* [dpdk-dev] [PATCH v2 1/5] bbdev: librte_bbdev library
@ 2017-10-18 2:14 Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 2/5] bbdev: PMD drivers (null/turbo_sw) Amr Mokhtar
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: Amr Mokhtar @ 2017-10-18 2:14 UTC (permalink / raw)
To: dev
Cc: thomas, anatoly.burakov, pablo.de.lara.guarch, niall.power,
chris.macnamara, Amr Mokhtar
- BBDEV library files
- BBDEV is tagged as EXPERIMENTAL
- Makefiles and configuration macros definition
- The bbdev framework and the 'null' driver are enabled by default
- The bbdev test framework is enabled by default
- Release Notes of the initial version and MAINTAINERS
Signed-off-by: Amr Mokhtar <amr.mokhtar@intel.com>
---
MAINTAINERS | 10 +
config/common_base | 23 +
doc/guides/rel_notes/release_17_11.rst | 10 +
lib/Makefile | 3 +
lib/librte_bbdev/Makefile | 56 ++
lib/librte_bbdev/rte_bbdev.c | 1095 ++++++++++++++++++++++++++++++++
lib/librte_bbdev/rte_bbdev.h | 741 +++++++++++++++++++++
lib/librte_bbdev/rte_bbdev_op.h | 514 +++++++++++++++
lib/librte_bbdev/rte_bbdev_pci.h | 288 +++++++++
lib/librte_bbdev/rte_bbdev_pmd.h | 223 +++++++
lib/librte_bbdev/rte_bbdev_vdev.h | 102 +++
lib/librte_bbdev/rte_bbdev_version.map | 37 ++
mk/rte.app.mk | 13 +
13 files changed, 3115 insertions(+)
create mode 100644 lib/librte_bbdev/Makefile
create mode 100644 lib/librte_bbdev/rte_bbdev.c
create mode 100644 lib/librte_bbdev/rte_bbdev.h
create mode 100644 lib/librte_bbdev/rte_bbdev_op.h
create mode 100644 lib/librte_bbdev/rte_bbdev_pci.h
create mode 100644 lib/librte_bbdev/rte_bbdev_pmd.h
create mode 100644 lib/librte_bbdev/rte_bbdev_vdev.h
create mode 100644 lib/librte_bbdev/rte_bbdev_version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 2a58378..df63f3f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -275,6 +275,16 @@ T: git://dpdk.org/next/dpdk-next-eventdev
F: lib/librte_eventdev/*eth_rx_adapter*
F: test/test/test_event_eth_rx_adapter.c
+BBDEV API - EXPERIMENTAL
+M: Amr Mokhtar <amr.mokhtar@intel.com>
+F: lib/librte_bbdev/
+F: drivers/bbdev/
+F: app/test-bbdev
+F: examples/bbdev_app/
+F: doc/guides/bbdevs/
+F: doc/guides/prog_guide/bbdev.rst
+F: doc/guides/sample_app_ug/bbdev_app.rst
+F: doc/guides/tools/testbbdev.rst
Networking Drivers
------------------
diff --git a/config/common_base b/config/common_base
index d9471e8..7e75701 100644
--- a/config/common_base
+++ b/config/common_base
@@ -573,6 +573,24 @@ CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV_DEBUG=n
CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=y
CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF_DEBUG=n
+# Compile generic wireless base band device library
+# EXPERIMENTAL: API may change without prior notice
+#
+CONFIG_RTE_LIBRTE_BBDEV=y
+CONFIG_RTE_LIBRTE_BBDEV_DEBUG=n
+CONFIG_RTE_BBDEV_MAX_DEVS=128
+CONFIG_RTE_BBDEV_NAME_MAX_LEN=64
+
+#
+# Compile PMD for NULL bbdev device
+#
+CONFIG_RTE_LIBRTE_PMD_BBDEV_NULL=y
+
+#
+# Compile PMD for turbo software bbdev device
+#
+CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW=n
+
#
# Compile librte_ring
#
@@ -780,6 +798,11 @@ CONFIG_RTE_APP_TEST=y
CONFIG_RTE_APP_TEST_RESOURCE_TAR=n
#
+# Compile the bbdev test application
+#
+CONFIG_RTE_TEST_BBDEV=y
+
+#
# Compile the PMD test application
#
CONFIG_RTE_TEST_PMD=y
diff --git a/doc/guides/rel_notes/release_17_11.rst b/doc/guides/rel_notes/release_17_11.rst
index 8db35f5..ab1c16b 100644
--- a/doc/guides/rel_notes/release_17_11.rst
+++ b/doc/guides/rel_notes/release_17_11.rst
@@ -41,6 +41,16 @@ New Features
Also, make sure to start the actual text at the margin.
=========================================================
+* **Added Wireless Base Band Device (bbdev).**
+
+ The Wireless Baseband library provides a common programming framework that
+ abstracts HW accelerators based on FPGA and/or Fixed Function Accelerators that
+ assist with 3gpp Physical Layer processing. Furthermore, it decouples the
+ application from the compute-intensive wireless functions by abstracting their
+ optimized libraries to appear as virtual bbdev devices.
+
+ The current release only supports Turbo Code FEC function.
+
* **Extended port_id range from uint8_t to uint16_t.**
Increased port_id range from 8 bits to 16 bits in order to support more than
diff --git a/lib/Makefile b/lib/Makefile
index 86d475f..3641eb5 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -52,6 +52,9 @@ DEPDIRS-librte_cryptodev := librte_eal librte_mempool librte_ring librte_mbuf
DEPDIRS-librte_cryptodev += librte_kvargs
DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += librte_eventdev
DEPDIRS-librte_eventdev := librte_eal librte_ring librte_ether librte_hash
+DIRS-$(CONFIG_RTE_LIBRTE_BBDEV) += librte_bbdev
+DEPDIRS-librte_bbdev := librte_eal librte_mempool librte_ring librte_mbuf
+DEPDIRS-librte_bbdev += librte_kvargs
DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += librte_vhost
DEPDIRS-librte_vhost := librte_eal librte_mempool librte_mbuf librte_ether
DIRS-$(CONFIG_RTE_LIBRTE_HASH) += librte_hash
diff --git a/lib/librte_bbdev/Makefile b/lib/librte_bbdev/Makefile
new file mode 100644
index 0000000..60c47e2
--- /dev/null
+++ b/lib/librte_bbdev/Makefile
@@ -0,0 +1,56 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_bbdev.a
+
+# library version
+LIBABIVER := 1
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+# library source files
+SRCS-y += rte_bbdev.c
+
+# export include files
+SYMLINK-y-include += rte_bbdev_op.h
+SYMLINK-y-include += rte_bbdev.h
+SYMLINK-y-include += rte_bbdev_pmd.h
+SYMLINK-y-include += rte_bbdev_pci.h
+SYMLINK-y-include += rte_bbdev_vdev.h
+
+# versioning export map
+EXPORT_MAP := rte_bbdev_version.map
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_bbdev/rte_bbdev.c b/lib/librte_bbdev/rte_bbdev.c
new file mode 100644
index 0000000..16f2544
--- /dev/null
+++ b/lib/librte_bbdev/rte_bbdev.c
@@ -0,0 +1,1095 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdint.h>
+#include <string.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_errno.h>
+#include <rte_log.h>
+#include <rte_debug.h>
+#include <rte_eal.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_memzone.h>
+#include <rte_lcore.h>
+#include <rte_dev.h>
+#include <rte_spinlock.h>
+#include <rte_tailq.h>
+#include <rte_interrupts.h>
+
+#include "rte_bbdev_op.h"
+#include "rte_bbdev.h"
+#include "rte_bbdev_pmd.h"
+
+#define DEV_NAME "BBDEV"
+
+
+/* Helper macro to check dev_id is valid */
+#define VALID_DEV_OR_RET_ERR(dev, dev_id) do { \
+ if (dev == NULL) { \
+ rte_bbdev_log(ERR, "device %u is invalid", dev_id); \
+ return -ENODEV; \
+ } \
+} while (0)
+
+/* Helper macro to check dev_ops is valid */
+#define VALID_DEV_OPS_OR_RET_ERR(dev, dev_id) do { \
+ if (dev->dev_ops == NULL) { \
+ rte_bbdev_log(ERR, "NULL dev_ops structure in device %u", \
+ dev_id); \
+ return -ENODEV; \
+ } \
+} while (0)
+
+/* Helper macro to check that driver implements required function pointer */
+#define VALID_FUNC_OR_RET_ERR(func, dev_id) do { \
+ if (func == NULL) { \
+ rte_bbdev_log(ERR, "device %u does not support %s", \
+ dev_id, #func); \
+ return -ENOTSUP; \
+ } \
+} while (0)
+
+/* Helper macro to check that queue is valid */
+#define VALID_QUEUE_OR_RET_ERR(queue_id, dev) do { \
+ if (queue_id >= dev->data->num_queues) { \
+ rte_bbdev_log(ERR, "Invalid queue_id %u for device %u", \
+ queue_id, dev->data->dev_id); \
+ return -ERANGE; \
+ } \
+} while (0)
+
+/* List of callback functions registered by an application */
+struct rte_bbdev_callback {
+ TAILQ_ENTRY(rte_bbdev_callback) next; /* Callbacks list */
+ rte_bbdev_cb_fn cb_fn; /* Callback address */
+ void *cb_arg; /* Parameter for callback */
+ void *ret_param; /* Return parameter */
+ enum rte_bbdev_event_type event; /* Interrupt event type */
+ uint32_t active; /* Callback is executing */
+};
+
+/* spinlock for bbdev device callbacks */
+static rte_spinlock_t rte_bbdev_cb_lock = RTE_SPINLOCK_INITIALIZER;
+
+/* Global array of all devices. This is not static because it's used by the
+ * inline enqueue and dequeue functions
+ */
+struct rte_bbdev rte_bbdev_devices[RTE_BBDEV_MAX_DEVS];
+
+/* Global array with rte_bbdev_data structures */
+static struct rte_bbdev_data *rte_bbdev_data;
+
+/* Memzone name for global bbdev data pool */
+static const char *MZ_RTE_BBDEV_DATA = "rte_bbdev_data";
+
+/* Number of currently valid devices */
+static uint16_t num_devs;
+
+/* Return pointer to device structure, with validity check */
+static struct rte_bbdev *
+get_dev(uint16_t dev_id)
+{
+ if (rte_bbdev_is_valid(dev_id))
+ return &rte_bbdev_devices[dev_id];
+ return NULL;
+}
+
+/* Allocate global data array */
+static void
+rte_bbdev_data_alloc(void)
+{
+ const unsigned int flags = 0;
+ const struct rte_memzone *mz;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ mz = rte_memzone_reserve(MZ_RTE_BBDEV_DATA,
+ RTE_BBDEV_MAX_DEVS * sizeof(*rte_bbdev_data),
+ rte_socket_id(), flags);
+ } else
+ mz = rte_memzone_lookup(MZ_RTE_BBDEV_DATA);
+ if (mz == NULL)
+ rte_panic("Cannot allocate memzone for bbdev port data\n");
+
+ rte_bbdev_data = mz->addr;
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+ memset(rte_bbdev_data, 0,
+ RTE_BBDEV_MAX_DEVS * sizeof(*rte_bbdev_data));
+}
+
+/* Find lowest device id with no attached device */
+static uint16_t
+find_free_dev_id(void)
+{
+ uint16_t i;
+ for (i = 0; i < RTE_BBDEV_MAX_DEVS; i++) {
+ if (rte_bbdev_devices[i].state == RTE_BBDEV_UNUSED)
+ return i;
+ }
+ return RTE_BBDEV_MAX_DEVS;
+}
+
+struct rte_bbdev *
+rte_bbdev_allocate(const char *name)
+{
+ int ret;
+ struct rte_bbdev *bbdev;
+ uint16_t dev_id;
+
+ if (name == NULL) {
+ rte_bbdev_log(ERR, "Invalid null device name");
+ return NULL;
+ }
+
+ if (rte_bbdev_get_named_dev(name) != NULL) {
+ rte_bbdev_log(ERR, "Device \"%s\" is already allocated", name);
+ return NULL;
+ }
+
+ dev_id = find_free_dev_id();
+ if (dev_id == RTE_BBDEV_MAX_DEVS) {
+ rte_bbdev_log(ERR, "Reached maximum number of devices");
+ return NULL;
+ }
+
+ bbdev = &rte_bbdev_devices[dev_id];
+
+ if (rte_bbdev_data == NULL)
+ rte_bbdev_data_alloc();
+
+ bbdev->data = &rte_bbdev_data[dev_id];
+ memset(bbdev->data, 0, sizeof(*bbdev->data));
+
+ bbdev->data->dev_id = dev_id;
+ bbdev->state = RTE_BBDEV_INITALIZED;
+
+ ret = snprintf(bbdev->data->name, RTE_BBDEV_NAME_MAX_LEN, "%s", name);
+ if ((ret < 0) || (ret >= RTE_BBDEV_NAME_MAX_LEN)) {
+ rte_bbdev_log(ERR, "Copying device name \"%s\" failed", name);
+ return NULL;
+ }
+
+ /* init user callbacks */
+ TAILQ_INIT(&(bbdev->list_cbs));
+
+ num_devs++;
+
+ rte_bbdev_log_debug("Initialised device %s (id = %u). Num devices = %u",
+ name, dev_id, num_devs);
+
+ return bbdev;
+}
+
+int
+rte_bbdev_release(struct rte_bbdev *bbdev)
+{
+ uint16_t dev_id;
+ struct rte_bbdev_callback *cb, *next;
+
+ if (bbdev == NULL) {
+ rte_bbdev_log(ERR, "NULL bbdev");
+ return -ENODEV;
+ }
+ dev_id = bbdev->data->dev_id;
+
+ /* free all callbacks from the device's list */
+ for (cb = TAILQ_FIRST(&bbdev->list_cbs); cb != NULL; cb = next) {
+
+ next = TAILQ_NEXT(cb, next);
+ TAILQ_REMOVE(&(bbdev->list_cbs), cb, next);
+ rte_free(cb);
+ }
+
+ memset(bbdev, 0, sizeof(*bbdev));
+ num_devs--;
+ bbdev->state = RTE_BBDEV_UNUSED;
+
+ rte_bbdev_log_debug(
+ "Un-initialised device id = %u. Num devices = %u",
+ dev_id, num_devs);
+ return 0;
+}
+
+struct rte_bbdev *
+rte_bbdev_get_named_dev(const char *name)
+{
+ unsigned int i;
+
+ if (name == NULL) {
+ rte_bbdev_log(ERR, "NULL driver name");
+ return NULL;
+ }
+
+ for (i = 0; i < RTE_BBDEV_MAX_DEVS; i++) {
+ struct rte_bbdev *dev = get_dev(i);
+ if (dev && (strncmp(dev->data->name,
+ name, RTE_BBDEV_NAME_MAX_LEN) == 0))
+ return dev;
+ }
+
+ return NULL;
+}
+
+uint16_t
+rte_bbdev_count(void)
+{
+ return num_devs;
+}
+
+bool
+rte_bbdev_is_valid(uint16_t dev_id)
+{
+ if ((dev_id < RTE_BBDEV_MAX_DEVS) &&
+ rte_bbdev_devices[dev_id].state == RTE_BBDEV_INITALIZED)
+ return true;
+ return false;
+}
+
+uint16_t
+rte_bbdev_find_next(uint16_t dev_id)
+{
+ dev_id++;
+ for (; dev_id < RTE_BBDEV_MAX_DEVS; dev_id++)
+ if (rte_bbdev_is_valid(dev_id))
+ break;
+ return dev_id;
+}
+
+int
+rte_bbdev_setup_queues(uint16_t dev_id, uint16_t num_queues, int socket_id)
+{
+ unsigned int i;
+ int ret;
+ struct rte_bbdev_driver_info dev_info;
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ if (dev->data->started) {
+ rte_bbdev_log(ERR,
+ "Device %u cannot be configured when started",
+ dev_id);
+ return -EBUSY;
+ }
+
+ /* Get device driver information to get max number of queues */
+ VALID_FUNC_OR_RET_ERR(dev->dev_ops->info_get, dev_id);
+ memset(&dev_info, 0, sizeof(dev_info));
+ dev->dev_ops->info_get(dev, &dev_info);
+
+ if ((num_queues == 0) || (num_queues > dev_info.max_num_queues)) {
+ rte_bbdev_log(ERR,
+ "Device %u supports 0 < N <= %u queues, not %u",
+ dev_id, dev_info.max_num_queues, num_queues);
+ return -EINVAL;
+ }
+
+ /* If re-configuration, get driver to free existing internal memory */
+ if (dev->data->queues != NULL) {
+ VALID_FUNC_OR_RET_ERR(dev->dev_ops->queue_release, dev_id);
+ for (i = 0; i < dev->data->num_queues; i++) {
+ int ret = dev->dev_ops->queue_release(dev, i);
+ if (ret < 0) {
+ rte_bbdev_log(ERR,
+ "Device %u queue %u release failed",
+ dev_id, i);
+ return ret;
+ }
+ }
+ /* Call optional device close */
+ if (dev->dev_ops->close) {
+ ret = dev->dev_ops->close(dev);
+ if (ret < 0) {
+ rte_bbdev_log(ERR,
+ "Device %u couldn't be closed",
+ dev_id);
+ return ret;
+ }
+ }
+ rte_free(dev->data->queues);
+ }
+
+ /* Allocate queue pointers */
+ dev->data->queues = rte_calloc_socket(DEV_NAME, num_queues,
+ sizeof(dev->data->queues[0]), RTE_CACHE_LINE_SIZE,
+ dev->data->socket_id);
+ if (dev->data->queues == NULL) {
+ rte_bbdev_log(ERR,
+ "calloc of %u queues for device %u on socket %i failed",
+ num_queues, dev_id, dev->data->socket_id);
+ return -ENOMEM;
+ }
+
+ dev->data->num_queues = num_queues;
+
+ /* Call optional device configuration */
+ if (dev->dev_ops->setup_queues) {
+ ret = dev->dev_ops->setup_queues(dev, num_queues, socket_id);
+ if (ret < 0) {
+ rte_bbdev_log(ERR,
+ "Device %u memory configuration failed",
+ dev_id);
+ goto error;
+ }
+ }
+
+ rte_bbdev_log_debug("Device %u set up with %u queues", dev_id,
+ num_queues);
+ return 0;
+
+error:
+ dev->data->num_queues = 0;
+ rte_free(dev->data->queues);
+ dev->data->queues = NULL;
+ return ret;
+}
+
+int
+rte_bbdev_intr_enable(uint16_t dev_id)
+{
+ int ret;
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ if (dev->data->started) {
+ rte_bbdev_log(ERR,
+ "Device %u cannot be configured when started",
+ dev_id);
+ return -EBUSY;
+ }
+
+ if (dev->dev_ops->intr_enable) {
+ ret = dev->dev_ops->intr_enable(dev);
+ if (ret < 0) {
+ rte_bbdev_log(ERR,
+ "Device %u interrupts configuration failed",
+ dev_id);
+ return ret;
+ }
+ rte_bbdev_log_debug("Enabled interrupts for dev %u", dev_id);
+ return 0;
+ }
+
+ rte_bbdev_log(ERR, "Device %u doesn't support interrupts", dev_id);
+ return -ENOTSUP;
+}
+
+int
+rte_bbdev_queue_configure(uint16_t dev_id, uint16_t queue_id,
+ const struct rte_bbdev_queue_conf *conf)
+{
+ int ret = 0;
+ struct rte_bbdev_driver_info dev_info;
+ struct rte_bbdev *dev = get_dev(dev_id);
+ const struct rte_bbdev_op_cap *p;
+ struct rte_bbdev_queue_conf *stored_conf;
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ VALID_QUEUE_OR_RET_ERR(queue_id, dev);
+
+ if (dev->data->queues[queue_id].started || dev->data->started) {
+ rte_bbdev_log(ERR,
+ "Queue %u of device %u cannot be configured when started",
+ queue_id, dev_id);
+ return -EBUSY;
+ }
+
+ VALID_FUNC_OR_RET_ERR(dev->dev_ops->queue_release, dev_id);
+ VALID_FUNC_OR_RET_ERR(dev->dev_ops->queue_setup, dev_id);
+
+ /* Get device driver information to verify config is valid */
+ VALID_FUNC_OR_RET_ERR(dev->dev_ops->info_get, dev_id);
+ memset(&dev_info, 0, sizeof(dev_info));
+ dev->dev_ops->info_get(dev, &dev_info);
+
+ /* Check configuration is valid */
+ if (conf != NULL) {
+ if ((conf->op_type == RTE_BBDEV_OP_NONE) &&
+ (dev_info.capabilities[0].type ==
+ RTE_BBDEV_OP_NONE)) {
+ ret = 1;
+ } else
+ for (p = dev_info.capabilities;
+ p->type != RTE_BBDEV_OP_NONE; p++) {
+ if (conf->op_type == p->type) {
+ ret = 1;
+ break;
+ }
+ }
+ if (ret == 0) {
+ rte_bbdev_log(ERR, "Invalid operation type");
+ return -EINVAL;
+ }
+ if (conf->queue_size > dev_info.queue_size_lim) {
+ rte_bbdev_log(ERR,
+ "Size (%u) of queue %u of device %u must be: <= %u",
+ conf->queue_size, queue_id, dev_id,
+ dev_info.queue_size_lim);
+ return -EINVAL;
+ }
+ if (!rte_is_power_of_2(conf->queue_size)) {
+ rte_bbdev_log(ERR,
+ "Size (%u) of queue %u of device %u must be a power of 2",
+ conf->queue_size, queue_id, dev_id);
+ return -EINVAL;
+ }
+ if (conf->priority > dev_info.max_queue_priority) {
+ rte_bbdev_log(ERR,
+ "Priority (%u) of queue %u of bdev %u must be <= %u",
+ conf->priority, queue_id, dev_id,
+ dev_info.max_queue_priority);
+ return -EINVAL;
+ }
+ }
+
+ /* Release existing queue (in case of queue reconfiguration) */
+ if (dev->data->queues[queue_id].queue_private != NULL) {
+ ret = dev->dev_ops->queue_release(dev, queue_id);
+ if (ret < 0) {
+ rte_bbdev_log(ERR, "Device %u queue %u release failed",
+ dev_id, queue_id);
+ return ret;
+ }
+ }
+
+ /* Get driver to setup the queue */
+ ret = dev->dev_ops->queue_setup(dev, queue_id, (conf != NULL) ?
+ conf : &dev_info.default_queue_conf);
+ if (ret < 0) {
+ rte_bbdev_log(ERR,
+ "Device %u queue %u setup failed", dev_id,
+ queue_id);
+ return ret;
+ }
+
+ /* Store configuration */
+ stored_conf = &dev->data->queues[queue_id].conf;
+ memcpy(stored_conf,
+ (conf != NULL) ? conf : &dev_info.default_queue_conf,
+ sizeof(*stored_conf));
+
+ rte_bbdev_log_debug("Configured dev%uq%u (size=%u, type=%s, prio=%u)",
+ dev_id, queue_id, stored_conf->queue_size,
+ rte_bbdev_op_type_str(stored_conf->op_type),
+ stored_conf->priority);
+
+ return 0;
+}
+
+int
+rte_bbdev_start(uint16_t dev_id)
+{
+ int i;
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ if (dev->data->started) {
+ rte_bbdev_log_debug("Device %u is already started", dev_id);
+ return 0;
+ }
+
+ if (dev->dev_ops->start) {
+ int ret = dev->dev_ops->start(dev);
+ if (ret < 0) {
+ rte_bbdev_log(ERR, "Device %u start failed", dev_id);
+ return ret;
+ }
+ }
+
+ /* Store new state */
+ for (i = 0; i < dev->data->num_queues; i++)
+ if (!dev->data->queues[i].conf.deferred_start)
+ dev->data->queues[i].started = true;
+ dev->data->started = true;
+
+ rte_bbdev_log_debug("Started device %u", dev_id);
+ return 0;
+}
+
+int
+rte_bbdev_stop(uint16_t dev_id)
+{
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ if (!dev->data->started) {
+ rte_bbdev_log_debug("Device %u is already stopped", dev_id);
+ return 0;
+ }
+
+ if (dev->dev_ops->stop)
+ dev->dev_ops->stop(dev);
+ dev->data->started = false;
+
+ rte_bbdev_log_debug("Stopped device %u", dev_id);
+ return 0;
+}
+
+int
+rte_bbdev_close(uint16_t dev_id)
+{
+ int ret;
+ uint16_t i;
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ if (dev->data->started) {
+ ret = rte_bbdev_stop(dev_id);
+ if (ret < 0) {
+ rte_bbdev_log(ERR, "Device %u stop failed", dev_id);
+ return ret;
+ }
+ }
+
+ /* Free memory used by queues */
+ for (i = 0; i < dev->data->num_queues; i++) {
+ ret = dev->dev_ops->queue_release(dev, i);
+ if (ret < 0) {
+ rte_bbdev_log(ERR, "Device %u queue %u release failed",
+ dev_id, i);
+ return ret;
+ }
+ }
+ rte_free(dev->data->queues);
+
+ if (dev->dev_ops->close) {
+ ret = dev->dev_ops->close(dev);
+ if (ret < 0) {
+ rte_bbdev_log(ERR, "Device %u close failed", dev_id);
+ return ret;
+ }
+ }
+
+ /* Clear configuration */
+ dev->data->queues = NULL;
+ dev->data->num_queues = 0;
+
+ rte_bbdev_log_debug("Closed device %u", dev_id);
+ return 0;
+}
+
+int
+rte_bbdev_queue_start(uint16_t dev_id, uint16_t queue_id)
+{
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ VALID_QUEUE_OR_RET_ERR(queue_id, dev);
+
+ if (dev->data->queues[queue_id].started) {
+ rte_bbdev_log_debug("Queue %u of device %u already started",
+ queue_id, dev_id);
+ return 0;
+ }
+
+ if (dev->dev_ops->queue_start) {
+ int ret = dev->dev_ops->queue_start(dev, queue_id);
+ if (ret < 0) {
+ rte_bbdev_log(ERR, "Device %u queue %u start failed",
+ dev_id, queue_id);
+ return ret;
+ }
+ }
+ dev->data->queues[queue_id].started = true;
+
+ rte_bbdev_log_debug("Started queue %u of device %u", queue_id, dev_id);
+ return 0;
+}
+
+int
+rte_bbdev_queue_stop(uint16_t dev_id, uint16_t queue_id)
+{
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ VALID_QUEUE_OR_RET_ERR(queue_id, dev);
+
+ if (!dev->data->queues[queue_id].started) {
+ rte_bbdev_log_debug("Queue %u of device %u already stopped",
+ queue_id, dev_id);
+ return 0;
+ }
+
+ if (dev->dev_ops->queue_stop) {
+ int ret = dev->dev_ops->queue_stop(dev, queue_id);
+ if (ret < 0) {
+ rte_bbdev_log(ERR, "Device %u queue %u stop failed",
+ dev_id, queue_id);
+ return ret;
+ }
+ }
+ dev->data->queues[queue_id].started = false;
+
+ rte_bbdev_log_debug("Stopped queue %u of device %u", queue_id, dev_id);
+ return 0;
+}
+
+/* Get device statistics */
+static void
+get_stats_from_queues(struct rte_bbdev *dev, struct rte_bbdev_stats *stats)
+{
+ unsigned int q_id;
+ for (q_id = 0; q_id < dev->data->num_queues; q_id++) {
+ struct rte_bbdev_stats *q_stats =
+ &dev->data->queues[q_id].queue_stats;
+
+ stats->enqueued_count += q_stats->enqueued_count;
+ stats->dequeued_count += q_stats->dequeued_count;
+ stats->enqueue_err_count += q_stats->enqueue_err_count;
+ stats->dequeue_err_count += q_stats->dequeue_err_count;
+ }
+ rte_bbdev_log_debug("Got stats on %u", dev->data->dev_id);
+}
+
+static void
+reset_stats_in_queues(struct rte_bbdev *dev)
+{
+ unsigned int q_id;
+ for (q_id = 0; q_id < dev->data->num_queues; q_id++) {
+ struct rte_bbdev_stats *q_stats =
+ &dev->data->queues[q_id].queue_stats;
+
+ memset(q_stats, 0, sizeof(*q_stats));
+ }
+ rte_bbdev_log_debug("Reset stats on %u", dev->data->dev_id);
+}
+
+int
+rte_bbdev_stats_get(uint16_t dev_id, struct rte_bbdev_stats *stats)
+{
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ if (stats == NULL) {
+ rte_bbdev_log(ERR, "NULL stats structure");
+ return -EINVAL;
+ }
+
+ memset(stats, 0, sizeof(*stats));
+ if (dev->dev_ops->stats_get != NULL)
+ dev->dev_ops->stats_get(dev, stats);
+ else
+ get_stats_from_queues(dev, stats);
+
+ rte_bbdev_log_debug("Retrieved stats of device %u", dev_id);
+ return 0;
+}
+
+int
+rte_bbdev_stats_reset(uint16_t dev_id)
+{
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+
+ if (dev->dev_ops->stats_reset != NULL)
+ dev->dev_ops->stats_reset(dev);
+ else
+ reset_stats_in_queues(dev);
+
+ rte_bbdev_log_debug("Reset stats of device %u", dev_id);
+ return 0;
+}
+
+int
+rte_bbdev_info_get(uint16_t dev_id, struct rte_bbdev_info *dev_info)
+{
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_FUNC_OR_RET_ERR(dev->dev_ops->info_get, dev_id);
+
+ if (dev_info == NULL) {
+ rte_bbdev_log(ERR, "NULL dev info structure");
+ return -EINVAL;
+ }
+
+ /* Copy data maintained by device interface layer */
+ memset(dev_info, 0, sizeof(*dev_info));
+ dev_info->dev_name = dev->data->name;
+ dev_info->num_queues = dev->data->num_queues;
+ dev_info->bus = rte_bus_find_by_device(dev->device);
+ dev_info->socket_id = dev->data->socket_id;
+ dev_info->started = dev->data->started;
+
+ /* Copy data maintained by device driver layer */
+ dev->dev_ops->info_get(dev, &dev_info->drv);
+
+ rte_bbdev_log_debug("Retrieved info of device %u", dev_id);
+ return 0;
+}
+
+int
+rte_bbdev_queue_info_get(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_queue_info *dev_info)
+{
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ VALID_QUEUE_OR_RET_ERR(queue_id, dev);
+
+ if (dev_info == NULL) {
+ rte_bbdev_log(ERR, "NULL queue info structure");
+ return -EINVAL;
+ }
+
+ /* Copy data to output */
+ memset(dev_info, 0, sizeof(*dev_info));
+ dev_info->conf = dev->data->queues[queue_id].conf;
+ dev_info->started = dev->data->queues[queue_id].started;
+
+ rte_bbdev_log_debug("Retrieved info of queue %u of device %u",
+ queue_id, dev_id);
+ return 0;
+}
+
+/* Calculate size needed to store bbdev_op, depending on type */
+static unsigned int
+get_bbdev_op_size(enum rte_bbdev_op_type type)
+{
+ unsigned int result = 0;
+ switch (type) {
+ case RTE_BBDEV_OP_NONE:
+ result = RTE_MAX(sizeof(struct rte_bbdev_dec_op),
+ sizeof(struct rte_bbdev_enc_op));
+ break;
+ case RTE_BBDEV_OP_TURBO_DEC:
+ result = sizeof(struct rte_bbdev_dec_op);
+ break;
+ case RTE_BBDEV_OP_TURBO_ENC:
+ result = sizeof(struct rte_bbdev_enc_op);
+ break;
+ default:
+ break;
+ }
+
+ return result;
+}
+
+/* Initialise a bbdev_op structure */
+static void
+bbdev_op_init(struct rte_mempool *mempool, void *arg, void *element,
+ __rte_unused unsigned int n)
+{
+ enum rte_bbdev_op_type type = *(enum rte_bbdev_op_type *)arg;
+
+ if (type == RTE_BBDEV_OP_TURBO_DEC) {
+ struct rte_bbdev_dec_op *op = element;
+ memset(op, 0, mempool->elt_size);
+ op->mempool = mempool;
+ } else if (type == RTE_BBDEV_OP_TURBO_ENC) {
+ struct rte_bbdev_enc_op *op = element;
+ memset(op, 0, mempool->elt_size);
+ op->mempool = mempool;
+ }
+}
+
+struct rte_mempool *
+rte_bbdev_op_pool_create(const char *name, enum rte_bbdev_op_type type,
+ unsigned int num_elements, unsigned int cache_size,
+ int socket_id)
+{
+ struct rte_bbdev_op_pool_private *priv;
+ struct rte_mempool *mp;
+
+ if (name == NULL) {
+ rte_bbdev_log(ERR, "NULL name for op pool");
+ return NULL;
+ }
+
+ if (type >= RTE_BBDEV_OP_TYPE_COUNT) {
+ rte_bbdev_log(ERR,
+ "Invalid op type (%u), should be less than %u",
+ type, RTE_BBDEV_OP_TYPE_COUNT);
+ return NULL;
+ }
+
+ mp = rte_mempool_create(name, num_elements, get_bbdev_op_size(type),
+ cache_size, sizeof(struct rte_bbdev_op_pool_private),
+ NULL, NULL, bbdev_op_init, &type, socket_id, 0);
+ if (mp == NULL) {
+ rte_bbdev_log(ERR,
+ "Failed to create op pool %s (num ops=%u, op size=%u) with error: %s",
+ name, num_elements, get_bbdev_op_size(type),
+ rte_strerror(rte_errno));
+ return NULL;
+ }
+
+ rte_bbdev_log_debug(
+ "Op pool %s created for %u ops (type=%s, cache=%u, socket=%u, size=%u)",
+ name, num_elements, rte_bbdev_op_type_str(type),
+ cache_size, socket_id, get_bbdev_op_size(type));
+
+ priv = (struct rte_bbdev_op_pool_private *)rte_mempool_get_priv(mp);
+ priv->type = type;
+
+ return mp;
+}
+
+int
+rte_bbdev_callback_register(uint16_t dev_id, enum rte_bbdev_event_type event,
+ rte_bbdev_cb_fn cb_fn, void *cb_arg)
+{
+ struct rte_bbdev_callback *user_cb;
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ if (event >= RTE_BBDEV_EVENT_MAX) {
+ rte_bbdev_log(ERR,
+ "Invalid event type (%u), should be less than %u",
+ event, RTE_BBDEV_EVENT_MAX);
+ return -EINVAL;
+ }
+
+ if (cb_fn == NULL) {
+ rte_bbdev_log(ERR, "NULL callback function");
+ return -EINVAL;
+ }
+
+ rte_spinlock_lock(&rte_bbdev_cb_lock);
+
+ TAILQ_FOREACH(user_cb, &(dev->list_cbs), next) {
+ if (user_cb->cb_fn == cb_fn &&
+ user_cb->cb_arg == cb_arg &&
+ user_cb->event == event)
+ break;
+ }
+
+ /* create a new callback. */
+ if (user_cb == NULL) {
+ user_cb = rte_zmalloc("INTR_USER_CALLBACK",
+ sizeof(struct rte_bbdev_callback), 0);
+ if (user_cb != NULL) {
+ user_cb->cb_fn = cb_fn;
+ user_cb->cb_arg = cb_arg;
+ user_cb->event = event;
+ TAILQ_INSERT_TAIL(&(dev->list_cbs), user_cb, next);
+ }
+ }
+
+ rte_spinlock_unlock(&rte_bbdev_cb_lock);
+ return (user_cb == NULL) ? -ENOMEM : 0;
+}
+
+int
+rte_bbdev_callback_unregister(uint16_t dev_id, enum rte_bbdev_event_type event,
+ rte_bbdev_cb_fn cb_fn, void *cb_arg)
+{
+ int ret = 0;
+ struct rte_bbdev_callback *cb, *next;
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+
+ if (event >= RTE_BBDEV_EVENT_MAX) {
+ rte_bbdev_log(ERR,
+ "Invalid event type (%u), should be less than %u",
+ event, RTE_BBDEV_EVENT_MAX);
+ return -EINVAL;
+ }
+
+ if (cb_fn == NULL) {
+ rte_bbdev_log(ERR,
+ "NULL callback function cannot be unregistered");
+ return -EINVAL;
+ }
+
+ dev = &rte_bbdev_devices[dev_id];
+ rte_spinlock_lock(&rte_bbdev_cb_lock);
+
+ for (cb = TAILQ_FIRST(&dev->list_cbs); cb != NULL; cb = next) {
+
+ next = TAILQ_NEXT(cb, next);
+
+ if (cb->cb_fn != cb_fn || cb->event != event ||
+ (cb_arg != (void *)-1 && cb->cb_arg != cb_arg))
+ continue;
+
+ /* If this callback is not executing right now, remove it. */
+ if (cb->active == 0) {
+ TAILQ_REMOVE(&(dev->list_cbs), cb, next);
+ rte_free(cb);
+ } else
+ ret = -EAGAIN;
+ }
+
+ rte_spinlock_unlock(&rte_bbdev_cb_lock);
+ return ret;
+}
+
+void
+rte_bbdev_pmd_callback_process(struct rte_bbdev *dev,
+ enum rte_bbdev_event_type event, void *ret_param)
+{
+ struct rte_bbdev_callback *cb_lst;
+ struct rte_bbdev_callback dev_cb;
+
+ if (dev == NULL) {
+ rte_bbdev_log(ERR, "NULL device");
+ return;
+ }
+
+ if (dev->data == NULL) {
+ rte_bbdev_log(ERR, "NULL data structure");
+ return;
+ }
+
+ if (event >= RTE_BBDEV_EVENT_MAX) {
+ rte_bbdev_log(ERR,
+ "Invalid event type (%u), should be less than %u",
+ event, RTE_BBDEV_EVENT_MAX);
+ return;
+ }
+
+ rte_spinlock_lock(&rte_bbdev_cb_lock);
+ TAILQ_FOREACH(cb_lst, &(dev->list_cbs), next) {
+ if (cb_lst->cb_fn == NULL || cb_lst->event != event)
+ continue;
+ dev_cb = *cb_lst;
+ cb_lst->active = 1;
+ if (ret_param != NULL)
+ dev_cb.ret_param = ret_param;
+
+ rte_spinlock_unlock(&rte_bbdev_cb_lock);
+ dev_cb.cb_fn(dev->data->dev_id, dev_cb.event,
+ dev_cb.cb_arg, dev_cb.ret_param);
+ rte_spinlock_lock(&rte_bbdev_cb_lock);
+ cb_lst->active = 0;
+ }
+ rte_spinlock_unlock(&rte_bbdev_cb_lock);
+}
+
+int
+rte_bbdev_queue_intr_enable(uint16_t dev_id, uint16_t queue_id)
+{
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+ VALID_QUEUE_OR_RET_ERR(queue_id, dev);
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+ VALID_FUNC_OR_RET_ERR(dev->dev_ops->queue_intr_enable, dev_id);
+ return dev->dev_ops->queue_intr_enable(dev, queue_id);
+}
+
+int
+rte_bbdev_queue_intr_disable(uint16_t dev_id, uint16_t queue_id)
+{
+ struct rte_bbdev *dev = get_dev(dev_id);
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+ VALID_QUEUE_OR_RET_ERR(queue_id, dev);
+ VALID_DEV_OPS_OR_RET_ERR(dev, dev_id);
+ VALID_FUNC_OR_RET_ERR(dev->dev_ops->queue_intr_disable, dev_id);
+ return dev->dev_ops->queue_intr_disable(dev, queue_id);
+}
+
+int
+rte_bbdev_queue_intr_ctl(uint16_t dev_id, uint16_t queue_id, int epfd, int op,
+ void *data)
+{
+ uint32_t vec;
+ struct rte_bbdev *dev = get_dev(dev_id);
+ struct rte_intr_handle *intr_handle;
+ int ret;
+
+ VALID_DEV_OR_RET_ERR(dev, dev_id);
+ VALID_QUEUE_OR_RET_ERR(queue_id, dev);
+
+ intr_handle = dev->intr_handle;
+ if (!intr_handle || !intr_handle->intr_vec) {
+ rte_bbdev_log(ERR, "Device %u intr handle unset\n", dev_id);
+ return -ENOTSUP;
+ }
+
+ if (queue_id >= RTE_MAX_RXTX_INTR_VEC_ID) {
+ rte_bbdev_log(ERR, "Device %u queue_id %u is too big\n",
+ dev_id, queue_id);
+ return -ENOTSUP;
+ }
+
+ vec = intr_handle->intr_vec[queue_id];
+ ret = rte_intr_rx_ctl(intr_handle, epfd, op, vec, data);
+ if (ret && (ret != -EEXIST)) {
+ rte_bbdev_log(ERR,
+ "dev %u q %u int ctl error op %d epfd %d vec %u\n",
+ dev_id, queue_id, op, epfd, vec);
+ return ret;
+ }
+
+ return 0;
+}
+
+
+const char *
+rte_bbdev_op_type_str(enum rte_bbdev_op_type op_type)
+{
+ static const char * const op_types[] = {
+ "RTE_BBDEV_OP_NONE",
+ "RTE_BBDEV_OP_TURBO_DEC",
+ "RTE_BBDEV_OP_TURBO_ENC",
+ };
+
+ if (op_type < RTE_BBDEV_OP_TYPE_COUNT)
+ return op_types[op_type];
+
+ rte_bbdev_log(ERR, "Invalid operation type");
+ return "";
+}
+
+
+int bbdev_logtype;
+
+RTE_INIT(rte_bbdev_init_log);
+static void
+rte_bbdev_init_log(void)
+{
+ bbdev_logtype = rte_log_register("lib.bbdev");
+ if (bbdev_logtype >= 0)
+ rte_log_set_level(bbdev_logtype, RTE_LOG_NOTICE);
+}
diff --git a/lib/librte_bbdev/rte_bbdev.h b/lib/librte_bbdev/rte_bbdev.h
new file mode 100644
index 0000000..9e6d283
--- /dev/null
+++ b/lib/librte_bbdev/rte_bbdev.h
@@ -0,0 +1,741 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_BBDEV_H_
+#define _RTE_BBDEV_H_
+
+/**
+ * @file rte_bbdev.h
+ *
+ * Wireless base band device application APIs.
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * This API allows an application to discover, configure and use a device to
+ * process operations. An asynchronous API (enqueue, followed by later dequeue)
+ * is used for processing operations.
+ *
+ * The functions in this API are not thread-safe when called on the same
+ * target object (a device, or a queue on a device), with the exception that
+ * one thread can enqueue operations to a queue while another thread dequeues
+ * from the same queue.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+#include <stdbool.h>
+#include <string.h>
+
+#include <rte_bus.h>
+#include <rte_cpuflags.h>
+#include <rte_memory.h>
+
+#include "rte_bbdev_op.h"
+
+#ifndef RTE_BBDEV_MAX_DEVS
+#define RTE_BBDEV_MAX_DEVS 128 /**< Max number of devices */
+#endif
+
+/** Flags indiciate current state of BBDEV device */
+enum rte_bbdev_state {
+ RTE_BBDEV_UNUSED,
+ RTE_BBDEV_INITALIZED
+};
+
+/**
+ * Get the total number of devices that have been successfully initialised.
+ *
+ * @return
+ * The total number of usable devices.
+ */
+uint16_t
+rte_bbdev_count(void);
+
+/**
+ * Check if a device is valid.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @return
+ * true if device ID is valid and device is attached, false otherwise.
+ */
+bool
+rte_bbdev_is_valid(uint16_t dev_id);
+
+/**
+ * Get the next enabled device.
+ *
+ * @param dev_id
+ * The current device
+ *
+ * @return
+ * - The next device, or
+ * - RTE_BBDEV_MAX_DEVS if none found
+ */
+uint16_t
+rte_bbdev_find_next(uint16_t dev_id);
+
+/** Iterate through all enabled devices */
+#define RTE_BBDEV_FOREACH(i) for (i = rte_bbdev_find_next(-1); \
+ i < RTE_BBDEV_MAX_DEVS; \
+ i = rte_bbdev_find_next(i))
+
+/**
+ * Setup up device queues.
+ * This function must be called on a device before setting up the queues and
+ * starting the device. It can also be called when a device is in the stopped
+ * state. If any device queues have been configured their configuration will be
+ * cleared by a call to this function.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param num_queues
+ * Number of queues to configure on device.
+ * @param socket_id
+ * ID of a socket which will be used to allocate memory.
+ *
+ * @return
+ * - 0 on success
+ * - -ENODEV if dev_id is invalid or the device is corrupted
+ * - -EINVAL if num_queues is invalid, 0 or greater than maximum
+ * - -EBUSY if the identified device has already started
+ * - -ENOMEM if unable to allocate memory
+ */
+int
+rte_bbdev_setup_queues(uint16_t dev_id, uint16_t num_queues, int socket_id);
+
+/**
+ * Enable interrupts.
+ * This function may be called before starting the device to enable the
+ * interrupts if they are available.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @return
+ * - 0 on success
+ * - -ENODEV if dev_id is invalid or the device is corrupted
+ * - -EBUSY if the identified device has already started
+ * - -ENOTSUP if the interrupts are not supported by the device
+ */
+int
+rte_bbdev_intr_enable(uint16_t dev_id);
+
+/** Device queue configuration structure */
+struct rte_bbdev_queue_conf {
+ int socket; /**< NUMA socket used for memory allocation */
+ uint32_t queue_size; /**< Size of queue */
+ uint8_t priority; /**< Queue priority */
+ bool deferred_start; /**< Do not start queue when device is started. */
+ enum rte_bbdev_op_type op_type; /**< Operation type */
+};
+
+/**
+ * Configure a queue on a device.
+ * This function can be called after device configuration, and before starting.
+ * It can also be called when the device or the queue is in the stopped state.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param queue_id
+ * The index of the queue.
+ * @param conf
+ * The queue configuration. If NULL, a default configuration will be used.
+ *
+ * @return
+ * - 0 on success
+ * - EINVAL if the identified queue size or priority are invalid
+ * - EBUSY if the identified queue or its device have already started
+ */
+int
+rte_bbdev_queue_configure(uint16_t dev_id, uint16_t queue_id,
+ const struct rte_bbdev_queue_conf *conf);
+
+/**
+ * Start a device.
+ * This is the last step needed before enqueueing operations is possible.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @return
+ * - 0 on success
+ * - negative value on failure - as returned from PMD driver
+ */
+int
+rte_bbdev_start(uint16_t dev_id);
+
+/**
+ * Stop a device.
+ * The device can be reconfigured, and restarted after being stopped.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @return
+ * - 0 on success
+ */
+int
+rte_bbdev_stop(uint16_t dev_id);
+
+/**
+ * Close a device.
+ * The device cannot be restarted without reconfiguration!
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @return
+ * - 0 on success
+ */
+int
+rte_bbdev_close(uint16_t dev_id);
+
+/**
+ * Start a specified queue on a device.
+ * This is only needed if the queue has been stopped, or if the deferred_start
+ * flag has been set when configuring the queue.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param queue_id
+ * The index of the queue.
+ *
+ * @return
+ * - 0 on success
+ * - negative value on failure - as returned from PMD driver
+ */
+int
+rte_bbdev_queue_start(uint16_t dev_id, uint16_t queue_id);
+
+/**
+ * Stop a specified queue on a device, to allow re configuration.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param queue_id
+ * The index of the queue.
+ *
+ * @return
+ * - 0 on success
+ * - negative value on failure - as returned from PMD driver
+ */
+int
+rte_bbdev_queue_stop(uint16_t dev_id, uint16_t queue_id);
+
+/** Device statistics. */
+struct rte_bbdev_stats {
+ uint64_t enqueued_count; /**< Count of all operations enqueued */
+ uint64_t dequeued_count; /**< Count of all operations dequeued */
+ /** Total error count on operations enqueued */
+ uint64_t enqueue_err_count;
+ /** Total error count on operations dequeued */
+ uint64_t dequeue_err_count;
+};
+
+/**
+ * Retrieve the general I/O statistics of a device.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param stats
+ * Pointer to structure to where statistics will be copied. On error, this
+ * location may or may not have been modified.
+ *
+ * @return
+ * - 0 on success
+ * - EINVAL if invalid parameter pointer is provided
+ */
+int
+rte_bbdev_stats_get(uint16_t dev_id, struct rte_bbdev_stats *stats);
+
+/**
+ * Reset the statistics of a device.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @return
+ * - 0 on success
+ */
+int
+rte_bbdev_stats_reset(uint16_t dev_id);
+
+/** Device information supplied by the device's driver */
+struct rte_bbdev_driver_info {
+ /** Driver name */
+ const char *driver_name;
+
+ /** Maximum number of queues supported by the device */
+ unsigned int max_num_queues;
+ /** Queue size limit (queue size must also be power of 2) */
+ uint32_t queue_size_lim;
+ /** Set if device off-loads operation to hardware */
+ bool hardware_accelerated;
+ /** Max value supported by queue priority */
+ uint8_t max_queue_priority;
+ /** Set if device supports per-queue interrupts */
+ bool queue_intr_supported;
+ /** Minimum alignment of buffers, in bytes */
+ uint16_t min_alignment;
+ /** Default queue configuration used if none is supplied */
+ struct rte_bbdev_queue_conf default_queue_conf;
+ /** Device operation capabilities */
+ const struct rte_bbdev_op_cap *capabilities;
+ /** Device cpu_flag requirements */
+ const enum rte_cpu_flag_t *cpu_flag_reqs;
+};
+
+/** Macro used at end of bbdev PMD list */
+#define RTE_BBDEV_END_OF_CAPABILITIES_LIST() \
+ { RTE_BBDEV_OP_NONE }
+
+/** Device information structure used by an application to discover a devices
+ * capabilities and current configuration
+ */
+struct rte_bbdev_info {
+ int socket_id; /**< NUMA socket that device is on */
+ const char *dev_name; /**< Unique device name */
+ const struct rte_bus *bus; /**< Bus information */
+ uint16_t num_queues; /**< Number of queues currently configured */
+ bool started; /**< Set if device is currently started */
+ struct rte_bbdev_driver_info drv; /**< Info from device driver */
+};
+
+/**
+ * Retrieve information about a device.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param dev_info
+ * Pointer to structure to where information will be copied. On error, this
+ * location may or may not have been modified.
+ *
+ * @return
+ * - 0 on success
+ * - EINVAL if invalid parameter pointer is provided
+ */
+int
+rte_bbdev_info_get(uint16_t dev_id, struct rte_bbdev_info *dev_info);
+
+/** Queue information */
+struct rte_bbdev_queue_info {
+ /** Current device configuration */
+ struct rte_bbdev_queue_conf conf;
+ /** Set if queue is currently started */
+ bool started;
+};
+
+/**
+ * Retrieve information about a specific queue on a device.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param queue_id
+ * The index of the queue.
+ * @param dev_info
+ * Pointer to structure to where information will be copied. On error, this
+ * location may or may not have been modified.
+ *
+ * @return
+ * - 0 on success
+ * - EINVAL if invalid parameter pointer is provided
+ */
+int
+rte_bbdev_queue_info_get(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_queue_info *dev_info);
+
+/** @internal The data structure associated with each queue of a device. */
+struct rte_bbdev_queue_data {
+ void *queue_private; /**< Driver-specific per-queue data */
+ struct rte_bbdev_queue_conf conf; /**< Current configuration */
+ struct rte_bbdev_stats queue_stats; /**< Queue statistics */
+ bool started; /**< Queue state */
+};
+
+/** @internal Enqueue encode operations for processing on queue of a device. */
+typedef uint16_t (*rte_bbdev_enqueue_enc_ops_t)(
+ struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_enc_op **ops,
+ uint16_t num);
+
+/** @internal Enqueue decode operations for processing on queue of a device. */
+typedef uint16_t (*rte_bbdev_enqueue_dec_ops_t)(
+ struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_dec_op **ops,
+ uint16_t num);
+
+/** @internal Dequeue encode operations from a queue of a device. */
+typedef uint16_t (*rte_bbdev_dequeue_enc_ops_t)(
+ struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_enc_op **ops, uint16_t num);
+
+/** @internal Dequeue decode operations from a queue of a device. */
+typedef uint16_t (*rte_bbdev_dequeue_dec_ops_t)(
+ struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_dec_op **ops, uint16_t num);
+
+#ifndef RTE_BBDEV_NAME_MAX_LEN
+#define RTE_BBDEV_NAME_MAX_LEN 64 /**< Max length of device name */
+#endif
+
+/**
+ * @internal The data associated with a device, with no function pointers.
+ * This structure is safe to place in shared memory to be common among
+ * different processes in a multi-process configuration. Drivers can access
+ * these fields, but should never write to them!
+ */
+struct rte_bbdev_data {
+ char name[RTE_BBDEV_NAME_MAX_LEN]; /**< Unique identifier name */
+ void *dev_private; /**< Driver-specific private data */
+ uint16_t num_queues; /**< Number of currently configured queues */
+ struct rte_bbdev_queue_data *queues; /**< Queue structures */
+ uint16_t dev_id; /**< Device ID */
+ int socket_id; /**< NUMA socket that device is on */
+ bool started; /**< Device run-time state */
+};
+
+/* Forward declarations */
+struct rte_bbdev_ops;
+struct rte_bbdev_callback;
+struct rte_intr_handle;
+
+/** Structure to keep track of registered callbacks */
+TAILQ_HEAD(rte_bbdev_cb_list, rte_bbdev_callback);
+
+/**
+ * @internal The data structure associated with a device. Drivers can access
+ * these fields, but should only write to the *_ops fields.
+ */
+struct __rte_cache_aligned rte_bbdev {
+ /**< Enqueue encode function */
+ rte_bbdev_enqueue_enc_ops_t enqueue_enc_ops;
+ /**< Enqueue decode function */
+ rte_bbdev_enqueue_dec_ops_t enqueue_dec_ops;
+ /**< Dequeue encode function */
+ rte_bbdev_dequeue_enc_ops_t dequeue_enc_ops;
+ /**< Dequeue decode function */
+ rte_bbdev_dequeue_dec_ops_t dequeue_dec_ops;
+ const struct rte_bbdev_ops *dev_ops; /**< Functions exported by PMD */
+ struct rte_bbdev_data *data; /**< Pointer to device data */
+ enum rte_bbdev_state state; /**< If device is currently used or not */
+ struct rte_device *device; /**< Backing device */
+ /** User application callback for interrupts if present */
+ struct rte_bbdev_cb_list list_cbs;
+ struct rte_intr_handle *intr_handle; /**< Device interrupt handle */
+};
+
+/** @internal array of all devices */
+extern struct rte_bbdev rte_bbdev_devices[];
+
+/**
+ * Enqueue a burst of processed encode operations to a queue of the device.
+ * This functions only enqueues as many operations as currently possible and
+ * does not block until @p num_ops entries in the queue are available.
+ * This function does not provide any error notification to avoid the
+ * corresponding overhead.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param queue_id
+ * The index of the queue.
+ * @param ops
+ * Pointer array containing operations to be enqueued Must have at least
+ * @p num_ops entries
+ * @param num_ops
+ * The maximum number of operations to enqueue.
+ *
+ * @return
+ * The number of operations actually enqueued (this is the number of processed
+ * entries in the @p ops array).
+ */
+static inline uint16_t
+rte_bbdev_enqueue_enc_ops(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_enc_op **ops, uint16_t num_ops)
+{
+ struct rte_bbdev *dev = &rte_bbdev_devices[dev_id];
+ struct rte_bbdev_queue_data *q_data = &dev->data->queues[queue_id];
+ uint16_t n = dev->enqueue_enc_ops(q_data, ops, num_ops);
+
+ rte_bbdev_log_verbose("%u encode ops enqueued to dev%u,q%u.\n",
+ num_ops, dev_id, queue_id);
+
+ return n;
+}
+
+/**
+ * Enqueue a burst of processed decode operations to a queue of the device.
+ * This functions only enqueues as many operations as currently possible and
+ * does not block until @p num_ops entries in the queue are available.
+ * This function does not provide any error notification to avoid the
+ * corresponding overhead.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param queue_id
+ * The index of the queue.
+ * @param ops
+ * Pointer array containing operations to be enqueued Must have at least
+ * @p num_ops entries
+ * @param num_ops
+ * The maximum number of operations to enqueue.
+ *
+ * @return
+ * The number of operations actually enqueued (this is the number of processed
+ * entries in the @p ops array).
+ */
+static inline uint16_t
+rte_bbdev_enqueue_dec_ops(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_dec_op **ops, uint16_t num_ops)
+{
+ struct rte_bbdev *dev = &rte_bbdev_devices[dev_id];
+ struct rte_bbdev_queue_data *q_data = &dev->data->queues[queue_id];
+ uint16_t n = dev->enqueue_dec_ops(q_data, ops, num_ops);
+
+ rte_bbdev_log_verbose("%u decode ops enqueued to dev%u,q%u.\n",
+ num_ops, dev_id, queue_id);
+
+ return n;
+}
+
+/**
+ * Dequeue a burst of processed encode operations from a queue of the device.
+ * This functions returns only the current contents of the queue, and does not
+ * block until @ num_ops is available.
+ * This function does not provide any error notification to avoid the
+ * corresponding overhead.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param queue_id
+ * The index of the queue.
+ * @param ops
+ * Pointer array where operations will be dequeued to. Must have at least
+ * @p num_ops entries
+ * @param num_ops
+ * The maximum number of operations to dequeue.
+ *
+ * @return
+ * The number of operations actually dequeued (this is the number of entries
+ * copied into the @p ops array).
+ */
+static inline uint16_t
+rte_bbdev_dequeue_enc_ops(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_enc_op **ops, uint16_t num_ops)
+{
+ struct rte_bbdev *dev = &rte_bbdev_devices[dev_id];
+ struct rte_bbdev_queue_data *q_data = &dev->data->queues[queue_id];
+ uint16_t n = dev->dequeue_enc_ops(q_data, ops, num_ops);
+
+ rte_bbdev_log_verbose("%u encode ops dequeued to dev%u,q%u\n",
+ n, dev_id, queue_id);
+
+ return n;
+}
+
+/**
+ * Dequeue a burst of processed decode operations from a queue of the device.
+ * This functions returns only the current contents of the queue, and does not
+ * block until @ num_ops is available.
+ * This function does not provide any error notification to avoid the
+ * corresponding overhead.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @param queue_id
+ * The index of the queue.
+ * @param ops
+ * Pointer array where operations will be dequeued to. Must have at least
+ * @p num_ops entries
+ * @param num_ops
+ * The maximum number of operations to dequeue.
+ *
+ * @return
+ * The number of operations actually dequeued (this is the number of entries
+ * copied into the @p ops array).
+ */
+
+static inline uint16_t
+rte_bbdev_dequeue_dec_ops(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_dec_op **ops, uint16_t num_ops)
+{
+ struct rte_bbdev *dev = &rte_bbdev_devices[dev_id];
+ struct rte_bbdev_queue_data *q_data = &dev->data->queues[queue_id];
+ uint16_t n = dev->dequeue_dec_ops(q_data, ops, num_ops);
+
+ rte_bbdev_log_verbose("%u decode ops dequeued to dev%u,q%u\n",
+ n, dev_id, queue_id);
+
+ return n;
+}
+
+/** Definitions of device event types */
+enum rte_bbdev_event_type {
+ RTE_BBDEV_EVENT_UNKNOWN, /**< unknown event type */
+ RTE_BBDEV_EVENT_ERROR, /**< error interrupt event */
+ RTE_BBDEV_EVENT_DEQUEUE, /**< dequeue event */
+ RTE_BBDEV_EVENT_MAX /**< max value of this enum */
+};
+
+/**
+ * Typedef for application callback function registered by application
+ * software for notification of device events
+ *
+ * @param dev_id
+ * Device identifier
+ * @param event
+ * Device event to register for notification of.
+ * @param cb_arg
+ * User specified parameter to be passed to user's callback function.
+ * @param ret_param
+ * To pass data back to user application.
+ */
+typedef void (*rte_bbdev_cb_fn)(uint16_t dev_id,
+ enum rte_bbdev_event_type event, void *cb_arg,
+ void *ret_param);
+
+/**
+ * Register a callback function for specific device id. Multiple callbacks can
+ * be added and will be called in the order they are added when an event is
+ * triggered. Callbacks are called in a separate thread created by the DPDK EAL.
+ *
+ * @param dev_id
+ * Device id.
+ * @param event
+ * The event that the callback will be registered for.
+ * @param cb_fn
+ * User supplied callback function to be called.
+ * @param cb_arg
+ * Pointer to parameter that will be passed to the callback.
+ *
+ * @return
+ * Zero on success, negative value on failure.
+ */
+int
+rte_bbdev_callback_register(uint16_t dev_id, enum rte_bbdev_event_type event,
+ rte_bbdev_cb_fn cb_fn, void *cb_arg);
+
+/**
+ * Unregister a callback function for specific device id.
+ *
+ * @param dev_id
+ * The device identifier.
+ * @param event
+ * The event that the callback will be unregistered for.
+ * @param cb_fn
+ * User supplied callback function to be unregistered.
+ * @param cb_arg
+ * Pointer to the parameter supplied when registering the callback.
+ * (void *)-1 means to remove all registered callbacks with the specified
+ * function address.
+ *
+ * @return
+ * - 0 on success
+ * - EINVAL if invalid parameter pointer is provided
+ * - EAGAIN if the provided callback pointer does not exist
+ */
+int
+rte_bbdev_callback_unregister(uint16_t dev_id, enum rte_bbdev_event_type event,
+ rte_bbdev_cb_fn cb_fn, void *cb_arg);
+
+/**
+ * Enable a one-shot interrupt on the next operation enqueued to a particular
+ * queue. The interrupt will be triggered when the operation is ready to be
+ * dequeued. To handle the interrupt, an epoll file descriptor must be
+ * registered using rte_bbdev_queue_intr_ctl(), and then an application
+ * thread/lcore can wait for the interrupt using rte_epoll_wait().
+ *
+ * @param dev_id
+ * The device identifier.
+ * @param queue_id
+ * The index of the queue.
+ *
+ * @return
+ * - 0 on success
+ * - negative value on failure - as returned from PMD driver
+ */
+int
+rte_bbdev_queue_intr_enable(uint16_t dev_id, uint16_t queue_id);
+
+/**
+ * Disable a one-shot interrupt on the next operation enqueued to a particular
+ * queue (if it has been enabled).
+ *
+ * @param dev_id
+ * The device identifier.
+ * @param queue_id
+ * The index of the queue.
+ *
+ * @return
+ * - 0 on success
+ * - negative value on failure - as returned from PMD driver
+ */
+int
+rte_bbdev_queue_intr_disable(uint16_t dev_id, uint16_t queue_id);
+
+/**
+ * Control interface for per-queue interrupts.
+ *
+ * @param dev_id
+ * The device identifier.
+ * @param queue_id
+ * The index of the queue.
+ * @param epfd
+ * Epoll file descriptor that will be associated with the interrupt source.
+ * If the special value RTE_EPOLL_PER_THREAD is provided, a per thread epoll
+ * file descriptor created by the EAL is used (RTE_EPOLL_PER_THREAD can also
+ * be used when calling rte_epoll_wait()).
+ * @param op
+ * The operation be performed for the vector.RTE_INTR_EVENT_ADD or
+ * RTE_INTR_EVENT_DEL.
+ * @param data
+ * User context, that will be returned in the epdata.data field of the
+ * rte_epoll_event structure filled in by rte_epoll_wait().
+ *
+ * @return
+ * - 0 on success
+ * - ENOTSUP if interrupts are not supported by the identified device
+ * - negative value on failure - as returned from PMD driver
+ */
+int
+rte_bbdev_queue_intr_ctl(uint16_t dev_id, uint16_t queue_id, int epfd, int op,
+ void *data);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_BBDEV_H_ */
diff --git a/lib/librte_bbdev/rte_bbdev_op.h b/lib/librte_bbdev/rte_bbdev_op.h
new file mode 100644
index 0000000..99ad899
--- /dev/null
+++ b/lib/librte_bbdev/rte_bbdev_op.h
@@ -0,0 +1,514 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_BBDEV_OP_H_
+#define _RTE_BBDEV_OP_H_
+
+/**
+ * @file rte_bbdev_op.h
+ *
+ * Defines wireless base band layer 1 operations and capabilities
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_mbuf.h>
+#include <rte_memory.h>
+#include <rte_mempool.h>
+
+#define RTE_BBDEV_MAX_CODE_BLOCKS 64
+
+extern int bbdev_logtype;
+
+/**
+ * Helper macro for logging
+ *
+ * @param level
+ * Log level: EMERG, ALERT, CRIT, ERR, WARNING, NOTICE, INFO, or DEBUG
+ * @param fmt
+ * The format string, as in printf(3).
+ * @param ...
+ * The variable arguments required by the format string.
+ *
+ * @return
+ * - 0 on success
+ * - Negative on error
+ */
+#define rte_bbdev_log(level, fmt, ...) \
+ rte_log(RTE_LOG_ ## level, bbdev_logtype, fmt "\n", ##__VA_ARGS__)
+
+/**
+ * Helper macro for debug logging with extra source info
+ *
+ * @param fmt
+ * The format string, as in printf(3).
+ * @param ...
+ * The variable arguments required by the format string.
+ *
+ * @return
+ * - 0 on success
+ * - Negative on error
+ */
+#define rte_bbdev_log_debug(fmt, ...) \
+ rte_bbdev_log(DEBUG, RTE_STR(__LINE__) ":%s() " fmt, __func__, \
+ ##__VA_ARGS__)
+
+/**
+ * Helper macro for extra conditional logging from datapath
+ *
+ * @param fmt
+ * The format string, as in printf(3).
+ * @param ...
+ * The variable arguments required by the format string.
+ *
+ * @return
+ * - 0 on success
+ * - Negative on error
+ */
+#define rte_bbdev_log_verbose(fmt, ...) \
+ (void)((RTE_LOG_DEBUG <= RTE_LOG_DP_LEVEL) ? \
+ rte_log(RTE_LOG_DEBUG, \
+ bbdev_logtype, ": " fmt "\n", ##__VA_ARGS__) : 0)
+
+/** Flags for turbo decoder operation and capability structure */
+enum rte_bbdev_op_td_flag_bitmasks {
+ /** If sub block de-interleaving is to be performed. */
+ RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE = (1ULL << 0),
+ /** To use CRC Type 24B (otherwise use CRC Type 24A). */
+ RTE_BBDEV_TURBO_CRC_TYPE_24B = (1ULL << 1),
+ /** If turbo equalization is to be performed. */
+ RTE_BBDEV_TURBO_EQUALIZER = (1ULL << 2),
+ /** If set, saturate soft output to +/-127 */
+ RTE_BBDEV_TURBO_SOFT_OUT_SATURATE = (1ULL << 3),
+ /**
+ * Set to 1 to start iteration from even, else odd; one iteration =
+ * max_iteration + 0.5
+ */
+ RTE_BBDEV_TURBO_HALF_ITERATION_EVEN = (1ULL << 4),
+ /**
+ * If 0, TD stops after CRC matches; else if 1, runs to end of next
+ * odd iteration after CRC matches
+ */
+ RTE_BBDEV_TURBO_CONTINUE_CRC_MATCH = (1ULL << 5),
+ /** Set if soft output is required to be output */
+ RTE_BBDEV_TURBO_SOFT_OUTPUT = (1ULL << 6),
+ /** Set to enable early termination mode */
+ RTE_BBDEV_TURBO_EARLY_TERMINATION = (1ULL << 7),
+ /**
+ * Set if the input is a raw data (E bytes, no NULL bytes). If not set
+ * the input is a full circular buffer with data (Kw bytes) as decribed
+ * in spec. 36.212, chapter 5.1.4.1.2.
+ */
+ RTE_BBDEV_TURBO_RAW_INPUT_DATA = (1ULL << 8),
+ /** Set if a device supports decoder dequeue interrupts */
+ RTE_BBDEV_TURBO_DEC_INTERRUPTS = (1ULL << 9),
+};
+
+/** Flags for turbo encoder operation and capability structure */
+enum rte_bbdev_op_te_flag_bitmasks {
+ /** Ignore rv_index and set K0 = 0 */
+ RTE_BBDEV_TURBO_RV_INDEX_BYPASS = (1ULL << 0),
+ /** If rate matching is to be performed */
+ RTE_BBDEV_TURBO_RATE_MATCH = (1ULL << 1),
+ /** This bit must be set to enable CRC-24B generation */
+ RTE_BBDEV_TURBO_CRC_24B_ATTACH = (1ULL << 2),
+ /** This bit must be set to enable CRC-24A generation */
+ RTE_BBDEV_TURBO_CRC_24A_ATTACH = (1ULL << 3),
+ /** Set if a device supports encoder dequeue interrupts */
+ RTE_BBDEV_TURBO_ENC_INTERRUPTS = (1ULL << 4)
+};
+
+/** Data input and output buffer for BBDEV operations */
+struct rte_bbdev_op_data {
+ struct rte_mbuf *data;
+ /**< First mbuf segment with input/output data. Each segment represents
+ * one Code Block. For the input operation a mbuf needs to contain
+ * all Code Blocks. For the output operation the mbuf should consist of
+ * only one segment and the driver will take care of allocating and
+ * chaining another segments for the consecutive Code Blocks if needed.
+ */
+ uint32_t offset;
+ /**< The starting point for the Turbo input/output, in bytes, from the
+ * start of the first segment's data buffer. It must be smaller than the
+ * first segment's data_len!
+ */
+ uint32_t length;
+ /**< Length of Transport Block - number of bytes for Turbo Encode/Decode
+ * operation for input; length of the output for output operation.
+ */
+};
+
+struct rte_bbdev_op_dec_cb_params {
+ uint16_t k; /**< size of the input code block in bits (40 - 6144) */
+ uint32_t e; /**< length in bits of the rate match output (17 bits) */
+};
+
+struct rte_bbdev_op_dec_tb_params {
+ /**< size of the input code block in bits (40 - 6144). Used when
+ * code block index r < c_neg
+ */
+ uint16_t k_neg;
+ /**< size of the input code block in bits (40 - 6144). Used when
+ * code block index r >= c_neg
+ */
+ uint16_t k_pos;
+ uint8_t c_neg; /**< number of code block using k_neg (0 - 63) */
+ uint8_t c; /**< total number of code blocks (1 - 64) */
+ uint8_t cab; /**< number of code blocks using ea before switch to eb */
+ /**< length in bits of the rate match output (17 bits). Used when
+ * code block index r < cab
+ */
+ uint32_t ea;
+ /**< length in bits of the rate match output (17 bits). Used when
+ * code block index r >= cab
+ */
+ uint32_t eb;
+ uint8_t cb_idx; /**< Code block index */
+};
+
+/** Operation structure for the Turbo Decoder */
+struct rte_bbdev_op_turbo_dec {
+ struct rte_bbdev_op_data input; /**< input src data */
+ struct rte_bbdev_op_data hard_output; /**< hard output buffer */
+ struct rte_bbdev_op_data soft_output; /**< soft output buffer */
+
+ uint32_t op_flags; /**< Flags from rte_bbdev_op_td_flag_bitmasks */
+ uint8_t rv_index; /**< Rv index for rate matching (0 - 3) */
+ uint8_t iter_min:4; /**< min number of iterations */
+ uint8_t iter_max:4; /**< max number of iterations */
+ uint8_t iter_count; /**< Actual num. of iterations performed */
+ /** 5 bit extrinsic scale (scale factor on extrinsic info) */
+ uint8_t ext_scale;
+ /** Number of MAP engines, must be power of 2 (or 0 to auto-select) */
+ uint8_t num_maps;
+ uint8_t code_block_mode; /**< 0 - transpot block, 1 - code block */
+ union {
+ /** Struct which stores Code Block specific parameters */
+ struct rte_bbdev_op_dec_cb_params cb_params;
+ /** Struct which stores Transport Block specific parameters */
+ struct rte_bbdev_op_dec_tb_params tb_params;
+ };
+};
+
+struct rte_bbdev_op_enc_cb_params {
+ uint16_t k; /**< size of the input code block in bits (40 - 6144) */
+ uint32_t e; /**< length in bits of the rate match output (17 bits) */
+ uint16_t ncb; /**< Ncb parameter for rate matching, range [k:3(k+4)] */
+};
+
+struct rte_bbdev_op_enc_tb_params {
+ /**< size of the input code block in bits (40 - 6144). Used when
+ * code block index r < c_neg
+ */
+ uint16_t k_neg;
+ /**< size of the input code block in bits (40 - 6144). Used when
+ * code block index r >= c_neg
+ */
+ uint16_t k_pos;
+ uint8_t c_neg; /**< number of code block using k_neg (0 - 63) */
+ uint8_t c; /**< total number of code blocks (1 - 64) */
+ uint8_t cab; /**< number of code blocks using ea before switch to eb */
+ /**< length in bits of the rate match output (17 bits). Used when
+ * code block index r < cab
+ */
+ uint32_t ea;
+ /**< length in bits of the rate match output (17 bits). Used when
+ * code block index r >= cab
+ */
+ uint32_t eb;
+ /**< Ncb parameter for rate matching, range [k : 3(k+4)]. Used when
+ * code block index r < c_neg
+ */
+ uint16_t ncb_neg;
+ /**< Ncb parameter for rate matching, range [k : 3(k+4)]. Used when
+ * code block index r >= c_neg
+ */
+ uint16_t ncb_pos;
+ uint8_t cb_idx; /**< Code block index */
+};
+
+/** Operation structure for the Turbo Encoder */
+struct rte_bbdev_op_turbo_enc {
+ struct rte_bbdev_op_data input; /**< input src data */
+ struct rte_bbdev_op_data output; /**< output buffer */
+
+ uint32_t op_flags; /**< Flags from rte_bbdev_op_te_flag_bitmasks */
+ int32_t n_soft; /**< total number of soft bits according to UE cat. */
+ int32_t k_mimo; /**< MIMO type */
+ int32_t mdl_harq; /**< the maximum number of DL HARQ processes */
+ /**< total number of bits available for transmission of one TB */
+ int32_t g;
+ int32_t nl; /**< number of layer */
+ int32_t qm; /**< modulation type */
+ uint8_t rv_index; /**< Rv index for rate matching (0 - 3) */
+ uint8_t code_block_mode; /**< 0 - transpot block, 1 - code block */
+ union {
+ /** Struct which stores Code Block specific parameters */
+ struct rte_bbdev_op_enc_cb_params cb_params;
+ /** Struct which stores Transport Block specific parameters */
+ struct rte_bbdev_op_enc_tb_params tb_params;
+ };
+};
+
+/** List of the capabilities for the Turbo Decoder */
+struct rte_bbdev_op_cap_turbo_dec {
+ /** Flags from rte_bbdev_op_td_flag_bitmasks */
+ uint32_t capability_flags;
+ uint8_t num_buffers_src; /**< Num input code block buffers */
+ /** Num hard output code block buffers */
+ uint8_t num_buffers_hard_out;
+ /** Num soft output code block buffers if supported by the driver */
+ uint8_t num_buffers_soft_out;
+};
+
+/** List of the capabilities for the Turbo Encoder */
+struct rte_bbdev_op_cap_turbo_enc {
+ /** Flags from rte_bbdev_op_te_flag_bitmasks */
+ uint32_t capability_flags;
+ uint8_t num_buffers_src; /**< Num input code block buffers */
+ uint8_t num_buffers_dst; /**< Num output code block buffers */
+};
+
+/** Different operation types supported by the device */
+enum rte_bbdev_op_type {
+ RTE_BBDEV_OP_NONE, /**< Dummy operation that does nothing */
+ RTE_BBDEV_OP_TURBO_DEC, /**< Turbo decode */
+ RTE_BBDEV_OP_TURBO_ENC, /**< Turbo encode */
+ RTE_BBDEV_OP_TYPE_COUNT, /**< Count of different op types */
+};
+
+/** Bit indexes of possible errors reported through status field */
+enum {
+ RTE_BBDEV_DRV_ERROR,
+ RTE_BBDEV_DATA_ERROR,
+ RTE_BBDEV_CRC_ERROR,
+};
+
+/** Structure specifying a single encode operation */
+struct rte_bbdev_enc_op {
+ int status; /**< Status of operation that was performed */
+ struct rte_mempool *mempool; /**< Mempool which op instance is in */
+ void *opaque_data; /**< Opaque pointer for user data */
+ /** Contains encoder specific parameters */
+ struct rte_bbdev_op_turbo_enc turbo_enc;
+};
+
+/** Structure specifying a single decode operation */
+struct rte_bbdev_dec_op {
+ int status; /**< Status of operation that was performed */
+ struct rte_mempool *mempool; /**< Mempool which op instance is in */
+ void *opaque_data; /**< Opaque pointer for user data */
+ /** Contains decoder specific parameters */
+ struct rte_bbdev_op_turbo_dec turbo_dec;
+};
+
+/** Operation capabilities supported by a device */
+struct rte_bbdev_op_cap {
+ enum rte_bbdev_op_type type; /**< Type of operation */
+ union {
+ struct rte_bbdev_op_cap_turbo_dec turbo_dec;
+ struct rte_bbdev_op_cap_turbo_enc turbo_enc;
+ } cap; /**< Operation-type specific capabilities */
+};
+
+/** @internal Private data structure stored with operation pool. */
+struct rte_bbdev_op_pool_private {
+ enum rte_bbdev_op_type type; /**< Type of operations in a pool */
+};
+
+/**
+ * Converts queue operation type from enum to string
+ *
+ * @param op_type
+ * Operation type as enum
+ *
+ * @returns
+ * Operation type as string
+ *
+ */
+const char*
+rte_bbdev_op_type_str(enum rte_bbdev_op_type op_type);
+
+/**
+ * Creates a bbdev operation mempool
+ *
+ * @param name
+ * Pool name.
+ * @param type
+ * Operation type, use RTE_BBDEV_OP_NONE for a pool which supports all
+ * operation types.
+ * @param num_elements
+ * Number of elements in the pool.
+ * @param cache_size
+ * Number of elements to cache on an lcore, see rte_mempool_create() for
+ * further details about cache size.
+ * @param socket_id
+ * Socket to allocate memory on.
+ *
+ * @return
+ * - Pointer to a mempool on success,
+ * - NULL pointer on failure.
+ */
+struct rte_mempool *
+rte_bbdev_op_pool_create(const char *name, enum rte_bbdev_op_type type,
+ unsigned int num_elements, unsigned int cache_size,
+ int socket_id);
+
+/**
+ * Bulk allocate encode operations from a mempool with parameter defaults reset.
+ *
+ * @param mempool
+ * Operation mempool, created by rte_bbdev_op_pool_create().
+ * @param ops
+ * Output array to place allocated operations
+ * @param num_ops
+ * Number of operations to allocate
+ *
+ * @returns
+ * - 0 on success
+ * - EINVAL if invalid mempool is provided
+ */
+static inline int
+rte_bbdev_enc_op_alloc_bulk(struct rte_mempool *mempool,
+ struct rte_bbdev_enc_op **ops, uint16_t num_ops)
+{
+ struct rte_bbdev_op_pool_private *priv;
+ int ret;
+
+ /* Check type */
+ priv = (struct rte_bbdev_op_pool_private *)
+ rte_mempool_get_priv(mempool);
+ if (unlikely(priv->type != RTE_BBDEV_OP_TURBO_ENC))
+ return -EINVAL;
+
+ /* Get elements */
+ ret = rte_mempool_get_bulk(mempool, (void **)ops, num_ops);
+ if (unlikely(ret < 0))
+ return ret;
+
+ rte_bbdev_log_verbose("%u encode ops allocated from %s\n",
+ num_ops, mempool->name);
+
+ return 0;
+}
+
+/**
+ * Bulk allocate decode operations from a mempool with parameter defaults reset.
+ *
+ * @param mempool
+ * Operation mempool, created by rte_bbdev_op_pool_create().
+ * @param ops
+ * Output array to place allocated operations
+ * @param num_ops
+ * Number of operations to allocate
+ *
+ * @returns
+ * - 0 on success
+ * - EINVAL if invalid mempool is provided
+ */
+static inline int
+rte_bbdev_dec_op_alloc_bulk(struct rte_mempool *mempool,
+ struct rte_bbdev_dec_op **ops, uint16_t num_ops)
+{
+ struct rte_bbdev_op_pool_private *priv;
+ int ret;
+
+ /* Check type */
+ priv = (struct rte_bbdev_op_pool_private *)
+ rte_mempool_get_priv(mempool);
+ if (unlikely(priv->type != RTE_BBDEV_OP_TURBO_DEC))
+ return -EINVAL;
+
+ /* Get elements */
+ ret = rte_mempool_get_bulk(mempool, (void **)ops, num_ops);
+ if (unlikely(ret < 0))
+ return ret;
+
+ rte_bbdev_log_verbose("%u encode ops allocated from %s\n",
+ num_ops, mempool->name);
+
+ return 0;
+}
+
+/**
+ * Free decode operation structures that were allocated by
+ * rte_bbdev_dec_op_alloc_bulk().
+ * All structures must belong to the same mempool.
+ *
+ * @param ops
+ * Operation structures
+ * @param num_ops
+ * Number of structures
+ */
+static inline void
+rte_bbdev_dec_op_free_bulk(struct rte_bbdev_dec_op **ops, unsigned int num_ops)
+{
+ if (num_ops > 0) {
+ rte_mempool_put_bulk(ops[0]->mempool, (void **)ops, num_ops);
+ rte_bbdev_log_verbose("%u decode ops freed to %s\n", num_ops,
+ ops[0]->mempool->name);
+ }
+}
+
+/**
+ * Free encode operation structures that were allocated by
+ * rte_bbdev_enc_op_alloc_bulk().
+ * All structures must belong to the same mempool.
+ *
+ * @param ops
+ * Operation structures
+ * @param num_ops
+ * Number of structures
+ */
+static inline void
+rte_bbdev_enc_op_free_bulk(struct rte_bbdev_enc_op **ops, unsigned int num_ops)
+{
+ if (num_ops > 0) {
+ rte_mempool_put_bulk(ops[0]->mempool, (void **)ops, num_ops);
+ rte_bbdev_log_verbose("%u encode ops freed to %s\n", num_ops,
+ ops[0]->mempool->name);
+ }
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_BBDEV_OP_H_ */
diff --git a/lib/librte_bbdev/rte_bbdev_pci.h b/lib/librte_bbdev/rte_bbdev_pci.h
new file mode 100644
index 0000000..1a32132
--- /dev/null
+++ b/lib/librte_bbdev/rte_bbdev_pci.h
@@ -0,0 +1,288 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_BBDEV_PCI_H_
+#define _RTE_BBDEV_PCI_H_
+
+/**
+ * @file rte_bbdev_pci.h
+ *
+ * Wireless base band PCI-driver-facing APIs.
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * This API provides the helper functions for device PCI drivers to register
+ * with the bbdev interface. User applications should not use this API.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+#include <rte_log.h>
+#include <rte_pci.h>
+
+#include "rte_bbdev_pmd.h"
+
+/**
+ * @internal
+ * Initialisation function of a HW driver invoked for each matching HW device
+ * detected during the EAL initialisation phase, or when a new device is
+ * attached. The driver should initialise the device and its own software
+ * context.
+ *
+ * @param dev
+ * This is a new device structure instance that is associated with the
+ * matching device.
+ * The driver *must* populate the following fields:
+ * - dev_ops
+ * - enqueue_ops
+ * - dequeue_ops
+ *
+ * @return
+ * - 0 on success
+ */
+typedef int (*rte_bbdev_init_t)(struct rte_bbdev *dev);
+
+/**
+ * @internal
+ * Finalization function of a HW driver invoked for each matching HW device
+ * detected during the closing phase, or when a device is detached.
+ *
+ * @param dev
+ * The device structure instance that is associated with the matching device.
+ *
+ * @return
+ * - 0 on success
+ */
+typedef int (*rte_bbdev_uninit_t)(struct rte_bbdev *dev);
+
+/**
+ * @internal
+ * Allocates a new bbdev slot for an PCI device and returns the pointer to that
+ * slot for the driver to use.
+ *
+ * @param dev
+ * Pointer to the PCI device
+ *
+ * @param private_data_size
+ * Size of private data structure
+ *
+ * @return
+ * A pointer to a rte_bbdev or NULL if allocation failed.
+ */
+static inline struct rte_bbdev *
+rte_bbdev_pci_allocate(struct rte_pci_device *dev, size_t private_data_size)
+{
+ const char *name;
+ struct rte_bbdev *bbdev = NULL;
+
+ if (dev == NULL) {
+ rte_bbdev_log(ERR, "NULL PCI device");
+ return NULL;
+ }
+
+ name = dev->device.name;
+
+ /* Allocate memory to be used privately by drivers */
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ bbdev = rte_bbdev_allocate(name);
+ if (!bbdev)
+ return NULL;
+
+ if (private_data_size) {
+ bbdev->data->dev_private = rte_zmalloc_socket(name,
+ private_data_size, RTE_CACHE_LINE_SIZE,
+ dev->device.numa_node);
+ if (bbdev->data->dev_private == NULL) {
+ rte_bbdev_log(CRIT,
+ "Allocate of %zu bytes for device \"%s\"failed",
+ private_data_size, name);
+ rte_bbdev_release(bbdev);
+ return NULL;
+ }
+ }
+ }
+
+ bbdev->data->socket_id = dev->device.numa_node;
+ return bbdev;
+}
+
+static inline void
+rte_bbdev_pci_release(struct rte_bbdev *bbdev)
+{
+ int ret;
+ uint16_t dev_id = bbdev->data->dev_id;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+ rte_free(bbdev->data->dev_private);
+
+ ret = rte_bbdev_release(bbdev);
+ if (ret)
+ rte_bbdev_log(ERR, "Device %i failed to uninit: %i", dev_id,
+ ret);
+
+ rte_bbdev_log_debug("Un-initialised HW device id = %u", dev_id);
+}
+
+/**
+ * @internal
+ * Wrapper for use by pci drivers as a .probe function to attach to a bbdev
+ * interface.
+ */
+static inline int
+rte_bbdev_pci_generic_probe(struct rte_pci_device *pci_dev,
+ size_t private_data_size,
+ rte_bbdev_init_t dev_init)
+{
+ struct rte_bbdev *bbdev = NULL;
+ char dev_name[RTE_BBDEV_NAME_MAX_LEN];
+ int ret;
+
+ rte_pci_device_name(&pci_dev->addr, dev_name, sizeof(dev_name));
+
+ bbdev = rte_bbdev_pci_allocate(pci_dev, private_data_size);
+ if (bbdev == NULL)
+ return -ENOMEM;
+
+ /* Fill HW specific part of device structure */
+ bbdev->device = &pci_dev->device;
+ bbdev->intr_handle = &pci_dev->intr_handle;
+
+ /* Invoke PMD device initialization function */
+ if (dev_init) {
+ ret = dev_init(bbdev);
+
+ if (bbdev->dev_ops == NULL) {
+ rte_bbdev_log(ERR, "NULL dev_ops structure in device %u",
+ bbdev->data->dev_id);
+ return -ENODEV;
+ }
+
+ if (bbdev->enqueue_enc_ops == NULL) {
+ rte_bbdev_log(ERR,
+ "NULL enqueue_enc_ops structure in device %u",
+ bbdev->data->dev_id);
+ return -ENODEV;
+ }
+ if (bbdev->enqueue_dec_ops == NULL) {
+ rte_bbdev_log(ERR,
+ "NULL enqueue_dec_ops structure in device %u",
+ bbdev->data->dev_id);
+ return -ENODEV;
+ }
+ if (bbdev->dequeue_enc_ops == NULL) {
+ rte_bbdev_log(ERR,
+ "NULL dequeue_enc_ops structure in device %u",
+ bbdev->data->dev_id);
+ return -ENODEV;
+ }
+ if (bbdev->dequeue_dec_ops == NULL) {
+ rte_bbdev_log(ERR,
+ "NULL dequeue_dec_ops structure in device %u",
+ bbdev->data->dev_id);
+ return -ENODEV;
+ }
+
+ if (ret < 0) {
+ rte_bbdev_log(ERR,
+ "Driver %s(vendor_id=0x%x device_id=0x%x): failed: %i",
+ pci_dev->driver->driver.name,
+ pci_dev->id.vendor_id,
+ pci_dev->id.device_id, ret);
+ rte_bbdev_pci_release(bbdev);
+ return -ENXIO;
+ }
+ } else {
+ rte_bbdev_log(ERR,
+ "Device init function doesn't exist for driver %s",
+ pci_dev->driver->driver.name);
+ return -ENODEV;
+ }
+
+ rte_bbdev_log_debug("Initialised HW device %s (id = %u)",
+ dev_name, bbdev->data->dev_id);
+ return 0;
+}
+
+/**
+ * @internal
+ * Wrapper for use by pci drivers as a .remove function to detach a bbdev
+ * interface.
+ */
+static inline int
+rte_bbdev_pci_generic_remove(struct rte_pci_device *pci_dev,
+ rte_bbdev_uninit_t dev_uninit)
+{
+ struct rte_bbdev *bbdev;
+ int ret;
+ uint8_t dev_id;
+
+ if (pci_dev == NULL)
+ return -EINVAL;
+
+ /* Find device */
+ bbdev = rte_bbdev_get_named_dev(pci_dev->device.name);
+ if (bbdev == NULL) {
+ rte_bbdev_log(CRIT,
+ "Couldn't find HW dev \"%s\" to uninitialise it",
+ pci_dev->device.name);
+ return -ENODEV;
+ }
+ dev_id = bbdev->data->dev_id;
+
+ /* Close device before uninit */
+ ret = rte_bbdev_close(dev_id);
+ if (ret < 0)
+ rte_bbdev_log(ERR,
+ "Device %i failed to close during uninit: %i",
+ dev_id, ret);
+
+ /* Invoke PMD device uninit function */
+ if (dev_uninit) {
+ ret = dev_uninit(bbdev);
+ if (ret)
+ rte_bbdev_log(ERR, "Device %i failed to uninit: %i",
+ dev_id, ret);
+ }
+
+ rte_bbdev_pci_release(bbdev);
+ return 0;
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_BBDEV_PCI_H_ */
diff --git a/lib/librte_bbdev/rte_bbdev_pmd.h b/lib/librte_bbdev/rte_bbdev_pmd.h
new file mode 100644
index 0000000..cf65de0
--- /dev/null
+++ b/lib/librte_bbdev/rte_bbdev_pmd.h
@@ -0,0 +1,223 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_BBDEV_PMD_H_
+#define _RTE_BBDEV_PMD_H_
+
+/**
+ * @file rte_bbdev_pmd.h
+ *
+ * Wireless base band driver-facing APIs.
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * This API provides the mechanism for device drivers to register with the
+ * bbdev interface. User applications should not use this API.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+#include <rte_log.h>
+
+#include "rte_bbdev.h"
+
+/** Suggested value for SW based devices */
+#define RTE_BBDEV_DEFAULT_MAX_NB_QUEUES RTE_MAX_LCORE
+
+/** Suggested value for SW based devices */
+#define RTE_BBDEV_QUEUE_SIZE_LIMIT 16384
+
+/**
+ * @internal
+ * Allocates a new slot for a bbdev and returns the pointer to that slot
+ * for the driver to use.
+ *
+ * @param name
+ * Unique identifier name for each bbdev device
+ *
+ * @return
+ * - Slot in the rte_bbdev array for a new device;
+ */
+struct rte_bbdev *
+rte_bbdev_allocate(const char *name);
+
+/**
+ * @internal
+ * Release the specified bbdev.
+ *
+ * @param bbdev
+ * The *bbdev* pointer is the address of the *rte_bbdev* structure.
+ * @return
+ * - 0 on success, negative on error
+ */
+int
+rte_bbdev_release(struct rte_bbdev *bbdev);
+
+/**
+ * Get the device structure for a named device.
+ *
+ * @param name
+ * Name of the device
+ *
+ * @return
+ * - The device structure pointer, or
+ * - NULL otherwise
+ *
+ */
+struct rte_bbdev *
+rte_bbdev_get_named_dev(const char *name);
+
+/**
+ * Definitions of all functions exported by a driver through the the generic
+ * structure of type *rte_bbdev_ops* supplied in the *rte_bbdev* structure
+ * associated with a device.
+ */
+
+/** @internal Function used to configure device memory. */
+typedef int (*rte_bbdev_setup_queues_t)(struct rte_bbdev *dev,
+ uint16_t num_queues, int socket_id);
+
+/** @internal Function used to configure interrupts for a device. */
+typedef int (*rte_bbdev_intr_enable_t)(struct rte_bbdev *dev);
+
+/** @internal Function to allocate and configure a device queue. */
+typedef int (*rte_bbdev_queue_setup_t)(struct rte_bbdev *dev,
+ uint16_t queue_id, const struct rte_bbdev_queue_conf *conf);
+
+/* @internal
+ * Function to release memory resources allocated for a device queue.
+ */
+typedef int (*rte_bbdev_queue_release_t)(struct rte_bbdev *dev,
+ uint16_t queue_id);
+
+/** @internal Function to start a configured device. */
+typedef int (*rte_bbdev_start_t)(struct rte_bbdev *dev);
+
+/** @internal Function to stop a device. */
+typedef void (*rte_bbdev_stop_t)(struct rte_bbdev *dev);
+
+/** @internal Function to close a device. */
+typedef int (*rte_bbdev_close_t)(struct rte_bbdev *dev);
+
+/** @internal Function to start a device queue. */
+typedef int (*rte_bbdev_queue_start_t)(struct rte_bbdev *dev,
+ uint16_t queue_id);
+
+/** @internal Function to stop a device queue. */
+typedef int (*rte_bbdev_queue_stop_t)(struct rte_bbdev *dev, uint16_t queue_id);
+
+/** @internal Function to read stats from a device. */
+typedef void (*rte_bbdev_stats_get_t)(struct rte_bbdev *dev,
+ struct rte_bbdev_stats *stats);
+
+/** @internal Function to reset stats on a device. */
+typedef void (*rte_bbdev_stats_reset_t)(struct rte_bbdev *dev);
+
+/** @internal Function to retrieve specific information of a device. */
+typedef void (*rte_bbdev_info_get_t)(struct rte_bbdev *dev,
+ struct rte_bbdev_driver_info *dev_info);
+
+/* @internal
+ * Function to enable interrupt for next op on a queue of a device.
+ */
+typedef int (*rte_bbdev_queue_intr_enable_t)(struct rte_bbdev *dev,
+ uint16_t queue_id);
+
+/* @internal
+ * Function to disable interrupt for next op on a queue of a device.
+ */
+typedef int (*rte_bbdev_queue_intr_disable_t)(struct rte_bbdev *dev,
+ uint16_t queue_id);
+
+/**
+ * Operations implemented by drivers. Fields marked as "Required" must be
+ * provided by a driver for a device to have basic functionality. "Optional"
+ * fields are for non-vital operations
+ */
+struct rte_bbdev_ops {
+ /**< Allocate and configure device memory. Optional. */
+ rte_bbdev_setup_queues_t setup_queues;
+ /**< Configure interrupts. Optional. */
+ rte_bbdev_intr_enable_t intr_enable;
+ /**< Start device. Optional. */
+ rte_bbdev_start_t start;
+ /**< Stop device. Optional. */
+ rte_bbdev_stop_t stop;
+ /**< Close device. Optional. */
+ rte_bbdev_close_t close;
+
+ /**< Get device info. Required. */
+ rte_bbdev_info_get_t info_get;
+ /** Get device statistics. Optional. */
+ rte_bbdev_stats_get_t stats_get;
+ /** Reset device statistics. Optional. */
+ rte_bbdev_stats_reset_t stats_reset;
+
+ /** Set up a device queue. Required. */
+ rte_bbdev_queue_setup_t queue_setup;
+ /** Release a queue. Required. */
+ rte_bbdev_queue_release_t queue_release;
+ /** Start a queue. Optional. */
+ rte_bbdev_queue_start_t queue_start;
+ /**< Stop a queue pair. Optional. */
+ rte_bbdev_queue_stop_t queue_stop;
+
+ /** Enable queue interrupt. Optional */
+ rte_bbdev_queue_intr_enable_t queue_intr_enable;
+ /** Disable queue interrupt. Optional */
+ rte_bbdev_queue_intr_disable_t queue_intr_disable;
+};
+
+/**
+ * Executes all the user application registered callbacks for the specific
+ * device and event type.
+ *
+ * @param dev
+ * Pointer to the device structure.
+ * @param event
+ * Event type.
+ * @param ret_param
+ * To pass data back to user application.
+ */
+void
+rte_bbdev_pmd_callback_process(struct rte_bbdev *dev,
+ enum rte_bbdev_event_type event, void *ret_param);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_BBDEV_PMD_H_ */
diff --git a/lib/librte_bbdev/rte_bbdev_vdev.h b/lib/librte_bbdev/rte_bbdev_vdev.h
new file mode 100644
index 0000000..fbaef2e
--- /dev/null
+++ b/lib/librte_bbdev/rte_bbdev_vdev.h
@@ -0,0 +1,102 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_BBDEV_VDEV_H_
+#define _RTE_BBDEV_VDEV_H_
+
+/**
+ * @file rte_bbdev_vdev.h
+ *
+ * Wireless base band virtual device driver-facing APIs.
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * This API provides the helper functions for virtual device drivers to register
+ * with the bbdev interface. User applications should not use this API.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+#include <rte_vdev.h>
+
+#include "rte_bbdev_pmd.h"
+
+/**
+ * @internal
+ * Allocates a new slot for a virtual bbdev and returns the pointer to that slot
+ * for the driver to use.
+ *
+ * @param dev
+ * Pointer to virtual device
+ *
+ * @param private_data_size
+ * Size of private data structure
+ *
+ * @return
+ * A pointer to a rte_bbdev or NULL if allocation failed.
+ */
+static inline struct rte_bbdev *
+rte_bbdev_vdev_allocate(struct rte_vdev_device *dev, size_t private_data_size)
+{
+ struct rte_bbdev *bbdev;
+ const char *name = rte_vdev_device_name(dev);
+
+ bbdev = rte_bbdev_allocate(name);
+ if (!bbdev)
+ return NULL;
+
+ if (private_data_size) {
+ bbdev->data->dev_private = rte_zmalloc_socket(name,
+ private_data_size, RTE_CACHE_LINE_SIZE,
+ dev->device.numa_node);
+ if (!bbdev->data->dev_private) {
+ rte_bbdev_release(bbdev);
+ return NULL;
+ }
+ }
+
+ bbdev->data->socket_id = dev->device.numa_node;
+ bbdev->device = &dev->device;
+ bbdev->intr_handle = NULL;
+
+ return bbdev;
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_BBDEV_VDEV_H_ */
diff --git a/lib/librte_bbdev/rte_bbdev_version.map b/lib/librte_bbdev/rte_bbdev_version.map
new file mode 100644
index 0000000..316a275
--- /dev/null
+++ b/lib/librte_bbdev/rte_bbdev_version.map
@@ -0,0 +1,37 @@
+EXPERIMENTAL {
+ global:
+
+ rte_bbdev_allocate;
+ rte_bbdev_callback_register;
+ rte_bbdev_callback_unregister;
+ rte_bbdev_close;
+ rte_bbdev_setup_queues;
+ rte_bbdev_intr_enable;
+ rte_bbdev_count;
+ rte_bbdev_dequeue_dec_ops;
+ rte_bbdev_dequeue_enc_ops;
+ rte_bbdev_devices;
+ rte_bbdev_enqueue_dec_ops;
+ rte_bbdev_enqueue_enc_ops;
+ rte_bbdev_find_next;
+ rte_bbdev_get_named_dev;
+ rte_bbdev_info_get;
+ rte_bbdev_is_valid;
+ rte_bbdev_op_pool_create;
+ rte_bbdev_op_type_str;
+ rte_bbdev_pmd_callback_process;
+ rte_bbdev_queue_configure;
+ rte_bbdev_queue_info_get;
+ rte_bbdev_queue_intr_ctl;
+ rte_bbdev_queue_intr_disable;
+ rte_bbdev_queue_intr_enable;
+ rte_bbdev_queue_start;
+ rte_bbdev_queue_stop;
+ rte_bbdev_release;
+ rte_bbdev_start;
+ rte_bbdev_stats_get;
+ rte_bbdev_stats_reset;
+ rte_bbdev_stop;
+
+ local: *;
+};
\ No newline at end of file
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 8192b98..e0f9d13 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -94,6 +94,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_NET) += -lrte_net
_LDLIBS-$(CONFIG_RTE_LIBRTE_ETHER) += -lrte_ethdev
_LDLIBS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += -lrte_cryptodev
_LDLIBS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += -lrte_eventdev
+_LDLIBS-$(CONFIG_RTE_LIBRTE_BBDEV) += -lrte_bbdev
_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL) += -lrte_mempool
_LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring
_LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring
@@ -156,6 +157,18 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += -lrte_pmd_vhost
endif # $(CONFIG_RTE_LIBRTE_VHOST)
_LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += -lrte_pmd_vmxnet3_uio
+ifeq ($(CONFIG_RTE_LIBRTE_BBDEV),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_NULL) += -lrte_pmd_bbdev_null
+
+# TURBO SOFTWARE PMD is dependent on the FLEXRAN library
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW) += -lrte_pmd_bbdev_turbo_sw
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW) += -L$(FLEXRAN_SDK)/lib_common -lcommon
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW) += -L$(FLEXRAN_SDK)/lib_crc -lcrc
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW) += -L$(FLEXRAN_SDK)/lib_turbo -lturbo
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW) += -L$(FLEXRAN_SDK)/lib_rate_matching -lrate_matching
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW) += -lirc -limf -lstdc++ -lipps
+endif # CONFIG_RTE_LIBRTE_BBDEV
+
ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += -lrte_pmd_aesni_mb
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
--
2.7.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [dpdk-dev] [PATCH v2 2/5] bbdev: PMD drivers (null/turbo_sw)
2017-10-18 2:14 [dpdk-dev] [PATCH v2 1/5] bbdev: librte_bbdev library Amr Mokhtar
@ 2017-10-18 2:14 ` Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 3/5] bbdev: test applications Amr Mokhtar
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Amr Mokhtar @ 2017-10-18 2:14 UTC (permalink / raw)
To: dev
Cc: thomas, anatoly.burakov, pablo.de.lara.guarch, niall.power,
chris.macnamara, Amr Mokhtar
- bbdev 'null' PMD enabled by default
- bbdev 'turbo_sw' PMD disabled by default
- 'turbo_sw' requires the external FLEXRAN SDK libraries
Signed-off-by: Amr Mokhtar <amr.mokhtar@intel.com>
---
drivers/Makefile | 2 +
drivers/bbdev/Makefile | 41 +
drivers/bbdev/null/Makefile | 49 +
drivers/bbdev/null/bbdev_null.c | 377 ++++++
drivers/bbdev/null/rte_pmd_bbdev_null_version.map | 3 +
drivers/bbdev/turbo_sw/Makefile | 59 +
drivers/bbdev/turbo_sw/bbdev_turbo_software.c | 1235 ++++++++++++++++++
.../bbdev/turbo_sw/bbdev_turbo_software_tables.h | 1344 ++++++++++++++++++++
.../turbo_sw/rte_pmd_bbdev_turbo_sw_version.map | 3 +
9 files changed, 3113 insertions(+)
create mode 100644 drivers/bbdev/Makefile
create mode 100644 drivers/bbdev/null/Makefile
create mode 100644 drivers/bbdev/null/bbdev_null.c
create mode 100644 drivers/bbdev/null/rte_pmd_bbdev_null_version.map
create mode 100644 drivers/bbdev/turbo_sw/Makefile
create mode 100644 drivers/bbdev/turbo_sw/bbdev_turbo_software.c
create mode 100644 drivers/bbdev/turbo_sw/bbdev_turbo_software_tables.h
create mode 100644 drivers/bbdev/turbo_sw/rte_pmd_bbdev_turbo_sw_version.map
diff --git a/drivers/Makefile b/drivers/Makefile
index 3a5b223..16f5d78 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -40,5 +40,7 @@ DIRS-y += net
DEPDIRS-net := bus mempool
DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
DEPDIRS-crypto := bus mempool
+DIRS-$(CONFIG_RTE_LIBRTE_BBDEV) += bbdev
+DEPDIRS-bbdev := mempool
include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/bbdev/Makefile b/drivers/bbdev/Makefile
new file mode 100644
index 0000000..c70cd06
--- /dev/null
+++ b/drivers/bbdev/Makefile
@@ -0,0 +1,41 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+core-libs := librte_eal librte_mbuf librte_mempool librte_ring
+core-libs += librte_bbdev librte_kvargs librte_cfgfile
+
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_NULL) += null
+DEPDIRS-null = $(core-libs)
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW) += turbo_sw
+DEPDIRS-turbo_sw = $(core-libs)
+
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/bbdev/null/Makefile b/drivers/bbdev/null/Makefile
new file mode 100644
index 0000000..7719eb6
--- /dev/null
+++ b/drivers/bbdev/null/Makefile
@@ -0,0 +1,49 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+# library name
+LIB = librte_pmd_bbdev_null.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+# versioning export map
+EXPORT_MAP := rte_pmd_bbdev_null_version.map
+
+# library version
+LIBABIVER := 1
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_NULL) += bbdev_null.c
+
+include $(RTE_SDK)/mk/rte.lib.mk
+
diff --git a/drivers/bbdev/null/bbdev_null.c b/drivers/bbdev/null/bbdev_null.c
new file mode 100644
index 0000000..e63904a
--- /dev/null
+++ b/drivers/bbdev/null/bbdev_null.c
@@ -0,0 +1,377 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <string.h>
+
+#include <rte_common.h>
+#include <rte_vdev.h>
+#include <rte_malloc.h>
+#include <rte_ring.h>
+#include <rte_bbdev_vdev.h>
+#include <rte_kvargs.h>
+
+#define DRIVER_NAME bbdev_null
+
+/* Initialisation params structure that can be used by null BBDEV driver */
+struct bbdev_null_params {
+ int socket_id; /*< Null BBDEV socket */
+ uint16_t queues_num; /*< Null BBDEV queues number */
+};
+
+/* Accecptable params for null BBDEV devices */
+#define BBDEV_NULL_MAX_NB_QUEUES_ARG "max_nb_queues"
+#define BBDEV_NULL_SOCKET_ID_ARG "socket_id"
+
+static const char * const bbdev_null_valid_params[] = {
+ BBDEV_NULL_MAX_NB_QUEUES_ARG,
+ BBDEV_NULL_SOCKET_ID_ARG
+};
+
+/* private data structure */
+struct bbdev_private {
+ unsigned int max_nb_queues; /**< Max number of queues */
+};
+
+/* queue */
+struct bbdev_queue {
+ struct rte_ring *processed_pkts; /* Ring for processed packets */
+} __rte_cache_aligned;
+
+/* Get device info */
+static void
+info_get(struct rte_bbdev *dev, struct rte_bbdev_driver_info *dev_info)
+{
+ struct bbdev_private *internals = dev->data->dev_private;
+
+ static const struct rte_bbdev_op_cap bbdev_capabilities[] = {
+ RTE_BBDEV_END_OF_CAPABILITIES_LIST(),
+ };
+
+ static struct rte_bbdev_queue_conf default_queue_conf = {
+ .queue_size = RTE_BBDEV_QUEUE_SIZE_LIMIT,
+ };
+
+ default_queue_conf.socket = dev->data->socket_id;
+
+ dev_info->driver_name = RTE_STR(DRIVER_NAME);
+ dev_info->max_num_queues = internals->max_nb_queues;
+ dev_info->queue_size_lim = RTE_BBDEV_QUEUE_SIZE_LIMIT;
+ dev_info->hardware_accelerated = false;
+ dev_info->max_queue_priority = 0;
+ dev_info->default_queue_conf = default_queue_conf;
+ dev_info->capabilities = bbdev_capabilities;
+ dev_info->cpu_flag_reqs = NULL;
+ dev_info->min_alignment = 0;
+
+ rte_bbdev_log_debug("got device info from %u", dev->data->dev_id);
+}
+
+/* Release queue */
+static int
+q_release(struct rte_bbdev *dev, uint16_t q_id)
+{
+ struct bbdev_queue *q = dev->data->queues[q_id].queue_private;
+
+ if (q != NULL) {
+ rte_ring_free(q->processed_pkts);
+ rte_free(q);
+ dev->data->queues[q_id].queue_private = NULL;
+ }
+
+ rte_bbdev_log_debug("released device queue %u:%u",
+ dev->data->dev_id, q_id);
+ return 0;
+}
+
+/* Setup a queue */
+static int
+q_setup(struct rte_bbdev *dev, uint16_t q_id,
+ const struct rte_bbdev_queue_conf *queue_conf)
+{
+ struct bbdev_queue *q;
+ char ring_name[RTE_RING_NAMESIZE];
+ snprintf(ring_name, RTE_RING_NAMESIZE, RTE_STR(DRIVER_NAME) "%u:%u",
+ dev->data->dev_id, q_id);
+
+ /* Allocate the queue data structure. */
+ q = rte_zmalloc_socket(RTE_STR(DRIVER_NAME), sizeof(*q),
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q == NULL) {
+ rte_bbdev_log(ERR, "Failed to allocate queue memory");
+ return -ENOMEM;
+ }
+
+ q->processed_pkts = rte_ring_create(ring_name, queue_conf->queue_size,
+ queue_conf->socket, RING_F_SP_ENQ | RING_F_SC_DEQ);
+ if (q->processed_pkts == NULL) {
+ rte_bbdev_log(ERR, "Failed to create ring");
+ goto free_q;
+ }
+
+ dev->data->queues[q_id].queue_private = q;
+ rte_bbdev_log_debug("setup device queue %s", ring_name);
+ return 0;
+
+free_q:
+ rte_free(q);
+ return -EFAULT;
+}
+
+static const struct rte_bbdev_ops pmd_ops = {
+ .setup_queues = NULL,
+ .intr_enable = NULL,
+ .start = NULL,
+ .stop = NULL,
+ .close = NULL,
+ .info_get = info_get,
+ .stats_get = NULL,
+ .stats_reset = NULL,
+ .queue_setup = q_setup,
+ .queue_release = q_release,
+ .queue_start = NULL,
+ .queue_stop = NULL,
+ .queue_intr_enable = NULL,
+ .queue_intr_disable = NULL
+};
+
+/* Enqueue decode burst */
+static uint16_t
+enqueue_dec_ops(struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_dec_op **ops, uint16_t nb_ops)
+{
+ struct bbdev_queue *q = q_data->queue_private;
+ uint16_t nb_enqueued = rte_ring_enqueue_burst(q->processed_pkts,
+ (void **)ops, nb_ops, NULL);
+
+ q_data->queue_stats.enqueue_err_count += nb_ops - nb_enqueued;
+ q_data->queue_stats.enqueued_count += nb_enqueued;
+
+ return nb_enqueued;
+}
+
+/* Enqueue encode burst */
+static uint16_t
+enqueue_enc_ops(struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_enc_op **ops, uint16_t nb_ops)
+{
+ struct bbdev_queue *q = q_data->queue_private;
+ uint16_t nb_enqueued = rte_ring_enqueue_burst(q->processed_pkts,
+ (void **)ops, nb_ops, NULL);
+
+ q_data->queue_stats.enqueue_err_count += nb_ops - nb_enqueued;
+ q_data->queue_stats.enqueued_count += nb_enqueued;
+
+ return nb_enqueued;
+}
+
+/* Dequeue decode burst */
+static uint16_t
+dequeue_dec_ops(struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_dec_op **ops, uint16_t nb_ops)
+{
+ struct bbdev_queue *q = q_data->queue_private;
+ uint16_t nb_dequeued = rte_ring_dequeue_burst(q->processed_pkts,
+ (void **)ops, nb_ops, NULL);
+ q_data->queue_stats.dequeued_count += nb_dequeued;
+
+ return nb_dequeued;
+}
+
+/* Dequeue encode burst */
+static uint16_t
+dequeue_enc_ops(struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_enc_op **ops, uint16_t nb_ops)
+{
+ struct bbdev_queue *q = q_data->queue_private;
+ uint16_t nb_dequeued = rte_ring_dequeue_burst(q->processed_pkts,
+ (void **)ops, nb_ops, NULL);
+ q_data->queue_stats.dequeued_count += nb_dequeued;
+
+ return nb_dequeued;
+}
+
+/* Parse 16bit integer from string argument */
+static inline int
+parse_u16_arg(const char *key, const char *value, void *extra_args)
+{
+ uint16_t *u16 = extra_args;
+ unsigned int long result;
+
+ if ((value == NULL) || (extra_args == NULL))
+ return -EINVAL;
+ errno = 0;
+ result = strtoul(value, NULL, 0);
+ if ((result >= (1 << 16)) || (errno != 0)) {
+ rte_bbdev_log(ERR, "Invalid value %lu for %s", result, key);
+ return -ERANGE;
+ }
+ *u16 = (uint16_t)result;
+ return 0;
+}
+
+/* Parse parameters used to create device */
+static int
+parse_bbdev_null_params(struct bbdev_null_params *params,
+ const char *input_args)
+{
+ struct rte_kvargs *kvlist = NULL;
+ int ret = 0;
+
+ if (params == NULL)
+ return -EINVAL;
+ if (input_args) {
+ kvlist = rte_kvargs_parse(input_args, bbdev_null_valid_params);
+ if (kvlist == NULL)
+ return -EFAULT;
+
+ ret = rte_kvargs_process(kvlist, bbdev_null_valid_params[0],
+ &parse_u16_arg, ¶ms->queues_num);
+ if (ret < 0)
+ goto exit;
+
+ ret = rte_kvargs_process(kvlist, bbdev_null_valid_params[1],
+ &parse_u16_arg, ¶ms->socket_id);
+ if (ret < 0)
+ goto exit;
+
+ if (params->socket_id >= RTE_MAX_NUMA_NODES) {
+ rte_bbdev_log(ERR, "Invalid socket, must be < %u",
+ RTE_MAX_NUMA_NODES);
+ goto exit;
+ }
+ }
+
+exit:
+ if (kvlist)
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+/* Create device */
+static int
+null_bbdev_create(struct rte_vdev_device *vdev,
+ struct bbdev_null_params *init_params)
+{
+ struct rte_bbdev *bbdev;
+ size_t dev_private_size = sizeof(struct bbdev_private);
+ struct bbdev_private *internals;
+
+ vdev->device.numa_node = init_params->socket_id;
+
+ bbdev = rte_bbdev_vdev_allocate(vdev, dev_private_size);
+ if (bbdev == NULL) {
+ rte_bbdev_log(ERR, "Failed to create %s",
+ rte_vdev_device_name(vdev));
+ return -EFAULT;
+ }
+
+ bbdev->dev_ops = &pmd_ops;
+
+ /* register rx/tx burst functions for data path */
+ bbdev->dequeue_enc_ops = dequeue_enc_ops;
+ bbdev->dequeue_dec_ops = dequeue_dec_ops;
+ bbdev->enqueue_enc_ops = enqueue_enc_ops;
+ bbdev->enqueue_dec_ops = enqueue_dec_ops;
+ internals = bbdev->data->dev_private;
+ internals->max_nb_queues = init_params->queues_num;
+
+ return 0;
+}
+
+/* Initialise device */
+static int
+null_bbdev_probe(struct rte_vdev_device *vdev)
+{
+ struct bbdev_null_params init_params = {
+ rte_socket_id(),
+ RTE_BBDEV_DEFAULT_MAX_NB_QUEUES
+ };
+ const char *name;
+ const char *input_args;
+
+ if (!vdev)
+ return -EINVAL;
+
+ name = rte_vdev_device_name(vdev);
+ if (name == NULL)
+ return -EINVAL;
+ input_args = rte_vdev_device_args(vdev);
+ parse_bbdev_null_params(&init_params, input_args);
+
+ rte_bbdev_log_debug("Init %s on NUMA node %d with max queues: %d",
+ name, init_params.socket_id, init_params.queues_num);
+
+ return null_bbdev_create(vdev, &init_params);
+}
+
+/* Uninitialise device */
+static int
+null_bbdev_remove(struct rte_vdev_device *vdev)
+{
+ struct rte_bbdev *bbdev;
+ const char *name;
+
+ if (!vdev)
+ return -EINVAL;
+
+ name = rte_vdev_device_name(vdev);
+ if (name == NULL)
+ return -EINVAL;
+
+ bbdev = rte_bbdev_get_named_dev(name);
+ if (bbdev == NULL)
+ return -EINVAL;
+
+ rte_free(bbdev->data->dev_private);
+
+ return rte_bbdev_release(bbdev);
+}
+
+static struct rte_vdev_driver bbdev_null_pmd_drv = {
+ .probe = null_bbdev_probe,
+ .remove = null_bbdev_remove
+};
+
+RTE_PMD_REGISTER_VDEV(DRIVER_NAME, bbdev_null_pmd_drv);
+RTE_PMD_REGISTER_ALIAS(DRIVER_NAME, bbdev_null_pmd);
+RTE_PMD_REGISTER_PARAM_STRING(DRIVER_NAME,
+ BBDEV_NULL_MAX_NB_QUEUES_ARG"=<int> "
+ BBDEV_NULL_SOCKET_ID_ARG"=<int>");
+
+int bbdev_logtype;
+RTE_INIT(null_bbdev_init_log);
+static void
+null_bbdev_init_log(void)
+{
+ bbdev_logtype = rte_log_register("pmd.null_bbdev");
+ if (bbdev_logtype >= 0)
+ rte_log_set_level(bbdev_logtype, RTE_LOG_NOTICE);
+}
diff --git a/drivers/bbdev/null/rte_pmd_bbdev_null_version.map b/drivers/bbdev/null/rte_pmd_bbdev_null_version.map
new file mode 100644
index 0000000..a753031
--- /dev/null
+++ b/drivers/bbdev/null/rte_pmd_bbdev_null_version.map
@@ -0,0 +1,3 @@
+DPDK_17.11 {
+ local: *;
+};
diff --git a/drivers/bbdev/turbo_sw/Makefile b/drivers/bbdev/turbo_sw/Makefile
new file mode 100644
index 0000000..35cfac0
--- /dev/null
+++ b/drivers/bbdev/turbo_sw/Makefile
@@ -0,0 +1,59 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifeq ($(FLEXRAN_SDK),)
+$(error "Please define FLEXRAN_SDK environment variable")
+endif
+
+# library name
+LIB = librte_pmd_bbdev_turbo_sw.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+# versioning export map
+EXPORT_MAP := rte_pmd_bbdev_turbo_sw_version.map
+
+# external library dependencies
+CFLAGS += -I$(FLEXRAN_SDK)/lib_common
+CFLAGS += -I$(FLEXRAN_SDK)/lib_turbo
+CFLAGS += -I$(FLEXRAN_SDK)/lib_crc
+CFLAGS += -I$(FLEXRAN_SDK)/lib_rate_matching
+
+# library version
+LIBABIVER := 1
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW) += bbdev_turbo_software.c
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/bbdev/turbo_sw/bbdev_turbo_software.c b/drivers/bbdev/turbo_sw/bbdev_turbo_software.c
new file mode 100644
index 0000000..477318b
--- /dev/null
+++ b/drivers/bbdev/turbo_sw/bbdev_turbo_software.c
@@ -0,0 +1,1235 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <string.h>
+
+#include <rte_common.h>
+#include <rte_vdev.h>
+#include <rte_malloc.h>
+#include <rte_ring.h>
+#include <rte_bbdev_vdev.h>
+#include <rte_kvargs.h>
+
+#include <phy_turbo.h>
+#include <phy_crc.h>
+#include <phy_rate_match.h>
+#include <divide.h>
+
+#include "bbdev_turbo_software_tables.h"
+
+#define DRIVER_NAME turbo_sw
+
+#define MAX_CB_SIZE (6144)
+#define MAX_NCB ((MAX_CB_SIZE + 4) * 3)
+
+/* private data structure */
+struct bbdev_private {
+ unsigned int max_nb_queues; /**< Max number of queues */
+};
+
+/* Initialisation params structure that can be used by Turbo SW driver */
+struct turbo_sw_params {
+ int socket_id; /*< Turbo SW device socket */
+ uint16_t queues_num; /*< Turbo SW device queues number */
+};
+
+/* Accecptable params for Turbo SW devices */
+#define TURBO_SW_MAX_NB_QUEUES_ARG "max_nb_queues"
+#define TURBO_SW_SOCKET_ID_ARG "socket_id"
+
+static const char * const turbo_sw_valid_params[] = {
+ TURBO_SW_MAX_NB_QUEUES_ARG,
+ TURBO_SW_SOCKET_ID_ARG
+};
+
+/* queue */
+struct turbo_sw_queue {
+ /* Ring for processed (encoded/decoded) operations which are ready to
+ * be dequeued.
+ */
+ struct rte_ring *processed_pkts;
+ /* Stores output from turbo encoder */
+ uint8_t *enc_out;
+ /* Stores output after rate-matching */
+ uint8_t *rm_out;
+ /* Alpha gamma buf for bblib_turbo_decoder() function */
+ int8_t *ag;
+ /* Temp buf for bblib_turbo_decoder() function */
+ uint16_t *code_block;
+ /* Input buf for bblib_harqcombine_lte() function */
+ uint8_t *harq_input;
+ /* Output buf for bblib_harqcombine_lte() function */
+ uint8_t *harq_output;
+ /* Output buf for bblib_rate_dematching_lte() function */
+ uint8_t *deint_output;
+ /* Output buf for bblib_turbodec_adapter_lte() function */
+ uint8_t *adapter_output;
+ /* Operation type of this queue */
+ enum rte_bbdev_op_type type;
+} __rte_cache_aligned;
+
+/* Calculate index based on Table 5.1.3-3 from TS34.212 */
+static inline int32_t
+compute_idx(uint16_t k)
+{
+ int32_t result = 0;
+
+ if (k < 40 || k > MAX_CB_SIZE)
+ return -1;
+
+ if (k > 2048) {
+ if ((k - 2048) % 64 != 0)
+ result = -1;
+
+ result = 124 + (k - 2048) / 64;
+ } else if (k <= 512) {
+ if ((k - 40) % 8 != 0)
+ result = -1;
+
+ result = (k - 40) / 8 + 1;
+ } else if (k <= 1024) {
+ if ((k - 512) % 16 != 0)
+ result = -1;
+
+ result = 60 + (k - 512) / 16;
+ } else { /* 1024 < k <= 2048 */
+ if ((k - 1024) % 32 != 0)
+ result = -1;
+
+ result = 92 + (k - 1024) / 32;
+ }
+
+ return result;
+}
+
+/* Read flag value 0/1 from bitmap */
+static inline bool
+check_bit(uint32_t bitmap, uint32_t bitmask)
+{
+ return bitmap & bitmask;
+}
+
+/* Get device info */
+static void
+info_get(struct rte_bbdev *dev, struct rte_bbdev_driver_info *dev_info)
+{
+ struct bbdev_private *internals = dev->data->dev_private;
+
+ static const struct rte_bbdev_op_cap bbdev_capabilities[] = {
+ {
+ .type = RTE_BBDEV_OP_TURBO_DEC,
+ .cap.turbo_dec = {
+ .capability_flags =
+ RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE |
+ RTE_BBDEV_TURBO_RAW_INPUT_DATA,
+ .num_buffers_src = RTE_BBDEV_MAX_CODE_BLOCKS,
+ .num_buffers_hard_out =
+ RTE_BBDEV_MAX_CODE_BLOCKS,
+ .num_buffers_soft_out = 0,
+ }
+ },
+ {
+ .type = RTE_BBDEV_OP_TURBO_ENC,
+ .cap.turbo_enc = {
+ .capability_flags =
+ RTE_BBDEV_TURBO_CRC_24B_ATTACH |
+ RTE_BBDEV_TURBO_RATE_MATCH |
+ RTE_BBDEV_TURBO_RV_INDEX_BYPASS,
+ .num_buffers_src = RTE_BBDEV_MAX_CODE_BLOCKS,
+ .num_buffers_dst = RTE_BBDEV_MAX_CODE_BLOCKS,
+ }
+ },
+ RTE_BBDEV_END_OF_CAPABILITIES_LIST()
+ };
+
+ static struct rte_bbdev_queue_conf default_queue_conf = {
+ .queue_size = RTE_BBDEV_QUEUE_SIZE_LIMIT,
+ };
+
+ static const enum rte_cpu_flag_t cpu_flag = RTE_CPUFLAG_SSE4_2;
+
+ default_queue_conf.socket = dev->data->socket_id;
+
+ dev_info->driver_name = RTE_STR(DRIVER_NAME);
+ dev_info->max_num_queues = internals->max_nb_queues;
+ dev_info->queue_size_lim = RTE_BBDEV_QUEUE_SIZE_LIMIT;
+ dev_info->hardware_accelerated = false;
+ dev_info->max_queue_priority = 0;
+ dev_info->default_queue_conf = default_queue_conf;
+ dev_info->capabilities = bbdev_capabilities;
+ dev_info->cpu_flag_reqs = &cpu_flag;
+ dev_info->min_alignment = 64;
+
+ rte_bbdev_log_debug("got device info from %u\n", dev->data->dev_id);
+}
+
+/* Release queue */
+static int
+q_release(struct rte_bbdev *dev, uint16_t q_id)
+{
+ struct turbo_sw_queue *q = dev->data->queues[q_id].queue_private;
+
+ if (q != NULL) {
+ rte_ring_free(q->processed_pkts);
+ rte_free(q->enc_out);
+ rte_free(q->rm_out);
+ rte_free(q->ag);
+ rte_free(q->code_block);
+ rte_free(q->harq_input);
+ rte_free(q->harq_output);
+ rte_free(q->deint_output);
+ rte_free(q->adapter_output);
+ rte_free(q);
+ dev->data->queues[q_id].queue_private = NULL;
+ }
+
+ rte_bbdev_log_debug("released device queue %u:%u",
+ dev->data->dev_id, q_id);
+ return 0;
+}
+
+/* Setup a queue */
+static int
+q_setup(struct rte_bbdev *dev, uint16_t q_id,
+ const struct rte_bbdev_queue_conf *queue_conf)
+{
+ int ret;
+ struct turbo_sw_queue *q;
+ char name[RTE_RING_NAMESIZE];
+
+ /* Allocate the queue data structure. */
+ q = rte_zmalloc_socket(RTE_STR(DRIVER_NAME), sizeof(*q),
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q == NULL) {
+ rte_bbdev_log(ERR, "Failed to allocate queue memory");
+ return -ENOMEM;
+ }
+
+ /* Allocate memory for encoder output. */
+ ret = snprintf(name, RTE_RING_NAMESIZE, RTE_STR(DRIVER_NAME)"_enc_out%u:%u",
+ dev->data->dev_id, q_id);
+ if ((ret < 0) || (ret >= (int)RTE_RING_NAMESIZE)) {
+ rte_bbdev_log(ERR,
+ "Creating queue name for device %u queue %u failed",
+ dev->data->dev_id, q_id);
+ return -ENAMETOOLONG;
+ }
+ q->enc_out = rte_zmalloc_socket(name,
+ ((MAX_CB_SIZE >> 3) + 3) * sizeof(*q->enc_out) * 3,
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q->enc_out == NULL) {
+ rte_bbdev_log(ERR,
+ "Failed to allocate queue memory for %s", name);
+ goto free_q;
+ }
+
+ /* Allocate memory for rate matching output. */
+ ret = snprintf(name, RTE_RING_NAMESIZE, RTE_STR(DRIVER_NAME)"_rm_out%u:%u",
+ dev->data->dev_id, q_id);
+ if ((ret < 0) || (ret >= (int)RTE_RING_NAMESIZE)) {
+ rte_bbdev_log(ERR,
+ "Creating queue name for device %u queue %u failed",
+ dev->data->dev_id, q_id);
+ return -ENAMETOOLONG;
+ }
+ q->rm_out = rte_zmalloc_socket(name,
+ ((MAX_CB_SIZE >> 3) + 3) * sizeof(*q->rm_out) * 3,
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q->rm_out == NULL) {
+ rte_bbdev_log(ERR,
+ "Failed to allocate queue memory for %s", name);
+ goto free_q;
+ }
+
+ /* Allocate memory for Aplha Gamma temp buffer. */
+ ret = snprintf(name, RTE_RING_NAMESIZE, RTE_STR(DRIVER_NAME)"_ag%u:%u",
+ dev->data->dev_id, q_id);
+ if ((ret < 0) || (ret >= (int)RTE_RING_NAMESIZE)) {
+ rte_bbdev_log(ERR,
+ "Creating queue name for device %u queue %u failed",
+ dev->data->dev_id, q_id);
+ return -ENAMETOOLONG;
+ }
+ q->ag = rte_zmalloc_socket(name,
+ MAX_CB_SIZE * 10 * sizeof(*q->ag),
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q->ag == NULL) {
+ rte_bbdev_log(ERR,
+ "Failed to allocate queue memory for %s", name);
+ goto free_q;
+ }
+
+ /* Allocate memory for code block temp buffer. */
+ ret = snprintf(name, RTE_RING_NAMESIZE, RTE_STR(DRIVER_NAME)"_cb%u:%u",
+ dev->data->dev_id, q_id);
+ if ((ret < 0) || (ret >= (int)RTE_RING_NAMESIZE)) {
+ rte_bbdev_log(ERR,
+ "Creating queue name for device %u queue %u failed",
+ dev->data->dev_id, q_id);
+ return -ENAMETOOLONG;
+ }
+ q->code_block = rte_zmalloc_socket(name,
+ (6144 >> 3) * sizeof(*q->code_block),
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q->code_block == NULL) {
+ rte_bbdev_log(ERR,
+ "Failed to allocate queue memory for %s", name);
+ goto free_q;
+ }
+
+ /* Allocate memory for HARQ input. */
+ ret = snprintf(name, RTE_RING_NAMESIZE,
+ RTE_STR(DRIVER_NAME)"_harq_input%u:%u",
+ dev->data->dev_id, q_id);
+ if ((ret < 0) || (ret >= (int)RTE_RING_NAMESIZE)) {
+ rte_bbdev_log(ERR,
+ "Creating queue name for device %u queue %u failed",
+ dev->data->dev_id, q_id);
+ return -ENAMETOOLONG;
+ }
+ q->harq_input = rte_zmalloc_socket(name,
+ MAX_NCB * sizeof(*q->harq_input),
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q->harq_input == NULL) {
+ rte_bbdev_log(ERR,
+ "Failed to allocate queue memory for %s", name);
+ goto free_q;
+ }
+
+ /* Allocate memory for HARQ output. */
+ ret = snprintf(name, RTE_RING_NAMESIZE,
+ RTE_STR(DRIVER_NAME)"_harq_output%u:%u",
+ dev->data->dev_id, q_id);
+ if ((ret < 0) || (ret >= (int)RTE_RING_NAMESIZE)) {
+ rte_bbdev_log(ERR,
+ "Creating queue name for device %u queue %u failed",
+ dev->data->dev_id, q_id);
+ return -ENAMETOOLONG;
+ }
+ q->harq_output = rte_zmalloc_socket(name,
+ MAX_NCB * sizeof(*q->harq_output),
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q->harq_output == NULL) {
+ rte_bbdev_log(ERR,
+ "Failed to allocate queue memory for %s", name);
+ goto free_q;
+ }
+
+ /* Allocate memory for Deinterleaver output. */
+ ret = snprintf(name, RTE_RING_NAMESIZE,
+ RTE_STR(DRIVER_NAME)"_deint_output%u:%u",
+ dev->data->dev_id, q_id);
+ if ((ret < 0) || (ret >= (int)RTE_RING_NAMESIZE)) {
+ rte_bbdev_log(ERR,
+ "Creating queue name for device %u queue %u failed",
+ dev->data->dev_id, q_id);
+ return -ENAMETOOLONG;
+ }
+ q->deint_output = rte_zmalloc_socket(NULL,
+ MAX_NCB * sizeof(*q->deint_output),
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q->deint_output == NULL) {
+ rte_bbdev_log(ERR,
+ "Failed to allocate queue memory for %s", name);
+ goto free_q;
+ }
+
+ /* Allocate memory for Adapter output. */
+ ret = snprintf(name, RTE_RING_NAMESIZE,
+ RTE_STR(DRIVER_NAME)"_adapter_output%u:%u",
+ dev->data->dev_id, q_id);
+ if ((ret < 0) || (ret >= (int)RTE_RING_NAMESIZE)) {
+ rte_bbdev_log(ERR,
+ "Creating queue name for device %u queue %u failed",
+ dev->data->dev_id, q_id);
+ return -ENAMETOOLONG;
+ }
+ q->adapter_output = rte_zmalloc_socket(NULL,
+ MAX_CB_SIZE * 6 * sizeof(*q->adapter_output),
+ RTE_CACHE_LINE_SIZE, queue_conf->socket);
+ if (q->adapter_output == NULL) {
+ rte_bbdev_log(ERR,
+ "Failed to allocate queue memory for %s", name);
+ goto free_q;
+ }
+
+ /* Create ring for packets awaiting to be dequeued. */
+ ret = snprintf(name, RTE_RING_NAMESIZE, RTE_STR(DRIVER_NAME)"%u:%u",
+ dev->data->dev_id, q_id);
+ if ((ret < 0) || (ret >= (int)RTE_RING_NAMESIZE)) {
+ rte_bbdev_log(ERR,
+ "Creating queue name for device %u queue %u failed",
+ dev->data->dev_id, q_id);
+ return -ENAMETOOLONG;
+ }
+ q->processed_pkts = rte_ring_create(name, queue_conf->queue_size,
+ queue_conf->socket, RING_F_SP_ENQ | RING_F_SC_DEQ);
+ if (q->processed_pkts == NULL) {
+ rte_bbdev_log(ERR, "Failed to create ring for %s", name);
+ goto free_q;
+ }
+
+ q->type = queue_conf->op_type;
+
+ dev->data->queues[q_id].queue_private = q;
+ rte_bbdev_log_debug("setup device queue %s", name);
+ return 0;
+
+free_q:
+ rte_ring_free(q->processed_pkts);
+ rte_free(q->enc_out);
+ rte_free(q->rm_out);
+ rte_free(q->ag);
+ rte_free(q->code_block);
+ rte_free(q->harq_input);
+ rte_free(q->harq_output);
+ rte_free(q->deint_output);
+ rte_free(q->adapter_output);
+ rte_free(q);
+ return -EFAULT;
+}
+
+static const struct rte_bbdev_ops pmd_ops = {
+ .setup_queues = NULL,
+ .intr_enable = NULL,
+ .start = NULL,
+ .stop = NULL,
+ .close = NULL,
+ .info_get = info_get,
+ .stats_get = NULL,
+ .stats_reset = NULL,
+ .queue_setup = q_setup,
+ .queue_release = q_release,
+ .queue_start = NULL,
+ .queue_stop = NULL,
+ .queue_intr_enable = NULL,
+ .queue_intr_disable = NULL
+};
+
+/* Checks if the encoder input buffer is correct.
+ * Returns 0 if it's valid, -1 otherwise.
+ */
+static inline int
+is_enc_input_valid(uint16_t k, int32_t k_idx, uint16_t in_length)
+{
+ if (k_idx < 0) {
+ rte_bbdev_log(ERR, "K Index is invalid");
+ return -1;
+ }
+
+ if (in_length != (k >> 3)) {
+ rte_bbdev_log(ERR,
+ "Mismatch between input length (%u bytes) and K (%u bits)",
+ in_length, k);
+ return -1;
+ }
+
+ if (in_length > (MAX_CB_SIZE >> 3)) {
+ rte_bbdev_log(ERR, "Input length (%u) is too big, max: %d",
+ in_length, (MAX_CB_SIZE >> 3));
+ return -1;
+ }
+
+ return 0;
+}
+
+/* Checks if the decoder input buffer is correct.
+ * Returns 0 if it's valid, -1 otherwise.
+ */
+static inline int
+is_dec_input_valid(int32_t k_idx, uint16_t in_length)
+{
+ if (k_idx < 0) {
+ rte_bbdev_log(ERR, "K index is invalid");
+ return -1;
+ }
+
+ if (in_length > MAX_NCB) {
+ rte_bbdev_log(ERR, "Input length (%u) is too big, max: %d",
+ in_length, MAX_NCB);
+ return -1;
+ }
+
+ return 0;
+}
+
+static inline void
+process_enc_cb(struct turbo_sw_queue *q, struct rte_bbdev_enc_op *op,
+ uint8_t cb_idx, uint8_t cb_total, uint16_t k, uint32_t e,
+ struct rte_mbuf *m_in, struct rte_mbuf *m_out,
+ uint16_t in_offset, uint16_t out_offset, uint16_t *total_left)
+{
+ int ret;
+ int16_t k_idx, in_length;
+ uint16_t m;
+ uint8_t *in, *out0, *out1, *out2, *tmp_out;
+ struct rte_bbdev_op_turbo_enc enc = op->turbo_enc;
+ struct bblib_crc_request crc_req;
+ struct bblib_turbo_encoder_request turbo_req;
+ struct bblib_turbo_encoder_response turbo_resp;
+ struct bblib_rate_match_dl_request rm_req;
+ struct bblib_rate_match_dl_response rm_resp;
+
+ k_idx = compute_idx(k);
+
+ in_length = (*total_left >= (m_in->data_len - in_offset)) ?
+ m_in->data_len - in_offset : *total_left;
+ *total_left -= in_length;
+
+ ret = is_enc_input_valid(k, k_idx, in_length);
+ if (ret != 0) {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ return;
+ }
+
+ in = rte_pktmbuf_mtod_offset(m_in, uint8_t *, in_offset);
+
+ /* CRC24B */
+ if (enc.op_flags & RTE_BBDEV_TURBO_CRC_24B_ATTACH) {
+ /* len is shortened by 3 bytes as CRC is attached to
+ * last 3 bytes.
+ */
+ crc_req.data = in;
+ crc_req.len = in_length - 3;
+ if (bblib_lte_crc24b_gen(&crc_req) == -1) {
+ op->status |= 1 << RTE_BBDEV_CRC_ERROR;
+ rte_bbdev_log(ERR, "CRC24b generation failed");
+ return;
+ }
+ }
+
+ /* Turbo encoder */
+
+ /* Each bit layer output from turbo encoder is (k+4) bits long, i.e.
+ * input length + 4 tail bits. That's (k/8) + 1 bytes after rounding up.
+ * So dst_data's length should be 3*(k/8) + 3 bytes.
+ */
+ out0 = q->enc_out;
+ out1 = RTE_PTR_ADD(out0, in_length + 1);
+ out2 = RTE_PTR_ADD(out1, in_length + 1);
+
+ turbo_req.case_id = k_idx;
+ turbo_req.input_win = in;
+ turbo_req.length = in_length;
+ turbo_resp.output_win_0 = out0;
+ turbo_resp.output_win_1 = out1;
+ turbo_resp.output_win_2 = out2;
+ if (bblib_turbo_encoder(&turbo_req, &turbo_resp) != 0) {
+ op->status |= 1 << RTE_BBDEV_DRV_ERROR;
+ rte_bbdev_log(ERR, "Turbo Encoder failed");
+ return;
+ }
+
+ /* Rate-matching */
+ if (enc.op_flags & RTE_BBDEV_TURBO_RATE_MATCH) {
+ /* index of current code block */
+ rm_req.r = cb_idx;
+ /* total number of code block */
+ rm_req.C = cb_total;
+ /* For DL - 1, UL - 0 */
+ rm_req.direction = 1;
+ rm_req.Nsoft = enc.n_soft;
+ rm_req.KMIMO = enc.k_mimo;
+ rm_req.MDL_HARQ = enc.mdl_harq;
+ rm_req.G = enc.g;
+ rm_req.NL = enc.nl;
+ rm_req.Qm = enc.qm;
+ rm_req.rvidx = enc.rv_index;
+ rm_req.Kidx = k_idx - 1;
+ rm_req.nLen = k + 4;
+ rm_req.tin0 = out0;
+ rm_req.tin1 = out1;
+ rm_req.tin2 = out2;
+ rm_resp.output = q->rm_out;
+ rm_resp.OutputLen = (e >> 3);
+ if (enc.op_flags & RTE_BBDEV_TURBO_RV_INDEX_BYPASS)
+ rm_req.bypass_rvidx = 1;
+ else
+ rm_req.bypass_rvidx = 0;
+
+ if (bblib_rate_match_dl(&rm_req, &rm_resp) != 0) {
+ op->status |= 1 << RTE_BBDEV_DRV_ERROR;
+ rte_bbdev_log(ERR, "Rate matching failed");
+ return;
+ }
+
+ /* copy rate-match output to turbo_enc entity */
+ out0 = (uint8_t *)rte_pktmbuf_append(m_out,
+ rm_resp.OutputLen);
+ if (out0 == NULL) {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ rte_bbdev_log(ERR,
+ "Too little space in output mbuf");
+ return;
+ }
+ enc.output.length += rm_resp.OutputLen;
+ /* rte_bbdev_op_data.offset can be different than the offset
+ * of the appended bytes
+ */
+ out0 = rte_pktmbuf_mtod_offset(m_out, uint8_t *,
+ out_offset);
+ rte_memcpy(out0, q->rm_out, rm_resp.OutputLen);
+ } else {
+ /* Rate matching is bypassed */
+
+ /* Completing last byte of out0 (where 4 tail bits are stored)
+ * by moving first 4 bits from out1
+ */
+ tmp_out = (uint8_t *) --out1;
+ *tmp_out = *tmp_out | ((*(tmp_out + 1) & 0xF0) >> 4);
+ tmp_out++;
+ /* Shifting out1 data by 4 bits to the left */
+ for (m = 0; m < in_length; ++m) {
+ uint8_t *first = tmp_out;
+ uint8_t second = *(tmp_out + 1);
+ *first = (*first << 4) | ((second & 0xF0) >> 4);
+ tmp_out++;
+ }
+ /* Shifting out2 data by 8 bits to the left */
+ for (m = 0; m < in_length + 1; ++m) {
+ *tmp_out = *(tmp_out + 1);
+ tmp_out++;
+ }
+ *tmp_out = 0;
+
+ /* copy shifted output to turbo_enc entity */
+ out0 = (uint8_t *)rte_pktmbuf_append(m_out,
+ in_length * 3 + 2);
+ if (out0 == NULL) {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ rte_bbdev_log(ERR,
+ "Too little space in output mbuf");
+ return;
+ }
+ enc.output.length += in_length * 3 + 2;
+ /* rte_bbdev_op_data.offset can be different than the
+ * offset of the appended bytes
+ */
+ out0 = rte_pktmbuf_mtod_offset(m_out, uint8_t *,
+ out_offset);
+ rte_memcpy(out0, q->enc_out, in_length * 3 + 2);
+ }
+}
+
+static inline void
+enqueue_enc_one_op(struct turbo_sw_queue *q, struct rte_bbdev_enc_op *op)
+{
+ uint8_t cb_idx = 0;
+ uint8_t cb_total, cab, c_neg;
+ uint16_t k_pos, k_neg, k;
+ uint32_t ea, eb, e;
+ struct rte_bbdev_op_turbo_enc enc = op->turbo_enc;
+ uint16_t in_offset = enc.input.offset;
+ uint16_t out_offset = enc.output.offset;
+ struct rte_mbuf *m_in = enc.input.data;
+ struct rte_mbuf *m_out = enc.output.data;
+ struct rte_mbuf *m_out_next;
+ uint16_t total_left = enc.input.length;
+
+ if (enc.code_block_mode == 0) { /* For Transport Block mode */
+ k_pos = enc.tb_params.k_pos;
+ k_neg = enc.tb_params.k_neg;
+ ea = enc.tb_params.ea;
+ eb = enc.tb_params.eb;
+ cb_total = enc.tb_params.c;
+ cab = enc.tb_params.cab;
+ c_neg = enc.tb_params.c_neg;
+ } else { /* For Code Block mode */
+ k_pos = enc.cb_params.k;
+ k_neg = enc.cb_params.k;
+ ea = enc.cb_params.e;
+ eb = enc.cb_params.e;
+ cb_total = 1;
+ cab = 1;
+ c_neg = 1;
+ }
+
+ /* Clear op status */
+ op->status = 0;
+
+ if (m_in == NULL || m_out == NULL) {
+ rte_bbdev_log(ERR, "Invalid mbuf pointer");
+ op->status = 1 << RTE_BBDEV_DATA_ERROR;
+ return;
+ }
+
+ k = (cb_idx < c_neg) ? k_neg : k_pos;
+ e = (cb_idx < cab) ? ea : eb;
+ process_enc_cb(q, op, cb_idx, cb_total, k, e, m_in, m_out, in_offset,
+ out_offset, &total_left);
+
+ for (cb_idx = 1; cb_idx < cb_total; ++cb_idx) {
+ k = (cb_idx < c_neg) ? k_neg : k_pos;
+ e = (cb_idx < cab) ? ea : eb;
+
+ m_in = m_in->next;
+ m_out_next = rte_pktmbuf_alloc(m_out->pool);
+ if (m_in == NULL || m_out_next == NULL) {
+ rte_bbdev_log(ERR, "Invalid mbuf pointer");
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ continue;
+ }
+ rte_pktmbuf_chain(m_out, m_out_next);
+
+ process_enc_cb(q, op, cb_idx, cb_total, k, e, m_in, m_out_next,
+ 0, 0, &total_left);
+ }
+
+ /* check if all input data was processed */
+ if (total_left != 0) {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ rte_bbdev_log(ERR,
+ "Mismatch between rte_bbdev_op_data.length and mbuf data");
+ }
+}
+
+static inline uint16_t
+enqueue_enc_all_ops(struct turbo_sw_queue *q, struct rte_bbdev_enc_op **ops,
+ uint16_t nb_ops)
+{
+ uint16_t i;
+
+ for (i = 0; i < nb_ops; ++i)
+ enqueue_enc_one_op(q, ops[i]);
+
+ return rte_ring_enqueue_burst(q->processed_pkts, (void **)ops, nb_ops,
+ NULL);
+}
+
+static inline int32_t
+calc_k0_without_null(int k0, int k_idx)
+{
+ int32_t null_num;
+ int32_t i;
+
+ k_idx--;
+ null_num = kidx_null_num[k_idx];
+ for (i = 0; i < null_num; ++i)
+ if (k0_null_num[k_idx][i] > k0)
+ break;
+ return k0 - i;
+}
+
+static inline int32_t
+calc_k0(int k, int rv_idx)
+{
+ int16_t temp;
+ int32_t ncb, k0;
+ int32_t r_tc_subblock = ceili(k+4, 32);
+ int32_t kpai = r_tc_subblock * 32;
+ int32_t kw = 3 * kpai;
+
+ ncb = kw;
+ temp = ceili(ncb, 8 * r_tc_subblock);
+ k0 = r_tc_subblock * (2 * temp * rv_idx + 2);
+
+ return k0;
+}
+
+/* Remove null bytes starting from k0, return number of non-null bytes */
+static inline uint16_t
+remove_null_bytes(const uint8_t *in, uint8_t *out, uint16_t k0, uint16_t len)
+{
+ uint16_t in_idx, out_idx;
+
+ for (in_idx = k0, out_idx = 0; in_idx < len; ++in_idx)
+ if (in[in_idx] != 0x00)
+ out[out_idx++] = in[in_idx];
+
+ for (in_idx = 0; in_idx < k0; ++in_idx)
+ if (in[in_idx] != 0x00)
+ out[out_idx++] = in[in_idx];
+
+ return out_idx;
+}
+
+static inline int
+process_raw_buf_dec_op(struct turbo_sw_queue *q,
+ struct rte_bbdev_op_turbo_dec *dec, uint8_t *in,
+ uint16_t in_len, int32_t k_idx, uint16_t k, int8_t **out)
+{
+ struct bblib_turbo_adapter_ul_request adapter_req;
+ struct bblib_turbo_adapter_ul_response adapter_resp;
+ struct bblib_harq_combine_ul_request harq_req;
+ struct bblib_harq_combine_ul_response harq_resp;
+ struct bblib_deinterleave_ul_request deint_req;
+ struct bblib_deinterleave_ul_response deint_resp;
+ int32_t ncb_without_null, k0, k0_without_null;
+
+ ncb_without_null = (k + 4) * 3;
+ k0 = calc_k0(k, dec->rv_index);
+ k0_without_null = calc_k0_without_null(k0, k_idx);
+
+ /* Input contains raw RX data - E bytes. */
+ harq_req.e = in_len;
+ /* Assume that it's never a retransmission. */
+ harq_req.isretx = 0;
+ harq_req.k0withoutnull = k0_without_null;
+ harq_req.ncb = ncb_without_null;
+ harq_req.pdmout = in;
+ harq_resp.pharqbuffer = q->harq_output;
+ bblib_harq_combine_ul(&harq_req, &harq_resp);
+
+ deint_req.pharqbuffer = q->harq_output;
+ deint_req.ncb = ncb_without_null;
+ deint_resp.pinteleavebuffer = q->deint_output;
+ bblib_deinterleave_ul(&deint_req, &deint_resp);
+
+ adapter_req.isinverted = 0;
+ adapter_req.ncb = ncb_without_null;
+ adapter_req.pinteleavebuffer = q->deint_output;
+ adapter_resp.pharqout = q->adapter_output;
+ bblib_turbo_adapter_ul(&adapter_req, &adapter_resp);
+
+ *out = (int8_t *)q->adapter_output;
+ return 0;
+}
+
+static inline int
+process_circular_buf_dec_op(struct turbo_sw_queue *q,
+ struct rte_bbdev_op_turbo_dec *dec, uint8_t *in,
+ uint16_t in_len, int32_t k_idx, int16_t k, int8_t **out)
+{
+ struct bblib_turbo_adapter_ul_request adapter_req;
+ struct bblib_turbo_adapter_ul_response adapter_resp;
+ int32_t ncb, ncb_without_null;
+ uint8_t *adapter_input = in;
+
+ /* Input contains the cyclic buffer. */
+ ncb = ncb_without_null = in_len;
+
+ if (check_bit(dec->op_flags, RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE)) {
+ struct bblib_harq_combine_ul_request harq_req;
+ struct bblib_harq_combine_ul_response harq_resp;
+ struct bblib_deinterleave_ul_request deint_req;
+ struct bblib_deinterleave_ul_response deint_resp;
+ int32_t e, k0, k0_without_null;
+
+ ncb_without_null = (k + 4) * 3;
+ k0 = calc_k0(k, dec->rv_index);
+ k0_without_null = calc_k0_without_null(k0, k_idx);
+
+ e = remove_null_bytes(in, q->harq_input, k0, ncb);
+
+ harq_req.e = e;
+ /* Assume that it's never a retransmission. */
+ harq_req.isretx = 0;
+ harq_req.k0withoutnull = k0_without_null;
+ harq_req.ncb = ncb_without_null;
+ harq_req.pdmout = q->harq_input;
+ harq_resp.pharqbuffer = q->harq_output;
+ bblib_harq_combine_ul(&harq_req, &harq_resp);
+
+ deint_req.pharqbuffer = q->harq_output;
+ deint_req.ncb = ncb_without_null;
+ deint_resp.pinteleavebuffer = q->deint_output;
+ bblib_deinterleave_ul(&deint_req, &deint_resp);
+ adapter_input = q->deint_output;
+ }
+
+ adapter_req.isinverted = 0;
+ adapter_req.ncb = ncb_without_null;
+ adapter_req.pinteleavebuffer = adapter_input;
+ adapter_resp.pharqout = q->adapter_output;
+ bblib_turbo_adapter_ul(&adapter_req, &adapter_resp);
+
+ *out = (int8_t *)q->adapter_output;
+ return 0;
+}
+
+static inline void
+process_dec_cb(struct turbo_sw_queue *q, struct rte_bbdev_dec_op *op,
+ uint8_t cb_idx, uint16_t k, struct rte_mbuf *m_in,
+ struct rte_mbuf *m_out, uint16_t in_offset, uint16_t out_offset,
+ uint16_t *total_left)
+{
+ int ret;
+ int32_t k_idx;
+ uint16_t in_length;
+ int32_t iter_cnt;
+ uint8_t *in, *out;
+ int8_t *dec_input;
+ struct bblib_turbo_decoder_request turbo_req;
+ struct bblib_turbo_decoder_response turbo_resp;
+ struct rte_bbdev_op_turbo_dec dec = op->turbo_dec;
+
+ k_idx = compute_idx(k);
+
+ in_length = (*total_left >= (m_in->data_len - in_offset)) ?
+ m_in->data_len - in_offset : *total_left;
+ *total_left -= in_length;
+
+ ret = is_dec_input_valid(k_idx, in_length);
+ if (ret != 0) {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ return;
+ }
+
+ in = rte_pktmbuf_mtod_offset(m_in, uint8_t *, in_offset);
+
+ if (check_bit(dec.op_flags, RTE_BBDEV_TURBO_RAW_INPUT_DATA)) {
+ ret = process_raw_buf_dec_op(q, &dec, in, in_length,
+ k_idx, k, &dec_input);
+ if (ret != 0) {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ return;
+ }
+ } else {
+ ret = process_circular_buf_dec_op(q, &dec, in, in_length,
+ k_idx, k, &dec_input);
+ if (ret != 0) {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ return;
+ }
+ }
+
+ out = (uint8_t *)rte_pktmbuf_append(m_out, (k >> 3));
+ if (out == NULL) {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ rte_bbdev_log(ERR, "Too little space in output mbuf");
+ return;
+ }
+ /* rte_bbdev_op_data.offset can be different than the offset of the
+ * appended bytes
+ */
+ out = rte_pktmbuf_mtod_offset(m_out, uint8_t *, out_offset);
+ turbo_req.c = cb_idx + 1;
+ turbo_req.input = dec_input;
+ turbo_req.k = k;
+ turbo_req.k_idx = k_idx;
+ turbo_req.max_iter_num = dec.iter_max;
+ turbo_resp.ag_buf = q->ag;
+ turbo_resp.cb_buf = q->code_block;
+ turbo_resp.output = out;
+ iter_cnt = bblib_turbo_decoder(&turbo_req, &turbo_resp);
+
+ dec.hard_output.length += (k >> 3);
+
+ if (iter_cnt > 0)
+ dec.iter_count = RTE_MAX(iter_cnt, dec.iter_count);
+ else {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ rte_bbdev_log(ERR, "Turbo Decoder failed");
+ return;
+ }
+}
+
+static inline void
+enqueue_dec_one_op(struct turbo_sw_queue *q, struct rte_bbdev_dec_op *op)
+{
+ uint8_t cb_idx = 0;
+ uint8_t cb_total, c_neg;
+ uint16_t k_pos, k_neg, k;
+ struct rte_bbdev_op_turbo_dec dec = op->turbo_dec;
+ struct rte_mbuf *m_in = dec.input.data;
+ struct rte_mbuf *m_out = dec.hard_output.data;
+ struct rte_mbuf *m_out_next;
+ uint16_t total_left = dec.input.length;
+ uint16_t in_offset = dec.input.offset;
+ uint16_t out_offset = dec.hard_output.offset;
+
+ if (dec.code_block_mode == 0) { /* For Transport Block mode */
+ k_pos = dec.tb_params.k_pos;
+ k_neg = dec.tb_params.k_neg;
+ cb_total = dec.tb_params.c;
+ c_neg = dec.tb_params.c_neg;
+ } else { /* For Code Block mode */
+ k_pos = dec.cb_params.k;
+ k_neg = dec.cb_params.k;
+ cb_total = 1;
+ c_neg = 1;
+ }
+
+ /* Clear op status */
+ op->status = 0;
+
+ if (m_in == NULL || m_out == NULL) {
+ rte_bbdev_log(ERR, "Invalid mbuf pointer");
+ op->status = 1 << RTE_BBDEV_DATA_ERROR;
+ return;
+ }
+
+ k = (cb_idx < c_neg) ? k_neg : k_pos;
+ process_dec_cb(q, op, cb_idx, k, m_in, m_out, in_offset, out_offset,
+ &total_left);
+
+ for (cb_idx = 1; cb_idx < cb_total; ++cb_idx) {
+ k = (cb_idx < c_neg) ? k_neg : k_pos;
+
+ m_in = m_in->next;
+ m_out_next = rte_pktmbuf_alloc(m_out->pool);
+ if (m_in == NULL || m_out_next == NULL) {
+ rte_bbdev_log(ERR, "Invalid mbuf pointer");
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ return;
+ }
+ rte_pktmbuf_chain(m_out, m_out_next);
+
+ process_dec_cb(q, op, cb_idx, k, m_in, m_out, in_offset,
+ out_offset, &total_left);
+ }
+ if (total_left != 0) {
+ op->status |= 1 << RTE_BBDEV_DATA_ERROR;
+ rte_bbdev_log(ERR,
+ "Mismatch between rte_bbdev_op_data.length and mbuf data");
+ }
+}
+
+static inline uint16_t
+enqueue_dec_all_ops(struct turbo_sw_queue *q, struct rte_bbdev_dec_op **ops,
+ uint16_t nb_ops)
+{
+ uint16_t i;
+
+ for (i = 0; i < nb_ops; ++i)
+ enqueue_dec_one_op(q, ops[i]);
+
+ return rte_ring_enqueue_burst(q->processed_pkts, (void **)ops, nb_ops,
+ NULL);
+}
+
+/* Enqueue burst */
+static uint16_t
+enqueue_enc_ops(struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_enc_op **ops, uint16_t nb_ops)
+{
+ void *queue = q_data->queue_private;
+ struct turbo_sw_queue *q = queue;
+ uint16_t nb_enqueued = 0;
+
+ nb_enqueued = enqueue_enc_all_ops(q, ops, nb_ops);
+
+ q_data->queue_stats.enqueue_err_count += nb_ops - nb_enqueued;
+ q_data->queue_stats.enqueued_count += nb_enqueued;
+
+ return nb_enqueued;
+}
+
+/* Enqueue burst */
+static uint16_t
+enqueue_dec_ops(struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_dec_op **ops, uint16_t nb_ops)
+{
+ void *queue = q_data->queue_private;
+ struct turbo_sw_queue *q = queue;
+ uint16_t nb_enqueued = 0;
+
+ nb_enqueued = enqueue_dec_all_ops(q, ops, nb_ops);
+
+ q_data->queue_stats.enqueue_err_count += nb_ops - nb_enqueued;
+ q_data->queue_stats.enqueued_count += nb_enqueued;
+
+ return nb_enqueued;
+}
+
+/* Dequeue decode burst */
+static uint16_t
+dequeue_dec_ops(struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_dec_op **ops, uint16_t nb_ops)
+{
+ struct turbo_sw_queue *q = q_data->queue_private;
+ uint16_t nb_dequeued = rte_ring_dequeue_burst(q->processed_pkts,
+ (void **)ops, nb_ops, NULL);
+ q_data->queue_stats.dequeued_count += nb_dequeued;
+
+ return nb_dequeued;
+}
+
+/* Dequeue encode burst */
+static uint16_t
+dequeue_enc_ops(struct rte_bbdev_queue_data *q_data,
+ struct rte_bbdev_enc_op **ops, uint16_t nb_ops)
+{
+ struct turbo_sw_queue *q = q_data->queue_private;
+ uint16_t nb_dequeued = rte_ring_dequeue_burst(q->processed_pkts,
+ (void **)ops, nb_ops, NULL);
+ q_data->queue_stats.dequeued_count += nb_dequeued;
+
+ return nb_dequeued;
+}
+
+/* Parse 16bit integer from string argument */
+static inline int
+parse_u16_arg(const char *key, const char *value, void *extra_args)
+{
+ uint16_t *u16 = extra_args;
+ unsigned int long result;
+
+ if ((value == NULL) || (extra_args == NULL))
+ return -EINVAL;
+ errno = 0;
+ result = strtoul(value, NULL, 0);
+ if ((result >= (1 << 16)) || (errno != 0)) {
+ rte_bbdev_log(ERR, "Invalid value %lu for %s", result, key);
+ return -ERANGE;
+ }
+ *u16 = (uint16_t)result;
+ return 0;
+}
+
+/* Parse parameters used to create device */
+static int
+parse_turbo_sw_params(struct turbo_sw_params *params, const char *input_args)
+{
+ struct rte_kvargs *kvlist = NULL;
+ int ret = 0;
+
+ if (params == NULL)
+ return -EINVAL;
+ if (input_args) {
+ kvlist = rte_kvargs_parse(input_args, turbo_sw_valid_params);
+ if (kvlist == NULL)
+ return -EFAULT;
+
+ ret = rte_kvargs_process(kvlist, turbo_sw_valid_params[0],
+ &parse_u16_arg, ¶ms->queues_num);
+ if (ret < 0)
+ goto exit;
+
+ ret = rte_kvargs_process(kvlist, turbo_sw_valid_params[1],
+ &parse_u16_arg, ¶ms->socket_id);
+ if (ret < 0)
+ goto exit;
+
+ if (params->socket_id >= RTE_MAX_NUMA_NODES) {
+ rte_bbdev_log(ERR, "Invalid socket, must be < %u",
+ RTE_MAX_NUMA_NODES);
+ goto exit;
+ }
+ }
+
+exit:
+ if (kvlist)
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+/* Create device */
+static int
+turbo_sw_bbdev_create(struct rte_vdev_device *vdev,
+ struct turbo_sw_params *init_params)
+{
+ struct rte_bbdev *bbdev;
+ size_t dev_private_size = sizeof(struct bbdev_private);
+ struct bbdev_private *internals;
+
+ vdev->device.numa_node = init_params->socket_id;
+
+ bbdev = rte_bbdev_vdev_allocate(vdev, dev_private_size);
+ if (bbdev == NULL) {
+ rte_bbdev_log(ERR, "Failed to create %s",
+ rte_vdev_device_name(vdev));
+ return -EFAULT;
+ }
+
+ bbdev->dev_ops = &pmd_ops;
+
+ /* register rx/tx burst functions for data path */
+ bbdev->dequeue_enc_ops = dequeue_enc_ops;
+ bbdev->dequeue_dec_ops = dequeue_dec_ops;
+ bbdev->enqueue_enc_ops = enqueue_enc_ops;
+ bbdev->enqueue_dec_ops = enqueue_dec_ops;
+ internals = bbdev->data->dev_private;
+ internals->max_nb_queues = init_params->queues_num;
+
+ return 0;
+}
+
+/* Initialise device */
+static int
+turbo_sw_bbdev_probe(struct rte_vdev_device *vdev)
+{
+ struct turbo_sw_params init_params = {
+ rte_socket_id(),
+ RTE_BBDEV_DEFAULT_MAX_NB_QUEUES
+ };
+ const char *name;
+ const char *input_args;
+
+ if (!vdev)
+ return -EINVAL;
+
+ name = rte_vdev_device_name(vdev);
+ if (name == NULL)
+ return -EINVAL;
+ input_args = rte_vdev_device_args(vdev);
+ parse_turbo_sw_params(&init_params, input_args);
+
+ rte_bbdev_log_debug(
+ "Initialising %s on NUMA node %d with max queues: %d\n",
+ name, init_params.socket_id, init_params.queues_num);
+
+ return turbo_sw_bbdev_create(vdev, &init_params);
+}
+
+/* Uninitialise device */
+static int
+turbo_sw_bbdev_remove(struct rte_vdev_device *vdev)
+{
+ struct rte_bbdev *bbdev;
+ const char *name;
+
+ if (!vdev)
+ return -EINVAL;
+
+ name = rte_vdev_device_name(vdev);
+ if (name == NULL)
+ return -EINVAL;
+
+ bbdev = rte_bbdev_get_named_dev(name);
+ if (bbdev == NULL)
+ return -EINVAL;
+
+ rte_free(bbdev->data->dev_private);
+
+ return rte_bbdev_release(bbdev);
+}
+
+static struct rte_vdev_driver bbdev_turbo_sw_pmd_drv = {
+ .probe = turbo_sw_bbdev_probe,
+ .remove = turbo_sw_bbdev_remove
+};
+
+RTE_PMD_REGISTER_VDEV(DRIVER_NAME, bbdev_turbo_sw_pmd_drv);
+RTE_PMD_REGISTER_ALIAS(DRIVER_NAME, bbdev_turbo_sw_pmd);
+RTE_PMD_REGISTER_PARAM_STRING(DRIVER_NAME,
+ TURBO_SW_MAX_NB_QUEUES_ARG"=<int> "
+ TURBO_SW_SOCKET_ID_ARG"=<int>");
+
+int bbdev_logtype;
+RTE_INIT(null_bbdev_init_log);
+static void
+null_bbdev_init_log(void)
+{
+ bbdev_logtype = rte_log_register("pmd.turbo_sw");
+ if (bbdev_logtype >= 0)
+ rte_log_set_level(bbdev_logtype, RTE_LOG_NOTICE);
+}
diff --git a/drivers/bbdev/turbo_sw/bbdev_turbo_software_tables.h b/drivers/bbdev/turbo_sw/bbdev_turbo_software_tables.h
new file mode 100644
index 0000000..97d9d0e
--- /dev/null
+++ b/drivers/bbdev/turbo_sw/bbdev_turbo_software_tables.h
@@ -0,0 +1,1344 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _BBDEV_TURBO_SW_TABLES_H
+#define _BBDEV_TURBO_SW_TABLES_H
+
+#include <stdint.h>
+
+#include <rte_memory.h>
+#include "bbdev_turbo_software_tables.h"
+
+/* Number of NULL bytes for each K index */
+const int32_t kidx_null_num[188] __rte_cache_aligned = {
+ 60, 36, 12, 84, 60, 36, 12, 84, 60, 36, 12, 84, 60, 36, 12, 84, 60, 36,
+ 12, 84, 60, 36, 12, 84, 60, 36, 12, 84, 60, 36, 12, 84, 60, 36, 12, 84,
+ 60, 36, 12, 84, 60, 36, 12, 84, 60, 36, 12, 84, 60, 36, 12, 84, 60, 36,
+ 12, 84, 60, 36, 12, 84, 36, 84, 36, 84, 36, 84, 36, 84, 36, 84, 36, 84,
+ 36, 84, 36, 84, 36, 84, 36, 84, 36, 84, 36, 84, 36, 84, 36, 84, 36, 84,
+ 36, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84,
+ 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84,
+ 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84,
+ 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84,
+ 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84,
+ 84, 84, 84, 84, 84, 84, 84, 84
+};
+
+/* Indexes of NULL bytes incremented by 1 for each K index */
+const int32_t k0_null_num[188][84] __rte_cache_aligned = {
+ { 1, 3, 5, 9, 13, 17, 19, 21, 25, 29, 33, 35, 37, 41, 45, 49, 51, 53,
+ 57, 61, 65, 66, 69, 70, 73, 74, 81, 82, 89, 90, 97, 98, 101, 102, 105,
+ 106, 113, 114, 121, 122, 129, 130, 133, 134, 137, 138, 145, 146, 153,
+ 154, 161, 162, 165, 169, 170, 177, 178, 185, 186, 192, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 5, 9, 17, 21, 25, 33, 37, 41, 49, 53, 57, 65, 66, 73, 74, 81, 82,
+ 97, 98, 105, 106, 113, 114, 129, 130, 137, 138, 145, 146, 161, 162,
+ 169, 177, 178, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 17, 33, 49, 65, 66, 97, 98, 129, 130, 161, 192, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 4, 7, 10, 13, 16, 19, 25, 28, 31, 34, 37, 40, 43, 49, 52, 55, 58,
+ 61, 64, 67, 73, 76, 79, 82, 85, 88, 91, 97, 98, 103, 104, 109, 110,
+ 115, 116, 121, 122, 127, 128, 133, 134, 145, 146, 151, 152, 157, 158,
+ 163, 164, 169, 170, 175, 176, 181, 182, 193, 194, 199, 200, 205, 206,
+ 211, 212, 217, 218, 223, 224, 229, 230, 241, 242, 247, 248, 253, 254,
+ 259, 265, 266, 271, 272, 277, 278, 288 },
+ { 1, 4, 7, 13, 19, 25, 28, 31, 37, 43, 49, 52, 55, 61, 67, 73, 76, 79,
+ 85, 91, 97, 98, 103, 104, 109, 110, 121, 122, 133, 134, 145, 146, 151,
+ 152, 157, 158, 169, 170, 181, 182, 193, 194, 199, 200, 205, 206, 217,
+ 218, 229, 230, 241, 242, 247, 253, 254, 265, 266, 277, 278, 288, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 7, 13, 25, 31, 37, 49, 55, 61, 73, 79, 85, 97, 98, 109, 110, 121,
+ 122, 145, 146, 157, 158, 169, 170, 193, 194, 205, 206, 217, 218, 241,
+ 242, 253, 265, 266, 288, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 25, 49, 73, 97, 98, 145, 146, 193, 194, 241, 288, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 5, 9, 13, 17, 21, 25, 33, 37, 41, 45, 49, 53, 57, 65, 69, 73, 77,
+ 81, 85, 89, 97, 101, 105, 109, 113, 117, 121, 129, 130, 137, 138, 145,
+ 146, 153, 154, 161, 162, 169, 170, 177, 178, 193, 194, 201, 202, 209,
+ 210, 217, 218, 225, 226, 233, 234, 241, 242, 257, 258, 265, 266, 273,
+ 274, 281, 282, 289, 290, 297, 298, 305, 306, 321, 322, 329, 330, 337,
+ 338, 345, 353, 354, 361, 362, 369, 370, 384 },
+ { 1, 5, 9, 17, 25, 33, 37, 41, 49, 57, 65, 69, 73, 81, 89, 97, 101, 105,
+ 113, 121, 129, 130, 137, 138, 145, 146, 161, 162, 177, 178, 193, 194,
+ 201, 202, 209, 210, 225, 226, 241, 242, 257, 258, 265, 266, 273, 274,
+ 289, 290, 305, 306, 321, 322, 329, 337, 338, 353, 354, 369, 370, 384,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0 },
+ { 1, 9, 17, 33, 41, 49, 65, 73, 81, 97, 105, 113, 129, 130, 145, 146,
+ 161, 162, 193, 194, 209, 210, 225, 226, 257, 258, 273, 274, 289, 290,
+ 321, 322, 337, 353, 354, 384, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 33, 65, 97, 129, 130, 193, 194, 257, 258, 321, 384, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 6, 11, 16, 21, 26, 31, 41, 46, 51, 56, 61, 66, 71, 81, 86, 91, 96,
+ 101, 106, 111, 121, 126, 131, 136, 141, 146, 151, 161, 162, 171, 172,
+ 181, 182, 191, 192, 201, 202, 211, 212, 221, 222, 241, 242, 251, 252,
+ 261, 262, 271, 272, 281, 282, 291, 292, 301, 302, 321, 322, 331, 332,
+ 341, 342, 351, 352, 361, 362, 371, 372, 381, 382, 401, 402, 411, 412,
+ 421, 422, 431, 441, 442, 451, 452, 461, 462, 480 },
+ { 1, 6, 11, 21, 31, 41, 46, 51, 61, 71, 81, 86, 91, 101, 111, 121, 126,
+ 131, 141, 151, 161, 162, 171, 172, 181, 182, 201, 202, 221, 222, 241,
+ 242, 251, 252, 261, 262, 281, 282, 301, 302, 321, 322, 331, 332, 341,
+ 342, 361, 362, 381, 382, 401, 402, 411, 421, 422, 441, 442, 461, 462,
+ 480, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0 },
+ { 1, 11, 21, 41, 51, 61, 81, 91, 101, 121, 131, 141, 161, 162, 181, 182,
+ 201, 202, 241, 242, 261, 262, 281, 282, 321, 322, 341, 342, 361, 362,
+ 401, 402, 421, 441, 442, 480, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 41, 81, 121, 161, 162, 241, 242, 321, 322, 401, 480, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 7, 13, 19, 25, 31, 37, 49, 55, 61, 67, 73, 79, 85, 97, 103, 109,
+ 115, 121, 127, 133, 145, 151, 157, 163, 169, 175, 181, 193, 194, 205,
+ 206, 217, 218, 229, 230, 241, 242, 253, 254, 265, 266, 289, 290, 301,
+ 302, 313, 314, 325, 326, 337, 338, 349, 350, 361, 362, 385, 386, 397,
+ 398, 409, 410, 421, 422, 433, 434, 445, 446, 457, 458, 481, 482, 493,
+ 494, 505, 506, 517, 529, 530, 541, 542, 553, 554, 576 },
+ { 1, 7, 13, 25, 37, 49, 55, 61, 73, 85, 97, 103, 109, 121, 133, 145,
+ 151, 157, 169, 181, 193, 194, 205, 206, 217, 218, 241, 242, 265, 266,
+ 289, 290, 301, 302, 313, 314, 337, 338, 361, 362, 385, 386, 397, 398,
+ 409, 410, 433, 434, 457, 458, 481, 482, 493, 505, 506, 529, 530, 553,
+ 554, 576, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0 },
+ { 1, 13, 25, 49, 61, 73, 97, 109, 121, 145, 157, 169, 193, 194, 217,
+ 218, 241, 242, 289, 290, 313, 314, 337, 338, 385, 386, 409, 410, 433,
+ 434, 481, 482, 505, 529, 530, 576, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 49, 97, 145, 193, 194, 289, 290, 385, 386, 481, 576, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 8, 15, 22, 29, 36, 43, 57, 64, 71, 78, 85, 92, 99, 113, 120, 127,
+ 134, 141, 148, 155, 169, 176, 183, 190, 197, 204, 211, 225, 226, 239,
+ 240, 253, 254, 267, 268, 281, 282, 295, 296, 309, 310, 337, 338, 351,
+ 352, 365, 366, 379, 380, 393, 394, 407, 408, 421, 422, 449, 450, 463,
+ 464, 477, 478, 491, 492, 505, 506, 519, 520, 533, 534, 561, 562, 575,
+ 576, 589, 590, 603, 617, 618, 631, 632, 645, 646, 672 },
+ { 1, 8, 15, 29, 43, 57, 64, 71, 85, 99, 113, 120, 127, 141, 155, 169,
+ 176, 183, 197, 211, 225, 226, 239, 240, 253, 254, 281, 282, 309, 310,
+ 337, 338, 351, 352, 365, 366, 393, 394, 421, 422, 449, 450, 463, 464,
+ 477, 478, 505, 506, 533, 534, 561, 562, 575, 589, 590, 617, 618, 645,
+ 646, 672, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0 },
+ { 1, 15, 29, 57, 71, 85, 113, 127, 141, 169, 183, 197, 225, 226, 253,
+ 254, 281, 282, 337, 338, 365, 366, 393, 394, 449, 450, 477, 478, 505,
+ 506, 561, 562, 589, 617, 618, 672, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 57, 113, 169, 225, 226, 337, 338, 449, 450, 561, 672, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 9, 17, 25, 33, 41, 49, 65, 73, 81, 89, 97, 105, 113, 129, 137, 145,
+ 153, 161, 169, 177, 193, 201, 209, 217, 225, 233, 241, 257, 258, 273,
+ 274, 289, 290, 305, 306, 321, 322, 337, 338, 353, 354, 385, 386, 401,
+ 402, 417, 418, 433, 434, 449, 450, 465, 466, 481, 482, 513, 514, 529,
+ 530, 545, 546, 561, 562, 577, 578, 593, 594, 609, 610, 641, 642, 657,
+ 658, 673, 674, 689, 705, 706, 721, 722, 737, 738, 768 },
+ { 1, 9, 17, 33, 49, 65, 73, 81, 97, 113, 129, 137, 145, 161, 177, 193,
+ 201, 209, 225, 241, 257, 258, 273, 274, 289, 290, 321, 322, 353, 354,
+ 385, 386, 401, 402, 417, 418, 449, 450, 481, 482, 513, 514, 529, 530,
+ 545, 546, 577, 578, 609, 610, 641, 642, 657, 673, 674, 705, 706, 737,
+ 738, 768, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0 },
+ { 1, 17, 33, 65, 81, 97, 129, 145, 161, 193, 209, 225, 257, 258, 289,
+ 290, 321, 322, 385, 386, 417, 418, 449, 450, 513, 514, 545, 546, 577,
+ 578, 641, 642, 673, 705, 706, 768, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 65, 129, 193, 257, 258, 385, 386, 513, 514, 641, 768, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 10, 19, 28, 37, 46, 55, 73, 82, 91, 100, 109, 118, 127, 145, 154,
+ 163, 172, 181, 190, 199, 217, 226, 235, 244, 253, 262, 271, 289, 290,
+ 307, 308, 325, 326, 343, 344, 361, 362, 379, 380, 397, 398, 433, 434,
+ 451, 452, 469, 470, 487, 488, 505, 506, 523, 524, 541, 542, 577, 578,
+ 595, 596, 613, 614, 631, 632, 649, 650, 667, 668, 685, 686, 721, 722,
+ 739, 740, 757, 758, 775, 793, 794, 811, 812, 829, 830, 864 },
+ { 1, 10, 19, 37, 55, 73, 82, 91, 109, 127, 145, 154, 163, 181, 199, 217,
+ 226, 235, 253, 271, 289, 290, 307, 308, 325, 326, 361, 362, 397, 398,
+ 433, 434, 451, 452, 469, 470, 505, 506, 541, 542, 577, 578, 595, 596,
+ 613, 614, 649, 650, 685, 686, 721, 722, 739, 757, 758, 793, 794, 829,
+ 830, 864, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0 },
+ { 1, 19, 37, 73, 91, 109, 145, 163, 181, 217, 235, 253, 289, 290, 325,
+ 326, 361, 362, 433, 434, 469, 470, 505, 506, 577, 578, 613, 614, 649,
+ 650, 721, 722, 757, 793, 794, 864, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 73, 145, 217, 289, 290, 433, 434, 577, 578, 721, 864, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 11, 21, 31, 41, 51, 61, 81, 91, 101, 111, 121, 131, 141, 161, 171,
+ 181, 191, 201, 211, 221, 241, 251, 261, 271, 281, 291, 301, 321, 322,
+ 341, 342, 361, 362, 381, 382, 401, 402, 421, 422, 441, 442, 481, 482,
+ 501, 502, 521, 522, 541, 542, 561, 562, 581, 582, 601, 602, 641, 642,
+ 661, 662, 681, 682, 701, 702, 721, 722, 741, 742, 761, 762, 801, 802,
+ 821, 822, 841, 842, 861, 881, 882, 901, 902, 921, 922, 960 },
+ { 1, 11, 21, 41, 61, 81, 91, 101, 121, 141, 161, 171, 181, 201, 221,
+ 241, 251, 261, 281, 301, 321, 322, 341, 342, 361, 362, 401, 402, 441,
+ 442, 481, 482, 501, 502, 521, 522, 561, 562, 601, 602, 641, 642, 661,
+ 662, 681, 682, 721, 722, 761, 762, 801, 802, 821, 841, 842, 881, 882,
+ 921, 922, 960, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0 },
+ { 1, 21, 41, 81, 101, 121, 161, 181, 201, 241, 261, 281, 321, 322, 361,
+ 362, 401, 402, 481, 482, 521, 522, 561, 562, 641, 642, 681, 682, 721,
+ 722, 801, 802, 841, 881, 882, 960, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 81, 161, 241, 321, 322, 481, 482, 641, 642, 801, 960, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 12, 23, 34, 45, 56, 67, 89, 100, 111, 122, 133, 144, 155, 177, 188,
+ 199, 210, 221, 232, 243, 265, 276, 287, 298, 309, 320, 331, 353, 354,
+ 375, 376, 397, 398, 419, 420, 441, 442, 463, 464, 485, 486, 529, 530,
+ 551, 552, 573, 574, 595, 596, 617, 618, 639, 640, 661, 662, 705, 706,
+ 727, 728, 749, 750, 771, 772, 793, 794, 815, 816, 837, 838, 881, 882,
+ 903, 904, 925, 926, 947, 969, 970, 991, 992, 1013, 1014, 1056 },
+ { 1, 12, 23, 45, 67, 89, 100, 111, 133, 155, 177, 188, 199, 221, 243,
+ 265, 276, 287, 309, 331, 353, 354, 375, 376, 397, 398, 441, 442, 485,
+ 486, 529, 530, 551, 552, 573, 574, 617, 618, 661, 662, 705, 706, 727,
+ 728, 749, 750, 793, 794, 837, 838, 881, 882, 903, 925, 926, 969, 970,
+ 1013, 1014, 1056, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 23, 45, 89, 111, 133, 177, 199, 221, 265, 287, 309, 353, 354, 397,
+ 398, 441, 442, 529, 530, 573, 574, 617, 618, 705, 706, 749, 750, 793,
+ 794, 881, 882, 925, 969, 970, 1056, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 89, 177, 265, 353, 354, 529, 530, 705, 706, 881, 1056, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 13, 25, 37, 49, 61, 73, 97, 109, 121, 133, 145, 157, 169, 193, 205,
+ 217, 229, 241, 253, 265, 289, 301, 313, 325, 337, 349, 361, 385, 386,
+ 409, 410, 433, 434, 457, 458, 481, 482, 505, 506, 529, 530, 577, 578,
+ 601, 602, 625, 626, 649, 650, 673, 674, 697, 698, 721, 722, 769, 770,
+ 793, 794, 817, 818, 841, 842, 865, 866, 889, 890, 913, 914, 961, 962,
+ 985, 986, 1009, 1010, 1033, 1057, 1058, 1081, 1082, 1105, 1106,
+ 1152 },
+ { 1, 13, 25, 49, 73, 97, 109, 121, 145, 169, 193, 205, 217, 241, 265,
+ 289, 301, 313, 337, 361, 385, 386, 409, 410, 433, 434, 481, 482, 529,
+ 530, 577, 578, 601, 602, 625, 626, 673, 674, 721, 722, 769, 770, 793,
+ 794, 817, 818, 865, 866, 913, 914, 961, 962, 985, 1009, 1010, 1057,
+ 1058, 1105, 1106, 1152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 25, 49, 97, 121, 145, 193, 217, 241, 289, 313, 337, 385, 386, 433,
+ 434, 481, 482, 577, 578, 625, 626, 673, 674, 769, 770, 817, 818, 865,
+ 866, 961, 962, 1009, 1057, 1058, 1152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 97, 193, 289, 385, 386, 577, 578, 769, 770, 961, 1152, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 14, 27, 40, 53, 66, 79, 105, 118, 131, 144, 157, 170, 183, 209,
+ 222, 235, 248, 261, 274, 287, 313, 326, 339, 352, 365, 378, 391, 417,
+ 418, 443, 444, 469, 470, 495, 496, 521, 522, 547, 548, 573, 574, 625,
+ 626, 651, 652, 677, 678, 703, 704, 729, 730, 755, 756, 781, 782, 833,
+ 834, 859, 860, 885, 886, 911, 912, 937, 938, 963, 964, 989, 990, 1041,
+ 1042, 1067, 1068, 1093, 1094, 1119, 1145, 1146, 1171, 1172, 1197,
+ 1198, 1248 },
+ { 1, 14, 27, 53, 79, 105, 118, 131, 157, 183, 209, 222, 235, 261, 287,
+ 313, 326, 339, 365, 391, 417, 418, 443, 444, 469, 470, 521, 522, 573,
+ 574, 625, 626, 651, 652, 677, 678, 729, 730, 781, 782, 833, 834, 859,
+ 860, 885, 886, 937, 938, 989, 990, 1041, 1042, 1067, 1093, 1094, 1145,
+ 1146, 1197, 1198, 1248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 27, 53, 105, 131, 157, 209, 235, 261, 313, 339, 365, 417, 418, 469,
+ 470, 521, 522, 625, 626, 677, 678, 729, 730, 833, 834, 885, 886, 937,
+ 938, 1041, 1042, 1093, 1145, 1146, 1248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 105, 209, 313, 417, 418, 625, 626, 833, 834, 1041, 1248, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 15, 29, 43, 57, 71, 85, 113, 127, 141, 155, 169, 183, 197, 225,
+ 239, 253, 267, 281, 295, 309, 337, 351, 365, 379, 393, 407, 421, 449,
+ 450, 477, 478, 505, 506, 533, 534, 561, 562, 589, 590, 617, 618, 673,
+ 674, 701, 702, 729, 730, 757, 758, 785, 786, 813, 814, 841, 842, 897,
+ 898, 925, 926, 953, 954, 981, 982, 1009, 1010, 1037, 1038, 1065, 1066,
+ 1121, 1122, 1149, 1150, 1177, 1178, 1205, 1233, 1234, 1261, 1262,
+ 1289, 1290, 1344 },
+ { 1, 15, 29, 57, 85, 113, 127, 141, 169, 197, 225, 239, 253, 281, 309,
+ 337, 351, 365, 393, 421, 449, 450, 477, 478, 505, 506, 561, 562, 617,
+ 618, 673, 674, 701, 702, 729, 730, 785, 786, 841, 842, 897, 898, 925,
+ 926, 953, 954, 1009, 1010, 1065, 1066, 1121, 1122, 1149, 1177, 1178,
+ 1233, 1234, 1289, 1290, 1344, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 29, 57, 113, 141, 169, 225, 253, 281, 337, 365, 393, 449, 450, 505,
+ 506, 561, 562, 673, 674, 729, 730, 785, 786, 897, 898, 953, 954, 1009,
+ 1010, 1121, 1122, 1177, 1233, 1234, 1344, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 113, 225, 337, 449, 450, 673, 674, 897, 898, 1121, 1344, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 16, 31, 46, 61, 76, 91, 121, 136, 151, 166, 181, 196, 211, 241,
+ 256, 271, 286, 301, 316, 331, 361, 376, 391, 406, 421, 436, 451, 481,
+ 482, 511, 512, 541, 542, 571, 572, 601, 602, 631, 632, 661, 662, 721,
+ 722, 751, 752, 781, 782, 811, 812, 841, 842, 871, 872, 901, 902, 961,
+ 962, 991, 992, 1021, 1022, 1051, 1052, 1081, 1082, 1111, 1112, 1141,
+ 1142, 1201, 1202, 1231, 1232, 1261, 1262, 1291, 1321, 1322, 1351,
+ 1352, 1381, 1382, 1440 },
+ { 1, 16, 31, 61, 91, 121, 136, 151, 181, 211, 241, 256, 271, 301, 331,
+ 361, 376, 391, 421, 451, 481, 482, 511, 512, 541, 542, 601, 602, 661,
+ 662, 721, 722, 751, 752, 781, 782, 841, 842, 901, 902, 961, 962, 991,
+ 992, 1021, 1022, 1081, 1082, 1141, 1142, 1201, 1202, 1231, 1261, 1262,
+ 1321, 1322, 1381, 1382, 1440, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 31, 61, 121, 151, 181, 241, 271, 301, 361, 391, 421, 481, 482, 541,
+ 542, 601, 602, 721, 722, 781, 782, 841, 842, 961, 962, 1021, 1022,
+ 1081, 1082, 1201, 1202, 1261, 1321, 1322, 1440, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 121, 241, 361, 481, 482, 721, 722, 961, 962, 1201, 1440, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 17, 33, 49, 65, 81, 97, 129, 145, 161, 177, 193, 209, 225, 257,
+ 273, 289, 305, 321, 337, 353, 385, 401, 417, 433, 449, 465, 481, 513,
+ 514, 545, 546, 577, 578, 609, 610, 641, 642, 673, 674, 705, 706, 769,
+ 770, 801, 802, 833, 834, 865, 866, 897, 898, 929, 930, 961, 962, 1025,
+ 1026, 1057, 1058, 1089, 1090, 1121, 1122, 1153, 1154, 1185, 1186,
+ 1217, 1218, 1281, 1282, 1313, 1314, 1345, 1346, 1377, 1409, 1410,
+ 1441, 1442, 1473, 1474, 1536 },
+ { 1, 17, 33, 65, 97, 129, 145, 161, 193, 225, 257, 273, 289, 321, 353,
+ 385, 401, 417, 449, 481, 513, 514, 545, 546, 577, 578, 641, 642, 705,
+ 706, 769, 770, 801, 802, 833, 834, 897, 898, 961, 962, 1025, 1026,
+ 1057, 1058, 1089, 1090, 1153, 1154, 1217, 1218, 1281, 1282, 1313,
+ 1345, 1346, 1409, 1410, 1473, 1474, 1536, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 33, 65, 129, 161, 193, 257, 289, 321, 385, 417, 449, 513, 514, 577,
+ 578, 641, 642, 769, 770, 833, 834, 897, 898, 1025, 1026, 1089, 1090,
+ 1153, 1154, 1281, 1282, 1345, 1409, 1410, 1536, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 129, 257, 385, 513, 514, 769, 770, 1025, 1026, 1281, 1536, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 18, 35, 52, 69, 86, 103, 137, 154, 171, 188, 205, 222, 239, 273,
+ 290, 307, 324, 341, 358, 375, 409, 426, 443, 460, 477, 494, 511, 545,
+ 546, 579, 580, 613, 614, 647, 648, 681, 682, 715, 716, 749, 750, 817,
+ 818, 851, 852, 885, 886, 919, 920, 953, 954, 987, 988, 1021, 1022,
+ 1089, 1090, 1123, 1124, 1157, 1158, 1191, 1192, 1225, 1226, 1259,
+ 1260, 1293, 1294, 1361, 1362, 1395, 1396, 1429, 1430, 1463, 1497,
+ 1498, 1531, 1532, 1565, 1566, 1632 },
+ { 1, 35, 69, 137, 171, 205, 273, 307, 341, 409, 443, 477, 545, 546, 613,
+ 614, 681, 682, 817, 818, 885, 886, 953, 954, 1089, 1090, 1157, 1158,
+ 1225, 1226, 1361, 1362, 1429, 1497, 1498, 1632, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 19, 37, 55, 73, 91, 109, 145, 163, 181, 199, 217, 235, 253, 289,
+ 307, 325, 343, 361, 379, 397, 433, 451, 469, 487, 505, 523, 541, 577,
+ 578, 613, 614, 649, 650, 685, 686, 721, 722, 757, 758, 793, 794, 865,
+ 866, 901, 902, 937, 938, 973, 974, 1009, 1010, 1045, 1046, 1081, 1082,
+ 1153, 1154, 1189, 1190, 1225, 1226, 1261, 1262, 1297, 1298, 1333,
+ 1334, 1369, 1370, 1441, 1442, 1477, 1478, 1513, 1514, 1549, 1585,
+ 1586, 1621, 1622, 1657, 1658, 1728 },
+ { 1, 37, 73, 145, 181, 217, 289, 325, 361, 433, 469, 505, 577, 578, 649,
+ 650, 721, 722, 865, 866, 937, 938, 1009, 1010, 1153, 1154, 1225, 1226,
+ 1297, 1298, 1441, 1442, 1513, 1585, 1586, 1728, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 20, 39, 58, 77, 96, 115, 153, 172, 191, 210, 229, 248, 267, 305,
+ 324, 343, 362, 381, 400, 419, 457, 476, 495, 514, 533, 552, 571, 609,
+ 610, 647, 648, 685, 686, 723, 724, 761, 762, 799, 800, 837, 838, 913,
+ 914, 951, 952, 989, 990, 1027, 1028, 1065, 1066, 1103, 1104, 1141,
+ 1142, 1217, 1218, 1255, 1256, 1293, 1294, 1331, 1332, 1369, 1370,
+ 1407, 1408, 1445, 1446, 1521, 1522, 1559, 1560, 1597, 1598, 1635,
+ 1673, 1674, 1711, 1712, 1749, 1750, 1824 },
+ { 1, 39, 77, 153, 191, 229, 305, 343, 381, 457, 495, 533, 609, 610, 685,
+ 686, 761, 762, 913, 914, 989, 990, 1065, 1066, 1217, 1218, 1293, 1294,
+ 1369, 1370, 1521, 1522, 1597, 1673, 1674, 1824, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 21, 41, 61, 81, 101, 121, 161, 181, 201, 221, 241, 261, 281, 321,
+ 341, 361, 381, 401, 421, 441, 481, 501, 521, 541, 561, 581, 601, 641,
+ 642, 681, 682, 721, 722, 761, 762, 801, 802, 841, 842, 881, 882, 961,
+ 962, 1001, 1002, 1041, 1042, 1081, 1082, 1121, 1122, 1161, 1162, 1201,
+ 1202, 1281, 1282, 1321, 1322, 1361, 1362, 1401, 1402, 1441, 1442,
+ 1481, 1482, 1521, 1522, 1601, 1602, 1641, 1642, 1681, 1682, 1721,
+ 1761, 1762, 1801, 1802, 1841, 1842, 1920 },
+ { 1, 41, 81, 161, 201, 241, 321, 361, 401, 481, 521, 561, 641, 642, 721,
+ 722, 801, 802, 961, 962, 1041, 1042, 1121, 1122, 1281, 1282, 1361,
+ 1362, 1441, 1442, 1601, 1602, 1681, 1761, 1762, 1920, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 22, 43, 64, 85, 106, 127, 169, 190, 211, 232, 253, 274, 295, 337,
+ 358, 379, 400, 421, 442, 463, 505, 526, 547, 568, 589, 610, 631, 673,
+ 674, 715, 716, 757, 758, 799, 800, 841, 842, 883, 884, 925, 926, 1009,
+ 1010, 1051, 1052, 1093, 1094, 1135, 1136, 1177, 1178, 1219, 1220,
+ 1261, 1262, 1345, 1346, 1387, 1388, 1429, 1430, 1471, 1472, 1513,
+ 1514, 1555, 1556, 1597, 1598, 1681, 1682, 1723, 1724, 1765, 1766,
+ 1807, 1849, 1850, 1891, 1892, 1933, 1934, 2016 },
+ { 1, 43, 85, 169, 211, 253, 337, 379, 421, 505, 547, 589, 673, 674, 757,
+ 758, 841, 842, 1009, 1010, 1093, 1094, 1177, 1178, 1345, 1346, 1429,
+ 1430, 1513, 1514, 1681, 1682, 1765, 1849, 1850, 2016, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 23, 45, 67, 89, 111, 133, 177, 199, 221, 243, 265, 287, 309, 353,
+ 375, 397, 419, 441, 463, 485, 529, 551, 573, 595, 617, 639, 661, 705,
+ 706, 749, 750, 793, 794, 837, 838, 881, 882, 925, 926, 969, 970, 1057,
+ 1058, 1101, 1102, 1145, 1146, 1189, 1190, 1233, 1234, 1277, 1278,
+ 1321, 1322, 1409, 1410, 1453, 1454, 1497, 1498, 1541, 1542, 1585,
+ 1586, 1629, 1630, 1673, 1674, 1761, 1762, 1805, 1806, 1849, 1850,
+ 1893, 1937, 1938, 1981, 1982, 2025, 2026, 2112 },
+ { 1, 45, 89, 177, 221, 265, 353, 397, 441, 529, 573, 617, 705, 706, 793,
+ 794, 881, 882, 1057, 1058, 1145, 1146, 1233, 1234, 1409, 1410, 1497,
+ 1498, 1585, 1586, 1761, 1762, 1849, 1937, 1938, 2112, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 24, 47, 70, 93, 116, 139, 185, 208, 231, 254, 277, 300, 323, 369,
+ 392, 415, 438, 461, 484, 507, 553, 576, 599, 622, 645, 668, 691, 737,
+ 738, 783, 784, 829, 830, 875, 876, 921, 922, 967, 968, 1013, 1014,
+ 1105, 1106, 1151, 1152, 1197, 1198, 1243, 1244, 1289, 1290, 1335,
+ 1336, 1381, 1382, 1473, 1474, 1519, 1520, 1565, 1566, 1611, 1612,
+ 1657, 1658, 1703, 1704, 1749, 1750, 1841, 1842, 1887, 1888, 1933,
+ 1934, 1979, 2025, 2026, 2071, 2072, 2117, 2118, 2208 },
+ { 1, 47, 93, 185, 231, 277, 369, 415, 461, 553, 599, 645, 737, 738, 829,
+ 830, 921, 922, 1105, 1106, 1197, 1198, 1289, 1290, 1473, 1474, 1565,
+ 1566, 1657, 1658, 1841, 1842, 1933, 2025, 2026, 2208, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 25, 49, 73, 97, 121, 145, 193, 217, 241, 265, 289, 313, 337, 385,
+ 409, 433, 457, 481, 505, 529, 577, 601, 625, 649, 673, 697, 721, 769,
+ 770, 817, 818, 865, 866, 913, 914, 961, 962, 1009, 1010, 1057, 1058,
+ 1153, 1154, 1201, 1202, 1249, 1250, 1297, 1298, 1345, 1346, 1393,
+ 1394, 1441, 1442, 1537, 1538, 1585, 1586, 1633, 1634, 1681, 1682,
+ 1729, 1730, 1777, 1778, 1825, 1826, 1921, 1922, 1969, 1970, 2017,
+ 2018, 2065, 2113, 2114, 2161, 2162, 2209, 2210, 2304 },
+ { 1, 49, 97, 193, 241, 289, 385, 433, 481, 577, 625, 673, 769, 770, 865,
+ 866, 961, 962, 1153, 1154, 1249, 1250, 1345, 1346, 1537, 1538, 1633,
+ 1634, 1729, 1730, 1921, 1922, 2017, 2113, 2114, 2304, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 26, 51, 76, 101, 126, 151, 201, 226, 251, 276, 301, 326, 351, 401,
+ 426, 451, 476, 501, 526, 551, 601, 626, 651, 676, 701, 726, 751, 801,
+ 802, 851, 852, 901, 902, 951, 952, 1001, 1002, 1051, 1052, 1101, 1102,
+ 1201, 1202, 1251, 1252, 1301, 1302, 1351, 1352, 1401, 1402, 1451,
+ 1452, 1501, 1502, 1601, 1602, 1651, 1652, 1701, 1702, 1751, 1752,
+ 1801, 1802, 1851, 1852, 1901, 1902, 2001, 2002, 2051, 2052, 2101,
+ 2102, 2151, 2201, 2202, 2251, 2252, 2301, 2302, 2400 },
+ { 1, 51, 101, 201, 251, 301, 401, 451, 501, 601, 651, 701, 801, 802,
+ 901, 902, 1001, 1002, 1201, 1202, 1301, 1302, 1401, 1402, 1601, 1602,
+ 1701, 1702, 1801, 1802, 2001, 2002, 2101, 2201, 2202, 2400, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 27, 53, 79, 105, 131, 157, 209, 235, 261, 287, 313, 339, 365, 417,
+ 443, 469, 495, 521, 547, 573, 625, 651, 677, 703, 729, 755, 781, 833,
+ 834, 885, 886, 937, 938, 989, 990, 1041, 1042, 1093, 1094, 1145, 1146,
+ 1249, 1250, 1301, 1302, 1353, 1354, 1405, 1406, 1457, 1458, 1509,
+ 1510, 1561, 1562, 1665, 1666, 1717, 1718, 1769, 1770, 1821, 1822,
+ 1873, 1874, 1925, 1926, 1977, 1978, 2081, 2082, 2133, 2134, 2185,
+ 2186, 2237, 2289, 2290, 2341, 2342, 2393, 2394, 2496 },
+ { 1, 53, 105, 209, 261, 313, 417, 469, 521, 625, 677, 729, 833, 834,
+ 937, 938, 1041, 1042, 1249, 1250, 1353, 1354, 1457, 1458, 1665, 1666,
+ 1769, 1770, 1873, 1874, 2081, 2082, 2185, 2289, 2290, 2496, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 28, 55, 82, 109, 136, 163, 217, 244, 271, 298, 325, 352, 379, 433,
+ 460, 487, 514, 541, 568, 595, 649, 676, 703, 730, 757, 784, 811, 865,
+ 866, 919, 920, 973, 974, 1027, 1028, 1081, 1082, 1135, 1136, 1189,
+ 1190, 1297, 1298, 1351, 1352, 1405, 1406, 1459, 1460, 1513, 1514,
+ 1567, 1568, 1621, 1622, 1729, 1730, 1783, 1784, 1837, 1838, 1891,
+ 1892, 1945, 1946, 1999, 2000, 2053, 2054, 2161, 2162, 2215, 2216,
+ 2269, 2270, 2323, 2377, 2378, 2431, 2432, 2485, 2486, 2592 },
+ { 1, 55, 109, 217, 271, 325, 433, 487, 541, 649, 703, 757, 865, 866,
+ 973, 974, 1081, 1082, 1297, 1298, 1405, 1406, 1513, 1514, 1729, 1730,
+ 1837, 1838, 1945, 1946, 2161, 2162, 2269, 2377, 2378, 2592, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ { 1, 29, 57, 85, 113, 141, 169, 225, 253, 281, 309, 337, 365, 393, 449,
+ 477, 505, 533, 561, 589, 617, 673, 701, 729, 757, 785, 813, 841, 897,
+ 898, 953, 954, 1009, 1010, 1065, 1066, 1121, 1122, 1177, 1178, 1233,
+ 1234, 1345, 1346, 1401, 1402, 1457, 1458, 1513, 1514, 1569, 1570,
+ 1625, 1626, 1681, 1682, 1793, 1794, 1849, 1850, 1905, 1906, 1961,
+ 1962, 2017, 2018, 2073, 2074, 2129, 2130, 2241, 2242, 2297, 2298,
+ 2353, 2354, 2409, 2465, 2466, 2521, 2522, 2577, 2578, 2688 },
+ { 1, 57, 113, 225, 281, 337, 449, 505, 561, 673, 729, 785, 897, 898,
+ 1009, 1010, 1121, 1122, 1345, 1346, 1457, 1458, 1569, 1570, 1793,
+ 1794, 1905, 1906, 2017, 2018, 2241, 2242, 2353, 2465, 2466, 2688, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0 },
+ { 1, 30, 59, 88, 117, 146, 175, 233, 262, 291, 320, 349, 378, 407, 465,
+ 494, 523, 552, 581, 610, 639, 697, 726, 755, 784, 813, 842, 871, 929,
+ 930, 987, 988, 1045, 1046, 1103, 1104, 1161, 1162, 1219, 1220, 1277,
+ 1278, 1393, 1394, 1451, 1452, 1509, 1510, 1567, 1568, 1625, 1626,
+ 1683, 1684, 1741, 1742, 1857, 1858, 1915, 1916, 1973, 1974, 2031,
+ 2032, 2089, 2090, 2147, 2148, 2205, 2206, 2321, 2322, 2379, 2380,
+ 2437, 2438, 2495, 2553, 2554, 2611, 2612, 2669, 2670, 2784 },
+ { 1, 59, 117, 233, 291, 349, 465, 523, 581, 697, 755, 813, 929, 930,
+ 1045, 1046, 1161, 1162, 1393, 1394, 1509, 1510, 1625, 1626, 1857,
+ 1858, 1973, 1974, 2089, 2090, 2321, 2322, 2437, 2553, 2554, 2784, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0 },
+ { 1, 31, 61, 91, 121, 151, 181, 241, 271, 301, 331, 361, 391, 421, 481,
+ 511, 541, 571, 601, 631, 661, 721, 751, 781, 811, 841, 871, 901, 961,
+ 962, 1021, 1022, 1081, 1082, 1141, 1142, 1201, 1202, 1261, 1262, 1321,
+ 1322, 1441, 1442, 1501, 1502, 1561, 1562, 1621, 1622, 1681, 1682,
+ 1741, 1742, 1801, 1802, 1921, 1922, 1981, 1982, 2041, 2042, 2101,
+ 2102, 2161, 2162, 2221, 2222, 2281, 2282, 2401, 2402, 2461, 2462,
+ 2521, 2522, 2581, 2641, 2642, 2701, 2702, 2761, 2762, 2880 },
+ { 1, 61, 121, 241, 301, 361, 481, 541, 601, 721, 781, 841, 961, 962,
+ 1081, 1082, 1201, 1202, 1441, 1442, 1561, 1562, 1681, 1682, 1921,
+ 1922, 2041, 2042, 2161, 2162, 2401, 2402, 2521, 2641, 2642, 2880,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0 },
+ { 1, 32, 63, 94, 125, 156, 187, 249, 280, 311, 342, 373, 404, 435, 497,
+ 528, 559, 590, 621, 652, 683, 745, 776, 807, 838, 869, 900, 931, 993,
+ 994, 1055, 1056, 1117, 1118, 1179, 1180, 1241, 1242, 1303, 1304, 1365,
+ 1366, 1489, 1490, 1551, 1552, 1613, 1614, 1675, 1676, 1737, 1738,
+ 1799, 1800, 1861, 1862, 1985, 1986, 2047, 2048, 2109, 2110, 2171,
+ 2172, 2233, 2234, 2295, 2296, 2357, 2358, 2481, 2482, 2543, 2544,
+ 2605, 2606, 2667, 2729, 2730, 2791, 2792, 2853, 2854, 2976 },
+ { 1, 63, 125, 249, 311, 373, 497, 559, 621, 745, 807, 869, 993, 994,
+ 1117, 1118, 1241, 1242, 1489, 1490, 1613, 1614, 1737, 1738, 1985,
+ 1986, 2109, 2110, 2233, 2234, 2481, 2482, 2605, 2729, 2730, 2976, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0 },
+ { 1, 33, 65, 97, 129, 161, 193, 257, 289, 321, 353, 385, 417, 449, 513,
+ 545, 577, 609, 641, 673, 705, 769, 801, 833, 865, 897, 929, 961, 1025,
+ 1026, 1089, 1090, 1153, 1154, 1217, 1218, 1281, 1282, 1345, 1346,
+ 1409, 1410, 1537, 1538, 1601, 1602, 1665, 1666, 1729, 1730, 1793,
+ 1794, 1857, 1858, 1921, 1922, 2049, 2050, 2113, 2114, 2177, 2178,
+ 2241, 2242, 2305, 2306, 2369, 2370, 2433, 2434, 2561, 2562, 2625,
+ 2626, 2689, 2690, 2753, 2817, 2818, 2881, 2882, 2945, 2946, 3072 },
+ { 1, 65, 129, 257, 321, 385, 513, 577, 641, 769, 833, 897, 1025, 1026,
+ 1153, 1154, 1281, 1282, 1537, 1538, 1665, 1666, 1793, 1794, 2049,
+ 2050, 2177, 2178, 2305, 2306, 2561, 2562, 2689, 2817, 2818, 3072, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0 },
+ { 1, 34, 67, 100, 133, 166, 199, 265, 298, 331, 364, 397, 430, 463, 529,
+ 562, 595, 628, 661, 694, 727, 793, 826, 859, 892, 925, 958, 991, 1057,
+ 1058, 1123, 1124, 1189, 1190, 1255, 1256, 1321, 1322, 1387, 1388,
+ 1453, 1454, 1585, 1586, 1651, 1652, 1717, 1718, 1783, 1784, 1849,
+ 1850, 1915, 1916, 1981, 1982, 2113, 2114, 2179, 2180, 2245, 2246,
+ 2311, 2312, 2377, 2378, 2443, 2444, 2509, 2510, 2641, 2642, 2707,
+ 2708, 2773, 2774, 2839, 2905, 2906, 2971, 2972, 3037, 3038, 3168 },
+ { 1, 35, 69, 103, 137, 171, 205, 273, 307, 341, 375, 409, 443, 477, 545,
+ 579, 613, 647, 681, 715, 749, 817, 851, 885, 919, 953, 987, 1021,
+ 1089, 1090, 1157, 1158, 1225, 1226, 1293, 1294, 1361, 1362, 1429,
+ 1430, 1497, 1498, 1633, 1634, 1701, 1702, 1769, 1770, 1837, 1838,
+ 1905, 1906, 1973, 1974, 2041, 2042, 2177, 2178, 2245, 2246, 2313,
+ 2314, 2381, 2382, 2449, 2450, 2517, 2518, 2585, 2586, 2721, 2722,
+ 2789, 2790, 2857, 2858, 2925, 2993, 2994, 3061, 3062, 3129, 3130,
+ 3264 },
+ { 1, 36, 71, 106, 141, 176, 211, 281, 316, 351, 386, 421, 456, 491, 561,
+ 596, 631, 666, 701, 736, 771, 841, 876, 911, 946, 981, 1016, 1051,
+ 1121, 1122, 1191, 1192, 1261, 1262, 1331, 1332, 1401, 1402, 1471,
+ 1472, 1541, 1542, 1681, 1682, 1751, 1752, 1821, 1822, 1891, 1892,
+ 1961, 1962, 2031, 2032, 2101, 2102, 2241, 2242, 2311, 2312, 2381,
+ 2382, 2451, 2452, 2521, 2522, 2591, 2592, 2661, 2662, 2801, 2802,
+ 2871, 2872, 2941, 2942, 3011, 3081, 3082, 3151, 3152, 3221, 3222,
+ 3360 },
+ { 1, 37, 73, 109, 145, 181, 217, 289, 325, 361, 397, 433, 469, 505, 577,
+ 613, 649, 685, 721, 757, 793, 865, 901, 937, 973, 1009, 1045, 1081,
+ 1153, 1154, 1225, 1226, 1297, 1298, 1369, 1370, 1441, 1442, 1513,
+ 1514, 1585, 1586, 1729, 1730, 1801, 1802, 1873, 1874, 1945, 1946,
+ 2017, 2018, 2089, 2090, 2161, 2162, 2305, 2306, 2377, 2378, 2449,
+ 2450, 2521, 2522, 2593, 2594, 2665, 2666, 2737, 2738, 2881, 2882,
+ 2953, 2954, 3025, 3026, 3097, 3169, 3170, 3241, 3242, 3313, 3314,
+ 3456 },
+ { 1, 38, 75, 112, 149, 186, 223, 297, 334, 371, 408, 445, 482, 519, 593,
+ 630, 667, 704, 741, 778, 815, 889, 926, 963, 1000, 1037, 1074, 1111,
+ 1185, 1186, 1259, 1260, 1333, 1334, 1407, 1408, 1481, 1482, 1555,
+ 1556, 1629, 1630, 1777, 1778, 1851, 1852, 1925, 1926, 1999, 2000,
+ 2073, 2074, 2147, 2148, 2221, 2222, 2369, 2370, 2443, 2444, 2517,
+ 2518, 2591, 2592, 2665, 2666, 2739, 2740, 2813, 2814, 2961, 2962,
+ 3035, 3036, 3109, 3110, 3183, 3257, 3258, 3331, 3332, 3405, 3406,
+ 3552 },
+ { 1, 39, 77, 115, 153, 191, 229, 305, 343, 381, 419, 457, 495, 533, 609,
+ 647, 685, 723, 761, 799, 837, 913, 951, 989, 1027, 1065, 1103, 1141,
+ 1217, 1218, 1293, 1294, 1369, 1370, 1445, 1446, 1521, 1522, 1597,
+ 1598, 1673, 1674, 1825, 1826, 1901, 1902, 1977, 1978, 2053, 2054,
+ 2129, 2130, 2205, 2206, 2281, 2282, 2433, 2434, 2509, 2510, 2585,
+ 2586, 2661, 2662, 2737, 2738, 2813, 2814, 2889, 2890, 3041, 3042,
+ 3117, 3118, 3193, 3194, 3269, 3345, 3346, 3421, 3422, 3497, 3498,
+ 3648 },
+ { 1, 40, 79, 118, 157, 196, 235, 313, 352, 391, 430, 469, 508, 547, 625,
+ 664, 703, 742, 781, 820, 859, 937, 976, 1015, 1054, 1093, 1132, 1171,
+ 1249, 1250, 1327, 1328, 1405, 1406, 1483, 1484, 1561, 1562, 1639,
+ 1640, 1717, 1718, 1873, 1874, 1951, 1952, 2029, 2030, 2107, 2108,
+ 2185, 2186, 2263, 2264, 2341, 2342, 2497, 2498, 2575, 2576, 2653,
+ 2654, 2731, 2732, 2809, 2810, 2887, 2888, 2965, 2966, 3121, 3122,
+ 3199, 3200, 3277, 3278, 3355, 3433, 3434, 3511, 3512, 3589, 3590,
+ 3744 },
+ { 1, 41, 81, 121, 161, 201, 241, 321, 361, 401, 441, 481, 521, 561, 641,
+ 681, 721, 761, 801, 841, 881, 961, 1001, 1041, 1081, 1121, 1161, 1201,
+ 1281, 1282, 1361, 1362, 1441, 1442, 1521, 1522, 1601, 1602, 1681,
+ 1682, 1761, 1762, 1921, 1922, 2001, 2002, 2081, 2082, 2161, 2162,
+ 2241, 2242, 2321, 2322, 2401, 2402, 2561, 2562, 2641, 2642, 2721,
+ 2722, 2801, 2802, 2881, 2882, 2961, 2962, 3041, 3042, 3201, 3202,
+ 3281, 3282, 3361, 3362, 3441, 3521, 3522, 3601, 3602, 3681, 3682,
+ 3840 },
+ { 1, 42, 83, 124, 165, 206, 247, 329, 370, 411, 452, 493, 534, 575, 657,
+ 698, 739, 780, 821, 862, 903, 985, 1026, 1067, 1108, 1149, 1190, 1231,
+ 1313, 1314, 1395, 1396, 1477, 1478, 1559, 1560, 1641, 1642, 1723,
+ 1724, 1805, 1806, 1969, 1970, 2051, 2052, 2133, 2134, 2215, 2216,
+ 2297, 2298, 2379, 2380, 2461, 2462, 2625, 2626, 2707, 2708, 2789,
+ 2790, 2871, 2872, 2953, 2954, 3035, 3036, 3117, 3118, 3281, 3282,
+ 3363, 3364, 3445, 3446, 3527, 3609, 3610, 3691, 3692, 3773, 3774,
+ 3936 },
+ { 1, 43, 85, 127, 169, 211, 253, 337, 379, 421, 463, 505, 547, 589, 673,
+ 715, 757, 799, 841, 883, 925, 1009, 1051, 1093, 1135, 1177, 1219,
+ 1261, 1345, 1346, 1429, 1430, 1513, 1514, 1597, 1598, 1681, 1682,
+ 1765, 1766, 1849, 1850, 2017, 2018, 2101, 2102, 2185, 2186, 2269,
+ 2270, 2353, 2354, 2437, 2438, 2521, 2522, 2689, 2690, 2773, 2774,
+ 2857, 2858, 2941, 2942, 3025, 3026, 3109, 3110, 3193, 3194, 3361,
+ 3362, 3445, 3446, 3529, 3530, 3613, 3697, 3698, 3781, 3782, 3865,
+ 3866, 4032 },
+ { 1, 44, 87, 130, 173, 216, 259, 345, 388, 431, 474, 517, 560, 603, 689,
+ 732, 775, 818, 861, 904, 947, 1033, 1076, 1119, 1162, 1205, 1248,
+ 1291, 1377, 1378, 1463, 1464, 1549, 1550, 1635, 1636, 1721, 1722,
+ 1807, 1808, 1893, 1894, 2065, 2066, 2151, 2152, 2237, 2238, 2323,
+ 2324, 2409, 2410, 2495, 2496, 2581, 2582, 2753, 2754, 2839, 2840,
+ 2925, 2926, 3011, 3012, 3097, 3098, 3183, 3184, 3269, 3270, 3441,
+ 3442, 3527, 3528, 3613, 3614, 3699, 3785, 3786, 3871, 3872, 3957,
+ 3958, 4128 },
+ { 1, 45, 89, 133, 177, 221, 265, 353, 397, 441, 485, 529, 573, 617, 705,
+ 749, 793, 837, 881, 925, 969, 1057, 1101, 1145, 1189, 1233, 1277,
+ 1321, 1409, 1410, 1497, 1498, 1585, 1586, 1673, 1674, 1761, 1762,
+ 1849, 1850, 1937, 1938, 2113, 2114, 2201, 2202, 2289, 2290, 2377,
+ 2378, 2465, 2466, 2553, 2554, 2641, 2642, 2817, 2818, 2905, 2906,
+ 2993, 2994, 3081, 3082, 3169, 3170, 3257, 3258, 3345, 3346, 3521,
+ 3522, 3609, 3610, 3697, 3698, 3785, 3873, 3874, 3961, 3962, 4049,
+ 4050, 4224 },
+ { 1, 46, 91, 136, 181, 226, 271, 361, 406, 451, 496, 541, 586, 631, 721,
+ 766, 811, 856, 901, 946, 991, 1081, 1126, 1171, 1216, 1261, 1306,
+ 1351, 1441, 1442, 1531, 1532, 1621, 1622, 1711, 1712, 1801, 1802,
+ 1891, 1892, 1981, 1982, 2161, 2162, 2251, 2252, 2341, 2342, 2431,
+ 2432, 2521, 2522, 2611, 2612, 2701, 2702, 2881, 2882, 2971, 2972,
+ 3061, 3062, 3151, 3152, 3241, 3242, 3331, 3332, 3421, 3422, 3601,
+ 3602, 3691, 3692, 3781, 3782, 3871, 3961, 3962, 4051, 4052, 4141,
+ 4142, 4320 },
+ { 1, 47, 93, 139, 185, 231, 277, 369, 415, 461, 507, 553, 599, 645, 737,
+ 783, 829, 875, 921, 967, 1013, 1105, 1151, 1197, 1243, 1289, 1335,
+ 1381, 1473, 1474, 1565, 1566, 1657, 1658, 1749, 1750, 1841, 1842,
+ 1933, 1934, 2025, 2026, 2209, 2210, 2301, 2302, 2393, 2394, 2485,
+ 2486, 2577, 2578, 2669, 2670, 2761, 2762, 2945, 2946, 3037, 3038,
+ 3129, 3130, 3221, 3222, 3313, 3314, 3405, 3406, 3497, 3498, 3681,
+ 3682, 3773, 3774, 3865, 3866, 3957, 4049, 4050, 4141, 4142, 4233,
+ 4234, 4416 },
+ { 1, 48, 95, 142, 189, 236, 283, 377, 424, 471, 518, 565, 612, 659, 753,
+ 800, 847, 894, 941, 988, 1035, 1129, 1176, 1223, 1270, 1317, 1364,
+ 1411, 1505, 1506, 1599, 1600, 1693, 1694, 1787, 1788, 1881, 1882,
+ 1975, 1976, 2069, 2070, 2257, 2258, 2351, 2352, 2445, 2446, 2539,
+ 2540, 2633, 2634, 2727, 2728, 2821, 2822, 3009, 3010, 3103, 3104,
+ 3197, 3198, 3291, 3292, 3385, 3386, 3479, 3480, 3573, 3574, 3761,
+ 3762, 3855, 3856, 3949, 3950, 4043, 4137, 4138, 4231, 4232, 4325,
+ 4326, 4512 },
+ { 1, 49, 97, 145, 193, 241, 289, 385, 433, 481, 529, 577, 625, 673, 769,
+ 817, 865, 913, 961, 1009, 1057, 1153, 1201, 1249, 1297, 1345, 1393,
+ 1441, 1537, 1538, 1633, 1634, 1729, 1730, 1825, 1826, 1921, 1922,
+ 2017, 2018, 2113, 2114, 2305, 2306, 2401, 2402, 2497, 2498, 2593,
+ 2594, 2689, 2690, 2785, 2786, 2881, 2882, 3073, 3074, 3169, 3170,
+ 3265, 3266, 3361, 3362, 3457, 3458, 3553, 3554, 3649, 3650, 3841,
+ 3842, 3937, 3938, 4033, 4034, 4129, 4225, 4226, 4321, 4322, 4417,
+ 4418, 4608 },
+ { 1, 50, 99, 148, 197, 246, 295, 393, 442, 491, 540, 589, 638, 687, 785,
+ 834, 883, 932, 981, 1030, 1079, 1177, 1226, 1275, 1324, 1373, 1422,
+ 1471, 1569, 1570, 1667, 1668, 1765, 1766, 1863, 1864, 1961, 1962,
+ 2059, 2060, 2157, 2158, 2353, 2354, 2451, 2452, 2549, 2550, 2647,
+ 2648, 2745, 2746, 2843, 2844, 2941, 2942, 3137, 3138, 3235, 3236,
+ 3333, 3334, 3431, 3432, 3529, 3530, 3627, 3628, 3725, 3726, 3921,
+ 3922, 4019, 4020, 4117, 4118, 4215, 4313, 4314, 4411, 4412, 4509,
+ 4510, 4704 },
+ { 1, 51, 101, 151, 201, 251, 301, 401, 451, 501, 551, 601, 651, 701,
+ 801, 851, 901, 951, 1001, 1051, 1101, 1201, 1251, 1301, 1351, 1401,
+ 1451, 1501, 1601, 1602, 1701, 1702, 1801, 1802, 1901, 1902, 2001,
+ 2002, 2101, 2102, 2201, 2202, 2401, 2402, 2501, 2502, 2601, 2602,
+ 2701, 2702, 2801, 2802, 2901, 2902, 3001, 3002, 3201, 3202, 3301,
+ 3302, 3401, 3402, 3501, 3502, 3601, 3602, 3701, 3702, 3801, 3802,
+ 4001, 4002, 4101, 4102, 4201, 4202, 4301, 4401, 4402, 4501, 4502,
+ 4601, 4602, 4800 },
+ { 1, 52, 103, 154, 205, 256, 307, 409, 460, 511, 562, 613, 664, 715,
+ 817, 868, 919, 970, 1021, 1072, 1123, 1225, 1276, 1327, 1378, 1429,
+ 1480, 1531, 1633, 1634, 1735, 1736, 1837, 1838, 1939, 1940, 2041,
+ 2042, 2143, 2144, 2245, 2246, 2449, 2450, 2551, 2552, 2653, 2654,
+ 2755, 2756, 2857, 2858, 2959, 2960, 3061, 3062, 3265, 3266, 3367,
+ 3368, 3469, 3470, 3571, 3572, 3673, 3674, 3775, 3776, 3877, 3878,
+ 4081, 4082, 4183, 4184, 4285, 4286, 4387, 4489, 4490, 4591, 4592,
+ 4693, 4694, 4896 },
+ { 1, 53, 105, 157, 209, 261, 313, 417, 469, 521, 573, 625, 677, 729,
+ 833, 885, 937, 989, 1041, 1093, 1145, 1249, 1301, 1353, 1405, 1457,
+ 1509, 1561, 1665, 1666, 1769, 1770, 1873, 1874, 1977, 1978, 2081,
+ 2082, 2185, 2186, 2289, 2290, 2497, 2498, 2601, 2602, 2705, 2706,
+ 2809, 2810, 2913, 2914, 3017, 3018, 3121, 3122, 3329, 3330, 3433,
+ 3434, 3537, 3538, 3641, 3642, 3745, 3746, 3849, 3850, 3953, 3954,
+ 4161, 4162, 4265, 4266, 4369, 4370, 4473, 4577, 4578, 4681, 4682,
+ 4785, 4786, 4992 },
+ { 1, 54, 107, 160, 213, 266, 319, 425, 478, 531, 584, 637, 690, 743,
+ 849, 902, 955, 1008, 1061, 1114, 1167, 1273, 1326, 1379, 1432, 1485,
+ 1538, 1591, 1697, 1698, 1803, 1804, 1909, 1910, 2015, 2016, 2121,
+ 2122, 2227, 2228, 2333, 2334, 2545, 2546, 2651, 2652, 2757, 2758,
+ 2863, 2864, 2969, 2970, 3075, 3076, 3181, 3182, 3393, 3394, 3499,
+ 3500, 3605, 3606, 3711, 3712, 3817, 3818, 3923, 3924, 4029, 4030,
+ 4241, 4242, 4347, 4348, 4453, 4454, 4559, 4665, 4666, 4771, 4772,
+ 4877, 4878, 5088 },
+ { 1, 55, 109, 163, 217, 271, 325, 433, 487, 541, 595, 649, 703, 757,
+ 865, 919, 973, 1027, 1081, 1135, 1189, 1297, 1351, 1405, 1459, 1513,
+ 1567, 1621, 1729, 1730, 1837, 1838, 1945, 1946, 2053, 2054, 2161,
+ 2162, 2269, 2270, 2377, 2378, 2593, 2594, 2701, 2702, 2809, 2810,
+ 2917, 2918, 3025, 3026, 3133, 3134, 3241, 3242, 3457, 3458, 3565,
+ 3566, 3673, 3674, 3781, 3782, 3889, 3890, 3997, 3998, 4105, 4106,
+ 4321, 4322, 4429, 4430, 4537, 4538, 4645, 4753, 4754, 4861, 4862,
+ 4969, 4970, 5184 },
+ { 1, 56, 111, 166, 221, 276, 331, 441, 496, 551, 606, 661, 716, 771,
+ 881, 936, 991, 1046, 1101, 1156, 1211, 1321, 1376, 1431, 1486, 1541,
+ 1596, 1651, 1761, 1762, 1871, 1872, 1981, 1982, 2091, 2092, 2201,
+ 2202, 2311, 2312, 2421, 2422, 2641, 2642, 2751, 2752, 2861, 2862,
+ 2971, 2972, 3081, 3082, 3191, 3192, 3301, 3302, 3521, 3522, 3631,
+ 3632, 3741, 3742, 3851, 3852, 3961, 3962, 4071, 4072, 4181, 4182,
+ 4401, 4402, 4511, 4512, 4621, 4622, 4731, 4841, 4842, 4951, 4952,
+ 5061, 5062, 5280 },
+ { 1, 57, 113, 169, 225, 281, 337, 449, 505, 561, 617, 673, 729, 785,
+ 897, 953, 1009, 1065, 1121, 1177, 1233, 1345, 1401, 1457, 1513, 1569,
+ 1625, 1681, 1793, 1794, 1905, 1906, 2017, 2018, 2129, 2130, 2241,
+ 2242, 2353, 2354, 2465, 2466, 2689, 2690, 2801, 2802, 2913, 2914,
+ 3025, 3026, 3137, 3138, 3249, 3250, 3361, 3362, 3585, 3586, 3697,
+ 3698, 3809, 3810, 3921, 3922, 4033, 4034, 4145, 4146, 4257, 4258,
+ 4481, 4482, 4593, 4594, 4705, 4706, 4817, 4929, 4930, 5041, 5042,
+ 5153, 5154, 5376 },
+ { 1, 58, 115, 172, 229, 286, 343, 457, 514, 571, 628, 685, 742, 799,
+ 913, 970, 1027, 1084, 1141, 1198, 1255, 1369, 1426, 1483, 1540, 1597,
+ 1654, 1711, 1825, 1826, 1939, 1940, 2053, 2054, 2167, 2168, 2281,
+ 2282, 2395, 2396, 2509, 2510, 2737, 2738, 2851, 2852, 2965, 2966,
+ 3079, 3080, 3193, 3194, 3307, 3308, 3421, 3422, 3649, 3650, 3763,
+ 3764, 3877, 3878, 3991, 3992, 4105, 4106, 4219, 4220, 4333, 4334,
+ 4561, 4562, 4675, 4676, 4789, 4790, 4903, 5017, 5018, 5131, 5132,
+ 5245, 5246, 5472 },
+ { 1, 59, 117, 175, 233, 291, 349, 465, 523, 581, 639, 697, 755, 813,
+ 929, 987, 1045, 1103, 1161, 1219, 1277, 1393, 1451, 1509, 1567, 1625,
+ 1683, 1741, 1857, 1858, 1973, 1974, 2089, 2090, 2205, 2206, 2321,
+ 2322, 2437, 2438, 2553, 2554, 2785, 2786, 2901, 2902, 3017, 3018,
+ 3133, 3134, 3249, 3250, 3365, 3366, 3481, 3482, 3713, 3714, 3829,
+ 3830, 3945, 3946, 4061, 4062, 4177, 4178, 4293, 4294, 4409, 4410,
+ 4641, 4642, 4757, 4758, 4873, 4874, 4989, 5105, 5106, 5221, 5222,
+ 5337, 5338, 5568 },
+ { 1, 60, 119, 178, 237, 296, 355, 473, 532, 591, 650, 709, 768, 827,
+ 945, 1004, 1063, 1122, 1181, 1240, 1299, 1417, 1476, 1535, 1594, 1653,
+ 1712, 1771, 1889, 1890, 2007, 2008, 2125, 2126, 2243, 2244, 2361,
+ 2362, 2479, 2480, 2597, 2598, 2833, 2834, 2951, 2952, 3069, 3070,
+ 3187, 3188, 3305, 3306, 3423, 3424, 3541, 3542, 3777, 3778, 3895,
+ 3896, 4013, 4014, 4131, 4132, 4249, 4250, 4367, 4368, 4485, 4486,
+ 4721, 4722, 4839, 4840, 4957, 4958, 5075, 5193, 5194, 5311, 5312,
+ 5429, 5430, 5664 },
+ { 1, 61, 121, 181, 241, 301, 361, 481, 541, 601, 661, 721, 781, 841,
+ 961, 1021, 1081, 1141, 1201, 1261, 1321, 1441, 1501, 1561, 1621, 1681,
+ 1741, 1801, 1921, 1922, 2041, 2042, 2161, 2162, 2281, 2282, 2401,
+ 2402, 2521, 2522, 2641, 2642, 2881, 2882, 3001, 3002, 3121, 3122,
+ 3241, 3242, 3361, 3362, 3481, 3482, 3601, 3602, 3841, 3842, 3961,
+ 3962, 4081, 4082, 4201, 4202, 4321, 4322, 4441, 4442, 4561, 4562,
+ 4801, 4802, 4921, 4922, 5041, 5042, 5161, 5281, 5282, 5401, 5402,
+ 5521, 5522, 5760 },
+ { 1, 62, 123, 184, 245, 306, 367, 489, 550, 611, 672, 733, 794, 855,
+ 977, 1038, 1099, 1160, 1221, 1282, 1343, 1465, 1526, 1587, 1648, 1709,
+ 1770, 1831, 1953, 1954, 2075, 2076, 2197, 2198, 2319, 2320, 2441,
+ 2442, 2563, 2564, 2685, 2686, 2929, 2930, 3051, 3052, 3173, 3174,
+ 3295, 3296, 3417, 3418, 3539, 3540, 3661, 3662, 3905, 3906, 4027,
+ 4028, 4149, 4150, 4271, 4272, 4393, 4394, 4515, 4516, 4637, 4638,
+ 4881, 4882, 5003, 5004, 5125, 5126, 5247, 5369, 5370, 5491, 5492,
+ 5613, 5614, 5856 },
+ { 1, 63, 125, 187, 249, 311, 373, 497, 559, 621, 683, 745, 807, 869,
+ 993, 1055, 1117, 1179, 1241, 1303, 1365, 1489, 1551, 1613, 1675, 1737,
+ 1799, 1861, 1985, 1986, 2109, 2110, 2233, 2234, 2357, 2358, 2481,
+ 2482, 2605, 2606, 2729, 2730, 2977, 2978, 3101, 3102, 3225, 3226,
+ 3349, 3350, 3473, 3474, 3597, 3598, 3721, 3722, 3969, 3970, 4093,
+ 4094, 4217, 4218, 4341, 4342, 4465, 4466, 4589, 4590, 4713, 4714,
+ 4961, 4962, 5085, 5086, 5209, 5210, 5333, 5457, 5458, 5581, 5582,
+ 5705, 5706, 5952 },
+ { 1, 64, 127, 190, 253, 316, 379, 505, 568, 631, 694, 757, 820, 883,
+ 1009, 1072, 1135, 1198, 1261, 1324, 1387, 1513, 1576, 1639, 1702,
+ 1765, 1828, 1891, 2017, 2018, 2143, 2144, 2269, 2270, 2395, 2396,
+ 2521, 2522, 2647, 2648, 2773, 2774, 3025, 3026, 3151, 3152, 3277,
+ 3278, 3403, 3404, 3529, 3530, 3655, 3656, 3781, 3782, 4033, 4034,
+ 4159, 4160, 4285, 4286, 4411, 4412, 4537, 4538, 4663, 4664, 4789,
+ 4790, 5041, 5042, 5167, 5168, 5293, 5294, 5419, 5545, 5546, 5671,
+ 5672, 5797, 5798, 6048 },
+ { 1, 65, 129, 193, 257, 321, 385, 513, 577, 641, 705, 769, 833, 897,
+ 1025, 1089, 1153, 1217, 1281, 1345, 1409, 1537, 1601, 1665, 1729,
+ 1793, 1857, 1921, 2049, 2050, 2177, 2178, 2305, 2306, 2433, 2434,
+ 2561, 2562, 2689, 2690, 2817, 2818, 3073, 3074, 3201, 3202, 3329,
+ 3330, 3457, 3458, 3585, 3586, 3713, 3714, 3841, 3842, 4097, 4098,
+ 4225, 4226, 4353, 4354, 4481, 4482, 4609, 4610, 4737, 4738, 4865,
+ 4866, 5121, 5122, 5249, 5250, 5377, 5378, 5505, 5633, 5634, 5761,
+ 5762, 5889, 5890, 6144 },
+ { 1, 66, 131, 196, 261, 326, 391, 521, 586, 651, 716, 781, 846, 911,
+ 1041, 1106, 1171, 1236, 1301, 1366, 1431, 1561, 1626, 1691, 1756,
+ 1821, 1886, 1951, 2081, 2082, 2211, 2212, 2341, 2342, 2471, 2472,
+ 2601, 2602, 2731, 2732, 2861, 2862, 3121, 3122, 3251, 3252, 3381,
+ 3382, 3511, 3512, 3641, 3642, 3771, 3772, 3901, 3902, 4161, 4162,
+ 4291, 4292, 4421, 4422, 4551, 4552, 4681, 4682, 4811, 4812, 4941,
+ 4942, 5201, 5202, 5331, 5332, 5461, 5462, 5591, 5721, 5722, 5851,
+ 5852, 5981, 5982, 6240 },
+ { 1, 68, 135, 202, 269, 336, 403, 537, 604, 671, 738, 805, 872, 939,
+ 1073, 1140, 1207, 1274, 1341, 1408, 1475, 1609, 1676, 1743, 1810,
+ 1877, 1944, 2011, 2145, 2146, 2279, 2280, 2413, 2414, 2547, 2548,
+ 2681, 2682, 2815, 2816, 2949, 2950, 3217, 3218, 3351, 3352, 3485,
+ 3486, 3619, 3620, 3753, 3754, 3887, 3888, 4021, 4022, 4289, 4290,
+ 4423, 4424, 4557, 4558, 4691, 4692, 4825, 4826, 4959, 4960, 5093,
+ 5094, 5361, 5362, 5495, 5496, 5629, 5630, 5763, 5897, 5898, 6031,
+ 6032, 6165, 6166, 6432 },
+ { 1, 70, 139, 208, 277, 346, 415, 553, 622, 691, 760, 829, 898, 967,
+ 1105, 1174, 1243, 1312, 1381, 1450, 1519, 1657, 1726, 1795, 1864,
+ 1933, 2002, 2071, 2209, 2210, 2347, 2348, 2485, 2486, 2623, 2624,
+ 2761, 2762, 2899, 2900, 3037, 3038, 3313, 3314, 3451, 3452, 3589,
+ 3590, 3727, 3728, 3865, 3866, 4003, 4004, 4141, 4142, 4417, 4418,
+ 4555, 4556, 4693, 4694, 4831, 4832, 4969, 4970, 5107, 5108, 5245,
+ 5246, 5521, 5522, 5659, 5660, 5797, 5798, 5935, 6073, 6074, 6211,
+ 6212, 6349, 6350, 6624 },
+ { 1, 72, 143, 214, 285, 356, 427, 569, 640, 711, 782, 853, 924, 995,
+ 1137, 1208, 1279, 1350, 1421, 1492, 1563, 1705, 1776, 1847, 1918,
+ 1989, 2060, 2131, 2273, 2274, 2415, 2416, 2557, 2558, 2699, 2700,
+ 2841, 2842, 2983, 2984, 3125, 3126, 3409, 3410, 3551, 3552, 3693,
+ 3694, 3835, 3836, 3977, 3978, 4119, 4120, 4261, 4262, 4545, 4546,
+ 4687, 4688, 4829, 4830, 4971, 4972, 5113, 5114, 5255, 5256, 5397,
+ 5398, 5681, 5682, 5823, 5824, 5965, 5966, 6107, 6249, 6250, 6391,
+ 6392, 6533, 6534, 6816 },
+ { 1, 74, 147, 220, 293, 366, 439, 585, 658, 731, 804, 877, 950, 1023,
+ 1169, 1242, 1315, 1388, 1461, 1534, 1607, 1753, 1826, 1899, 1972,
+ 2045, 2118, 2191, 2337, 2338, 2483, 2484, 2629, 2630, 2775, 2776,
+ 2921, 2922, 3067, 3068, 3213, 3214, 3505, 3506, 3651, 3652, 3797,
+ 3798, 3943, 3944, 4089, 4090, 4235, 4236, 4381, 4382, 4673, 4674,
+ 4819, 4820, 4965, 4966, 5111, 5112, 5257, 5258, 5403, 5404, 5549,
+ 5550, 5841, 5842, 5987, 5988, 6133, 6134, 6279, 6425, 6426, 6571,
+ 6572, 6717, 6718, 7008 },
+ { 1, 76, 151, 226, 301, 376, 451, 601, 676, 751, 826, 901, 976, 1051,
+ 1201, 1276, 1351, 1426, 1501, 1576, 1651, 1801, 1876, 1951, 2026,
+ 2101, 2176, 2251, 2401, 2402, 2551, 2552, 2701, 2702, 2851, 2852,
+ 3001, 3002, 3151, 3152, 3301, 3302, 3601, 3602, 3751, 3752, 3901,
+ 3902, 4051, 4052, 4201, 4202, 4351, 4352, 4501, 4502, 4801, 4802,
+ 4951, 4952, 5101, 5102, 5251, 5252, 5401, 5402, 5551, 5552, 5701,
+ 5702, 6001, 6002, 6151, 6152, 6301, 6302, 6451, 6601, 6602, 6751,
+ 6752, 6901, 6902, 7200 },
+ { 1, 78, 155, 232, 309, 386, 463, 617, 694, 771, 848, 925, 1002, 1079,
+ 1233, 1310, 1387, 1464, 1541, 1618, 1695, 1849, 1926, 2003, 2080,
+ 2157, 2234, 2311, 2465, 2466, 2619, 2620, 2773, 2774, 2927, 2928,
+ 3081, 3082, 3235, 3236, 3389, 3390, 3697, 3698, 3851, 3852, 4005,
+ 4006, 4159, 4160, 4313, 4314, 4467, 4468, 4621, 4622, 4929, 4930,
+ 5083, 5084, 5237, 5238, 5391, 5392, 5545, 5546, 5699, 5700, 5853,
+ 5854, 6161, 6162, 6315, 6316, 6469, 6470, 6623, 6777, 6778, 6931,
+ 6932, 7085, 7086, 7392 },
+ { 1, 80, 159, 238, 317, 396, 475, 633, 712, 791, 870, 949, 1028, 1107,
+ 1265, 1344, 1423, 1502, 1581, 1660, 1739, 1897, 1976, 2055, 2134,
+ 2213, 2292, 2371, 2529, 2530, 2687, 2688, 2845, 2846, 3003, 3004,
+ 3161, 3162, 3319, 3320, 3477, 3478, 3793, 3794, 3951, 3952, 4109,
+ 4110, 4267, 4268, 4425, 4426, 4583, 4584, 4741, 4742, 5057, 5058,
+ 5215, 5216, 5373, 5374, 5531, 5532, 5689, 5690, 5847, 5848, 6005,
+ 6006, 6321, 6322, 6479, 6480, 6637, 6638, 6795, 6953, 6954, 7111,
+ 7112, 7269, 7270, 7584 },
+ { 1, 82, 163, 244, 325, 406, 487, 649, 730, 811, 892, 973, 1054, 1135,
+ 1297, 1378, 1459, 1540, 1621, 1702, 1783, 1945, 2026, 2107, 2188,
+ 2269, 2350, 2431, 2593, 2594, 2755, 2756, 2917, 2918, 3079, 3080,
+ 3241, 3242, 3403, 3404, 3565, 3566, 3889, 3890, 4051, 4052, 4213,
+ 4214, 4375, 4376, 4537, 4538, 4699, 4700, 4861, 4862, 5185, 5186,
+ 5347, 5348, 5509, 5510, 5671, 5672, 5833, 5834, 5995, 5996, 6157,
+ 6158, 6481, 6482, 6643, 6644, 6805, 6806, 6967, 7129, 7130, 7291,
+ 7292, 7453, 7454, 7776 },
+ { 1, 84, 167, 250, 333, 416, 499, 665, 748, 831, 914, 997, 1080, 1163,
+ 1329, 1412, 1495, 1578, 1661, 1744, 1827, 1993, 2076, 2159, 2242,
+ 2325, 2408, 2491, 2657, 2658, 2823, 2824, 2989, 2990, 3155, 3156,
+ 3321, 3322, 3487, 3488, 3653, 3654, 3985, 3986, 4151, 4152, 4317,
+ 4318, 4483, 4484, 4649, 4650, 4815, 4816, 4981, 4982, 5313, 5314,
+ 5479, 5480, 5645, 5646, 5811, 5812, 5977, 5978, 6143, 6144, 6309,
+ 6310, 6641, 6642, 6807, 6808, 6973, 6974, 7139, 7305, 7306, 7471,
+ 7472, 7637, 7638, 7968 },
+ { 1, 86, 171, 256, 341, 426, 511, 681, 766, 851, 936, 1021, 1106, 1191,
+ 1361, 1446, 1531, 1616, 1701, 1786, 1871, 2041, 2126, 2211, 2296,
+ 2381, 2466, 2551, 2721, 2722, 2891, 2892, 3061, 3062, 3231, 3232,
+ 3401, 3402, 3571, 3572, 3741, 3742, 4081, 4082, 4251, 4252, 4421,
+ 4422, 4591, 4592, 4761, 4762, 4931, 4932, 5101, 5102, 5441, 5442,
+ 5611, 5612, 5781, 5782, 5951, 5952, 6121, 6122, 6291, 6292, 6461,
+ 6462, 6801, 6802, 6971, 6972, 7141, 7142, 7311, 7481, 7482, 7651,
+ 7652, 7821, 7822, 8160 },
+ { 1, 88, 175, 262, 349, 436, 523, 697, 784, 871, 958, 1045, 1132, 1219,
+ 1393, 1480, 1567, 1654, 1741, 1828, 1915, 2089, 2176, 2263, 2350,
+ 2437, 2524, 2611, 2785, 2786, 2959, 2960, 3133, 3134, 3307, 3308,
+ 3481, 3482, 3655, 3656, 3829, 3830, 4177, 4178, 4351, 4352, 4525,
+ 4526, 4699, 4700, 4873, 4874, 5047, 5048, 5221, 5222, 5569, 5570,
+ 5743, 5744, 5917, 5918, 6091, 6092, 6265, 6266, 6439, 6440, 6613,
+ 6614, 6961, 6962, 7135, 7136, 7309, 7310, 7483, 7657, 7658, 7831,
+ 7832, 8005, 8006, 8352 },
+ { 1, 90, 179, 268, 357, 446, 535, 713, 802, 891, 980, 1069, 1158, 1247,
+ 1425, 1514, 1603, 1692, 1781, 1870, 1959, 2137, 2226, 2315, 2404,
+ 2493, 2582, 2671, 2849, 2850, 3027, 3028, 3205, 3206, 3383, 3384,
+ 3561, 3562, 3739, 3740, 3917, 3918, 4273, 4274, 4451, 4452, 4629,
+ 4630, 4807, 4808, 4985, 4986, 5163, 5164, 5341, 5342, 5697, 5698,
+ 5875, 5876, 6053, 6054, 6231, 6232, 6409, 6410, 6587, 6588, 6765,
+ 6766, 7121, 7122, 7299, 7300, 7477, 7478, 7655, 7833, 7834, 8011,
+ 8012, 8189, 8190, 8544 },
+ { 1, 92, 183, 274, 365, 456, 547, 729, 820, 911, 1002, 1093, 1184, 1275,
+ 1457, 1548, 1639, 1730, 1821, 1912, 2003, 2185, 2276, 2367, 2458,
+ 2549, 2640, 2731, 2913, 2914, 3095, 3096, 3277, 3278, 3459, 3460,
+ 3641, 3642, 3823, 3824, 4005, 4006, 4369, 4370, 4551, 4552, 4733,
+ 4734, 4915, 4916, 5097, 5098, 5279, 5280, 5461, 5462, 5825, 5826,
+ 6007, 6008, 6189, 6190, 6371, 6372, 6553, 6554, 6735, 6736, 6917,
+ 6918, 7281, 7282, 7463, 7464, 7645, 7646, 7827, 8009, 8010, 8191,
+ 8192, 8373, 8374, 8736 },
+ { 1, 94, 187, 280, 373, 466, 559, 745, 838, 931, 1024, 1117, 1210, 1303,
+ 1489, 1582, 1675, 1768, 1861, 1954, 2047, 2233, 2326, 2419, 2512,
+ 2605, 2698, 2791, 2977, 2978, 3163, 3164, 3349, 3350, 3535, 3536,
+ 3721, 3722, 3907, 3908, 4093, 4094, 4465, 4466, 4651, 4652, 4837,
+ 4838, 5023, 5024, 5209, 5210, 5395, 5396, 5581, 5582, 5953, 5954,
+ 6139, 6140, 6325, 6326, 6511, 6512, 6697, 6698, 6883, 6884, 7069,
+ 7070, 7441, 7442, 7627, 7628, 7813, 7814, 7999, 8185, 8186, 8371,
+ 8372, 8557, 8558, 8928 },
+ { 1, 96, 191, 286, 381, 476, 571, 761, 856, 951, 1046, 1141, 1236, 1331,
+ 1521, 1616, 1711, 1806, 1901, 1996, 2091, 2281, 2376, 2471, 2566,
+ 2661, 2756, 2851, 3041, 3042, 3231, 3232, 3421, 3422, 3611, 3612,
+ 3801, 3802, 3991, 3992, 4181, 4182, 4561, 4562, 4751, 4752, 4941,
+ 4942, 5131, 5132, 5321, 5322, 5511, 5512, 5701, 5702, 6081, 6082,
+ 6271, 6272, 6461, 6462, 6651, 6652, 6841, 6842, 7031, 7032, 7221,
+ 7222, 7601, 7602, 7791, 7792, 7981, 7982, 8171, 8361, 8362, 8551,
+ 8552, 8741, 8742, 9120 },
+ { 1, 98, 195, 292, 389, 486, 583, 777, 874, 971, 1068, 1165, 1262, 1359,
+ 1553, 1650, 1747, 1844, 1941, 2038, 2135, 2329, 2426, 2523, 2620,
+ 2717, 2814, 2911, 3105, 3106, 3299, 3300, 3493, 3494, 3687, 3688,
+ 3881, 3882, 4075, 4076, 4269, 4270, 4657, 4658, 4851, 4852, 5045,
+ 5046, 5239, 5240, 5433, 5434, 5627, 5628, 5821, 5822, 6209, 6210,
+ 6403, 6404, 6597, 6598, 6791, 6792, 6985, 6986, 7179, 7180, 7373,
+ 7374, 7761, 7762, 7955, 7956, 8149, 8150, 8343, 8537, 8538, 8731,
+ 8732, 8925, 8926, 9312 },
+ { 1, 100, 199, 298, 397, 496, 595, 793, 892, 991, 1090, 1189, 1288,
+ 1387, 1585, 1684, 1783, 1882, 1981, 2080, 2179, 2377, 2476, 2575,
+ 2674, 2773, 2872, 2971, 3169, 3170, 3367, 3368, 3565, 3566, 3763,
+ 3764, 3961, 3962, 4159, 4160, 4357, 4358, 4753, 4754, 4951, 4952,
+ 5149, 5150, 5347, 5348, 5545, 5546, 5743, 5744, 5941, 5942, 6337,
+ 6338, 6535, 6536, 6733, 6734, 6931, 6932, 7129, 7130, 7327, 7328,
+ 7525, 7526, 7921, 7922, 8119, 8120, 8317, 8318, 8515, 8713, 8714,
+ 8911, 8912, 9109, 9110, 9504 },
+ { 1, 102, 203, 304, 405, 506, 607, 809, 910, 1011, 1112, 1213, 1314,
+ 1415, 1617, 1718, 1819, 1920, 2021, 2122, 2223, 2425, 2526, 2627,
+ 2728, 2829, 2930, 3031, 3233, 3234, 3435, 3436, 3637, 3638, 3839,
+ 3840, 4041, 4042, 4243, 4244, 4445, 4446, 4849, 4850, 5051, 5052,
+ 5253, 5254, 5455, 5456, 5657, 5658, 5859, 5860, 6061, 6062, 6465,
+ 6466, 6667, 6668, 6869, 6870, 7071, 7072, 7273, 7274, 7475, 7476,
+ 7677, 7678, 8081, 8082, 8283, 8284, 8485, 8486, 8687, 8889, 8890,
+ 9091, 9092, 9293, 9294, 9696 },
+ { 1, 104, 207, 310, 413, 516, 619, 825, 928, 1031, 1134, 1237, 1340,
+ 1443, 1649, 1752, 1855, 1958, 2061, 2164, 2267, 2473, 2576, 2679,
+ 2782, 2885, 2988, 3091, 3297, 3298, 3503, 3504, 3709, 3710, 3915,
+ 3916, 4121, 4122, 4327, 4328, 4533, 4534, 4945, 4946, 5151, 5152,
+ 5357, 5358, 5563, 5564, 5769, 5770, 5975, 5976, 6181, 6182, 6593,
+ 6594, 6799, 6800, 7005, 7006, 7211, 7212, 7417, 7418, 7623, 7624,
+ 7829, 7830, 8241, 8242, 8447, 8448, 8653, 8654, 8859, 9065, 9066,
+ 9271, 9272, 9477, 9478, 9888 },
+ { 1, 106, 211, 316, 421, 526, 631, 841, 946, 1051, 1156, 1261, 1366,
+ 1471, 1681, 1786, 1891, 1996, 2101, 2206, 2311, 2521, 2626, 2731,
+ 2836, 2941, 3046, 3151, 3361, 3362, 3571, 3572, 3781, 3782, 3991,
+ 3992, 4201, 4202, 4411, 4412, 4621, 4622, 5041, 5042, 5251, 5252,
+ 5461, 5462, 5671, 5672, 5881, 5882, 6091, 6092, 6301, 6302, 6721,
+ 6722, 6931, 6932, 7141, 7142, 7351, 7352, 7561, 7562, 7771, 7772,
+ 7981, 7982, 8401, 8402, 8611, 8612, 8821, 8822, 9031, 9241, 9242,
+ 9451, 9452, 9661, 9662, 10080 },
+ { 1, 108, 215, 322, 429, 536, 643, 857, 964, 1071, 1178, 1285, 1392,
+ 1499, 1713, 1820, 1927, 2034, 2141, 2248, 2355, 2569, 2676, 2783,
+ 2890, 2997, 3104, 3211, 3425, 3426, 3639, 3640, 3853, 3854, 4067,
+ 4068, 4281, 4282, 4495, 4496, 4709, 4710, 5137, 5138, 5351, 5352,
+ 5565, 5566, 5779, 5780, 5993, 5994, 6207, 6208, 6421, 6422, 6849,
+ 6850, 7063, 7064, 7277, 7278, 7491, 7492, 7705, 7706, 7919, 7920,
+ 8133, 8134, 8561, 8562, 8775, 8776, 8989, 8990, 9203, 9417, 9418,
+ 9631, 9632, 9845, 9846, 10272 },
+ { 1, 110, 219, 328, 437, 546, 655, 873, 982, 1091, 1200, 1309, 1418,
+ 1527, 1745, 1854, 1963, 2072, 2181, 2290, 2399, 2617, 2726, 2835,
+ 2944, 3053, 3162, 3271, 3489, 3490, 3707, 3708, 3925, 3926, 4143,
+ 4144, 4361, 4362, 4579, 4580, 4797, 4798, 5233, 5234, 5451, 5452,
+ 5669, 5670, 5887, 5888, 6105, 6106, 6323, 6324, 6541, 6542, 6977,
+ 6978, 7195, 7196, 7413, 7414, 7631, 7632, 7849, 7850, 8067, 8068,
+ 8285, 8286, 8721, 8722, 8939, 8940, 9157, 9158, 9375, 9593, 9594,
+ 9811, 9812, 10029, 10030, 10464 },
+ { 1, 112, 223, 334, 445, 556, 667, 889, 1000, 1111, 1222, 1333, 1444,
+ 1555, 1777, 1888, 1999, 2110, 2221, 2332, 2443, 2665, 2776, 2887,
+ 2998, 3109, 3220, 3331, 3553, 3554, 3775, 3776, 3997, 3998, 4219,
+ 4220, 4441, 4442, 4663, 4664, 4885, 4886, 5329, 5330, 5551, 5552,
+ 5773, 5774, 5995, 5996, 6217, 6218, 6439, 6440, 6661, 6662, 7105,
+ 7106, 7327, 7328, 7549, 7550, 7771, 7772, 7993, 7994, 8215, 8216,
+ 8437, 8438, 8881, 8882, 9103, 9104, 9325, 9326, 9547, 9769, 9770,
+ 9991, 9992, 10213, 10214, 10656 },
+ { 1, 114, 227, 340, 453, 566, 679, 905, 1018, 1131, 1244, 1357, 1470,
+ 1583, 1809, 1922, 2035, 2148, 2261, 2374, 2487, 2713, 2826, 2939,
+ 3052, 3165, 3278, 3391, 3617, 3618, 3843, 3844, 4069, 4070, 4295,
+ 4296, 4521, 4522, 4747, 4748, 4973, 4974, 5425, 5426, 5651, 5652,
+ 5877, 5878, 6103, 6104, 6329, 6330, 6555, 6556, 6781, 6782, 7233,
+ 7234, 7459, 7460, 7685, 7686, 7911, 7912, 8137, 8138, 8363, 8364,
+ 8589, 8590, 9041, 9042, 9267, 9268, 9493, 9494, 9719, 9945, 9946,
+ 10171, 10172, 10397, 10398, 10848 },
+ { 1, 116, 231, 346, 461, 576, 691, 921, 1036, 1151, 1266, 1381, 1496,
+ 1611, 1841, 1956, 2071, 2186, 2301, 2416, 2531, 2761, 2876, 2991,
+ 3106, 3221, 3336, 3451, 3681, 3682, 3911, 3912, 4141, 4142, 4371,
+ 4372, 4601, 4602, 4831, 4832, 5061, 5062, 5521, 5522, 5751, 5752,
+ 5981, 5982, 6211, 6212, 6441, 6442, 6671, 6672, 6901, 6902, 7361,
+ 7362, 7591, 7592, 7821, 7822, 8051, 8052, 8281, 8282, 8511, 8512,
+ 8741, 8742, 9201, 9202, 9431, 9432, 9661, 9662, 9891, 10121, 10122,
+ 10351, 10352, 10581, 10582, 11040 },
+ { 1, 118, 235, 352, 469, 586, 703, 937, 1054, 1171, 1288, 1405, 1522,
+ 1639, 1873, 1990, 2107, 2224, 2341, 2458, 2575, 2809, 2926, 3043,
+ 3160, 3277, 3394, 3511, 3745, 3746, 3979, 3980, 4213, 4214, 4447,
+ 4448, 4681, 4682, 4915, 4916, 5149, 5150, 5617, 5618, 5851, 5852,
+ 6085, 6086, 6319, 6320, 6553, 6554, 6787, 6788, 7021, 7022, 7489,
+ 7490, 7723, 7724, 7957, 7958, 8191, 8192, 8425, 8426, 8659, 8660,
+ 8893, 8894, 9361, 9362, 9595, 9596, 9829, 9830, 10063, 10297, 10298,
+ 10531, 10532, 10765, 10766, 11232 },
+ { 1, 120, 239, 358, 477, 596, 715, 953, 1072, 1191, 1310, 1429, 1548,
+ 1667, 1905, 2024, 2143, 2262, 2381, 2500, 2619, 2857, 2976, 3095,
+ 3214, 3333, 3452, 3571, 3809, 3810, 4047, 4048, 4285, 4286, 4523,
+ 4524, 4761, 4762, 4999, 5000, 5237, 5238, 5713, 5714, 5951, 5952,
+ 6189, 6190, 6427, 6428, 6665, 6666, 6903, 6904, 7141, 7142, 7617,
+ 7618, 7855, 7856, 8093, 8094, 8331, 8332, 8569, 8570, 8807, 8808,
+ 9045, 9046, 9521, 9522, 9759, 9760, 9997, 9998, 10235, 10473, 10474,
+ 10711, 10712, 10949, 10950, 11424 },
+ { 1, 122, 243, 364, 485, 606, 727, 969, 1090, 1211, 1332, 1453, 1574,
+ 1695, 1937, 2058, 2179, 2300, 2421, 2542, 2663, 2905, 3026, 3147,
+ 3268, 3389, 3510, 3631, 3873, 3874, 4115, 4116, 4357, 4358, 4599,
+ 4600, 4841, 4842, 5083, 5084, 5325, 5326, 5809, 5810, 6051, 6052,
+ 6293, 6294, 6535, 6536, 6777, 6778, 7019, 7020, 7261, 7262, 7745,
+ 7746, 7987, 7988, 8229, 8230, 8471, 8472, 8713, 8714, 8955, 8956,
+ 9197, 9198, 9681, 9682, 9923, 9924, 10165, 10166, 10407, 10649, 10650,
+ 10891, 10892, 11133, 11134, 11616 },
+ { 1, 124, 247, 370, 493, 616, 739, 985, 1108, 1231, 1354, 1477, 1600,
+ 1723, 1969, 2092, 2215, 2338, 2461, 2584, 2707, 2953, 3076, 3199,
+ 3322, 3445, 3568, 3691, 3937, 3938, 4183, 4184, 4429, 4430, 4675,
+ 4676, 4921, 4922, 5167, 5168, 5413, 5414, 5905, 5906, 6151, 6152,
+ 6397, 6398, 6643, 6644, 6889, 6890, 7135, 7136, 7381, 7382, 7873,
+ 7874, 8119, 8120, 8365, 8366, 8611, 8612, 8857, 8858, 9103, 9104,
+ 9349, 9350, 9841, 9842, 10087, 10088, 10333, 10334, 10579, 10825,
+ 10826, 11071, 11072, 11317, 11318, 11808 },
+ { 1, 126, 251, 376, 501, 626, 751, 1001, 1126, 1251, 1376, 1501, 1626,
+ 1751, 2001, 2126, 2251, 2376, 2501, 2626, 2751, 3001, 3126, 3251,
+ 3376, 3501, 3626, 3751, 4001, 4002, 4251, 4252, 4501, 4502, 4751,
+ 4752, 5001, 5002, 5251, 5252, 5501, 5502, 6001, 6002, 6251, 6252,
+ 6501, 6502, 6751, 6752, 7001, 7002, 7251, 7252, 7501, 7502, 8001,
+ 8002, 8251, 8252, 8501, 8502, 8751, 8752, 9001, 9002, 9251, 9252,
+ 9501, 9502, 10001, 10002, 10251, 10252, 10501, 10502, 10751, 11001,
+ 11002, 11251, 11252, 11501, 11502, 12000 },
+ { 1, 128, 255, 382, 509, 636, 763, 1017, 1144, 1271, 1398, 1525, 1652,
+ 1779, 2033, 2160, 2287, 2414, 2541, 2668, 2795, 3049, 3176, 3303,
+ 3430, 3557, 3684, 3811, 4065, 4066, 4319, 4320, 4573, 4574, 4827,
+ 4828, 5081, 5082, 5335, 5336, 5589, 5590, 6097, 6098, 6351, 6352,
+ 6605, 6606, 6859, 6860, 7113, 7114, 7367, 7368, 7621, 7622, 8129,
+ 8130, 8383, 8384, 8637, 8638, 8891, 8892, 9145, 9146, 9399, 9400,
+ 9653, 9654, 10161, 10162, 10415, 10416, 10669, 10670, 10923, 11177,
+ 11178, 11431, 11432, 11685, 11686, 12192 },
+ { 1, 130, 259, 388, 517, 646, 775, 1033, 1162, 1291, 1420, 1549, 1678,
+ 1807, 2065, 2194, 2323, 2452, 2581, 2710, 2839, 3097, 3226, 3355,
+ 3484, 3613, 3742, 3871, 4129, 4130, 4387, 4388, 4645, 4646, 4903,
+ 4904, 5161, 5162, 5419, 5420, 5677, 5678, 6193, 6194, 6451, 6452,
+ 6709, 6710, 6967, 6968, 7225, 7226, 7483, 7484, 7741, 7742, 8257,
+ 8258, 8515, 8516, 8773, 8774, 9031, 9032, 9289, 9290, 9547, 9548,
+ 9805, 9806, 10321, 10322, 10579, 10580, 10837, 10838, 11095, 11353,
+ 11354, 11611, 11612, 11869, 11870, 12384 },
+ { 1, 132, 263, 394, 525, 656, 787, 1049, 1180, 1311, 1442, 1573, 1704,
+ 1835, 2097, 2228, 2359, 2490, 2621, 2752, 2883, 3145, 3276, 3407,
+ 3538, 3669, 3800, 3931, 4193, 4194, 4455, 4456, 4717, 4718, 4979,
+ 4980, 5241, 5242, 5503, 5504, 5765, 5766, 6289, 6290, 6551, 6552,
+ 6813, 6814, 7075, 7076, 7337, 7338, 7599, 7600, 7861, 7862, 8385,
+ 8386, 8647, 8648, 8909, 8910, 9171, 9172, 9433, 9434, 9695, 9696,
+ 9957, 9958, 10481, 10482, 10743, 10744, 11005, 11006, 11267, 11529,
+ 11530, 11791, 11792, 12053, 12054, 12576 },
+ { 1, 134, 267, 400, 533, 666, 799, 1065, 1198, 1331, 1464, 1597, 1730,
+ 1863, 2129, 2262, 2395, 2528, 2661, 2794, 2927, 3193, 3326, 3459,
+ 3592, 3725, 3858, 3991, 4257, 4258, 4523, 4524, 4789, 4790, 5055,
+ 5056, 5321, 5322, 5587, 5588, 5853, 5854, 6385, 6386, 6651, 6652,
+ 6917, 6918, 7183, 7184, 7449, 7450, 7715, 7716, 7981, 7982, 8513,
+ 8514, 8779, 8780, 9045, 9046, 9311, 9312, 9577, 9578, 9843, 9844,
+ 10109, 10110, 10641, 10642, 10907, 10908, 11173, 11174, 11439, 11705,
+ 11706, 11971, 11972, 12237, 12238, 12768 },
+ { 1, 136, 271, 406, 541, 676, 811, 1081, 1216, 1351, 1486, 1621, 1756,
+ 1891, 2161, 2296, 2431, 2566, 2701, 2836, 2971, 3241, 3376, 3511,
+ 3646, 3781, 3916, 4051, 4321, 4322, 4591, 4592, 4861, 4862, 5131,
+ 5132, 5401, 5402, 5671, 5672, 5941, 5942, 6481, 6482, 6751, 6752,
+ 7021, 7022, 7291, 7292, 7561, 7562, 7831, 7832, 8101, 8102, 8641,
+ 8642, 8911, 8912, 9181, 9182, 9451, 9452, 9721, 9722, 9991, 9992,
+ 10261, 10262, 10801, 10802, 11071, 11072, 11341, 11342, 11611, 11881,
+ 11882, 12151, 12152, 12421, 12422, 12960 },
+ { 1, 138, 275, 412, 549, 686, 823, 1097, 1234, 1371, 1508, 1645, 1782,
+ 1919, 2193, 2330, 2467, 2604, 2741, 2878, 3015, 3289, 3426, 3563,
+ 3700, 3837, 3974, 4111, 4385, 4386, 4659, 4660, 4933, 4934, 5207,
+ 5208, 5481, 5482, 5755, 5756, 6029, 6030, 6577, 6578, 6851, 6852,
+ 7125, 7126, 7399, 7400, 7673, 7674, 7947, 7948, 8221, 8222, 8769,
+ 8770, 9043, 9044, 9317, 9318, 9591, 9592, 9865, 9866, 10139, 10140,
+ 10413, 10414, 10961, 10962, 11235, 11236, 11509, 11510, 11783, 12057,
+ 12058, 12331, 12332, 12605, 12606, 13152 },
+ { 1, 140, 279, 418, 557, 696, 835, 1113, 1252, 1391, 1530, 1669, 1808,
+ 1947, 2225, 2364, 2503, 2642, 2781, 2920, 3059, 3337, 3476, 3615,
+ 3754, 3893, 4032, 4171, 4449, 4450, 4727, 4728, 5005, 5006, 5283,
+ 5284, 5561, 5562, 5839, 5840, 6117, 6118, 6673, 6674, 6951, 6952,
+ 7229, 7230, 7507, 7508, 7785, 7786, 8063, 8064, 8341, 8342, 8897,
+ 8898, 9175, 9176, 9453, 9454, 9731, 9732, 10009, 10010, 10287, 10288,
+ 10565, 10566, 11121, 11122, 11399, 11400, 11677, 11678, 11955, 12233,
+ 12234, 12511, 12512, 12789, 12790, 13344 },
+ { 1, 142, 283, 424, 565, 706, 847, 1129, 1270, 1411, 1552, 1693, 1834,
+ 1975, 2257, 2398, 2539, 2680, 2821, 2962, 3103, 3385, 3526, 3667,
+ 3808, 3949, 4090, 4231, 4513, 4514, 4795, 4796, 5077, 5078, 5359,
+ 5360, 5641, 5642, 5923, 5924, 6205, 6206, 6769, 6770, 7051, 7052,
+ 7333, 7334, 7615, 7616, 7897, 7898, 8179, 8180, 8461, 8462, 9025,
+ 9026, 9307, 9308, 9589, 9590, 9871, 9872, 10153, 10154, 10435, 10436,
+ 10717, 10718, 11281, 11282, 11563, 11564, 11845, 11846, 12127, 12409,
+ 12410, 12691, 12692, 12973, 12974, 13536 },
+ { 1, 144, 287, 430, 573, 716, 859, 1145, 1288, 1431, 1574, 1717, 1860,
+ 2003, 2289, 2432, 2575, 2718, 2861, 3004, 3147, 3433, 3576, 3719,
+ 3862, 4005, 4148, 4291, 4577, 4578, 4863, 4864, 5149, 5150, 5435,
+ 5436, 5721, 5722, 6007, 6008, 6293, 6294, 6865, 6866, 7151, 7152,
+ 7437, 7438, 7723, 7724, 8009, 8010, 8295, 8296, 8581, 8582, 9153,
+ 9154, 9439, 9440, 9725, 9726, 10011, 10012, 10297, 10298, 10583,
+ 10584, 10869, 10870, 11441, 11442, 11727, 11728, 12013, 12014, 12299,
+ 12585, 12586, 12871, 12872, 13157, 13158, 13728 },
+ { 1, 146, 291, 436, 581, 726, 871, 1161, 1306, 1451, 1596, 1741, 1886,
+ 2031, 2321, 2466, 2611, 2756, 2901, 3046, 3191, 3481, 3626, 3771,
+ 3916, 4061, 4206, 4351, 4641, 4642, 4931, 4932, 5221, 5222, 5511,
+ 5512, 5801, 5802, 6091, 6092, 6381, 6382, 6961, 6962, 7251, 7252,
+ 7541, 7542, 7831, 7832, 8121, 8122, 8411, 8412, 8701, 8702, 9281,
+ 9282, 9571, 9572, 9861, 9862, 10151, 10152, 10441, 10442, 10731,
+ 10732, 11021, 11022, 11601, 11602, 11891, 11892, 12181, 12182, 12471,
+ 12761, 12762, 13051, 13052, 13341, 13342, 13920 },
+ { 1, 148, 295, 442, 589, 736, 883, 1177, 1324, 1471, 1618, 1765, 1912,
+ 2059, 2353, 2500, 2647, 2794, 2941, 3088, 3235, 3529, 3676, 3823,
+ 3970, 4117, 4264, 4411, 4705, 4706, 4999, 5000, 5293, 5294, 5587,
+ 5588, 5881, 5882, 6175, 6176, 6469, 6470, 7057, 7058, 7351, 7352,
+ 7645, 7646, 7939, 7940, 8233, 8234, 8527, 8528, 8821, 8822, 9409,
+ 9410, 9703, 9704, 9997, 9998, 10291, 10292, 10585, 10586, 10879,
+ 10880, 11173, 11174, 11761, 11762, 12055, 12056, 12349, 12350, 12643,
+ 12937, 12938, 13231, 13232, 13525, 13526, 14112 },
+ { 1, 150, 299, 448, 597, 746, 895, 1193, 1342, 1491, 1640, 1789, 1938,
+ 2087, 2385, 2534, 2683, 2832, 2981, 3130, 3279, 3577, 3726, 3875,
+ 4024, 4173, 4322, 4471, 4769, 4770, 5067, 5068, 5365, 5366, 5663,
+ 5664, 5961, 5962, 6259, 6260, 6557, 6558, 7153, 7154, 7451, 7452,
+ 7749, 7750, 8047, 8048, 8345, 8346, 8643, 8644, 8941, 8942, 9537,
+ 9538, 9835, 9836, 10133, 10134, 10431, 10432, 10729, 10730, 11027,
+ 11028, 11325, 11326, 11921, 11922, 12219, 12220, 12517, 12518, 12815,
+ 13113, 13114, 13411, 13412, 13709, 13710, 14304 },
+ { 1, 152, 303, 454, 605, 756, 907, 1209, 1360, 1511, 1662, 1813, 1964,
+ 2115, 2417, 2568, 2719, 2870, 3021, 3172, 3323, 3625, 3776, 3927,
+ 4078, 4229, 4380, 4531, 4833, 4834, 5135, 5136, 5437, 5438, 5739,
+ 5740, 6041, 6042, 6343, 6344, 6645, 6646, 7249, 7250, 7551, 7552,
+ 7853, 7854, 8155, 8156, 8457, 8458, 8759, 8760, 9061, 9062, 9665,
+ 9666, 9967, 9968, 10269, 10270, 10571, 10572, 10873, 10874, 11175,
+ 11176, 11477, 11478, 12081, 12082, 12383, 12384, 12685, 12686, 12987,
+ 13289, 13290, 13591, 13592, 13893, 13894, 14496 },
+ { 1, 154, 307, 460, 613, 766, 919, 1225, 1378, 1531, 1684, 1837, 1990,
+ 2143, 2449, 2602, 2755, 2908, 3061, 3214, 3367, 3673, 3826, 3979,
+ 4132, 4285, 4438, 4591, 4897, 4898, 5203, 5204, 5509, 5510, 5815,
+ 5816, 6121, 6122, 6427, 6428, 6733, 6734, 7345, 7346, 7651, 7652,
+ 7957, 7958, 8263, 8264, 8569, 8570, 8875, 8876, 9181, 9182, 9793,
+ 9794, 10099, 10100, 10405, 10406, 10711, 10712, 11017, 11018, 11323,
+ 11324, 11629, 11630, 12241, 12242, 12547, 12548, 12853, 12854, 13159,
+ 13465, 13466, 13771, 13772, 14077, 14078, 14688 },
+ { 1, 156, 311, 466, 621, 776, 931, 1241, 1396, 1551, 1706, 1861, 2016,
+ 2171, 2481, 2636, 2791, 2946, 3101, 3256, 3411, 3721, 3876, 4031,
+ 4186, 4341, 4496, 4651, 4961, 4962, 5271, 5272, 5581, 5582, 5891,
+ 5892, 6201, 6202, 6511, 6512, 6821, 6822, 7441, 7442, 7751, 7752,
+ 8061, 8062, 8371, 8372, 8681, 8682, 8991, 8992, 9301, 9302, 9921,
+ 9922, 10231, 10232, 10541, 10542, 10851, 10852, 11161, 11162, 11471,
+ 11472, 11781, 11782, 12401, 12402, 12711, 12712, 13021, 13022, 13331,
+ 13641, 13642, 13951, 13952, 14261, 14262, 14880 },
+ { 1, 158, 315, 472, 629, 786, 943, 1257, 1414, 1571, 1728, 1885, 2042,
+ 2199, 2513, 2670, 2827, 2984, 3141, 3298, 3455, 3769, 3926, 4083,
+ 4240, 4397, 4554, 4711, 5025, 5026, 5339, 5340, 5653, 5654, 5967,
+ 5968, 6281, 6282, 6595, 6596, 6909, 6910, 7537, 7538, 7851, 7852,
+ 8165, 8166, 8479, 8480, 8793, 8794, 9107, 9108, 9421, 9422, 10049,
+ 10050, 10363, 10364, 10677, 10678, 10991, 10992, 11305, 11306, 11619,
+ 11620, 11933, 11934, 12561, 12562, 12875, 12876, 13189, 13190, 13503,
+ 13817, 13818, 14131, 14132, 14445, 14446, 15072 },
+ { 1, 160, 319, 478, 637, 796, 955, 1273, 1432, 1591, 1750, 1909, 2068,
+ 2227, 2545, 2704, 2863, 3022, 3181, 3340, 3499, 3817, 3976, 4135,
+ 4294, 4453, 4612, 4771, 5089, 5090, 5407, 5408, 5725, 5726, 6043,
+ 6044, 6361, 6362, 6679, 6680, 6997, 6998, 7633, 7634, 7951, 7952,
+ 8269, 8270, 8587, 8588, 8905, 8906, 9223, 9224, 9541, 9542, 10177,
+ 10178, 10495, 10496, 10813, 10814, 11131, 11132, 11449, 11450, 11767,
+ 11768, 12085, 12086, 12721, 12722, 13039, 13040, 13357, 13358, 13675,
+ 13993, 13994, 14311, 14312, 14629, 14630, 15264 },
+ { 1, 162, 323, 484, 645, 806, 967, 1289, 1450, 1611, 1772, 1933, 2094,
+ 2255, 2577, 2738, 2899, 3060, 3221, 3382, 3543, 3865, 4026, 4187,
+ 4348, 4509, 4670, 4831, 5153, 5154, 5475, 5476, 5797, 5798, 6119,
+ 6120, 6441, 6442, 6763, 6764, 7085, 7086, 7729, 7730, 8051, 8052,
+ 8373, 8374, 8695, 8696, 9017, 9018, 9339, 9340, 9661, 9662, 10305,
+ 10306, 10627, 10628, 10949, 10950, 11271, 11272, 11593, 11594, 11915,
+ 11916, 12237, 12238, 12881, 12882, 13203, 13204, 13525, 13526, 13847,
+ 14169, 14170, 14491, 14492, 14813, 14814, 15456 },
+ { 1, 164, 327, 490, 653, 816, 979, 1305, 1468, 1631, 1794, 1957, 2120,
+ 2283, 2609, 2772, 2935, 3098, 3261, 3424, 3587, 3913, 4076, 4239,
+ 4402, 4565, 4728, 4891, 5217, 5218, 5543, 5544, 5869, 5870, 6195,
+ 6196, 6521, 6522, 6847, 6848, 7173, 7174, 7825, 7826, 8151, 8152,
+ 8477, 8478, 8803, 8804, 9129, 9130, 9455, 9456, 9781, 9782, 10433,
+ 10434, 10759, 10760, 11085, 11086, 11411, 11412, 11737, 11738, 12063,
+ 12064, 12389, 12390, 13041, 13042, 13367, 13368, 13693, 13694, 14019,
+ 14345, 14346, 14671, 14672, 14997, 14998, 15648 },
+ { 1, 166, 331, 496, 661, 826, 991, 1321, 1486, 1651, 1816, 1981, 2146,
+ 2311, 2641, 2806, 2971, 3136, 3301, 3466, 3631, 3961, 4126, 4291,
+ 4456, 4621, 4786, 4951, 5281, 5282, 5611, 5612, 5941, 5942, 6271,
+ 6272, 6601, 6602, 6931, 6932, 7261, 7262, 7921, 7922, 8251, 8252,
+ 8581, 8582, 8911, 8912, 9241, 9242, 9571, 9572, 9901, 9902, 10561,
+ 10562, 10891, 10892, 11221, 11222, 11551, 11552, 11881, 11882, 12211,
+ 12212, 12541, 12542, 13201, 13202, 13531, 13532, 13861, 13862, 14191,
+ 14521, 14522, 14851, 14852, 15181, 15182, 15840 },
+ { 1, 168, 335, 502, 669, 836, 1003, 1337, 1504, 1671, 1838, 2005, 2172,
+ 2339, 2673, 2840, 3007, 3174, 3341, 3508, 3675, 4009, 4176, 4343,
+ 4510, 4677, 4844, 5011, 5345, 5346, 5679, 5680, 6013, 6014, 6347,
+ 6348, 6681, 6682, 7015, 7016, 7349, 7350, 8017, 8018, 8351, 8352,
+ 8685, 8686, 9019, 9020, 9353, 9354, 9687, 9688, 10021, 10022, 10689,
+ 10690, 11023, 11024, 11357, 11358, 11691, 11692, 12025, 12026, 12359,
+ 12360, 12693, 12694, 13361, 13362, 13695, 13696, 14029, 14030, 14363,
+ 14697, 14698, 15031, 15032, 15365, 15366, 16032 },
+ { 1, 170, 339, 508, 677, 846, 1015, 1353, 1522, 1691, 1860, 2029, 2198,
+ 2367, 2705, 2874, 3043, 3212, 3381, 3550, 3719, 4057, 4226, 4395,
+ 4564, 4733, 4902, 5071, 5409, 5410, 5747, 5748, 6085, 6086, 6423,
+ 6424, 6761, 6762, 7099, 7100, 7437, 7438, 8113, 8114, 8451, 8452,
+ 8789, 8790, 9127, 9128, 9465, 9466, 9803, 9804, 10141, 10142, 10817,
+ 10818, 11155, 11156, 11493, 11494, 11831, 11832, 12169, 12170, 12507,
+ 12508, 12845, 12846, 13521, 13522, 13859, 13860, 14197, 14198, 14535,
+ 14873, 14874, 15211, 15212, 15549, 15550, 16224 },
+ { 1, 172, 343, 514, 685, 856, 1027, 1369, 1540, 1711, 1882, 2053, 2224,
+ 2395, 2737, 2908, 3079, 3250, 3421, 3592, 3763, 4105, 4276, 4447,
+ 4618, 4789, 4960, 5131, 5473, 5474, 5815, 5816, 6157, 6158, 6499,
+ 6500, 6841, 6842, 7183, 7184, 7525, 7526, 8209, 8210, 8551, 8552,
+ 8893, 8894, 9235, 9236, 9577, 9578, 9919, 9920, 10261, 10262, 10945,
+ 10946, 11287, 11288, 11629, 11630, 11971, 11972, 12313, 12314, 12655,
+ 12656, 12997, 12998, 13681, 13682, 14023, 14024, 14365, 14366, 14707,
+ 15049, 15050, 15391, 15392, 15733, 15734, 16416 },
+ { 1, 174, 347, 520, 693, 866, 1039, 1385, 1558, 1731, 1904, 2077, 2250,
+ 2423, 2769, 2942, 3115, 3288, 3461, 3634, 3807, 4153, 4326, 4499,
+ 4672, 4845, 5018, 5191, 5537, 5538, 5883, 5884, 6229, 6230, 6575,
+ 6576, 6921, 6922, 7267, 7268, 7613, 7614, 8305, 8306, 8651, 8652,
+ 8997, 8998, 9343, 9344, 9689, 9690, 10035, 10036, 10381, 10382, 11073,
+ 11074, 11419, 11420, 11765, 11766, 12111, 12112, 12457, 12458, 12803,
+ 12804, 13149, 13150, 13841, 13842, 14187, 14188, 14533, 14534, 14879,
+ 15225, 15226, 15571, 15572, 15917, 15918, 16608 },
+ { 1, 176, 351, 526, 701, 876, 1051, 1401, 1576, 1751, 1926, 2101, 2276,
+ 2451, 2801, 2976, 3151, 3326, 3501, 3676, 3851, 4201, 4376, 4551,
+ 4726, 4901, 5076, 5251, 5601, 5602, 5951, 5952, 6301, 6302, 6651,
+ 6652, 7001, 7002, 7351, 7352, 7701, 7702, 8401, 8402, 8751, 8752,
+ 9101, 9102, 9451, 9452, 9801, 9802, 10151, 10152, 10501, 10502, 11201,
+ 11202, 11551, 11552, 11901, 11902, 12251, 12252, 12601, 12602, 12951,
+ 12952, 13301, 13302, 14001, 14002, 14351, 14352, 14701, 14702, 15051,
+ 15401, 15402, 15751, 15752, 16101, 16102, 16800 },
+ { 1, 178, 355, 532, 709, 886, 1063, 1417, 1594, 1771, 1948, 2125, 2302,
+ 2479, 2833, 3010, 3187, 3364, 3541, 3718, 3895, 4249, 4426, 4603,
+ 4780, 4957, 5134, 5311, 5665, 5666, 6019, 6020, 6373, 6374, 6727,
+ 6728, 7081, 7082, 7435, 7436, 7789, 7790, 8497, 8498, 8851, 8852,
+ 9205, 9206, 9559, 9560, 9913, 9914, 10267, 10268, 10621, 10622, 11329,
+ 11330, 11683, 11684, 12037, 12038, 12391, 12392, 12745, 12746, 13099,
+ 13100, 13453, 13454, 14161, 14162, 14515, 14516, 14869, 14870, 15223,
+ 15577, 15578, 15931, 15932, 16285, 16286, 16992 },
+ { 1, 180, 359, 538, 717, 896, 1075, 1433, 1612, 1791, 1970, 2149, 2328,
+ 2507, 2865, 3044, 3223, 3402, 3581, 3760, 3939, 4297, 4476, 4655,
+ 4834, 5013, 5192, 5371, 5729, 5730, 6087, 6088, 6445, 6446, 6803,
+ 6804, 7161, 7162, 7519, 7520, 7877, 7878, 8593, 8594, 8951, 8952,
+ 9309, 9310, 9667, 9668, 10025, 10026, 10383, 10384, 10741, 10742,
+ 11457, 11458, 11815, 11816, 12173, 12174, 12531, 12532, 12889, 12890,
+ 13247, 13248, 13605, 13606, 14321, 14322, 14679, 14680, 15037, 15038,
+ 15395, 15753, 15754, 16111, 16112, 16469, 16470, 17184 },
+ { 1, 182, 363, 544, 725, 906, 1087, 1449, 1630, 1811, 1992, 2173, 2354,
+ 2535, 2897, 3078, 3259, 3440, 3621, 3802, 3983, 4345, 4526, 4707,
+ 4888, 5069, 5250, 5431, 5793, 5794, 6155, 6156, 6517, 6518, 6879,
+ 6880, 7241, 7242, 7603, 7604, 7965, 7966, 8689, 8690, 9051, 9052,
+ 9413, 9414, 9775, 9776, 10137, 10138, 10499, 10500, 10861, 10862,
+ 11585, 11586, 11947, 11948, 12309, 12310, 12671, 12672, 13033, 13034,
+ 13395, 13396, 13757, 13758, 14481, 14482, 14843, 14844, 15205, 15206,
+ 15567, 15929, 15930, 16291, 16292, 16653, 16654, 17376 },
+ { 1, 184, 367, 550, 733, 916, 1099, 1465, 1648, 1831, 2014, 2197, 2380,
+ 2563, 2929, 3112, 3295, 3478, 3661, 3844, 4027, 4393, 4576, 4759,
+ 4942, 5125, 5308, 5491, 5857, 5858, 6223, 6224, 6589, 6590, 6955,
+ 6956, 7321, 7322, 7687, 7688, 8053, 8054, 8785, 8786, 9151, 9152,
+ 9517, 9518, 9883, 9884, 10249, 10250, 10615, 10616, 10981, 10982,
+ 11713, 11714, 12079, 12080, 12445, 12446, 12811, 12812, 13177, 13178,
+ 13543, 13544, 13909, 13910, 14641, 14642, 15007, 15008, 15373, 15374,
+ 15739, 16105, 16106, 16471, 16472, 16837, 16838, 17568 },
+ { 1, 186, 371, 556, 741, 926, 1111, 1481, 1666, 1851, 2036, 2221, 2406,
+ 2591, 2961, 3146, 3331, 3516, 3701, 3886, 4071, 4441, 4626, 4811,
+ 4996, 5181, 5366, 5551, 5921, 5922, 6291, 6292, 6661, 6662, 7031,
+ 7032, 7401, 7402, 7771, 7772, 8141, 8142, 8881, 8882, 9251, 9252,
+ 9621, 9622, 9991, 9992, 10361, 10362, 10731, 10732, 11101, 11102,
+ 11841, 11842, 12211, 12212, 12581, 12582, 12951, 12952, 13321, 13322,
+ 13691, 13692, 14061, 14062, 14801, 14802, 15171, 15172, 15541, 15542,
+ 15911, 16281, 16282, 16651, 16652, 17021, 17022, 17760 },
+ { 1, 188, 375, 562, 749, 936, 1123, 1497, 1684, 1871, 2058, 2245, 2432,
+ 2619, 2993, 3180, 3367, 3554, 3741, 3928, 4115, 4489, 4676, 4863,
+ 5050, 5237, 5424, 5611, 5985, 5986, 6359, 6360, 6733, 6734, 7107,
+ 7108, 7481, 7482, 7855, 7856, 8229, 8230, 8977, 8978, 9351, 9352,
+ 9725, 9726, 10099, 10100, 10473, 10474, 10847, 10848, 11221, 11222,
+ 11969, 11970, 12343, 12344, 12717, 12718, 13091, 13092, 13465, 13466,
+ 13839, 13840, 14213, 14214, 14961, 14962, 15335, 15336, 15709, 15710,
+ 16083, 16457, 16458, 16831, 16832, 17205, 17206, 17952 },
+ { 1, 190, 379, 568, 757, 946, 1135, 1513, 1702, 1891, 2080, 2269, 2458,
+ 2647, 3025, 3214, 3403, 3592, 3781, 3970, 4159, 4537, 4726, 4915,
+ 5104, 5293, 5482, 5671, 6049, 6050, 6427, 6428, 6805, 6806, 7183,
+ 7184, 7561, 7562, 7939, 7940, 8317, 8318, 9073, 9074, 9451, 9452,
+ 9829, 9830, 10207, 10208, 10585, 10586, 10963, 10964, 11341, 11342,
+ 12097, 12098, 12475, 12476, 12853, 12854, 13231, 13232, 13609, 13610,
+ 13987, 13988, 14365, 14366, 15121, 15122, 15499, 15500, 15877, 15878,
+ 16255, 16633, 16634, 17011, 17012, 17389, 17390, 18144 },
+ { 1, 192, 383, 574, 765, 956, 1147, 1529, 1720, 1911, 2102, 2293, 2484,
+ 2675, 3057, 3248, 3439, 3630, 3821, 4012, 4203, 4585, 4776, 4967,
+ 5158, 5349, 5540, 5731, 6113, 6114, 6495, 6496, 6877, 6878, 7259,
+ 7260, 7641, 7642, 8023, 8024, 8405, 8406, 9169, 9170, 9551, 9552,
+ 9933, 9934, 10315, 10316, 10697, 10698, 11079, 11080, 11461, 11462,
+ 12225, 12226, 12607, 12608, 12989, 12990, 13371, 13372, 13753, 13754,
+ 14135, 14136, 14517, 14518, 15281, 15282, 15663, 15664, 16045, 16046,
+ 16427, 16809, 16810, 17191, 17192, 17573, 17574, 18336 },
+ { 1, 194, 387, 580, 773, 966, 1159, 1545, 1738, 1931, 2124, 2317, 2510,
+ 2703, 3089, 3282, 3475, 3668, 3861, 4054, 4247, 4633, 4826, 5019,
+ 5212, 5405, 5598, 5791, 6177, 6178, 6563, 6564, 6949, 6950, 7335,
+ 7336, 7721, 7722, 8107, 8108, 8493, 8494, 9265, 9266, 9651, 9652,
+ 10037, 10038, 10423, 10424, 10809, 10810, 11195, 11196, 11581, 11582,
+ 12353, 12354, 12739, 12740, 13125, 13126, 13511, 13512, 13897, 13898,
+ 14283, 14284, 14669, 14670, 15441, 15442, 15827, 15828, 16213, 16214,
+ 16599, 16985, 16986, 17371, 17372, 17757, 17758, 18528 }
+};
+
+#endif /* _BBDEV_TURBO_SW_TABLES_H */
diff --git a/drivers/bbdev/turbo_sw/rte_pmd_bbdev_turbo_sw_version.map b/drivers/bbdev/turbo_sw/rte_pmd_bbdev_turbo_sw_version.map
new file mode 100644
index 0000000..a753031
--- /dev/null
+++ b/drivers/bbdev/turbo_sw/rte_pmd_bbdev_turbo_sw_version.map
@@ -0,0 +1,3 @@
+DPDK_17.11 {
+ local: *;
+};
--
2.7.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [dpdk-dev] [PATCH v2 3/5] bbdev: test applications
2017-10-18 2:14 [dpdk-dev] [PATCH v2 1/5] bbdev: librte_bbdev library Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 2/5] bbdev: PMD drivers (null/turbo_sw) Amr Mokhtar
@ 2017-10-18 2:14 ` Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 4/5] bbdev: sample app Amr Mokhtar
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Amr Mokhtar @ 2017-10-18 2:14 UTC (permalink / raw)
To: dev
Cc: thomas, anatoly.burakov, pablo.de.lara.guarch, niall.power,
chris.macnamara, Amr Mokhtar
- Full test suite for bbdev
- Test App works seamlessly on all PMDs registered with bbdev
framework
- A python script is provided to make our life easier
- Supports execution of tests by parsing Test Vector files
- Test Vectors can be added/deleted/modified with no need for
re-compilation
- Various Tests can be executed:
(a) Throughput test
(b) Offload latency test
(c) Operation latency test
(d) Validation test
(c) Sanity checks
Signed-off-by: Amr Mokhtar <amr.mokhtar@intel.com>
---
app/Makefile | 4 +
app/test-bbdev/Makefile | 53 +
app/test-bbdev/main.c | 317 +++
app/test-bbdev/main.h | 144 ++
app/test-bbdev/test-bbdev.py | 132 ++
app/test-bbdev/test_bbdev.c | 1406 +++++++++++++
app/test-bbdev/test_bbdev_perf.c | 2090 ++++++++++++++++++++
app/test-bbdev/test_bbdev_vector.c | 884 +++++++++
app/test-bbdev/test_bbdev_vector.h | 98 +
app/test-bbdev/test_vectors/bbdev_vector_null.data | 32 +
.../test_vectors/bbdev_vector_td_default.data | 80 +
.../test_vectors/bbdev_vector_te_default.data | 60 +
12 files changed, 5300 insertions(+)
create mode 100644 app/test-bbdev/Makefile
create mode 100644 app/test-bbdev/main.c
create mode 100644 app/test-bbdev/main.h
create mode 100755 app/test-bbdev/test-bbdev.py
create mode 100644 app/test-bbdev/test_bbdev.c
create mode 100644 app/test-bbdev/test_bbdev_perf.c
create mode 100644 app/test-bbdev/test_bbdev_vector.c
create mode 100644 app/test-bbdev/test_bbdev_vector.h
create mode 100644 app/test-bbdev/test_vectors/bbdev_vector_null.data
create mode 100644 app/test-bbdev/test_vectors/bbdev_vector_td_default.data
create mode 100644 app/test-bbdev/test_vectors/bbdev_vector_te_default.data
diff --git a/app/Makefile b/app/Makefile
index 7ea02b0..6b0ed67 100644
--- a/app/Makefile
+++ b/app/Makefile
@@ -43,4 +43,8 @@ ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
DIRS-$(CONFIG_RTE_APP_EVENTDEV) += test-eventdev
endif
+ifeq ($(CONFIG_RTE_LIBRTE_BBDEV),y)
+DIRS-$(CONFIG_RTE_TEST_BBDEV) += test-bbdev
+endif
+
include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/app/test-bbdev/Makefile b/app/test-bbdev/Makefile
new file mode 100644
index 0000000..29c9be5
--- /dev/null
+++ b/app/test-bbdev/Makefile
@@ -0,0 +1,53 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifeq ($(CONFIG_RTE_TEST_BBDEV),y)
+
+#
+# library name
+#
+APP = testbbdev
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+#
+# all sources are stored in SRCS-y
+#
+SRCS-$(CONFIG_RTE_LIBRTE_BBDEV) += main.c
+SRCS-$(CONFIG_RTE_LIBRTE_BBDEV) += test_bbdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_BBDEV) += test_bbdev_perf.c
+SRCS-$(CONFIG_RTE_LIBRTE_BBDEV) += test_bbdev_vector.c
+
+include $(RTE_SDK)/mk/rte.app.mk
+
+endif
diff --git a/app/test-bbdev/main.c b/app/test-bbdev/main.c
new file mode 100644
index 0000000..59b01f9
--- /dev/null
+++ b/app/test-bbdev/main.c
@@ -0,0 +1,317 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <getopt.h>
+#include <inttypes.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdbool.h>
+
+#include <rte_eal.h>
+#include <rte_common.h>
+#include <rte_string_fns.h>
+#include <rte_cycles.h>
+
+#include "main.h"
+
+/* Defines how many testcases can be specified as cmdline args */
+#define MAX_CMDLINE_TESTCASES 8
+
+static const char tc_sep = ',';
+
+static struct test_params {
+ struct test_command *test_to_run[MAX_CMDLINE_TESTCASES];
+ unsigned int num_tests;
+ unsigned int num_ops;
+ unsigned int burst_sz;
+ char test_vector_filename[PATH_MAX];
+} test_params;
+
+static struct test_commands_list commands_list =
+ TAILQ_HEAD_INITIALIZER(commands_list);
+
+void
+add_test_command(struct test_command *t)
+{
+ TAILQ_INSERT_TAIL(&commands_list, t, next);
+}
+
+int
+unit_test_suite_runner(struct unit_test_suite *suite)
+{
+ int test_result = TEST_SUCCESS;
+ unsigned int total = 0, skipped = 0, succeeded = 0, failed = 0;
+ uint64_t start, end;
+
+ printf(
+ "\n + ------------------------------------------------------- +\n");
+ printf(" + Starting Test Suite : %s\n", suite->suite_name);
+
+ start = rte_rdtsc_precise();
+
+ if (suite->setup) {
+ test_result = suite->setup();
+ if (test_result == TEST_FAILED) {
+ printf(" + Test suite setup %s failed!\n",
+ suite->suite_name);
+ printf(
+ " + ------------------------------------------------------- +\n");
+ return 1;
+ }
+ if (test_result == TEST_SKIPPED) {
+ printf(" + Test suite setup %s skipped!\n",
+ suite->suite_name);
+ printf(
+ " + ------------------------------------------------------- +\n");
+ return 0;
+ }
+ }
+
+ while (suite->unit_test_cases[total].testcase) {
+ if (suite->unit_test_cases[total].setup)
+ test_result = suite->unit_test_cases[total].setup();
+
+ if (test_result == TEST_SUCCESS)
+ test_result = suite->unit_test_cases[total].testcase();
+
+ if (suite->unit_test_cases[total].teardown)
+ suite->unit_test_cases[total].teardown();
+
+ if (test_result == TEST_SUCCESS) {
+ succeeded++;
+ printf(" + TestCase [%2d] : %s passed\n", total,
+ suite->unit_test_cases[total].name);
+ } else if (test_result == TEST_SKIPPED) {
+ skipped++;
+ printf(" + TestCase [%2d] : %s skipped\n", total,
+ suite->unit_test_cases[total].name);
+ } else {
+ failed++;
+ printf(" + TestCase [%2d] : %s failed\n", total,
+ suite->unit_test_cases[total].name);
+ }
+
+ total++;
+ }
+
+ /* Run test suite teardown */
+ if (suite->teardown)
+ suite->teardown();
+
+ end = rte_rdtsc_precise();
+
+ printf(" + ------------------------------------------------------- +\n");
+ printf(" + Test Suite Summary : %s\n", suite->suite_name);
+ printf(" + Tests Total : %2d\n", total);
+ printf(" + Tests Skipped : %2d\n", skipped);
+ printf(" + Tests Passed : %2d\n", succeeded);
+ printf(" + Tests Failed : %2d\n", failed);
+ printf(" + Tests Lasted : %lg ms\n",
+ ((end - start) * 1000) / (double)rte_get_tsc_hz());
+ printf(" + ------------------------------------------------------- +\n");
+
+ return (failed > 0) ? 1 : 0;
+}
+
+const char *
+get_vector_filename(void)
+{
+ return test_params.test_vector_filename;
+}
+
+unsigned int
+get_num_ops(void)
+{
+ return test_params.num_ops;
+}
+
+unsigned int
+get_burst_sz(void)
+{
+ return test_params.burst_sz;
+}
+
+static void
+print_usage(const char *prog_name)
+{
+ struct test_command *t;
+
+ printf("Usage: %s [EAL params] [-- [-n/--num-ops NUM_OPS]\n"
+ "\t[-b/--burst-size BURST_SIZE]\n"
+ "\t[-v/--test-vector VECTOR_FILE]\n"
+ "\t[-c/--test-cases TEST_CASE[,TEST_CASE,...]]]\n",
+ prog_name);
+
+ printf("Available testcases: ");
+ TAILQ_FOREACH(t, &commands_list, next)
+ printf("%s ", t->command);
+ printf("\n");
+}
+
+static int
+parse_args(int argc, char **argv, struct test_params *tp)
+{
+ int opt, option_index;
+ unsigned int num_tests = 0;
+ bool test_cases_present = false;
+ bool test_vector_present = false;
+ struct test_command *t;
+ char *tokens[MAX_CMDLINE_TESTCASES];
+ int tc, ret;
+
+ static struct option lgopts[] = {
+ { "num-ops", 1, 0, 'n' },
+ { "burst-size", 1, 0, 'b' },
+ { "test-cases", 1, 0, 'c' },
+ { "test-vector", 1, 0, 'v' },
+ { "help", 0, 0, 'h' },
+ { NULL, 0, 0, 0 }
+ };
+
+ while ((opt = getopt_long(argc, argv, "hn:b:c:v:", lgopts,
+ &option_index)) != EOF)
+ switch (opt) {
+ case 'n':
+ TEST_ASSERT(strlen(optarg) > 0,
+ "Num of operations is not provided");
+ tp->num_ops = strtol(optarg, NULL, 10);
+ break;
+ case 'b':
+ TEST_ASSERT(strlen(optarg) > 0,
+ "Burst size is not provided");
+ tp->burst_sz = strtol(optarg, NULL, 10);
+ TEST_ASSERT(tp->burst_sz <= MAX_BURST,
+ "Burst size mustn't be greater than %u",
+ MAX_BURST);
+ break;
+ case 'c':
+ TEST_ASSERT(test_cases_present == false,
+ "Test cases provided more than once");
+ test_cases_present = true;
+
+ ret = rte_strsplit(optarg, strlen(optarg),
+ tokens, MAX_CMDLINE_TESTCASES, tc_sep);
+
+ TEST_ASSERT(ret <= MAX_CMDLINE_TESTCASES,
+ "Too many test cases (max=%d)",
+ MAX_CMDLINE_TESTCASES);
+
+ for (tc = 0; tc < ret; ++tc) {
+ /* Find matching test case */
+ TAILQ_FOREACH(t, &commands_list, next)
+ if (!strcmp(tokens[tc], t->command))
+ tp->test_to_run[num_tests] = t;
+
+ TEST_ASSERT(tp->test_to_run[num_tests] != NULL,
+ "Unknown test case: %s",
+ tokens[tc]);
+ ++num_tests;
+ }
+ break;
+ case 'v':
+ TEST_ASSERT(test_vector_present == false,
+ "Test vector provided more than once");
+ test_vector_present = true;
+
+ TEST_ASSERT(strlen(optarg) > 0,
+ "Config file name is null");
+
+ strncpy(tp->test_vector_filename, optarg,
+ sizeof(tp->test_vector_filename));
+ break;
+ case 'h':
+ print_usage(argv[0]);
+ return 0;
+ default:
+ printf("ERROR: Unknown option: -%c\n", opt);
+ return -1;
+ }
+
+ TEST_ASSERT(tp->burst_sz <= tp->num_ops,
+ "Burst size (%u) mustn't be greater than num ops (%u)",
+ tp->burst_sz, tp->num_ops);
+
+ tp->num_tests = num_tests;
+ return 0;
+}
+
+static int
+run_all_tests(void)
+{
+ int ret = TEST_SUCCESS;
+ struct test_command *t;
+
+ TAILQ_FOREACH(t, &commands_list, next)
+ ret |= t->callback();
+
+ return ret;
+}
+
+static int
+run_parsed_tests(struct test_params *tp)
+{
+ int ret = TEST_SUCCESS;
+ unsigned int i;
+
+ for (i = 0; i < tp->num_tests; ++i)
+ ret |= tp->test_to_run[i]->callback();
+
+ return ret;
+}
+
+int
+main(int argc, char **argv)
+{
+ int ret;
+
+ /* Init EAL */
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ return 1;
+ argc -= ret;
+ argv += ret;
+
+ /* Parse application arguments (after the EAL ones) */
+ ret = parse_args(argc, argv, &test_params);
+ if (ret < 0) {
+ print_usage(argv[0]);
+ return 1;
+ }
+
+ rte_log_set_global_level(RTE_LOG_INFO);
+
+ /* If no argument provided - run all tests */
+ if (test_params.num_tests == 0)
+ return run_all_tests();
+ else
+ return run_parsed_tests(&test_params);
+}
diff --git a/app/test-bbdev/main.h b/app/test-bbdev/main.h
new file mode 100644
index 0000000..a60076d
--- /dev/null
+++ b/app/test-bbdev/main.h
@@ -0,0 +1,144 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _MAIN_H_
+#define _MAIN_H_
+
+#include <stddef.h>
+#include <sys/queue.h>
+
+#include <rte_common.h>
+#include <rte_hexdump.h>
+#include <rte_log.h>
+
+#define TEST_SUCCESS 0
+#define TEST_FAILED -1
+#define TEST_SKIPPED 1
+
+#define MAX_BURST 512U
+
+#define TEST_ASSERT(cond, msg, ...) do { \
+ if (!(cond)) { \
+ printf("TestCase %s() line %d failed: " \
+ msg "\n", __func__, __LINE__, ##__VA_ARGS__); \
+ return TEST_FAILED; \
+ } \
+} while (0)
+
+/* Compare two buffers (length in bytes) */
+#define TEST_ASSERT_BUFFERS_ARE_EQUAL(a, b, len, msg, ...) do { \
+ if (memcmp((a), (b), len)) { \
+ printf("TestCase %s() line %d failed: " \
+ msg "\n", __func__, __LINE__, ##__VA_ARGS__); \
+ rte_memdump(stdout, "Buffer A", (a), len); \
+ rte_memdump(stdout, "Buffer B", (b), len); \
+ return TEST_FAILED; \
+ } \
+} while (0)
+
+#define TEST_ASSERT_SUCCESS(val, msg, ...) do { \
+ typeof(val) _val = (val); \
+ if (!(_val == 0)) { \
+ printf("TestCase %s() line %d failed (err %d): " \
+ msg "\n", __func__, __LINE__, _val, \
+ ##__VA_ARGS__); \
+ return TEST_FAILED; \
+ } \
+} while (0)
+
+#define TEST_ASSERT_FAIL(val, msg, ...) \
+ TEST_ASSERT_SUCCESS(!(val), msg, ##__VA_ARGS__)
+
+#define TEST_ASSERT_NOT_NULL(val, msg, ...) do { \
+ if ((val) == NULL) { \
+ printf("TestCase %s() line %d failed (null): " \
+ msg "\n", __func__, __LINE__, ##__VA_ARGS__); \
+ return TEST_FAILED; \
+ } \
+} while (0)
+
+struct unit_test_case {
+ int (*setup)(void);
+ void (*teardown)(void);
+ int (*testcase)(void);
+ const char *name;
+};
+
+#define TEST_CASE(testcase) {NULL, NULL, testcase, #testcase}
+
+#define TEST_CASE_ST(setup, teardown, testcase) \
+ {setup, teardown, testcase, #testcase}
+
+#define TEST_CASES_END() {NULL, NULL, NULL, NULL}
+
+struct unit_test_suite {
+ const char *suite_name;
+ int (*setup)(void);
+ void (*teardown)(void);
+ struct unit_test_case unit_test_cases[];
+};
+
+int unit_test_suite_runner(struct unit_test_suite *suite);
+
+typedef int (test_callback)(void);
+TAILQ_HEAD(test_commands_list, test_command);
+struct test_command {
+ TAILQ_ENTRY(test_command) next;
+ const char *command;
+ test_callback *callback;
+};
+
+void add_test_command(struct test_command *t);
+
+/* Register a test function */
+#define REGISTER_TEST_COMMAND(name, testsuite) \
+ static int test_func_##name(void) \
+ { \
+ return unit_test_suite_runner(&testsuite); \
+ } \
+ static struct test_command test_struct_##name = { \
+ .command = RTE_STR(name), \
+ .callback = test_func_##name, \
+ }; \
+ static void __attribute__((constructor, used)) \
+ test_register_##name(void) \
+ { \
+ add_test_command(&test_struct_##name); \
+ }
+
+const char *get_vector_filename(void);
+
+unsigned int get_num_ops(void);
+
+unsigned int get_burst_sz(void);
+
+#endif
diff --git a/app/test-bbdev/test-bbdev.py b/app/test-bbdev/test-bbdev.py
new file mode 100755
index 0000000..09dd2d7
--- /dev/null
+++ b/app/test-bbdev/test-bbdev.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python
+
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+import sys
+import os
+import argparse
+import subprocess
+import shlex
+
+from threading import Timer
+
+def kill(process):
+ print "ERROR: Test app timed out"
+ process.kill()
+
+if "RTE_SDK" in os.environ:
+ dpdk_path = os.environ["RTE_SDK"]
+else:
+ dpdk_path = "../.."
+
+if "RTE_TARGET" in os.environ:
+ dpdk_target = os.environ["RTE_TARGET"]
+else:
+ dpdk_target = "x86_64-native-linuxapp-gcc"
+
+parser = argparse.ArgumentParser(
+ description='BBdev Unit Test Application',
+ formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+parser.add_argument("-p", "--testapp-path",
+ help="specifies path to the bbdev test app",
+ default=dpdk_path + "/" + dpdk_target + "/app/testbbdev")
+parser.add_argument("-e", "--eal-params",
+ help="EAL arguments which are passed to the test app",
+ default="--vdev=bbdev_null0")
+parser.add_argument("-t", "--timeout",
+ type=int,
+ help="Timeout in seconds",
+ default=300)
+parser.add_argument("-c", "--test-cases",
+ nargs="+",
+ help="Defines test cases to run. Run all if not specified")
+parser.add_argument("-v", "--test-vector",
+ nargs="+",
+ help="Specifies paths to the test vector files.",
+ default=[dpdk_path +
+ "/app/test-bbdev/test_vectors/bbdev_vector_null.data"])
+parser.add_argument("-n", "--num-ops",
+ type=int,
+ help="Number of operations to process on device.",
+ default=32)
+parser.add_argument("-b", "--burst-size",
+ nargs="+",
+ type=int,
+ help="Operations enqueue/dequeue burst size.",
+ default=[32])
+
+args = parser.parse_args()
+
+if not os.path.exists(args.testapp_path):
+ print "No such file: " + args.testapp_path
+ sys.exit(1)
+
+params = [args.testapp_path]
+if args.eal_params:
+ params.extend(shlex.split(args.eal_params))
+
+params.extend(["--"])
+
+if args.num_ops:
+ params.extend(["-n", str(args.num_ops)])
+
+if args.test_cases:
+ params.extend(["-c"])
+ params.extend([",".join(args.test_cases)])
+
+exit_status = 0
+for vector in args.test_vector:
+ for burst_size in args.burst_size:
+ call_params = params[:]
+ call_params.extend(["-v", vector])
+ call_params.extend(["-b", str(burst_size)])
+ params_string = " ".join(call_params)
+
+ print("Executing: {}".format(params_string))
+ app_proc = subprocess.Popen(call_params)
+ if args.timeout > 0:
+ timer = Timer(args.timeout, kill, [app_proc])
+ timer.start()
+
+ try:
+ app_proc.communicate()
+ except:
+ print("Error: failed to execute: {}".format(params_string))
+ finally:
+ timer.cancel()
+
+ if app_proc.returncode != 0:
+ exit_status = 1
+ print("ERROR TestCase failed. Failed test for vector {}. Return code: {}".format(
+ vector, app_proc.returncode))
+
+sys.exit(exit_status)
diff --git a/app/test-bbdev/test_bbdev.c b/app/test-bbdev/test_bbdev.c
new file mode 100644
index 0000000..982546d
--- /dev/null
+++ b/app/test-bbdev/test_bbdev.c
@@ -0,0 +1,1406 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_common.h>
+#include <rte_hexdump.h>
+#include <rte_mbuf.h>
+#include <rte_malloc.h>
+#include <rte_memcpy.h>
+#include <rte_cycles.h>
+
+#include <rte_dev.h>
+
+#include <rte_bbdev.h>
+#include <rte_bbdev_op.h>
+#include <rte_bbdev_pmd.h>
+
+#include "main.h"
+
+
+#define BBDEV_NAME_NULL ("bbdev_null")
+
+struct bbdev_testsuite_params {
+ struct rte_bbdev_queue_conf qconf;
+};
+
+static struct bbdev_testsuite_params testsuite_params;
+
+static uint8_t null_dev_id;
+
+static int
+testsuite_setup(void)
+{
+ uint8_t nb_devs;
+ int ret;
+ char buf[RTE_BBDEV_NAME_MAX_LEN];
+
+ /* Create test device */
+ snprintf(buf, sizeof(buf), "%s_unittest", BBDEV_NAME_NULL);
+ ret = rte_vdev_init(buf, NULL);
+ TEST_ASSERT(ret == 0, "Failed to create instance of pmd: %s", buf);
+
+ nb_devs = rte_bbdev_count();
+ TEST_ASSERT(nb_devs != 0, "No devices found");
+
+ /* Most recently created device is our device */
+ null_dev_id = nb_devs - 1;
+
+ return TEST_SUCCESS;
+}
+
+static void
+testsuite_teardown(void)
+{
+ char buf[RTE_BBDEV_NAME_MAX_LEN];
+
+ snprintf(buf, sizeof(buf), "%s_unittest", BBDEV_NAME_NULL);
+ rte_vdev_uninit(buf);
+}
+
+static int
+ut_setup(void)
+{
+ struct bbdev_testsuite_params *ts_params = &testsuite_params;
+ uint8_t num_queues;
+
+ /* Valid queue configuration */
+ ts_params->qconf.priority = 0;
+ ts_params->qconf.socket = SOCKET_ID_ANY;
+ ts_params->qconf.deferred_start = 1;
+
+ num_queues = 1;
+ TEST_ASSERT_SUCCESS(rte_bbdev_setup_queues(null_dev_id, num_queues,
+ SOCKET_ID_ANY), "Failed to setup queues for bbdev %u",
+ 0);
+
+ /* Start the device */
+ TEST_ASSERT_SUCCESS(rte_bbdev_start(null_dev_id),
+ "Failed to start bbdev %u", 0);
+
+ return TEST_SUCCESS;
+}
+
+static void
+ut_teardown(void)
+{
+ rte_bbdev_close(null_dev_id);
+}
+
+static int
+test_bbdev_configure_invalid_dev_id(void)
+{
+ uint8_t dev_id;
+ uint8_t num_queues;
+
+ num_queues = 1;
+ for (dev_id = 0; dev_id < RTE_BBDEV_MAX_DEVS; dev_id++) {
+ if (!rte_bbdev_is_valid(dev_id)) {
+ TEST_ASSERT_FAIL(rte_bbdev_setup_queues(dev_id,
+ num_queues, SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "invalid dev_num %u", dev_id);
+ TEST_ASSERT(rte_bbdev_intr_enable(dev_id) == -ENODEV,
+ "Failed test for rte_bbdev_intr_enable: "
+ "invalid dev_num %u", dev_id);
+ break;
+ }
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_configure_invalid_num_queues(void)
+{
+ struct rte_bbdev_info info;
+ uint8_t dev_id, num_devs;
+ uint8_t num_queues;
+ int return_value;
+
+ TEST_ASSERT((num_devs = rte_bbdev_count()) >= 1,
+ "Need at least %d devices for test", 1);
+
+ /* valid num_queues values */
+ num_queues = 8;
+
+ /* valid dev_id values */
+ dev_id = null_dev_id;
+
+ /* Stop the device in case it's started so it can be configured */
+ rte_bbdev_stop(dev_id);
+
+ TEST_ASSERT_FAIL(rte_bbdev_setup_queues(dev_id, 0, SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "invalid num_queues %d", 0);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_setup_queues(dev_id, num_queues,
+ SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "invalid dev_num %u", dev_id);
+
+ TEST_ASSERT_FAIL(return_value = rte_bbdev_info_get(dev_id, NULL),
+ "Failed test for rte_bbdev_info_get: "
+ "returned value:%i", return_value);
+
+ TEST_ASSERT_SUCCESS(return_value = rte_bbdev_info_get(dev_id, &info),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid return value:%i", return_value);
+
+ TEST_ASSERT(info.num_queues == num_queues,
+ "Failed test for rte_bbdev_info_get: "
+ "invalid num_queues:%u", info.num_queues);
+
+ num_queues = info.drv.max_num_queues;
+ TEST_ASSERT_SUCCESS(rte_bbdev_setup_queues(dev_id, num_queues,
+ SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "invalid num_queues: %u", num_queues);
+
+ num_queues++;
+ TEST_ASSERT_FAIL(rte_bbdev_setup_queues(dev_id, num_queues,
+ SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "invalid num_queues: %u", num_queues);
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_configure_stop_device(void)
+{
+ struct rte_bbdev_info info;
+ uint8_t dev_id;
+ int return_value;
+
+ /* valid dev_id values */
+ dev_id = null_dev_id;
+
+ /* Stop the device so it can be configured */
+ rte_bbdev_stop(dev_id);
+
+ TEST_ASSERT_SUCCESS(return_value = rte_bbdev_info_get(dev_id, &info),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid return value from "
+ "rte_bbdev_info_get function: %i", return_value);
+
+ TEST_ASSERT_SUCCESS(info.started, "Failed test for rte_bbdev_info_get: "
+ "started value: %u", info.started);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_setup_queues(dev_id,
+ info.drv.max_num_queues, SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "device should be stopped, dev_id: %u", dev_id);
+
+ return_value = rte_bbdev_intr_enable(dev_id);
+ TEST_ASSERT(return_value != -EBUSY,
+ "Failed test for rte_bbdev_intr_enable: device should be stopped, dev_id: %u",
+ dev_id);
+
+ /* Start the device so it cannot be configured */
+ TEST_ASSERT_FAIL(rte_bbdev_start(RTE_BBDEV_MAX_DEVS),
+ "Failed to start bbdev %u", dev_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_start(dev_id),
+ "Failed to start bbdev %u", dev_id);
+
+ TEST_ASSERT_SUCCESS(return_value = rte_bbdev_info_get(dev_id, &info),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid return value from "
+ "rte_bbdev_info_get function: %i", return_value);
+
+ TEST_ASSERT_FAIL(info.started, "Failed test for rte_bbdev_info_get: "
+ "started value: %u", info.started);
+
+ TEST_ASSERT_FAIL(rte_bbdev_setup_queues(dev_id,
+ info.drv.max_num_queues, SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "device should be started, dev_id: %u", dev_id);
+
+ return_value = rte_bbdev_intr_enable(dev_id);
+ TEST_ASSERT(return_value == -EBUSY,
+ "Failed test for rte_bbdev_intr_enable: device should be started, dev_id: %u",
+ dev_id);
+
+ /* Stop again the device so it can be once again configured */
+ TEST_ASSERT_FAIL(rte_bbdev_stop(RTE_BBDEV_MAX_DEVS),
+ "Failed to start bbdev %u", dev_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_stop(dev_id), "Failed to stop bbdev %u",
+ dev_id);
+
+ TEST_ASSERT_SUCCESS(return_value = rte_bbdev_info_get(dev_id, &info),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid return value from "
+ "rte_bbdev_info_get function: %i", return_value);
+
+ TEST_ASSERT_SUCCESS(info.started, "Failed test for rte_bbdev_info_get: "
+ "started value: %u", info.started);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_setup_queues(dev_id,
+ info.drv.max_num_queues, SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "device should be stopped, dev_id: %u", dev_id);
+
+ return_value = rte_bbdev_intr_enable(dev_id);
+ TEST_ASSERT(return_value != -EBUSY,
+ "Failed test for rte_bbdev_intr_enable: device should be stopped, dev_id: %u",
+ dev_id);
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_configure_stop_queue(void)
+{
+ struct bbdev_testsuite_params *ts_params = &testsuite_params;
+ struct rte_bbdev_info info;
+ struct rte_bbdev_queue_info qinfo;
+ uint8_t dev_id;
+ uint16_t queue_id;
+ int return_value;
+
+ /* Valid dev_id values */
+ dev_id = null_dev_id;
+
+ /* Valid queue_id values */
+ queue_id = 0;
+
+ rte_bbdev_stop(dev_id);
+ TEST_ASSERT_SUCCESS(return_value = rte_bbdev_info_get(dev_id, &info),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid return value:%i", return_value);
+
+ /* Valid queue configuration */
+ ts_params->qconf.queue_size = info.drv.queue_size_lim;
+ ts_params->qconf.priority = info.drv.max_queue_priority;
+
+ /* Device - started; queue - started */
+ rte_bbdev_start(dev_id);
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "queue:%u on device:%u should be stopped",
+ queue_id, dev_id);
+
+ /* Device - stopped; queue - started */
+ rte_bbdev_stop(dev_id);
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "queue:%u on device:%u should be stopped",
+ queue_id, dev_id);
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_stop(RTE_BBDEV_MAX_DEVS, queue_id),
+ "Failed test for rte_bbdev_queue_stop "
+ "invalid dev_id ");
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_stop(dev_id, RTE_MAX_QUEUES_PER_PORT),
+ "Failed test for rte_bbdev_queue_stop "
+ "invalid queue_id ");
+
+ /* Device - stopped; queue - stopped */
+ rte_bbdev_queue_stop(dev_id, queue_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "queue:%u on device:%u should be stopped", queue_id,
+ dev_id);
+
+ TEST_ASSERT_SUCCESS(return_value = rte_bbdev_queue_info_get(dev_id,
+ queue_id, &qinfo),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid return value from "
+ "rte_bbdev_queue_info_get function: %i", return_value);
+
+ TEST_ASSERT(qinfo.conf.socket == ts_params->qconf.socket,
+ "Failed test for rte_bbdev_queue_info_get: "
+ "invalid queue_size:%u", qinfo.conf.socket);
+
+ TEST_ASSERT(qinfo.conf.queue_size == ts_params->qconf.queue_size,
+ "Failed test for rte_bbdev_queue_info_get: "
+ "invalid queue_size:%u", qinfo.conf.queue_size);
+
+ TEST_ASSERT(qinfo.conf.priority == ts_params->qconf.priority,
+ "Failed test for rte_bbdev_queue_info_get: "
+ "invalid queue_size:%u", qinfo.conf.priority);
+
+ TEST_ASSERT(qinfo.conf.deferred_start ==
+ ts_params->qconf.deferred_start,
+ "Failed test for rte_bbdev_queue_info_get: "
+ "invalid queue_size:%u", qinfo.conf.deferred_start);
+
+ /* Device - started; queue - stopped */
+ rte_bbdev_start(dev_id);
+ rte_bbdev_queue_stop(dev_id, queue_id);
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "queue:%u on device:%u should be stopped", queue_id,
+ dev_id);
+
+ rte_bbdev_stop(dev_id);
+
+ /* After rte_bbdev_start(dev_id):
+ * - queue should be still stopped if deferred_start ==
+ */
+ rte_bbdev_start(dev_id);
+
+ TEST_ASSERT_SUCCESS(return_value = rte_bbdev_queue_info_get(dev_id,
+ queue_id, &qinfo),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid return value from "
+ "rte_bbdev_queue_info_get function: %i", return_value);
+
+ TEST_ASSERT(qinfo.started == 0,
+ "Failed test for rte_bbdev_queue_info_get: "
+ "invalid value for qinfo.started:%u", qinfo.started);
+
+ rte_bbdev_stop(dev_id);
+
+ /* After rte_bbdev_start(dev_id):
+ * - queue should be started if deferred_start ==
+ */
+ ts_params->qconf.deferred_start = 0;
+ rte_bbdev_queue_configure(dev_id, queue_id, &ts_params->qconf);
+ rte_bbdev_start(dev_id);
+
+ TEST_ASSERT_SUCCESS(return_value = rte_bbdev_queue_info_get(dev_id,
+ queue_id, &qinfo),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid return value from "
+ "rte_bbdev_queue_info_get function: %i", return_value);
+
+ TEST_ASSERT(qinfo.started == 1,
+ "Failed test for rte_bbdev_queue_info_get: "
+ "invalid value for qinfo.started:%u", qinfo.started);
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_configure_invalid_queue_configure(void)
+{
+ struct bbdev_testsuite_params *ts_params = &testsuite_params;
+ int return_value;
+ struct rte_bbdev_info info;
+ uint8_t dev_id;
+ uint16_t queue_id;
+
+ /* Valid dev_id values */
+ dev_id = null_dev_id;
+
+ /* Valid queue_id values */
+ queue_id = 0;
+
+ rte_bbdev_stop(dev_id);
+
+ TEST_ASSERT_SUCCESS(return_value = rte_bbdev_info_get(dev_id, &info),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid return value:%i", return_value);
+
+ rte_bbdev_queue_stop(dev_id, queue_id);
+
+ ts_params->qconf.queue_size = info.drv.queue_size_lim + 1;
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "invalid value qconf.queue_size: %u",
+ ts_params->qconf.queue_size);
+
+ ts_params->qconf.queue_size = info.drv.queue_size_lim;
+ ts_params->qconf.priority = info.drv.max_queue_priority + 1;
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "invalid value qconf.queue_size: %u",
+ ts_params->qconf.queue_size);
+
+ ts_params->qconf.priority = info.drv.max_queue_priority;
+ queue_id = info.num_queues;
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "invalid value queue_id: %u", queue_id);
+
+ queue_id = 0;
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_configure(dev_id, queue_id, NULL),
+ "Failed test for rte_bbdev_queue_configure: "
+ "NULL qconf structure ");
+
+ ts_params->qconf.socket = RTE_MAX_NUMA_NODES;
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "invalid socket number ");
+
+ ts_params->qconf.socket = SOCKET_ID_ANY;
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "invalid value qconf.queue_size: %u",
+ ts_params->qconf.queue_size);
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(RTE_BBDEV_MAX_DEVS, queue_id,
+ &ts_params->qconf),
+ "Failed test for rte_bbdev_queue_configure: "
+ "invalid dev_id");
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_configure(dev_id, queue_id, NULL),
+ "Failed test for rte_bbdev_queue_configure: "
+ "invalid value qconf.queue_size: %u",
+ ts_params->qconf.queue_size);
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_op_pool(void)
+{
+ struct rte_mempool *mp;
+
+ unsigned int dec_size = sizeof(struct rte_bbdev_dec_op);
+ unsigned int enc_size = sizeof(struct rte_bbdev_enc_op);
+
+ const char *pool_dec = "Test_DEC";
+ const char *pool_enc = "Test_ENC";
+
+ /* Valid pool configuration */
+ uint32_t size = 256;
+ uint32_t cache_size = 128;
+
+ TEST_ASSERT(rte_bbdev_op_pool_create(NULL,
+ RTE_BBDEV_OP_TURBO_DEC, size, cache_size, 0) == NULL,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "NULL name parameter");
+
+ TEST_ASSERT((mp = rte_bbdev_op_pool_create(pool_dec,
+ RTE_BBDEV_OP_TURBO_DEC, size, cache_size, 0)) != NULL,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "returned value is empty");
+
+ TEST_ASSERT(mp->size == size,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "invalid size of the mempool, mp->size: %u", mp->size);
+
+ TEST_ASSERT(mp->cache_size == cache_size,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "invalid size of the mempool, mp->size: %u",
+ mp->cache_size);
+
+ TEST_ASSERT_SUCCESS(strcmp(mp->name, pool_dec),
+ "Failed test for rte_bbdev_op_pool_create: "
+ "invalid name of mempool, mp->name: %s", mp->name);
+
+ TEST_ASSERT(mp->elt_size == dec_size,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "invalid element size for RTE_BBDEV_OP_TURBO_DEC, "
+ "mp->elt_size: %u", mp->elt_size);
+
+ rte_mempool_free(mp);
+
+ TEST_ASSERT((mp = rte_bbdev_op_pool_create(pool_enc,
+ RTE_BBDEV_OP_TURBO_ENC, size, cache_size, 0)) != NULL,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "returned value is empty");
+
+ TEST_ASSERT(mp->elt_size == enc_size,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "invalid element size for RTE_BBDEV_OP_TURBO_ENC, "
+ "mp->elt_size: %u", mp->elt_size);
+
+ rte_mempool_free(mp);
+
+ TEST_ASSERT((mp = rte_bbdev_op_pool_create("Test_NONE",
+ RTE_BBDEV_OP_NONE, size, cache_size, 0)) != NULL,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "returned value is empty for RTE_BBDEV_OP_NONE");
+
+ TEST_ASSERT(mp->elt_size == (enc_size > dec_size ? enc_size : dec_size),
+ "Failed test for rte_bbdev_op_pool_create: "
+ "invalid size for RTE_BBDEV_OP_NONE, mp->elt_size: %u",
+ mp->elt_size);
+
+ rte_mempool_free(mp);
+
+ TEST_ASSERT((mp = rte_bbdev_op_pool_create("Test_INV",
+ RTE_BBDEV_OP_TYPE_COUNT, size, cache_size, 0)) == NULL,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "returned value is not NULL for invalid type");
+
+ /* Invalid pool configuration */
+ size = 128;
+ cache_size = 256;
+
+ TEST_ASSERT((mp = rte_bbdev_op_pool_create("Test_InvSize",
+ RTE_BBDEV_OP_NONE, size, cache_size, 0)) == NULL,
+ "Failed test for rte_bbdev_op_pool_create: "
+ "returned value should be empty "
+ "because size of per-lcore local cache "
+ "is greater than size of the mempool.");
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Create pool of OP types RTE_BBDEV_OP_NONE, RTE_BBDEV_OP_TURBO_DEC and
+ * RTE_BBDEV_OP_TURBO_ENC and check that only ops of that type can be
+ * allocated
+ */
+static int
+test_bbdev_op_type(void)
+{
+ struct rte_mempool *mp_dec;
+
+ const unsigned int OPS_COUNT = 32;
+ struct rte_bbdev_dec_op *dec_ops_arr[OPS_COUNT];
+ struct rte_bbdev_enc_op *enc_ops_arr[OPS_COUNT];
+
+ const char *pool_dec = "Test_op_dec";
+
+ /* Valid pool configuration */
+ uint32_t num_elements = 256;
+ uint32_t cache_size = 128;
+
+ /* mempool type : RTE_BBDEV_OP_TURBO_DEC */
+ mp_dec = rte_bbdev_op_pool_create(pool_dec,
+ RTE_BBDEV_OP_TURBO_DEC, num_elements, cache_size, 0);
+ TEST_ASSERT(mp_dec != NULL, "Failed to create %s mempool", pool_dec);
+
+ TEST_ASSERT(rte_bbdev_dec_op_alloc_bulk(mp_dec, dec_ops_arr, 1) == 0,
+ "Failed test for rte_bbdev_op_alloc_bulk TURBO_DEC: "
+ "OPs type: RTE_BBDEV_OP_TURBO_DEC");
+
+ TEST_ASSERT(rte_bbdev_enc_op_alloc_bulk(mp_dec, enc_ops_arr, 1) != 0,
+ "Failed test for rte_bbdev_op_alloc_bulk TURBO_DEC: "
+ "OPs type: RTE_BBDEV_OP_TURBO_ENC");
+
+ rte_mempool_free(mp_dec);
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_op_pool_size(void)
+{
+ struct rte_mempool *mp_none;
+
+ const unsigned int OPS_COUNT = 128;
+ struct rte_bbdev_enc_op *ops_enc_arr[OPS_COUNT];
+ struct rte_bbdev_enc_op *ops_ext_arr[OPS_COUNT];
+ struct rte_bbdev_enc_op *ops_ext2_arr[OPS_COUNT];
+
+ const char *pool_none = "Test_pool_size";
+
+ /* Valid pool configuration */
+ uint32_t num_elements = 256;
+ uint32_t cache_size = 0;
+
+ /* Create mempool type : RTE_BBDEV_OP_TURBO_ENC, size : 256 */
+ mp_none = rte_bbdev_op_pool_create(pool_none, RTE_BBDEV_OP_TURBO_ENC,
+ num_elements, cache_size, 0);
+ TEST_ASSERT(mp_none != NULL, "Failed to create %s mempool", pool_none);
+
+ /* Add 128 RTE_BBDEV_OP_TURBO_ENC ops */
+ rte_bbdev_enc_op_alloc_bulk(mp_none, ops_enc_arr, OPS_COUNT);
+
+ /* Add 128 RTE_BBDEV_OP_TURBO_ENC ops */
+ TEST_ASSERT(rte_bbdev_enc_op_alloc_bulk(mp_none, ops_ext_arr,
+ OPS_COUNT) == 0,
+ "Failed test for allocating bbdev ops: "
+ "Mempool size: 256, Free : 128, Attempted to add: 128");
+
+ /* Try adding 128 more RTE_BBDEV_OP_TURBO_ENC ops, this should fail */
+ TEST_ASSERT(rte_bbdev_enc_op_alloc_bulk(mp_none, ops_ext2_arr,
+ OPS_COUNT) != 0,
+ "Failed test for allocating bbdev ops: "
+ "Mempool size: 256, Free : 0, Attempted to add: 128");
+
+ /* Free-up 128 RTE_BBDEV_OP_TURBO_ENC ops */
+ rte_bbdev_enc_op_free_bulk(ops_enc_arr, OPS_COUNT);
+
+ /* Try adding 128 RTE_BBDEV_OP_TURBO_DEC ops, this should succeed */
+ /* Cache size > 0 causes reallocation of ops size > 127 fail */
+ TEST_ASSERT(rte_bbdev_enc_op_alloc_bulk(mp_none, ops_ext2_arr,
+ OPS_COUNT) == 0,
+ "Failed test for allocating ops after mempool freed: "
+ "Mempool size: 256, Free : 128, Attempted to add: 128");
+
+ rte_mempool_free(mp_none);
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_count(void)
+{
+ uint8_t num_devs, num_valid_devs = 0;
+
+ for (num_devs = 0; num_devs < RTE_BBDEV_MAX_DEVS; num_devs++) {
+ if (rte_bbdev_is_valid(num_devs))
+ num_valid_devs++;
+ }
+
+ num_devs = rte_bbdev_count();
+ TEST_ASSERT(num_valid_devs == num_devs,
+ "Failed test for rte_bbdev_is_valid: "
+ "invalid num_devs %u ", num_devs);
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_stats(void)
+{
+ uint8_t dev_id = null_dev_id;
+ uint16_t queue_id = 0;
+ struct rte_bbdev_dec_op *dec_ops[4096] = { 0 };
+ struct rte_bbdev_dec_op *dec_proc_ops[4096] = { 0 };
+ struct rte_bbdev_enc_op *enc_ops[4096] = { 0 };
+ struct rte_bbdev_enc_op *enc_proc_ops[4096] = { 0 };
+ uint16_t num_ops = 236;
+ struct rte_bbdev_stats stats;
+ struct bbdev_testsuite_params *ts_params = &testsuite_params;
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_stop(dev_id, queue_id),
+ "Failed to stop queue %u on device %u ", queue_id,
+ dev_id);
+ TEST_ASSERT_SUCCESS(rte_bbdev_stop(dev_id),
+ "Failed to stop bbdev %u ", dev_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed to configure queue %u on device %u ",
+ queue_id, dev_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_start(dev_id),
+ "Failed to start bbdev %u ", dev_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_start(dev_id, queue_id),
+ "Failed to start queue %u on device %u ", queue_id,
+ dev_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_start(dev_id, queue_id),
+ "Failed to start queue %u on device %u ", queue_id,
+ dev_id);
+
+ /* Tests after enqueue operation */
+ rte_bbdev_enqueue_enc_ops(dev_id, queue_id, enc_ops, num_ops);
+ rte_bbdev_enqueue_dec_ops(dev_id, queue_id, dec_ops, num_ops);
+
+ TEST_ASSERT_FAIL(rte_bbdev_stats_get(RTE_BBDEV_MAX_DEVS, &stats),
+ "Failed test for rte_bbdev_stats_get on device %u ",
+ dev_id);
+
+ TEST_ASSERT_FAIL(rte_bbdev_stats_get(dev_id, NULL),
+ "Failed test for rte_bbdev_stats_get on device %u ",
+ dev_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_stats_get(dev_id, &stats),
+ "Failed test for rte_bbdev_stats_get on device %u ",
+ dev_id);
+
+ TEST_ASSERT(stats.enqueued_count == 2 * num_ops,
+ "Failed test for rte_bbdev_enqueue_ops: "
+ "invalid enqueued_count %" PRIu64 " ",
+ stats.enqueued_count);
+
+ TEST_ASSERT(stats.dequeued_count == 0,
+ "Failed test for rte_bbdev_stats_reset: "
+ "invalid dequeued_count %" PRIu64 " ",
+ stats.dequeued_count);
+
+ /* Tests after dequeue operation */
+ rte_bbdev_dequeue_enc_ops(dev_id, queue_id, enc_proc_ops, num_ops);
+ rte_bbdev_dequeue_dec_ops(dev_id, queue_id, dec_proc_ops, num_ops);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_stats_get(dev_id, &stats),
+ "Failed test for rte_bbdev_stats_get on device %u ",
+ dev_id);
+
+ TEST_ASSERT(stats.dequeued_count == 2 * num_ops,
+ "Failed test for rte_bbdev_dequeue_ops: "
+ "invalid enqueued_count %" PRIu64 " ",
+ stats.dequeued_count);
+
+ TEST_ASSERT(stats.enqueue_err_count == 0,
+ "Failed test for rte_bbdev_stats_reset: "
+ "invalid enqueue_err_count %" PRIu64 " ",
+ stats.enqueue_err_count);
+
+ TEST_ASSERT(stats.dequeue_err_count == 0,
+ "Failed test for rte_bbdev_stats_reset: "
+ "invalid dequeue_err_count %" PRIu64 " ",
+ stats.dequeue_err_count);
+
+ /* Tests after reset operation */
+ TEST_ASSERT_FAIL(rte_bbdev_stats_reset(RTE_BBDEV_MAX_DEVS),
+ "Failed to reset statistic for device %u ", dev_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_stats_reset(dev_id),
+ "Failed to reset statistic for device %u ", dev_id);
+ TEST_ASSERT_SUCCESS(rte_bbdev_stats_get(dev_id, &stats),
+ "Failed test for rte_bbdev_stats_get on device %u ",
+ dev_id);
+
+ TEST_ASSERT(stats.enqueued_count == 0,
+ "Failed test for rte_bbdev_stats_reset: "
+ "invalid enqueued_count %" PRIu64 " ",
+ stats.enqueued_count);
+
+ TEST_ASSERT(stats.dequeued_count == 0,
+ "Failed test for rte_bbdev_stats_reset: "
+ "invalid dequeued_count %" PRIu64 " ",
+ stats.dequeued_count);
+
+ TEST_ASSERT(stats.enqueue_err_count == 0,
+ "Failed test for rte_bbdev_stats_reset: "
+ "invalid enqueue_err_count %" PRIu64 " ",
+ stats.enqueue_err_count);
+
+ TEST_ASSERT(stats.dequeue_err_count == 0,
+ "Failed test for rte_bbdev_stats_reset: "
+ "invalid dequeue_err_count %" PRIu64 " ",
+ stats.dequeue_err_count);
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_driver_init(void)
+{
+ struct rte_bbdev *dev1, *dev2;
+ const char *name = "dev_name";
+ char name_tmp[16];
+ int num_devs, num_devs_tmp;
+
+ dev1 = rte_bbdev_allocate(NULL);
+ TEST_ASSERT(dev1 == NULL,
+ "Failed initialize bbdev driver with NULL name");
+
+ dev1 = rte_bbdev_allocate(name);
+ TEST_ASSERT(dev1 != NULL, "Failed to initialize bbdev driver");
+
+ dev2 = rte_bbdev_allocate(name);
+ TEST_ASSERT(dev2 == NULL,
+ "Failed to initialize bbdev driver: "
+ "driver with the same name has been initialized before");
+
+ num_devs = rte_bbdev_count() - 1;
+ num_devs_tmp = num_devs;
+
+ /* Initialize the maximum amount of devices */
+ do {
+ sprintf(name_tmp, "%s%i", "name_", num_devs);
+ dev2 = rte_bbdev_allocate(name_tmp);
+ TEST_ASSERT(dev2 != NULL,
+ "Failed to initialize bbdev driver");
+ ++num_devs;
+ } while (num_devs < (RTE_BBDEV_MAX_DEVS - 1));
+
+ sprintf(name_tmp, "%s%i", "name_", num_devs);
+ dev2 = rte_bbdev_allocate(name_tmp);
+ TEST_ASSERT(dev2 == NULL, "Failed to initialize bbdev driver number %d "
+ "more drivers than RTE_BBDEV_MAX_DEVS: %d ", num_devs,
+ RTE_BBDEV_MAX_DEVS);
+
+ num_devs--;
+
+ while (num_devs >= num_devs_tmp) {
+ sprintf(name_tmp, "%s%i", "name_", num_devs);
+ dev2 = rte_bbdev_get_named_dev(name_tmp);
+ TEST_ASSERT_SUCCESS(rte_bbdev_release(dev2),
+ "Failed to uninitialize bbdev driver %s ",
+ name_tmp);
+ num_devs--;
+ }
+
+ TEST_ASSERT(dev1->data->dev_id < RTE_BBDEV_MAX_DEVS,
+ "Failed test rte_bbdev_allocate: "
+ "invalid dev_id %" PRIu8 ", max number of devices %d ",
+ dev1->data->dev_id, RTE_BBDEV_MAX_DEVS);
+
+ TEST_ASSERT(dev1->state == RTE_BBDEV_INITALIZED,
+ "Failed test rte_bbdev_allocate: "
+ "invalid state %d (0 - RTE_BBDEV_UNUSED, 1 - RTE_BBDEV_INITALIZED",
+ dev1->state);
+
+ TEST_ASSERT_FAIL(rte_bbdev_release(NULL),
+ "Failed to uninitialize bbdev driver with NULL bbdev");
+
+ sprintf(name_tmp, "%s", "invalid_name");
+ dev2 = rte_bbdev_get_named_dev(name_tmp);
+ TEST_ASSERT_FAIL(rte_bbdev_release(dev2),
+ "Failed to uninitialize bbdev driver with invalid name");
+
+ dev2 = rte_bbdev_get_named_dev(name);
+ TEST_ASSERT_SUCCESS(rte_bbdev_release(dev2),
+ "Failed to uninitialize bbdev driver: %s ", name);
+
+ return TEST_SUCCESS;
+}
+
+static void
+event_callback(uint16_t dev_id, enum rte_bbdev_event_type type, void *param,
+ void *ret_param)
+{
+ RTE_SET_USED(dev_id);
+ RTE_SET_USED(ret_param);
+
+ if (param == NULL)
+ return;
+
+ if (type == RTE_BBDEV_EVENT_UNKNOWN ||
+ type == RTE_BBDEV_EVENT_ERROR ||
+ type == RTE_BBDEV_EVENT_MAX)
+ *(int *)param = type;
+}
+
+static int
+test_bbdev_callback(void)
+{
+ struct rte_bbdev *dev1, *dev2;
+ const char *name = "dev_name1";
+ const char *name2 = "dev_name2";
+ int event_status;
+ uint8_t invalid_dev_id = 7;
+ enum rte_bbdev_event_type invalid_event_type = RTE_BBDEV_EVENT_MAX;
+ uint8_t dev_id;
+
+ dev1 = rte_bbdev_allocate(name);
+ TEST_ASSERT(dev1 != NULL, "Failed to initialize bbdev driver");
+
+ /*
+ * RTE_BBDEV_EVENT_UNKNOWN - unregistered
+ * RTE_BBDEV_EVENT_ERROR - unregistered
+ */
+ event_status = -1;
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_ERROR, NULL);
+ TEST_ASSERT(event_status == -1,
+ "Failed test for rte_bbdev_pmd_callback_process: "
+ "events were not registered ");
+
+ TEST_ASSERT_FAIL(rte_bbdev_callback_register(dev1->data->dev_id,
+ RTE_BBDEV_EVENT_MAX, event_callback, NULL),
+ "Failed to callback register for RTE_BBDEV_EVENT_MAX ");
+
+ TEST_ASSERT_FAIL(rte_bbdev_callback_unregister(dev1->data->dev_id,
+ RTE_BBDEV_EVENT_MAX, event_callback, NULL),
+ "Failed to unregister RTE_BBDEV_EVENT_MAX ");
+
+ /*
+ * RTE_BBDEV_EVENT_UNKNOWN - registered
+ * RTE_BBDEV_EVENT_ERROR - unregistered
+ */
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_register(dev1->data->dev_id,
+ RTE_BBDEV_EVENT_UNKNOWN, event_callback, &event_status),
+ "Failed to callback rgstr for RTE_BBDEV_EVENT_UNKNOWN");
+
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ TEST_ASSERT(event_status == 0,
+ "Failed test for rte_bbdev_pmd_callback_process "
+ "for RTE_BBDEV_EVENT_UNKNOWN ");
+
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_ERROR, NULL);
+ TEST_ASSERT(event_status == 0,
+ "Failed test for rte_bbdev_pmd_callback_process: "
+ "event RTE_BBDEV_EVENT_ERROR was not registered ");
+
+ /*
+ * RTE_BBDEV_EVENT_UNKNOWN - registered
+ * RTE_BBDEV_EVENT_ERROR - registered
+ */
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_register(dev1->data->dev_id,
+ RTE_BBDEV_EVENT_ERROR, event_callback, &event_status),
+ "Failed to callback rgstr for RTE_BBDEV_EVENT_ERROR ");
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_register(dev1->data->dev_id,
+ RTE_BBDEV_EVENT_ERROR, event_callback, &event_status),
+ "Failed to callback register for RTE_BBDEV_EVENT_ERROR"
+ "(re-registration) ");
+
+ event_status = -1;
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ TEST_ASSERT(event_status == 0,
+ "Failed test for rte_bbdev_pmd_callback_process "
+ "for RTE_BBDEV_EVENT_UNKNOWN ");
+
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_ERROR, NULL);
+ TEST_ASSERT(event_status == 1,
+ "Failed test for rte_bbdev_pmd_callback_process "
+ "for RTE_BBDEV_EVENT_ERROR ");
+
+ /*
+ * RTE_BBDEV_EVENT_UNKNOWN - registered
+ * RTE_BBDEV_EVENT_ERROR - unregistered
+ */
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_unregister(dev1->data->dev_id,
+ RTE_BBDEV_EVENT_ERROR, event_callback, &event_status),
+ "Failed to unregister RTE_BBDEV_EVENT_ERROR ");
+
+ event_status = -1;
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ TEST_ASSERT(event_status == 0,
+ "Failed test for rte_bbdev_pmd_callback_process "
+ "for RTE_BBDEV_EVENT_UNKNOWN ");
+
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_ERROR, NULL);
+ TEST_ASSERT(event_status == 0,
+ "Failed test for rte_bbdev_pmd_callback_process: "
+ "event RTE_BBDEV_EVENT_ERROR was unregistered ");
+
+ /* rte_bbdev_callback_register with invalid inputs */
+ TEST_ASSERT_FAIL(rte_bbdev_callback_register(invalid_dev_id,
+ RTE_BBDEV_EVENT_ERROR, event_callback, &event_status),
+ "Failed test for rte_bbdev_callback_register "
+ "for invalid_dev_id ");
+
+ TEST_ASSERT_FAIL(rte_bbdev_callback_register(dev1->data->dev_id,
+ invalid_event_type, event_callback, &event_status),
+ "Failed to callback register for invalid event type ");
+
+ TEST_ASSERT_FAIL(rte_bbdev_callback_register(dev1->data->dev_id,
+ RTE_BBDEV_EVENT_ERROR, NULL, &event_status),
+ "Failed to callback register - no callback function ");
+
+ /* The impact of devices on each other */
+ dev2 = rte_bbdev_allocate(name2);
+ TEST_ASSERT(dev2 != NULL,
+ "Failed to initialize bbdev driver");
+
+ /*
+ * dev2:
+ * RTE_BBDEV_EVENT_UNKNOWN - unregistered
+ * RTE_BBDEV_EVENT_ERROR - unregistered
+ */
+ event_status = -1;
+ rte_bbdev_pmd_callback_process(dev2, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ rte_bbdev_pmd_callback_process(dev2, RTE_BBDEV_EVENT_ERROR, NULL);
+ TEST_ASSERT(event_status == -1,
+ "Failed test for rte_bbdev_pmd_callback_process: "
+ "events were not registered ");
+
+ /*
+ * dev1: RTE_BBDEV_EVENT_ERROR - unregistered
+ * dev2: RTE_BBDEV_EVENT_ERROR - registered
+ */
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_register(dev2->data->dev_id,
+ RTE_BBDEV_EVENT_ERROR, event_callback, &event_status),
+ "Failed to callback rgstr for RTE_BBDEV_EVENT_ERROR");
+
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_ERROR, NULL);
+ TEST_ASSERT(event_status == -1,
+ "Failed test for rte_bbdev_pmd_callback_process in dev1 "
+ "for RTE_BBDEV_EVENT_ERROR ");
+
+ rte_bbdev_pmd_callback_process(dev2, RTE_BBDEV_EVENT_ERROR, NULL);
+ TEST_ASSERT(event_status == 1,
+ "Failed test for rte_bbdev_pmd_callback_process in dev2 "
+ "for RTE_BBDEV_EVENT_ERROR ");
+
+ /*
+ * dev1: RTE_BBDEV_EVENT_UNKNOWN - registered
+ * dev2: RTE_BBDEV_EVENT_UNKNOWN - unregistered
+ */
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_register(dev2->data->dev_id,
+ RTE_BBDEV_EVENT_UNKNOWN, event_callback, &event_status),
+ "Failed to callback register for RTE_BBDEV_EVENT_UNKNOWN "
+ "in dev 2 ");
+
+ rte_bbdev_pmd_callback_process(dev2, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ TEST_ASSERT(event_status == 0,
+ "Failed test for rte_bbdev_pmd_callback_process in dev2"
+ " for RTE_BBDEV_EVENT_UNKNOWN ");
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_unregister(dev2->data->dev_id,
+ RTE_BBDEV_EVENT_UNKNOWN, event_callback, &event_status),
+ "Failed to unregister RTE_BBDEV_EVENT_UNKNOWN ");
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_unregister(dev2->data->dev_id,
+ RTE_BBDEV_EVENT_UNKNOWN, event_callback, &event_status),
+ "Failed to unregister RTE_BBDEV_EVENT_UNKNOWN : "
+ "unregister function called once again ");
+
+ event_status = -1;
+ rte_bbdev_pmd_callback_process(dev2, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ TEST_ASSERT(event_status == -1,
+ "Failed test for rte_bbdev_pmd_callback_process in dev2"
+ " for RTE_BBDEV_EVENT_UNKNOWN ");
+
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ TEST_ASSERT(event_status == 0,
+ "Failed test for rte_bbdev_pmd_callback_process in dev2 "
+ "for RTE_BBDEV_EVENT_UNKNOWN ");
+
+ /* rte_bbdev_pmd_callback_process with invalid inputs */
+ rte_bbdev_pmd_callback_process(NULL, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+
+ event_status = -1;
+ rte_bbdev_pmd_callback_process(dev1, invalid_event_type, NULL);
+ TEST_ASSERT(event_status == -1,
+ "Failed test for rte_bbdev_pmd_callback_process: "
+ "for invalid event type ");
+
+ /* rte_dev_callback_unregister with invalid inputs */
+ TEST_ASSERT_FAIL(rte_bbdev_callback_unregister(invalid_dev_id,
+ RTE_BBDEV_EVENT_UNKNOWN, event_callback, &event_status),
+ "Failed test for rte_dev_callback_unregister "
+ "for invalid_dev_id ");
+
+ TEST_ASSERT_FAIL(rte_bbdev_callback_unregister(dev1->data->dev_id,
+ invalid_event_type, event_callback, &event_status),
+ "Failed rte_dev_callback_unregister "
+ "for invalid event type ");
+
+ TEST_ASSERT_FAIL(rte_bbdev_callback_unregister(dev1->data->dev_id,
+ invalid_event_type, NULL, &event_status),
+ "Failed rte_dev_callback_unregister "
+ "when no callback function ");
+
+ dev_id = dev1->data->dev_id;
+
+ rte_bbdev_release(dev1);
+ rte_bbdev_release(dev2);
+
+ TEST_ASSERT_FAIL(rte_bbdev_callback_register(dev_id,
+ RTE_BBDEV_EVENT_ERROR, event_callback, &event_status),
+ "Failed test for rte_bbdev_callback_register: "
+ "function called after rte_bbdev_driver_uninit .");
+
+ TEST_ASSERT_FAIL(rte_bbdev_callback_unregister(dev_id,
+ RTE_BBDEV_EVENT_ERROR, event_callback, &event_status),
+ "Failed test for rte_dev_callback_unregister: "
+ "function called after rte_bbdev_driver_uninit. ");
+
+ event_status = -1;
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ rte_bbdev_pmd_callback_process(dev1, RTE_BBDEV_EVENT_ERROR, NULL);
+ rte_bbdev_pmd_callback_process(dev2, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+ rte_bbdev_pmd_callback_process(dev2, RTE_BBDEV_EVENT_ERROR, NULL);
+ TEST_ASSERT(event_status == -1,
+ "Failed test for rte_bbdev_pmd_callback_process: "
+ "callback function was called after rte_bbdev_driver_uninit");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_invalid_driver(void)
+{
+ struct rte_bbdev dev1, *dev2;
+ uint8_t dev_id = null_dev_id;
+ uint16_t queue_id = 0;
+ struct rte_bbdev_stats stats;
+ struct bbdev_testsuite_params *ts_params = &testsuite_params;
+ struct rte_bbdev_queue_info qinfo;
+ struct rte_bbdev_ops dev_ops_tmp;
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_stop(dev_id), "Failed to stop bbdev %u ",
+ dev_id);
+
+ dev1 = rte_bbdev_devices[dev_id];
+ dev2 = &rte_bbdev_devices[dev_id];
+
+ /* Tests for rte_bbdev_setup_queues */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_FAIL(rte_bbdev_setup_queues(dev_id, 1, SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "NULL dev_ops structure ");
+ dev2->dev_ops = dev1.dev_ops;
+
+ dev_ops_tmp = *dev2->dev_ops;
+ dev_ops_tmp.info_get = NULL;
+ dev2->dev_ops = &dev_ops_tmp;
+ TEST_ASSERT_FAIL(rte_bbdev_setup_queues(dev_id, 1, SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "NULL info_get ");
+ dev2->dev_ops = dev1.dev_ops;
+
+ dev_ops_tmp = *dev2->dev_ops;
+ dev_ops_tmp.queue_release = NULL;
+ dev2->dev_ops = &dev_ops_tmp;
+ TEST_ASSERT_FAIL(rte_bbdev_setup_queues(dev_id, 1, SOCKET_ID_ANY),
+ "Failed test for rte_bbdev_setup_queues: "
+ "NULL queue_release ");
+ dev2->dev_ops = dev1.dev_ops;
+
+ dev2->data->socket_id = SOCKET_ID_ANY;
+ TEST_ASSERT_SUCCESS(rte_bbdev_setup_queues(dev_id, 1,
+ SOCKET_ID_ANY), "Failed to configure bbdev %u", dev_id);
+
+ /* Test for rte_bbdev_queue_configure */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed to configure queue %u on device %u "
+ "with NULL dev_ops structure ", queue_id, dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ dev_ops_tmp = *dev2->dev_ops;
+ dev_ops_tmp.queue_setup = NULL;
+ dev2->dev_ops = &dev_ops_tmp;
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed to configure queue %u on device %u "
+ "with NULL queue_setup ", queue_id, dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ dev_ops_tmp = *dev2->dev_ops;
+ dev_ops_tmp.info_get = NULL;
+ dev2->dev_ops = &dev_ops_tmp;
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed to configure queue %u on device %u "
+ "with NULL info_get ", queue_id, dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_configure(RTE_BBDEV_MAX_DEVS,
+ queue_id, &ts_params->qconf),
+ "Failed to configure queue %u on device %u ",
+ queue_id, dev_id);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_configure(dev_id, queue_id,
+ &ts_params->qconf),
+ "Failed to configure queue %u on device %u ",
+ queue_id, dev_id);
+
+ /* Test for rte_bbdev_queue_info_get */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_info_get(dev_id, queue_id, &qinfo),
+ "Failed test for rte_bbdev_info_get: "
+ "NULL dev_ops structure ");
+ dev2->dev_ops = dev1.dev_ops;
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_info_get(RTE_BBDEV_MAX_DEVS,
+ queue_id, &qinfo),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid dev_id ");
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_info_get(dev_id,
+ RTE_MAX_QUEUES_PER_PORT, &qinfo),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid queue_id ");
+
+ TEST_ASSERT_FAIL(rte_bbdev_queue_info_get(dev_id, queue_id, NULL),
+ "Failed test for rte_bbdev_info_get: "
+ "invalid dev_info ");
+
+ /* Test for rte_bbdev_start */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_FAIL(rte_bbdev_start(dev_id),
+ "Failed to start bbdev %u "
+ "with NULL dev_ops structure ", dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_start(dev_id),
+ "Failed to start bbdev %u ", dev_id);
+
+ /* Test for rte_bbdev_queue_start */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_FAIL(rte_bbdev_queue_start(dev_id, queue_id),
+ "Failed to start queue %u on device %u: "
+ "NULL dev_ops structure", queue_id, dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_start(dev_id, queue_id),
+ "Failed to start queue %u on device %u ", queue_id,
+ dev_id);
+
+ /* Tests for rte_bbdev_stats_get */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_FAIL(rte_bbdev_stats_get(dev_id, &stats),
+ "Failed test for rte_bbdev_stats_get on device %u ",
+ dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ dev_ops_tmp = *dev2->dev_ops;
+ dev_ops_tmp.stats_reset = NULL;
+ dev2->dev_ops = &dev_ops_tmp;
+ TEST_ASSERT_SUCCESS(rte_bbdev_stats_get(dev_id, &stats),
+ "Failed test for rte_bbdev_stats_get: "
+ "NULL stats_get ");
+ dev2->dev_ops = dev1.dev_ops;
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_stats_get(dev_id, &stats),
+ "Failed test for rte_bbdev_stats_get on device %u ",
+ dev_id);
+
+ /*
+ * Tests for:
+ * rte_bbdev_callback_register,
+ * rte_bbdev_pmd_callback_process,
+ * rte_dev_callback_unregister
+ */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_register(dev_id,
+ RTE_BBDEV_EVENT_UNKNOWN, event_callback, NULL),
+ "Failed to callback rgstr for RTE_BBDEV_EVENT_UNKNOWN");
+ rte_bbdev_pmd_callback_process(dev2, RTE_BBDEV_EVENT_UNKNOWN, NULL);
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_callback_unregister(dev_id,
+ RTE_BBDEV_EVENT_UNKNOWN, event_callback, NULL),
+ "Failed to unregister RTE_BBDEV_EVENT_ERROR ");
+ dev2->dev_ops = dev1.dev_ops;
+
+ /* Tests for rte_bbdev_stats_reset */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_FAIL(rte_bbdev_stats_reset(dev_id),
+ "Failed to reset statistic for device %u ", dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ dev_ops_tmp = *dev2->dev_ops;
+ dev_ops_tmp.stats_reset = NULL;
+ dev2->dev_ops = &dev_ops_tmp;
+ TEST_ASSERT_SUCCESS(rte_bbdev_stats_reset(dev_id),
+ "Failed test for rte_bbdev_stats_reset: "
+ "NULL stats_reset ");
+ dev2->dev_ops = dev1.dev_ops;
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_stats_reset(dev_id),
+ "Failed to reset statistic for device %u ", dev_id);
+
+ /* Tests for rte_bbdev_queue_stop */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_FAIL(rte_bbdev_queue_stop(dev_id, queue_id),
+ "Failed to stop queue %u on device %u: "
+ "NULL dev_ops structure", queue_id, dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_stop(dev_id, queue_id),
+ "Failed to stop queue %u on device %u ", queue_id,
+ dev_id);
+
+ /* Tests for rte_bbdev_stop */
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_FAIL(rte_bbdev_stop(dev_id),
+ "Failed to stop bbdev %u with NULL dev_ops structure ",
+ dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_stop(dev_id),
+ "Failed to stop bbdev %u ", dev_id);
+
+ /* Tests for rte_bbdev_close */
+ TEST_ASSERT_FAIL(rte_bbdev_close(RTE_BBDEV_MAX_DEVS),
+ "Failed to close bbdev with invalid dev_id");
+
+ dev2->dev_ops = NULL;
+ TEST_ASSERT_FAIL(rte_bbdev_close(dev_id),
+ "Failed to close bbdev %u with NULL dev_ops structure ",
+ dev_id);
+ dev2->dev_ops = dev1.dev_ops;
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_close(dev_id),
+ "Failed to close bbdev %u ", dev_id);
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_bbdev_get_named_dev(void)
+{
+ struct rte_bbdev *dev, *dev_tmp;
+ const char *name = "name";
+
+ dev = rte_bbdev_allocate(name);
+ TEST_ASSERT(dev != NULL, "Failed to initialize bbdev driver");
+
+ dev_tmp = rte_bbdev_get_named_dev(NULL);
+ TEST_ASSERT(dev_tmp == NULL, "Failed test for rte_bbdev_get_named_dev: "
+ "function called with NULL parameter");
+
+ dev_tmp = rte_bbdev_get_named_dev(name);
+
+ TEST_ASSERT(dev == dev_tmp, "Failed test for rte_bbdev_get_named_dev: "
+ "wrong device was returned ");
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_release(dev),
+ "Failed to uninitialize bbdev driver %s ", name);
+
+ return TEST_SUCCESS;
+}
+
+static struct unit_test_suite bbdev_null_testsuite = {
+ .suite_name = "BBDEV NULL Unit Test Suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+
+ TEST_CASE(test_bbdev_configure_invalid_dev_id),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_configure_invalid_num_queues),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_configure_stop_device),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_configure_stop_queue),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_configure_invalid_queue_configure),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_op_pool),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_op_type),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_op_pool_size),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_stats),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_driver_init),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_callback),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_invalid_driver),
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_bbdev_get_named_dev),
+
+ TEST_CASE(test_bbdev_count),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+REGISTER_TEST_COMMAND(unittest, bbdev_null_testsuite);
diff --git a/app/test-bbdev/test_bbdev_perf.c b/app/test-bbdev/test_bbdev_perf.c
new file mode 100644
index 0000000..1ab67ca
--- /dev/null
+++ b/app/test-bbdev/test_bbdev_perf.c
@@ -0,0 +1,2090 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <inttypes.h>
+
+#include <rte_eal.h>
+#include <rte_common.h>
+#include <rte_dev.h>
+#include <rte_launch.h>
+#include <rte_bbdev.h>
+#include <rte_cycles.h>
+#include <rte_lcore.h>
+#include <rte_malloc.h>
+#include <rte_random.h>
+#include <rte_hexdump.h>
+
+#include "main.h"
+#include "test_bbdev_vector.h"
+
+#define GET_SOCKET(socket_id) (((socket_id) == SOCKET_ID_ANY) ? 0 : (socket_id))
+
+#define MAX_QUEUES RTE_MAX_LCORE
+
+#define OPS_CACHE_SIZE 256U
+#define OPS_POOL_SIZE_MIN 511U /* 0.5K per queue */
+
+#define SYNC_WAIT 0
+#define SYNC_START 1
+
+#define INVALID_QUEUE_ID -1
+
+static struct test_bbdev_vector test_vector;
+
+/* Switch between PMD and Interrupt for throughput TC */
+static bool intr_enabled;
+
+/* Represents tested active devices */
+static struct active_device {
+ const char *driver_name;
+ uint8_t dev_id;
+ uint16_t supported_ops;
+ uint16_t queue_ids[MAX_QUEUES];
+ uint16_t nb_queues;
+ struct rte_mempool *ops_mempool;
+ struct rte_mempool *in_mbuf_pool;
+ struct rte_mempool *hard_out_mbuf_pool;
+ struct rte_mempool *soft_out_mbuf_pool;
+} active_devs[RTE_BBDEV_MAX_DEVS];
+
+static uint8_t nb_active_devs;
+
+/* Data buffers used by BBDEV ops */
+struct test_buffers {
+ struct rte_bbdev_op_data *inputs;
+ struct rte_bbdev_op_data *hard_outputs;
+ struct rte_bbdev_op_data *soft_outputs;
+};
+
+/* Operation parameters specific for given test case */
+struct test_op_params {
+ struct rte_mempool *mp;
+ struct rte_bbdev_dec_op *ref_dec_op;
+ struct rte_bbdev_enc_op *ref_enc_op;
+ uint16_t burst_sz;
+ uint16_t num_to_process;
+ int vector_mask;
+ rte_atomic16_t sync;
+ struct test_buffers q_bufs[RTE_MAX_NUMA_NODES][MAX_QUEUES];
+};
+
+/* Contains per lcore params */
+struct thread_params {
+ uint8_t dev_id;
+ uint16_t queue_id;
+ uint64_t start_time;
+ rte_atomic16_t nb_dequeued;
+ rte_atomic16_t processing_status;
+ struct test_op_params *op_params;
+};
+
+typedef int (test_case_function)(struct active_device *ad,
+ struct test_op_params *op_params);
+
+static inline void
+set_avail_op(struct active_device *ad, enum rte_bbdev_op_type op_type)
+{
+ ad->supported_ops |= (1 << op_type);
+}
+
+static inline bool
+is_avail_op(struct active_device *ad, enum rte_bbdev_op_type op_type)
+{
+ return ad->supported_ops & (1 << op_type);
+}
+
+/* calculates optimal mempool size not smaller than the val */
+static unsigned int
+optimal_mempool_size(unsigned int val)
+{
+ return rte_align32pow2(val + 1) - 1;
+}
+
+/* allocates mbuf mempool for inputs and outputs */
+static struct rte_mempool *
+create_mbuf_pool(struct op_data_entries *entries, uint8_t dev_id,
+ int socket_id, unsigned int mbuf_pool_size,
+ const char *op_type_str)
+{
+ unsigned int i;
+ uint32_t max_seg_sz = 0;
+ char pool_name[RTE_MEMPOOL_NAMESIZE];
+
+ /* find max input segment size */
+ for (i = 0; i < entries->nb_segments; ++i)
+ if (entries->segments[i].length > max_seg_sz)
+ max_seg_sz = entries->segments[i].length;
+
+ snprintf(pool_name, sizeof(pool_name), "%s_pool_%u", op_type_str,
+ dev_id);
+ return rte_pktmbuf_pool_create(pool_name, mbuf_pool_size, 0, 0,
+ RTE_MAX(max_seg_sz + RTE_PKTMBUF_HEADROOM,
+ (unsigned int)RTE_MBUF_DEFAULT_BUF_SIZE), socket_id);
+}
+
+static int
+create_mempools(struct active_device *ad, int socket_id,
+ enum rte_bbdev_op_type op_type, uint16_t num_ops)
+{
+ struct rte_mempool *mp;
+ unsigned int ops_pool_size, mbuf_pool_size = 0;
+ char pool_name[RTE_MEMPOOL_NAMESIZE];
+
+ struct op_data_entries *in = &test_vector.entries[DATA_INPUT];
+ struct op_data_entries *hard_out =
+ &test_vector.entries[DATA_HARD_OUTPUT];
+ struct op_data_entries *soft_out =
+ &test_vector.entries[DATA_SOFT_OUTPUT];
+
+ /* allocate ops mempool */
+ ops_pool_size = optimal_mempool_size(RTE_MAX(
+ /* Ops used plus 1 reference op */
+ RTE_MAX((unsigned int)(ad->nb_queues * num_ops + 1),
+ /* Minimal cache size plus 1 reference op */
+ (unsigned int)(1.5 * rte_lcore_count() *
+ OPS_CACHE_SIZE + 1)),
+ OPS_POOL_SIZE_MIN));
+ snprintf(pool_name, sizeof(pool_name), "%s_pool_%u",
+ rte_bbdev_op_type_str(op_type), ad->dev_id);
+ mp = rte_bbdev_op_pool_create(pool_name, op_type,
+ ops_pool_size, OPS_CACHE_SIZE, socket_id);
+ TEST_ASSERT_NOT_NULL(mp,
+ "ERROR Failed to create %u items ops pool for dev %u on socket %u.",
+ ops_pool_size,
+ ad->dev_id,
+ socket_id);
+ ad->ops_mempool = mp;
+
+ /* Inputs */
+ mbuf_pool_size = optimal_mempool_size(ops_pool_size * in->nb_segments);
+ mp = create_mbuf_pool(in, ad->dev_id, socket_id, mbuf_pool_size, "in");
+ TEST_ASSERT_NOT_NULL(mp,
+ "ERROR Failed to create %u items input pktmbuf pool for dev %u on socket %u.",
+ mbuf_pool_size,
+ ad->dev_id,
+ socket_id);
+ ad->in_mbuf_pool = mp;
+
+ /* Hard outputs */
+ mbuf_pool_size = optimal_mempool_size(ops_pool_size *
+ hard_out->nb_segments);
+ mp = create_mbuf_pool(hard_out, ad->dev_id, socket_id, mbuf_pool_size,
+ "hard_out");
+ TEST_ASSERT_NOT_NULL(mp,
+ "ERROR Failed to create %u items hard output pktmbuf pool for dev %u on socket %u.",
+ mbuf_pool_size,
+ ad->dev_id,
+ socket_id);
+ ad->hard_out_mbuf_pool = mp;
+
+ if (soft_out->nb_segments == 0)
+ return TEST_SUCCESS;
+
+ /* Soft outputs */
+ mbuf_pool_size = optimal_mempool_size(ops_pool_size *
+ soft_out->nb_segments);
+ mp = create_mbuf_pool(soft_out, ad->dev_id, socket_id, mbuf_pool_size,
+ "soft_out");
+ TEST_ASSERT_NOT_NULL(mp,
+ "ERROR Failed to create %uB soft output pktmbuf pool for dev %u on socket %u.",
+ mbuf_pool_size,
+ ad->dev_id,
+ socket_id);
+ ad->soft_out_mbuf_pool = mp;
+
+ return 0;
+}
+
+static int
+add_bbdev_dev(uint8_t dev_id, struct rte_bbdev_info *info,
+ struct test_bbdev_vector *vector)
+{
+ int ret;
+ unsigned int queue_id;
+ struct rte_bbdev_queue_conf qconf;
+ struct active_device *ad = &active_devs[nb_active_devs];
+ unsigned int nb_queues;
+ enum rte_bbdev_op_type op_type = vector->op_type;
+
+ nb_queues = RTE_MIN(rte_lcore_count(), info->drv.max_num_queues);
+ /* setup device */
+ ret = rte_bbdev_setup_queues(dev_id, nb_queues, info->socket_id);
+ if (ret < 0) {
+ printf("rte_bbdev_setup_queues(%u, %u, %d) ret %i\n",
+ dev_id, nb_queues, info->socket_id, ret);
+ return TEST_FAILED;
+ }
+
+ /* configure interrupts if needed */
+ if (intr_enabled) {
+ ret = rte_bbdev_intr_enable(dev_id);
+ if (ret < 0) {
+ printf("rte_bbdev_intr_enable(%u) ret %i\n", dev_id,
+ ret);
+ return TEST_FAILED;
+ }
+ }
+
+ /* setup device queues */
+ qconf.socket = info->socket_id;
+ qconf.queue_size = info->drv.default_queue_conf.queue_size;
+ qconf.priority = 0;
+ qconf.deferred_start = 0;
+ qconf.op_type = op_type;
+
+ for (queue_id = 0; queue_id < nb_queues; ++queue_id) {
+ ret = rte_bbdev_queue_configure(dev_id, queue_id, &qconf);
+ if (ret != 0) {
+ printf(
+ "Allocated all queues (id=%u) at prio%u on dev%u\n",
+ queue_id, qconf.priority, dev_id);
+ qconf.priority++;
+ ret = rte_bbdev_queue_configure(ad->dev_id, queue_id,
+ &qconf);
+ }
+ if (ret != 0) {
+ printf("All queues on dev %u allocated: %u\n",
+ dev_id, queue_id);
+ break;
+ }
+ ad->queue_ids[queue_id] = queue_id;
+ }
+ TEST_ASSERT(queue_id != 0,
+ "ERROR Failed to configure any queues on dev %u",
+ dev_id);
+ ad->nb_queues = queue_id;
+
+ set_avail_op(ad, op_type);
+
+ return TEST_SUCCESS;
+}
+
+static int
+add_active_device(uint8_t dev_id, struct rte_bbdev_info *info,
+ struct test_bbdev_vector *vector)
+{
+ int ret;
+
+ active_devs[nb_active_devs].driver_name = info->drv.driver_name;
+ active_devs[nb_active_devs].dev_id = dev_id;
+
+ ret = add_bbdev_dev(dev_id, info, vector);
+ if (ret == TEST_SUCCESS)
+ ++nb_active_devs;
+ return ret;
+}
+
+static inline bool
+flags_match(uint32_t flags_req, uint32_t flags_present)
+{
+ return (flags_req & flags_present) == flags_req;
+}
+
+static int
+check_dev_cap(const struct rte_bbdev_info *dev_info)
+{
+ unsigned int i;
+ unsigned int nb_inputs, nb_soft_outputs, nb_hard_outputs;
+ const struct rte_bbdev_op_cap *op_cap = dev_info->drv.capabilities;
+
+ nb_inputs = test_vector.entries[DATA_INPUT].nb_segments;
+ nb_soft_outputs = test_vector.entries[DATA_SOFT_OUTPUT].nb_segments;
+ nb_hard_outputs = test_vector.entries[DATA_HARD_OUTPUT].nb_segments;
+
+ for (i = 0; op_cap->type != RTE_BBDEV_OP_NONE; ++i, ++op_cap) {
+ if (op_cap->type != test_vector.op_type)
+ continue;
+
+ if (op_cap->type == RTE_BBDEV_OP_TURBO_DEC) {
+ const struct rte_bbdev_op_cap_turbo_dec *cap =
+ &op_cap->cap.turbo_dec;
+ /* Ignore lack of soft output capability, just skip
+ * checking if soft output is valid.
+ */
+ if ((test_vector.turbo_dec.op_flags &
+ RTE_BBDEV_TURBO_SOFT_OUTPUT) &&
+ !(cap->capability_flags &
+ RTE_BBDEV_TURBO_SOFT_OUTPUT)) {
+ printf(
+ "WARNING: Device \"%s\" does not support soft output, RTE_BBDEV_TURBO_SOFT_OUTPUT flag will be ignored.\n",
+ dev_info->dev_name);
+ test_vector.turbo_dec.op_flags &=
+ ~RTE_BBDEV_TURBO_SOFT_OUTPUT;
+ }
+
+ if (!flags_match(test_vector.turbo_dec.op_flags,
+ cap->capability_flags))
+ return TEST_FAILED;
+ if (nb_inputs > cap->num_buffers_src) {
+ printf("Too many inputs defined: %u, max: %u\n",
+ nb_inputs, cap->num_buffers_src);
+ return TEST_FAILED;
+ }
+ if (nb_soft_outputs > cap->num_buffers_soft_out &&
+ (test_vector.turbo_dec.op_flags &
+ RTE_BBDEV_TURBO_SOFT_OUTPUT)) {
+ printf(
+ "Too many soft outputs defined: %u, max: %u\n",
+ nb_soft_outputs,
+ cap->num_buffers_soft_out);
+ return TEST_FAILED;
+ }
+ if (nb_hard_outputs > cap->num_buffers_hard_out) {
+ printf(
+ "Too many hard outputs defined: %u, max: %u\n",
+ nb_hard_outputs,
+ cap->num_buffers_hard_out);
+ return TEST_FAILED;
+ }
+ if (intr_enabled && !(cap->capability_flags &
+ RTE_BBDEV_TURBO_DEC_INTERRUPTS)) {
+ printf(
+ "Dequeue interrupts are not supported!\n");
+ return TEST_FAILED;
+ }
+
+ return TEST_SUCCESS;
+ } else if (op_cap->type == RTE_BBDEV_OP_TURBO_ENC) {
+ const struct rte_bbdev_op_cap_turbo_enc *cap =
+ &op_cap->cap.turbo_enc;
+
+ if (!flags_match(test_vector.turbo_enc.op_flags,
+ cap->capability_flags))
+ return TEST_FAILED;
+ if (nb_inputs > cap->num_buffers_src) {
+ printf("Too many inputs defined: %u, max: %u\n",
+ nb_inputs, cap->num_buffers_src);
+ return TEST_FAILED;
+ }
+ if (nb_hard_outputs > cap->num_buffers_dst) {
+ printf(
+ "Too many hard outputs defined: %u, max: %u\n",
+ nb_hard_outputs, cap->num_buffers_src);
+ return TEST_FAILED;
+ }
+ if (intr_enabled && !(cap->capability_flags &
+ RTE_BBDEV_TURBO_ENC_INTERRUPTS)) {
+ printf(
+ "Dequeue interrupts are not supported!\n");
+ return TEST_FAILED;
+ }
+
+ return TEST_SUCCESS;
+ }
+ }
+
+ if ((i == 0) && (test_vector.op_type == RTE_BBDEV_OP_NONE))
+ return TEST_SUCCESS; /* Special case for NULL device */
+
+ return TEST_FAILED;
+}
+
+static uint8_t
+populate_active_devices(void)
+{
+ int ret;
+ uint8_t dev_id;
+ uint8_t nb_devs_added = 0;
+ struct rte_bbdev_info info;
+
+ RTE_BBDEV_FOREACH(dev_id) {
+ rte_bbdev_info_get(dev_id, &info);
+
+ if (check_dev_cap(&info)) {
+ printf(
+ "Device %d (%s) does not support specified capabilities\n",
+ dev_id, info.dev_name);
+ continue;
+ }
+
+ ret = add_active_device(dev_id, &info, &test_vector);
+ if (ret != 0) {
+ printf("Adding active bbdev %s skipped\n",
+ info.dev_name);
+ continue;
+ }
+ nb_devs_added++;
+ }
+
+ return nb_devs_added;
+}
+
+static int
+read_test_vector(void)
+{
+ int ret;
+
+ memset(&test_vector, 0, sizeof(test_vector));
+ printf("Test vector file = %s\n", get_vector_filename());
+ ret = test_bbdev_vector_read(get_vector_filename(), &test_vector);
+ TEST_ASSERT_SUCCESS(ret, "Failed to parse file %s\n",
+ get_vector_filename());
+
+ return TEST_SUCCESS;
+}
+
+static int
+testsuite_setup(void)
+{
+ TEST_ASSERT_SUCCESS(read_test_vector(), "Test suite setup failed\n");
+
+ if (populate_active_devices() == 0) {
+ printf("No suitable devices found!\n");
+ return TEST_SKIPPED;
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+interrupt_testsuite_setup(void)
+{
+ TEST_ASSERT_SUCCESS(read_test_vector(), "Test suite setup failed\n");
+
+ /* Enable interrupts */
+ intr_enabled = true;
+
+ /* Special case for NULL device (RTE_BBDEV_OP_NONE) */
+ if (populate_active_devices() == 0 ||
+ test_vector.op_type == RTE_BBDEV_OP_NONE) {
+ intr_enabled = false;
+ printf("No suitable devices found!\n");
+ return TEST_SKIPPED;
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+testsuite_teardown(void)
+{
+ uint8_t dev_id;
+
+ /* Unconfigure devices */
+ RTE_BBDEV_FOREACH(dev_id)
+ rte_bbdev_close(dev_id);
+
+ /* Clear active devices structs. */
+ memset(active_devs, 0, sizeof(active_devs));
+ nb_active_devs = 0;
+}
+
+static int
+ut_setup(void)
+{
+ uint8_t i, dev_id;
+
+ for (i = 0; i < nb_active_devs; i++) {
+ dev_id = active_devs[i].dev_id;
+ /* reset bbdev stats */
+ TEST_ASSERT_SUCCESS(rte_bbdev_stats_reset(dev_id),
+ "Failed to reset stats of bbdev %u", dev_id);
+ /* start the device */
+ TEST_ASSERT_SUCCESS(rte_bbdev_start(dev_id),
+ "Failed to start bbdev %u", dev_id);
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+ut_teardown(void)
+{
+ uint8_t i, dev_id;
+ struct rte_bbdev_stats stats;
+
+ for (i = 0; i < nb_active_devs; i++) {
+ dev_id = active_devs[i].dev_id;
+ /* read stats and print */
+ rte_bbdev_stats_get(dev_id, &stats);
+ /* Stop the device */
+ rte_bbdev_stop(dev_id);
+ }
+}
+
+static int
+init_op_data_objs(struct rte_bbdev_op_data *bufs,
+ struct op_data_entries *ref_entries,
+ struct rte_mempool *mbuf_pool, const uint16_t n,
+ enum op_data_type op_type, uint16_t min_alignment)
+{
+ int ret;
+ unsigned int i, j;
+
+ for (i = 0; i < n; ++i) {
+ char *data;
+ struct op_data_buf *seg = &ref_entries->segments[0];
+ struct rte_mbuf *m_head = rte_pktmbuf_alloc(mbuf_pool);
+ TEST_ASSERT_NOT_NULL(m_head,
+ "Not enough mbufs in %d data type mbuf pool (needed %u, available %u)",
+ op_type, n * ref_entries->nb_segments,
+ mbuf_pool->size);
+
+ bufs[i].data = m_head;
+ bufs[i].offset = 0;
+ bufs[i].length = 0;
+
+ if (op_type == DATA_INPUT) {
+ data = rte_pktmbuf_append(m_head, seg->length);
+ TEST_ASSERT_NOT_NULL(data,
+ "Couldn't append %u bytes to mbuf from %d data type mbuf pool",
+ seg->length, op_type);
+
+ TEST_ASSERT(data == RTE_PTR_ALIGN(data, min_alignment),
+ "Data addr in mbuf (%p) is not aligned to device min alignment (%u)",
+ data, min_alignment);
+ rte_memcpy(data, seg->addr, seg->length);
+ bufs[i].length += seg->length;
+
+
+ for (j = 1; j < ref_entries->nb_segments; ++j) {
+ struct rte_mbuf *m_tail =
+ rte_pktmbuf_alloc(mbuf_pool);
+ TEST_ASSERT_NOT_NULL(m_tail,
+ "Not enough mbufs in %d data type mbuf pool (needed %u, available %u)",
+ op_type,
+ n * ref_entries->nb_segments,
+ mbuf_pool->size);
+ seg += 1;
+
+ data = rte_pktmbuf_append(m_tail, seg->length);
+ TEST_ASSERT_NOT_NULL(data,
+ "Couldn't append %u bytes to mbuf from %d data type mbuf pool",
+ seg->length, op_type);
+
+ TEST_ASSERT(data == RTE_PTR_ALIGN(data,
+ min_alignment),
+ "Data addr in mbuf (%p) is not aligned to device min alignment (%u)",
+ data, min_alignment);
+ rte_memcpy(data, seg->addr, seg->length);
+ bufs[i].length += seg->length;
+
+ ret = rte_pktmbuf_chain(m_head, m_tail);
+ TEST_ASSERT_SUCCESS(ret,
+ "Couldn't chain mbufs from %d data type mbuf pool",
+ op_type);
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int
+allocate_buffers_on_socket(struct rte_bbdev_op_data **buffers, const int len,
+ const int socket)
+{
+ int i;
+
+ *buffers = rte_zmalloc_socket(NULL, len, 0, socket);
+ if (*buffers == NULL) {
+ printf("WARNING: Failed to allocate op_data on socket %d\n",
+ socket);
+ /* try to allocate memory on other detected sockets */
+ for (i = 0; i < socket; i++) {
+ *buffers = rte_zmalloc_socket(NULL, len, 0, i);
+ if (*buffers != NULL)
+ break;
+ }
+ }
+
+ return (*buffers == NULL) ? TEST_FAILED : TEST_SUCCESS;
+}
+
+static int
+fill_queue_buffers(struct test_op_params *op_params,
+ struct rte_mempool *in_mp, struct rte_mempool *hard_out_mp,
+ struct rte_mempool *soft_out_mp, uint16_t queue_id,
+ uint16_t min_alignment, const int socket_id)
+{
+ int ret;
+ enum op_data_type type;
+ const uint16_t n = op_params->num_to_process;
+
+ struct rte_mempool *mbuf_pools[DATA_NUM_TYPES] = {
+ in_mp,
+ soft_out_mp,
+ hard_out_mp,
+ };
+
+ struct rte_bbdev_op_data **queue_ops[DATA_NUM_TYPES] = {
+ &op_params->q_bufs[socket_id][queue_id].inputs,
+ &op_params->q_bufs[socket_id][queue_id].soft_outputs,
+ &op_params->q_bufs[socket_id][queue_id].hard_outputs,
+ };
+
+ for (type = DATA_INPUT; type < DATA_NUM_TYPES; ++type) {
+ struct op_data_entries *ref_entries =
+ &test_vector.entries[type];
+ if (ref_entries->nb_segments == 0)
+ continue;
+
+ ret = allocate_buffers_on_socket(queue_ops[type],
+ n * sizeof(struct rte_bbdev_op_data),
+ socket_id);
+ TEST_ASSERT_SUCCESS(ret,
+ "Couldn't allocate memory for rte_bbdev_op_data structs");
+
+ ret = init_op_data_objs(*queue_ops[type], ref_entries,
+ mbuf_pools[type], n, type, min_alignment);
+ TEST_ASSERT_SUCCESS(ret,
+ "Couldn't init rte_bbdev_op_data structs");
+ }
+
+ return 0;
+}
+
+static void
+free_buffers(struct active_device *ad, struct test_op_params *op_params)
+{
+ unsigned int i, j;
+
+ rte_mempool_free(ad->ops_mempool);
+ rte_mempool_free(ad->in_mbuf_pool);
+ rte_mempool_free(ad->hard_out_mbuf_pool);
+ rte_mempool_free(ad->soft_out_mbuf_pool);
+
+ for (i = 0; i < rte_lcore_count(); ++i) {
+ for (j = 0; j < RTE_MAX_NUMA_NODES; ++j) {
+ rte_free(op_params->q_bufs[j][i].inputs);
+ rte_free(op_params->q_bufs[j][i].hard_outputs);
+ rte_free(op_params->q_bufs[j][i].soft_outputs);
+ }
+ }
+}
+
+static void
+copy_reference_dec_op(struct rte_bbdev_dec_op **ops, unsigned int n,
+ unsigned int start_idx,
+ struct rte_bbdev_op_data *inputs,
+ struct rte_bbdev_op_data *hard_outputs,
+ struct rte_bbdev_op_data *soft_outputs,
+ struct rte_bbdev_dec_op *ref_op)
+{
+ unsigned int i;
+ struct rte_bbdev_op_turbo_dec *turbo_dec = &ref_op->turbo_dec;
+
+ for (i = 0; i < n; ++i) {
+ if (turbo_dec->code_block_mode == 0) {
+ ops[i]->turbo_dec.tb_params.ea =
+ turbo_dec->tb_params.ea;
+ ops[i]->turbo_dec.tb_params.eb =
+ turbo_dec->tb_params.eb;
+ ops[i]->turbo_dec.tb_params.k_pos =
+ turbo_dec->tb_params.k_pos;
+ ops[i]->turbo_dec.tb_params.k_neg =
+ turbo_dec->tb_params.k_neg;
+ ops[i]->turbo_dec.tb_params.c =
+ turbo_dec->tb_params.c;
+ ops[i]->turbo_dec.tb_params.c_neg =
+ turbo_dec->tb_params.c_neg;
+ ops[i]->turbo_dec.tb_params.cab =
+ turbo_dec->tb_params.cab;
+ } else {
+ ops[i]->turbo_dec.cb_params.e = turbo_dec->cb_params.e;
+ ops[i]->turbo_dec.cb_params.k = turbo_dec->cb_params.k;
+ }
+
+ ops[i]->turbo_dec.ext_scale = turbo_dec->ext_scale;
+ ops[i]->turbo_dec.iter_max = turbo_dec->iter_max;
+ ops[i]->turbo_dec.iter_min = turbo_dec->iter_min;
+ ops[i]->turbo_dec.op_flags = turbo_dec->op_flags;
+ ops[i]->turbo_dec.rv_index = turbo_dec->rv_index;
+ ops[i]->turbo_dec.num_maps = turbo_dec->num_maps;
+ ops[i]->turbo_dec.code_block_mode = turbo_dec->code_block_mode;
+
+ ops[i]->turbo_dec.hard_output = hard_outputs[start_idx + i];
+ ops[i]->turbo_dec.input = inputs[start_idx + i];
+ if (soft_outputs != NULL)
+ ops[i]->turbo_dec.soft_output =
+ soft_outputs[start_idx + i];
+ }
+}
+
+static void
+copy_reference_enc_op(struct rte_bbdev_enc_op **ops, unsigned int n,
+ unsigned int start_idx,
+ struct rte_bbdev_op_data *inputs,
+ struct rte_bbdev_op_data *outputs,
+ struct rte_bbdev_enc_op *ref_op)
+{
+ unsigned int i;
+ struct rte_bbdev_op_turbo_enc *turbo_enc = &ref_op->turbo_enc;
+ for (i = 0; i < n; ++i) {
+ if (turbo_enc->code_block_mode == 0) {
+ ops[i]->turbo_enc.tb_params.ea =
+ turbo_enc->tb_params.ea;
+ ops[i]->turbo_enc.tb_params.eb =
+ turbo_enc->tb_params.eb;
+ ops[i]->turbo_enc.tb_params.k_pos =
+ turbo_enc->tb_params.k_pos;
+ ops[i]->turbo_enc.tb_params.k_neg =
+ turbo_enc->tb_params.k_neg;
+ ops[i]->turbo_enc.tb_params.c =
+ turbo_enc->tb_params.c;
+ ops[i]->turbo_enc.tb_params.c_neg =
+ turbo_enc->tb_params.c_neg;
+ ops[i]->turbo_enc.tb_params.cab =
+ turbo_enc->tb_params.cab;
+ ops[i]->turbo_enc.tb_params.ncb_pos =
+ turbo_enc->tb_params.ncb_pos;
+ ops[i]->turbo_enc.tb_params.ncb_neg =
+ turbo_enc->tb_params.ncb_neg;
+ /* total number of bits available for the
+ * transmission of one TB
+ */
+ ops[i]->turbo_enc.g = turbo_enc->tb_params.ncb_pos * 3;
+ } else {
+ ops[i]->turbo_enc.cb_params.e = turbo_enc->cb_params.e;
+ ops[i]->turbo_enc.cb_params.k = turbo_enc->cb_params.k;
+ ops[i]->turbo_enc.cb_params.ncb =
+ turbo_enc->cb_params.ncb;
+ /* total number of bits available for the
+ * transmission of one TB
+ */
+ ops[i]->turbo_enc.g = turbo_enc->cb_params.ncb * 3;
+ }
+ /* For UE category 1 number of soft bits is 250368
+ * according to TS 36.306 Table 4.1-1
+ */
+ ops[i]->turbo_enc.n_soft = 250368;
+ ops[i]->turbo_enc.k_mimo = 1;
+ /* Max number of DL HARQ depends on UL/DL configuration
+ * according to TS 36.213 Table 7-1
+ */
+ ops[i]->turbo_enc.mdl_harq = 4;
+ ops[i]->turbo_enc.nl = 1;
+ ops[i]->turbo_enc.qm = 2;
+ ops[i]->turbo_enc.rv_index = turbo_enc->rv_index;
+ ops[i]->turbo_enc.op_flags = turbo_enc->op_flags;
+ ops[i]->turbo_enc.code_block_mode = turbo_enc->code_block_mode;
+
+ ops[i]->turbo_enc.output = outputs[start_idx + i];
+ ops[i]->turbo_enc.input = inputs[start_idx + i];
+ }
+}
+
+static int
+check_dec_status_and_ordering(struct rte_bbdev_dec_op *op,
+ unsigned int order_idx, const int expected_status)
+{
+ TEST_ASSERT(op->status == expected_status,
+ "op_status (%d) != expected_status (%d)",
+ op->status, expected_status);
+
+ TEST_ASSERT((void *)(uintptr_t)order_idx == op->opaque_data,
+ "Ordering error, expected %p, got %p",
+ (void *)(uintptr_t)order_idx, op->opaque_data);
+
+ return TEST_SUCCESS;
+}
+
+static int
+check_enc_status_and_ordering(struct rte_bbdev_enc_op *op,
+ unsigned int order_idx, const int expected_status)
+{
+ TEST_ASSERT(op->status == expected_status,
+ "op_status (%d) != expected_status (%d)",
+ op->status, expected_status);
+
+ TEST_ASSERT((void *)(uintptr_t)order_idx == op->opaque_data,
+ "Ordering error, expected %p, got %p",
+ (void *)(uintptr_t)order_idx, op->opaque_data);
+
+ return TEST_SUCCESS;
+}
+
+static inline int
+validate_op_chain(struct rte_bbdev_op_data *op,
+ struct op_data_entries *orig_op)
+{
+ uint8_t i;
+ struct rte_mbuf *m = op->data;
+ uint8_t nb_dst_segments = orig_op->nb_segments;
+
+ TEST_ASSERT(nb_dst_segments == m->nb_segs,
+ "Number of segments differ in original (%u) and filled (%u) op",
+ nb_dst_segments, m->nb_segs);
+
+ for (i = 0; i < nb_dst_segments; ++i) {
+ /* Apply offset to the first mbuf segment */
+ uint16_t offset = (i == 0) ? op->offset : 0;
+ uint16_t data_len = m->data_len - offset;
+
+ TEST_ASSERT(orig_op->segments[i].length == data_len,
+ "Length of segment differ in original (%u) and filled (%u) op",
+ orig_op->segments[i].length, data_len);
+ TEST_ASSERT_BUFFERS_ARE_EQUAL(orig_op->segments[i].addr,
+ rte_pktmbuf_mtod_offset(m, uint32_t *, offset),
+ data_len,
+ "Output buffers (CB=%u) are not equal", i);
+ m = m->next;
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+validate_dec_buffers(struct rte_bbdev_dec_op *ref_op, struct test_buffers *bufs,
+ const uint16_t num_to_process)
+{
+ int i;
+
+ struct op_data_entries *hard_data_orig =
+ &test_vector.entries[DATA_HARD_OUTPUT];
+ struct op_data_entries *soft_data_orig =
+ &test_vector.entries[DATA_SOFT_OUTPUT];
+
+ for (i = 0; i < num_to_process; i++) {
+ TEST_ASSERT_SUCCESS(validate_op_chain(&bufs->hard_outputs[i],
+ hard_data_orig),
+ "Hard output buffers are not equal");
+ if (ref_op->turbo_dec.op_flags &
+ RTE_BBDEV_TURBO_SOFT_OUTPUT)
+ TEST_ASSERT_SUCCESS(validate_op_chain(
+ &bufs->soft_outputs[i],
+ soft_data_orig),
+ "Soft output buffers are not equal");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+validate_enc_buffers(struct test_buffers *bufs, const uint16_t num_to_process)
+{
+ int i;
+
+ struct op_data_entries *hard_data_orig =
+ &test_vector.entries[DATA_HARD_OUTPUT];
+
+ for (i = 0; i < num_to_process; i++)
+ TEST_ASSERT_SUCCESS(validate_op_chain(&bufs->hard_outputs[i],
+ hard_data_orig), "");
+
+ return TEST_SUCCESS;
+}
+
+static int
+validate_dec_op(struct rte_bbdev_dec_op **ops, const uint16_t n,
+ struct rte_bbdev_dec_op *ref_op, const int vector_mask)
+{
+ unsigned int i;
+ int ret;
+ struct op_data_entries *hard_data_orig =
+ &test_vector.entries[DATA_HARD_OUTPUT];
+ struct op_data_entries *soft_data_orig =
+ &test_vector.entries[DATA_SOFT_OUTPUT];
+ struct rte_bbdev_op_turbo_dec *ops_td;
+ struct rte_bbdev_op_data *hard_output;
+ struct rte_bbdev_op_data *soft_output;
+ struct rte_bbdev_op_turbo_dec *ref_td = &ref_op->turbo_dec;
+
+ for (i = 0; i < n; ++i) {
+ ops_td = &ops[i]->turbo_dec;
+ hard_output = &ops_td->hard_output;
+ soft_output = &ops_td->soft_output;
+
+ if (vector_mask & TEST_BBDEV_VF_EXPECTED_ITER_COUNT)
+ TEST_ASSERT(ops_td->iter_count <= ref_td->iter_count,
+ "Returned iter_count (%d) > expected iter_count (%d)",
+ ops_td->iter_count, ref_td->iter_count);
+ ret = check_dec_status_and_ordering(ops[i], i, ref_op->status);
+ TEST_ASSERT_SUCCESS(ret,
+ "Checking status and ordering for decoder failed");
+
+ TEST_ASSERT_SUCCESS(validate_op_chain(hard_output,
+ hard_data_orig),
+ "Hard output buffers (CB=%u) are not equal",
+ i);
+
+ if (ref_op->turbo_dec.op_flags & RTE_BBDEV_TURBO_SOFT_OUTPUT)
+ TEST_ASSERT_SUCCESS(validate_op_chain(soft_output,
+ soft_data_orig),
+ "Soft output buffers (CB=%u) are not equal",
+ i);
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+validate_enc_op(struct rte_bbdev_enc_op **ops, const uint16_t n,
+ struct rte_bbdev_enc_op *ref_op)
+{
+ unsigned int i;
+ int ret;
+ struct op_data_entries *hard_data_orig =
+ &test_vector.entries[DATA_HARD_OUTPUT];
+
+ for (i = 0; i < n; ++i) {
+ ret = check_enc_status_and_ordering(ops[i], i, ref_op->status);
+ TEST_ASSERT_SUCCESS(ret,
+ "Checking status and ordering for encoder failed");
+ TEST_ASSERT_SUCCESS(validate_op_chain(
+ &ops[i]->turbo_enc.output,
+ hard_data_orig),
+ "Output buffers (CB=%u) are not equal",
+ i);
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+create_reference_dec_op(struct rte_bbdev_dec_op *op)
+{
+ op->turbo_dec = test_vector.turbo_dec;
+}
+
+static void
+create_reference_enc_op(struct rte_bbdev_enc_op *op)
+{
+ op->turbo_enc = test_vector.turbo_enc;
+}
+
+static int
+init_test_op_params(struct test_op_params *op_params,
+ enum rte_bbdev_op_type op_type, const int expected_status,
+ const int vector_mask, struct rte_mempool *ops_mp,
+ uint16_t burst_sz, uint16_t num_to_process)
+{
+ int ret = 0;
+ if (op_type == RTE_BBDEV_OP_TURBO_DEC)
+ ret = rte_bbdev_dec_op_alloc_bulk(ops_mp,
+ &op_params->ref_dec_op, 1);
+ else
+ ret = rte_bbdev_enc_op_alloc_bulk(ops_mp,
+ &op_params->ref_enc_op, 1);
+
+ TEST_ASSERT_SUCCESS(ret, "rte_bbdev_op_alloc_bulk() failed");
+
+ op_params->mp = ops_mp;
+ op_params->burst_sz = burst_sz;
+ op_params->num_to_process = num_to_process;
+ op_params->vector_mask = vector_mask;
+ if (op_type == RTE_BBDEV_OP_TURBO_DEC)
+ op_params->ref_dec_op->status = expected_status;
+ else if (op_type == RTE_BBDEV_OP_TURBO_ENC)
+ op_params->ref_enc_op->status = expected_status;
+
+ return 0;
+}
+
+static int
+run_test_case_on_device(test_case_function *test_case_func, uint8_t dev_id,
+ struct test_op_params *op_params)
+{
+ int t_ret, f_ret, socket_id = SOCKET_ID_ANY;
+ unsigned int i;
+ struct active_device *ad;
+ unsigned int burst_sz = get_burst_sz();
+ enum rte_bbdev_op_type op_type = test_vector.op_type;
+
+ ad = &active_devs[dev_id];
+
+ /* Check if device supports op_type */
+ if (!is_avail_op(ad, test_vector.op_type))
+ return TEST_SUCCESS;
+
+ struct rte_bbdev_info info;
+ rte_bbdev_info_get(ad->dev_id, &info);
+ socket_id = GET_SOCKET(info.socket_id);
+
+ if (op_type == RTE_BBDEV_OP_NONE)
+ op_type = RTE_BBDEV_OP_TURBO_ENC;
+ f_ret = create_mempools(ad, socket_id, op_type,
+ get_num_ops());
+ if (f_ret != TEST_SUCCESS) {
+ printf("Couldn't create mempools");
+ goto fail;
+ }
+
+ f_ret = init_test_op_params(op_params, test_vector.op_type,
+ test_vector.expected_status,
+ test_vector.mask,
+ ad->ops_mempool,
+ burst_sz,
+ get_num_ops());
+ if (f_ret != TEST_SUCCESS) {
+ printf("Couldn't init test op params");
+ goto fail;
+ }
+
+ if (test_vector.op_type == RTE_BBDEV_OP_TURBO_DEC)
+ create_reference_dec_op(op_params->ref_dec_op);
+ else if (test_vector.op_type == RTE_BBDEV_OP_TURBO_ENC)
+ create_reference_enc_op(op_params->ref_enc_op);
+
+ for (i = 0; i < ad->nb_queues; ++i) {
+ f_ret = fill_queue_buffers(op_params,
+ ad->in_mbuf_pool,
+ ad->hard_out_mbuf_pool,
+ ad->soft_out_mbuf_pool,
+ ad->queue_ids[i],
+ info.drv.min_alignment,
+ socket_id);
+ if (f_ret != TEST_SUCCESS) {
+ printf("Couldn't init queue buffers");
+ goto fail;
+ }
+ }
+
+ /* Run test case function */
+ t_ret = test_case_func(ad, op_params);
+
+ /* Free active device resources and return */
+ free_buffers(ad, op_params);
+ return t_ret;
+
+fail:
+ free_buffers(ad, op_params);
+ return TEST_FAILED;
+}
+
+/* Run given test function per active device per supported op type
+ * per burst size.
+ */
+static int
+run_test_case(test_case_function *test_case_func)
+{
+ int ret = 0;
+ uint8_t dev;
+
+ /* Alloc op_params */
+ struct test_op_params *op_params = rte_zmalloc(NULL,
+ sizeof(struct test_op_params), RTE_CACHE_LINE_SIZE);
+ TEST_ASSERT_NOT_NULL(op_params, "Failed to alloc %zuB for op_params",
+ RTE_ALIGN(sizeof(struct test_op_params),
+ RTE_CACHE_LINE_SIZE));
+
+ /* For each device run test case function */
+ for (dev = 0; dev < nb_active_devs; ++dev)
+ ret |= run_test_case_on_device(test_case_func, dev, op_params);
+
+ rte_free(op_params);
+
+ return ret;
+}
+
+static void
+dequeue_event_callback(uint16_t dev_id,
+ enum rte_bbdev_event_type event, void *cb_arg,
+ void *ret_param)
+{
+ int ret;
+ uint16_t i;
+ uint64_t total_time;
+ uint16_t deq, burst_sz, num_to_process;
+ uint16_t queue_id = INVALID_QUEUE_ID;
+ struct rte_bbdev_dec_op *dec_ops[MAX_BURST];
+ struct rte_bbdev_enc_op *enc_ops[MAX_BURST];
+ struct test_buffers *bufs;
+ struct rte_bbdev_info info;
+
+ /* Input length in bytes, million operations per second,
+ * million bits per second.
+ */
+ double in_len, mops, mbps;
+
+ struct thread_params *tp = cb_arg;
+
+ RTE_SET_USED(ret_param);
+
+ /* Find matching thread params using queue_id */
+ for (i = 0; i < MAX_QUEUES; ++i, ++tp)
+ if (tp->queue_id == queue_id)
+ break;
+
+ if (i == MAX_QUEUES) {
+ printf("%s: Queue_id from interrupt details was not found!\n",
+ __func__);
+ return;
+ }
+
+ if (unlikely(event != RTE_BBDEV_EVENT_DEQUEUE)) {
+ rte_atomic16_set(&tp->processing_status, TEST_FAILED);
+ printf(
+ "Dequeue interrupt handler called for incorrect event!\n");
+ return;
+ }
+
+ burst_sz = tp->op_params->burst_sz;
+ num_to_process = tp->op_params->num_to_process;
+
+ if (test_vector.op_type == RTE_BBDEV_OP_TURBO_DEC)
+ deq = rte_bbdev_dequeue_dec_ops(dev_id, queue_id, dec_ops,
+ burst_sz);
+ else
+ deq = rte_bbdev_dequeue_enc_ops(dev_id, queue_id, enc_ops,
+ burst_sz);
+
+ if (deq < burst_sz) {
+ printf(
+ "After receiving the interrupt all operations should be dequeued. Expected: %u, got: %u\n",
+ burst_sz, deq);
+ rte_atomic16_set(&tp->processing_status, TEST_FAILED);
+ return;
+ }
+
+ if (rte_atomic16_read(&tp->nb_dequeued) + deq < num_to_process) {
+ rte_atomic16_add(&tp->nb_dequeued, deq);
+ return;
+ }
+
+ total_time = rte_rdtsc_precise() - tp->start_time;
+
+ rte_bbdev_info_get(dev_id, &info);
+
+ bufs = &tp->op_params->q_bufs[GET_SOCKET(info.socket_id)][queue_id];
+
+ ret = TEST_SUCCESS;
+ if (test_vector.op_type == RTE_BBDEV_OP_TURBO_DEC)
+ ret = validate_dec_buffers(tp->op_params->ref_dec_op, bufs,
+ num_to_process);
+ else if (test_vector.op_type == RTE_BBDEV_OP_TURBO_ENC)
+ ret = validate_dec_buffers(tp->op_params->ref_dec_op, bufs,
+ num_to_process);
+
+ if (ret) {
+ printf("Buffers validation failed\n");
+ rte_atomic16_set(&tp->processing_status, TEST_FAILED);
+ }
+
+ switch (test_vector.op_type) {
+ case RTE_BBDEV_OP_TURBO_DEC:
+ in_len = tp->op_params->ref_dec_op->turbo_dec.input.length;
+ break;
+ case RTE_BBDEV_OP_TURBO_ENC:
+ in_len = tp->op_params->ref_enc_op->turbo_enc.input.length;
+ break;
+ case RTE_BBDEV_OP_NONE:
+ in_len = 0.0;
+ break;
+ default:
+ printf("Unknown op type: %d\n", test_vector.op_type);
+ rte_atomic16_set(&tp->processing_status, TEST_FAILED);
+ return;
+ }
+
+ mops = ((double)num_to_process / 1000000.0) /
+ ((double)total_time / (double)rte_get_tsc_hz());
+ mbps = ((double)num_to_process * in_len * 8 / 1000000.0) /
+ ((double)total_time / (double)rte_get_tsc_hz());
+
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE)
+ printf("\tqueue_id: %u, throughput: %lg MOPS, %lg Mbps\n",
+ queue_id, mops, mbps);
+ else
+ printf("\tqueue_id: %u, throughput: %lg MOPS\n",
+ queue_id, mops);
+
+ rte_atomic16_add(&tp->nb_dequeued, deq);
+}
+
+static int
+throughput_intr_lcore_dec(void *arg)
+{
+ struct thread_params *tp = arg;
+ unsigned int enqueued;
+ struct rte_bbdev_dec_op *ops[MAX_BURST];
+ const uint16_t queue_id = tp->queue_id;
+ const uint16_t burst_sz = tp->op_params->burst_sz;
+ const uint16_t num_to_process = tp->op_params->num_to_process;
+ struct test_buffers *bufs = NULL;
+ unsigned int allocs_failed = 0;
+ struct rte_bbdev_info info;
+ int ret;
+
+ if (burst_sz > MAX_BURST) {
+ printf("Operations table size > MAX_BURST\n");
+ return TEST_FAILED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_intr_enable(tp->dev_id, queue_id),
+ "Failed to enable interrupts for dev: %u, queue_id: %u",
+ tp->dev_id, queue_id);
+
+ rte_bbdev_info_get(tp->dev_id, &info);
+ bufs = &tp->op_params->q_bufs[GET_SOCKET(info.socket_id)][queue_id];
+
+ rte_atomic16_clear(&tp->processing_status);
+ rte_atomic16_clear(&tp->nb_dequeued);
+
+ while (rte_atomic16_read(&tp->op_params->sync) == SYNC_WAIT)
+ rte_pause();
+
+ tp->start_time = rte_rdtsc_precise();
+ for (enqueued = 0; enqueued < num_to_process;) {
+
+ uint16_t num_to_enq = burst_sz;
+
+ if (unlikely(num_to_process - enqueued < num_to_enq))
+ num_to_enq = num_to_process - enqueued;
+
+ ret = rte_bbdev_dec_op_alloc_bulk(tp->op_params->mp, ops,
+ num_to_enq);
+ if (ret != 0) {
+ allocs_failed++;
+ continue;
+ }
+
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE)
+ copy_reference_dec_op(ops, num_to_enq, enqueued,
+ bufs->inputs,
+ bufs->hard_outputs,
+ bufs->soft_outputs,
+ tp->op_params->ref_dec_op);
+
+ enqueued += rte_bbdev_enqueue_dec_ops(tp->dev_id, queue_id, ops,
+ num_to_enq);
+
+ rte_bbdev_dec_op_free_bulk(ops, num_to_enq);
+ }
+
+ if (allocs_failed > 0)
+ printf("WARNING: op allocations failed: %u times\n",
+ allocs_failed);
+
+ return TEST_SUCCESS;
+}
+
+static int
+throughput_intr_lcore_enc(void *arg)
+{
+ struct thread_params *tp = arg;
+ unsigned int enqueued;
+ struct rte_bbdev_enc_op *ops[MAX_BURST];
+ const uint16_t queue_id = tp->queue_id;
+ const uint16_t burst_sz = tp->op_params->burst_sz;
+ const uint16_t num_to_process = tp->op_params->num_to_process;
+ struct test_buffers *bufs = NULL;
+ unsigned int allocs_failed = 0;
+ struct rte_bbdev_info info;
+ int ret;
+
+ if (burst_sz > MAX_BURST) {
+ printf("Operations table size > MAX_BURST\n");
+ return TEST_FAILED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_bbdev_queue_intr_enable(tp->dev_id, queue_id),
+ "Failed to enable interrupts for dev: %u, queue_id: %u",
+ tp->dev_id, queue_id);
+
+ rte_bbdev_info_get(tp->dev_id, &info);
+ bufs = &tp->op_params->q_bufs[GET_SOCKET(info.socket_id)][queue_id];
+
+ rte_atomic16_clear(&tp->processing_status);
+ rte_atomic16_clear(&tp->nb_dequeued);
+
+ while (rte_atomic16_read(&tp->op_params->sync) == SYNC_WAIT)
+ rte_pause();
+
+ tp->start_time = rte_rdtsc_precise();
+ for (enqueued = 0; enqueued < num_to_process;) {
+
+ uint16_t num_to_enq = burst_sz;
+
+ if (unlikely(num_to_process - enqueued < num_to_enq))
+ num_to_enq = num_to_process - enqueued;
+
+ ret = rte_bbdev_enc_op_alloc_bulk(tp->op_params->mp, ops,
+ num_to_enq);
+ if (ret != 0) {
+ allocs_failed++;
+ continue;
+ }
+
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE)
+ copy_reference_enc_op(ops, num_to_enq, enqueued,
+ bufs->inputs,
+ bufs->hard_outputs,
+ tp->op_params->ref_enc_op);
+
+ enqueued += rte_bbdev_enqueue_enc_ops(tp->dev_id, queue_id, ops,
+ num_to_enq);
+
+ rte_bbdev_enc_op_free_bulk(ops, num_to_enq);
+ }
+
+ if (allocs_failed > 0)
+ printf("WARNING: op allocations failed: %u times\n",
+ allocs_failed);
+
+ return TEST_SUCCESS;
+}
+
+static int
+throughput_pmd_lcore_dec(void *arg)
+{
+ struct thread_params *tp = arg;
+ unsigned int enqueued, dequeued;
+ struct rte_bbdev_dec_op *ops[MAX_BURST];
+ uint64_t total_time, start_time;
+ const uint16_t queue_id = tp->queue_id;
+ const uint16_t burst_sz = tp->op_params->burst_sz;
+ const uint16_t num_to_process = tp->op_params->num_to_process;
+ struct rte_bbdev_dec_op *ref_op = tp->op_params->ref_dec_op;
+ struct test_buffers *bufs = NULL;
+ unsigned int allocs_failed = 0;
+ int ret;
+ struct rte_bbdev_info info;
+
+ /* Input length in bytes, million operations per second, million bits
+ * per second.
+ */
+ double in_len, mops, mbps;
+
+ if (burst_sz > MAX_BURST) {
+ printf("Operations table size > MAX_BURST\n");
+ return TEST_FAILED;
+ }
+
+ rte_bbdev_info_get(tp->dev_id, &info);
+ bufs = &tp->op_params->q_bufs[GET_SOCKET(info.socket_id)][queue_id];
+
+ while (rte_atomic16_read(&tp->op_params->sync) == SYNC_WAIT)
+ rte_pause();
+
+ start_time = rte_rdtsc_precise();
+ for (enqueued = 0, dequeued = 0; dequeued < num_to_process;) {
+ uint16_t deq;
+
+ if (likely(enqueued < num_to_process)) {
+
+ uint16_t num_to_enq = burst_sz;
+
+ if (unlikely(num_to_process - enqueued < num_to_enq))
+ num_to_enq = num_to_process - enqueued;
+
+ ret = rte_bbdev_dec_op_alloc_bulk(tp->op_params->mp,
+ ops, num_to_enq);
+ if (ret != 0) {
+ allocs_failed++;
+ goto do_dequeue;
+ }
+
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE)
+ copy_reference_dec_op(ops, num_to_enq, enqueued,
+ bufs->inputs,
+ bufs->hard_outputs,
+ bufs->soft_outputs,
+ ref_op);
+
+ enqueued += rte_bbdev_enqueue_dec_ops(tp->dev_id,
+ queue_id, ops, num_to_enq);
+ }
+do_dequeue:
+ deq = rte_bbdev_dequeue_dec_ops(tp->dev_id, queue_id, ops,
+ burst_sz);
+ dequeued += deq;
+ rte_bbdev_dec_op_free_bulk(ops, deq);
+ }
+ total_time = rte_rdtsc_precise() - start_time;
+
+ if (allocs_failed > 0)
+ printf("WARNING: op allocations failed: %u times\n",
+ allocs_failed);
+
+ TEST_ASSERT(enqueued == dequeued, "enqueued (%u) != dequeued (%u)",
+ enqueued, dequeued);
+
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE) {
+ ret = validate_dec_buffers(ref_op, bufs, num_to_process);
+ TEST_ASSERT_SUCCESS(ret, "Buffers validation failed");
+ }
+
+ in_len = ref_op->turbo_dec.input.length;
+ mops = ((double)num_to_process / 1000000.0) /
+ ((double)total_time / (double)rte_get_tsc_hz());
+ mbps = ((double)in_len * 8 / 1000000.0) /
+ ((double)total_time / (double)rte_get_tsc_hz());
+
+ printf("\tlcore_id: %u, throughput: %lg MOPS, %lg Mbps\n",
+ rte_lcore_id(), mops, mbps);
+
+ return TEST_SUCCESS;
+}
+
+static int
+throughput_pmd_lcore_enc(void *arg)
+{
+ struct thread_params *tp = arg;
+ unsigned int enqueued, dequeued;
+ struct rte_bbdev_enc_op *ops[MAX_BURST];
+ uint64_t total_time, start_time;
+ const uint16_t queue_id = tp->queue_id;
+ const uint16_t burst_sz = tp->op_params->burst_sz;
+ const uint16_t num_to_process = tp->op_params->num_to_process;
+ struct rte_bbdev_enc_op *ref_op = tp->op_params->ref_enc_op;
+ struct test_buffers *bufs = NULL;
+ unsigned int allocs_failed = 0;
+ int ret;
+ struct rte_bbdev_info info;
+
+ /* Input length in bytes, million operations per second, million bits
+ * per second.
+ */
+ double in_len, mops, mbps;
+
+ if (burst_sz > MAX_BURST) {
+ printf("Operations table size > MAX_BURST\n");
+ return TEST_FAILED;
+ }
+
+ rte_bbdev_info_get(tp->dev_id, &info);
+ bufs = &tp->op_params->q_bufs[GET_SOCKET(info.socket_id)][queue_id];
+
+ while (rte_atomic16_read(&tp->op_params->sync) == SYNC_WAIT)
+ rte_pause();
+
+ start_time = rte_rdtsc_precise();
+ for (enqueued = 0, dequeued = 0; dequeued < num_to_process;) {
+ uint16_t deq;
+
+ if (likely(enqueued < num_to_process)) {
+
+ uint16_t num_to_enq = burst_sz;
+
+ if (unlikely(num_to_process - enqueued < num_to_enq))
+ num_to_enq = num_to_process - enqueued;
+
+ ret = rte_bbdev_enc_op_alloc_bulk(tp->op_params->mp,
+ ops, num_to_enq);
+ if (ret != 0) {
+ allocs_failed++;
+ goto do_dequeue;
+ }
+
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE)
+ copy_reference_enc_op(ops, num_to_enq, enqueued,
+ bufs->inputs,
+ bufs->hard_outputs,
+ ref_op);
+
+ enqueued += rte_bbdev_enqueue_enc_ops(tp->dev_id,
+ queue_id, ops, num_to_enq);
+ }
+do_dequeue:
+ deq = rte_bbdev_dequeue_enc_ops(tp->dev_id, queue_id, ops,
+ burst_sz);
+ dequeued += deq;
+ rte_bbdev_enc_op_free_bulk(ops, deq);
+ }
+ total_time = rte_rdtsc_precise() - start_time;
+
+ if (allocs_failed > 0)
+ printf("WARNING: op allocations failed: %u times\n",
+ allocs_failed);
+
+ TEST_ASSERT(enqueued == dequeued, "enqueued (%u) != dequeued (%u)",
+ enqueued, dequeued);
+
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE) {
+ ret = validate_enc_buffers(bufs, num_to_process);
+ TEST_ASSERT_SUCCESS(ret, "Buffers validation failed");
+ }
+
+ in_len = ref_op->turbo_enc.input.length;
+
+ mops = ((double)num_to_process / 1000000.0) /
+ ((double)total_time / (double)rte_get_tsc_hz());
+ mbps = ((double)in_len * 8 / 1000000.0) /
+ ((double)total_time / (double)rte_get_tsc_hz());
+
+ printf("\tlcore_id: %u, throughput: %lg MOPS, %lg Mbps\n",
+ rte_lcore_id(), mops, mbps);
+
+ return TEST_SUCCESS;
+}
+
+/*
+ * Test function that determines how long an enqueue + dequeue of a burst
+ * takes on available lcores.
+ */
+static int
+throughput_test(struct active_device *ad,
+ struct test_op_params *op_params)
+{
+ int ret;
+ unsigned int lcore_id, used_cores = 0;
+ struct thread_params t_params[MAX_QUEUES];
+ struct rte_bbdev_info info;
+ lcore_function_t *throughput_function;
+
+ rte_bbdev_info_get(ad->dev_id, &info);
+
+ printf(
+ "Throughput test: dev: %s, burst size: %u, num ops: %u, op type: %s, int mode: %s, GHz: %lg\n",
+ info.dev_name, op_params->burst_sz,
+ op_params->num_to_process,
+ rte_bbdev_op_type_str(test_vector.op_type),
+ intr_enabled ? "Interrupt mode" : "PMD mode",
+ (double)rte_get_tsc_hz() / 1000000000.0);
+
+ if (intr_enabled) {
+ if (test_vector.op_type == RTE_BBDEV_OP_TURBO_DEC)
+ throughput_function = throughput_intr_lcore_dec;
+ else
+ throughput_function = throughput_intr_lcore_enc;
+
+ /* Dequeue interrupt callback registration */
+ rte_bbdev_callback_register(ad->dev_id, RTE_BBDEV_EVENT_DEQUEUE,
+ dequeue_event_callback,
+ &t_params);
+ } else {
+ if (test_vector.op_type == RTE_BBDEV_OP_TURBO_DEC)
+ throughput_function = throughput_pmd_lcore_dec;
+ else
+ throughput_function = throughput_pmd_lcore_enc;
+ }
+
+ rte_atomic16_set(&op_params->sync, SYNC_WAIT);
+
+ t_params[rte_lcore_id()].dev_id = ad->dev_id;
+ t_params[rte_lcore_id()].op_params = op_params;
+ t_params[rte_lcore_id()].queue_id =
+ ad->queue_ids[used_cores++];
+
+ RTE_LCORE_FOREACH_SLAVE(lcore_id) {
+ if (used_cores >= ad->nb_queues)
+ break;
+
+ t_params[lcore_id].dev_id = ad->dev_id;
+ t_params[lcore_id].op_params = op_params;
+ t_params[lcore_id].queue_id = ad->queue_ids[used_cores++];
+
+ rte_eal_remote_launch(throughput_function, &t_params[lcore_id],
+ lcore_id);
+ }
+
+ rte_atomic16_set(&op_params->sync, SYNC_START);
+ ret = throughput_function(&t_params[rte_lcore_id()]);
+
+ /* Master core is always used */
+ used_cores = 1;
+ RTE_LCORE_FOREACH_SLAVE(lcore_id) {
+ if (used_cores++ >= ad->nb_queues)
+ break;
+
+ ret |= rte_eal_wait_lcore(lcore_id);
+ }
+
+ /* If interrupts are disabled or throughput_intr_lcore failed, return */
+ if (!intr_enabled || ret)
+ return ret;
+
+ /* In interrupt TC we need to wait for the interrupt callback to deqeue
+ * all pending operations. Skip waiting for queues which reported an
+ * error using processing_status variable.
+ */
+ used_cores = 1;
+ RTE_LCORE_FOREACH_SLAVE(lcore_id) {
+ struct thread_params *tp = &t_params[lcore_id];
+ if (used_cores++ >= ad->nb_queues)
+ break;
+
+ while ((rte_atomic16_read(&tp->nb_dequeued) <
+ op_params->num_to_process) &&
+ (rte_atomic16_read(&tp->processing_status) !=
+ TEST_FAILED))
+ rte_pause();
+
+ ret |= rte_atomic16_read(&tp->processing_status);
+ }
+
+ return ret;
+}
+
+static int
+operation_latency_test_dec(struct rte_mempool *mempool,
+ struct test_buffers *bufs, struct rte_bbdev_dec_op *ref_op,
+ int vector_mask, uint16_t dev_id, uint16_t queue_id,
+ const uint16_t num_to_process, uint16_t burst_sz,
+ uint64_t *total_time)
+{
+ int ret = TEST_SUCCESS;
+ uint16_t i, j, dequeued;
+ struct rte_bbdev_dec_op *ops[MAX_BURST];
+ uint64_t start_time = 0;
+
+ for (i = 0, dequeued = 0; dequeued < num_to_process; ++i) {
+ uint16_t enq = 0, deq = 0;
+ bool first_time = true;
+
+ if (unlikely(num_to_process - dequeued < burst_sz))
+ burst_sz = num_to_process - dequeued;
+
+ rte_bbdev_dec_op_alloc_bulk(mempool, ops, burst_sz);
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE)
+ copy_reference_dec_op(ops, burst_sz, dequeued,
+ bufs->inputs,
+ bufs->hard_outputs,
+ bufs->soft_outputs,
+ ref_op);
+
+ /* Set counter to validate the ordering */
+ for (j = 0; j < burst_sz; ++j)
+ ops[j]->opaque_data = (void *)(uintptr_t)j;
+
+ start_time = rte_rdtsc_precise();
+
+ enq = rte_bbdev_enqueue_dec_ops(dev_id, queue_id, &ops[enq],
+ burst_sz);
+ TEST_ASSERT(enq == burst_sz,
+ "Error enqueueing burst, expected %u, got %u",
+ burst_sz, enq);
+
+ /* Dequeue */
+ do {
+ deq += rte_bbdev_dequeue_dec_ops(dev_id, queue_id,
+ &ops[deq], burst_sz - deq);
+ if (likely(first_time && (deq > 0))) {
+ *total_time += rte_rdtsc_precise() - start_time;
+ first_time = false;
+ }
+ } while (unlikely(burst_sz != deq));
+
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE) {
+ ret = validate_dec_op(ops, burst_sz, ref_op,
+ vector_mask);
+ TEST_ASSERT_SUCCESS(ret, "Validation failed!");
+ }
+
+ rte_bbdev_dec_op_free_bulk(ops, deq);
+ dequeued += deq;
+ }
+
+ return i;
+}
+
+static int
+operation_latency_test_enc(struct rte_mempool *mempool,
+ struct test_buffers *bufs, struct rte_bbdev_enc_op *ref_op,
+ uint16_t dev_id, uint16_t queue_id,
+ const uint16_t num_to_process, uint16_t burst_sz,
+ uint64_t *total_time)
+{
+ int ret = TEST_SUCCESS;
+ uint16_t i, j, dequeued;
+ struct rte_bbdev_enc_op *ops[MAX_BURST];
+ uint64_t start_time = 0;
+
+ for (i = 0, dequeued = 0; dequeued < num_to_process; ++i) {
+ uint16_t enq = 0, deq = 0;
+ bool first_time = true;
+
+ if (unlikely(num_to_process - dequeued < burst_sz))
+ burst_sz = num_to_process - dequeued;
+
+ rte_bbdev_enc_op_alloc_bulk(mempool, ops, burst_sz);
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE)
+ copy_reference_enc_op(ops, burst_sz, dequeued,
+ bufs->inputs,
+ bufs->hard_outputs,
+ ref_op);
+
+ /* Set counter to validate the ordering */
+ for (j = 0; j < burst_sz; ++j)
+ ops[j]->opaque_data = (void *)(uintptr_t)j;
+
+ start_time = rte_rdtsc_precise();
+
+ enq = rte_bbdev_enqueue_enc_ops(dev_id, queue_id, &ops[enq],
+ burst_sz);
+ TEST_ASSERT(enq == burst_sz,
+ "Error enqueueing burst, expected %u, got %u",
+ burst_sz, enq);
+
+ /* Dequeue */
+ do {
+ deq += rte_bbdev_dequeue_enc_ops(dev_id, queue_id,
+ &ops[deq], burst_sz - deq);
+ if (likely(first_time && (deq > 0))) {
+ *total_time += rte_rdtsc_precise() - start_time;
+ first_time = false;
+ }
+ } while (unlikely(burst_sz != deq));
+
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE) {
+ ret = validate_enc_op(ops, burst_sz, ref_op);
+ TEST_ASSERT_SUCCESS(ret, "Validation failed!");
+ }
+
+ rte_bbdev_enc_op_free_bulk(ops, deq);
+ dequeued += deq;
+ }
+
+ return i;
+}
+
+static int
+operation_latency_test(struct active_device *ad,
+ struct test_op_params *op_params)
+{
+ uint16_t iter;
+ uint16_t burst_sz = op_params->burst_sz;
+ const uint16_t num_to_process = op_params->num_to_process;
+ const enum rte_bbdev_op_type op_type = test_vector.op_type;
+ const uint16_t queue_id = ad->queue_ids[0];
+ struct test_buffers *bufs = NULL;
+ struct rte_bbdev_info info;
+ uint64_t total_time = 0;
+
+ rte_bbdev_info_get(ad->dev_id, &info);
+ bufs = &op_params->q_bufs[GET_SOCKET(info.socket_id)][queue_id];
+
+ printf(
+ "Validation/Latency test: dev: %s, burst size: %u, num ops: %u, op type: %s\n",
+ info.dev_name, burst_sz, num_to_process,
+ rte_bbdev_op_type_str(op_type));
+
+ if (op_type == RTE_BBDEV_OP_TURBO_DEC)
+ iter = operation_latency_test_dec(op_params->mp, bufs,
+ op_params->ref_dec_op, op_params->vector_mask,
+ ad->dev_id, queue_id, num_to_process,
+ burst_sz, &total_time);
+ else
+ iter = operation_latency_test_enc(op_params->mp, bufs,
+ op_params->ref_enc_op, ad->dev_id, queue_id,
+ num_to_process, burst_sz, &total_time);
+
+ printf("\toperation avg. latency: %lg cycles, %lg us\n",
+ (double)total_time / (double)iter,
+ (double)(total_time * 1000000) / (double)iter /
+ (double)rte_get_tsc_hz());
+
+ return TEST_SUCCESS;
+}
+
+static uint16_t
+offload_latency_test_dec(struct rte_mempool *mempool, struct test_buffers *bufs,
+ struct rte_bbdev_dec_op *ref_op, uint16_t dev_id,
+ uint16_t queue_id, const uint16_t num_to_process,
+ uint16_t burst_sz, uint64_t *enq_total_time,
+ uint64_t *deq_total_time)
+{
+ uint16_t i, dequeued;
+ struct rte_bbdev_dec_op *ops[MAX_BURST];
+ uint64_t enq_start_time, deq_start_time;
+
+ for (i = 0, dequeued = 0; dequeued < num_to_process; ++i) {
+ uint16_t enq = 0, deq = 0;
+
+ if (unlikely(num_to_process - dequeued < burst_sz))
+ burst_sz = num_to_process - dequeued;
+
+ rte_bbdev_dec_op_alloc_bulk(mempool, ops, burst_sz);
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE)
+ copy_reference_dec_op(ops, burst_sz, dequeued,
+ bufs->inputs,
+ bufs->hard_outputs,
+ bufs->soft_outputs,
+ ref_op);
+
+ /* Start time measurment for enqueue function offload latency */
+ enq_start_time = rte_rdtsc();
+ do {
+ enq += rte_bbdev_enqueue_dec_ops(dev_id, queue_id,
+ &ops[enq], burst_sz - enq);
+ } while (unlikely(burst_sz != enq));
+ *enq_total_time += rte_rdtsc() - enq_start_time;
+
+ /* ensure enqueue has been completed */
+ rte_delay_ms(10);
+
+ /* Start time measurment for dequeue function offload latency */
+ deq_start_time = rte_rdtsc();
+ do {
+ deq += rte_bbdev_dequeue_dec_ops(dev_id, queue_id,
+ &ops[deq], burst_sz - deq);
+ } while (unlikely(burst_sz != deq));
+ *deq_total_time += rte_rdtsc() - deq_start_time;
+
+ rte_bbdev_dec_op_free_bulk(ops, deq);
+ dequeued += deq;
+ }
+
+ return i;
+}
+
+static uint16_t
+offload_latency_test_enc(struct rte_mempool *mempool, struct test_buffers *bufs,
+ struct rte_bbdev_enc_op *ref_op, uint16_t dev_id,
+ uint16_t queue_id, const uint16_t num_to_process,
+ uint16_t burst_sz, uint64_t *enq_total_time,
+ uint64_t *deq_total_time)
+{
+ uint16_t i, dequeued;
+ struct rte_bbdev_enc_op *ops[MAX_BURST];
+ uint64_t enq_start_time, deq_start_time;
+
+ for (i = 0, dequeued = 0; dequeued < num_to_process; ++i) {
+ uint16_t enq = 0, deq = 0;
+
+ if (unlikely(num_to_process - dequeued < burst_sz))
+ burst_sz = num_to_process - dequeued;
+
+ rte_bbdev_enc_op_alloc_bulk(mempool, ops, burst_sz);
+ if (test_vector.op_type != RTE_BBDEV_OP_NONE)
+ copy_reference_enc_op(ops, burst_sz, dequeued,
+ bufs->inputs,
+ bufs->hard_outputs,
+ ref_op);
+
+ /* Start time measurment for enqueue function offload latency */
+ enq_start_time = rte_rdtsc();
+ do {
+ enq += rte_bbdev_enqueue_enc_ops(dev_id, queue_id,
+ &ops[enq], burst_sz - enq);
+ } while (unlikely(burst_sz != enq));
+ *enq_total_time += rte_rdtsc() - enq_start_time;
+
+ /* ensure enqueue has been completed */
+ rte_delay_ms(10);
+
+ /* Start time measurment for dequeue function offload latency */
+ deq_start_time = rte_rdtsc();
+ do {
+ deq += rte_bbdev_dequeue_enc_ops(dev_id, queue_id,
+ &ops[deq], burst_sz - deq);
+ } while (unlikely(burst_sz != deq));
+ *deq_total_time += rte_rdtsc() - deq_start_time;
+
+ rte_bbdev_enc_op_free_bulk(ops, deq);
+ dequeued += deq;
+ }
+
+ return i;
+}
+
+static int
+offload_latency_test(struct active_device *ad,
+ struct test_op_params *op_params)
+{
+ uint16_t iter;
+ uint64_t enq_total_time = 0, deq_total_time = 0;
+ uint16_t burst_sz = op_params->burst_sz;
+ const uint16_t num_to_process = op_params->num_to_process;
+ const enum rte_bbdev_op_type op_type = test_vector.op_type;
+ const uint16_t queue_id = ad->queue_ids[0];
+ struct test_buffers *bufs = NULL;
+ struct rte_bbdev_info info;
+
+ rte_bbdev_info_get(ad->dev_id, &info);
+ bufs = &op_params->q_bufs[GET_SOCKET(info.socket_id)][queue_id];
+
+ printf(
+ "Offload latency test: dev: %s, burst size: %u, num ops: %u, op type: %s\n",
+ info.dev_name, burst_sz, num_to_process,
+ rte_bbdev_op_type_str(op_type));
+
+ if (op_type == RTE_BBDEV_OP_TURBO_DEC)
+ iter = offload_latency_test_dec(op_params->mp, bufs,
+ op_params->ref_dec_op, ad->dev_id, queue_id,
+ num_to_process, burst_sz, &enq_total_time,
+ &deq_total_time);
+ else
+ iter = offload_latency_test_enc(op_params->mp, bufs,
+ op_params->ref_enc_op, ad->dev_id, queue_id,
+ num_to_process, burst_sz, &enq_total_time,
+ &deq_total_time);
+
+ printf("\tenq offload avg. latency: %lg cycles, %lg us\n",
+ (double)enq_total_time / (double)iter,
+ (double)(enq_total_time * 1000000) / (double)iter /
+ (double)rte_get_tsc_hz());
+
+ printf("\tdeq offload avg. latency: %lg cycles, %lg us\n",
+ (double)deq_total_time / (double)iter,
+ (double)(deq_total_time * 1000000) / (double)iter /
+ (double)rte_get_tsc_hz());
+
+ return TEST_SUCCESS;
+}
+
+static uint16_t
+offload_latency_empty_q_test_dec(uint16_t dev_id, uint16_t queue_id,
+ const uint16_t num_to_process, uint16_t burst_sz,
+ uint64_t *deq_total_time)
+{
+ uint16_t i, deq_total;
+ struct rte_bbdev_dec_op *ops[MAX_BURST];
+ uint64_t deq_start_time;
+
+ /* Test deq offload latency from an empty queue */
+ deq_start_time = rte_rdtsc_precise();
+ for (i = 0, deq_total = 0; deq_total < num_to_process;
+ ++i, deq_total += burst_sz) {
+ if (unlikely(num_to_process - deq_total < burst_sz))
+ burst_sz = num_to_process - deq_total;
+ rte_bbdev_dequeue_dec_ops(dev_id, queue_id, ops, burst_sz);
+ }
+ *deq_total_time = rte_rdtsc_precise() - deq_start_time;
+
+ return i;
+}
+
+static uint16_t
+offload_latency_empty_q_test_enc(uint16_t dev_id, uint16_t queue_id,
+ const uint16_t num_to_process, uint16_t burst_sz,
+ uint64_t *deq_total_time)
+{
+ uint16_t i, deq_total;
+ struct rte_bbdev_enc_op *ops[MAX_BURST];
+ uint64_t deq_start_time;
+
+ /* Test deq offload latency from an empty queue */
+ deq_start_time = rte_rdtsc_precise();
+ for (i = 0, deq_total = 0; deq_total < num_to_process;
+ ++i, deq_total += burst_sz) {
+ if (unlikely(num_to_process - deq_total < burst_sz))
+ burst_sz = num_to_process - deq_total;
+ rte_bbdev_dequeue_enc_ops(dev_id, queue_id, ops, burst_sz);
+ }
+ *deq_total_time = rte_rdtsc_precise() - deq_start_time;
+
+ return i;
+}
+
+static int
+offload_latency_empty_q_test(struct active_device *ad,
+ struct test_op_params *op_params)
+{
+ uint16_t iter;
+ uint64_t deq_total_time = 0;
+ uint16_t burst_sz = op_params->burst_sz;
+ const uint16_t num_to_process = op_params->num_to_process;
+ const enum rte_bbdev_op_type op_type = test_vector.op_type;
+ const uint16_t queue_id = ad->queue_ids[0];
+ struct rte_bbdev_info info;
+
+ rte_bbdev_info_get(ad->dev_id, &info);
+
+ printf(
+ "Offload latency empty dequeue test: dev: %s, burst size: %u, num ops: %u, op type: %s\n",
+ info.dev_name, burst_sz, num_to_process,
+ rte_bbdev_op_type_str(op_type));
+
+ if (op_type == RTE_BBDEV_OP_TURBO_DEC)
+ iter = offload_latency_empty_q_test_dec(ad->dev_id, queue_id,
+ num_to_process, burst_sz, &deq_total_time);
+ else
+ iter = offload_latency_empty_q_test_enc(ad->dev_id, queue_id,
+ num_to_process, burst_sz, &deq_total_time);
+
+ printf("\tempty deq offload avg. latency: %lg cycles, %lg us\n",
+ (double)deq_total_time / (double)iter,
+ (double)(deq_total_time * 1000000) / (double)iter /
+ (double)rte_get_tsc_hz());
+
+ return TEST_SUCCESS;
+}
+
+static int
+throughput_tc(void)
+{
+ return run_test_case(throughput_test);
+}
+
+static int
+offload_latency_tc(void)
+{
+ return run_test_case(offload_latency_test);
+}
+
+static int
+offload_latency_empty_q_tc(void)
+{
+ return run_test_case(offload_latency_empty_q_test);
+}
+
+static int
+operation_latency_tc(void)
+{
+ return run_test_case(operation_latency_test);
+}
+
+static int
+interrupt_tc(void)
+{
+ return run_test_case(throughput_test);
+}
+
+static struct unit_test_suite bbdev_throughput_testsuite = {
+ .suite_name = "BBdev Throughput Tests",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+ TEST_CASE_ST(ut_setup, ut_teardown, throughput_tc),
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static struct unit_test_suite bbdev_validation_testsuite = {
+ .suite_name = "BBdev Validation Tests",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+ TEST_CASE_ST(ut_setup, ut_teardown, operation_latency_tc),
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static struct unit_test_suite bbdev_latency_testsuite = {
+ .suite_name = "BBdev Latency Tests",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+ TEST_CASE_ST(ut_setup, ut_teardown, offload_latency_tc),
+ TEST_CASE_ST(ut_setup, ut_teardown, offload_latency_empty_q_tc),
+ TEST_CASE_ST(ut_setup, ut_teardown, operation_latency_tc),
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static struct unit_test_suite bbdev_interrupt_testsuite = {
+ .suite_name = "BBdev Interrupt Tests",
+ .setup = interrupt_testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+ TEST_CASE_ST(ut_setup, ut_teardown, interrupt_tc),
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+REGISTER_TEST_COMMAND(throughput, bbdev_throughput_testsuite);
+REGISTER_TEST_COMMAND(validation, bbdev_validation_testsuite);
+REGISTER_TEST_COMMAND(latency, bbdev_latency_testsuite);
+REGISTER_TEST_COMMAND(interrupt, bbdev_interrupt_testsuite);
diff --git a/app/test-bbdev/test_bbdev_vector.c b/app/test-bbdev/test_bbdev_vector.c
new file mode 100644
index 0000000..b9e1340
--- /dev/null
+++ b/app/test-bbdev/test_bbdev_vector.c
@@ -0,0 +1,884 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifdef RTE_EXEC_ENV_BSDAPP
+ #define _WITH_GETLINE
+#endif
+#include <stdio.h>
+#include <stdbool.h>
+#include <rte_malloc.h>
+
+#include "test_bbdev_vector.h"
+
+#define VALUE_DELIMITER ","
+#define ENTRY_DELIMITER "="
+
+const char *op_data_prefixes[] = {
+ "input",
+ "soft_output",
+ "hard_output",
+};
+
+/* trim leading and trailing spaces */
+static void
+trim_space(char *str)
+{
+ char *start, *end;
+
+ for (start = str; *start; start++) {
+ if (!isspace((unsigned char) start[0]))
+ break;
+ }
+
+ for (end = start + strlen(start); end > start + 1; end--) {
+ if (!isspace((unsigned char) end[-1]))
+ break;
+ }
+
+ *end = 0;
+
+ /* Shift from "start" to the beginning of the string */
+ if (start > str)
+ memmove(str, start, (end - start) + 1);
+}
+
+static bool
+starts_with(const char *str, const char *pre)
+{
+ return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+/* tokenization test values separated by a comma */
+static int
+parse_values(char *tokens, uint32_t **data, uint32_t *data_length)
+{
+ uint32_t n_tokens = 0;
+ uint32_t data_size = 32;
+
+ uint32_t *values, *values_resized;
+ char *tok, *error = NULL;
+
+ tok = strtok(tokens, VALUE_DELIMITER);
+ if (tok == NULL)
+ return -1;
+
+ values = (uint32_t *)
+ rte_zmalloc(NULL, sizeof(uint32_t) * data_size, 0);
+ if (values == NULL)
+ return -1;
+
+ while (tok != NULL) {
+ values_resized = NULL;
+
+ if (n_tokens >= data_size) {
+ data_size *= 2;
+
+ values_resized = (uint32_t *) rte_realloc(values,
+ sizeof(uint32_t) * data_size, 0);
+ if (values_resized == NULL) {
+ rte_free(values);
+ return -1;
+ }
+ values = values_resized;
+ }
+
+ values[n_tokens] = (uint32_t) strtoul(tok, &error, 0);
+ if ((error == NULL) || (*error != '\0')) {
+ printf("Failed with convert '%s'\n", tok);
+ rte_free(values);
+ return -1;
+ }
+
+ *data_length = *data_length + (strlen(tok) - strlen("0x"))/2;
+
+ tok = strtok(NULL, VALUE_DELIMITER);
+ if (tok == NULL)
+ break;
+
+ n_tokens++;
+ }
+
+ values_resized = (uint32_t *) rte_realloc(values,
+ sizeof(uint32_t) * (n_tokens + 1), 0);
+
+ if (values_resized == NULL) {
+ rte_free(values);
+ return -1;
+ }
+
+ *data = values_resized;
+
+ return 0;
+}
+
+/* convert turbo decoder flag from string to unsigned long int*/
+static int
+op_decoder_flag_strtoul(char *token, uint32_t *op_flag_value)
+{
+ if (!strcmp(token, "RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE"))
+ *op_flag_value = RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_CRC_TYPE_24B"))
+ *op_flag_value = RTE_BBDEV_TURBO_CRC_TYPE_24B;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_EQUALIZER"))
+ *op_flag_value = RTE_BBDEV_TURBO_EQUALIZER;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_SOFT_OUT_SATURATE"))
+ *op_flag_value = RTE_BBDEV_TURBO_SOFT_OUT_SATURATE;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_HALF_ITERATION_EVEN"))
+ *op_flag_value = RTE_BBDEV_TURBO_HALF_ITERATION_EVEN;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_CONTINUE_CRC_MATCH"))
+ *op_flag_value = RTE_BBDEV_TURBO_CONTINUE_CRC_MATCH;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_SOFT_OUTPUT"))
+ *op_flag_value = RTE_BBDEV_TURBO_SOFT_OUTPUT;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_EARLY_TERMINATION"))
+ *op_flag_value = RTE_BBDEV_TURBO_EARLY_TERMINATION;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_RAW_INPUT_DATA"))
+ *op_flag_value = RTE_BBDEV_TURBO_RAW_INPUT_DATA;
+ else {
+ printf("The given value is not a turbo decoder flag\n");
+ return -1;
+ }
+
+ return 0;
+}
+
+/* convert turbo encoder flag from string to unsigned long int*/
+static int
+op_encoder_flag_strtoul(char *token, uint32_t *op_flag_value)
+{
+ if (!strcmp(token, "RTE_BBDEV_TURBO_RV_INDEX_BYPASS"))
+ *op_flag_value = RTE_BBDEV_TURBO_RV_INDEX_BYPASS;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_RATE_MATCH"))
+ *op_flag_value = RTE_BBDEV_TURBO_RATE_MATCH;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_CRC_24B_ATTACH"))
+ *op_flag_value = RTE_BBDEV_TURBO_CRC_24B_ATTACH;
+ else if (!strcmp(token, "RTE_BBDEV_TURBO_CRC_24A_ATTACH"))
+ *op_flag_value = RTE_BBDEV_TURBO_CRC_24A_ATTACH;
+ else {
+ printf("The given value is not a turbo encoder flag\n");
+ return -1;
+ }
+
+ return 0;
+}
+
+/* tokenization turbo decoder/encoder flags values separated by a comma */
+static int
+parse_turbo_flags(char *tokens, uint32_t *op_flags,
+ enum rte_bbdev_op_type op_type)
+{
+ char *tok = NULL;
+ uint32_t op_flag_value = 0;
+
+ tok = strtok(tokens, VALUE_DELIMITER);
+ if (tok == NULL)
+ return -1;
+
+ while (tok != NULL) {
+ trim_space(tok);
+ if (op_type == RTE_BBDEV_OP_TURBO_DEC) {
+ if (op_decoder_flag_strtoul(tok, &op_flag_value) == -1)
+ return -1;
+ } else if (op_type == RTE_BBDEV_OP_TURBO_ENC) {
+ if (op_encoder_flag_strtoul(tok, &op_flag_value) == -1)
+ return -1;
+ } else {
+ return -1;
+ }
+
+ *op_flags = *op_flags | op_flag_value;
+
+ tok = strtok(NULL, VALUE_DELIMITER);
+ if (tok == NULL)
+ break;
+ }
+
+ return 0;
+}
+
+/* convert turbo encoder/decoder op_type from string to enum*/
+static int
+op_turbo_type_strtol(char *token, enum rte_bbdev_op_type *op_type)
+{
+ trim_space(token);
+ if (!strcmp(token, "RTE_BBDEV_OP_TURBO_DEC"))
+ *op_type = RTE_BBDEV_OP_TURBO_DEC;
+ else if (!strcmp(token, "RTE_BBDEV_OP_TURBO_ENC"))
+ *op_type = RTE_BBDEV_OP_TURBO_ENC;
+ else if (!strcmp(token, "RTE_BBDEV_OP_NONE"))
+ *op_type = RTE_BBDEV_OP_NONE;
+ else {
+ printf("Not valid turbo op_type: '%s'\n", token);
+ return -1;
+ }
+
+ return 0;
+}
+
+/* tokenization expected status values separated by a comma */
+static int
+parse_expected_status(char *tokens, int *status, enum rte_bbdev_op_type op_type)
+{
+ char *tok = NULL;
+ bool status_ok = false;
+
+ tok = strtok(tokens, VALUE_DELIMITER);
+ if (tok == NULL)
+ return -1;
+
+ while (tok != NULL) {
+ trim_space(tok);
+ if (!strcmp(tok, "OK"))
+ status_ok = true;
+ else if (!strcmp(tok, "DMA"))
+ *status = *status | (1 << RTE_BBDEV_DRV_ERROR);
+ else if (!strcmp(tok, "FCW"))
+ *status = *status | (1 << RTE_BBDEV_DATA_ERROR);
+ else if (!strcmp(tok, "CRC")) {
+ if (op_type == RTE_BBDEV_OP_TURBO_DEC)
+ *status = *status | (1 << RTE_BBDEV_CRC_ERROR);
+ else {
+ printf(
+ "CRC is only a valid value for turbo decoder\n");
+ return -1;
+ }
+ } else {
+ printf("Not valid status: '%s'\n", tok);
+ return -1;
+ }
+
+ tok = strtok(NULL, VALUE_DELIMITER);
+ if (tok == NULL)
+ break;
+ }
+
+ if (status_ok && *status != 0) {
+ printf(
+ "Not valid status values. Cannot be OK and ERROR at the same time.\n");
+ return -1;
+ }
+
+ return 0;
+}
+
+/* parse ops data entry (there can be more than 1 input entry, each will be
+ * contained in a separate op_data_buf struct)
+ */
+static int
+parse_data_entry(const char *key_token, char *token,
+ struct test_bbdev_vector *vector, enum op_data_type type,
+ const char *prefix)
+{
+ int ret;
+ uint32_t data_length = 0;
+ uint32_t *data = NULL;
+ unsigned int id;
+ struct op_data_buf *op_data;
+ unsigned int *nb_ops;
+
+ if (type > DATA_NUM_TYPES) {
+ printf("Unknown op type: %d!\n", type);
+ return -1;
+ }
+
+ op_data = vector->entries[type].segments;
+ nb_ops = &vector->entries[type].nb_segments;
+
+ if (*nb_ops >= RTE_BBDEV_MAX_CODE_BLOCKS) {
+ printf("Too many segments (code blocks defined): %u, max %d!\n",
+ *nb_ops, RTE_BBDEV_MAX_CODE_BLOCKS);
+ return -1;
+ }
+
+ if (sscanf(key_token + strlen(prefix), "%u", &id) != 1) {
+ printf("Missing ID of %s\n", prefix);
+ return -1;
+ }
+ if (id != *nb_ops) {
+ printf(
+ "Please order data entries sequentially, i.e. %s0, %s1, ...\n",
+ prefix, prefix);
+ return -1;
+ }
+
+ /* Clear new op data struct */
+ memset(op_data + *nb_ops, 0, sizeof(struct op_data_buf));
+
+ ret = parse_values(token, &data, &data_length);
+ if (!ret) {
+ op_data[*nb_ops].addr = data;
+ op_data[*nb_ops].length = data_length;
+ ++(*nb_ops);
+ }
+
+ return ret;
+}
+
+/* parses turbo decoder parameters and assigns to global variable */
+static int
+parse_decoder_params(const char *key_token, char *token,
+ struct test_bbdev_vector *vector)
+{
+ int ret = 0, status = 0;
+ uint32_t op_flags = 0;
+ char *err = NULL;
+
+ struct rte_bbdev_op_turbo_dec *turbo_dec = &vector->turbo_dec;
+
+ /* compare keys */
+ if (starts_with(key_token, op_data_prefixes[DATA_INPUT]))
+ ret = parse_data_entry(key_token, token, vector,
+ DATA_INPUT, op_data_prefixes[DATA_INPUT]);
+
+ else if (starts_with(key_token, op_data_prefixes[DATA_SOFT_OUTPUT]))
+ ret = parse_data_entry(key_token, token, vector,
+ DATA_SOFT_OUTPUT,
+ op_data_prefixes[DATA_SOFT_OUTPUT]);
+
+ else if (starts_with(key_token, op_data_prefixes[DATA_HARD_OUTPUT]))
+ ret = parse_data_entry(key_token, token, vector,
+ DATA_HARD_OUTPUT,
+ op_data_prefixes[DATA_HARD_OUTPUT]);
+ else if (!strcmp(key_token, "e")) {
+ vector->mask |= TEST_BBDEV_VF_E;
+ turbo_dec->cb_params.e = (uint32_t) strtoul(token, &err, 0);
+ } else if (!strcmp(key_token, "ea")) {
+ vector->mask |= TEST_BBDEV_VF_EA;
+ turbo_dec->tb_params.ea = (uint32_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "eb")) {
+ vector->mask |= TEST_BBDEV_VF_EB;
+ turbo_dec->tb_params.eb = (uint32_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "k")) {
+ vector->mask |= TEST_BBDEV_VF_K;
+ turbo_dec->cb_params.k = (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "k_pos")) {
+ vector->mask |= TEST_BBDEV_VF_K_POS;
+ turbo_dec->tb_params.k_pos = (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "k_neg")) {
+ vector->mask |= TEST_BBDEV_VF_K_NEG;
+ turbo_dec->tb_params.k_neg = (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "c")) {
+ vector->mask |= TEST_BBDEV_VF_C;
+ turbo_dec->tb_params.c = (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "c_neg")) {
+ vector->mask |= TEST_BBDEV_VF_C_NEG;
+ turbo_dec->tb_params.c_neg = (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "cab")) {
+ vector->mask |= TEST_BBDEV_VF_CAB;
+ turbo_dec->tb_params.cab = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "rv_index")) {
+ vector->mask |= TEST_BBDEV_VF_RV_INDEX;
+ turbo_dec->rv_index = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "iter_max")) {
+ vector->mask |= TEST_BBDEV_VF_ITER_MAX;
+ turbo_dec->iter_max = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "iter_min")) {
+ vector->mask |= TEST_BBDEV_VF_ITER_MIN;
+ turbo_dec->iter_min = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "expected_iter_count")) {
+ vector->mask |= TEST_BBDEV_VF_EXPECTED_ITER_COUNT;
+ turbo_dec->iter_count = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "ext_scale")) {
+ vector->mask |= TEST_BBDEV_VF_EXT_SCALE;
+ turbo_dec->ext_scale = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "num_maps")) {
+ vector->mask |= TEST_BBDEV_VF_NUM_MAPS;
+ turbo_dec->num_maps = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "code_block_mode")) {
+ vector->mask |= TEST_BBDEV_VF_CODE_BLOCK_MODE;
+ turbo_dec->code_block_mode = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "op_flags")) {
+ vector->mask |= TEST_BBDEV_VF_OP_FLAGS;
+ ret = parse_turbo_flags(token, &op_flags,
+ vector->op_type);
+ if (!ret)
+ turbo_dec->op_flags = op_flags;
+ } else if (!strcmp(key_token, "expected_status")) {
+ vector->mask |= TEST_BBDEV_VF_EXPECTED_STATUS;
+ ret = parse_expected_status(token, &status, vector->op_type);
+ if (!ret)
+ vector->expected_status = status;
+ } else {
+ printf("Not valid dec key: '%s'\n", key_token);
+ return -1;
+ }
+
+ if (ret != 0) {
+ printf("Failed with convert '%s\t%s'\n", key_token, token);
+ return -1;
+ }
+
+ return 0;
+}
+
+/* parses turbo encoder parameters and assigns to global variable */
+static int
+parse_encoder_params(const char *key_token, char *token,
+ struct test_bbdev_vector *vector)
+{
+ int ret = 0, status = 0;
+ uint32_t op_flags = 0;
+ char *err = NULL;
+
+
+ struct rte_bbdev_op_turbo_enc *turbo_enc = &vector->turbo_enc;
+
+ if (starts_with(key_token, op_data_prefixes[DATA_INPUT]))
+ ret = parse_data_entry(key_token, token, vector,
+ DATA_INPUT, op_data_prefixes[DATA_INPUT]);
+ else if (starts_with(key_token, "output"))
+ ret = parse_data_entry(key_token, token, vector,
+ DATA_HARD_OUTPUT, "output");
+ else if (!strcmp(key_token, "e")) {
+ vector->mask |= TEST_BBDEV_VF_E;
+ turbo_enc->cb_params.e = (uint32_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "ea")) {
+ vector->mask |= TEST_BBDEV_VF_EA;
+ turbo_enc->tb_params.ea = (uint32_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "eb")) {
+ vector->mask |= TEST_BBDEV_VF_EB;
+ turbo_enc->tb_params.eb = (uint32_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "k")) {
+ vector->mask |= TEST_BBDEV_VF_K;
+ turbo_enc->cb_params.k = (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "k_neg")) {
+ vector->mask |= TEST_BBDEV_VF_K_NEG;
+ turbo_enc->tb_params.k_neg = (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "k_pos")) {
+ vector->mask |= TEST_BBDEV_VF_K_POS;
+ turbo_enc->tb_params.k_pos = (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "c_neg")) {
+ vector->mask |= TEST_BBDEV_VF_C_NEG;
+ turbo_enc->tb_params.c_neg = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "c")) {
+ vector->mask |= TEST_BBDEV_VF_C;
+ turbo_enc->tb_params.c = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "cab")) {
+ vector->mask |= TEST_BBDEV_VF_CAB;
+ turbo_enc->tb_params.cab = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "rv_index")) {
+ vector->mask |= TEST_BBDEV_VF_RV_INDEX;
+ turbo_enc->rv_index = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "ncb")) {
+ vector->mask |= TEST_BBDEV_VF_NCB;
+ turbo_enc->cb_params.ncb = (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "ncb_neg")) {
+ vector->mask |= TEST_BBDEV_VF_NCB_NEG;
+ turbo_enc->tb_params.ncb_neg =
+ (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "ncb_pos")) {
+ vector->mask |= TEST_BBDEV_VF_NCB_POS;
+ turbo_enc->tb_params.ncb_pos =
+ (uint16_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "code_block_mode")) {
+ vector->mask |= TEST_BBDEV_VF_CODE_BLOCK_MODE;
+ turbo_enc->code_block_mode = (uint8_t) strtoul(token, &err, 0);
+ ret = ((err == NULL) || (*err != '\0')) ? -1 : 0;
+ } else if (!strcmp(key_token, "op_flags")) {
+ vector->mask |= TEST_BBDEV_VF_OP_FLAGS;
+ ret = parse_turbo_flags(token, &op_flags,
+ vector->op_type);
+ if (!ret)
+ turbo_enc->op_flags = op_flags;
+ } else if (!strcmp(key_token, "expected_status")) {
+ vector->mask |= TEST_BBDEV_VF_EXPECTED_STATUS;
+ ret = parse_expected_status(token, &status, vector->op_type);
+ if (!ret)
+ vector->expected_status = status;
+ } else {
+ printf("Not valid enc key: '%s'\n", key_token);
+ return -1;
+ }
+
+ if (ret != 0) {
+ printf("Failed with convert '%s\t%s'\n", key_token, token);
+ return -1;
+ }
+
+ return 0;
+}
+
+/* checks the type of key and assigns data */
+static int
+parse_entry(char *entry, struct test_bbdev_vector *vector)
+{
+ int ret = 0;
+ char *token, *key_token;
+ enum rte_bbdev_op_type op_type = RTE_BBDEV_OP_NONE;
+
+ if (entry == NULL) {
+ printf("Expected entry value\n");
+ return -1;
+ }
+
+ /* get key */
+ token = strtok(entry, ENTRY_DELIMITER);
+ key_token = token;
+ /* get values for key */
+ token = strtok(NULL, ENTRY_DELIMITER);
+
+ if (key_token == NULL || token == NULL) {
+ printf("Expected 'key = values' but was '%.40s'..\n", entry);
+ return -1;
+ }
+ trim_space(key_token);
+
+ /* first key_token has to specify type of operation */
+ if (vector->op_type == RTE_BBDEV_OP_NONE) {
+ if (!strcmp(key_token, "op_type")) {
+ ret = op_turbo_type_strtol(token, &op_type);
+ if (!ret)
+ vector->op_type = op_type;
+ return (!ret) ? 0 : -1;
+ }
+ printf("First key_token (%s) does not specify op_type\n",
+ key_token);
+ return -1;
+ }
+
+ /* compare keys */
+ if (vector->op_type == RTE_BBDEV_OP_TURBO_DEC) {
+ if (parse_decoder_params(key_token, token, vector) == -1)
+ return -1;
+ } else if (vector->op_type == RTE_BBDEV_OP_TURBO_ENC) {
+ if (parse_encoder_params(key_token, token, vector) == -1)
+ return -1;
+ }
+
+ return 0;
+}
+
+/* checks decoder parameters */
+static int
+check_decoder(struct test_bbdev_vector *vector, const int mask)
+{
+ unsigned char i;
+ struct rte_bbdev_op_turbo_dec *turbo_dec = &vector->turbo_dec;
+
+ if (vector->entries[DATA_INPUT].nb_segments == 0)
+ return -1;
+
+ for (i = 0; i < vector->entries[DATA_INPUT].nb_segments; i++)
+ if (vector->entries[DATA_INPUT].
+ segments[i].addr == NULL)
+ return -1;
+
+ if (vector->entries[DATA_HARD_OUTPUT].nb_segments == 0)
+ return -1;
+
+ for (i = 0; i < vector->entries[DATA_HARD_OUTPUT].nb_segments;
+ i++)
+ if (vector->entries[DATA_HARD_OUTPUT].
+ segments[i].addr == NULL)
+ return -1;
+
+ if ((turbo_dec->op_flags & RTE_BBDEV_TURBO_SOFT_OUTPUT) &&
+ (vector->entries[DATA_SOFT_OUTPUT].nb_segments == 0))
+ return -1;
+
+ for (i = 0; i < vector->entries[DATA_SOFT_OUTPUT].nb_segments;
+ i++)
+ if (vector->entries[DATA_SOFT_OUTPUT].
+ segments[i].addr == NULL)
+ return -1;
+
+ if (!(mask & TEST_BBDEV_VF_CODE_BLOCK_MODE)) {
+ printf(
+ "WARNING: code_block_mode was not specified in vector file and will be set to 1 (0 - TB Mode, 1 - CB mode)\n");
+ turbo_dec->code_block_mode = 1;
+ }
+ if (turbo_dec->code_block_mode == 0) {
+ if (!(mask & TEST_BBDEV_VF_EA))
+ printf(
+ "WARNING: ea was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_EB))
+ printf(
+ "WARNING: eb was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_K_NEG))
+ printf(
+ "WARNING: k_neg was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_K_POS))
+ printf(
+ "WARNING: k_pos was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_C_NEG))
+ printf(
+ "WARNING: c_neg was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_C)) {
+ printf(
+ "WARNING: c was not specified in vector file and will be set to 1\n");
+ turbo_dec->tb_params.c = 1;
+ }
+ if (!(mask & TEST_BBDEV_VF_CAB))
+ printf(
+ "WARNING: cab was not specified in vector file and will be set to 0\n");
+ } else {
+ if (!(mask & TEST_BBDEV_VF_E))
+ printf(
+ "WARNING: e was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_K))
+ printf(
+ "WARNING: k was not specified in vector file and will be set to 0\n");
+ }
+ if (!(mask & TEST_BBDEV_VF_RV_INDEX))
+ printf(
+ "WARNING: rv_index was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_ITER_MIN))
+ printf(
+ "WARNING: iter_min was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_ITER_MAX))
+ printf(
+ "WARNING: iter_max was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_EXPECTED_ITER_COUNT))
+ printf(
+ "WARNING: expected_iter_count was not specified in vector file and iter_count will not be validated\n");
+ if (!(mask & TEST_BBDEV_VF_EXT_SCALE))
+ printf(
+ "WARNING: ext_scale was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_NUM_MAPS))
+ printf(
+ "WARNING: num_maps was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_OP_FLAGS))
+ printf(
+ "WARNING: op_flags was not specified in vector file and capabilities will not be validated\n");
+ if (!(mask & TEST_BBDEV_VF_EXPECTED_STATUS))
+ printf(
+ "WARNING: expected_status was not specified in vector file and will be set to 0\n");
+ return 0;
+}
+
+/* checks encoder parameters */
+static int
+check_encoder(struct test_bbdev_vector *vector, const int mask)
+{
+ unsigned char i;
+
+ if (vector->entries[DATA_INPUT].nb_segments == 0)
+ return -1;
+
+ for (i = 0; i < vector->entries[DATA_INPUT].nb_segments; i++)
+ if (vector->entries[DATA_INPUT].
+ segments[i].addr == NULL)
+ return -1;
+
+ if (vector->entries[DATA_HARD_OUTPUT].nb_segments == 0)
+ return -1;
+
+ for (i = 0; i < vector->entries[DATA_HARD_OUTPUT].nb_segments;
+ i++)
+ if (vector->entries[DATA_HARD_OUTPUT].
+ segments[i].addr == NULL)
+ return -1;
+
+ if (!(mask & TEST_BBDEV_VF_CODE_BLOCK_MODE)) {
+ printf(
+ "WARNING: code_block_mode was not specified in vector file and will be set to 1\n");
+ vector->turbo_enc.code_block_mode = 1;
+ }
+ if (vector->turbo_enc.code_block_mode == 0) {
+ if (!(mask & TEST_BBDEV_VF_EA))
+ printf(
+ "WARNING: ea was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_EB))
+ printf(
+ "WARNING: eb was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_K_NEG))
+ printf(
+ "WARNING: k_neg was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_K_POS))
+ printf(
+ "WARNING: k_pos was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_C_NEG))
+ printf(
+ "WARNING: c_neg was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_C)) {
+ printf(
+ "WARNING: c was not specified in vector file and will be set to 1\n");
+ vector->turbo_enc.tb_params.c = 1;
+ }
+ if (!(mask & TEST_BBDEV_VF_CAB))
+ printf(
+ "WARNING: cab was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_NCB_NEG))
+ printf(
+ "WARNING: ncb_neg was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_NCB_POS))
+ printf(
+ "WARNING: ncb_pos was not specified in vector file and will be set to 0\n");
+ } else {
+ if (!(mask & TEST_BBDEV_VF_E))
+ printf(
+ "WARNING: e was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_K))
+ printf(
+ "WARNING: k was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_NCB))
+ printf(
+ "WARNING: ncb was not specified in vector file and will be set to 0\n");
+ }
+ if (!(mask & TEST_BBDEV_VF_RV_INDEX))
+ printf(
+ "WARNING: rv_index was not specified in vector file and will be set to 0\n");
+ if (!(mask & TEST_BBDEV_VF_OP_FLAGS))
+ printf(
+ "WARNING: op_flags was not specified in vector file and capabilities will not be validated\n");
+ if (!(mask & TEST_BBDEV_VF_EXPECTED_STATUS))
+ printf(
+ "WARNING: expected_status was not specified in vector file and will be set to 0\n");
+
+ return 0;
+}
+
+static int
+bbdev_check_vector(struct test_bbdev_vector *vector)
+{
+ if (vector->op_type == RTE_BBDEV_OP_TURBO_DEC) {
+ if (check_decoder(vector, vector->mask) == -1)
+ return -1;
+ } else if (vector->op_type == RTE_BBDEV_OP_TURBO_ENC) {
+ if (check_encoder(vector, vector->mask) == -1)
+ return -1;
+ } else if (vector->op_type != RTE_BBDEV_OP_NONE) {
+ printf("Vector was not filled\n");
+ return -1;
+ }
+
+ return 0;
+}
+
+int
+test_bbdev_vector_read(const char *filename,
+ struct test_bbdev_vector *vector)
+{
+ int ret = 0;
+ size_t len = 0;
+
+ FILE *fp = NULL;
+ char *line = NULL;
+ char *entry = NULL;
+
+ fp = fopen(filename, "r");
+ if (fp == NULL) {
+ printf("File %s does not exist\n", filename);
+ return -1;
+ }
+
+ while (getline(&line, &len, fp) != -1) {
+
+ /* ignore comments and new lines */
+ if (line[0] == '#' || line[0] == '/' || line[0] == '\n'
+ || line[0] == '\r')
+ continue;
+
+ trim_space(line);
+
+ /* buffer for multiline */
+ entry = realloc(entry, strlen(line) + 1);
+ if (entry == NULL) {
+ printf("Fail to realloc %zu bytes\n", strlen(line) + 1);
+ ret = -ENOMEM;
+ goto exit;
+ }
+
+ memset(entry, 0, strlen(line) + 1);
+ strncpy(entry, line, strlen(line));
+
+ /* check if entry ends with , or = */
+ if (entry[strlen(entry) - 1] == ','
+ || entry[strlen(entry) - 1] == '=') {
+ while (getline(&line, &len, fp) != -1) {
+ trim_space(line);
+
+ /* extend entry about length of new line */
+ char *entry_extended = realloc(entry,
+ strlen(line) +
+ strlen(entry) + 1);
+
+ if (entry_extended == NULL) {
+ printf("Fail to allocate %zu bytes\n",
+ strlen(line) +
+ strlen(entry) + 1);
+ ret = -ENOMEM;
+ goto exit;
+ }
+
+ entry = entry_extended;
+ strncat(entry, line, strlen(line));
+
+ if (entry[strlen(entry) - 1] != ',')
+ break;
+ }
+ }
+ ret = parse_entry(entry, vector);
+ if (ret != 0) {
+ printf("An error occurred while parsing!\n");
+ goto exit;
+ }
+ }
+
+ ret = bbdev_check_vector(vector);
+ if (ret != 0)
+ printf("An error occurred while checking!\n");
+
+exit:
+ fclose(fp);
+ free(line);
+ free(entry);
+
+ return ret;
+}
diff --git a/app/test-bbdev/test_bbdev_vector.h b/app/test-bbdev/test_bbdev_vector.h
new file mode 100644
index 0000000..75de330
--- /dev/null
+++ b/app/test-bbdev/test_bbdev_vector.h
@@ -0,0 +1,98 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef TEST_BBDEV_VECTOR_H_
+#define TEST_BBDEV_VECTOR_H_
+
+#include <rte_bbdev_op.h>
+
+/* Flags which are set when specific parameter is define in vector file */
+enum {
+ TEST_BBDEV_VF_E = (1ULL << 0),
+ TEST_BBDEV_VF_EA = (1ULL << 1),
+ TEST_BBDEV_VF_EB = (1ULL << 2),
+ TEST_BBDEV_VF_K = (1ULL << 3),
+ TEST_BBDEV_VF_K_NEG = (1ULL << 4),
+ TEST_BBDEV_VF_K_POS = (1ULL << 5),
+ TEST_BBDEV_VF_C_NEG = (1ULL << 6),
+ TEST_BBDEV_VF_C = (1ULL << 7),
+ TEST_BBDEV_VF_CAB = (1ULL << 8),
+ TEST_BBDEV_VF_RV_INDEX = (1ULL << 9),
+ TEST_BBDEV_VF_ITER_MAX = (1ULL << 10),
+ TEST_BBDEV_VF_ITER_MIN = (1ULL << 11),
+ TEST_BBDEV_VF_EXPECTED_ITER_COUNT = (1ULL << 12),
+ TEST_BBDEV_VF_EXT_SCALE = (1ULL << 13),
+ TEST_BBDEV_VF_NUM_MAPS = (1ULL << 14),
+ TEST_BBDEV_VF_NCB = (1ULL << 15),
+ TEST_BBDEV_VF_NCB_NEG = (1ULL << 16),
+ TEST_BBDEV_VF_NCB_POS = (1ULL << 17),
+ TEST_BBDEV_VF_CODE_BLOCK_MODE = (1ULL << 18),
+ TEST_BBDEV_VF_OP_FLAGS = (1ULL << 19),
+ TEST_BBDEV_VF_EXPECTED_STATUS = (1ULL << 20),
+};
+
+enum op_data_type {
+ DATA_INPUT = 0,
+ DATA_SOFT_OUTPUT,
+ DATA_HARD_OUTPUT,
+ DATA_NUM_TYPES,
+};
+
+struct op_data_buf {
+ uint32_t *addr;
+ uint32_t length;
+};
+
+struct op_data_entries {
+ struct op_data_buf segments[RTE_BBDEV_MAX_CODE_BLOCKS];
+ unsigned int nb_segments;
+};
+
+struct test_bbdev_vector {
+ enum rte_bbdev_op_type op_type;
+ int expected_status;
+ int mask;
+ union {
+ struct rte_bbdev_op_turbo_dec turbo_dec;
+ struct rte_bbdev_op_turbo_enc turbo_enc;
+ };
+ /* Additional storage for op data entries */
+ struct op_data_entries entries[DATA_NUM_TYPES];
+};
+
+/* fills test vector paramaters based on test file */
+int
+test_bbdev_vector_read(const char *filename,
+ struct test_bbdev_vector *vector);
+
+
+#endif /* TEST_BBDEV_VECTOR_H_ */
diff --git a/app/test-bbdev/test_vectors/bbdev_vector_null.data b/app/test-bbdev/test_vectors/bbdev_vector_null.data
new file mode 100644
index 0000000..91aea62
--- /dev/null
+++ b/app/test-bbdev/test_vectors/bbdev_vector_null.data
@@ -0,0 +1,32 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+op_type =
+RTE_BBDEV_OP_NONE
\ No newline at end of file
diff --git a/app/test-bbdev/test_vectors/bbdev_vector_td_default.data b/app/test-bbdev/test_vectors/bbdev_vector_td_default.data
new file mode 100644
index 0000000..db20640
--- /dev/null
+++ b/app/test-bbdev/test_vectors/bbdev_vector_td_default.data
@@ -0,0 +1,80 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+op_type =
+RTE_BBDEV_OP_TURBO_DEC
+
+input0 =
+0x7f007f00, 0x7f817f00, 0x767f8100, 0x817f8100, 0x81008100, 0x7f818100, 0x81817f00, 0x7f818100,
+0x86007f00, 0x7f818100, 0x887f8100, 0x81815200, 0x81008100, 0x817f7f00, 0x7f7f8100, 0x9e817f00,
+0x7f7f0000, 0xb97f0000, 0xa7810000, 0x7f7f4a7f, 0x7f810000, 0x7f7f7f7f, 0x81720000, 0x40658181,
+0x84810000, 0x817f0000, 0x81810000, 0x7f818181, 0x7f810000, 0x81815a81, 0x817f0000, 0x7a867f7b,
+0x817f0000, 0x6b7f0000, 0x7f810000, 0x81818181, 0x817f0000, 0x7f7f817f, 0x7f7f0000, 0xab7f4f7f,
+0x817f0000, 0x817f6c00, 0x81810000, 0x817f8181, 0x7f810000, 0x81816981, 0x7f7f0000, 0x007f8181
+
+hard_output0 =
+0xa7d6732e, 0x61
+
+soft_output0 =
+0x7f7f7f7f, 0x81817f7f, 0x7f817f81, 0x817f7f81, 0x81817f81, 0x81817f81, 0x8181817f, 0x7f81817f,
+0x7f81817f, 0x7f817f7f, 0x81817f7f, 0x817f8181, 0x81818181, 0x817f7f7f, 0x7f818181, 0x817f817f,
+0x81818181, 0x81817f7f, 0x7f817f81, 0x7f81817f, 0x817f7f7f, 0x817f7f7f, 0x7f81817f, 0x817f817f,
+0x81817f7f, 0x81817f7f, 0x81817f7f, 0x7f817f7f, 0x817f7f81, 0x7f7f8181, 0x81817f81, 0x817f7f7f,
+0x7f7f8181
+
+e =
+17280
+
+k =
+40
+
+rv_index =
+1
+
+iter_max =
+8
+
+iter_min =
+4
+
+expected_iter_count =
+8
+
+ext_scale =
+15
+
+num_maps =
+0
+
+op_flags =
+RTE_BBDEV_TURBO_SOFT_OUTPUT, RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE
+
+expected_status =
+OK
diff --git a/app/test-bbdev/test_vectors/bbdev_vector_te_default.data b/app/test-bbdev/test_vectors/bbdev_vector_te_default.data
new file mode 100644
index 0000000..b5cecf4
--- /dev/null
+++ b/app/test-bbdev/test_vectors/bbdev_vector_te_default.data
@@ -0,0 +1,60 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+op_type =
+RTE_BBDEV_OP_TURBO_ENC
+
+input0 =
+0x11d2bcac, 0x4d
+
+output0 =
+0xd2399179, 0x640eb999, 0x2cbaf577, 0xaf224ae2, 0x9d139927, 0xe6909b29, 0xa25b7f47, 0x2aa224ce,
+0x79f2
+
+e =
+272
+
+k =
+40
+
+ncb =
+192
+
+rv_index =
+0
+
+code_block_mode =
+1
+
+op_flags =
+RTE_BBDEV_TURBO_RATE_MATCH
+
+expected_status =
+OK
--
2.7.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [dpdk-dev] [PATCH v2 4/5] bbdev: sample app
2017-10-18 2:14 [dpdk-dev] [PATCH v2 1/5] bbdev: librte_bbdev library Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 2/5] bbdev: PMD drivers (null/turbo_sw) Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 3/5] bbdev: test applications Amr Mokhtar
@ 2017-10-18 2:14 ` Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 5/5] bbdev: documentation Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 0/5] Wireless Base Band Device (bbdev) Amr Mokhtar
4 siblings, 0 replies; 6+ messages in thread
From: Amr Mokhtar @ 2017-10-18 2:14 UTC (permalink / raw)
To: dev
Cc: thomas, anatoly.burakov, pablo.de.lara.guarch, niall.power,
chris.macnamara, Amr Mokhtar
- Sample application performing a loop-back over ethernet using
a bbbdev device
- A packet is received on an Rx ethdev port -> enqueued for baseband
operation -> dequeued -> looped-back to a Tx ethdev port
- 'Turbo_sw' PMD must be enabled for the app to be functional
Signed-off-by: Amr Mokhtar <amr.mokhtar@intel.com>
---
examples/Makefile | 1 +
examples/bbdev_app/Makefile | 50 ++
examples/bbdev_app/main.c | 1396 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 1447 insertions(+)
create mode 100644 examples/bbdev_app/Makefile
create mode 100644 examples/bbdev_app/main.c
diff --git a/examples/Makefile b/examples/Makefile
index d27eddd..535f36f 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -59,6 +59,7 @@ ifneq ($(PQOS_INSTALL_PATH),)
DIRS-y += l2fwd-cat
endif
DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += l2fwd-crypto
+DIRS-$(CONFIG_RTE_LIBRTE_BBDEV) += bbdev_app
DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += l2fwd-jobstats
DIRS-y += l2fwd-keepalive
DIRS-y += l2fwd-keepalive/ka-agent
diff --git a/examples/bbdev_app/Makefile b/examples/bbdev_app/Makefile
new file mode 100644
index 0000000..1583251
--- /dev/null
+++ b/examples/bbdev_app/Makefile
@@ -0,0 +1,50 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# binary name
+APP = bbdev
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/bbdev_app/main.c b/examples/bbdev_app/main.c
new file mode 100644
index 0000000..55bf23f
--- /dev/null
+++ b/examples/bbdev_app/main.c
@@ -0,0 +1,1396 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <sys/types.h>
+#include <sys/unistd.h>
+#include <sys/queue.h>
+#include <stdarg.h>
+#include <ctype.h>
+#include <errno.h>
+#include <math.h>
+#include <assert.h>
+#include <getopt.h>
+#include <signal.h>
+#include <inttypes.h>
+
+#include "rte_common.h"
+#include "rte_eal.h"
+#include "rte_cycles.h"
+#include "rte_eal.h"
+#include "rte_ether.h"
+#include "rte_ethdev.h"
+#include "rte_ip.h"
+#include "rte_lcore.h"
+#include "rte_malloc.h"
+#include "rte_mbuf.h"
+#include "rte_memory.h"
+#include "rte_mempool.h"
+#include "rte_log.h"
+#include "rte_bbdev.h"
+#include "rte_bbdev_op.h"
+
+#define MAX_PKT_BURST 32
+#define NB_MBUF 8163
+#define MAX_RX_QUEUE_PER_LCORE 16
+#define MAX_TX_QUEUE_PER_LCORE 16
+#define MEMPOOL_CACHE_SIZE 256
+
+/*Configurable number of RX/TX ring descriptors */
+#define RTE_TEST_RX_DESC_DEFAULT 128
+#define RTE_TEST_TX_DESC_DEFAULT 512
+
+#define BBDEV_ASSERT(a) do { \
+ if (!(a)) { \
+ usage(prgname); \
+ return -1; \
+ } \
+} while (0)
+
+static struct rte_mempool *ethdev_mbuf_mempool;
+
+struct dev_qs {
+ unsigned int port_id;
+ unsigned int queues_list[RTE_MAX_QUEUES_PER_PORT];
+ unsigned int queues;
+};
+
+static struct rte_mempool *bbdev_op_pool[RTE_BBDEV_OP_TYPE_COUNT];
+
+/**all lcores used*/
+struct lcore_setup {
+ unsigned int rxte_lcores;
+ unsigned int rxte_lcore_list[RTE_MAX_LCORE];
+ unsigned int tetx_lcores;
+ unsigned int tetx_lcore_list[RTE_MAX_LCORE];
+ unsigned int rxtd_lcores;
+ unsigned int rxtd_lcore_list[RTE_MAX_LCORE];
+ unsigned int tdtx_lcores;
+ unsigned int tdtx_lcore_list[RTE_MAX_LCORE];
+};
+
+/** each lcore configuration */
+struct lcore_queue_conf {
+ unsigned int nb_ports;
+ struct dev_qs port_list[RTE_MAX_ETHPORTS];
+ /* ethernet addresses of ports */
+ struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS];
+
+ unsigned int bbdev_id;
+
+ unsigned int bbdev_qs[128];
+ unsigned int nb_bbdev_qs;
+
+ struct rte_mempool *mbuf_pool;
+} __rte_cache_aligned;
+
+static struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];
+
+static const struct rte_eth_conf port_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_NONE,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+};
+
+struct rte_bbdev_op_turbo_enc def_op_enc = {
+ /* These values are arbitrarily put, and does not map to the real
+ * values for the data received from ethdev ports
+ */
+ .rv_index = 0,
+ .code_block_mode = 1,
+ .cb_params.e = 272,
+ .cb_params.ncb = 192,
+ .op_flags = RTE_BBDEV_TURBO_CRC_24B_ATTACH
+};
+
+struct rte_bbdev_op_turbo_dec def_op_dec = {
+ /* These values are arbitrarily put, and does not map to the real
+ * values for the data received from ethdev ports
+ */
+ .cb_params.e = 44,
+ .rv_index = 0,
+ .iter_max = 8,
+ .iter_min = 4,
+ .ext_scale = 15,
+ .num_maps = 0,
+ .op_flags = RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE |
+ RTE_BBDEV_TURBO_EQUALIZER | RTE_BBDEV_TURBO_SOFT_OUTPUT
+};
+
+struct bbdev_config_params {
+ uint8_t downlink_lcores;
+ uint8_t uplink_lcores;
+
+ uint8_t downlink_rx_ports;
+ uint8_t downlink_tx_ports;
+
+ uint8_t uplink_rx_ports;
+ uint8_t uplink_tx_ports;
+
+ unsigned int rx_port_list[MAX_RX_QUEUE_PER_LCORE];
+ unsigned int tx_port_list[MAX_TX_QUEUE_PER_LCORE];
+};
+
+struct lcore_statistics {
+ unsigned int enqueued;
+ unsigned int dequeued;
+ unsigned int lost_packets;
+} __rte_cache_aligned;
+
+static struct lcore_statistics lcore_stats[RTE_MAX_LCORE];
+
+static volatile int global_exit_flag;
+
+struct port_queues {
+ unsigned int nb_rx_qs;
+ unsigned int nb_tx_qs;
+};
+
+static struct port_queues ports_qs[RTE_MAX_ETHPORTS];
+
+/* display usage */
+static inline void
+usage(const char *prgname)
+{
+ printf("%s [EAL options] "
+ " --\n"
+ " --downlink_cores - downlink cores mask\n"
+ " --uplink_cores - uplink cores mask\n"
+ " --downlink_rx_ports - downlink Rx ports mask\n"
+ " --downlink_tx_ports - downlink Tx ports mask\n"
+ " --uplink_tx_ports -uplink Tx ports mask\n"
+ " --uplink_rx_ports -uplink Rx ports msk\n"
+ "\n", prgname);
+}
+
+/* parse port or core mask */
+static inline
+uint8_t bbdev_parse_mask(const char *mask)
+{
+ char *end = NULL;
+ unsigned long pm;
+
+ /* parse hexadecimal string */
+ pm = strtoul(mask, &end, 16);
+ if ((mask[0] == '\0') || (end == NULL) || (*end != '\0'))
+ return 0;
+
+ if (pm == 0)
+ return 0;
+
+ return pm;
+}
+
+static int
+bbdev_parse_args(int argc, char **argv,
+ struct bbdev_config_params *bbdev_params)
+{
+ int optind = 0;
+ int opt;
+ int opt_indx = 0;
+ char *prgname = argv[0];
+
+ memset(bbdev_params, 0, sizeof(*bbdev_params));
+
+ static struct option lgopts[] = {
+ { "downlink_cores", required_argument, 0, 'c' },
+ { "uplink_cores", required_argument, 0, 'C' },
+ { "downlink_rx_ports", required_argument, 0, 'r' },
+ { "downlink_tx_ports", required_argument, 0, 't' },
+ { "uplink_rx_ports", required_argument, 0, 'R' },
+ { "uplink_tx_ports", required_argument, 0, 'T' },
+ { NULL, 0, 0, 0 }
+ };
+
+ BBDEV_ASSERT(argc != 0);
+ BBDEV_ASSERT(argv != NULL);
+ BBDEV_ASSERT(bbdev_params != NULL);
+
+ while ((opt = getopt_long(argc, argv, "c:C:r:t:R:T:", lgopts,
+ &opt_indx)) != EOF) {
+
+ switch (opt) {
+ case 'c':
+ bbdev_params->downlink_lcores =
+ bbdev_parse_mask(optarg);
+ if (bbdev_params->downlink_lcores == 0) {
+ usage(prgname);
+ return -1;
+ }
+ break;
+
+ case 'C':
+ bbdev_params->uplink_lcores =
+ bbdev_parse_mask(optarg);
+ if (bbdev_params->uplink_lcores == 0) {
+ usage(prgname);
+ return -1;
+ }
+ break;
+
+ case 'r':
+ bbdev_params->downlink_rx_ports =
+ bbdev_parse_mask(optarg);
+ if (bbdev_params->downlink_rx_ports == 0) {
+ usage(prgname);
+ return -1;
+ }
+ break;
+
+ case 't':
+ bbdev_params->downlink_tx_ports =
+ bbdev_parse_mask(optarg);
+ if (bbdev_params->downlink_tx_ports == 0) {
+ usage(prgname);
+ return -1;
+ }
+ break;
+
+ case 'R':
+ bbdev_params->uplink_rx_ports =
+ bbdev_parse_mask(optarg);
+ if (bbdev_params->uplink_rx_ports == 0) {
+ usage(prgname);
+ return -1;
+ }
+ break;
+
+ case 'T':
+ bbdev_params->uplink_tx_ports =
+ bbdev_parse_mask(optarg);
+ if (bbdev_params->uplink_tx_ports == 0) {
+ usage(prgname);
+ return -1;
+ }
+ break;
+
+ default:
+ usage(prgname);
+ return -1;
+ }
+ }
+ optind = 0;
+ return optind;
+}
+
+static void
+signal_handler(int signum)
+{
+ printf("\nSignal %d received", signum);
+ global_exit_flag = 1;
+}
+
+static void
+print_mac(unsigned int portid, struct ether_addr *bbdev_ports_eth_address)
+{
+ printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+ (unsigned int) portid,
+ bbdev_ports_eth_address[portid].addr_bytes[0],
+ bbdev_ports_eth_address[portid].addr_bytes[1],
+ bbdev_ports_eth_address[portid].addr_bytes[2],
+ bbdev_ports_eth_address[portid].addr_bytes[3],
+ bbdev_ports_eth_address[portid].addr_bytes[4],
+ bbdev_ports_eth_address[portid].addr_bytes[5]);
+}
+
+/* Check the link status of all ports in up to 9s, and print them finally */
+static void
+check_all_ports_link_status(uint8_t port_num, uint32_t port_mask)
+{
+#define CHECK_INTERVAL 100 /* 100ms */
+#define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */
+ uint8_t portid, count, all_ports_up, print_flag = 0;
+ struct rte_eth_link link;
+
+ printf("\nChecking link status");
+ fflush(stdout);
+
+ for (count = 0; count <= MAX_CHECK_TIME; count++) {
+ all_ports_up = 1;
+ for (portid = 0; portid < port_num; portid++) {
+ if ((port_mask & (1 << portid)) == 0)
+ continue;
+ memset(&link, 0, sizeof(link));
+ rte_eth_link_get_nowait(portid, &link);
+ /* print link status if flag set */
+ if (print_flag == 1) {
+ if (link.link_status) {
+ const char *dp = (link.link_duplex ==
+ ETH_LINK_FULL_DUPLEX) ?
+ "full-duplex" :
+ "half-duplex";
+ printf("Port %u Link Up - speed %u Mbps"
+ " - %s\n", portid,
+ link.link_speed,
+ dp);
+ } else
+ printf("Port %d Link Down\n", portid);
+ continue;
+ }
+ /* clear all_ports_up flag if any link down */
+ if (link.link_status == 0) {
+ all_ports_up = 0;
+ break;
+ }
+ }
+ /* after finally printing all link status, get out */
+ if (print_flag == 1)
+ break;
+
+ if (all_ports_up == 0) {
+ printf(".");
+ fflush(stdout);
+ rte_delay_ms(CHECK_INTERVAL);
+ }
+
+ /* set the print_flag if all ports up or timeout */
+ if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) {
+ print_flag = 1;
+ printf("done\n");
+ }
+ }
+}
+
+static void
+update_pkt_mac(uint16_t n, struct rte_mbuf *pkts_burst[],
+ struct ether_addr *addr, unsigned int dest_portid)
+{
+ struct ether_hdr *eth;
+ void *tmp;
+
+ for (uint16_t i = 0; i < n; i++) {
+ eth = rte_pktmbuf_mtod(pkts_burst[i], struct ether_hdr *);
+ /* 02:00:00:00:00:xx */
+ tmp = ð->d_addr.addr_bytes[0];
+ *((uint64_t *)tmp) = 0x000000000002 +
+ ((uint64_t)dest_portid << 40);
+ /* src addr */
+ ether_addr_copy(addr, ð->s_addr);
+ }
+}
+
+static int
+initialize_ports(uint8_t ports_mask, char link, char port_type,
+ uint32_t *bbdev_enabled_port_mask)
+{
+ uint8_t portid;
+ unsigned int enabled_portcount = 0, q;
+ int ret;
+ struct port_queues *port_qs;
+ /* ethernet addresses of ports */
+ struct ether_addr bbdev_dl_ports_eth_addr[RTE_MAX_ETHPORTS];
+ struct ether_addr bbdev_ul_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+ uint8_t nb_ports = rte_eth_dev_count();
+ if (nb_ports == 0) {
+ printf("No Ethernet ports available\n");
+ return -1;
+ }
+
+ for (portid = 0; portid < nb_ports; portid++) {
+
+ /* Skip ports that are not enabled */
+ if ((ports_mask & (1 << portid)) == 0)
+ continue;
+
+ port_qs = &ports_qs[portid];
+ /* initialize ports */
+ printf("\nInitializing %cx port %u... ",
+ port_type, portid);
+ fflush(stdout);
+ ret = rte_eth_dev_configure(portid, port_qs->nb_rx_qs,
+ port_qs->nb_tx_qs, &port_conf);
+
+ if (ret < 0) {
+ printf("Cannot configure device: err=%d, port=%u\n",
+ ret, portid);
+ return -1;
+ }
+ fflush(stdout);
+
+ /* initialize one RX or TX queue on each port*/
+ if (port_type == 'R')
+ for (q = 0; q < port_qs->nb_rx_qs; q++)
+ ret = rte_eth_rx_queue_setup(portid, q,
+ RTE_TEST_RX_DESC_DEFAULT,
+ rte_eth_dev_socket_id(portid),
+ NULL, ethdev_mbuf_mempool);
+ else if (port_type == 'T')
+ for (q = 0; q < port_qs->nb_rx_qs; q++)
+ ret = rte_eth_tx_queue_setup(portid, q,
+ RTE_TEST_TX_DESC_DEFAULT,
+ rte_eth_dev_socket_id(portid),
+ NULL);
+
+ if (ret < 0) {
+ printf("%cL rte_eth_%cx_queue_setup:err=%d, port=%u\n",
+ port_type, link, ret,
+ (unsigned int) portid);
+ return -1;
+ }
+
+ rte_eth_promiscuous_enable(portid);
+
+ if (link == 'D') {
+ rte_eth_macaddr_get(portid,
+ &bbdev_dl_ports_eth_addr[portid]);
+ print_mac(portid, bbdev_dl_ports_eth_addr);
+ } else {
+ rte_eth_macaddr_get(portid,
+ &bbdev_ul_ports_eth_addr[portid]);
+ print_mac(portid, bbdev_ul_ports_eth_addr);
+ }
+
+ *bbdev_enabled_port_mask |= (1 << portid);
+ enabled_portcount++;
+ }
+ return enabled_portcount;
+}
+
+static int
+start_ethdev_ports(uint8_t nb_ports, uint32_t bbdev_enabled_port_mask)
+{
+ unsigned int portid;
+ int ret, ports_started = 0;
+
+ for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) {
+
+ if ((bbdev_enabled_port_mask & (uint32_t)(1 << portid)) == 0)
+ continue;
+ ret = rte_eth_dev_start(portid);
+ if (ret < 0) {
+ printf("rte_eth_dev_start:err=%d, port=%u\n", ret,
+ (unsigned int) portid);
+ return -1;
+ }
+
+ ports_started++;
+ if (ports_started == nb_ports)
+ break;
+ }
+ return ports_started;
+}
+
+static int
+lcore_queue_init(uint8_t nb_ports, struct bbdev_config_params *bbdev_params,
+ struct lcore_setup *l_setup)
+{
+ uint16_t port;
+ unsigned int lcore;
+ int lcount, total_count = 0;
+ struct lcore_queue_conf *qconf;
+
+ /*
+ * for each core in the uplink populate the array of ports (rx or tx).
+ * core with odd id receives rx ports, core with even id receives all
+ * tx ports.
+ */
+ for (lcore = 0, lcount = 0; lcore < sizeof(int) * 8; lcore++) {
+
+ /* get the lcore_id */
+ if ((bbdev_params->downlink_lcores & (1 << lcore)) == 0)
+ continue;
+
+ lcount++;
+ for (port = 0; port <= nb_ports; port++) {
+ if ((lcount & 1) == 1) {
+ if (port < nb_ports) {
+ /*get the port id*/
+ if ((bbdev_params->downlink_rx_ports &
+ (1 << port)) == 0)
+ continue;
+
+ l_setup->rxte_lcore_list[
+ l_setup->rxte_lcores] =
+ lcore;
+ printf("Lcore %d: DL RX port %d\n",
+ lcore, port);
+ } else {
+ l_setup->rxte_lcores++;
+ continue;
+ }
+
+ } else {
+ if (port < nb_ports) {
+ if ((bbdev_params->downlink_tx_ports &
+ (1 << port)) == 0)
+ continue;
+
+ l_setup->tdtx_lcore_list[
+ l_setup->tdtx_lcores] =
+ lcore;
+ printf("Lcore %d: DL TX port %d\n",
+ lcore, port);
+ } else {
+ l_setup->tdtx_lcores++;
+ continue;
+ }
+ }
+ qconf = &lcore_queue_conf[lcore];
+ qconf->port_list[qconf->nb_ports].queues++;
+ qconf->port_list[qconf->nb_ports].port_id = port;
+ rte_eth_macaddr_get(port,
+ &qconf->ports_eth_addr
+ [qconf->nb_ports]);
+ qconf->nb_ports++;
+ }
+ }
+
+ if (lcount % 2) {
+ printf("\nNumber of DL lcores is not an even number");
+ return -1;
+ }
+
+ total_count += lcount;
+
+ /*
+ * for each core in the downlink populate the array of ports (rx or tx).
+ * core with odd id receives rx ports, core with even id receives all
+ * tx ports
+ */
+ for (lcore = 0, lcount = 0; lcore < (sizeof(int) * 8); lcore++) {
+ if ((bbdev_params->uplink_lcores & (1 << lcore)) == 0)
+ continue;
+
+ lcount++;
+ for (port = 0; port <= nb_ports; port++) {
+
+ if ((lcount & 1) == 1) {
+ if (port < nb_ports) {
+ if ((bbdev_params->uplink_rx_ports &
+ (1 << port)) == 0)
+ continue;
+
+ l_setup->rxtd_lcore_list[
+ l_setup->rxtd_lcores] =
+ lcore;
+ printf("Lcore %u: UL RX port %u\n",
+ lcore, port);
+ } else {
+ l_setup->rxtd_lcores++;
+ continue;
+ }
+
+ } else {
+ if (port < nb_ports) {
+ if ((bbdev_params->uplink_tx_ports &
+ (1 << port)) == 0)
+ continue;
+
+ l_setup->tetx_lcore_list[
+ l_setup->tetx_lcores] =
+ lcore;
+ printf("Lcore %d: UL TX port %d\n",
+ lcore, port);
+ } else {
+ l_setup->tetx_lcores++;
+ continue;
+ }
+ }
+ qconf = &lcore_queue_conf[lcore];
+ qconf->port_list[qconf->nb_ports].queues++;
+ qconf->port_list[qconf->nb_ports].port_id = port;
+ rte_eth_macaddr_get(port,
+ &qconf->ports_eth_addr
+ [qconf->nb_ports]);
+ qconf->nb_ports++;
+ }
+ }
+ if (lcount % 2) {
+ printf("\nNumber of DL lcores is not an even number");
+ return -1;
+ }
+ total_count += lcount;
+ /*return number of lcores processed*/
+ return total_count;
+}
+
+static void
+bbdev_dequeue_dec(void)
+{
+ struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+ struct rte_bbdev_dec_op *ops_burst[MAX_PKT_BURST];
+ unsigned int i, j, portid, nb_tx, lcoreid, qid = 0, q, nb_tx_sent = 0;
+ struct lcore_queue_conf *qconf;
+ struct lcore_statistics *lcore_stat;
+
+ lcoreid = rte_lcore_id();
+ lcore_stat = &lcore_stats[lcoreid];
+ qconf = &lcore_queue_conf[lcoreid];
+
+ if (qconf->nb_ports == 0)
+ return;
+
+ while (!global_exit_flag) {
+ for (i = 0; i < qconf->nb_ports; i++) {
+ portid = qconf->port_list[i].port_id;
+
+ for (q = 0; q < qconf->port_list[i].queues; q++) {
+ qid = qconf->port_list[i].queues_list[q];
+ /* Dequeue packets from bbdev device*/
+ nb_tx = rte_bbdev_dequeue_dec_ops(
+ qconf->bbdev_id,
+ qconf->bbdev_qs[q], ops_burst,
+ MAX_PKT_BURST);
+
+ if (!nb_tx)
+ continue;
+
+ lcore_stat->dequeued += nb_tx;
+
+ for (j = 0; j < nb_tx; j++) {
+ pkts_burst[j] =
+ ops_burst[j]->turbo_dec
+ .hard_output.data;
+ /* input mbufs are no longer needed,
+ * release!
+ */
+ rte_pktmbuf_free(ops_burst[j]->
+ turbo_dec.input.data);
+ }
+
+ rte_bbdev_dec_op_free_bulk(ops_burst, nb_tx);
+
+ update_pkt_mac(nb_tx, pkts_burst,
+ &(qconf->ports_eth_addr
+ [portid]),
+ portid);
+
+ /*Enqueue packets to ethdev*/
+ nb_tx_sent = rte_eth_tx_burst(
+ (uint8_t)portid, qid,
+ pkts_burst, nb_tx);
+ if (unlikely(nb_tx_sent < nb_tx)) {
+ lcore_stat->lost_packets +=
+ nb_tx - nb_tx_sent;
+ for (j = nb_tx_sent; j < nb_tx; j++)
+ rte_pktmbuf_free(pkts_burst[j]);
+ }
+ }
+ }
+ }
+}
+
+static void
+bbdev_dequeue_enc(void)
+{
+ struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+ struct rte_bbdev_enc_op *ops_burst[MAX_PKT_BURST];
+ unsigned int i, j, portid, nb_tx, lcoreid, qid = 0, q, nb_tx_sent = 0;
+ struct lcore_queue_conf *qconf;
+ struct lcore_statistics *lcore_stat;
+
+ lcoreid = rte_lcore_id();
+ lcore_stat = &lcore_stats[lcoreid];
+ qconf = &lcore_queue_conf[lcoreid];
+
+ if (qconf->nb_ports == 0)
+ return;
+
+ while (!global_exit_flag) {
+ for (i = 0; i < qconf->nb_ports; i++) {
+ portid = qconf->port_list[i].port_id;
+
+ for (q = 0; q < qconf->port_list[i].queues; q++) {
+ qid = qconf->port_list[i].queues_list[q];
+ /* Dequeue packets from bbdev device*/
+ nb_tx = rte_bbdev_dequeue_enc_ops(
+ qconf->bbdev_id,
+ qconf->bbdev_qs[q], ops_burst,
+ MAX_PKT_BURST);
+
+ if (!nb_tx)
+ continue;
+
+ lcore_stat->dequeued += nb_tx;
+
+ for (j = 0; j < nb_tx; j++) {
+ pkts_burst[j] =
+ ops_burst[j]->
+ turbo_enc.output.data;
+ /* input mbufs are no longer needed,
+ * release!
+ */
+ rte_pktmbuf_free(ops_burst[j]->
+ turbo_enc.input.data);
+ }
+
+ rte_bbdev_enc_op_free_bulk(ops_burst, nb_tx);
+
+ update_pkt_mac(nb_tx, pkts_burst,
+ &(qconf->ports_eth_addr
+ [portid]), portid);
+
+ /*Enqueue packets to ethdev*/
+ nb_tx_sent = rte_eth_tx_burst(
+ (uint8_t)portid, qid,
+ pkts_burst, nb_tx);
+ if (unlikely(nb_tx_sent < nb_tx)) {
+ lcore_stat->lost_packets +=
+ nb_tx - nb_tx_sent;
+ for (j = nb_tx_sent; j < nb_tx; j++)
+ rte_pktmbuf_free(pkts_burst[j]);
+ }
+ }
+ }
+ }
+}
+
+static void
+bbdev_enqueue_dec(void)
+{
+ unsigned int i, portid, nb_rx = 0, j, lcoreid, q, qid = 0;
+ int ret;
+ struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+ struct rte_mbuf *bbdev_pkts[MAX_PKT_BURST];
+ struct rte_bbdev_dec_op *ops_burst[MAX_PKT_BURST];
+ struct lcore_statistics *lcore_stat;
+ struct lcore_queue_conf *qconf = &lcore_queue_conf[rte_lcore_id()];
+
+ if (qconf->nb_ports == 0)
+ return;
+
+ lcoreid = rte_lcore_id();
+ lcore_stat = &lcore_stats[lcoreid];
+
+ /* Read packet from RX queues*/
+ while (!global_exit_flag) {
+
+ for (i = 0; i < qconf->nb_ports; i++) {
+ portid = qconf->port_list[i].port_id;
+
+ for (q = 0; q < qconf->port_list[i].queues; q++) {
+
+ qid = qconf->port_list[i].queues_list[q];
+ /* Dequeue packets from ethdev */
+ nb_rx = rte_eth_rx_burst((uint8_t) portid, qid,
+ pkts_burst, MAX_PKT_BURST);
+
+ if (!nb_rx)
+ continue;
+
+ if (rte_bbdev_dec_op_alloc_bulk(
+ bbdev_op_pool
+ [RTE_BBDEV_OP_TURBO_DEC],
+ ops_burst, nb_rx) != 0)
+ continue;
+
+ ret = rte_pktmbuf_alloc_bulk(qconf->mbuf_pool,
+ bbdev_pkts, MAX_PKT_BURST);
+ if (ret < 0)
+ continue;
+
+ for (j = 0; j < nb_rx; j++) {
+ struct rte_bbdev_op_turbo_dec *td;
+ /* append the size of the etherneit
+ * header
+ */
+ rte_pktmbuf_append(bbdev_pkts[j],
+ sizeof(struct ether_hdr));
+ /* copy the ethernet header */
+ void *in_buf = rte_pktmbuf_mtod(
+ pkts_burst[j], void *);
+ void *out_buf = rte_pktmbuf_mtod(
+ bbdev_pkts[j], void *);
+ rte_memcpy(out_buf, in_buf,
+ sizeof(struct ether_hdr));
+ /* set op */
+ ops_burst[j]->turbo_dec = def_op_dec;
+ td = &ops_burst[j]->turbo_dec;
+ td->cb_params.k =
+ rte_pktmbuf_pkt_len(
+ bbdev_pkts[j])
+ * 8;
+ td->input.offset =
+ sizeof(
+ struct
+ ether_hdr);
+ td->input.length =
+ rte_pktmbuf_pkt_len(
+ bbdev_pkts[j]);
+ td->input.data =
+ pkts_burst[j];
+ td->hard_output.offset =
+ sizeof(
+ struct
+ ether_hdr);
+ td->hard_output.data =
+ bbdev_pkts[j];
+ }
+
+ /* Enqueue packets on BBDEV device */
+ nb_rx = rte_bbdev_enqueue_dec_ops(
+ qconf->bbdev_id,
+ qconf->bbdev_qs[q], ops_burst,
+ nb_rx);
+ lcore_stat->enqueued += nb_rx;
+ }
+ }
+ }
+}
+
+static void
+bbdev_enqueue_enc(void)
+{
+ unsigned int i, portid, nb_rx = 0, j, lcoreid, q, qid = 0;
+ int ret;
+ struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+ struct rte_mbuf *bbdev_pkts[MAX_PKT_BURST];
+ struct rte_bbdev_enc_op *ops_burst[MAX_PKT_BURST];
+ struct lcore_statistics *lcore_stat;
+ struct lcore_queue_conf *qconf = &lcore_queue_conf[rte_lcore_id()];
+
+ lcoreid = rte_lcore_id();
+ lcore_stat = &lcore_stats[lcoreid];
+
+ /* Read packet from RX queues*/
+ while (!global_exit_flag) {
+
+ for (i = 0; i < qconf->nb_ports; i++) {
+ portid = qconf->port_list[i].port_id;
+
+ for (q = 0; q < qconf->port_list[i].queues; q++) {
+
+ qid = qconf->port_list[i].queues_list[q];
+ /* Dequeue packets from ethdev */
+ nb_rx = rte_eth_rx_burst((uint8_t)portid, qid,
+ pkts_burst, MAX_PKT_BURST);
+
+ if (!nb_rx)
+ continue;
+
+ if (rte_bbdev_enc_op_alloc_bulk(
+ bbdev_op_pool
+ [RTE_BBDEV_OP_TURBO_ENC],
+ ops_burst, nb_rx) != 0)
+ continue;
+
+ ret = rte_pktmbuf_alloc_bulk(qconf->mbuf_pool,
+ bbdev_pkts, MAX_PKT_BURST);
+ if (ret < 0)
+ continue;
+
+ for (j = 0; j < nb_rx; j++) {
+ /* append the size of the ethernet
+ * header
+ */
+ rte_pktmbuf_append(bbdev_pkts[j],
+ sizeof(
+ struct
+ ether_hdr));
+ /* copy the ethernet header */
+ void *in_buf = rte_pktmbuf_mtod(
+ pkts_burst[j], void *);
+ void *out_buf = rte_pktmbuf_mtod(
+ bbdev_pkts[j], void *);
+ rte_memcpy(out_buf, in_buf,
+ sizeof(
+ struct
+ ether_hdr));
+ /* set op */
+ ops_burst[j]->turbo_enc = def_op_enc;
+ ops_burst[j]->turbo_enc.cb_params.k =
+ rte_pktmbuf_pkt_len(
+ bbdev_pkts[j])
+ * 8;
+ ops_burst[j]->turbo_enc.input.offset =
+ sizeof(
+ struct
+ ether_hdr);
+ ops_burst[j]->turbo_enc.input.length =
+ rte_pktmbuf_pkt_len(
+ bbdev_pkts[j]);
+ ops_burst[j]->turbo_enc.input.data =
+ pkts_burst[j];
+ ops_burst[j]->turbo_enc.output.offset =
+ sizeof(
+ struct
+ ether_hdr);
+ ops_burst[j]->turbo_enc.output.data =
+ bbdev_pkts[j];
+ }
+
+ /* Enqueue packets on BBDEV device */
+ nb_rx = rte_bbdev_enqueue_enc_ops(
+ qconf->bbdev_id,
+ qconf->bbdev_qs[q], ops_burst,
+ nb_rx);
+ lcore_stat->enqueued += nb_rx;
+ }
+ }
+ }
+}
+
+static void
+print_lcore_stats(unsigned int lcore_id)
+{
+ struct lcore_statistics *lcore_stat;
+ static const char *stats_border = "_______";
+
+ lcore_stat = &lcore_stats[lcore_id];
+ printf("\nLcore %d: %s enqueued count:\t\t%u\n",
+ lcore_id, stats_border, lcore_stat->enqueued);
+ printf("Lcore %d: %s dequeued count:\t\t%u\n",
+ lcore_id, stats_border, lcore_stat->dequeued);
+}
+
+static void
+print_stats(struct lcore_setup *lcore_setup)
+{
+ unsigned int l_id, dev_id;
+ unsigned char x, nb_ports;
+ int len, ret, i;
+
+ struct rte_eth_xstat *xstats;
+ struct rte_eth_xstat_name *xstats_names;
+ struct rte_bbdev_stats bbstats;
+ static const char *stats_border = "_______";
+
+ const char clr[] = { 27, '[', '2', 'J', '\0' };
+ const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+ /* Clear screen and move to top left */
+ printf("%s%s", clr, topLeft);
+
+ nb_ports = rte_eth_dev_count();
+ printf("PORT STATISTICS:\n================\n");
+ for (x = 0; x < nb_ports; x++) {
+
+ len = rte_eth_xstats_get(x, NULL, 0);
+ if (len < 0)
+ rte_exit(EXIT_FAILURE,
+ "rte_eth_xstats_get(%hhu) failed: %d",
+ x, len);
+
+ xstats = calloc(len, sizeof(*xstats));
+ if (xstats == NULL)
+ rte_exit(EXIT_FAILURE,
+ "Failed to calloc memory for xstats");
+
+ ret = rte_eth_xstats_get(x, xstats, len);
+ if (ret < 0 || ret > len)
+ rte_exit(EXIT_FAILURE,
+ "rte_eth_xstats_get(%hhu) len%i failed: %d",
+ x, len, ret);
+
+ xstats_names = calloc(len, sizeof(*xstats_names));
+ if (xstats_names == NULL)
+ rte_exit(EXIT_FAILURE,
+ "Failed to calloc memory for xstats_names");
+
+ ret = rte_eth_xstats_get_names(x, xstats_names, len);
+ if (ret < 0 || ret > len)
+ rte_exit(EXIT_FAILURE,
+ "rte_eth_xstats_get_names(%hhu) len%i failed: %d",
+ x, len, ret);
+
+ for (i = 0; i < len; i++) {
+ if (xstats[i].value > 0)
+ printf("Port %u: %s %s:\t\t%"PRIu64"\n",
+ x, stats_border,
+ xstats_names[i].name,
+ xstats[i].value);
+ }
+ }
+
+ printf("\nBBDEV STATISTICS:\n=================\n");
+ RTE_BBDEV_FOREACH(dev_id) {
+ rte_bbdev_stats_get(dev_id, &bbstats);
+ printf("BBDEV %u: %s enqueue count:\t\t%"PRIu64"\n",
+ dev_id, stats_border,
+ bbstats.enqueued_count);
+ printf("BBDEV %u: %s dequeue count:\t\t%"PRIu64"\n",
+ dev_id, stats_border,
+ bbstats.dequeued_count);
+ printf("BBDEV %u: %s enqueue error count:\t\t%"PRIu64"\n",
+ dev_id, stats_border,
+ bbstats.enqueue_err_count);
+ printf("BBDEV %u: %s dequeue error count:\t\t%"PRIu64"\n\n",
+ dev_id, stats_border,
+ bbstats.dequeue_err_count);
+ }
+
+ printf("LCORE STATISTICS:\n=================\n");
+ for (l_id = 0; l_id < lcore_setup->tdtx_lcores; l_id++)
+ print_lcore_stats(lcore_setup->tdtx_lcore_list[l_id]);
+ for (l_id = 0; l_id < lcore_setup->tetx_lcores; l_id++)
+ print_lcore_stats(lcore_setup->tetx_lcore_list[l_id]);
+ for (l_id = 0; l_id < lcore_setup->rxtd_lcores; l_id++)
+ print_lcore_stats(lcore_setup->rxtd_lcore_list[l_id]);
+ for (l_id = 0; l_id < lcore_setup->rxte_lcores; l_id++)
+ print_lcore_stats(lcore_setup->rxte_lcore_list[l_id]);
+}
+
+static void
+qs_to_lcores(unsigned int nb_lcores, unsigned int lcores[])
+{
+ unsigned int port = 0, lcore, nb_ports = 0, q_id;
+ unsigned int l = 0, y, x;
+ struct lcore_queue_conf *qconf;
+ struct dev_qs temp[RTE_MAX_LCORE];
+ int count = 0;
+ struct dev_qs *p;
+ struct port_queues *port_qs;
+
+ /*
+ * for each core copy array of ports (holding all ports available to
+ * this lcore) from qconf into temp array and empty the one in qconf
+ */
+ for (x = 0; x < nb_lcores; x++) {
+
+ lcore = lcores[x];
+ qconf = &lcore_queue_conf[lcore];
+ nb_ports = qconf->nb_ports;
+ qconf->nb_ports = 0;
+ p = qconf->port_list;
+
+ for (y = 0; y < nb_ports; y++, p++) {
+ temp[y] = *p;
+ p->port_id = 0;
+ }
+ }
+ /*
+ * re-populate array of ports and queues in qconf according to number
+ * of available lcores, ports, and queues so only the required ports
+ * and queues stay in the configuration of each particular lcore
+ */
+ while (port < nb_ports || l < nb_lcores) {
+
+ if (port < nb_ports && l == nb_lcores) {
+ l = 0;
+ count++;
+ }
+ if (port == nb_ports && l < nb_lcores)
+ port = 0;
+
+ lcore = lcores[l];
+ qconf = &lcore_queue_conf[lcore];
+ port_qs = &ports_qs[temp[port].port_id];
+
+ if ((lcore & 1) == 1) {
+ qconf->port_list[count].port_id =
+ temp[port].port_id;
+
+ for (q_id = 0; q_id < qconf->port_list[count].queues;
+ q_id++) {
+ qconf->port_list[count].queues_list[q_id] =
+ port_qs->nb_tx_qs;
+ port_qs->nb_tx_qs++;
+ }
+
+ qconf->nb_ports++;
+ } else {
+ qconf->port_list[count].port_id =
+ temp[port].port_id;
+ for (q_id = 0; q_id < qconf->port_list[count].queues;
+ q_id++) {
+ qconf->port_list[count].queues_list[q_id] =
+ port_qs->nb_rx_qs;
+ port_qs->nb_rx_qs++;
+ }
+ qconf->nb_ports++;
+ }
+ printf("Lcore: %u connected to port: %u\n",
+ lcores[l], temp[port].port_id);
+ port++;
+ l++;
+ }
+}
+
+static void
+main_loop(struct lcore_setup *lcore_setup)
+{
+ unsigned int i = 0, this_lcore;
+ this_lcore = rte_lcore_id();
+
+ /* print stats on master core */
+ if (rte_get_master_lcore() == this_lcore) {
+ while (!global_exit_flag) {
+ print_stats(lcore_setup);
+ rte_delay_ms(500);
+ }
+ }
+
+ for (i = 0; i < lcore_setup->rxte_lcores; i++) {
+ if (lcore_setup->rxte_lcore_list[i] == this_lcore)
+ bbdev_enqueue_enc();
+ }
+
+ for (i = 0; i < lcore_setup->tetx_lcores; i++) {
+ if (lcore_setup->tetx_lcore_list[i] == this_lcore)
+ bbdev_dequeue_enc();
+ }
+
+ for (i = 0; i < lcore_setup->rxtd_lcores; i++) {
+ if (lcore_setup->rxtd_lcore_list[i] == this_lcore)
+ bbdev_enqueue_dec();
+ }
+
+ for (i = 0; i < lcore_setup->tdtx_lcores; i++) {
+ if (lcore_setup->tdtx_lcore_list[i] == this_lcore)
+ bbdev_dequeue_dec();
+ }
+}
+
+static int
+launch_one_lcore(void *arg)
+{
+ main_loop((struct lcore_setup *)arg);
+ return 0;
+}
+
+static int
+prepare_bbdev_device(unsigned int dev_id, uint8_t qs_nb)
+{
+ int ret;
+ unsigned int q_id;
+ struct rte_bbdev_info info;
+ struct rte_bbdev_queue_conf qconf = {0};
+
+ rte_bbdev_info_get(dev_id, &info);
+
+ ret = rte_bbdev_setup_queues(dev_id, qs_nb, info.socket_id);
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE,
+ "ERROR(%d): BBDEV %u not configured properly\n",
+ ret, dev_id);
+
+ /* setup device queues */
+ qconf.socket = info.socket_id;
+ qconf.queue_size = info.drv.queue_size_lim;
+ qconf.op_type = (dev_id & 0x1) ? RTE_BBDEV_OP_TURBO_DEC :
+ RTE_BBDEV_OP_TURBO_ENC;
+
+ for (q_id = 0; q_id < qs_nb; q_id++) {
+
+ ret = rte_bbdev_queue_configure(dev_id, q_id, &qconf);
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE,
+ "ERROR(%d): BBDEV %u queue %u not configured properly\n",
+ ret, dev_id, q_id);
+ }
+
+ ret = rte_bbdev_start(dev_id);
+
+ if (ret != 0)
+ rte_exit(EXIT_FAILURE, "ERROR(%d): BBDEV %u not started\n",
+ ret, dev_id);
+
+ printf("BBdev %u started\n", dev_id);
+
+ return 0;
+}
+
+static void
+enable_bbdev(unsigned int lcores[], unsigned int nb_lcores, unsigned int dev_id)
+{
+ unsigned int i, nb_qs, tot_nb_bbdev_qs = 1, lcore, lcoreid;
+ struct lcore_queue_conf *qconf, *qconf_next;
+ unsigned int nb_bbdev_qs = 0;
+ char pool_name[RTE_MEMPOOL_NAMESIZE];
+
+ /* set num of port queues and bbdev queues for each lcore on the link */
+ for (lcore = 0; lcore < nb_lcores; lcore++) {
+ nb_qs = 0;
+ lcoreid = lcores[lcore];
+ qconf = &lcore_queue_conf[lcoreid];
+
+ for (i = 0; i < qconf->nb_ports; i++)
+ nb_qs += qconf->port_list->queues;
+
+ for (i = 0; i < nb_qs; i++) {
+ qconf->bbdev_qs[i] = nb_bbdev_qs;
+ nb_bbdev_qs++;
+ }
+
+ /* create the mbuf mempool for ethdev pkts */
+ snprintf(pool_name, sizeof(pool_name), "bbdev%d_mbuf_pool%d",
+ dev_id, lcore);
+ qconf->mbuf_pool = rte_pktmbuf_pool_create(pool_name,
+ NB_MBUF, MEMPOOL_CACHE_SIZE, 0,
+ RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id());
+ if (qconf->mbuf_pool == NULL)
+ rte_exit(EXIT_FAILURE,
+ "Unable to create '%s' pool\n",
+ pool_name);
+
+ qconf->nb_bbdev_qs = nb_qs;
+ qconf->bbdev_id = dev_id;
+ printf("BBdev: %u, lcore %u\n", dev_id, lcoreid);
+
+ qconf_next = &lcore_queue_conf[lcoreid + 1];
+ qconf_next->bbdev_id = dev_id;
+
+ for (i = 0; i < nb_qs; i++)
+ qconf_next->bbdev_qs[i] = qconf->bbdev_qs[i];
+
+ qconf_next->nb_bbdev_qs = nb_qs;
+ printf("BBdev: %u, lcore %u\n", dev_id, lcoreid + 1);
+ }
+
+ /* count the number of queues needed on the bbdev device on this link */
+ for (lcore = 0; lcore < nb_lcores; lcore++) {
+ lcoreid = lcores[lcore];
+ qconf = &lcore_queue_conf[lcoreid];
+ tot_nb_bbdev_qs += qconf->nb_bbdev_qs;
+ }
+
+ prepare_bbdev_device(dev_id, tot_nb_bbdev_qs);
+}
+
+int
+main(int argc, char **argv)
+{
+ int ret, nb_bbdevs, nb_ports;
+ int enabled_portcount = 0;
+ uint8_t lcore_id;
+ char rx = 'R', tx = 'T', up_link = 'U', down_link = 'D';
+ void *sigret;
+ uint32_t bbdev_enabled_port_mask = 0;
+ struct lcore_setup lcore_setup = { 0 };
+ struct bbdev_config_params bbdev_params = { 0 };
+
+ sigret = signal(SIGTERM, signal_handler);
+ if (sigret == SIG_ERR)
+ rte_exit(EXIT_FAILURE, "signal(%d, ...) failed", SIGTERM);
+
+ sigret = signal(SIGINT, signal_handler);
+ if (sigret == SIG_ERR)
+ rte_exit(EXIT_FAILURE, "signal(%d, ...) failed", SIGINT);
+
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+
+ argc -= ret;
+ argv += ret;
+
+ /* parse application arguments (after the EAL ones) */
+ ret = bbdev_parse_args(argc, argv, &bbdev_params);
+
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE, "Invalid BBDEV arguments\n");
+
+ /* Get number of available bbdev devices */
+ nb_bbdevs = rte_bbdev_count();
+ if (nb_bbdevs == 0)
+ rte_exit(EXIT_FAILURE, "No bbdevs detected!\n");
+ printf("Number of bbdevs detected: %d\n", nb_bbdevs);
+
+ /* Get number of available ethdev devices */
+ nb_ports = rte_eth_dev_count();
+ /* Initialize the port/queue configuration of each logical core */
+ ret = lcore_queue_init(nb_ports, &bbdev_params, &lcore_setup);
+ if (ret < nb_bbdevs * 2)
+ rte_exit(EXIT_FAILURE, "Failed to initialize queues.\n");
+
+ /* reconfigure ports to lcores assignment to allow for multiple lcores
+ * usage
+ */
+ qs_to_lcores(lcore_setup.rxte_lcores, lcore_setup.rxte_lcore_list);
+ qs_to_lcores(lcore_setup.rxtd_lcores, lcore_setup.rxtd_lcore_list);
+ qs_to_lcores(lcore_setup.tdtx_lcores, lcore_setup.tdtx_lcore_list);
+ qs_to_lcores(lcore_setup.tetx_lcores, lcore_setup.tetx_lcore_list);
+
+ /* create the mbuf mempool for ethdev pkts */
+ ethdev_mbuf_mempool = rte_pktmbuf_pool_create("ethdev_mbuf_pool",
+ NB_MBUF, MEMPOOL_CACHE_SIZE, 0,
+ RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id());
+ if (ethdev_mbuf_mempool == NULL)
+ rte_exit(EXIT_FAILURE, "Cannot create ethdev mbuf mempool\n");
+
+ /*initialize ports*/
+ enabled_portcount = initialize_ports(bbdev_params.downlink_rx_ports,
+ down_link, rx, &bbdev_enabled_port_mask);
+ enabled_portcount += initialize_ports(bbdev_params.downlink_tx_ports,
+ down_link, tx, &bbdev_enabled_port_mask);
+ enabled_portcount += initialize_ports(bbdev_params.uplink_rx_ports,
+ up_link, rx, &bbdev_enabled_port_mask);
+ enabled_portcount += initialize_ports(bbdev_params.uplink_tx_ports,
+ up_link, tx, &bbdev_enabled_port_mask);
+
+ if (enabled_portcount < 1)
+ rte_exit(EXIT_FAILURE, "Failed to initialize Ethernet ports\n");
+
+ /*start Ethernet devices*/
+ ret = start_ethdev_ports(nb_ports, bbdev_enabled_port_mask);
+ if (ret <= 0)
+ rte_exit(EXIT_FAILURE,
+ "Failed to start Ethernet devices on ports\n");
+ check_all_ports_link_status(nb_ports, bbdev_enabled_port_mask);
+
+ /*create bbdev op pools*/
+ bbdev_op_pool[RTE_BBDEV_OP_TURBO_DEC] =
+ rte_bbdev_op_pool_create("bbdev_op_pool_dec",
+ RTE_BBDEV_OP_TURBO_DEC, NB_MBUF, 128, rte_socket_id());
+ bbdev_op_pool[RTE_BBDEV_OP_TURBO_ENC] =
+ rte_bbdev_op_pool_create("bbdev_op_pool_enc",
+ RTE_BBDEV_OP_TURBO_ENC, NB_MBUF, 128, rte_socket_id());
+
+ if ((bbdev_op_pool[RTE_BBDEV_OP_TURBO_DEC] == NULL) ||
+ (bbdev_op_pool[RTE_BBDEV_OP_TURBO_ENC] == NULL))
+ rte_exit(EXIT_FAILURE, "Cannot create bbdev op pools\n");
+
+ /*start DL bbdev device*/
+ if (lcore_setup.rxte_lcores)
+ enable_bbdev(lcore_setup.rxte_lcore_list,
+ lcore_setup.rxte_lcores, 0);
+ /*start UL bbdev device*/
+ if (lcore_setup.rxtd_lcores)
+ enable_bbdev(lcore_setup.rxtd_lcore_list,
+ lcore_setup.rxtd_lcores, 1);
+
+ /*launch per-lcore init on every lcore*/
+ rte_eal_mp_remote_launch(launch_one_lcore, (void *)&lcore_setup,
+ CALL_MASTER);
+
+ RTE_LCORE_FOREACH_SLAVE(lcore_id) {
+ if (rte_eal_wait_lcore(lcore_id) < 0)
+ return -1;
+ }
+
+ return 0;
+}
--
2.7.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [dpdk-dev] [PATCH v2 5/5] bbdev: documentation
2017-10-18 2:14 [dpdk-dev] [PATCH v2 1/5] bbdev: librte_bbdev library Amr Mokhtar
` (2 preceding siblings ...)
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 4/5] bbdev: sample app Amr Mokhtar
@ 2017-10-18 2:14 ` Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 0/5] Wireless Base Band Device (bbdev) Amr Mokhtar
4 siblings, 0 replies; 6+ messages in thread
From: Amr Mokhtar @ 2017-10-18 2:14 UTC (permalink / raw)
To: dev
Cc: thomas, anatoly.burakov, pablo.de.lara.guarch, niall.power,
chris.macnamara, Amr Mokhtar
- Wireless Baseband Device Library Programmer’s Guide
- test-bbdev User Guide
- BBDEV Sample Application User Guides
- Baseband Device Drivers Guides
- Doxygen API
Signed-off-by: Amr Mokhtar <amr.mokhtar@intel.com>
---
doc/api/doxy-api-index.md | 1 +
doc/api/doxy-api.conf | 1 +
doc/guides/bbdevs/index.rst | 40 +++
doc/guides/bbdevs/null.rst | 77 ++++
doc/guides/bbdevs/turbo_sw.rst | 169 +++++++++
doc/guides/index.rst | 1 +
doc/guides/prog_guide/bbdev.rst | 621 +++++++++++++++++++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/sample_app_ug/bbdev_app.rst | 160 +++++++++
doc/guides/sample_app_ug/index.rst | 1 +
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testbbdev.rst | 546 +++++++++++++++++++++++++++++
12 files changed, 1619 insertions(+)
create mode 100644 doc/guides/bbdevs/index.rst
create mode 100644 doc/guides/bbdevs/null.rst
create mode 100644 doc/guides/bbdevs/turbo_sw.rst
create mode 100644 doc/guides/prog_guide/bbdev.rst
create mode 100644 doc/guides/sample_app_ug/bbdev_app.rst
create mode 100644 doc/guides/tools/testbbdev.rst
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index 990815f..0e2227b 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -48,6 +48,7 @@ The public API headers are grouped by topics:
[bitrate] (@ref rte_bitrate.h),
[latency] (@ref rte_latencystats.h),
[devargs] (@ref rte_devargs.h),
+ [bbdev] (@ref rte_bbdev.h),
[PCI] (@ref rte_pci.h)
- **device specific**:
diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf
index 9e9fa56..d4a0890 100644
--- a/doc/api/doxy-api.conf
+++ b/doc/api/doxy-api.conf
@@ -39,6 +39,7 @@ INPUT = doc/api/doxy-api-index.md \
lib/librte_eal/common/include \
lib/librte_eal/common/include/generic \
lib/librte_acl \
+ lib/librte_bbdev \
lib/librte_bitratestats \
lib/librte_cfgfile \
lib/librte_cmdline \
diff --git a/doc/guides/bbdevs/index.rst b/doc/guides/bbdevs/index.rst
new file mode 100644
index 0000000..c9aa1b0
--- /dev/null
+++ b/doc/guides/bbdevs/index.rst
@@ -0,0 +1,40 @@
+..
+ BSD LICENSE
+
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Baseband Device Drivers
+=======================
+
+.. toctree::
+ :maxdepth: 2
+ :numbered:
+
+ null
+ turbo_sw
diff --git a/doc/guides/bbdevs/null.rst b/doc/guides/bbdevs/null.rst
new file mode 100644
index 0000000..0f40232
--- /dev/null
+++ b/doc/guides/bbdevs/null.rst
@@ -0,0 +1,77 @@
+..
+ BSD LICENSE
+
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+BBDEV null Poll Mode Driver
+============================
+
+The (**bbdev_null**) is a bbdev poll mode driver which provides a minimal
+implementation of a software bbdev device. As a null device it does not modify
+the data in the mbuf on which the bbdev operation is to operate and it only
+works for operation type ``RTE_BBDEV_OP_NONE``.
+
+When a burst of mbufs is submitted to a *bbdev null PMD* for processing then
+each mbuf in the burst will be enqueued in an internal buffer ring to be
+collected on a dequeue call.
+
+
+Limitations
+-----------
+
+* In-place operations for Turbo encode and decode are not supported
+
+Installation
+------------
+
+The *bbdev null PMD* is enabled and built by default in both the Linux and
+FreeBSD builds.
+
+Initialization
+--------------
+
+To use the PMD in an application, user must:
+
+- Call ``rte_vdev_init("bbdev_null")`` within the application.
+
+- Use ``--vdev="bbdev_null"`` in the EAL options, which will call ``rte_vdev_init()`` internally.
+
+The following parameters (all optional) can be provided in the previous two calls:
+
+* ``socket_id``: Specify the socket where the memory for the device is going to be allocated
+ (by default, *socket_id* will be the socket where the core that is creating the PMD is running on).
+
+* ``max_nb_queues``: Specify the maximum number of queues in the device (default is ``RTE_MAX_LCORE``).
+
+Example:
+~~~~~~~~
+
+.. code-block:: console
+
+ ./test-bbdev.py -e="--vdev=bbdev_null,socket_id=0,max_nb_queues=8"
diff --git a/doc/guides/bbdevs/turbo_sw.rst b/doc/guides/bbdevs/turbo_sw.rst
new file mode 100644
index 0000000..46833b5
--- /dev/null
+++ b/doc/guides/bbdevs/turbo_sw.rst
@@ -0,0 +1,169 @@
+..
+ BSD LICENSE
+
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+SW Turbo Poll Mode Driver
+=========================
+
+The SW Turbo PMD (**turbo_sw**) provides a poll mode bbdev driver that utilizes
+Intel optimized libraries for LTE Layer 1 workloads acceleration. This PMD
+supports the functions: Turbo FEC, Rate Matching and CRC functions.
+
+Features
+--------
+
+SW Turbo PMD has support for the following capabilities:
+
+For the encode operation:
+
+* ``RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE``
+* ``RTE_BBDEV_TURBO_RAW_INPUT_DATA``
+
+For the decode operation:
+
+* ``RTE_BBDEV_TURBO_CRC_24B_ATTACH``
+* ``RTE_BBDEV_TURBO_RATE_MATCH``
+* ``RTE_BBDEV_TURBO_RV_INDEX_BYPASS``
+
+
+Limitations
+-----------
+
+* In-place operations for Turbo encode and decode are not supported
+
+Installation
+------------
+
+To build DPDK with the *turbo_sw* the user is required to download
+the export controlled ``FlexRAN SDK`` Libraries, by requesting them from
+`<https://networkbuilders.intel.com/network-technologies/dpdk>`_.
+Once approval has been granted, the user needs to log in
+`<https://networkbuilders.intel.com/dpdklogin>`_
+and click on "FlexRAN SDK Libraries" link, to download the libraries.
+After downloading the libraries, the user needs to unpack and compile on their
+system before building DPDK.
+
+FlexRAN SDK Installation
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following are pre-requisites for building FlexRAN SDK Libraries:
+ (a) An AVX2 supporting machine
+ (b) Windriver TS 2 or CentOS 7 operating systems
+ (c) Intel ICC compiler installed
+
+The following instructions should be followed in this exact order:
+
+#. Set the environment variables:
+
+ .. code-block:: console
+
+ export ICC_LOCATION=<path-to-icc-compiler-install-folder>
+ export CPATH=$ICC_LOCATION/linux/mkl/include/:$ICC_LOCATION/linux/ipp/include/
+
+
+#. Extract the SDK-R1.3.0 package:
+
+ .. code-block:: console
+
+ cd <path-to-workspace>/
+ ./SDK-R1.3.0.sh
+
+
+#. To allow FlexRAN SDK R1.3.0 to work with bbdev properly, the following
+ hotfix is required. Change the return of function ``bit_selection_complete_avx2()``
+ located in file
+ ``<path-to-workspace>/SDK-R1.3.0/sdk/source/phy/lib_rate_matching/phy_rate_match_avx2.cpp``
+ to return 0 instead of 1.
+
+ .. code-block:: c
+
+ - return 1;
+ + return 0;
+
+#. Install SDK-R1.3.0 package:
+
+ .. code-block:: console
+
+ cd SDK-R1.3.0/sdk/
+ ./create-makefiles-linux.sh
+ cd build-avx2-icc/
+ make install
+
+
+Initialization
+--------------
+
+In order to enable this virtual bbdev PMD, the user must:
+
+* Build the ``FLEXRAN SDK`` libraries (explained in Installation section).
+
+* Export ICC environment variables:
+
+.. code-block:: console
+
+ export LIBRARY_PATH=$ICC_LOCATION/linux/compiler/lib/intel64_lin/:$ICC_LOCATION/linux/ipp/lib/intel64_lin/
+ export LD_LIBRARY_PATH=$ICC_LOCATION/linux/compiler/lib/intel64_lin/:$ICC_LOCATION/linux/ipp/lib/intel64_lin/
+
+
+* Export the environmental variables ``FLEXRAN_SDK`` to the path where the
+ FlexRAN library was installed. And ``DIR_WIRELESS_SDK`` to the path where the
+ library was extracted.
+
+Example:
+
+.. code-block:: console
+
+ export FLEXRAN_SDK=<path-to-workspace>/SDK-R1.3.0/sdk/build-avx2-icc/install
+ export DIR_WIRELESS_SDK=<path-to-workspace>/SDK-R1.3.0/sdk/
+
+
+* Set ``CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW=y`` in DPDK common configuration
+ file ``config/common_base``.
+
+To use the PMD in an application, user must:
+
+- Call ``rte_vdev_init("turbo_sw")`` within the application.
+
+- Use ``--vdev="turbo_sw"`` in the EAL options, which will call ``rte_vdev_init()`` internally.
+
+The following parameters (all optional) can be provided in the previous two calls:
+
+* ``socket_id``: Specify the socket where the memory for the device is going to be allocated
+ (by default, *socket_id* will be the socket where the core that is creating the PMD is running on).
+
+* ``max_nb_queues``: Specify the maximum number of queues in the device (default is ``RTE_MAX_LCORE``).
+
+Example:
+~~~~~~~~
+
+.. code-block:: console
+
+ ./test-bbdev.py -e="--vdev=turbo_sw,socket_id=0,max_nb_queues=8" \
+ -c validation -v ./test_vectors/bbdev_vector_t?_default.data
diff --git a/doc/guides/index.rst b/doc/guides/index.rst
index 5b6eb7e..e3df7ae 100644
--- a/doc/guides/index.rst
+++ b/doc/guides/index.rst
@@ -44,6 +44,7 @@ DPDK documentation
nics/index
cryptodevs/index
eventdevs/index
+ bbdevs/index
contributing/index
rel_notes/index
faq/index
diff --git a/doc/guides/prog_guide/bbdev.rst b/doc/guides/prog_guide/bbdev.rst
new file mode 100644
index 0000000..7e7cde6
--- /dev/null
+++ b/doc/guides/prog_guide/bbdev.rst
@@ -0,0 +1,621 @@
+..
+ BSD LICENSE
+
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Wireless Baseband Device Library
+================================
+
+The Wireless Baseband library provides a common programming framework that
+abstracts HW accelerators based on FPGA and/or Fixed Function Accelerators that
+assist with 3gpp Physical Layer processing. Furthermore, it decouples the
+application from the compute-intensive wireless functions by abstracting their
+optimized libraries to appear as virtual bbdev devices.
+
+The functional scope of the BBDEV library are those functions in relation to
+the 3gpp Layer 1 signal processing (channel coding, modulation, ...).
+
+The framework currently only supports Turbo Code FEC function.
+
+
+Design Principles
+-----------------
+
+The Wireless Baseband library follows the same ideology of DPDK's Ethernet
+Device and Crypto Device frameworks. Wireless Baseband provides a generic
+acceleration abstraction framework which supports both physical (hardware) and
+virtual (software) wireless acceleration functions.
+
+Device Management
+-----------------
+
+Device Creation
+~~~~~~~~~~~~~~~
+
+Physical bbdev devices are discovered during the PCI probe/enumeration of the
+EAL function which is executed at DPDK initialization, based on
+their PCI device identifier, each unique PCI BDF (bus/bridge, device,
+function).
+
+Virtual devices can be created by two mechanisms, either using the EAL command
+line options or from within the application using an EAL API directly.
+
+From the command line using the --vdev EAL option
+
+.. code-block:: console
+
+ --vdev 'turbo_sw,max_nb_queues=8,socket_id=0'
+
+Our using the rte_vdev_init API within the application code.
+
+.. code-block:: c
+
+ rte_vdev_init("turbo_sw", "max_nb_queues=2,socket_id=0")
+
+All virtual bbdev devices support the following initialization parameters:
+
+- ``max_nb_queues`` - maximum number of queues supported by the device.
+
+- ``socket_id`` - socket on which to allocate the device resources on.
+
+
+Device Identification
+~~~~~~~~~~~~~~~~~~~~~
+
+Each device, whether virtual or physical is uniquely designated by two
+identifiers:
+
+- A unique device index used to designate the bbdev device in all functions
+ exported by the bbdev API.
+
+- A device name used to designate the bbdev device in console messages, for
+ administration or debugging purposes. For ease of use, the port name includes
+ the port index.
+
+
+Device Configuration
+~~~~~~~~~~~~~~~~~~~~
+
+From the application point of view, each instance of a bbdev device consists of
+one or more queues identified by queue IDs. While different devices may have
+different capabilities (e.g. support different operation types), all queues on
+a device support identical configuration possibilities. A queue is configured
+for only one type of operation and is configured at initializations time.
+When an operation is enqueued to a specific queue ID, the result is dequeued
+from the same queue ID.
+
+Configuration of a device has two different levels: configuration that applies
+to the whole device, and configuration that applies to a single queue.
+
+Device configuration is applied with
+``rte_bbdev_setup_queues(dev_id,num_queues,socket_id)``
+and queue configuration is applied with
+``rte_bbdev_queue_configure(dev_id,queue_id,conf)``. Note that, although all
+queues on a device support same capabilities, they can be configured differently
+and will then behave differently.
+Devices supporting interrupts can enable them by using
+``rte_bbdev_intr_enable(dev_id)``.
+
+The configuration of each bbdev device includes the following operations:
+
+- Allocation of resources, including hardware resources if a physical device.
+- Resetting the device into a well-known default state.
+- Initialization of statistics counters.
+
+The ``rte_bbdev_setup_queues`` API is used to setup queues for a bbdev device.
+
+.. code-block:: c
+
+ int rte_bbdev_setup_queues(uint16_t dev_id, uint16_t num_queues,
+ int socket_id);
+
+- ``num_queues`` argument identifies the total number of queues to setup for
+ this device.
+
+- ``socket_id`` specifies which socket will be used to allocate the memory.
+
+
+The ``rte_bbdev_intr_enable`` API is used to enable interrupts for a bbdev
+device, if supported by the driver. Should be called before starting the device.
+
+.. code-block:: c
+
+ int rte_bbdev_intr_enable(uint16_t dev_id);
+
+
+Queues Configuration
+~~~~~~~~~~~~~~~~~~~~
+
+Each bbdev devices queue is individually configured through the
+``rte_bbdev_queue_configure()`` API.
+Each queue resources may be allocated on a specified socket.
+
+.. code-block:: c
+
+ struct rte_bbdev_queue_conf {
+ int socket; /**< NUMA socket used for memory allocation */
+ uint32_t queue_size; /**< Size of queue */
+ uint8_t priority; /**< Queue priority */
+ bool deferred_start; /**< Do not start queue when device is started. */
+ enum rte_bbdev_op_type op_type; /**< Operation type */
+ };
+
+Device & Queues Management
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+After initialization, devices are in a stopped state, so must be started by the
+application. If an application is finished using a device it can close the
+device. Once closed, it cannot be restarted.
+
+.. code-block:: c
+
+ int rte_bbdev_start(uint16_t dev_id)
+ int rte_bbdev_stop(uint16_t dev_id)
+ int rte_bbdev_close(uint16_t dev_id)
+ int rte_bbdev_queue_start(uint16_t dev_id, uint16_t queue_id)
+ int rte_bbdev_queue_stop(uint16_t dev_id, uint16_t queue_id)
+
+
+By default, all queues are started when the device is started, but they can be
+stopped individually.
+
+.. code-block:: c
+
+ int rte_bbdev_queue_start(uint16_t dev_id, uint16_t queue_id)
+ int rte_bbdev_queue_stop(uint16_t dev_id, uint16_t queue_id)
+
+
+Logical Cores, Memory and Queues Relationships
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The bbdev device Library as the Poll Mode Driver library support NUMA for when
+a processor’s logical cores and interfaces utilize its local memory. Therefore
+baseband operations, the mbuf being operated on should be allocated from memory
+pools created in the local memory. The buffers should, if possible, remain on
+the local processor to obtain the best performance results and buffer
+descriptors should be populated with mbufs allocated from a mempool allocated
+from local memory.
+
+The run-to-completion model also performs better, especially in the case of
+virtual bbdev devices, if the baseband operation and data buffers are in local
+memory instead of a remote processor's memory. This is also true for the
+pipe-line model provided all logical cores used are located on the same processor.
+
+Multiple logical cores should never share the same queue for enqueuing
+operations or dequeuing operations on the same bbdev device since this would
+require global locks and hinder performance. It is however possible to use a
+different logical core to dequeue an operation on a queue pair from the logical
+core which it was enqueued on. This means that a baseband burst enqueue/dequeue
+APIs are a logical place to transition from one logical core to another in a
+packet processing pipeline.
+
+
+Device Operation Capabilities
+-----------------------------
+
+Capabilities (in terms of operations supported, max number of queues, etc.)
+identify what a bbdev is capable of performing that differs from one device to
+another. For the full scope of the bbdev capability see the definition of the
+structure in the *DPDK API Reference*.
+
+.. code-block:: c
+
+ struct rte_bbdev_op_cap;
+
+A device reports its capabilities when registering itself in the bbdev framework.
+With the aid of this capabilities mechanism, an application can query devices to
+discover which operations within the 3gpp physical layer they are capable of
+performing. Below is an example of the capabilities for a PMD it supports in
+relation to Turbo Encoding and Decoding operations.
+
+.. code-block:: c
+
+ static const struct rte_bbdev_op_cap bbdev_capabilities[] = {
+ {
+ .type = RTE_BBDEV_OP_TURBO_DEC,
+ .cap.turbo_dec = {
+ .capability_flags =
+ RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE |
+ RTE_BBDEV_TURBO_RAW_INPUT_DATA,
+ .num_buffers_src = 1,
+ .num_buffers_hard_out = 1,
+ .num_buffers_soft_out = 0,
+ }
+ },
+ {
+ .type = RTE_BBDEV_OP_TURBO_ENC,
+ .cap.turbo_enc = {
+ .capability_flags =
+ RTE_BBDEV_TURBO_CRC_24B_ATTACH |
+ RTE_BBDEV_TURBO_RATE_MATCH |
+ RTE_BBDEV_TURBO_RV_INDEX_BYPASS,
+ .num_buffers_src = 1,
+ .num_buffers_dst = 1,
+ }
+ },
+ RTE_BBDEV_END_OF_CAPABILITIES_LIST()
+ };
+
+Capabilities Discovery
+~~~~~~~~~~~~~~~~~~~~~~
+
+Discovering the features and capabilities of a bbdev device poll mode driver
+is achieved through the ``rte_bbdev_info_get()`` function.
+
+.. code-block:: c
+
+ int rte_bbdev_info_get(uint16_t dev_id, struct rte_bbdev_info *dev_info)
+
+This allows the user to query a specific bbdev PMD and get all the device
+capabilities. The ``rte_bbdev_info`` structure provides two levels of
+information:
+
+- Device relevant information, like: name and related rte_bus.
+
+- Driver specific information, as defined by the ``struct rte_bbdev_driver_info``
+ structure, this is where capabilities reside along with other specifics like:
+ maximum queue sizes and priority level.
+
+.. code-block:: c
+
+ struct rte_bbdev_info {
+ int socket_id; /**< NUMA socket that device is on */
+ const char *dev_name; /**< Unique device name */
+ const struct rte_bus *bus; /**< Bus information */
+ uint16_t num_queues; /**< Number of queues currently configured */
+ bool started; /**< Set if device is currently started */
+ struct rte_bbdev_driver_info drv; /**< Info from device driver */
+ };
+
+Operation Processing
+--------------------
+
+Scheduling of baseband operations on DPDK's application data path is
+performed using a burst oriented asynchronous API set. A queue on a bbdev
+device accepts a burst of baseband operations using enqueue burst API. On physical
+bbdev devices the enqueue burst API will place the operations to be processed
+on the device's hardware input queue, for virtual devices the processing of the
+baseband operations is usually completed during the enqueue call to the bbdev
+device. The dequeue burst API will retrieve any processed operations available
+from the queue on the bbdev device, from physical devices this is usually
+directly from the device's processed queue, and for virtual device's from a
+``rte_ring`` where processed operations are place after being processed on the
+enqueue call.
+
+
+Enqueue / Dequeue Burst APIs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The burst enqueue API uses a bbdev device identifier and a queue
+identifier to specify the bbdev device queue to schedule the processing on.
+The ``num_ops`` parameter is the number of operations to process which are
+supplied in the ``ops`` array of ``rte_bbdev_*_op`` structures.
+The enqueue function returns the number of operations it actually enqueued for
+processing, a return value equal to ``num_ops`` means that all packets have been
+enqueued.
+
+.. code-block:: c
+
+ uint16_t rte_bbdev_enqueue_enc_ops(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_enc_op **ops, uint16_t num_ops)
+
+ uint16_t rte_bbdev_enqueue_dec_ops(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_dec_op **ops, uint16_t num_ops)
+
+The dequeue API uses the same format as the enqueue API of processed but
+the ``num_ops`` and ``ops`` parameters are now used to specify the max processed
+operations the user wishes to retrieve and the location in which to store them.
+The API call returns the actual number of processed operations returned, this
+can never be larger than ``num_ops``.
+
+.. code-block:: c
+
+ uint16_t rte_bbdev_dequeue_enc_ops(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_enc_op **ops, uint16_t num_ops)
+
+ uint16_t rte_bbdev_dequeue_dec_ops(uint16_t dev_id, uint16_t queue_id,
+ struct rte_bbdev_dec_op **ops, uint16_t num_ops)
+
+Operation Representation
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+An encode bbdev operation is represented by ``rte_bbdev_enc_op`` structure,
+and by ``rte_bbdev_dec_op`` for decode. These structures act as metadata
+containers for all necessary information required for the bbdev operation to be
+processed on a particular bbdev device poll mode driver.
+
+.. code-block:: c
+
+ struct rte_bbdev_enc_op {
+ int status;
+ struct rte_mempool *mempool;
+ void *opaque_data;
+ /** Contains encoder specific parameters */
+ struct rte_bbdev_op_turbo_enc turbo_enc;
+ };
+
+ struct rte_bbdev_dec_op {
+ int status;
+ struct rte_mempool *mempool;
+ void *opaque_data;
+ /** Contains decoder specific parameters */
+ struct rte_bbdev_op_turbo_dec turbo_dec;
+ };
+
+The operation structure by itself defines the operation type. It includes an
+operation status, a reference to the operation specific data, which can vary in
+size and content depending on the operation being provisioned. It also contains
+the source mempool for the operation, if it is allocated from a mempool.
+
+If bbdev operations are allocated from a bbdev operation mempool, see next
+section, there is also the ability to allocate private memory with the
+operation for applications purposes.
+
+Application software is responsible for specifying all the operation specific
+fields in the ``rte_bbdev_*_op`` structure which are then used by the bbdev PMD
+to process the requested operation.
+
+
+Operation Management and Allocation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The bbdev library provides an API set for managing bbdev operations which
+utilize the Mempool Library to allocate operation buffers. Therefore, it ensures
+that the bbdev operation is interleaved optimally across the channels and
+ranks for optimal processing.
+
+.. code-block:: c
+
+ struct rte_mempool *
+ rte_bbdev_op_pool_create(const char *name, enum rte_bbdev_op_type type,
+ unsigned int num_elements, unsigned int cache_size,
+ int socket_id)
+
+``rte_bbdev_*_op_alloc_bulk()`` and ``rte_bbdev_*_op_free_bulk()`` are used to
+allocate bbdev operations of a specific type from a given bbdev operation mempool.
+
+.. code-block:: c
+
+ int rte_bbdev_enc_op_alloc_bulk(struct rte_mempool *mempool,
+ struct rte_bbdev_enc_op **ops, uint16_t num_ops)
+
+ int rte_bbdev_dec_op_alloc_bulk(struct rte_mempool *mempool,
+ struct rte_bbdev_dec_op **ops, uint16_t num_ops)
+
+``rte_bbdev_*_op_free_bulk()`` is called by the application to return an
+operation to its allocating pool.
+
+.. code-block:: c
+
+ void rte_bbdev_dec_op_free_bulk(struct rte_bbdev_dec_op **ops,
+ unsigned int num_ops)
+ void rte_bbdev_enc_op_free_bulk(struct rte_bbdev_enc_op **ops,
+ unsigned int num_ops)
+
+BBDEV Operations
+~~~~~~~~~~~~~~~~
+
+The bbdev operation structure contains all the mutable data relating to
+performing Turbo code processing on a referenced mbuf data buffer. It is used
+for either encode or decode operations.
+
+Turbo Encode operation accepts one input and one output.
+
+Turbo Decode operation accepts one input and two outputs, called *hard-decision*
+and *soft-decision* outputs. *Soft-decision* output is optional.
+
+It is expected that the application provides input and output ``mbuf`` pointers
+allocated and ready to use. The baseband framework supports turbo coding on
+Code Blocks (CB) and Transport Blocks (TB).
+
+For the output buffer(s), the application needs only to provide an allocated and
+free mbuf (containing only one mbuf segment), so that bbdev can write the
+operation outcome.
+
+**Turbo Encode Op structure**
+
+.. code-block:: c
+
+ struct rte_bbdev_op_turbo_enc {
+ struct rte_bbdev_op_data input; /**< input src data */
+ struct rte_bbdev_op_data output; /**< output buffer */
+
+ uint32_t op_flags;
+ int32_t n_soft;
+ int32_t k_mimo;
+ int32_t mdl_harq;
+
+ int32_t g;
+ int32_t nl;
+ int32_t qm;
+ uint8_t rv_index;
+
+ uint8_t code_block_mode; /**< 0 - transpot block, 1 - code block */
+ union {
+ struct rte_bbdev_op_enc_cb_params cb_params;
+ struct rte_bbdev_op_enc_tb_params tb_params;
+ };
+ };
+
+
+**Turbo Decode Op structure**
+
+.. code-block:: c
+
+ struct rte_bbdev_op_turbo_dec {
+ struct rte_bbdev_op_data input; /**< input src data */
+ struct rte_bbdev_op_data hard_output; /**< hard output buffer */
+ struct rte_bbdev_op_data soft_output; /**< soft output buffer */
+
+ uint32_t op_flags;
+ uint8_t rv_index;
+ uint8_t iter_min:4;
+ uint8_t iter_max:4;
+ uint8_t iter_count;
+ uint8_t ext_scale;
+ uint8_t num_maps;
+
+ uint8_t code_block_mode; /**< 0 - transpot block, 1 - code block */
+ union {
+ struct rte_bbdev_op_dec_cb_params cb_params;
+ struct rte_bbdev_op_dec_tb_params tb_params;
+ };
+ };
+
+Input and output data buffers are identified by ``rte_bbdev_op_data`` structure.
+This strucutre has three elements:
+
+- ``data`` - This is the mbuf reference
+
+- ``offset`` - The starting point for the Turbo input/output, in bytes, from the
+ start of the data in the data buffer. It must be smaller than data_len of the
+ mbuf's first segment
+
+- ``length`` - The length, in bytes, of the buffer on which the Turbo operation
+ will or has been computed. For the input, the length is set by the application.
+ For the output(s), the length is computed by the bbdev PMD driver.
+
+Sample code
+-----------
+
+The baseband device sample application gives an introduction on how to use the
+bbdev framework, by giving a sample code performing a loop-back operation with a
+baseband processor capable of transceiving data packets.
+
+The following sample pseudo-code shows the basic steps to encode several buffers
+using (**sw_trubo**) bbdev PMD.
+
+.. code-block:: c
+
+ /* EAL Init */
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+
+ /* Get number of available bbdev devices */
+ nb_bbdevs = rte_bbdev_count();
+ if (nb_bbdevs == 0)
+ rte_exit(EXIT_FAILURE, "No bbdevs detected!\n");
+
+ /* Create bbdev op pools */
+ bbdev_op_pool[RTE_BBDEV_OP_TURBO_ENC] =
+ rte_bbdev_op_pool_create("bbdev_op_pool_enc",
+ RTE_BBDEV_OP_TURBO_ENC, NB_MBUF, 128, rte_socket_id());
+
+ /* Get information for this device */
+ rte_bbdev_info_get(dev_id, &info);
+
+ /* Setup BBDEV device queues */
+ ret = rte_bbdev_setup_queues(dev_id, qs_nb, info.socket_id);
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE,
+ "ERROR(%d): BBDEV %u not configured properly\n",
+ ret, dev_id);
+
+ /* setup device queues */
+ qconf.socket = info.socket_id;
+ qconf.queue_size = info.drv.queue_size_lim;
+ qconf.op_type = RTE_BBDEV_OP_TURBO_ENC;
+
+ for (q_id = 0; q_id < qs_nb; q_id++) {
+ /* Configure all queues belonging to this bbdev device */
+ ret = rte_bbdev_queue_configure(dev_id, q_id, &qconf);
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE,
+ "ERROR(%d): BBDEV %u queue %u not configured properly\n",
+ ret, dev_id, q_id);
+ }
+
+ /* Start bbdev device */
+ ret = rte_bbdev_start(dev_id);
+
+ /* Create the mbuf mempool for pkts */
+ mbuf_pool = rte_pktmbuf_pool_create("bbdev_mbuf_pool",
+ NB_MBUF, MEMPOOL_CACHE_SIZE, 0,
+ RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id());
+ if (mbuf_pool == NULL)
+ rte_exit(EXIT_FAILURE,
+ "Unable to create '%s' pool\n", pool_name);
+
+ while (!global_exit_flag) {
+
+ /* Allocate burst of op structures in preparation for enqueue */
+ if (rte_bbdev_enc_op_alloc_bulk(bbdev_op_pool[RTE_BBDEV_OP_TURBO_ENC],
+ op_type, ops_burst, op_num) != 0)
+ continue;
+
+ /* Allocate input mbuf pkts */
+ ret = rte_pktmbuf_alloc_bulk(mbuf_pool, input_pkts_burst, MAX_PKT_BURST);
+ if (ret < 0)
+ continue;
+
+ /* Allocate output mbuf pkts */
+ ret = rte_pktmbuf_alloc_bulk(mbuf_pool, output_pkts_burst, MAX_PKT_BURST);
+ if (ret < 0)
+ continue;
+
+ for (j = 0; j < op_num; j++) {
+ /* Append the size of the ethernet header */
+ rte_pktmbuf_append(input_pkts_burst[j],
+ sizeof(struct ether_hdr));
+
+ /* set op */
+
+ ops_burst[j]->turbo_enc->input.offset =
+ sizeof(struct ether_hdr);
+
+ ops_burst[j]->turbo_enc->input.length =
+ rte_pktmbuf_pkt_len(bbdev_pkts[j]);
+
+ ops_burst[j]->turbo_enc->input.data =
+ input_pkts_burst[j];
+
+ ops_burst[j]->turbo_enc->output.offset =
+ sizeof(struct ether_hdr);
+
+ ops_burst[j]->turbo_enc->output.data =
+ output_pkts_burst[j];
+ }
+
+ /* Enqueue packets on BBDEV device */
+ op_num = rte_bbdev_enqueue_ops(qconf->bbdev_id,
+ qconf->bbdev_qs[q], ops_burst,
+ MAX_PKT_BURST);
+
+ /* Dequeue packets from BBDEV device*/
+ op_num = rte_bbdev_dequeue_ops(qconf->bbdev_id,
+ qconf->bbdev_qs[q], ops_burst,
+ MAX_PKT_BURST);
+ }
+
+
+BBDEV Device API
+~~~~~~~~~~~~~~~~
+
+The bbdev Library API is described in the *DPDK API Reference* document.
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index b5ad6b8..421c0f6 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -63,6 +63,7 @@ Programmer's Guide
kernel_nic_interface
thread_safety_dpdk_functions
eventdev
+ bbdev
qos_framework
power_man
packet_classif_access_ctrl
diff --git a/doc/guides/sample_app_ug/bbdev_app.rst b/doc/guides/sample_app_ug/bbdev_app.rst
new file mode 100644
index 0000000..dee37ad
--- /dev/null
+++ b/doc/guides/sample_app_ug/bbdev_app.rst
@@ -0,0 +1,160 @@
+.. BSD LICENSE
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+.. bbdev_app:
+
+Loop-back Sample Application using Baseband Device (bbdev)
+==========================================================
+
+The baseband sample application is a simple example of packet processing using
+the Data Plane Development Kit (DPDK) for baseband workloads using the bbdev
+library.
+
+Overview
+--------
+
+The Baseband device sample application performs a loop-back operation using a
+baseband device capable of transceiving data packets.
+A packet is received on a DOWNLINK_RX_PORT or UPLINK_RX_PORTS -> enqueued for
+baseband operation -> dequeued from the baseband device then looped back to
+DOWNLINK_TX_PORTS or UPLINK_TX_PORTS, respectively.
+
+* The source MAC address is replaced by the TX_PORT MAC address
+
+* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID
+
+Limitations
+-----------
+
+* Although, the baseband sample application is designed to work in full-duplex
+ mode, but due to theplain format of the received packets from pkt-gen, the
+ Turbo decode (uplink) is not operable. Only the Turbo encode direction
+ (downlink) is currently supported in the bbdev sample application.
+
+Compiling the Application
+-------------------------
+
+#. DPDK needs to be built with ``turbo_sw`` PMD driver enabled along with
+ ``FLEXRAN SDK`` Libraries. Refer to *SW Turbo Poll Mode Driver*
+ documentation for more details on this.
+
+#. Go to the example directory:
+
+ .. code-block:: console
+
+ export RTE_SDK=/path/to/rte_sdk
+ cd ${RTE_SDK}/examples/bbdev_app
+
+#. Set the target (a default target is used if not specified). For example:
+
+ .. code-block:: console
+
+ export RTE_TARGET=x86_64-native-linuxapp-gcc
+
+ See the *DPDK Getting Started Guide* for possible RTE_TARGET values.
+
+#. Build the application:
+
+ .. code-block:: console
+
+ make
+
+Running the Application
+-----------------------
+
+The application requires a number of command line options:
+
+.. code-block:: console
+
+ $ ./build/bbdev [EAL options] -- [-r DOWNLINK_RX_PORTS] [-R UPLINK_RX_PORTS] /
+ [-t DOWNLINK_TX_PORTS ] [-T UPLINK_TX_PORTS] /
+ [-c DOWNLINK_CORES] [-C UPLINK_CORES]
+
+where:
+
+* ``r DOWNLINK_RX_PORTS``: decimal number of downlink receiver ports
+* ``R UPLINK_RX_PORTS``: decimal number of uplink receiver ports
+* ``t DOWNLINK_TX_PORTS``: decimal number of downlink transceiver ports
+* ``T UPLINK_TX_PORTS``: decimal number of uplink transceiver ports
+* ``c DOWNLINK_CORES``: hexmask for downlink cores number
+* ``C UPLINK_CORES``: hexmask for uplink cores number
+
+The application requires that baseband devices is capable of performing
+the specified baseband operation are available on application initialization.
+This means that HW baseband device/s must be bound to a DPDK driver or
+a SW baseband device/s (virtual BBdev) must be created (using --vdev).
+
+To run the application in linuxapp environment with one baseband device for
+uplink, running on 1 port and 2 logical cores, issue the command:
+
+.. code-block:: console
+
+ $ ./build/bbdev --vdev='turbo_sw' -w <NIC0PCIADDR> -c 0x38 --socket-mem=2,2 \
+ --file-prefix=bbdev -- -r 1 -t 1 -c 0x30
+
+where, NIC0PCIADDR is the PCI addresse of the Rx port
+
+This command creates one virtual bbdev devices ``turbo_sw`` where the device
+gets linked to a corresponding ethernet port as whitelisted by the parameter -w.
+3 cores are allocated to the application, and assigned as:
+
+ - core 3 is the master and used to print the stats live on screen,
+
+ - cores 4 & 5 are the downlink cores, Tx & Rx
+
+
+Refer to the *DPDK Getting Started Guide* for general information on running
+applications and the Environment Abstraction Layer (EAL) options.
+
+Using Packet Generator with baseband device sample application
+--------------------------------------------------------------
+
+To allow the bbdev sample app to do the loopback, an influx of traffic is required.
+This can be done by using DPDK Pktgen to burst traffic on two ethernet ports, and
+it will print the transmitted along with the looped-back traffic on Rx ports.
+Executing the command below will generate traffic on the two whitelisted ethernet
+ports.
+
+.. code-block:: console
+
+ $ ./pktgen-3.4.0/app/x86_64-native-linuxapp-gcc/pktgen -c 0x3 \
+ --socket-mem=1,1 --file-prefix=pg -w <NIC1PCIADDR> -- -m 1.0 -P
+
+where:
+
+* ``-c COREMASK``: A hexadecimal bitmask of cores to run on
+* ``--socket-mem``: Memory to allocate on specific sockets (use comma separated values)
+* ``--file-prefix``: Prefix for hugepage filenames
+* ``-w <NIC1PCIADDR>``: Add a PCI device in white list. The argument format is <[domain:]bus:devid.func>.
+* ``-m <string>``: Matrix for mapping ports to logical cores.
+* ``-P``: PROMISCUOUS mode
+
+
+Refer to *The Pktgen Application* documents for general information on running
+Pktgen with DPDK applications.
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index 069d4f1..cc568a9 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -77,6 +77,7 @@ Sample Applications User Guides
ptpclient
performance_thread
ipsec_secgw
+ bbdev_app
**Figures**
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index c9133ec..a390fe7 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -41,3 +41,4 @@ DPDK Tools User Guides
devbind
cryptoperf
testeventdev
+ testbbdev
diff --git a/doc/guides/tools/testbbdev.rst b/doc/guides/tools/testbbdev.rst
new file mode 100644
index 0000000..d257b22
--- /dev/null
+++ b/doc/guides/tools/testbbdev.rst
@@ -0,0 +1,546 @@
+..
+ BSD LICENSE
+
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+dpdk-test-bbdev Application
+===========================
+
+The ``dpdk-test-bbdev`` tool is a Data Plane Development Kit (DPDK) utility that
+allows measuring performance parameters of PMDs available in the bbdev framework.
+Available tests available for execution are: latency, throughput, validation and
+sanity tests. Execution of tests can be customized using various parameters
+passed to a python running script.
+
+Compiling the Application
+-------------------------
+
+**Step 1: PMD setting**
+
+The ``dpdk-test-bbdev`` tool depends on crypto device drivers PMD which
+are disabled by default in the build configuration file ``common_base``.
+The bbdevice drivers PMD which should be tested can be enabled by setting
+
+ ``CONFIG_RTE_LIBRTE_PMD_<name>=y``
+
+Setting example for (*turbo_sw*) PMD
+
+ ``CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW=y``
+
+**Step 2: Build the application**
+
+Execute the ``dpdk-setup.sh`` script to build the DPDK library together with the
+``dpdk-test-bbdev`` application.
+
+Initially, the user must select a DPDK target to choose the correct target type
+and compiler options to use when building the libraries.
+The user must have all libraries, modules, updates and compilers installed
+in the system prior to this, as described in the earlier chapters in this
+Getting Started Guide.
+
+Running the Application
+-----------------------
+
+The tool application has a number of command line options:
+
+.. code-block:: console
+
+ python test-bbdev.py [-h] [-p TESTAPP_PATH] [-e EAL_PARAMS] [-t TIMEOUT]
+ [-c TEST_CASE [TEST_CASE ...]]
+ [-v TEST_VECTOR [TEST_VECTOR...]] [-n NUM_OPS]
+ [-b BURST_SIZE [BURST_SIZE ...]]
+
+command-line Options
+~~~~~~~~~~~~~~~~~~~~
+
+The following are the command-line options:
+
+``-h, --help``
+ Shows help message and exit.
+
+``-p TESTAPP_PATH, --testapp_path TESTAPP_PATH``
+ Indicates the path to the bbdev test app. If not specified path is set based
+ on *$RTE_SDK* environment variable concatenated with "*/build/app/testbbdev*".
+
+``-e EAL_PARAMS, --eal_params EAL_PARAMS``
+ Specifies EAL arguments which are passed to the test app. For more details,
+ refer to DPDK documentation at http://dpdk.org/doc.
+
+``-t TIMEOUT, --timeout TIMEOUT``
+ Specifies timeout in seconds. If not specified timeout is set to 300 seconds.
+
+``-c TEST_CASE [TEST_CASE ...], --test_cases TEST_CASE [TEST_CASE ...]``
+ Defines test cases to run. If not specified all available tests are run.
+
+ The following tests can be run:
+ * unittest
+ Small unit tests witch check basic functionality of bbdev library.
+ * latency
+ Test calculates three latency metrics:
+ * offload_latency_tc
+ measures the cost of offloading enqueue and dequeue operations.
+ * offload_latency_empty_q_tc
+ measures the cost of offloading a dequeue operation from an empty queue.
+ checks how long last dequeueing if there is no operations to dequeue
+ * operation_latency_tc
+ measures the time difference from the first attempt to enqueue till the
+ first successful dequeue.
+ * validation
+ Test do enqueue on given vector and compare output after dequeueing.
+ * throughput
+ Test measures the achieved throughput on the available lcores.
+ Results are printed in million operations per second and million bits per second.
+ * interrupt
+ The same test as 'throughput' but uses interrupts instead of PMD to perform
+ the dequeue.
+
+ **Example usage:**
+
+ ``./test-bbdev.py -c validation``
+ Runs validation test suite
+
+ ``./test-bbdev.py -c latency throughput``
+ Runs latency and throughput test suites
+
+``-v TEST_VECTOR [TEST_VECTOR ...], --test_vector TEST_VECTOR [TEST_VECTOR ...]``
+ Specifies paths to the test vector files. If not specified path is set based
+ on *$RTE_SDK* environment variable concatenated with
+ "*/app/test-bbdev/test_vectors/bbdev_vector_null.data*" and indicates default
+ data file.
+
+ **Example usage:**
+
+ ``./test-bbdev.py -v app/test-bbdev/test_vectors/bbdev_vector_td_test1.data``
+ Fills vector based on bbdev_vector_td_test1.data file and runs all tests
+
+ ``./test-bbdev.py -v bbdev_vector_td_test1.data bbdev_vector_te_test2.data``
+ The bbdev test app is executed twice. First time vector is filled based on
+ *bbdev_vector_td_test1.data* file and second time based on
+ *bbdev_vector_te_test2.data* file. For both executions all tests are run.
+
+``-n NUM_OPS, --num_ops NUM_OPS``
+ Specifies number of operations to process on device. If not specified num_ops
+ is set to 32 operations.
+
+``-v BURST_SIZE [BURST_SIZE ...], --burst-size BURST_SIZE [BURST_SIZE ...]``
+ Specifies operations enqueue/dequeue burst size. If not specified burst_size is
+ set to 32.
+
+
+Parameter globbing
+~~~~~~~~~~~~~~~~~~
+
+Thanks to the globbing functionality in python test-bbdev.py script allows to
+run tests with different set of vector files without giving all of them explicitly.
+
+**Example usage:**
+
+.. code-block:: console
+
+ ./test-bbdev.py -v app/test-bbdev/test_vectors/bbdev_vector_*.data
+
+It runs all tests with following vectors:
+
+- ``bbdev_vector_null.data``
+
+- ``bbdev_vector_te_default.data``
+
+- ``bbdev_vector_td_default.data``
+
+
+.. code-block:: console
+
+ ./test-bbdev.py -v app/test-bbdev/test_vectors/bbdev_vector_t?_default.data
+
+It runs all tests with "default" vectors:
+
+- ``bbdev_vector_te_default.data``
+
+- ``bbdev_vector_td_default.data``
+
+
+Running Tests
+-------------
+
+Shortened tree of isg_cid-wireless_dpdk_ae with dpdk compiled for
+x86_64-native-linuxapp-icc target:
+
+::
+
+ |-- app
+ |-- test-bbdev
+ |-- test_vectors
+ |-- bbdev_vector_null.data
+ |-- bbdev_vector_te_default.data
+ |-- bbdev_vector_td_default.data
+
+ |-- x86_64-native-linuxapp-icc
+ |-- app
+ |-- testbbdev
+
+All bbdev devices
+~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+ ./test-bbdev.py -p ../../x86_64-native-linuxapp-icc/app/testbbdev
+ -v ./test_vectors/bbdev_vector_td_default.data
+
+It runs all available tests using the test vector filled based on
+*bbdev_vector_td_default.data* file.
+By default number of operations to process on device is set to 32, timeout is
+set to 300s and operations enqueue/dequeue burst size is set to 32.
+Moreover a bbdev (*bbdev_null*) device will be created.
+
+bbdev turbo_sw device
+~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+ ./test-bbdev.py -p ../../x86_64-native-linuxapp-icc/app/testbbdev
+ -e="--vdev=turbo_sw" -t 120 -c validation
+ -v ./test_vectors/bbdev_vector_t?_default.data -n 64 -b 8 32
+
+It runs **validation** test for each vector file that matches the given pattern.
+Number of operations to process on device is set to 64 and operations timeout is
+set to 120s and enqueue/dequeue burst size is set to 8 and to 32.
+Moreover a bbdev (*turbo_sw*) device will be created.
+
+
+bbdev null device
+~~~~~~~~~~~~~~~~~
+
+Executing bbdev null device with *bbdev_vector_null.data* helps in measuring the
+overhead introduced by the bbdev framework.
+
+.. code-block:: console
+
+ ./test-bbdev.py -e="--vdev=bbdev_null0"
+ -v ./test_vectors/bbdev_vector_null.data
+
+**Note:**
+
+bbdev_null device does not have to be defined explicitly as it is created by default.
+
+
+
+Test Vector files
+=================
+
+Test Vector files contain the data which is used to set turbo decoder/encoder
+parameters and buffers for validation purpose. New test vector files should be
+stored in ``app/test-bbdev/test_vectors/`` directory. Detailed description of
+the systax of the test vector files is in the following section.
+
+
+Basic principles for test vector files
+--------------------------------------
+Line started with ``#`` is treated as a comment and is ignored.
+
+If variable is a chain of values, values should be separated by a comma. If
+assignment is split into several lines, each line (except the last one) has to
+be ended with a comma.
+There is no comma after last value in last line. Correct assignment should
+look like the following:
+
+.. parsed-literal::
+
+ variable =
+ value, value, value, value,
+ value, value
+
+In case where variable is a single value correct assignment looks like the
+following:
+
+.. parsed-literal::
+
+ variable =
+ value
+
+Length of chain variable is calculated by parser. Can not be defined
+explicitly.
+
+Variable op_type has to be defined as a first variable in file. It specifies
+what type of operations will be executed. For decoder op_type has to be set to
+``RTE_BBDEV_OP_TURBO_DEC`` and for encoder to ``RTE_BBDEV_OP_TURBO_ENC``.
+
+Full details of the meaning and valid values for the below fields are
+documented in *rte_bbdev_op.h*
+
+
+Turbo decoder test vectors template
+-----------------------------------
+
+For turbo decoder it has to be always set to ``RTE_BBDEV_OP_TURBO_DEC``
+
+.. parsed-literal::
+
+ op_type =
+ RTE_BBDEV_OP_TURBO_DEC
+
+Chain of uint32_t values. Note that it is possible to define more than one
+input/output entries which will result in chaining two or more data structures
+for *segmented Transport Blocks*
+
+.. parsed-literal::
+
+ input0 =
+ 0x00000000, 0x7f817f00, 0x7f7f8100, 0x817f8100, 0x81008100, 0x7f818100, 0x81817f00, 0x7f818100,
+ 0x81007f00, 0x7f818100, 0x817f8100, 0x81817f00, 0x81008100, 0x817f7f00, 0x7f7f8100, 0x81817f00
+
+Chain of uint32_t values
+
+.. parsed-literal::
+
+ input1 =
+ 0x7f7f0000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000,
+ 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000
+
+Chain of uint32_t values
+
+.. parsed-literal::
+
+ input2 =
+ 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000,
+ 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000
+
+Chain of uint32_t values
+
+.. parsed-literal::
+
+ hard_output0 =
+ 0xa7d6732e
+
+Chain of uint32_t values
+
+.. parsed-literal::
+
+ hard_output1 =
+ 0xa61
+
+Chain of uint32_t values
+
+.. parsed-literal::
+
+ soft_output0 =
+ 0x817f817f, 0x7f817f7f, 0x81818181, 0x817f7f81, 0x7f818181, 0x8181817f, 0x817f817f, 0x8181817f
+
+Chain of uint32_t values
+
+.. parsed-literal::
+
+ soft_output1 =
+ 0x817f7f81, 0x7f7f7f81, 0x7f7f8181
+
+uint32_t value
+
+.. parsed-literal::
+
+ e =
+ 44
+
+uint16_t value
+
+.. parsed-literal::
+
+ k =
+ 40
+
+uint8_t value
+
+.. parsed-literal::
+
+ rv_index =
+ 0
+
+uint8_t value
+
+.. parsed-literal::
+
+ iter_max =
+ 8
+
+uint8_t value
+
+.. parsed-literal::
+
+ iter_min =
+ 4
+
+uint8_t value
+
+.. parsed-literal::
+
+ expected_iter_count =
+ 8
+
+uint8_t value
+
+.. parsed-literal::
+
+ ext_scale =
+ 15
+
+uint8_t value
+
+.. parsed-literal::
+
+ num_maps =
+ 0
+
+Chain of flags for turbo decoder operation. Following flags can be used:
+
+- ``RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE``
+
+- ``RTE_BBDEV_TURBO_CRC_TYPE_24B``
+
+- ``RTE_BBDEV_TURBO_EQUALIZER``
+
+- ``RTE_BBDEV_TURBO_SOFT_OUT_SATURATE``
+
+- ``RTE_BBDEV_TURBO_HALF_ITERATION_EVEN``
+
+- ``RTE_BBDEV_TURBO_CONTINUE_CRC_MATCH``
+
+- ``RTE_BBDEV_TURBO_SOFT_OUTPUT``
+
+- ``RTE_BBDEV_TURBO_EARLY_TERMINATION``
+
+If ``RTE_BBDEV_TURBO_CRC_TYPE_24B`` is set input has to be increased by three
+bytes for CRC24B purposes
+
+.. parsed-literal::
+
+ op_flags =
+ RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE, RTE_BBDEV_TURBO_EQUALIZER,
+ RTE_BBDEV_TURBO_SOFT_OUTPUT
+
+Chain of operation statuses that are expected after operation is performed.
+Following statuses can be used:
+
+- ``DMA``
+
+- ``FCW``
+
+- ``CRC``
+
+- ``OK``
+
+``OK`` means no errors are expected. Cannot be used with other values.
+
+.. parsed-literal::
+
+ expected_status =
+ FCW, CRC
+
+
+Turbo encoder test vectors template
+-----------------------------------
+
+For turbo encoder it has to be always set to ``RTE_BBDEV_OP_TURBO_ENC``
+
+.. parsed-literal::
+
+ op_type =
+ RTE_BBDEV_OP_TURBO_ENC
+
+Chain of uint32_t values
+
+.. parsed-literal::
+
+ input0 =
+ 0x11d2bcac, 0x4d
+
+Chain of uint32_t values
+
+.. parsed-literal::
+
+ output0 =
+ 0xd2399179, 0x640eb999, 0x2cbaf577, 0xaf224ae2, 0x9d139927, 0xe6909b29,
+ 0xa25b7f47, 0x2aa224ce, 0x79f2
+
+uint32_t value
+
+.. parsed-literal::
+
+ e =
+ 272
+
+uint16_t value
+
+.. parsed-literal::
+
+ k =
+ 40
+
+uint16_t value
+
+.. parsed-literal::
+
+ ncb =
+ 192
+
+uint8_t value
+
+.. parsed-literal::
+
+ rv_index =
+ 0
+
+Chain of flags for turbo encoder operation. Following flags can be used:
+
+- ``RTE_BBDEV_TURBO_RV_INDEX_BYPASS``
+
+- ``RTE_BBDEV_TURBO_RATE_MATCH``
+
+- ``RTE_BBDEV_TURBO_CRC_24B_ATTACH``
+
+- ``RTE_BBDEV_TURBO_CRC_24A_ATTACH``
+
+
+.. parsed-literal::
+
+ op_flags =
+ RTE_BBDEV_TURBO_RATE_MATCH
+
+Chain of operation statuses that are expected after operation is performed.
+Following statuses can be used:
+
+- ``DMA``
+
+- ``FCW``
+
+- ``OK``
+
+``OK`` means no errors are expected. Cannot be used with other values.
+
+.. parsed-literal::
+
+ expected_status =
+ OK
--
2.7.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [dpdk-dev] [PATCH v2 0/5] Wireless Base Band Device (bbdev)
2017-10-18 2:14 [dpdk-dev] [PATCH v2 1/5] bbdev: librte_bbdev library Amr Mokhtar
` (3 preceding siblings ...)
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 5/5] bbdev: documentation Amr Mokhtar
@ 2017-10-18 2:14 ` Amr Mokhtar
4 siblings, 0 replies; 6+ messages in thread
From: Amr Mokhtar @ 2017-10-18 2:14 UTC (permalink / raw)
To: dev
Cc: thomas, anatoly.burakov, pablo.de.lara.guarch, niall.power,
chris.macnamara, Amr Mokhtar
Hello again,
A v2 patch of the Wireless Base Band Device (bbdev) RFC is enclosed.
Addressing the feedback received from the community and more crafting for the
application interface.
---
v2:
* Split the functionality of rte_bbdev_configure() into smaller portions ->
rte_bbdev_setup_queues() and rte_bbdev_enable_intr()
* Split rte_bbdev_enqueue() -> rte_bbdev_enc_enqueue() and rte_bbdev_dec_enqueue()
* Split rte_bbdev_dequeue() -> rte_bbdev_enc_dequeue() and rte_bbdev_dec_dequeue()
* Removed attached flag until hotplug is properly supported in DPDK
* More details on the installation of FlexRAN SDK libraries in accordance with Turbo_sw PMD
* Minor build fixes for other targets: bsdapp-gcc, bsdapp-clang and linuxapp-clang.
* Better-organized patchwork
v1:
* Initial release of BBDEV library.
* Support Turbo Code FEC with two virtual devices (vdev):
- Null Turbo PMD
- Turbo_sw PMD
* A complete Test suite for Turbo Encode/Decode and None operations
* Test Vectors parsing and testing functionality
* Sample App for a looped-back bbdev with ethdev
* Documentation in rst format for all new components
[1] http://dpdk.org/dev/patchwork/patch/29447/
[2] http://dpdk.org/dev/patchwork/patch/29448/
[3] http://dpdk.org/dev/patchwork/patch/29450/
[4] http://dpdk.org/dev/patchwork/patch/29449/
[5] http://dpdk.org/dev/patchwork/patch/29452/
[6] http://dpdk.org/dev/patchwork/patch/29451/
RFC:
[1] http://dpdk.org/ml/archives/dev/2017-August/073585.html
[2] http://dpdk.org/ml/archives/dev/2017-August/073584.html
Amr Mokhtar (5):
bbdev: librte_bbdev library
bbdev: PMD drivers (null/turbo_sw)
bbdev: test applications
bbdev: sample app
bbdev: documentation
MAINTAINERS | 10 +
app/Makefile | 4 +
app/test-bbdev/Makefile | 53 +
app/test-bbdev/main.c | 317 +++
app/test-bbdev/main.h | 144 ++
app/test-bbdev/test-bbdev.py | 132 ++
app/test-bbdev/test_bbdev.c | 1406 +++++++++++++
app/test-bbdev/test_bbdev_perf.c | 2090 ++++++++++++++++++++
app/test-bbdev/test_bbdev_vector.c | 884 +++++++++
app/test-bbdev/test_bbdev_vector.h | 98 +
app/test-bbdev/test_vectors/bbdev_vector_null.data | 32 +
.../test_vectors/bbdev_vector_td_default.data | 80 +
.../test_vectors/bbdev_vector_te_default.data | 60 +
config/common_base | 23 +
doc/api/doxy-api-index.md | 1 +
doc/api/doxy-api.conf | 1 +
doc/guides/bbdevs/index.rst | 40 +
doc/guides/bbdevs/null.rst | 77 +
doc/guides/bbdevs/turbo_sw.rst | 169 ++
doc/guides/index.rst | 1 +
doc/guides/prog_guide/bbdev.rst | 621 ++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_17_11.rst | 10 +
doc/guides/sample_app_ug/bbdev_app.rst | 160 ++
doc/guides/sample_app_ug/index.rst | 1 +
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testbbdev.rst | 546 +++++
drivers/Makefile | 2 +
drivers/bbdev/Makefile | 41 +
drivers/bbdev/null/Makefile | 49 +
drivers/bbdev/null/bbdev_null.c | 377 ++++
drivers/bbdev/null/rte_pmd_bbdev_null_version.map | 3 +
drivers/bbdev/turbo_sw/Makefile | 59 +
drivers/bbdev/turbo_sw/bbdev_turbo_software.c | 1235 ++++++++++++
.../bbdev/turbo_sw/bbdev_turbo_software_tables.h | 1344 +++++++++++++
.../turbo_sw/rte_pmd_bbdev_turbo_sw_version.map | 3 +
examples/Makefile | 1 +
examples/bbdev_app/Makefile | 50 +
examples/bbdev_app/main.c | 1396 +++++++++++++
lib/Makefile | 3 +
lib/librte_bbdev/Makefile | 56 +
lib/librte_bbdev/rte_bbdev.c | 1095 ++++++++++
lib/librte_bbdev/rte_bbdev.h | 741 +++++++
lib/librte_bbdev/rte_bbdev_op.h | 514 +++++
lib/librte_bbdev/rte_bbdev_pci.h | 288 +++
lib/librte_bbdev/rte_bbdev_pmd.h | 223 +++
lib/librte_bbdev/rte_bbdev_vdev.h | 102 +
lib/librte_bbdev/rte_bbdev_version.map | 37 +
mk/rte.app.mk | 13 +
49 files changed, 14594 insertions(+)
create mode 100644 app/test-bbdev/Makefile
create mode 100644 app/test-bbdev/main.c
create mode 100644 app/test-bbdev/main.h
create mode 100755 app/test-bbdev/test-bbdev.py
create mode 100644 app/test-bbdev/test_bbdev.c
create mode 100644 app/test-bbdev/test_bbdev_perf.c
create mode 100644 app/test-bbdev/test_bbdev_vector.c
create mode 100644 app/test-bbdev/test_bbdev_vector.h
create mode 100644 app/test-bbdev/test_vectors/bbdev_vector_null.data
create mode 100644 app/test-bbdev/test_vectors/bbdev_vector_td_default.data
create mode 100644 app/test-bbdev/test_vectors/bbdev_vector_te_default.data
create mode 100644 doc/guides/bbdevs/index.rst
create mode 100644 doc/guides/bbdevs/null.rst
create mode 100644 doc/guides/bbdevs/turbo_sw.rst
create mode 100644 doc/guides/prog_guide/bbdev.rst
create mode 100644 doc/guides/sample_app_ug/bbdev_app.rst
create mode 100644 doc/guides/tools/testbbdev.rst
create mode 100644 drivers/bbdev/Makefile
create mode 100644 drivers/bbdev/null/Makefile
create mode 100644 drivers/bbdev/null/bbdev_null.c
create mode 100644 drivers/bbdev/null/rte_pmd_bbdev_null_version.map
create mode 100644 drivers/bbdev/turbo_sw/Makefile
create mode 100644 drivers/bbdev/turbo_sw/bbdev_turbo_software.c
create mode 100644 drivers/bbdev/turbo_sw/bbdev_turbo_software_tables.h
create mode 100644 drivers/bbdev/turbo_sw/rte_pmd_bbdev_turbo_sw_version.map
create mode 100644 examples/bbdev_app/Makefile
create mode 100644 examples/bbdev_app/main.c
create mode 100644 lib/librte_bbdev/Makefile
create mode 100644 lib/librte_bbdev/rte_bbdev.c
create mode 100644 lib/librte_bbdev/rte_bbdev.h
create mode 100644 lib/librte_bbdev/rte_bbdev_op.h
create mode 100644 lib/librte_bbdev/rte_bbdev_pci.h
create mode 100644 lib/librte_bbdev/rte_bbdev_pmd.h
create mode 100644 lib/librte_bbdev/rte_bbdev_vdev.h
create mode 100644 lib/librte_bbdev/rte_bbdev_version.map
--
2.7.4
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-10-18 2:15 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-10-18 2:14 [dpdk-dev] [PATCH v2 1/5] bbdev: librte_bbdev library Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 2/5] bbdev: PMD drivers (null/turbo_sw) Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 3/5] bbdev: test applications Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 4/5] bbdev: sample app Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 5/5] bbdev: documentation Amr Mokhtar
2017-10-18 2:14 ` [dpdk-dev] [PATCH v2 0/5] Wireless Base Band Device (bbdev) Amr Mokhtar
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).