From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9F148A034C; Thu, 28 Apr 2022 12:01:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5140640E50; Thu, 28 Apr 2022 12:01:50 +0200 (CEST) Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by mails.dpdk.org (Postfix) with ESMTP id 2ACF34013F for ; Thu, 28 Apr 2022 12:01:48 +0200 (CEST) Received: by mail-pf1-f180.google.com with SMTP id y38so3832429pfa.6 for ; Thu, 28 Apr 2022 03:01:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=atomicrules-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Tyy/orWfktfB57mSZZFWaIIftYjImTJ8wHkV9GhL2lw=; b=6eae8wouE6l2UWK5erx3bh0ALAcpxd/iD3d5mN6Uu27e2QpiAsAv369pndiHBIoDYi PaeJi56tw+TtVygUdPpaYE5G0FrLVUPJp6GPT+3S3vb7OSpZpF0z0+Uls+FhfN2HQJEv 2K6DXCJnCSwSwn1rAUYYfkYATf/wwou6b81jzly9uIY22LFuoR9YcTeP91VZd2xbaqa2 TwZJ8+2ACIkyr2lyROU+u/L4od5lSmr9Az9YQ6e1qu7CiyZYMx0ieWyldkMc0ndH2CsR H1OQ1ZnrewWSKi17UORoHQgDlgzRw1hv/i4GBvGoXD60Fzjr0hjLBnAan4zliQt+ZU6j MhwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Tyy/orWfktfB57mSZZFWaIIftYjImTJ8wHkV9GhL2lw=; b=U40nvOTzWXjwOIhzsTgj3fE8mEPRDrwXnzq6JSQcRRmbQCX3/nEbnPqbspsu5hoNb7 m1UYK53C1qlQXkLeh2keopLZ3BqMYYR0sKRyiQhhEH7+tWtq7sDqlreyCBjuj83D/JCs ya3NOc43jfOMJSVsizHUClEr2dxfkUb7ffAwtwlTrBtol08aR/jli/2iXDR4xtigZ9FH 04T8CEQhFbgS3yYGH68FNxhGwAE8i3DtQpLfqVGJ4QLsgYvx2EfTHiQQ6ZR0a5o6aL/3 YYJSsX6mMP2nElOHCEVn8cogo1lTPyvNzaW2nDC0M9GX6mwM1k1ay5ixiv5XgWjGjgN+ kQ+g== X-Gm-Message-State: AOAM530zAKRyMaaLFAaKaSvw/UdMjWCB3/1nofYo/W6zWCOpAIzlCL0M zxM5VSSaZQa3Y+4LQYakxMA2li/+222bSUkJ/ZxLKQ== X-Google-Smtp-Source: ABdhPJw2DzxGzv2PIOIdnUoE/+tAzQ0tcDwNayGBNzusbJtr+efB5VIjZQg1f5al26BNQ4OUFPKQblxRoLceB2DBu6Q= X-Received: by 2002:aa7:8b0e:0:b0:50d:4310:c11f with SMTP id f14-20020aa78b0e000000b0050d4310c11fmr8137773pfd.49.1651140106348; Thu, 28 Apr 2022 03:01:46 -0700 (PDT) MIME-Version: 1.0 References: <20220421151900.703467-1-john.miller@atomicrules.com> <20220421151900.703467-5-john.miller@atomicrules.com> In-Reply-To: From: John Miller Date: Thu, 28 Apr 2022 06:01:35 -0400 Message-ID: Subject: Re: [PATCH 05/10] baseband/ark: add ark baseband device To: "Chautru, Nicolas" Cc: "dev@dpdk.org" , "ferruh.yigit@xilinx.com" , "ed.czeck@atomicrules.com" , Shepard Siegel Content-Type: multipart/alternative; boundary="000000000000837e7f05ddb40304" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000837e7f05ddb40304 Content-Type: text/plain; charset="UTF-8" Hi Nic, Yes, i will split this into smaller patches and take care of the TODO's. I will have a v1 patch set ready with these changes within a couple of days. Given the 4/10 deadline for 22.07 it would seem that 22.11 is our target. Thank you, -John On Wed, Apr 27, 2022 at 2:38 PM Chautru, Nicolas wrote: > Hi John, > > Do you think this one can be split into a few incremental commits? > > There are a few TODOs, is that v1 ready for review? Also you are targeting > 22.11 right? > > Thanks > Nic > > > -----Original Message----- > > From: John Miller > > Sent: Thursday, April 21, 2022 8:19 AM > > To: dev@dpdk.org > > Cc: ferruh.yigit@xilinx.com; ed.czeck@atomicrules.com; John Miller > > > > Subject: [PATCH 05/10] baseband/ark: add ark baseband device > > > > Add new ark baseband device. > > > > Signed-off-by: John Miller > > --- > > drivers/baseband/ark/ark_bbdev.c | 1064 +++++++++++++++++++++++ > > drivers/baseband/ark/ark_bbdev_common.c | 125 +++ > > drivers/baseband/ark/ark_bbdev_common.h | 92 ++ > > drivers/baseband/ark/ark_bbdev_custom.c | 201 +++++ > > drivers/baseband/ark/ark_bbdev_custom.h | 30 + > > drivers/baseband/ark/meson.build | 11 + > > drivers/baseband/ark/version.map | 3 + > > 7 files changed, 1526 insertions(+) > > create mode 100644 drivers/baseband/ark/ark_bbdev.c create mode > > 100644 drivers/baseband/ark/ark_bbdev_common.c > > create mode 100644 drivers/baseband/ark/ark_bbdev_common.h > > create mode 100644 drivers/baseband/ark/ark_bbdev_custom.c > > create mode 100644 drivers/baseband/ark/ark_bbdev_custom.h > > create mode 100644 drivers/baseband/ark/meson.build create mode > > 100644 drivers/baseband/ark/version.map > > > > diff --git a/drivers/baseband/ark/ark_bbdev.c > > b/drivers/baseband/ark/ark_bbdev.c > > new file mode 100644 > > index 0000000000..b23bbd44d1 > > --- /dev/null > > +++ b/drivers/baseband/ark/ark_bbdev.c > > @@ -0,0 +1,1064 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(c) 2016-2021 Atomic Rules LLC */ > > + > > +#include "ark_common.h" > > +#include "ark_bbdev_common.h" > > +#include "ark_bbdev_custom.h" > > +#include "ark_ddm.h" > > +#include "ark_mpu.h" > > +#include "ark_rqp.h" > > +#include "ark_udm.h" > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#include > > + > > +#define DRIVER_NAME baseband_ark > > + > > +RTE_LOG_REGISTER_DEFAULT(ark_bbdev_logtype, DEBUG); > > + > > +#define ARK_SYSCTRL_BASE 0x0 > > +#define ARK_PKTGEN_BASE 0x10000 > > +#define ARK_MPU_RX_BASE 0x20000 > > +#define ARK_UDM_BASE 0x30000 > > +#define ARK_MPU_TX_BASE 0x40000 > > +#define ARK_DDM_BASE 0x60000 > > +#define ARK_PKTDIR_BASE 0xa0000 > > +#define ARK_PKTCHKR_BASE 0x90000 > > +#define ARK_RCPACING_BASE 0xb0000 > > +#define ARK_MPU_QOFFSET 0x00100 > > + > > +#define BB_ARK_TX_Q_FACTOR 4 > > + > > +/* TODO move to UDM, verify configuration */ #define ARK_RX_META_SIZE > > +32 #define ARK_RX_META_OFFSET (RTE_PKTMBUF_HEADROOM - > > ARK_RX_META_SIZE) > > +#define ARK_RX_MAX_NOCHAIN (RTE_MBUF_DEFAULT_DATAROOM) > > + > > +static_assert(sizeof(struct ark_rx_meta) == ARK_RX_META_SIZE, > > +"Unexpected struct size ark_rx_meta"); static_assert(sizeof(union > > +ark_tx_meta) == 8, "Unexpected struct size ark_tx_meta"); > > + > > +static struct rte_pci_id pci_id_ark[] = { > > + {RTE_PCI_DEVICE(AR_VENDOR_ID, 0x1015)}, > > + {RTE_PCI_DEVICE(AR_VENDOR_ID, 0x1016)}, > > + {.device_id = 0}, > > +}; > > + > > +static const struct ark_dev_caps > > +ark_device_caps[] = { > > + SET_DEV_CAPS(0x1015, true), > > + SET_DEV_CAPS(0x1016, true), > > + {.device_id = 0,} > > +}; > > + > > + > > +/* Forward declarations */ > > +static const struct rte_bbdev_ops ark_bbdev_pmd_ops; > > + > > + > > +/* queue */ > > +struct ark_bbdev_queue { > > + struct rte_ring *active_ops; /* Ring for processed packets */ > > + > > + /* RX components */ > > + /* array of physical addresses of the mbuf data pointer */ > > + rte_iova_t *rx_paddress_q; > > + struct ark_udm_t *udm; > > + struct ark_mpu_t *rx_mpu; > > + > > + /* TX components */ > > + union ark_tx_meta *tx_meta_q; > > + struct ark_mpu_t *tx_mpu; > > + struct ark_ddm_t *ddm; > > + > > + /* */ > > + uint32_t tx_queue_mask; > > + uint32_t rx_queue_mask; > > + > > + int32_t rx_seed_index; /* step 1 set with empty mbuf */ > > + int32_t rx_cons_index; /* step 3 consumed by driver */ > > + > > + /* 3 indexes to the paired data rings. */ > > + int32_t tx_prod_index; /* where to put the next one */ > > + int32_t tx_free_index; /* local copy of tx_cons_index */ > > + > > + /* separate cache line -- written by FPGA -- RX announce */ > > + RTE_MARKER cacheline1 __rte_cache_min_aligned; > > + volatile int32_t rx_prod_index; /* step 2 filled by FPGA */ > > + > > + /* Separate cache line -- written by FPGA -- RX completion */ > > + RTE_MARKER cacheline2 __rte_cache_min_aligned; > > + volatile int32_t tx_cons_index; /* hw is done, can be freed */ } > > +__rte_cache_aligned; > > + > > +static int > > +ark_bb_hw_q_setup(struct rte_bbdev *bbdev, uint16_t q_id, uint16_t > > +queue_size) { > > + struct ark_bbdev_queue *q = bbdev->data- > > >queues[q_id].queue_private; > > + > > + rte_iova_t queue_base; > > + rte_iova_t phys_addr_q_base; > > + rte_iova_t phys_addr_prod_index; > > + rte_iova_t phys_addr_cons_index; > > + > > + uint32_t write_interval_ns = 500; /* TODO this seems big */ > > + > > + if (ark_mpu_verify(q->rx_mpu, sizeof(rte_iova_t))) { > > + ARK_BBDEV_LOG(ERR, "Illegal hw/sw configuration RX > > queue"); > > + return -1; > > + } > > + ARK_BBDEV_LOG(DEBUG, "ark_bb_q setup %u:%u", > > + bbdev->data->dev_id, q_id); > > + > > + /* RX MPU */ > > + phys_addr_q_base = rte_malloc_virt2iova(q->rx_paddress_q); > > + /* Force TX mode on MPU to match bbdev behavior */ > > + ark_mpu_configure(q->rx_mpu, phys_addr_q_base, queue_size, 1); > > + ark_mpu_reset_stats(q->rx_mpu); > > + ark_mpu_start(q->rx_mpu); > > + > > + /* UDM */ > > + queue_base = rte_malloc_virt2iova(q); > > + phys_addr_prod_index = queue_base + > > + offsetof(struct ark_bbdev_queue, rx_prod_index); > > + ark_udm_write_addr(q->udm, phys_addr_prod_index); > > + ark_udm_queue_enable(q->udm, 1); > > + > > + /* TX MPU */ > > + phys_addr_q_base = rte_malloc_virt2iova(q->tx_meta_q); > > + ark_mpu_configure(q->tx_mpu, phys_addr_q_base, > > + BB_ARK_TX_Q_FACTOR * queue_size, 1); > > + ark_mpu_start(q->tx_mpu); > > + > > + /* DDM */ > > + phys_addr_cons_index = queue_base + > > + offsetof(struct ark_bbdev_queue, tx_cons_index); > > + ark_ddm_setup(q->ddm, phys_addr_cons_index, write_interval_ns); > > + > > + return 0; > > +} > > + > > +/* Setup a queue */ > > +static int > > +ark_bb_q_setup(struct rte_bbdev *bbdev, uint16_t q_id, > > + const struct rte_bbdev_queue_conf *queue_conf) { > > + struct ark_bbdev_queue *q; > > + struct ark_bbdevice *ark_bb = bbdev->data->dev_private; > > + > > + const uint32_t queue_size = queue_conf->queue_size; > > + const int socket_id = queue_conf->socket; > > + const uint64_t pg_sz = sysconf(_SC_PAGESIZE); > > + char ring_name[RTE_RING_NAMESIZE]; > > + > > + /* Configuration checks */ > > + if (!rte_is_power_of_2(queue_size)) { > > + ARK_BBDEV_LOG(ERR, > > + "Configuration queue size" > > + " must be power of two %u", > > + queue_size); > > + return -EINVAL; > > + } > > + > > + if (RTE_PKTMBUF_HEADROOM < ARK_RX_META_SIZE) { > > + ARK_BBDEV_LOG(ERR, > > + "Error: Ark bbdev requires head room > %d > bytes > > (%s)", > > + ARK_RX_META_SIZE, __func__); > > + return -EINVAL; > > + } > > + > > + /* Allocate the queue data structure. */ > > + q = rte_zmalloc_socket(RTE_STR(DRIVER_NAME), sizeof(*q), > > + RTE_CACHE_LINE_SIZE, queue_conf->socket); > > + if (q == NULL) { > > + ARK_BBDEV_LOG(ERR, "Failed to allocate queue memory"); > > + return -ENOMEM; > > + } > > + bbdev->data->queues[q_id].queue_private = q; > > + > > + /* RING */ > > + snprintf(ring_name, RTE_RING_NAMESIZE, RTE_STR(DRIVER_NAME) > > "%u:%u", > > + bbdev->data->dev_id, q_id); > > + q->active_ops = rte_ring_create(ring_name, > > + queue_size, > > + queue_conf->socket, > > + RING_F_SP_ENQ | RING_F_SC_DEQ); > > + if (q->active_ops == NULL) { > > + ARK_BBDEV_LOG(ERR, "Failed to create ring"); > > + goto free_all; > > + } > > + > > + q->rx_queue_mask = queue_size - 1; > > + q->tx_queue_mask = (BB_ARK_TX_Q_FACTOR * queue_size) - 1; > > + > > + /* Each mbuf requires 2 to 4 objects, factor by > > BB_ARK_TX_Q_FACTOR */ > > + q->tx_meta_q = > > + rte_zmalloc_socket("Ark_bb_txqueue meta", > > + queue_size * BB_ARK_TX_Q_FACTOR * > > + sizeof(union ark_tx_meta), > > + pg_sz, > > + socket_id); > > + > > + if (q->tx_meta_q == 0) { > > + ARK_BBDEV_LOG(ERR, "Failed to allocate " > > + "queue memory in %s", __func__); > > + goto free_all; > > + } > > + > > + q->ddm = RTE_PTR_ADD(ark_bb->ddm.v, q_id * > > ARK_DDM_QOFFSET); > > + q->tx_mpu = RTE_PTR_ADD(ark_bb->mputx.v, q_id * > > ARK_MPU_QOFFSET); > > + > > + q->rx_paddress_q = > > + rte_zmalloc_socket("ark_bb_rx_paddress_q", > > + queue_size * sizeof(rte_iova_t), > > + pg_sz, > > + socket_id); > > + > > + if (q->rx_paddress_q == 0) { > > + ARK_BBDEV_LOG(ERR, > > + "Failed to allocate queue memory in %s", > > + __func__); > > + goto free_all; > > + } > > + q->udm = RTE_PTR_ADD(ark_bb->udm.v, q_id * > > ARK_UDM_QOFFSET); > > + q->rx_mpu = RTE_PTR_ADD(ark_bb->mpurx.v, q_id * > > ARK_MPU_QOFFSET); > > + > > + /* Structure have been configured, set the hardware */ > > + return ark_bb_hw_q_setup(bbdev, q_id, queue_size); > > + > > +free_all: > > + rte_free(q->tx_meta_q); > > + rte_free(q->rx_paddress_q); > > + rte_free(q); > > + return -EFAULT; > > +} > > + > > +/* Release queue */ > > +static int > > +ark_bb_q_release(struct rte_bbdev *bbdev, uint16_t q_id) { > > + struct ark_bbdev_queue *q = bbdev->data- > > >queues[q_id].queue_private; > > + > > + /* TODO Wait for ddm to send out all packets in flight, > > + * Is this only called after q stop? > > + */ > > + > > + ark_mpu_dump(q->rx_mpu, "rx_MPU release", q_id); > > + ark_mpu_dump(q->tx_mpu, "tx_MPU release", q_id); > > + > > + rte_ring_free(q->active_ops); > > + rte_free(q->tx_meta_q); > > + rte_free(q->rx_paddress_q); > > + rte_free(q); > > + bbdev->data->queues[q_id].queue_private = NULL; > > + > > + ARK_BBDEV_LOG(DEBUG, "released device queue %u:%u", > > + bbdev->data->dev_id, q_id); > > + return 0; > > +} > > + > > +static int > > +ark_bbdev_start(struct rte_bbdev *bbdev) { > > + struct ark_bbdevice *ark_bb = bbdev->data->dev_private; > > + > > + ARK_BBDEV_LOG(DEBUG, "Starting device %u", bbdev->data- > > >dev_id); > > + if (ark_bb->started) > > + return 0; > > + > > + /* start UDM */ > > + ark_udm_start(ark_bb->udm.v); > > + > > + /* start DDM */ > > + ark_ddm_start(ark_bb->ddm.v); > > + > > + ark_bb->started = 1; > > + > > + if (ark_bb->start_pg) > > + ark_pktchkr_run(ark_bb->pc); > > + > > + if (ark_bb->start_pg) { > > + pthread_t thread; > > + > > + /* Delay packet generator start allow the hardware to be > > ready > > + * This is only used for sanity checking with internal > generator > > + */ > > + if (pthread_create(&thread, NULL, > > + ark_pktgen_delay_start, ark_bb->pg)) { > > + ARK_BBDEV_LOG(ERR, "Could not create pktgen " > > + "starter thread"); > > + return -1; > > + } > > + } > > + > > + return 0; > > +} > > + > > +static void > > +ark_bbdev_stop(struct rte_bbdev *bbdev) { > > + struct ark_bbdevice *ark_bb = bbdev->data->dev_private; > > + struct ark_mpu_t *mpu; > > + unsigned int i; > > + int status; > > + > > + ARK_BBDEV_LOG(DEBUG, "Stopping device %u", bbdev->data- > > >dev_id); > > + > > + if (!ark_bb->started) > > + return; > > + > > + /* Stop the packet generator */ > > + if (ark_bb->start_pg) > > + ark_pktgen_pause(ark_bb->pg); > > + > > + /* Stop DDM */ > > + /* Wait up to 0.1 second. each stop is up to 1000 * 10 useconds */ > > + for (i = 0; i < 10; i++) { > > + status = ark_ddm_stop(ark_bb->ddm.v, 1); > > + if (status == 0) > > + break; > > + } > > + if (status || i != 0) { > > + ARK_BBDEV_LOG(ERR, "DDM stop anomaly. status:" > > + " %d iter: %u. (%s)", > > + status, > > + i, > > + __func__); > > + ark_ddm_dump(ark_bb->ddm.v, "Stop anomaly"); > > + > > + mpu = ark_bb->mputx.v; > > + for (i = 0; i < ark_bb->max_nb_queues; i++) { > > + ark_mpu_dump(mpu, "DDM failure dump", i); > > + mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET); > > + } > > + } > > + ark_ddm_dump_stats(ark_bb->ddm.v, "bbdev stop"); > > + > > + /* STOP RX Side */ > > + /* Stop UDM multiple tries attempted */ > > + for (i = 0; i < 10; i++) { > > + status = ark_udm_stop(ark_bb->udm.v, 1); > > + if (status == 0) > > + break; > > + } > > + if (status || i != 0) { > > + ARK_BBDEV_LOG(WARNING, "UDM stop anomaly. status %d > > iter: %u. (%s)", > > + status, i, __func__); > > + ark_udm_dump(ark_bb->udm.v, "Stop anomaly"); > > + > > + mpu = ark_bb->mpurx.v; > > + for (i = 0; i < ark_bb->max_nb_queues; i++) { > > + ark_mpu_dump(mpu, "UDM Stop anomaly", i); > > + mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET); > > + } > > + } > > + > > + ark_udm_dump_stats(ark_bb->udm.v, "Post stop"); > > + ark_udm_dump_perf(ark_bb->udm.v, "Post stop"); > > + > > + /* Stop the packet checker if it is running */ > > + if (ark_bb->start_pg) { > > + ark_pktchkr_dump_stats(ark_bb->pc); > > + ark_pktchkr_stop(ark_bb->pc); > > + } > > +} > > + > > +static int > > +ark_bb_q_start(struct rte_bbdev *bbdev, uint16_t q_id) { > > + struct ark_bbdev_queue *q = bbdev->data- > > >queues[q_id].queue_private; > > + ARK_BBDEV_LOG(DEBUG, "ark_bb_q start %u:%u", bbdev->data- > > >dev_id, q_id); > > + ark_mpu_start(q->tx_mpu); > > + ark_mpu_start(q->rx_mpu); > > + return 0; > > +} > > +static int > > +ark_bb_q_stop(struct rte_bbdev *bbdev, uint16_t q_id) { > > + struct ark_bbdev_queue *q = bbdev->data- > > >queues[q_id].queue_private; > > + ARK_BBDEV_LOG(DEBUG, "ark_bb_q stop %u:%u", bbdev->data- > > >dev_id, q_id); > > + ark_mpu_stop(q->tx_mpu); > > + ark_mpu_stop(q->rx_mpu); > > + return 0; > > +} > > + > > +/* > > +**************************************************************** > > ******* > > +** */ > > +/* Common function for all enqueue and dequeue ops */ static inline > > +void ark_bb_enqueue_desc_fill(struct ark_bbdev_queue *q, > > + struct rte_mbuf *mbuf, > > + uint16_t offset, /* Extra offset */ > > + uint8_t flags, > > + uint32_t *meta, > > + uint8_t meta_cnt /* 0, 1 or 2 */ > > + ) > > +{ > > + union ark_tx_meta *tx_meta; > > + int32_t tx_idx; > > + uint8_t m; > > + > > + /* Header */ > > + tx_idx = q->tx_prod_index & q->tx_queue_mask; > > + tx_meta = &q->tx_meta_q[tx_idx]; > > + tx_meta->data_len = rte_pktmbuf_data_len(mbuf) - offset; > > + tx_meta->flags = flags; > > + tx_meta->meta_cnt = meta_cnt; > > + tx_meta->user1 = *meta++; > > + q->tx_prod_index++; > > + > > + for (m = 0; m < meta_cnt; m++) { > > + tx_idx = q->tx_prod_index & q->tx_queue_mask; > > + tx_meta = &q->tx_meta_q[tx_idx]; > > + tx_meta->usermeta0 = *meta++; > > + tx_meta->usermeta1 = *meta++; > > + q->tx_prod_index++; > > + } > > + > > + tx_idx = q->tx_prod_index & q->tx_queue_mask; > > + tx_meta = &q->tx_meta_q[tx_idx]; > > + tx_meta->physaddr = rte_mbuf_data_iova(mbuf) + offset; > > + q->tx_prod_index++; > > +} > > + > > +static inline void > > +ark_bb_enqueue_segmented_pkt(struct ark_bbdev_queue *q, > > + struct rte_mbuf *mbuf, > > + uint16_t offset, > > + uint32_t *meta, uint8_t meta_cnt) { > > + struct rte_mbuf *next; > > + uint8_t flags = ARK_DDM_SOP; > > + > > + while (mbuf != NULL) { > > + next = mbuf->next; > > + flags |= (next == NULL) ? ARK_DDM_EOP : 0; > > + > > + ark_bb_enqueue_desc_fill(q, mbuf, offset, flags, > > + meta, meta_cnt); > > + > > + flags &= ~ARK_DDM_SOP; /* drop SOP flags */ > > + meta_cnt = 0; > > + offset = 0; > > + > > + mbuf = next; > > + } > > +} > > + > > +static inline int > > +ark_bb_enqueue_common(struct ark_bbdev_queue *q, > > + struct rte_mbuf *m_in, struct rte_mbuf *m_out, > > + uint16_t offset, > > + uint32_t *meta, uint8_t meta_cnt) { > > + int32_t free_queue_space; > > + int32_t rx_idx; > > + > > + /* TX side limit */ > > + free_queue_space = q->tx_queue_mask - > > + (q->tx_prod_index - q->tx_free_index); > > + if (unlikely(free_queue_space < (2 + (2 * m_in->nb_segs)))) > > + return 1; > > + > > + /* RX side limit */ > > + free_queue_space = q->rx_queue_mask - > > + (q->rx_seed_index - q->rx_cons_index); > > + if (unlikely(free_queue_space < m_out->nb_segs)) > > + return 1; > > + > > + if (unlikely(m_in->nb_segs > 1)) > > + ark_bb_enqueue_segmented_pkt(q, m_in, offset, meta, > > meta_cnt); > > + else > > + ark_bb_enqueue_desc_fill(q, m_in, offset, > > + ARK_DDM_SOP | ARK_DDM_EOP, > > + meta, meta_cnt); > > + > > + /* We assume that the return mubf has exactly enough segments for > > + * return data, which is 2048 bytes per segment. > > + */ > > + do { > > + rx_idx = q->rx_seed_index & q->rx_queue_mask; > > + q->rx_paddress_q[rx_idx] = m_out->buf_iova; > > + q->rx_seed_index++; > > + m_out = m_out->next; > > + } while (m_out); > > + > > + return 0; > > +} > > + > > +static inline void > > +ark_bb_enqueue_finalize(struct rte_bbdev_queue_data *q_data, > > + struct ark_bbdev_queue *q, > > + void **ops, > > + uint16_t nb_ops, uint16_t nb) > > +{ > > + /* BBDEV global stats */ > > + /* These are not really errors, not sure why bbdev counts these. */ > > + q_data->queue_stats.enqueue_err_count += nb_ops - nb; > > + q_data->queue_stats.enqueued_count += nb; > > + > > + /* Notify HW that */ > > + if (unlikely(nb == 0)) > > + return; > > + > > + ark_mpu_set_producer(q->tx_mpu, q->tx_prod_index); > > + ark_mpu_set_producer(q->rx_mpu, q->rx_seed_index); > > + > > + /* Queue info for dequeue-side processing */ > > + rte_ring_enqueue_burst(q->active_ops, > > + (void **)ops, nb, NULL); > > +} > > + > > +static int > > +ark_bb_dequeue_segmented(struct rte_mbuf *mbuf0, > > + int32_t *prx_cons_index, > > + uint16_t pkt_len > > + ) > > +{ > > + struct rte_mbuf *mbuf; > > + uint16_t data_len; > > + uint16_t remaining; > > + uint16_t segments = 1; > > + > > + data_len = RTE_MIN(pkt_len, RTE_MBUF_DEFAULT_DATAROOM); > > + remaining = pkt_len - data_len; > > + > > + mbuf = mbuf0; > > + mbuf0->data_len = data_len; > > + while (remaining) { > > + segments += 1; > > + mbuf = mbuf->next; > > + if (unlikely(mbuf == 0)) { > > + ARK_BBDEV_LOG(CRIT, "Expected chained mbuf with > > " > > + "at least %d segments for dequeue " > > + "of packet length %d", > > + segments, pkt_len); > > + return 1; > > + } > > + > > + data_len = RTE_MIN(remaining, > > + RTE_MBUF_DEFAULT_DATAROOM); > > + remaining -= data_len; > > + > > + mbuf->data_len = data_len; > > + *prx_cons_index += 1; > > + } > > + > > + if (mbuf->next != 0) { > > + ARK_BBDEV_LOG(CRIT, "Expected chained mbuf with " > > + "at exactly %d segments for dequeue " > > + "of packet length %d. Found %d " > > + "segments", > > + segments, pkt_len, mbuf0->nb_segs); > > + return 1; > > + } > > + return 0; > > +} > > + > > +/* > > +**************************************************************** > > ******* > > +** */ > > +/* LDPC Decode ops */ > > +static int16_t > > +ark_bb_enqueue_ldpc_dec_one_op(struct ark_bbdev_queue *q, > > + struct rte_bbdev_dec_op *this_op) { > > + struct rte_bbdev_op_ldpc_dec *ldpc_dec_op = &this_op->ldpc_dec; > > + struct rte_mbuf *m_in = ldpc_dec_op->input.data; > > + struct rte_mbuf *m_out = ldpc_dec_op->hard_output.data; > > + uint16_t offset = ldpc_dec_op->input.offset; > > + uint32_t meta[5] = {0}; > > + uint8_t meta_cnt = 0; > > + > > + /* User's meta move from bbdev op to Arkville HW */ > > + if (ark_bb_user_enqueue_ldpc_dec(this_op, meta, &meta_cnt)) { > > + ARK_BBDEV_LOG(ERR, "%s failed", __func__); > > + return 1; > > + } > > + > > + return ark_bb_enqueue_common(q, m_in, m_out, offset, meta, > > meta_cnt); > > +} > > + > > +/* Enqueue LDPC Decode -- burst */ > > +static uint16_t > > +ark_bb_enqueue_ldpc_dec_ops(struct rte_bbdev_queue_data *q_data, > > + struct rte_bbdev_dec_op **ops, uint16_t > nb_ops) { > > + struct ark_bbdev_queue *q = q_data->queue_private; > > + unsigned int max_enq; > > + uint16_t nb; > > + > > + max_enq = rte_ring_free_count(q->active_ops); > > + max_enq = RTE_MIN(max_enq, nb_ops); > > + for (nb = 0; nb < max_enq; nb++) { > > + if (ark_bb_enqueue_ldpc_dec_one_op(q, ops[nb])) > > + break; > > + } > > + > > + ark_bb_enqueue_finalize(q_data, q, (void **)ops, nb_ops, nb); > > + return nb; > > +} > > + > > +/* > > +**************************************************************** > > ******* > > +** */ > > +/* Dequeue LDPC Decode -- burst */ > > +static uint16_t > > +ark_bb_dequeue_ldpc_dec_ops(struct rte_bbdev_queue_data *q_data, > > + struct rte_bbdev_dec_op **ops, uint16_t > nb_ops) { > > + struct ark_bbdev_queue *q = q_data->queue_private; > > + struct rte_mbuf *mbuf; > > + struct rte_bbdev_dec_op *this_op; > > + struct ark_rx_meta *meta; > > + uint32_t *usermeta; > > + > > + uint16_t nb = 0; > > + int32_t prod_index = q->rx_prod_index; > > + int32_t cons_index = q->rx_cons_index; > > + > > + q->tx_free_index = q->tx_cons_index; > > + > > + while ((prod_index - cons_index) > 0) { > > + if (rte_ring_dequeue(q->active_ops, (void **)&this_op)) { > > + ARK_BBDEV_LOG(ERR, "%s data ready but no op!", > > + __func__); > > + q_data->queue_stats.dequeue_err_count += 1; > > + break; > > + } > > + ops[nb] = this_op; > > + > > + mbuf = this_op->ldpc_dec.hard_output.data; > > + > > + /* META DATA embedded in headroom */ > > + meta = RTE_PTR_ADD(mbuf->buf_addr, > > ARK_RX_META_OFFSET); > > + > > + mbuf->pkt_len = meta->pkt_len; > > + mbuf->data_len = meta->pkt_len; > > + > > + if (unlikely(meta->pkt_len > ARK_RX_MAX_NOCHAIN)) { > > + if (ark_bb_dequeue_segmented(mbuf, &cons_index, > > + meta->pkt_len)) > > + q_data->queue_stats.dequeue_err_count += > > 1; > > + } else if (mbuf->next != 0) { > > + ARK_BBDEV_LOG(CRIT, "Expected mbuf with " > > + "at exactly 1 segments for dequeue " > > + "of packet length %d. Found %d " > > + "segments", > > + meta->pkt_len, mbuf->nb_segs); > > + q_data->queue_stats.dequeue_err_count += 1; > > + } > > + > > + usermeta = meta->user_meta; > > + /* User's meta move from Arkville HW to bbdev OP */ > > + ark_bb_user_dequeue_ldpc_dec(this_op, usermeta); > > + nb++; > > + cons_index++; > > + if (nb >= nb_ops) > > + break; > > + } > > + > > + q->rx_cons_index = cons_index; > > + > > + /* BBdev stats */ > > + q_data->queue_stats.dequeued_count += nb; > > + > > + return nb; > > +} > > + > > +/*************************************************************** > > ******* > > +****/ > > +/* Enqueue LDPC Encode */ > > +static int16_t > > +ark_bb_enqueue_ldpc_enc_one_op(struct ark_bbdev_queue *q, > > + struct rte_bbdev_enc_op *this_op) { > > + struct rte_bbdev_op_ldpc_enc *ldpc_enc_op = &this_op->ldpc_enc; > > + struct rte_mbuf *m_in = ldpc_enc_op->input.data; > > + struct rte_mbuf *m_out = ldpc_enc_op->output.data; > > + uint16_t offset = ldpc_enc_op->input.offset; > > + uint32_t meta[5] = {0}; > > + uint8_t meta_cnt = 0; > > + > > + /* User's meta move from bbdev op to Arkville HW */ > > + if (ark_bb_user_enqueue_ldpc_enc(this_op, meta, &meta_cnt)) { > > + ARK_BBDEV_LOG(ERR, "%s failed", __func__); > > + return 1; > > + } > > + > > + return ark_bb_enqueue_common(q, m_in, m_out, offset, meta, > > meta_cnt); > > +} > > + > > +/* Enqueue LDPC Encode -- burst */ > > +static uint16_t > > +ark_bb_enqueue_ldpc_enc_ops(struct rte_bbdev_queue_data *q_data, > > + struct rte_bbdev_enc_op **ops, uint16_t > nb_ops) { > > + struct ark_bbdev_queue *q = q_data->queue_private; > > + unsigned int max_enq; > > + uint16_t nb; > > + > > + max_enq = rte_ring_free_count(q->active_ops); > > + max_enq = RTE_MIN(max_enq, nb_ops); > > + for (nb = 0; nb < max_enq; nb++) { > > + if (ark_bb_enqueue_ldpc_enc_one_op(q, ops[nb])) > > + break; > > + } > > + > > + ark_bb_enqueue_finalize(q_data, q, (void **)ops, nb_ops, nb); > > + return nb; > > +} > > + > > +/* Dequeue LDPC Encode -- burst */ > > +static uint16_t > > +ark_bb_dequeue_ldpc_enc_ops(struct rte_bbdev_queue_data *q_data, > > + struct rte_bbdev_enc_op **ops, uint16_t > nb_ops) { > > + struct ark_bbdev_queue *q = q_data->queue_private; > > + struct rte_mbuf *mbuf; > > + struct rte_bbdev_enc_op *this_op; > > + struct ark_rx_meta *meta; > > + uint32_t *usermeta; > > + > > + uint16_t nb = 0; > > + int32_t prod_index = q->rx_prod_index; > > + int32_t cons_index = q->rx_cons_index; > > + > > + q->tx_free_index = q->tx_cons_index; > > + > > + while ((prod_index - cons_index) > 0) { > > + if (rte_ring_dequeue(q->active_ops, (void **)&this_op)) { > > + ARK_BBDEV_LOG(ERR, "%s data ready but no op!", > > + __func__); > > + q_data->queue_stats.dequeue_err_count += 1; > > + break; > > + } > > + ops[nb] = this_op; > > + > > + mbuf = this_op->ldpc_enc.output.data; > > + > > + /* META DATA embedded in headroom */ > > + meta = RTE_PTR_ADD(mbuf->buf_addr, > > ARK_RX_META_OFFSET); > > + > > + mbuf->pkt_len = meta->pkt_len; > > + mbuf->data_len = meta->pkt_len; > > + usermeta = meta->user_meta; > > + > > + if (unlikely(meta->pkt_len > ARK_RX_MAX_NOCHAIN)) { > > + if (ark_bb_dequeue_segmented(mbuf, &cons_index, > > + meta->pkt_len)) > > + q_data->queue_stats.dequeue_err_count += > > 1; > > + } else if (mbuf->next != 0) { > > + ARK_BBDEV_LOG(CRIT, "Expected mbuf with " > > + "at exactly 1 segments for dequeue " > > + "of packet length %d. Found %d " > > + "segments", > > + meta->pkt_len, mbuf->nb_segs); > > + q_data->queue_stats.dequeue_err_count += 1; > > + } > > + > > + /* User's meta move from Arkville HW to bbdev OP */ > > + ark_bb_user_dequeue_ldpc_enc(this_op, usermeta); > > + nb++; > > + cons_index++; > > + if (nb >= nb_ops) > > + break; > > + } > > + > > + q->rx_cons_index = cons_index; > > + > > + /* BBdev stats */ > > + q_data->queue_stats.dequeued_count += nb; > > + > > + return nb; > > +} > > + > > +/*************************************************************** > > ******* > > +****/ > > +/* > > + *Initial device hardware configuration when device is opened > > + * setup the DDM, and UDM; called once per PCIE device */ static int > > +ark_bb_config_device(struct ark_bbdevice *ark_bb) { > > + uint16_t num_q, i; > > + struct ark_mpu_t *mpu; > > + > > + /* > > + * Make sure that the packet director, generator and checker are > in a > > + * known state > > + */ > > + ark_bb->start_pg = 0; > > + ark_bb->pg = ark_pktgen_init(ark_bb->pktgen.v, 0, 1); > > + if (ark_bb->pg == NULL) > > + return -1; > > + ark_pktgen_reset(ark_bb->pg); > > + ark_bb->pc = ark_pktchkr_init(ark_bb->pktchkr.v, 0, 1); > > + if (ark_bb->pc == NULL) > > + return -1; > > + ark_pktchkr_stop(ark_bb->pc); > > + ark_bb->pd = ark_pktdir_init(ark_bb->pktdir.v); > > + if (ark_bb->pd == NULL) > > + return -1; > > + > > + /* Verify HW */ > > + if (ark_udm_verify(ark_bb->udm.v)) > > + return -1; > > + if (ark_ddm_verify(ark_bb->ddm.v)) > > + return -1; > > + > > + /* UDM */ > > + if (ark_udm_reset(ark_bb->udm.v)) { > > + ARK_BBDEV_LOG(ERR, "Unable to stop and reset UDM"); > > + return -1; > > + } > > + /* Keep in reset until the MPU are cleared */ > > + > > + /* MPU reset */ > > + mpu = ark_bb->mpurx.v; > > + num_q = ark_api_num_queues(mpu); > > + ark_bb->max_nb_queues = num_q; > > + > > + for (i = 0; i < num_q; i++) { > > + ark_mpu_reset(mpu); > > + mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET); > > + } > > + > > + /* Only 1 queue supported in the udm */ > > + ark_udm_stop(ark_bb->udm.v, 0); > > + ark_udm_configure(ark_bb->udm.v, > > + RTE_PKTMBUF_HEADROOM, > > + bbdev->data->queues[q_id]->dataroom, > > + ARK_RX_WRITE_TIME_NS); > > + > > + > > + ark_udm_stats_reset(ark_bb->udm.v); > > + ark_udm_stop(ark_bb->udm.v, 0); > > + > > + /* TX -- DDM */ > > + if (ark_ddm_stop(ark_bb->ddm.v, 1)) > > + ARK_BBDEV_LOG(ERR, "Unable to stop DDM"); > > + > > + mpu = ark_bb->mputx.v; > > + num_q = ark_api_num_queues(mpu); > > + for (i = 0; i < num_q; i++) { > > + ark_mpu_reset(mpu); > > + mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET); > > + } > > + > > + ark_ddm_reset(ark_bb->ddm.v); > > + ark_ddm_stats_reset(ark_bb->ddm.v); > > + > > + ark_ddm_stop(ark_bb->ddm.v, 0); > > + if (ark_bb->rqpacing) > > + ark_rqp_stats_reset(ark_bb->rqpacing); > > + > > + ARK_BBDEV_LOG(INFO, "packet director set to 0x%x", ark_bb- > > >pkt_dir_v); > > + ark_pktdir_setup(ark_bb->pd, ark_bb->pkt_dir_v); > > + > > + if (ark_bb->pkt_gen_args[0]) { > > + ARK_BBDEV_LOG(INFO, "Setting up the packet generator"); > > + ark_pktgen_parse(ark_bb->pkt_gen_args); > > + ark_pktgen_reset(ark_bb->pg); > > + ark_pktgen_setup(ark_bb->pg); > > + ark_bb->start_pg = 1; > > + } > > + > > + return 0; > > +} > > + > > +static int > > +ark_bbdev_init(struct rte_bbdev *bbdev, struct rte_pci_driver *pci_drv) > > +{ > > + struct ark_bbdevice *ark_bb = bbdev->data->dev_private; > > + struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(bbdev->device); > > + bool rqpacing = false; > > + int p; > > + > > + RTE_SET_USED(pci_drv); > > + > > + ark_bb->bar0 = (uint8_t *)pci_dev->mem_resource[0].addr; > > + ark_bb->a_bar = (uint8_t *)pci_dev->mem_resource[2].addr; > > + > > + ark_bb->sysctrl.v = (void *)&ark_bb->bar0[ARK_SYSCTRL_BASE]; > > + ark_bb->mpurx.v = (void *)&ark_bb->bar0[ARK_MPU_RX_BASE]; > > + ark_bb->udm.v = (void *)&ark_bb->bar0[ARK_UDM_BASE]; > > + ark_bb->mputx.v = (void *)&ark_bb->bar0[ARK_MPU_TX_BASE]; > > + ark_bb->ddm.v = (void *)&ark_bb->bar0[ARK_DDM_BASE]; > > + ark_bb->pktdir.v = (void *)&ark_bb->bar0[ARK_PKTDIR_BASE]; > > + ark_bb->pktgen.v = (void *)&ark_bb->bar0[ARK_PKTGEN_BASE]; > > + ark_bb->pktchkr.v = (void *)&ark_bb->bar0[ARK_PKTCHKR_BASE]; > > + > > + p = 0; > > + while (ark_device_caps[p].device_id != 0) { > > + if (pci_dev->id.device_id == ark_device_caps[p].device_id) > { > > + rqpacing = ark_device_caps[p].caps.rqpacing; > > + break; > > + } > > + p++; > > + } > > + > > + if (rqpacing) > > + ark_bb->rqpacing = > > + (struct ark_rqpace_t *)(ark_bb->bar0 + > > ARK_RCPACING_BASE); > > + else > > + ark_bb->rqpacing = NULL; > > + > > + ark_bb->started = 0; > > + > > + ARK_BBDEV_LOG(INFO, "Sys Ctrl Const = 0x%x HW Commit_ID: > > %08x", > > + ark_bb->sysctrl.t32[4], > > + rte_be_to_cpu_32(ark_bb->sysctrl.t32[0x20 / 4])); > > + ARK_BBDEV_LOG(INFO, "Arkville HW Commit_ID: %08x", > > + rte_be_to_cpu_32(ark_bb->sysctrl.t32[0x20 / 4])); > > + > > + /* If HW sanity test fails, return an error */ > > + if (ark_bb->sysctrl.t32[4] != 0xcafef00d) { > > + ARK_BBDEV_LOG(ERR, > > + "HW Sanity test has failed, expected > constant" > > + " 0x%x, read 0x%x (%s)", > > + 0xcafef00d, > > + ark_bb->sysctrl.t32[4], __func__); > > + return -1; > > + } > > + > > + return ark_bb_config_device(ark_bb); > > +} > > + > > +static int > > +ark_bbdev_uninit(struct rte_bbdev *bbdev) { > > + struct ark_bbdevice *ark_bb = bbdev->data->dev_private; > > + > > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > > + return 0; > > + > > + ark_pktgen_uninit(ark_bb->pg); > > + ark_pktchkr_uninit(ark_bb->pc); > > + > > + return 0; > > +} > > + > > +static int > > +ark_bbdev_probe(struct rte_pci_driver *pci_drv, > > + struct rte_pci_device *pci_dev) > > +{ > > + struct rte_bbdev *bbdev = NULL; > > + char dev_name[RTE_BBDEV_NAME_MAX_LEN]; > > + struct ark_bbdevice *ark_bb; > > + > > + if (pci_dev == NULL) > > + return -EINVAL; > > + > > + rte_pci_device_name(&pci_dev->addr, dev_name, sizeof(dev_name)); > > + > > + /* Allocate memory to be used privately by drivers */ > > + bbdev = rte_bbdev_allocate(pci_dev->device.name); > > + if (bbdev == NULL) > > + return -ENODEV; > > + > > + /* allocate device private memory */ > > + bbdev->data->dev_private = rte_zmalloc_socket(dev_name, > > + sizeof(struct ark_bbdevice), > > + RTE_CACHE_LINE_SIZE, > > + pci_dev->device.numa_node); > > + > > + if (bbdev->data->dev_private == NULL) { > > + ARK_BBDEV_LOG(CRIT, > > + "Allocate of %zu bytes for device \"%s\" > > failed", > > + sizeof(struct ark_bbdevice), dev_name); > > + rte_bbdev_release(bbdev); > > + return -ENOMEM; > > + } > > + ark_bb = bbdev->data->dev_private; > > + /* Initialize ark_bb */ > > + ark_bb->pkt_dir_v = 0x00110110; > > + > > + /* Fill HW specific part of device structure */ > > + bbdev->device = &pci_dev->device; > > + bbdev->intr_handle = NULL; > > + bbdev->data->socket_id = pci_dev->device.numa_node; > > + bbdev->dev_ops = &ark_bbdev_pmd_ops; > > + if (pci_dev->device.devargs) > > + parse_ark_bbdev_params(pci_dev->device.devargs->args, > > ark_bb); > > + > > + > > + /* Device specific initialization */ > > + if (ark_bbdev_init(bbdev, pci_drv)) > > + return -EIO; > > + if (ark_bbdev_start(bbdev)) > > + return -EIO; > > + > > + /* Core operations LDPC encode amd decode */ > > + bbdev->enqueue_ldpc_enc_ops = ark_bb_enqueue_ldpc_enc_ops; > > + bbdev->dequeue_ldpc_enc_ops = ark_bb_dequeue_ldpc_enc_ops; > > + bbdev->enqueue_ldpc_dec_ops = ark_bb_enqueue_ldpc_dec_ops; > > + bbdev->dequeue_ldpc_dec_ops = ark_bb_dequeue_ldpc_dec_ops; > > + > > + ARK_BBDEV_LOG(DEBUG, "bbdev id = %u [%s]", > > + bbdev->data->dev_id, dev_name); > > + > > + return 0; > > +} > > + > > +/* Uninitialize device */ > > +static int > > +ark_bbdev_remove(struct rte_pci_device *pci_dev) { > > + struct rte_bbdev *bbdev; > > + int ret; > > + > > + if (pci_dev == NULL) > > + return -EINVAL; > > + > > + /* Find device */ > > + bbdev = rte_bbdev_get_named_dev(pci_dev->device.name); > > + if (bbdev == NULL) { > > + ARK_BBDEV_LOG(CRIT, > > + "Couldn't find HW dev \"%s\" to > Uninitialize > > it", > > + pci_dev->device.name); > > + return -ENODEV; > > + } > > + > > + /* Arkville device close */ > > + ark_bbdev_uninit(bbdev); > > + rte_free(bbdev->data->dev_private); > > + > > + /* Close device */ > > + ret = rte_bbdev_close(bbdev->data->dev_id); > > + if (ret < 0) > > + ARK_BBDEV_LOG(ERR, > > + "Device %i failed to close during remove: > %i", > > + bbdev->data->dev_id, ret); > > + > > + return rte_bbdev_release(bbdev); > > +} > > + > > +/* Operation for the PMD */ > > +static const struct rte_bbdev_ops ark_bbdev_pmd_ops = { > > + .info_get = ark_bbdev_info_get, > > + .start = ark_bbdev_start, > > + .stop = ark_bbdev_stop, > > + .queue_setup = ark_bb_q_setup, > > + .queue_release = ark_bb_q_release, > > + .queue_start = ark_bb_q_start, > > + .queue_stop = ark_bb_q_stop, > > +}; > > + > > + > > + > > +static struct rte_pci_driver ark_bbdev_pmd_drv = { > > + .probe = ark_bbdev_probe, > > + .remove = ark_bbdev_remove, > > + .id_table = pci_id_ark, > > + .drv_flags = RTE_PCI_DRV_NEED_MAPPING > > +}; > > + > > +RTE_PMD_REGISTER_PCI(DRIVER_NAME, ark_bbdev_pmd_drv); > > +RTE_PMD_REGISTER_PCI_TABLE(DRIVER_NAME, pci_id_ark); > > +RTE_PMD_REGISTER_PARAM_STRING(DRIVER_NAME, > > + ARK_BBDEV_PKTGEN_ARG "= " > > + ARK_BBDEV_PKTCHKR_ARG "= " > > + ARK_BBDEV_PKTDIR_ARG "=" > > + ); > > diff --git a/drivers/baseband/ark/ark_bbdev_common.c > > b/drivers/baseband/ark/ark_bbdev_common.c > > new file mode 100644 > > index 0000000000..6ef0f43654 > > --- /dev/null > > +++ b/drivers/baseband/ark/ark_bbdev_common.c > > @@ -0,0 +1,125 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(c) 2016-2021 Atomic Rules LLC */ > > + > > +#include > > + > > +#include > > +#include > > + > > +#include "ark_bbdev_common.h" > > + > > +static const char * const ark_bbdev_valid_params[] = { > > + ARK_BBDEV_PKTDIR_ARG, > > + ARK_BBDEV_PKTGEN_ARG, > > + ARK_BBDEV_PKTCHKR_ARG, > > + NULL > > +}; > > + > > +/* Parse 16-bit integer from string argument */ static inline int > > +parse_u16_arg(const char *key, const char *value, void *extra_args) { > > + uint16_t *u16 = extra_args; > > + unsigned int long result; > > + > > + if ((value == NULL) || (extra_args == NULL)) > > + return -EINVAL; > > + errno = 0; > > + result = strtoul(value, NULL, 0); > > + if ((result >= (1 << 16)) || (errno != 0)) { > > + ARK_BBDEV_LOG(ERR, "Invalid value %" PRIu64 " for %s", > > result, key); > > + return -ERANGE; > > + } > > + *u16 = (uint16_t)result; > > + return 0; > > +} > > + > > +static inline int > > +process_pktdir_arg(const char *key, const char *value, > > + void *extra_args) > > +{ > > + uint32_t *u32 = extra_args; > > + ARK_BBDEV_LOG(DEBUG, "key = %s, value = %s", key, value); > > + > > + *u32 = strtol(value, NULL, 0); > > + ARK_BBDEV_LOG(DEBUG, "pkt_dir_v = 0x%x", *u32); > > + return 0; > > +} > > + > > +static inline int > > +process_file_args(const char *key, const char *value, void *extra_args) > > +{ > > + char *args = (char *)extra_args; > > + ARK_BBDEV_LOG(DEBUG, "key = %s, value = %s", key, value); > > + > > + /* Open the configuration file */ > > + FILE *file = fopen(value, "r"); > > + char line[ARK_MAX_ARG_LEN]; > > + int size = 0; > > + int first = 1; > > + > > + if (file == NULL) { > > + ARK_BBDEV_LOG(ERR, "Unable to open config file %s", > > + value); > > + return -1; > > + } > > + > > + while (fgets(line, sizeof(line), file)) { > > + size += strlen(line); > > + if (size >= ARK_MAX_ARG_LEN) { > > + ARK_BBDEV_LOG(ERR, "Unable to parse file %s args, > > " > > + "parameter list is too long", value); > > + fclose(file); > > + return -1; > > + } > > + if (first) { > > + strncpy(args, line, ARK_MAX_ARG_LEN); > > + first = 0; > > + } else { > > + strncat(args, line, ARK_MAX_ARG_LEN); > > + } > > + } > > + ARK_BBDEV_LOG(DEBUG, "file = %s", args); > > + fclose(file); > > + return 0; > > +} > > + > > + > > +/* Parse parameters used to create device */ int > > +parse_ark_bbdev_params(const char *input_args, > > + struct ark_bbdevice *ark_bb) > > +{ > > + struct rte_kvargs *kvlist = NULL; > > + int ret = 0; > > + > > + if (ark_bb == NULL) > > + return -EINVAL; > > + if (input_args == NULL) > > + return ret; > > + > > + kvlist = rte_kvargs_parse(input_args, ark_bbdev_valid_params); > > + if (kvlist == NULL) > > + return -EFAULT; > > + > > + ret = rte_kvargs_process(kvlist, ARK_BBDEV_PKTDIR_ARG, > > + &process_pktdir_arg, &ark_bb->pkt_dir_v); > > + if (ret < 0) > > + goto exit; > > + > > + ret = rte_kvargs_process(kvlist, ARK_BBDEV_PKTGEN_ARG, > > + &process_file_args, &ark_bb- > > >pkt_gen_args); > > + if (ret < 0) > > + goto exit; > > + > > + ret = rte_kvargs_process(kvlist, ARK_BBDEV_PKTCHKR_ARG, > > + &process_file_args, &ark_bb- > > >pkt_chkr_args); > > + if (ret < 0) > > + goto exit; > > + > > + exit: > > + if (kvlist) > > + rte_kvargs_free(kvlist); > > + return ret; > > +} > > diff --git a/drivers/baseband/ark/ark_bbdev_common.h > > b/drivers/baseband/ark/ark_bbdev_common.h > > new file mode 100644 > > index 0000000000..670e7e86d6 > > --- /dev/null > > +++ b/drivers/baseband/ark/ark_bbdev_common.h > > @@ -0,0 +1,92 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(c) 2016-2021 Atomic Rules LLC */ > > + > > +#ifndef _ARK_BBDEV_COMMON_H_ > > +#define _ARK_BBDEV_COMMON_H_ > > + > > +#include "ark_pktchkr.h" > > +#include "ark_pktdir.h" > > +#include "ark_pktgen.h" > > + > > +#define ARK_MAX_ARG_LEN 256 > > + > > +/* Acceptable params for ark BBDEV devices */ > > +/* > > + * The packet generator is a functional block used to generate packet > > + * patterns for testing. It is not intended for nominal use. > > + */ > > +#define ARK_BBDEV_PKTGEN_ARG "Pkt_gen" > > + > > +/* > > + * The packet checker is a functional block used to verify packet > > + * patterns for testing. It is not intended for nominal use. > > + */ > > +#define ARK_BBDEV_PKTCHKR_ARG "Pkt_chkr" > > + > > +/* > > + * The packet director is used to select the internal ingress and > > + * egress packets paths during testing. It is not intended for > > + * nominal use. > > + */ > > +#define ARK_BBDEV_PKTDIR_ARG "Pkt_dir" > > + > > + > > +#define def_ptr(type, name) \ > > + union type { \ > > + uint64_t *t64; \ > > + uint32_t *t32; \ > > + uint16_t *t16; \ > > + uint8_t *t8; \ > > + void *v; \ > > + } name > > + > > +/* > > + * Structure to store private data for each PF/VF instance. > > + */ > > +struct ark_bbdevice { > > + /* Our Bar 0 */ > > + uint8_t *bar0; > > + > > + /* Application Bar needed for extensions */ > > + uint8_t *a_bar; > > + > > + /* Arkville hardware block offsets */ > > + def_ptr(sys_ctrl, sysctrl); > > + def_ptr(pkt_gen, pktgen); > > + def_ptr(mpu_rx, mpurx); > > + def_ptr(UDM, udm); > > + def_ptr(mpu_tx, mputx); > > + def_ptr(DDM, ddm); > > + def_ptr(pkt_dir, pktdir); > > + def_ptr(pkt_chkr, pktchkr); > > + struct ark_rqpace_t *rqpacing; > > + > > + /* Pointers to packet generator and checker */ > > + int start_pg; > > + ark_pkt_gen_t pg; > > + ark_pkt_chkr_t pc; > > + ark_pkt_dir_t pd; > > + > > + /* Packet generator/checker args */ > > + char pkt_gen_args[ARK_MAX_ARG_LEN]; > > + char pkt_chkr_args[ARK_MAX_ARG_LEN]; > > + uint32_t pkt_dir_v; > > + > > + int started; > > + unsigned int max_nb_queues; /**< Max number of queues */ > > + > > +}; > > + > > + > > +/* Log message for PMD */ > > +extern int ark_bbdev_logtype; > > + > > +/* Helper macro for logging */ > > +#define ARK_BBDEV_LOG(level, fmt, ...) \ > > + rte_log(RTE_LOG_ ## level, ark_bbdev_logtype, \ > > + "ARK_BBD: " fmt "\n", ##__VA_ARGS__) > > + > > +int parse_ark_bbdev_params(const char *argv, struct ark_bbdevice *dev); > > + > > +#endif > > diff --git a/drivers/baseband/ark/ark_bbdev_custom.c > > b/drivers/baseband/ark/ark_bbdev_custom.c > > new file mode 100644 > > index 0000000000..6b1553abe1 > > --- /dev/null > > +++ b/drivers/baseband/ark/ark_bbdev_custom.c > > @@ -0,0 +1,201 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(c) 2016-2021 Atomic Rules LLC */ > > + > > +#include > > +#include > > + > > +#include > > +#include /* For debug */ > > + > > + > > +#include "ark_bbdev_common.h" > > +#include "ark_bbdev_custom.h" > > + > > +/* It is expected that functions in this file will be modified based on > > + * specifics of the FPGA hardware beyond the core Arkville > > + * components. > > + */ > > + > > +/* bytyes must be range of 0 to 20 */ > > +static inline > > +uint8_t ark_bb_cvt_bytes_meta_cnt(size_t bytes) { > > + return (bytes + 3) / 8; > > +} > > + > > +void > > +ark_bbdev_info_get(struct rte_bbdev *dev, > > + struct rte_bbdev_driver_info *dev_info) { > > + struct ark_bbdevice *ark_bb = dev->data->dev_private; > > + > > + static const struct rte_bbdev_op_cap bbdev_capabilities[] = { > > + { > > + .type = RTE_BBDEV_OP_LDPC_DEC, > > + .cap.ldpc_dec = { > > + .capability_flags = > > + RTE_BBDEV_LDPC_CRC_24B_ATTACH > > | > > + RTE_BBDEV_LDPC_RATE_MATCH, > > + .num_buffers_src = > > + > > RTE_BBDEV_LDPC_MAX_CODE_BLOCKS, > > + .num_buffers_hard_out = > > + > > RTE_BBDEV_LDPC_MAX_CODE_BLOCKS > > + } > > + }, > > + { > > + .type = RTE_BBDEV_OP_LDPC_ENC, > > + .cap.ldpc_enc = { > > + .capability_flags = > > + RTE_BBDEV_LDPC_CRC_24B_ATTACH > > | > > + RTE_BBDEV_LDPC_RATE_MATCH, > > + .num_buffers_src = > > + > > RTE_BBDEV_LDPC_MAX_CODE_BLOCKS, > > + .num_buffers_dst = > > + > > RTE_BBDEV_LDPC_MAX_CODE_BLOCKS > > + } > > + }, > > + RTE_BBDEV_END_OF_CAPABILITIES_LIST(), > > + }; > > + > > + static struct rte_bbdev_queue_conf default_queue_conf = { > > + .queue_size = RTE_BBDEV_QUEUE_SIZE_LIMIT, > > + }; > > + > > + default_queue_conf.socket = dev->data->socket_id; > > + > > + dev_info->driver_name = RTE_STR(DRIVER_NAME); > > + dev_info->max_num_queues = ark_bb->max_nb_queues; > > + dev_info->queue_size_lim = RTE_BBDEV_QUEUE_SIZE_LIMIT; > > + dev_info->hardware_accelerated = true; > > + dev_info->max_dl_queue_priority = 0; > > + dev_info->max_ul_queue_priority = 0; > > + dev_info->default_queue_conf = default_queue_conf; > > + dev_info->capabilities = bbdev_capabilities; > > + dev_info->cpu_flag_reqs = NULL; > > + dev_info->min_alignment = 4; > > + > > +} > > + > > +/* Structure defining layout of the ldpc command struct */ struct > > +ark_bb_ldpc_enc_meta { > > + uint16_t header; > > + uint8_t rv_index:2, > > + basegraph:1, > > + code_block_mode:1, > > + rfu_71_68:4; > > + > > + uint8_t q_m; > > + uint32_t e_ea; > > + uint32_t eb; > > + uint8_t c; > > + uint8_t cab; > > + uint16_t n_cb; > > + uint16_t pad; > > + uint16_t trailer; > > +} __rte_packed; > > + > > +/* The size must be less then 20 Bytes */ static_assert(sizeof(struct > > +ark_bb_ldpc_enc_meta) <= 20, "struct size"); > > + > > +/* Custom operation on equeue ldpc operation */ > > +/* Do these function need queue number? */ > > +/* Maximum of 20 bytes */ > > +int > > +ark_bb_user_enqueue_ldpc_enc(struct rte_bbdev_enc_op *enc_op, > > + uint32_t *meta, uint8_t *meta_cnt) { > > + struct rte_bbdev_op_ldpc_enc *ldpc_enc_op = &enc_op->ldpc_enc; > > + struct ark_bb_ldpc_enc_meta *src = (struct ark_bb_ldpc_enc_meta > > +*)meta; > > + > > + src->header = 0x4321; /* For testings */ > > + src->trailer = 0xFEDC; > > + > > + src->rv_index = ldpc_enc_op->rv_index; > > + src->basegraph = ldpc_enc_op->basegraph; > > + src->code_block_mode = ldpc_enc_op->code_block_mode; > > + > > + src->q_m = ldpc_enc_op->q_m; > > + src->e_ea = 0xABCD; > > + src->eb = ldpc_enc_op->tb_params.eb; > > + src->c = ldpc_enc_op->tb_params.c; > > + src->cab = ldpc_enc_op->tb_params.cab; > > + > > + src->n_cb = 0; > > + > > + meta[0] = 0x11111110; > > + meta[1] = 0x22222220; > > + meta[2] = 0x33333330; > > + meta[3] = 0x44444440; > > + meta[4] = 0x55555550; > > + > > + *meta_cnt = ark_bb_cvt_bytes_meta_cnt( > > + sizeof(struct ark_bb_ldpc_enc_meta)); > > + return 0; > > +} > > + > > +/* Custom operation on dequeue ldpc operation */ int > > +ark_bb_user_dequeue_ldpc_enc(struct rte_bbdev_enc_op *enc_op, > > + const uint32_t *usermeta) > > +{ > > + static int dump; /* = 0 */ > > + /* Just compare with what was sent? */ > > + uint32_t meta_in[5] = {0}; > > + uint8_t meta_cnt; > > + > > + ark_bb_user_enqueue_ldpc_enc(enc_op, meta_in, &meta_cnt); > > + if (memcmp(usermeta, meta_in, 3 + (meta_cnt * 8))) { > > + fprintf(stderr, > > + "------------------------------------------\n"); > > + rte_hexdump(stdout, "meta difference for lpdc_enc IN", > > + meta_in, 20); > > + rte_hexdump(stdout, "meta difference for lpdc_enc OUT", > > + usermeta, 20); > > + } else if (dump) { > > + rte_hexdump(stdout, "DUMP lpdc_enc IN", usermeta, 20); > > + dump--; > > + } > > + > > + return 0; > > +} > > + > > + > > +/* Turbo op call backs for user meta data */ int > > +ark_bb_user_enqueue_ldpc_dec(struct rte_bbdev_dec_op *enc_op, > > + uint32_t *meta, uint8_t *meta_cnt) { > > + RTE_SET_USED(enc_op); > > + meta[0] = 0xF1111110; > > + meta[1] = 0xF2222220; > > + meta[2] = 0xF3333330; > > + meta[3] = 0xF4444440; > > + meta[4] = 0xF5555550; > > + > > + *meta_cnt = ark_bb_cvt_bytes_meta_cnt(20); > > + return 0; > > +} > > + > > +int ark_bb_user_dequeue_ldpc_dec(struct rte_bbdev_dec_op *enc_op, > > + const uint32_t *usermeta) > > +{ > > + RTE_SET_USED(enc_op); > > + static int dump; /* = 0 */ > > + /* Just compare with what was sent? */ > > + uint32_t meta_in[5] = {0}; > > + uint8_t meta_cnt; > > + > > + ark_bb_user_enqueue_ldpc_dec(enc_op, meta_in, &meta_cnt); > > + if (memcmp(usermeta, meta_in, 3 + (meta_cnt * 8))) { > > + fprintf(stderr, > > + "------------------------------------------\n"); > > + rte_hexdump(stdout, "meta difference for lpdc_enc IN", > > + meta_in, 20); > > + rte_hexdump(stdout, "meta difference for lpdc_enc OUT", > > + usermeta, 20); > > + } else if (dump) { > > + rte_hexdump(stdout, "DUMP lpdc_enc IN", usermeta, 20); > > + dump--; > > + } > > + return 0; > > +} > > diff --git a/drivers/baseband/ark/ark_bbdev_custom.h > > b/drivers/baseband/ark/ark_bbdev_custom.h > > new file mode 100644 > > index 0000000000..32a2ef6bb6 > > --- /dev/null > > +++ b/drivers/baseband/ark/ark_bbdev_custom.h > > @@ -0,0 +1,30 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(c) 2016-2021 Atomic Rules LLC */ > > + > > +#ifndef _ARK_BBDEV_CUSTOM_H_ > > +#define _ARK_BBDEV_CUSTOM_H_ > > + > > +#include > > + > > +/* Forward declarations */ > > +struct rte_bbdev; > > +struct rte_bbdev_driver_info; > > +struct rte_bbdev_enc_op; > > +struct rte_bbdev_dec_op; > > +struct rte_mbuf; > > + > > +void ark_bbdev_info_get(struct rte_bbdev *dev, > > + struct rte_bbdev_driver_info *dev_info); > > + > > +int ark_bb_user_enqueue_ldpc_dec(struct rte_bbdev_dec_op *enc_op, > > + uint32_t *meta, uint8_t *meta_cnt); int > > +ark_bb_user_dequeue_ldpc_dec(struct rte_bbdev_dec_op *enc_op, > > + const uint32_t *usermeta); > > + > > +int ark_bb_user_enqueue_ldpc_enc(struct rte_bbdev_enc_op *enc_op, > > + uint32_t *meta, uint8_t *meta_cnt); int > > +ark_bb_user_dequeue_ldpc_enc(struct rte_bbdev_enc_op *enc_op, > > + const uint32_t *usermeta); > > + > > +#endif > > diff --git a/drivers/baseband/ark/meson.build > > b/drivers/baseband/ark/meson.build > > new file mode 100644 > > index 0000000000..b876f05c6e > > --- /dev/null > > +++ b/drivers/baseband/ark/meson.build > > @@ -0,0 +1,11 @@ > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca > > +Boccassi > > + > > +deps += ['common_ark', 'bbdev', 'bus_pci', 'pci', 'ring'] sources = > > +files( > > + 'ark_bbdev.c', > > + 'ark_bbdev_common.c', > > + 'ark_bbdev_custom.c' > > + ) > > + > > +includes += include_directories('../../common/ark') > > diff --git a/drivers/baseband/ark/version.map > > b/drivers/baseband/ark/version.map > > new file mode 100644 > > index 0000000000..4a76d1d52d > > --- /dev/null > > +++ b/drivers/baseband/ark/version.map > > @@ -0,0 +1,3 @@ > > +DPDK_21 { > > + local: *; > > +}; > > -- > > 2.25.1 > > --000000000000837e7f05ddb40304 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Nic,

Yes, i will split this into sma= ller patches and take care of the TODO's.=C2=A0 I will have a v1 patch = set ready with these changes within a couple of days.=C2=A0 Given the 4/10 = deadline for 22.07 it would seem that 22.11 is our target.

Thank you,
-John


On Wed, Apr 27, 202= 2 at 2:38 PM Chautru, Nicolas <nicolas.chautru@intel.com> wrote:
Hi John,

Do you think this one can be split into a few incremental commits?

There are a few TODOs, is that v1 ready for review? Also you are targeting = 22.11 right?

Thanks
Nic

> -----Original Message-----
> From: John Miller <john.miller@atomicrules.com>
> Sent: Thursday, April 21, 2022 8:19 AM
> To: dev@dpdk.org=
> Cc: ferru= h.yigit@xilinx.com; ed.czeck@atomicrules.com; John Miller
> <j= ohn.miller@atomicrules.com>
> Subject: [PATCH 05/10] baseband/ark: add ark baseband device
>
> Add new ark baseband device.
>
> Signed-off-by: John Miller <john.miller@atomicrules.com>
> ---
>=C2=A0 drivers/baseband/ark/ark_bbdev.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 | 10= 64 +++++++++++++++++++++++
>=C2=A0 drivers/baseband/ark/ark_bbdev_common.c |=C2=A0 125 +++
>=C2=A0 drivers/baseband/ark/ark_bbdev_common.h |=C2=A0 =C2=A092 ++
>=C2=A0 drivers/baseband/ark/ark_bbdev_custom.c |=C2=A0 201 +++++
>=C2=A0 drivers/baseband/ark/ark_bbdev_custom.h |=C2=A0 =C2=A030 +
>=C2=A0 drivers/baseband/ark/meson.build=C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2= =A0 =C2=A011 +
>=C2=A0 drivers/baseband/ark/version.map=C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2= =A0 =C2=A0 3 +
>=C2=A0 7 files changed, 1526 insertions(+)
>=C2=A0 create mode 100644 drivers/baseband/ark/ark_bbdev.c=C2=A0 create= mode
> 100644 drivers/baseband/ark/ark_bbdev_common.c
>=C2=A0 create mode 100644 drivers/baseband/ark/ark_bbdev_common.h
>=C2=A0 create mode 100644 drivers/baseband/ark/ark_bbdev_custom.c
>=C2=A0 create mode 100644 drivers/baseband/ark/ark_bbdev_custom.h
>=C2=A0 create mode 100644 drivers/baseband/ark/meson.build=C2=A0 create= mode
> 100644 drivers/baseband/ark/version.map
>
> diff --git a/drivers/baseband/ark/ark_bbdev.c
> b/drivers/baseband/ark/ark_bbdev.c
> new file mode 100644
> index 0000000000..b23bbd44d1
> --- /dev/null
> +++ b/drivers/baseband/ark/ark_bbdev.c
> @@ -0,0 +1,1064 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2016-2021 Atomic Rules LLC=C2=A0 */
> +
> +#include "ark_common.h"
> +#include "ark_bbdev_common.h"
> +#include "ark_bbdev_custom.h"
> +#include "ark_ddm.h"
> +#include "ark_mpu.h"
> +#include "ark_rqp.h"
> +#include "ark_udm.h"
> +
> +#include <rte_bbdev.h>
> +#include <rte_bbdev_pmd.h>
> +#include <rte_bus_pci.h>
> +#include <rte_common.h>
> +#include <rte_devargs.h>
> +#include <rte_malloc.h>
> +#include <rte_ring.h>
> +
> +#include <unistd.h>
> +
> +#define DRIVER_NAME baseband_ark
> +
> +RTE_LOG_REGISTER_DEFAULT(ark_bbdev_logtype, DEBUG);
> +
> +#define ARK_SYSCTRL_BASE=C2=A0 0x0
> +#define ARK_PKTGEN_BASE=C2=A0 =C2=A00x10000
> +#define ARK_MPU_RX_BASE=C2=A0 =C2=A00x20000
> +#define ARK_UDM_BASE=C2=A0 =C2=A0 =C2=A0 0x30000
> +#define ARK_MPU_TX_BASE=C2=A0 =C2=A00x40000
> +#define ARK_DDM_BASE=C2=A0 =C2=A0 =C2=A0 0x60000
> +#define ARK_PKTDIR_BASE=C2=A0 =C2=A00xa0000
> +#define ARK_PKTCHKR_BASE=C2=A0 0x90000
> +#define ARK_RCPACING_BASE 0xb0000
> +#define ARK_MPU_QOFFSET=C2=A0 =C2=A00x00100
> +
> +#define BB_ARK_TX_Q_FACTOR 4
> +
> +/* TODO move to UDM, verify configuration */ #define ARK_RX_META_SIZE=
> +32 #define ARK_RX_META_OFFSET (RTE_PKTMBUF_HEADROOM -
> ARK_RX_META_SIZE)
> +#define ARK_RX_MAX_NOCHAIN (RTE_MBUF_DEFAULT_DATAROOM)
> +
> +static_assert(sizeof(struct ark_rx_meta) =3D=3D ARK_RX_META_SIZE,
> +"Unexpected struct size ark_rx_meta"); static_assert(sizeof= (union
> +ark_tx_meta) =3D=3D 8, "Unexpected struct size ark_tx_meta"= );
> +
> +static struct rte_pci_id pci_id_ark[] =3D {
> +=C2=A0 =C2=A0 =C2=A0{RTE_PCI_DEVICE(AR_VENDOR_ID, 0x1015)},
> +=C2=A0 =C2=A0 =C2=A0{RTE_PCI_DEVICE(AR_VENDOR_ID, 0x1016)},
> +=C2=A0 =C2=A0 =C2=A0{.device_id =3D 0},
> +};
> +
> +static const struct ark_dev_caps
> +ark_device_caps[] =3D {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 SET_DE= V_CAPS(0x1015, true),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 SET_DE= V_CAPS(0x1016, true),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 {.devi= ce_id =3D 0,}
> +};
> +
> +
> +/* Forward declarations */
> +static const struct rte_bbdev_ops ark_bbdev_pmd_ops;
> +
> +
> +/* queue */
> +struct ark_bbdev_queue {
> +=C2=A0 =C2=A0 =C2=A0struct rte_ring *active_ops;=C2=A0 /* Ring for pr= ocessed packets */
> +
> +=C2=A0 =C2=A0 =C2=A0/* RX components */
> +=C2=A0 =C2=A0 =C2=A0/* array of physical addresses of the mbuf data p= ointer */
> +=C2=A0 =C2=A0 =C2=A0rte_iova_t *rx_paddress_q;
> +=C2=A0 =C2=A0 =C2=A0struct ark_udm_t *udm;
> +=C2=A0 =C2=A0 =C2=A0struct ark_mpu_t *rx_mpu;
> +
> +=C2=A0 =C2=A0 =C2=A0/* TX components */
> +=C2=A0 =C2=A0 =C2=A0union ark_tx_meta *tx_meta_q;
> +=C2=A0 =C2=A0 =C2=A0struct ark_mpu_t *tx_mpu;
> +=C2=A0 =C2=A0 =C2=A0struct ark_ddm_t *ddm;
> +
> +=C2=A0 =C2=A0 =C2=A0/*=C2=A0 */
> +=C2=A0 =C2=A0 =C2=A0uint32_t tx_queue_mask;
> +=C2=A0 =C2=A0 =C2=A0uint32_t rx_queue_mask;
> +
> +=C2=A0 =C2=A0 =C2=A0int32_t rx_seed_index;=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 /* step 1 set with empty mbuf */
> +=C2=A0 =C2=A0 =C2=A0int32_t rx_cons_index;=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 /* step 3 consumed by driver */
> +
> +=C2=A0 =C2=A0 =C2=A0/* 3 indexes to the paired data rings. */
> +=C2=A0 =C2=A0 =C2=A0int32_t tx_prod_index;=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 /* where to put the next one */
> +=C2=A0 =C2=A0 =C2=A0int32_t tx_free_index;=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 /* local copy of tx_cons_index */
> +
> +=C2=A0 =C2=A0 =C2=A0/* separate cache line -- written by FPGA -- RX a= nnounce */
> +=C2=A0 =C2=A0 =C2=A0RTE_MARKER cacheline1 __rte_cache_min_aligned; > +=C2=A0 =C2=A0 =C2=A0volatile int32_t rx_prod_index; /* step 2 filled = by FPGA */
> +
> +=C2=A0 =C2=A0 =C2=A0/* Separate cache line -- written by FPGA -- RX c= ompletion */
> +=C2=A0 =C2=A0 =C2=A0RTE_MARKER cacheline2 __rte_cache_min_aligned; > +=C2=A0 =C2=A0 =C2=A0volatile int32_t tx_cons_index; /* hw is done, ca= n be freed */ }
> +__rte_cache_aligned;
> +
> +static int
> +ark_bb_hw_q_setup(struct rte_bbdev *bbdev, uint16_t q_id, uint16_t > +queue_size) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdev_queue *q =3D bbdev->data-
> >queues[q_id].queue_private;
> +
> +=C2=A0 =C2=A0 =C2=A0rte_iova_t queue_base;
> +=C2=A0 =C2=A0 =C2=A0rte_iova_t phys_addr_q_base;
> +=C2=A0 =C2=A0 =C2=A0rte_iova_t phys_addr_prod_index;
> +=C2=A0 =C2=A0 =C2=A0rte_iova_t phys_addr_cons_index;
> +
> +=C2=A0 =C2=A0 =C2=A0uint32_t write_interval_ns =3D 500; /* TODO this = seems big */
> +
> +=C2=A0 =C2=A0 =C2=A0if (ark_mpu_verify(q->rx_mpu, sizeof(rte_iova_= t))) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;Illegal hw/sw configuration RX
> queue");
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "ark_bb_q setup %u:%u&q= uot;,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= bbdev->data->dev_id, q_id);
> +
> +=C2=A0 =C2=A0 =C2=A0/* RX MPU */
> +=C2=A0 =C2=A0 =C2=A0phys_addr_q_base =3D rte_malloc_virt2iova(q->r= x_paddress_q);
> +=C2=A0 =C2=A0 =C2=A0/* Force TX mode on MPU to match bbdev behavior *= /
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_configure(q->rx_mpu, phys_addr_q_base,= queue_size, 1);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_reset_stats(q->rx_mpu);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_start(q->rx_mpu);
> +
> +=C2=A0 =C2=A0 =C2=A0/* UDM */
> +=C2=A0 =C2=A0 =C2=A0queue_base =3D rte_malloc_virt2iova(q);
> +=C2=A0 =C2=A0 =C2=A0phys_addr_prod_index =3D queue_base +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0offsetof(struct ark_b= bdev_queue, rx_prod_index);
> +=C2=A0 =C2=A0 =C2=A0ark_udm_write_addr(q->udm, phys_addr_prod_inde= x);
> +=C2=A0 =C2=A0 =C2=A0ark_udm_queue_enable(q->udm, 1);
> +
> +=C2=A0 =C2=A0 =C2=A0/* TX MPU */
> +=C2=A0 =C2=A0 =C2=A0phys_addr_q_base =3D rte_malloc_virt2iova(q->t= x_meta_q);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_configure(q->tx_mpu, phys_addr_q_base,=
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0BB_ARK_TX_Q_FACTOR * queue_size, 1);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_start(q->tx_mpu);
> +
> +=C2=A0 =C2=A0 =C2=A0/* DDM */
> +=C2=A0 =C2=A0 =C2=A0phys_addr_cons_index =3D queue_base +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0offsetof(struct ark_b= bdev_queue, tx_cons_index);
> +=C2=A0 =C2=A0 =C2=A0ark_ddm_setup(q->ddm, phys_addr_cons_index, wr= ite_interval_ns);
> +
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +/* Setup a queue */
> +static int
> +ark_bb_q_setup(struct rte_bbdev *bbdev, uint16_t q_id,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 const struct rte_bbdev_queu= e_conf *queue_conf) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdev_queue *q;
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdevice *ark_bb =3D=C2=A0 bbdev->d= ata->dev_private;
> +
> +=C2=A0 =C2=A0 =C2=A0const uint32_t queue_size =3D queue_conf->queu= e_size;
> +=C2=A0 =C2=A0 =C2=A0const int socket_id =3D queue_conf->socket; > +=C2=A0 =C2=A0 =C2=A0const uint64_t pg_sz =3D sysconf(_SC_PAGESIZE); > +=C2=A0 =C2=A0 =C2=A0char ring_name[RTE_RING_NAMESIZE];
> +
> +=C2=A0 =C2=A0 =C2=A0/* Configuration checks */
> +=C2=A0 =C2=A0 =C2=A0if (!rte_is_power_of_2(queue_size)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0"Configuration queue size"
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0" must be power of two %u",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0queue_size);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EINVAL;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0if (RTE_PKTMBUF_HEADROOM < ARK_RX_META_SIZE) {=
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0"Error: Ark bbdev requires head room > = %d bytes
> (%s)",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_RX_META_SIZE, __func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EINVAL;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0/* Allocate the queue data structure. */
> +=C2=A0 =C2=A0 =C2=A0q =3D rte_zmalloc_socket(RTE_STR(DRIVER_NAME), si= zeof(*q),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0RTE_CACHE_LINE_SIZE, queue_conf->socket);
> +=C2=A0 =C2=A0 =C2=A0if (q =3D=3D NULL) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;Failed to allocate queue memory");
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -ENOMEM;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0bbdev->data->queues[q_id].queue_private =3D= q;
> +
> +=C2=A0 =C2=A0 =C2=A0/* RING */
> +=C2=A0 =C2=A0 =C2=A0snprintf(ring_name, RTE_RING_NAMESIZE, RTE_STR(DR= IVER_NAME)
> "%u:%u",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 bbdev->data->d= ev_id, q_id);
> +=C2=A0 =C2=A0 =C2=A0q->active_ops =3D rte_ring_create(ring_name, > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0queue_size,<= br> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0queue_conf-&= gt;socket,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0RING_F_SP_EN= Q | RING_F_SC_DEQ);
> +=C2=A0 =C2=A0 =C2=A0if (q->active_ops =3D=3D NULL) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;Failed to create ring");
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto free_all;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0q->rx_queue_mask =3D queue_size - 1;
> +=C2=A0 =C2=A0 =C2=A0q->tx_queue_mask =3D (BB_ARK_TX_Q_FACTOR * que= ue_size) - 1;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Each mbuf requires 2 to 4 objects, factor by > BB_ARK_TX_Q_FACTOR */
> +=C2=A0 =C2=A0 =C2=A0q->tx_meta_q =3D
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_zmalloc_socket(&q= uot;Ark_bb_txqueue meta",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 queue_size * BB_ARK_TX_Q_FACTOR = *
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sizeof(union ark_tx_meta),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pg_sz,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 socket_id);
> +
> +=C2=A0 =C2=A0 =C2=A0if (q->tx_meta_q =3D=3D 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;Failed to allocate "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0"queue memory in %s", __func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto free_all;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0q->ddm =3D RTE_PTR_ADD(ark_bb->ddm.v, q_id = *
> ARK_DDM_QOFFSET);
> +=C2=A0 =C2=A0 =C2=A0q->tx_mpu =3D RTE_PTR_ADD(ark_bb->mputx.v, = q_id *
> ARK_MPU_QOFFSET);
> +
> +=C2=A0 =C2=A0 =C2=A0q->rx_paddress_q =3D
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_zmalloc_socket(&q= uot;ark_bb_rx_paddress_q",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 queue_size * sizeof(rte_iova_t),=
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pg_sz,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 socket_id);
> +
> +=C2=A0 =C2=A0 =C2=A0if (q->rx_paddress_q =3D=3D 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0"Failed to allocate queue memory in %s&quo= t;,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0__func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto free_all;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0q->udm =3D RTE_PTR_ADD(ark_bb->udm.v, q_id = *
> ARK_UDM_QOFFSET);
> +=C2=A0 =C2=A0 =C2=A0q->rx_mpu =3D RTE_PTR_ADD(ark_bb->mpurx.v, = q_id *
> ARK_MPU_QOFFSET);
> +
> +=C2=A0 =C2=A0 =C2=A0/* Structure have been configured, set the hardwa= re */
> +=C2=A0 =C2=A0 =C2=A0return ark_bb_hw_q_setup(bbdev, q_id, queue_size)= ;
> +
> +free_all:
> +=C2=A0 =C2=A0 =C2=A0rte_free(q->tx_meta_q);
> +=C2=A0 =C2=A0 =C2=A0rte_free(q->rx_paddress_q);
> +=C2=A0 =C2=A0 =C2=A0rte_free(q);
> +=C2=A0 =C2=A0 =C2=A0return -EFAULT;
> +}
> +
> +/* Release queue */
> +static int
> +ark_bb_q_release(struct rte_bbdev *bbdev, uint16_t q_id) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdev_queue *q =3D bbdev->data-
> >queues[q_id].queue_private;
> +
> +=C2=A0 =C2=A0 =C2=A0/* TODO Wait for ddm to send out all packets in f= light,
> +=C2=A0 =C2=A0 =C2=A0 * Is this only called after q stop?
> +=C2=A0 =C2=A0 =C2=A0 */
> +
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_dump(q->rx_mpu, "rx_MPU release&q= uot;, q_id);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_dump(q->tx_mpu, "tx_MPU release&q= uot;, q_id);
> +
> +=C2=A0 =C2=A0 =C2=A0rte_ring_free(q->active_ops);
> +=C2=A0 =C2=A0 =C2=A0rte_free(q->tx_meta_q);
> +=C2=A0 =C2=A0 =C2=A0rte_free(q->rx_paddress_q);
> +=C2=A0 =C2=A0 =C2=A0rte_free(q);
> +=C2=A0 =C2=A0 =C2=A0bbdev->data->queues[q_id].queue_private =3D= NULL;
> +
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "released device queue = %u:%u",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= bbdev->data->dev_id, q_id);
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +static int
> +ark_bbdev_start(struct rte_bbdev *bbdev) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdevice *ark_bb =3D bbdev->data-&g= t;dev_private;
> +
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "Starting device %u&quo= t;, bbdev->data-
> >dev_id);
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb->started)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 0;
> +
> +=C2=A0 =C2=A0 =C2=A0/* start UDM */
> +=C2=A0 =C2=A0 =C2=A0ark_udm_start(ark_bb->udm.v);
> +
> +=C2=A0 =C2=A0 =C2=A0/* start DDM */
> +=C2=A0 =C2=A0 =C2=A0ark_ddm_start(ark_bb->ddm.v);
> +
> +=C2=A0 =C2=A0 =C2=A0ark_bb->started =3D 1;
> +
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb->start_pg)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_pktchkr_run(ark_b= b->pc);
> +
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb->start_pg) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pthread_t thread;
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* Delay packet gener= ator start allow the hardware to be
> ready
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 * This is only used = for sanity checking with internal generator
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (pthread_create(&a= mp;thread, NULL,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ark_pktgen_delay_start, ark_bb-&= gt;pg)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0ARK_BBDEV_LOG(ERR, "Could not create pktgen "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"starter thread"= );
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0return -1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +static void
> +ark_bbdev_stop(struct rte_bbdev *bbdev) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdevice *ark_bb =3D bbdev->data-&g= t;dev_private;
> +=C2=A0 =C2=A0 =C2=A0struct ark_mpu_t *mpu;
> +=C2=A0 =C2=A0 =C2=A0unsigned int i;
> +=C2=A0 =C2=A0 =C2=A0int status;
> +
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "Stopping device %u&quo= t;, bbdev->data-
> >dev_id);
> +
> +=C2=A0 =C2=A0 =C2=A0if (!ark_bb->started)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Stop the packet generator */
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb->start_pg)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_pktgen_pause(ark_= bb->pg);
> +
> +=C2=A0 =C2=A0 =C2=A0/* Stop DDM */
> +=C2=A0 =C2=A0 =C2=A0/* Wait up to 0.1 second.=C2=A0 each stop is up t= o 1000 * 10 useconds */
> +=C2=A0 =C2=A0 =C2=A0for (i =3D 0; i < 10; i++) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0status =3D ark_ddm_st= op(ark_bb->ddm.v, 1);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (status =3D=3D 0)<= br> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0break;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0if (status || i !=3D 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;DDM stop anomaly. status:"
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0" %d iter: %u. (%s)",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0status,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0i,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0__func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_ddm_dump(ark_bb-&= gt;ddm.v, "Stop anomaly");
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mpu =3D ark_bb->mp= utx.v;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for (i =3D 0; i < = ark_bb->max_nb_queues; i++) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0ark_mpu_dump(mpu, "DDM failure dump", i);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0mpu =3D RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0ark_ddm_dump_stats(ark_bb->ddm.v, "bbdev = stop");
> +
> +=C2=A0 =C2=A0 =C2=A0/* STOP RX Side */
> +=C2=A0 =C2=A0 =C2=A0/* Stop UDM=C2=A0 multiple tries attempted */
> +=C2=A0 =C2=A0 =C2=A0for (i =3D 0; i < 10; i++) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0status =3D ark_udm_st= op(ark_bb->udm.v, 1);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (status =3D=3D 0)<= br> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0break;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0if (status || i !=3D 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(WARNING= , "UDM stop anomaly. status %d
> iter: %u. (%s)",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0status, i, __func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_udm_dump(ark_bb-&= gt;udm.v, "Stop anomaly");
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mpu =3D ark_bb->mp= urx.v;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for (i =3D 0; i < = ark_bb->max_nb_queues; i++) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0ark_mpu_dump(mpu, "UDM Stop anomaly", i);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0mpu =3D RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0ark_udm_dump_stats(ark_bb->udm.v, "Post s= top");
> +=C2=A0 =C2=A0 =C2=A0ark_udm_dump_perf(ark_bb->udm.v, "Post st= op");
> +
> +=C2=A0 =C2=A0 =C2=A0/* Stop the packet checker if it is running */ > +=C2=A0 =C2=A0 =C2=A0if (ark_bb->start_pg) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_pktchkr_dump_stat= s(ark_bb->pc);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_pktchkr_stop(ark_= bb->pc);
> +=C2=A0 =C2=A0 =C2=A0}
> +}
> +
> +static int
> +ark_bb_q_start(struct rte_bbdev *bbdev, uint16_t q_id) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdev_queue *q =3D bbdev->data-
> >queues[q_id].queue_private;
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "ark_bb_q start %u:%u&q= uot;, bbdev->data-
> >dev_id, q_id);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_start(q->tx_mpu);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_start(q->rx_mpu);
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +static int
> +ark_bb_q_stop(struct rte_bbdev *bbdev, uint16_t q_id) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdev_queue *q =3D bbdev->data-
> >queues[q_id].queue_private;
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "ark_bb_q stop %u:%u&qu= ot;, bbdev->data-
> >dev_id, q_id);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_stop(q->tx_mpu);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_stop(q->rx_mpu);
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +/*
> +****************************************************************
> *******
> +** */
> +/* Common function for all enqueue and dequeue ops */ static inline > +void ark_bb_enqueue_desc_fill(struct ark_bbdev_queue *q,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 struct rte_mbuf *mbuf,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 uint16_t offset, /* Extra offset */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 uint8_t=C2=A0 flags,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 uint32_t *meta,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 uint8_t=C2=A0 meta_cnt /* 0, 1 or 2 */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 )
> +{
> +=C2=A0 =C2=A0 =C2=A0union ark_tx_meta *tx_meta;
> +=C2=A0 =C2=A0 =C2=A0int32_t tx_idx;
> +=C2=A0 =C2=A0 =C2=A0uint8_t m;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Header */
> +=C2=A0 =C2=A0 =C2=A0tx_idx =3D q->tx_prod_index & q->tx_que= ue_mask;
> +=C2=A0 =C2=A0 =C2=A0tx_meta =3D &q->tx_meta_q[tx_idx];
> +=C2=A0 =C2=A0 =C2=A0tx_meta->data_len =3D rte_pktmbuf_data_len(mbu= f) - offset;
> +=C2=A0 =C2=A0 =C2=A0tx_meta->flags =3D flags;
> +=C2=A0 =C2=A0 =C2=A0tx_meta->meta_cnt =3D meta_cnt;
> +=C2=A0 =C2=A0 =C2=A0tx_meta->user1 =3D *meta++;
> +=C2=A0 =C2=A0 =C2=A0q->tx_prod_index++;
> +
> +=C2=A0 =C2=A0 =C2=A0for (m =3D 0; m < meta_cnt; m++) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tx_idx =3D q->tx_p= rod_index & q->tx_queue_mask;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tx_meta =3D &q-&g= t;tx_meta_q[tx_idx];
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tx_meta->usermeta0= =3D *meta++;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tx_meta->usermeta1= =3D *meta++;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0q->tx_prod_index++= ;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0tx_idx =3D q->tx_prod_index & q->tx_que= ue_mask;
> +=C2=A0 =C2=A0 =C2=A0tx_meta =3D &q->tx_meta_q[tx_idx];
> +=C2=A0 =C2=A0 =C2=A0tx_meta->physaddr =3D rte_mbuf_data_iova(mbuf)= + offset;
> +=C2=A0 =C2=A0 =C2=A0q->tx_prod_index++;
> +}
> +
> +static inline void
> +ark_bb_enqueue_segmented_pkt(struct ark_bbdev_queue *q,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 struct rte_mbuf *mbuf,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 uint16_t offset,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 uint32_t *meta, uint8_t meta_cnt) {
> +=C2=A0 =C2=A0 =C2=A0struct rte_mbuf *next;
> +=C2=A0 =C2=A0 =C2=A0uint8_t flags =3D ARK_DDM_SOP;
> +
> +=C2=A0 =C2=A0 =C2=A0while (mbuf !=3D NULL) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0next =3D mbuf->nex= t;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0flags |=3D (next =3D= =3D NULL) ? ARK_DDM_EOP : 0;
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_bb_enqueue_desc_f= ill(q, mbuf, offset, flags,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 meta, meta_= cnt);
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0flags &=3D ~ARK_D= DM_SOP;=C2=A0 /* drop SOP flags */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0meta_cnt =3D 0;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0offset =3D 0;
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mbuf =3D next;
> +=C2=A0 =C2=A0 =C2=A0}
> +}
> +
> +static inline int
> +ark_bb_enqueue_common(struct ark_bbdev_queue *q,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= struct rte_mbuf *m_in, struct rte_mbuf *m_out,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= uint16_t offset,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= uint32_t *meta, uint8_t meta_cnt) {
> +=C2=A0 =C2=A0 =C2=A0int32_t free_queue_space;
> +=C2=A0 =C2=A0 =C2=A0int32_t rx_idx;
> +
> +=C2=A0 =C2=A0 =C2=A0/* TX side limit */
> +=C2=A0 =C2=A0 =C2=A0free_queue_space =3D q->tx_queue_mask -
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(q->tx_prod_index = - q->tx_free_index);
> +=C2=A0 =C2=A0 =C2=A0if (unlikely(free_queue_space < (2 + (2 * m_in= ->nb_segs))))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1;
> +
> +=C2=A0 =C2=A0 =C2=A0/* RX side limit */
> +=C2=A0 =C2=A0 =C2=A0free_queue_space =3D q->rx_queue_mask -
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(q->rx_seed_index = - q->rx_cons_index);
> +=C2=A0 =C2=A0 =C2=A0if (unlikely(free_queue_space < m_out->nb_s= egs))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1;
> +
> +=C2=A0 =C2=A0 =C2=A0if (unlikely(m_in->nb_segs > 1))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_bb_enqueue_segmen= ted_pkt(q, m_in, offset, meta,
> meta_cnt);
> +=C2=A0 =C2=A0 =C2=A0else
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_bb_enqueue_desc_f= ill(q, m_in, offset,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ARK_DDM_SOP= | ARK_DDM_EOP,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 meta, meta_= cnt);
> +
> +=C2=A0 =C2=A0 =C2=A0/* We assume that the return mubf has exactly eno= ugh segments for
> +=C2=A0 =C2=A0 =C2=A0 * return data, which is 2048 bytes per segment.<= br> > +=C2=A0 =C2=A0 =C2=A0 */
> +=C2=A0 =C2=A0 =C2=A0do {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rx_idx =3D q->rx_s= eed_index & q->rx_queue_mask;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0q->rx_paddress_q[r= x_idx] =3D m_out->buf_iova;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0q->rx_seed_index++= ;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0m_out =3D m_out->n= ext;
> +=C2=A0 =C2=A0 =C2=A0} while (m_out);
> +
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +static inline void
> +ark_bb_enqueue_finalize(struct rte_bbdev_queue_data *q_data,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0struct ark_bbdev_queue *q,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0void **ops,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0uint16_t nb_ops, uint16_t nb)
> +{
> +=C2=A0 =C2=A0 =C2=A0/* BBDEV global stats */
> +=C2=A0 =C2=A0 =C2=A0/* These are not really errors, not sure why bbde= v counts these. */
> +=C2=A0 =C2=A0 =C2=A0q_data->queue_stats.enqueue_err_count +=3D nb_= ops - nb;
> +=C2=A0 =C2=A0 =C2=A0q_data->queue_stats.enqueued_count +=3D nb; > +
> +=C2=A0 =C2=A0 =C2=A0/* Notify HW that=C2=A0 */
> +=C2=A0 =C2=A0 =C2=A0if (unlikely(nb =3D=3D 0))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return;
> +
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_set_producer(q->tx_mpu, q->tx_prod_= index);
> +=C2=A0 =C2=A0 =C2=A0ark_mpu_set_producer(q->rx_mpu, q->rx_seed_= index);
> +
> +=C2=A0 =C2=A0 =C2=A0/* Queue info for dequeue-side processing */
> +=C2=A0 =C2=A0 =C2=A0rte_ring_enqueue_burst(q->active_ops,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 (void **)ops, nb, NULL);
> +}
> +
> +static int
> +ark_bb_dequeue_segmented(struct rte_mbuf *mbuf0,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 int32_t *prx_cons_index,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 uint16_t pkt_len
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 )
> +{
> +=C2=A0 =C2=A0 =C2=A0struct rte_mbuf *mbuf;
> +=C2=A0 =C2=A0 =C2=A0uint16_t data_len;
> +=C2=A0 =C2=A0 =C2=A0uint16_t remaining;
> +=C2=A0 =C2=A0 =C2=A0uint16_t segments =3D 1;
> +
> +=C2=A0 =C2=A0 =C2=A0data_len =3D RTE_MIN(pkt_len, RTE_MBUF_DEFAULT_DA= TAROOM);
> +=C2=A0 =C2=A0 =C2=A0remaining =3D pkt_len - data_len;
> +
> +=C2=A0 =C2=A0 =C2=A0mbuf =3D mbuf0;
> +=C2=A0 =C2=A0 =C2=A0mbuf0->data_len =3D data_len;
> +=C2=A0 =C2=A0 =C2=A0while (remaining) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0segments +=3D 1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mbuf =3D mbuf->nex= t;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (unlikely(mbuf =3D= =3D 0)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0ARK_BBDEV_LOG(CRIT, "Expected chained mbuf with
> "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"at least %d s= egments for dequeue "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"of packet len= gth %d",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0segments, pkt_len);=
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0return 1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0data_len =3D RTE_MIN(= remaining,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 RTE_MBUF_DEFAULT_DATAROOM);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0remaining -=3D data_l= en;
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mbuf->data_len =3D= data_len;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*prx_cons_index +=3D = 1;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0if (mbuf->next !=3D 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(CRIT, &= quot;Expected chained mbuf with "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0"at exactly %d segments for dequeue "=
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0"of packet length %d. Found %d "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0"segments",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0segments, pkt_len, mbuf0->nb_segs);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +/*
> +****************************************************************
> *******
> +** */
> +/* LDPC Decode ops */
> +static int16_t
> +ark_bb_enqueue_ldpc_dec_one_op(struct ark_bbdev_queue *q,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 struct rte_bbdev_dec_op *this_op) {
> +=C2=A0 =C2=A0 =C2=A0struct rte_bbdev_op_ldpc_dec *ldpc_dec_op =3D &am= p;this_op->ldpc_dec;
> +=C2=A0 =C2=A0 =C2=A0struct rte_mbuf *m_in =3D ldpc_dec_op->input.d= ata;
> +=C2=A0 =C2=A0 =C2=A0struct rte_mbuf *m_out =3D ldpc_dec_op->hard_o= utput.data;
> +=C2=A0 =C2=A0 =C2=A0uint16_t offset =3D ldpc_dec_op->input.offset;=
> +=C2=A0 =C2=A0 =C2=A0uint32_t meta[5] =3D {0};
> +=C2=A0 =C2=A0 =C2=A0uint8_t meta_cnt =3D 0;
> +
> +=C2=A0 =C2=A0 =C2=A0/* User's meta move from bbdev op to Arkville= HW */
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb_user_enqueue_ldpc_dec(this_op, meta, &= amp;meta_cnt)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;%s failed", __func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0return ark_bb_enqueue_common(q, m_in, m_out, offs= et, meta,
> meta_cnt);
> +}
> +
> +/* Enqueue LDPC Decode -- burst */
> +static uint16_t
> +ark_bb_enqueue_ldpc_dec_ops(struct rte_bbdev_queue_data *q_data,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0struct rte_bbdev_dec_op **ops, uint16_t nb_ops) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdev_queue *q =3D q_data->queue_pr= ivate;
> +=C2=A0 =C2=A0 =C2=A0unsigned int max_enq;
> +=C2=A0 =C2=A0 =C2=A0uint16_t nb;
> +
> +=C2=A0 =C2=A0 =C2=A0max_enq =3D rte_ring_free_count(q->active_ops)= ;
> +=C2=A0 =C2=A0 =C2=A0max_enq =3D RTE_MIN(max_enq, nb_ops);
> +=C2=A0 =C2=A0 =C2=A0for (nb =3D 0; nb < max_enq; nb++) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (ark_bb_enqueue_ld= pc_dec_one_op(q, ops[nb]))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0break;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0ark_bb_enqueue_finalize(q_data, q, (void **)ops, = nb_ops, nb);
> +=C2=A0 =C2=A0 =C2=A0return nb;
> +}
> +
> +/*
> +****************************************************************
> *******
> +** */
> +/* Dequeue LDPC Decode -- burst */
> +static uint16_t
> +ark_bb_dequeue_ldpc_dec_ops(struct rte_bbdev_queue_data *q_data,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0struct rte_bbdev_dec_op **ops, uint16_t nb_ops) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdev_queue *q =3D q_data->queue_pr= ivate;
> +=C2=A0 =C2=A0 =C2=A0struct rte_mbuf *mbuf;
> +=C2=A0 =C2=A0 =C2=A0struct rte_bbdev_dec_op *this_op;
> +=C2=A0 =C2=A0 =C2=A0struct ark_rx_meta *meta;
> +=C2=A0 =C2=A0 =C2=A0uint32_t *usermeta;
> +
> +=C2=A0 =C2=A0 =C2=A0uint16_t nb =3D 0;
> +=C2=A0 =C2=A0 =C2=A0int32_t prod_index =3D q->rx_prod_index;
> +=C2=A0 =C2=A0 =C2=A0int32_t cons_index =3D q->rx_cons_index;
> +
> +=C2=A0 =C2=A0 =C2=A0q->tx_free_index =3D q->tx_cons_index;
> +
> +=C2=A0 =C2=A0 =C2=A0while ((prod_index - cons_index) > 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (rte_ring_dequeue(= q->active_ops, (void **)&this_op)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0ARK_BBDEV_LOG(ERR, "%s data ready but no op!",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0__func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0q_data->queue_stats.dequeue_err_count +=3D 1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0break;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ops[nb] =3D this_op;<= br> > +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mbuf =3D this_op->= ldpc_dec.hard_output.data;
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* META DATA embedded= in headroom */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0meta =3D RTE_PTR_ADD(= mbuf->buf_addr,
> ARK_RX_META_OFFSET);
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mbuf->pkt_len =3D = meta->pkt_len;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mbuf->data_len =3D= meta->pkt_len;
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (unlikely(meta->= ;pkt_len > ARK_RX_MAX_NOCHAIN)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0if (ark_bb_dequeue_segmented(mbuf, &cons_index,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 meta->pkt_len))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0q_data->queue_stats.dequeue_err_count= +=3D
> 1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0} else if (mbuf->n= ext !=3D 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0ARK_BBDEV_LOG(CRIT, "Expected mbuf with "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"at exactly 1 = segments for dequeue "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"of packet len= gth %d. Found %d "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"segments"= ;,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0meta->pkt_len, m= buf->nb_segs);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0q_data->queue_stats.dequeue_err_count +=3D 1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0usermeta =3D meta->= ;user_meta;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* User's meta mo= ve from Arkville HW to bbdev OP */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_bb_user_dequeue_l= dpc_dec(this_op, usermeta);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nb++;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cons_index++;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (nb >=3D nb_ops= )
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0break;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0q->rx_cons_index =3D cons_index;
> +
> +=C2=A0 =C2=A0 =C2=A0/* BBdev stats */
> +=C2=A0 =C2=A0 =C2=A0q_data->queue_stats.dequeued_count +=3D nb; > +
> +=C2=A0 =C2=A0 =C2=A0return nb;
> +}
> +
> +/***************************************************************
> *******
> +****/
> +/* Enqueue LDPC Encode */
> +static int16_t
> +ark_bb_enqueue_ldpc_enc_one_op(struct ark_bbdev_queue *q,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 struct rte_bbdev_enc_op *this_op) {
> +=C2=A0 =C2=A0 =C2=A0struct rte_bbdev_op_ldpc_enc *ldpc_enc_op =3D &am= p;this_op->ldpc_enc;
> +=C2=A0 =C2=A0 =C2=A0struct rte_mbuf *m_in =3D ldpc_enc_op->input.d= ata;
> +=C2=A0 =C2=A0 =C2=A0struct rte_mbuf *m_out =3D ldpc_enc_op->output= .data;
> +=C2=A0 =C2=A0 =C2=A0uint16_t offset =3D ldpc_enc_op->input.offset;=
> +=C2=A0 =C2=A0 =C2=A0uint32_t meta[5] =3D {0};
> +=C2=A0 =C2=A0 =C2=A0uint8_t meta_cnt =3D 0;
> +
> +=C2=A0 =C2=A0 =C2=A0/* User's meta move from bbdev op to Arkville= HW */
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb_user_enqueue_ldpc_enc(this_op, meta, &= amp;meta_cnt)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;%s failed", __func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0return ark_bb_enqueue_common(q, m_in, m_out, offs= et, meta,
> meta_cnt);
> +}
> +
> +/* Enqueue LDPC Encode -- burst */
> +static uint16_t
> +ark_bb_enqueue_ldpc_enc_ops(struct rte_bbdev_queue_data *q_data,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0struct rte_bbdev_enc_op **ops, uint16_t nb_ops) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdev_queue *q =3D q_data->queue_pr= ivate;
> +=C2=A0 =C2=A0 =C2=A0unsigned int max_enq;
> +=C2=A0 =C2=A0 =C2=A0uint16_t nb;
> +
> +=C2=A0 =C2=A0 =C2=A0max_enq =3D rte_ring_free_count(q->active_ops)= ;
> +=C2=A0 =C2=A0 =C2=A0max_enq =3D RTE_MIN(max_enq, nb_ops);
> +=C2=A0 =C2=A0 =C2=A0for (nb =3D 0; nb < max_enq; nb++) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (ark_bb_enqueue_ld= pc_enc_one_op(q, ops[nb]))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0break;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0ark_bb_enqueue_finalize(q_data, q, (void **)ops, = nb_ops, nb);
> +=C2=A0 =C2=A0 =C2=A0return nb;
> +}
> +
> +/* Dequeue LDPC Encode -- burst */
> +static uint16_t
> +ark_bb_dequeue_ldpc_enc_ops(struct rte_bbdev_queue_data *q_data,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0struct rte_bbdev_enc_op **ops, uint16_t nb_ops) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdev_queue *q =3D q_data->queue_pr= ivate;
> +=C2=A0 =C2=A0 =C2=A0struct rte_mbuf *mbuf;
> +=C2=A0 =C2=A0 =C2=A0struct rte_bbdev_enc_op *this_op;
> +=C2=A0 =C2=A0 =C2=A0struct ark_rx_meta *meta;
> +=C2=A0 =C2=A0 =C2=A0uint32_t *usermeta;
> +
> +=C2=A0 =C2=A0 =C2=A0uint16_t nb =3D 0;
> +=C2=A0 =C2=A0 =C2=A0int32_t prod_index =3D q->rx_prod_index;
> +=C2=A0 =C2=A0 =C2=A0int32_t cons_index =3D q->rx_cons_index;
> +
> +=C2=A0 =C2=A0 =C2=A0q->tx_free_index =3D q->tx_cons_index;
> +
> +=C2=A0 =C2=A0 =C2=A0while ((prod_index - cons_index) > 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (rte_ring_dequeue(= q->active_ops, (void **)&this_op)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0ARK_BBDEV_LOG(ERR, "%s data ready but no op!",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0__func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0q_data->queue_stats.dequeue_err_count +=3D 1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0break;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ops[nb] =3D this_op;<= br> > +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mbuf =3D this_op->= ldpc_enc.output.data;
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* META DATA embedded= in headroom */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0meta =3D RTE_PTR_ADD(= mbuf->buf_addr,
> ARK_RX_META_OFFSET);
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mbuf->pkt_len =3D = meta->pkt_len;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mbuf->data_len =3D= meta->pkt_len;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0usermeta =3D meta->= ;user_meta;
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (unlikely(meta->= ;pkt_len > ARK_RX_MAX_NOCHAIN)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0if (ark_bb_dequeue_segmented(mbuf, &cons_index,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 meta->pkt_len))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0q_data->queue_stats.dequeue_err_count= +=3D
> 1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0} else if (mbuf->n= ext !=3D 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0ARK_BBDEV_LOG(CRIT, "Expected mbuf with "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"at exactly 1 = segments for dequeue "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"of packet len= gth %d. Found %d "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"segments"= ;,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0meta->pkt_len, m= buf->nb_segs);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0q_data->queue_stats.dequeue_err_count +=3D 1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* User's meta mo= ve from Arkville HW to bbdev OP */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_bb_user_dequeue_l= dpc_enc(this_op, usermeta);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nb++;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cons_index++;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (nb >=3D nb_ops= )
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0break;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0q->rx_cons_index =3D cons_index;
> +
> +=C2=A0 =C2=A0 =C2=A0/* BBdev stats */
> +=C2=A0 =C2=A0 =C2=A0q_data->queue_stats.dequeued_count +=3D nb; > +
> +=C2=A0 =C2=A0 =C2=A0return nb;
> +}
> +
> +/***************************************************************
> *******
> +****/
> +/*
> + *Initial device hardware configuration when device is opened
> + * setup the DDM, and UDM; called once per PCIE device=C2=A0 */ stati= c int
> +ark_bb_config_device(struct ark_bbdevice *ark_bb) {
> +=C2=A0 =C2=A0 =C2=A0uint16_t num_q, i;
> +=C2=A0 =C2=A0 =C2=A0struct ark_mpu_t *mpu;
> +
> +=C2=A0 =C2=A0 =C2=A0/*
> +=C2=A0 =C2=A0 =C2=A0 * Make sure that the packet director, generator = and checker are in a
> +=C2=A0 =C2=A0 =C2=A0 * known state
> +=C2=A0 =C2=A0 =C2=A0 */
> +=C2=A0 =C2=A0 =C2=A0ark_bb->start_pg =3D 0;
> +=C2=A0 =C2=A0 =C2=A0ark_bb->pg =3D ark_pktgen_init(ark_bb->pktg= en.v, 0, 1);
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb->pg =3D=3D NULL)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
> +=C2=A0 =C2=A0 =C2=A0ark_pktgen_reset(ark_bb->pg);
> +=C2=A0 =C2=A0 =C2=A0ark_bb->pc =3D ark_pktchkr_init(ark_bb->pkt= chkr.v, 0, 1);
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb->pc =3D=3D NULL)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
> +=C2=A0 =C2=A0 =C2=A0ark_pktchkr_stop(ark_bb->pc);
> +=C2=A0 =C2=A0 =C2=A0ark_bb->pd =3D ark_pktdir_init(ark_bb->pktd= ir.v);
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb->pd =3D=3D NULL)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Verify HW */
> +=C2=A0 =C2=A0 =C2=A0if (ark_udm_verify(ark_bb->udm.v))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
> +=C2=A0 =C2=A0 =C2=A0if (ark_ddm_verify(ark_bb->ddm.v))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
> +
> +=C2=A0 =C2=A0 =C2=A0/* UDM */
> +=C2=A0 =C2=A0 =C2=A0if (ark_udm_reset(ark_bb->udm.v)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;Unable to stop and reset UDM");
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0/* Keep in reset until the MPU are cleared */
> +
> +=C2=A0 =C2=A0 =C2=A0/* MPU reset */
> +=C2=A0 =C2=A0 =C2=A0mpu =3D ark_bb->mpurx.v;
> +=C2=A0 =C2=A0 =C2=A0num_q =3D ark_api_num_queues(mpu);
> +=C2=A0 =C2=A0 =C2=A0ark_bb->max_nb_queues =3D num_q;
> +
> +=C2=A0 =C2=A0 =C2=A0for (i =3D 0; i < num_q; i++) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_mpu_reset(mpu); > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mpu =3D RTE_PTR_ADD(m= pu, ARK_MPU_QOFFSET);
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0/* Only 1 queue supported in the udm */
> +=C2=A0 =C2=A0 =C2=A0ark_udm_stop(ark_bb->udm.v, 0);
> +=C2=A0 =C2=A0 =C2=A0ark_udm_configure(ark_bb->udm.v,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0RTE_PKTMBUF_HEADROOM,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0bbdev->data->queues[q_id]->dataroom,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0ARK_RX_WRITE_TIME_NS);
> +
> +
> +=C2=A0 =C2=A0 =C2=A0ark_udm_stats_reset(ark_bb->udm.v);
> +=C2=A0 =C2=A0 =C2=A0ark_udm_stop(ark_bb->udm.v, 0);
> +
> +=C2=A0 =C2=A0 =C2=A0/* TX -- DDM */
> +=C2=A0 =C2=A0 =C2=A0if (ark_ddm_stop(ark_bb->ddm.v, 1))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;Unable to stop DDM");
> +
> +=C2=A0 =C2=A0 =C2=A0mpu =3D ark_bb->mputx.v;
> +=C2=A0 =C2=A0 =C2=A0num_q =3D ark_api_num_queues(mpu);
> +=C2=A0 =C2=A0 =C2=A0for (i =3D 0; i < num_q; i++) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_mpu_reset(mpu); > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mpu =3D RTE_PTR_ADD(m= pu, ARK_MPU_QOFFSET);
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0ark_ddm_reset(ark_bb->ddm.v);
> +=C2=A0 =C2=A0 =C2=A0ark_ddm_stats_reset(ark_bb->ddm.v);
> +
> +=C2=A0 =C2=A0 =C2=A0ark_ddm_stop(ark_bb->ddm.v, 0);
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb->rqpacing)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_rqp_stats_reset(a= rk_bb->rqpacing);
> +
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(INFO, "packet director set to = 0x%x", ark_bb-
> >pkt_dir_v);
> +=C2=A0 =C2=A0 =C2=A0ark_pktdir_setup(ark_bb->pd, ark_bb->pkt_di= r_v);
> +
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb->pkt_gen_args[0]) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(INFO, &= quot;Setting up the packet generator");
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_pktgen_parse(ark_= bb->pkt_gen_args);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_pktgen_reset(ark_= bb->pg);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_pktgen_setup(ark_= bb->pg);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_bb->start_pg = =3D 1;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +static int
> +ark_bbdev_init(struct rte_bbdev *bbdev, struct rte_pci_driver *pci_dr= v)
> +{
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdevice *ark_bb =3D bbdev->data-&g= t;dev_private;
> +=C2=A0 =C2=A0 =C2=A0struct rte_pci_device *pci_dev =3D RTE_DEV_TO_PCI= (bbdev->device);
> +=C2=A0 =C2=A0 =C2=A0bool rqpacing =3D false;
> +=C2=A0 =C2=A0 =C2=A0int p;
> +
> +=C2=A0 =C2=A0 =C2=A0RTE_SET_USED(pci_drv);
> +
> +=C2=A0 =C2=A0 =C2=A0ark_bb->bar0 =3D (uint8_t *)pci_dev->mem_re= source[0].addr;
> +=C2=A0 =C2=A0 =C2=A0ark_bb->a_bar =3D (uint8_t *)pci_dev->mem_r= esource[2].addr;
> +
> +=C2=A0 =C2=A0 =C2=A0ark_bb->sysctrl.v=C2=A0 =3D (void *)&ark_b= b->bar0[ARK_SYSCTRL_BASE];
> +=C2=A0 =C2=A0 =C2=A0ark_bb->mpurx.v=C2=A0 =3D (void *)&ark_bb-= >bar0[ARK_MPU_RX_BASE];
> +=C2=A0 =C2=A0 =C2=A0ark_bb->udm.v=C2=A0 =3D (void *)&ark_bb-&g= t;bar0[ARK_UDM_BASE];
> +=C2=A0 =C2=A0 =C2=A0ark_bb->mputx.v=C2=A0 =3D (void *)&ark_bb-= >bar0[ARK_MPU_TX_BASE];
> +=C2=A0 =C2=A0 =C2=A0ark_bb->ddm.v=C2=A0 =3D (void *)&ark_bb-&g= t;bar0[ARK_DDM_BASE];
> +=C2=A0 =C2=A0 =C2=A0ark_bb->pktdir.v=C2=A0 =3D (void *)&ark_bb= ->bar0[ARK_PKTDIR_BASE];
> +=C2=A0 =C2=A0 =C2=A0ark_bb->pktgen.v=C2=A0 =3D (void *)&ark_bb= ->bar0[ARK_PKTGEN_BASE];
> +=C2=A0 =C2=A0 =C2=A0ark_bb->pktchkr.v=C2=A0 =3D (void *)&ark_b= b->bar0[ARK_PKTCHKR_BASE];
> +
> +=C2=A0 =C2=A0 =C2=A0p =3D 0;
> +=C2=A0 =C2=A0 =C2=A0while (ark_device_caps[p].device_id !=3D 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (pci_dev->id.de= vice_id =3D=3D ark_device_caps[p].device_id) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0rqpacing =3D ark_device_caps[p].caps.rqpacing;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0break;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0p++;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0if (rqpacing)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_bb->rqpacing = =3D
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0(struct ark_rqpace_t *)(ark_bb->bar0 +
> ARK_RCPACING_BASE);
> +=C2=A0 =C2=A0 =C2=A0else
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_bb->rqpacing = =3D NULL;
> +
> +=C2=A0 =C2=A0 =C2=A0ark_bb->started =3D 0;
> +
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(INFO, "Sys Ctrl Const =3D 0x%x= =C2=A0 HW Commit_ID:
> %08x",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= ark_bb->sysctrl.t32[4],
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= rte_be_to_cpu_32(ark_bb->sysctrl.t32[0x20 / 4]));
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(INFO, "Arkville HW Commit_ID: = %08x",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_be_= to_cpu_32(ark_bb->sysctrl.t32[0x20 / 4]));
> +
> +=C2=A0 =C2=A0 =C2=A0/* If HW sanity test fails, return an error */ > +=C2=A0 =C2=A0 =C2=A0if (ark_bb->sysctrl.t32[4] !=3D 0xcafef00d) {<= br> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0"HW Sanity test has failed, expected const= ant"
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0" 0x%x, read 0x%x (%s)",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A00xcafef00d,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0ark_bb->sysctrl.t32[4], __func__);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0return ark_bb_config_device(ark_bb);
> +}
> +
> +static int
> +ark_bbdev_uninit(struct rte_bbdev *bbdev) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdevice *ark_bb =3D bbdev->data-&g= t;dev_private;
> +
> +=C2=A0 =C2=A0 =C2=A0if (rte_eal_process_type() !=3D RTE_PROC_PRIMARY)=
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 0;
> +
> +=C2=A0 =C2=A0 =C2=A0ark_pktgen_uninit(ark_bb->pg);
> +=C2=A0 =C2=A0 =C2=A0ark_pktchkr_uninit(ark_bb->pc);
> +
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +static int
> +ark_bbdev_probe(struct rte_pci_driver *pci_drv,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0struct rte_pci_device= *pci_dev)
> +{
> +=C2=A0 =C2=A0 =C2=A0struct rte_bbdev *bbdev =3D NULL;
> +=C2=A0 =C2=A0 =C2=A0char dev_name[RTE_BBDEV_NAME_MAX_LEN];
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdevice *ark_bb;
> +
> +=C2=A0 =C2=A0 =C2=A0if (pci_dev =3D=3D NULL)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EINVAL;
> +
> +=C2=A0 =C2=A0 =C2=A0rte_pci_device_name(&pci_dev->addr, dev_na= me, sizeof(dev_name));
> +
> +=C2=A0 =C2=A0 =C2=A0/* Allocate memory to be used privately by driver= s */
> +=C2=A0 =C2=A0 =C2=A0bbdev =3D rte_bbdev_allocate(pci_dev->device.name);
> +=C2=A0 =C2=A0 =C2=A0if (bbdev =3D=3D NULL)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -ENODEV;
> +
> +=C2=A0 =C2=A0 =C2=A0/* allocate device private memory */
> +=C2=A0 =C2=A0 =C2=A0bbdev->data->dev_private =3D rte_zmalloc_so= cket(dev_name,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0sizeof(struct ark_bbdevice),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0RTE_CACHE_LINE_SIZE,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0pci_dev->device.numa_node);
> +
> +=C2=A0 =C2=A0 =C2=A0if (bbdev->data->dev_private =3D=3D NULL) {=
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(CRIT, > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"Allocate of %zu bytes for device \= "%s\"
> failed",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sizeof(struct ark_bbdevice), dev_name);<= br> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_bbdev_release(bbdev);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0return -ENOMEM;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0ark_bb =3D bbdev->data->dev_private;
> +=C2=A0 =C2=A0 =C2=A0/* Initialize ark_bb */
> +=C2=A0 =C2=A0 =C2=A0ark_bb->pkt_dir_v =3D 0x00110110;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Fill HW specific part of device structure */ > +=C2=A0 =C2=A0 =C2=A0bbdev->device =3D &pci_dev->device;
> +=C2=A0 =C2=A0 =C2=A0bbdev->intr_handle =3D NULL;
> +=C2=A0 =C2=A0 =C2=A0bbdev->data->socket_id =3D pci_dev->devi= ce.numa_node;
> +=C2=A0 =C2=A0 =C2=A0bbdev->dev_ops =3D &ark_bbdev_pmd_ops;
> +=C2=A0 =C2=A0 =C2=A0if (pci_dev->device.devargs)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0parse_ark_bbdev_param= s(pci_dev->device.devargs->args,
> ark_bb);
> +
> +
> +=C2=A0 =C2=A0 =C2=A0/* Device specific initialization */
> +=C2=A0 =C2=A0 =C2=A0if (ark_bbdev_init(bbdev, pci_drv))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EIO;
> +=C2=A0 =C2=A0 =C2=A0if (ark_bbdev_start(bbdev))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EIO;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Core operations LDPC encode amd decode */
> +=C2=A0 =C2=A0 =C2=A0bbdev->enqueue_ldpc_enc_ops =3D ark_bb_enqueue= _ldpc_enc_ops;
> +=C2=A0 =C2=A0 =C2=A0bbdev->dequeue_ldpc_enc_ops =3D ark_bb_dequeue= _ldpc_enc_ops;
> +=C2=A0 =C2=A0 =C2=A0bbdev->enqueue_ldpc_dec_ops =3D ark_bb_enqueue= _ldpc_dec_ops;
> +=C2=A0 =C2=A0 =C2=A0bbdev->dequeue_ldpc_dec_ops =3D ark_bb_dequeue= _ldpc_dec_ops;
> +
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "bbdev id =3D %u [%s]&q= uot;,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= bbdev->data->dev_id, dev_name);
> +
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +/* Uninitialize device */
> +static int
> +ark_bbdev_remove(struct rte_pci_device *pci_dev) {
> +=C2=A0 =C2=A0 =C2=A0struct rte_bbdev *bbdev;
> +=C2=A0 =C2=A0 =C2=A0int ret;
> +
> +=C2=A0 =C2=A0 =C2=A0if (pci_dev =3D=3D NULL)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EINVAL;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Find device */
> +=C2=A0 =C2=A0 =C2=A0bbdev =3D rte_bbdev_get_named_dev(pci_dev->device.nam= e);
> +=C2=A0 =C2=A0 =C2=A0if (bbdev =3D=3D NULL) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(CRIT, > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"Couldn't find HW dev \"%s= \" to Uninitialize
> it",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pci_dev->device.name);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -ENODEV;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0/* Arkville device close */
> +=C2=A0 =C2=A0 =C2=A0ark_bbdev_uninit(bbdev);
> +=C2=A0 =C2=A0 =C2=A0rte_free(bbdev->data->dev_private);
> +
> +=C2=A0 =C2=A0 =C2=A0/* Close device */
> +=C2=A0 =C2=A0 =C2=A0ret =3D rte_bbdev_close(bbdev->data->dev_id= );
> +=C2=A0 =C2=A0 =C2=A0if (ret < 0)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"Device %i failed to close during r= emove: %i",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bbdev->data->dev_id, ret);
> +
> +=C2=A0 =C2=A0 =C2=A0return rte_bbdev_release(bbdev);
> +}
> +
> +/* Operation for the PMD */
> +static const struct rte_bbdev_ops ark_bbdev_pmd_ops =3D {
> +=C2=A0 =C2=A0 =C2=A0.info_get =3D ark_bbdev_info_get,
> +=C2=A0 =C2=A0 =C2=A0.start =3D ark_bbdev_start,
> +=C2=A0 =C2=A0 =C2=A0.stop =3D ark_bbdev_stop,
> +=C2=A0 =C2=A0 =C2=A0.queue_setup =3D ark_bb_q_setup,
> +=C2=A0 =C2=A0 =C2=A0.queue_release =3D ark_bb_q_release,
> +=C2=A0 =C2=A0 =C2=A0.queue_start =3D ark_bb_q_start,
> +=C2=A0 =C2=A0 =C2=A0.queue_stop =3D ark_bb_q_stop,
> +};
> +
> +
> +
> +static struct rte_pci_driver ark_bbdev_pmd_drv =3D {
> +=C2=A0 =C2=A0 =C2=A0.probe =3D ark_bbdev_probe,
> +=C2=A0 =C2=A0 =C2=A0.remove =3D ark_bbdev_remove,
> +=C2=A0 =C2=A0 =C2=A0.id_table =3D pci_id_ark,
> +=C2=A0 =C2=A0 =C2=A0.drv_flags =3D RTE_PCI_DRV_NEED_MAPPING
> +};
> +
> +RTE_PMD_REGISTER_PCI(DRIVER_NAME, ark_bbdev_pmd_drv);
> +RTE_PMD_REGISTER_PCI_TABLE(DRIVER_NAME, pci_id_ark);
> +RTE_PMD_REGISTER_PARAM_STRING(DRIVER_NAME,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_PKTGEN_ARG "=3D<filename> = "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_PKTCHKR_ARG "=3D<filename>= "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_PKTDIR_ARG "=3D<bitmap>&qu= ot;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0);
> diff --git a/drivers/baseband/ark/ark_bbdev_common.c
> b/drivers/baseband/ark/ark_bbdev_common.c
> new file mode 100644
> index 0000000000..6ef0f43654
> --- /dev/null
> +++ b/drivers/baseband/ark/ark_bbdev_common.c
> @@ -0,0 +1,125 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2016-2021 Atomic Rules LLC=C2=A0 */
> +
> +#include <string.h>
> +
> +#include <rte_kvargs.h>
> +#include <rte_log.h>
> +
> +#include "ark_bbdev_common.h"
> +
> +static const char * const ark_bbdev_valid_params[] =3D {
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_PKTDIR_ARG,
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_PKTGEN_ARG,
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_PKTCHKR_ARG,
> +=C2=A0 =C2=A0 =C2=A0NULL
> +};
> +
> +/* Parse 16-bit integer from string argument */ static inline int
> +parse_u16_arg(const char *key, const char *value, void *extra_args) {=
> +=C2=A0 =C2=A0 =C2=A0uint16_t *u16 =3D extra_args;
> +=C2=A0 =C2=A0 =C2=A0unsigned int long result;
> +
> +=C2=A0 =C2=A0 =C2=A0if ((value =3D=3D NULL) || (extra_args =3D=3D NUL= L))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EINVAL;
> +=C2=A0 =C2=A0 =C2=A0errno =3D 0;
> +=C2=A0 =C2=A0 =C2=A0result =3D strtoul(value, NULL, 0);
> +=C2=A0 =C2=A0 =C2=A0if ((result >=3D (1 << 16)) || (errno != =3D 0)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;Invalid value %" PRIu64 " for %s",
> result, key);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -ERANGE;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0*u16 =3D (uint16_t)result;
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +static inline int
> +process_pktdir_arg(const char *key, const char *value,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 void *extra_a= rgs)
> +{
> +=C2=A0 =C2=A0 =C2=A0uint32_t *u32 =3D extra_args;
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "key =3D %s, value =3D = %s", key, value);
> +
> +=C2=A0 =C2=A0 =C2=A0*u32 =3D strtol(value, NULL, 0);
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "pkt_dir_v =3D 0x%x&quo= t;, *u32);
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +static inline int
> +process_file_args(const char *key, const char *value, void *extra_arg= s)
> +{
> +=C2=A0 =C2=A0 =C2=A0char *args =3D (char *)extra_args;
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "key =3D %s, value =3D = %s", key, value);
> +
> +=C2=A0 =C2=A0 =C2=A0/* Open the configuration file */
> +=C2=A0 =C2=A0 =C2=A0FILE *file =3D fopen(value, "r");
> +=C2=A0 =C2=A0 =C2=A0char line[ARK_MAX_ARG_LEN];
> +=C2=A0 =C2=A0 =C2=A0int=C2=A0 size =3D 0;
> +=C2=A0 =C2=A0 =C2=A0int first =3D 1;
> +
> +=C2=A0 =C2=A0 =C2=A0if (file =3D=3D NULL) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(ERR, &q= uot;Unable to open config file %s",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0value);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0while (fgets(line, sizeof(line), file)) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0size +=3D strlen(line= );
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (size >=3D ARK_= MAX_ARG_LEN) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0ARK_BBDEV_LOG(ERR, "Unable to parse file %s args,
> "
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"parameter lis= t is too long", value);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0fclose(file);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0return -1;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (first) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0strncpy(args, line, ARK_MAX_ARG_LEN);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0first =3D 0;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0} else {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0strncat(args, line, ARK_MAX_ARG_LEN);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0ARK_BBDEV_LOG(DEBUG, "file =3D %s", arg= s);
> +=C2=A0 =C2=A0 =C2=A0fclose(file);
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +
> +/* Parse parameters used to create device */ int
> +parse_ark_bbdev_params(const char *input_args,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= struct ark_bbdevice *ark_bb)
> +{
> +=C2=A0 =C2=A0 =C2=A0struct rte_kvargs *kvlist =3D NULL;
> +=C2=A0 =C2=A0 =C2=A0int ret =3D 0;
> +
> +=C2=A0 =C2=A0 =C2=A0if (ark_bb =3D=3D NULL)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EINVAL;
> +=C2=A0 =C2=A0 =C2=A0if (input_args =3D=3D NULL)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return ret;
> +
> +=C2=A0 =C2=A0 =C2=A0kvlist =3D rte_kvargs_parse(input_args, ark_bbdev= _valid_params);
> +=C2=A0 =C2=A0 =C2=A0if (kvlist =3D=3D NULL)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EFAULT;
> +
> +=C2=A0 =C2=A0 =C2=A0ret =3D rte_kvargs_process(kvlist, ARK_BBDEV_PKTD= IR_ARG,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0&process_pktdir_arg, &ark= _bb->pkt_dir_v);
> +=C2=A0 =C2=A0 =C2=A0if (ret < 0)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto exit;
> +
> +=C2=A0 =C2=A0 =C2=A0ret =3D rte_kvargs_process(kvlist, ARK_BBDEV_PKTG= EN_ARG,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &process_file_args, &ark_bb- > >pkt_gen_args);
> +=C2=A0 =C2=A0 =C2=A0if (ret < 0)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto exit;
> +
> +=C2=A0 =C2=A0 =C2=A0ret =3D rte_kvargs_process(kvlist, ARK_BBDEV_PKTC= HKR_ARG,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &process_file_args, &ark_bb- > >pkt_chkr_args);
> +=C2=A0 =C2=A0 =C2=A0if (ret < 0)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto exit;
> +
> + exit:
> +=C2=A0 =C2=A0 =C2=A0if (kvlist)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_kvargs_free(kvlis= t);
> +=C2=A0 =C2=A0 =C2=A0return ret;
> +}
> diff --git a/drivers/baseband/ark/ark_bbdev_common.h
> b/drivers/baseband/ark/ark_bbdev_common.h
> new file mode 100644
> index 0000000000..670e7e86d6
> --- /dev/null
> +++ b/drivers/baseband/ark/ark_bbdev_common.h
> @@ -0,0 +1,92 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2016-2021 Atomic Rules LLC=C2=A0 */
> +
> +#ifndef _ARK_BBDEV_COMMON_H_
> +#define _ARK_BBDEV_COMMON_H_
> +
> +#include "ark_pktchkr.h"
> +#include "ark_pktdir.h"
> +#include "ark_pktgen.h"
> +
> +#define ARK_MAX_ARG_LEN 256
> +
> +/* Acceptable params for ark BBDEV devices */
> +/*
> + * The packet generator is a functional block used to generate packet=
> + * patterns for testing.=C2=A0 It is not intended for nominal use. > + */
> +#define ARK_BBDEV_PKTGEN_ARG "Pkt_gen"
> +
> +/*
> + * The packet checker is a functional block used to verify packet
> + * patterns for testing.=C2=A0 It is not intended for nominal use. > + */
> +#define ARK_BBDEV_PKTCHKR_ARG "Pkt_chkr"
> +
> +/*
> + * The packet director is used to select the internal ingress and
> + * egress packets paths during testing.=C2=A0 It is not intended for<= br> > + * nominal use.
> + */
> +#define ARK_BBDEV_PKTDIR_ARG "Pkt_dir"
> +
> +
> +#define def_ptr(type, name) \
> +=C2=A0 =C2=A0 =C2=A0union type {=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0\
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0uint64_t *t64;=C2=A0 = =C2=A0 =C2=A0\
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0uint32_t *t32;=C2=A0 = =C2=A0 =C2=A0\
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0uint16_t *t16;=C2=A0 = =C2=A0 =C2=A0\
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0uint8_t=C2=A0 *t8;=C2= =A0 =C2=A0 =C2=A0 \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0void=C2=A0 =C2=A0 =C2= =A0*v;=C2=A0 =C2=A0 =C2=A0 =C2=A0\
> +=C2=A0 =C2=A0 =C2=A0} name
> +
> +/*
> + * Structure to store private data for each PF/VF instance.
> + */
> +struct ark_bbdevice {
> +=C2=A0 =C2=A0 =C2=A0/* Our Bar 0 */
> +=C2=A0 =C2=A0 =C2=A0uint8_t *bar0;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Application Bar needed for extensions */
> +=C2=A0 =C2=A0 =C2=A0uint8_t *a_bar;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Arkville hardware block offsets */
> +=C2=A0 =C2=A0 =C2=A0def_ptr(sys_ctrl, sysctrl);
> +=C2=A0 =C2=A0 =C2=A0def_ptr(pkt_gen, pktgen);
> +=C2=A0 =C2=A0 =C2=A0def_ptr(mpu_rx, mpurx);
> +=C2=A0 =C2=A0 =C2=A0def_ptr(UDM, udm);
> +=C2=A0 =C2=A0 =C2=A0def_ptr(mpu_tx, mputx);
> +=C2=A0 =C2=A0 =C2=A0def_ptr(DDM, ddm);
> +=C2=A0 =C2=A0 =C2=A0def_ptr(pkt_dir, pktdir);
> +=C2=A0 =C2=A0 =C2=A0def_ptr(pkt_chkr, pktchkr);
> +=C2=A0 =C2=A0 =C2=A0struct ark_rqpace_t *rqpacing;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Pointers to packet generator and checker */ > +=C2=A0 =C2=A0 =C2=A0int start_pg;
> +=C2=A0 =C2=A0 =C2=A0ark_pkt_gen_t pg;
> +=C2=A0 =C2=A0 =C2=A0ark_pkt_chkr_t pc;
> +=C2=A0 =C2=A0 =C2=A0ark_pkt_dir_t pd;
> +
> +=C2=A0 =C2=A0 =C2=A0/* Packet generator/checker args */
> +=C2=A0 =C2=A0 =C2=A0char pkt_gen_args[ARK_MAX_ARG_LEN];
> +=C2=A0 =C2=A0 =C2=A0char pkt_chkr_args[ARK_MAX_ARG_LEN];
> +=C2=A0 =C2=A0 =C2=A0uint32_t pkt_dir_v;
> +
> +=C2=A0 =C2=A0 =C2=A0int started;
> +=C2=A0 =C2=A0 =C2=A0unsigned int max_nb_queues;=C2=A0 /**< Max num= ber of queues */
> +
> +};
> +
> +
> +/* Log message for PMD */
> +extern int ark_bbdev_logtype;
> +
> +/* Helper macro for logging */
> +#define ARK_BBDEV_LOG(level, fmt, ...) \
> +=C2=A0 =C2=A0 =C2=A0rte_log(RTE_LOG_ ## level, ark_bbdev_logtype, \ > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"ARK_BBD: "= fmt "\n", ##__VA_ARGS__)
> +
> +int parse_ark_bbdev_params(const char *argv, struct ark_bbdevice *dev= );
> +
> +#endif
> diff --git a/drivers/baseband/ark/ark_bbdev_custom.c
> b/drivers/baseband/ark/ark_bbdev_custom.c
> new file mode 100644
> index 0000000000..6b1553abe1
> --- /dev/null
> +++ b/drivers/baseband/ark/ark_bbdev_custom.c
> @@ -0,0 +1,201 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2016-2021 Atomic Rules LLC=C2=A0 */
> +
> +#include <rte_bbdev.h>
> +#include <rte_bbdev_pmd.h>
> +
> +#include <rte_mbuf.h>
> +#include <rte_hexdump.h>=C2=A0 =C2=A0 =C2=A0/* For debug */
> +
> +
> +#include "ark_bbdev_common.h"
> +#include "ark_bbdev_custom.h"
> +
> +/* It is expected that functions in this file will be modified based = on
> + * specifics of the FPGA hardware beyond the core Arkville
> + * components.
> + */
> +
> +/* bytyes must be range of 0 to 20 */
> +static inline
> +uint8_t ark_bb_cvt_bytes_meta_cnt(size_t bytes) {
> +=C2=A0 =C2=A0 =C2=A0return (bytes + 3) / 8;
> +}
> +
> +void
> +ark_bbdev_info_get(struct rte_bbdev *dev,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 struct rte_bb= dev_driver_info *dev_info) {
> +=C2=A0 =C2=A0 =C2=A0struct ark_bbdevice *ark_bb =3D=C2=A0 dev->dat= a->dev_private;
> +
> +=C2=A0 =C2=A0 =C2=A0static const struct rte_bbdev_op_cap bbdev_capabi= lities[] =3D {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0.type =3D RTE_BBDEV_OP_LDPC_DEC,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0.cap.ldpc_dec =3D {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.capability_flags =3D
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_BBDEV_LD= PC_CRC_24B_ATTACH
> |
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_BBDEV_LD= PC_RATE_MATCH,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.num_buffers_src =3D
> +
>=C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_BBDEV_LDPC_MAX_CODE_BLOCKS,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.num_buffers_hard_out =3D
> +
>=C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_BBDEV_LDPC_MAX_CODE_BLOCKS
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0},
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0.type =3D RTE_BBDEV_OP_LDPC_ENC,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0.cap.ldpc_enc =3D {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.capability_flags =3D
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_BBDEV_LD= PC_CRC_24B_ATTACH
> |
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_BBDEV_LD= PC_RATE_MATCH,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.num_buffers_src =3D
> +
>=C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_BBDEV_LDPC_MAX_CODE_BLOCKS,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.num_buffers_dst =3D
> +
>=C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_BBDEV_LDPC_MAX_CODE_BLOCKS
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0},
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_BBDEV_END_OF_CAPA= BILITIES_LIST(),
> +=C2=A0 =C2=A0 =C2=A0};
> +
> +=C2=A0 =C2=A0 =C2=A0static struct rte_bbdev_queue_conf default_queue_= conf =3D {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.queue_size =3D RTE_B= BDEV_QUEUE_SIZE_LIMIT,
> +=C2=A0 =C2=A0 =C2=A0};
> +
> +=C2=A0 =C2=A0 =C2=A0default_queue_conf.socket =3D dev->data->so= cket_id;
> +
> +=C2=A0 =C2=A0 =C2=A0dev_info->driver_name =3D RTE_STR(DRIVER_NAME)= ;
> +=C2=A0 =C2=A0 =C2=A0dev_info->max_num_queues =3D ark_bb->max_nb= _queues;
> +=C2=A0 =C2=A0 =C2=A0dev_info->queue_size_lim =3D RTE_BBDEV_QUEUE_S= IZE_LIMIT;
> +=C2=A0 =C2=A0 =C2=A0dev_info->hardware_accelerated =3D true;
> +=C2=A0 =C2=A0 =C2=A0dev_info->max_dl_queue_priority =3D 0;
> +=C2=A0 =C2=A0 =C2=A0dev_info->max_ul_queue_priority =3D 0;
> +=C2=A0 =C2=A0 =C2=A0dev_info->default_queue_conf =3D default_queue= _conf;
> +=C2=A0 =C2=A0 =C2=A0dev_info->capabilities =3D bbdev_capabilities;=
> +=C2=A0 =C2=A0 =C2=A0dev_info->cpu_flag_reqs =3D NULL;
> +=C2=A0 =C2=A0 =C2=A0dev_info->min_alignment =3D 4;
> +
> +}
> +
> +/* Structure defining layout of the ldpc command struct */ struct
> +ark_bb_ldpc_enc_meta {
> +=C2=A0 =C2=A0 =C2=A0uint16_t header;
> +=C2=A0 =C2=A0 =C2=A0uint8_t rv_index:2,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0basegraph:1,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0code_block_mode:1, > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rfu_71_68:4;
> +
> +=C2=A0 =C2=A0 =C2=A0uint8_t q_m;
> +=C2=A0 =C2=A0 =C2=A0uint32_t e_ea;
> +=C2=A0 =C2=A0 =C2=A0uint32_t eb;
> +=C2=A0 =C2=A0 =C2=A0uint8_t c;
> +=C2=A0 =C2=A0 =C2=A0uint8_t cab;
> +=C2=A0 =C2=A0 =C2=A0uint16_t n_cb;
> +=C2=A0 =C2=A0 =C2=A0uint16_t pad;
> +=C2=A0 =C2=A0 =C2=A0uint16_t trailer;
> +} __rte_packed;
> +
> +/* The size must be less then 20 Bytes */ static_assert(sizeof(struct=
> +ark_bb_ldpc_enc_meta) <=3D 20, "struct size");
> +
> +/* Custom operation on equeue ldpc operation=C2=A0 */
> +/* Do these function need queue number? */
> +/* Maximum of 20 bytes */
> +int
> +ark_bb_user_enqueue_ldpc_enc(struct rte_bbdev_enc_op *enc_op,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0uint32_t *meta, uint8_t *meta_cnt) {
> +=C2=A0 =C2=A0 =C2=A0struct rte_bbdev_op_ldpc_enc *ldpc_enc_op =3D &am= p;enc_op->ldpc_enc;
> +=C2=A0 =C2=A0 =C2=A0struct ark_bb_ldpc_enc_meta *src =3D (struct ark_= bb_ldpc_enc_meta
> +*)meta;
> +
> +=C2=A0 =C2=A0 =C2=A0src->header =3D 0x4321;=C2=A0 =C2=A0/* For tes= tings */
> +=C2=A0 =C2=A0 =C2=A0src->trailer =3D 0xFEDC;
> +
> +=C2=A0 =C2=A0 =C2=A0src->rv_index =3D ldpc_enc_op->rv_index; > +=C2=A0 =C2=A0 =C2=A0src->basegraph =3D ldpc_enc_op->basegraph;<= br> > +=C2=A0 =C2=A0 =C2=A0src->code_block_mode =3D ldpc_enc_op->code_= block_mode;
> +
> +=C2=A0 =C2=A0 =C2=A0src->q_m =3D ldpc_enc_op->q_m;
> +=C2=A0 =C2=A0 =C2=A0src->e_ea =3D 0xABCD;
> +=C2=A0 =C2=A0 =C2=A0src->eb =3D ldpc_enc_op->tb_params.eb;
> +=C2=A0 =C2=A0 =C2=A0src->c =3D ldpc_enc_op->tb_params.c;
> +=C2=A0 =C2=A0 =C2=A0src->cab =3D ldpc_enc_op->tb_params.cab; > +
> +=C2=A0 =C2=A0 =C2=A0src->n_cb =3D 0;
> +
> +=C2=A0 =C2=A0 =C2=A0meta[0] =3D 0x11111110;
> +=C2=A0 =C2=A0 =C2=A0meta[1] =3D 0x22222220;
> +=C2=A0 =C2=A0 =C2=A0meta[2] =3D 0x33333330;
> +=C2=A0 =C2=A0 =C2=A0meta[3] =3D 0x44444440;
> +=C2=A0 =C2=A0 =C2=A0meta[4] =3D 0x55555550;
> +
> +=C2=A0 =C2=A0 =C2=A0*meta_cnt =3D ark_bb_cvt_bytes_meta_cnt(
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0sizeof(struct ark_bb_ldpc_enc_meta));
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +/* Custom operation on dequeue ldpc operation=C2=A0 */ int
> +ark_bb_user_dequeue_ldpc_enc(struct rte_bbdev_enc_op *enc_op,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 const uint32_t *usermeta)
> +{
> +=C2=A0 =C2=A0 =C2=A0static int dump;=C2=A0 =C2=A0 =C2=A0 =C2=A0 /* = =3D 0 */
> +=C2=A0 =C2=A0 =C2=A0/* Just compare with what was sent? */
> +=C2=A0 =C2=A0 =C2=A0uint32_t meta_in[5] =3D {0};
> +=C2=A0 =C2=A0 =C2=A0uint8_t=C2=A0 meta_cnt;
> +
> +=C2=A0 =C2=A0 =C2=A0ark_bb_user_enqueue_ldpc_enc(enc_op, meta_in, &am= p;meta_cnt);
> +=C2=A0 =C2=A0 =C2=A0if (memcmp(usermeta, meta_in, 3 + (meta_cnt * 8))= ) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fprintf(stderr,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0"------------------------------------------\n");
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hexdump(stdout, &= quot;meta difference for lpdc_enc IN",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0meta_in, 20);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hexdump(stdout, &= quot;meta difference for lpdc_enc OUT",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0usermeta, 20);
> +=C2=A0 =C2=A0 =C2=A0} else if (dump) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hexdump(stdout, &= quot;DUMP lpdc_enc IN", usermeta, 20);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dump--;
> +=C2=A0 =C2=A0 =C2=A0}
> +
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +
> +/* Turbo op call backs for user meta data */ int
> +ark_bb_user_enqueue_ldpc_dec(struct rte_bbdev_dec_op *enc_op,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 uint32_t *meta, uint8_t *meta_cnt) { > +=C2=A0 =C2=A0 =C2=A0RTE_SET_USED(enc_op);
> +=C2=A0 =C2=A0 =C2=A0meta[0] =3D 0xF1111110;
> +=C2=A0 =C2=A0 =C2=A0meta[1] =3D 0xF2222220;
> +=C2=A0 =C2=A0 =C2=A0meta[2] =3D 0xF3333330;
> +=C2=A0 =C2=A0 =C2=A0meta[3] =3D 0xF4444440;
> +=C2=A0 =C2=A0 =C2=A0meta[4] =3D 0xF5555550;
> +
> +=C2=A0 =C2=A0 =C2=A0*meta_cnt =3D ark_bb_cvt_bytes_meta_cnt(20);
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> +
> +int ark_bb_user_dequeue_ldpc_dec(struct rte_bbdev_dec_op *enc_op,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 const uint32_t *usermeta)
> +{
> +=C2=A0 =C2=A0 =C2=A0RTE_SET_USED(enc_op);
> +=C2=A0 =C2=A0 =C2=A0static int dump;=C2=A0 =C2=A0 =C2=A0 =C2=A0 /* = =3D 0 */
> +=C2=A0 =C2=A0 =C2=A0/* Just compare with what was sent? */
> +=C2=A0 =C2=A0 =C2=A0uint32_t meta_in[5] =3D {0};
> +=C2=A0 =C2=A0 =C2=A0uint8_t=C2=A0 meta_cnt;
> +
> +=C2=A0 =C2=A0 =C2=A0ark_bb_user_enqueue_ldpc_dec(enc_op, meta_in, &am= p;meta_cnt);
> +=C2=A0 =C2=A0 =C2=A0if (memcmp(usermeta, meta_in, 3 + (meta_cnt * 8))= ) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fprintf(stderr,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0"------------------------------------------\n");
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hexdump(stdout, &= quot;meta difference for lpdc_enc IN",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0meta_in, 20);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hexdump(stdout, &= quot;meta difference for lpdc_enc OUT",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0usermeta, 20);
> +=C2=A0 =C2=A0 =C2=A0} else if (dump) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hexdump(stdout, &= quot;DUMP lpdc_enc IN", usermeta, 20);
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dump--;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0return 0;
> +}
> diff --git a/drivers/baseband/ark/ark_bbdev_custom.h
> b/drivers/baseband/ark/ark_bbdev_custom.h
> new file mode 100644
> index 0000000000..32a2ef6bb6
> --- /dev/null
> +++ b/drivers/baseband/ark/ark_bbdev_custom.h
> @@ -0,0 +1,30 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2016-2021 Atomic Rules LLC=C2=A0 */
> +
> +#ifndef _ARK_BBDEV_CUSTOM_H_
> +#define _ARK_BBDEV_CUSTOM_H_
> +
> +#include <stdint.h>
> +
> +/* Forward declarations */
> +struct rte_bbdev;
> +struct rte_bbdev_driver_info;
> +struct rte_bbdev_enc_op;
> +struct rte_bbdev_dec_op;
> +struct rte_mbuf;
> +
> +void ark_bbdev_info_get(struct rte_bbdev *dev,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0struct rte_bbdev_driver_info *dev_info);
> +
> +int ark_bb_user_enqueue_ldpc_dec(struct rte_bbdev_dec_op *enc_op,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 uint32_t *meta, uint8_t *meta_cnt); int=
> +ark_bb_user_dequeue_ldpc_dec(struct rte_bbdev_dec_op *enc_op,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 const uint32_t *usermeta);
> +
> +int ark_bb_user_enqueue_ldpc_enc(struct rte_bbdev_enc_op *enc_op,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 uint32_t *meta, uint8_t *meta_cnt); int=
> +ark_bb_user_dequeue_ldpc_enc(struct rte_bbdev_enc_op *enc_op,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 const uint32_t *usermeta);
> +
> +#endif
> diff --git a/drivers/baseband/ark/meson.build
> b/drivers/baseband/ark/meson.build
> new file mode 100644
> index 0000000000..b876f05c6e
> --- /dev/null
> +++ b/drivers/baseband/ark/meson.build
> @@ -0,0 +1,11 @@
> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca
> +Boccassi <bl= uca@debian.org>
> +
> +deps +=3D ['common_ark', 'bbdev', 'bus_pci', = 'pci', 'ring'] sources =3D
> +files(
> +=C2=A0 =C2=A0 =C2=A0'ark_bbdev.c',
> +=C2=A0 =C2=A0 =C2=A0'ark_bbdev_common.c',
> +=C2=A0 =C2=A0 =C2=A0'ark_bbdev_custom.c'
> +=C2=A0 =C2=A0 =C2=A0)
> +
> +includes +=3D include_directories('../../common/ark')
> diff --git a/drivers/baseband/ark/version.map
> b/drivers/baseband/ark/version.map
> new file mode 100644
> index 0000000000..4a76d1d52d
> --- /dev/null
> +++ b/drivers/baseband/ark/version.map
> @@ -0,0 +1,3 @@
> +DPDK_21 {
> +=C2=A0 =C2=A0 =C2=A0local: *;
> +};
> --
> 2.25.1

--000000000000837e7f05ddb40304--