From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BF60AA0540; Thu, 22 Sep 2022 02:28:34 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D2B8842B6C; Thu, 22 Sep 2022 02:28:16 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 0F751410D0 for ; Thu, 22 Sep 2022 02:28:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663806493; x=1695342493; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YTepv5bRQgjGjE7SN6J5BKhKEjyVkPTX1038AsTfN6g=; b=VPYxDsy4vX4iWxNntFLN7GXxwgkXfaR3KSQofuEaUBePcBxXNw8ZXlrs wy3xMF2YhwBwZ1laacuo10b1nk3NTtbP+NhMO3fJcrntxD6xg2YtrLcrT jRhZ5nLbM3J91Ha92G/XkuS8ZbvITeVIOMODdObDIgF4B7Oc9aXnO3yLK W+sy363kQ0/slSso2DVDV6JbVd9CsEalFKcpvYcWRACa8hH+nsW+u650T +LTcDpb+Rtu2Z/2mZo7xtgGnXaV7VaXa90G3W92tMM3/RjnFR4HmX3li+ WAUy/3rx3HlZOLyn+qziPGK+Ko5+D594k4iu0NU+Wvm3T05mXxewhWLw0 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10477"; a="279883402" X-IronPort-AV: E=Sophos;i="5.93,334,1654585200"; d="scan'208";a="279883402" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Sep 2022 17:28:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,334,1654585200"; d="scan'208";a="615010949" Received: from unknown (HELO icx-npg-scs1-cp1.localdomain) ([10.233.180.245]) by orsmga007.jf.intel.com with ESMTP; 21 Sep 2022 17:28:10 -0700 From: Nic Chautru To: dev@dpdk.org, thomas@monjalon.net Cc: maxime.coquelin@redhat.com, trix@redhat.com, mdr@ashroe.eu, bruce.richardson@intel.com, hemant.agrawal@nxp.com, david.marchand@redhat.com, stephen@networkplumber.org, hernan.vargas@intel.com, Nic Chautru Subject: [PATCH v4 04/14] baseband/acc: introduce PMD for ACC200 Date: Wed, 21 Sep 2022 17:27:30 -0700 Message-Id: <1663806460-45162-5-git-send-email-nicolas.chautru@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663806460-45162-1-git-send-email-nicolas.chautru@intel.com> References: <1663806460-45162-1-git-send-email-nicolas.chautru@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduce stubs for device driver for the ACC200 integrated VRAN accelerator on SPR-EEC Signed-off-by: Nic Chautru --- MAINTAINERS | 3 + doc/guides/bbdevs/acc200.rst | 244 ++++++++++++++++++++++++++++++++++ doc/guides/bbdevs/index.rst | 1 + drivers/baseband/acc/acc200_pmd.h | 32 +++++ drivers/baseband/acc/meson.build | 2 +- drivers/baseband/acc/rte_acc200_pmd.c | 142 ++++++++++++++++++++ 6 files changed, 423 insertions(+), 1 deletion(-) create mode 100644 doc/guides/bbdevs/acc200.rst create mode 100644 drivers/baseband/acc/acc200_pmd.h create mode 100644 drivers/baseband/acc/rte_acc200_pmd.c diff --git a/MAINTAINERS b/MAINTAINERS index 52afd5d..909c855 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1340,6 +1340,9 @@ F: drivers/baseband/acc/ F: doc/guides/bbdevs/acc100.rst F: doc/guides/bbdevs/features/acc100.ini F: doc/guides/bbdevs/features/acc101.ini +F: drivers/baseband/acc200/ +F: doc/guides/bbdevs/acc200.rst +F: doc/guides/bbdevs/features/acc200.ini Null baseband M: Nicolas Chautru diff --git a/doc/guides/bbdevs/acc200.rst b/doc/guides/bbdevs/acc200.rst new file mode 100644 index 0000000..c5e1bd7 --- /dev/null +++ b/doc/guides/bbdevs/acc200.rst @@ -0,0 +1,244 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2022 Intel Corporation + +Intel(R) ACC200 vRAN Dedicated Accelerator Poll Mode Driver +=========================================================== + +The Intel® vRAN Dedicated Accelerator ACC200 peripheral enables cost-effective 4G +and 5G next-generation virtualized Radio Access Network (vRAN) solutions integrated on +Sapphire Rapids EEC Intel(R)7 based Xeon(R) multi-core Serverprocessor. + +Features +-------- + +The ACC200 includes a 5G Low Density Parity Check (LDPC) encoder/decoder, rate match/dematch, +Hybrid Automatic Repeat Request (HARQ) with access to DDR memory for buffer management, a 4G +Turbo encoder/decoder, a Fast Fourier Transform (FFT) block providing DFT/iDFT processing offload +for the 5G Sounding Reference Signal (SRS), a Queue Manager (QMGR), and a DMA subsystem. +There is no dedicated on-card memory for HARQ, this is using coherent memory on the CPU side. + +These correspond to the following features exposed by the PMD: + +- LDPC Encode in the Downlink (5GNR) +- LDPC Decode in the Uplink (5GNR) +- Turbo Encode in the Downlink (4G) +- Turbo Decode in the Uplink (4G) +- FFT processing +- SR-IOV with 16 VFs per PF +- Maximum of 256 queues per VF +- MSI + +ACC200 PMD supports the following BBDEV capabilities: + +* For the LDPC encode operation: + - ``RTE_BBDEV_LDPC_CRC_24B_ATTACH`` : set to attach CRC24B to CB(s) + - ``RTE_BBDEV_LDPC_RATE_MATCH`` : if set then do not do Rate Match bypass + - ``RTE_BBDEV_LDPC_INTERLEAVER_BYPASS`` : if set then bypass interleaver + +* For the LDPC decode operation: + - ``RTE_BBDEV_LDPC_CRC_TYPE_24B_CHECK`` : check CRC24B from CB(s) + - ``RTE_BBDEV_LDPC_CRC_TYPE_24B_DROP`` : drops CRC24B bits appended while decoding + - ``RTE_BBDEV_LDPC_CRC_TYPE_24A_CHECK`` : check CRC24A from CB(s) + - ``RTE_BBDEV_LDPC_CRC_TYPE_16_CHECK`` : check CRC16 from CB(s) + - ``RTE_BBDEV_LDPC_HQ_COMBINE_IN_ENABLE`` : provides an input for HARQ combining + - ``RTE_BBDEV_LDPC_HQ_COMBINE_OUT_ENABLE`` : provides an input for HARQ combining + - ``RTE_BBDEV_LDPC_ITERATION_STOP_ENABLE`` : disable early termination + - ``RTE_BBDEV_LDPC_DEC_SCATTER_GATHER`` : supports scatter-gather for input/output data + - ``RTE_BBDEV_LDPC_HARQ_6BIT_COMPRESSION`` : supports compression of the HARQ input/output + - ``RTE_BBDEV_LDPC_LLR_COMPRESSION`` : supports LLR input compression + +* For the turbo encode operation: + - ``RTE_BBDEV_TURBO_CRC_24B_ATTACH`` : set to attach CRC24B to CB(s) + - ``RTE_BBDEV_TURBO_RATE_MATCH`` : if set then do not do Rate Match bypass + - ``RTE_BBDEV_TURBO_ENC_INTERRUPTS`` : set for encoder dequeue interrupts + - ``RTE_BBDEV_TURBO_RV_INDEX_BYPASS`` : set to bypass RV index + - ``RTE_BBDEV_TURBO_ENC_SCATTER_GATHER`` : supports scatter-gather for input/output data + +* For the turbo decode operation: + - ``RTE_BBDEV_TURBO_CRC_TYPE_24B`` : check CRC24B from CB(s) + - ``RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE`` : perform subblock de-interleave + - ``RTE_BBDEV_TURBO_DEC_INTERRUPTS`` : set for decoder dequeue interrupts + - ``RTE_BBDEV_TURBO_NEG_LLR_1_BIT_IN`` : set if negative LLR input is supported + - ``RTE_BBDEV_TURBO_DEC_TB_CRC_24B_KEEP`` : keep CRC24B bits appended while decoding + - ``RTE_BBDEV_TURBO_DEC_CRC_24B_DROP`` : option to drop the code block CRC after decoding + - ``RTE_BBDEV_TURBO_EARLY_TERMINATION`` : set early termination feature + - ``RTE_BBDEV_TURBO_DEC_SCATTER_GATHER`` : supports scatter-gather for input/output data + - ``RTE_BBDEV_TURBO_HALF_ITERATION_EVEN`` : set half iteration granularity + - ``RTE_BBDEV_TURBO_SOFT_OUTPUT`` : set the APP LLR soft output + - ``RTE_BBDEV_TURBO_EQUALIZER`` : set the turbo equalizer feature + - ``RTE_BBDEV_TURBO_SOFT_OUT_SATURATE`` : set the soft output saturation + - ``RTE_BBDEV_TURBO_CONTINUE_CRC_MATCH`` : set to run an extra odd iteration after CRC match + - ``RTE_BBDEV_TURBO_NEG_LLR_1_BIT_SOFT_OUT`` : set if negative APP LLR output supported + - ``RTE_BBDEV_TURBO_MAP_DEC`` : supports flexible parallel MAP engine decoding + +Installation +------------ + +Section 3 of the DPDK manual provides instructions on installing and compiling DPDK. + +DPDK requires hugepages to be configured as detailed in section 2 of the DPDK manual. +The bbdev test application has been tested with a configuration 40 x 1GB hugepages. The +hugepage configuration of a server may be examined using: + +.. code-block:: console + + grep Huge* /proc/meminfo + + +Initialization +-------------- + +When the device first powers up, its PCI Physical Functions (PF) can be listed through these +commands for ACC200: + +.. code-block:: console + + sudo lspci -vd8086:57c0 + +The physical and virtual functions are compatible with Linux UIO drivers: +``vfio`` and ``igb_uio``. However, in order to work the 5G/4G +FEC device first needs to be bound to one of these linux drivers through DPDK. + + +Bind PF UIO driver(s) +~~~~~~~~~~~~~~~~~~~~~ + +Install the DPDK igb_uio driver, bind it with the PF PCI device ID and use +``lspci`` to confirm the PF device is under use by ``igb_uio`` DPDK UIO driver. + +The igb_uio driver may be bound to the PF PCI device using one of two methods for ACC200: + + +1. PCI functions (physical or virtual, depending on the use case) can be bound to +the UIO driver by repeating this command for every function. + +.. code-block:: console + + cd + insmod ./build/kmod/igb_uio.ko + echo "8086 57c0" > /sys/bus/pci/drivers/igb_uio/new_id + lspci -vd8086:57c0 + + +2. Another way to bind PF with DPDK UIO driver is by using the ``dpdk-devbind.py`` tool + +.. code-block:: console + + cd + ./usertools/dpdk-devbind.py -b igb_uio 0000:f7:00.0 + +where the PCI device ID (example: 0000:f7:00.0) is obtained using lspci -vd8086:57c0 + + +In a similar way the PF may be bound with vfio-pci as any PCIe device. + + +Enable Virtual Functions +~~~~~~~~~~~~~~~~~~~~~~~~ + +Now, it should be visible in the printouts that PCI PF is under igb_uio control +"``Kernel driver in use: igb_uio``" + +To show the number of available VFs on the device, read ``sriov_totalvfs`` file.. + +.. code-block:: console + + cat /sys/bus/pci/devices/0000\:\:./sriov_totalvfs + + where 0000\:\:. is the PCI device ID + + +To enable VFs via igb_uio, echo the number of virtual functions intended to +enable to ``max_vfs`` file.. + +.. code-block:: console + + echo > /sys/bus/pci/devices/0000\:\:./max_vfs + + +Afterwards, all VFs must be bound to appropriate UIO drivers as required, same +way it was done with the physical function previously. + +Enabling SR-IOV via vfio driver is pretty much the same, except that the file +name is different: + +.. code-block:: console + + echo > /sys/bus/pci/devices/0000\:\:./sriov_numvfs + + +Configure the VFs through PF +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The PCI virtual functions must be configured before working or getting assigned +to VMs/Containers. The configuration involves allocating the number of hardware +queues, priorities, load balance, bandwidth and other settings necessary for the +device to perform FEC functions. + +This configuration needs to be executed at least once after reboot or PCI FLR and can +be achieved by using the functions ``rte_acc200_configure()``, +which sets up the parameters defined in the compatible ``acc200_conf`` structure. + +Test Application +---------------- + +BBDEV provides a test application, ``test-bbdev.py`` and range of test data for testing +the functionality of the device, depending on the device's +capabilities. The test application is located under app->test-bbdev folder and has the +following options: + +.. code-block:: console + + "-p", "--testapp-path": specifies path to the bbdev test app. + "-e", "--eal-params" : EAL arguments which are passed to the test app. + "-t", "--timeout" : Timeout in seconds (default=300). + "-c", "--test-cases" : Defines test cases to run. Run all if not specified. + "-v", "--test-vector" : Test vector path (default=dpdk_path+/app/test-bbdev/test_vectors/bbdev_null.data). + "-n", "--num-ops" : Number of operations to process on device (default=32). + "-b", "--burst-size" : Operations enqueue/dequeue burst size (default=32). + "-s", "--snr" : SNR in dB used when generating LLRs for bler tests. + "-s", "--iter_max" : Number of iterations for LDPC decoder. + "-l", "--num-lcores" : Number of lcores to run (default=16). + "-i", "--init-device" : Initialise PF device with default values. + + +To execute the test application tool using simple decode or encode data, +type one of the following: + +.. code-block:: console + + ./test-bbdev.py -c validation -n 64 -b 1 -v ./ldpc_dec_default.data + ./test-bbdev.py -c validation -n 64 -b 1 -v ./ldpc_enc_default.data + + +The test application ``test-bbdev.py``, supports the ability to configure the PF device with +a default set of values, if the "-i" or "- -init-device" option is included. The default values +are defined in test_bbdev_perf.c. + + +Test Vectors +~~~~~~~~~~~~ + +In addition to the simple LDPC decoder and LDPC encoder tests, bbdev also provides +a range of additional tests under the test_vectors folder, which may be useful. The results +of these tests will depend on the device capabilities which may cause some +testcases to be skipped, but no failure should be reported. + + +Alternate Baseband Device configuration tool +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +On top of the embedded configuration feature supported in test-bbdev using "- -init-device" +option mentioned above, there is also a tool available to perform that device configuration +using a companion application. +The ``pf_bb_config`` application notably enables then to run bbdev-test from the VF +and not only limited to the PF as captured above. + +See for more details: https://github.com/intel/pf-bb-config + +Specifically for the BBDEV ACC200 PMD, the command below can be used: + +.. code-block:: console + + ./pf_bb_config ACC200 -c ./acc200/acc200_config_vf_5g.cfg + ./test-bbdev.py -e="-c 0xff0 -a${VF_PCI_ADDR}" -c validation -n 64 -b 64 -l 1 -v ./ldpc_dec_default.data diff --git a/doc/guides/bbdevs/index.rst b/doc/guides/bbdevs/index.rst index cedd706..4e9dea8 100644 --- a/doc/guides/bbdevs/index.rst +++ b/doc/guides/bbdevs/index.rst @@ -14,4 +14,5 @@ Baseband Device Drivers fpga_lte_fec fpga_5gnr_fec acc100 + acc200 la12xx diff --git a/drivers/baseband/acc/acc200_pmd.h b/drivers/baseband/acc/acc200_pmd.h new file mode 100644 index 0000000..aaa6b77 --- /dev/null +++ b/drivers/baseband/acc/acc200_pmd.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#ifndef _RTE_ACC200_PMD_H_ +#define _RTE_ACC200_PMD_H_ + +#include "acc_common.h" + +/* Helper macro for logging */ +#define rte_bbdev_log(level, fmt, ...) \ + rte_log(RTE_LOG_ ## level, acc200_logtype, fmt "\n", \ + ##__VA_ARGS__) + +#ifdef RTE_LIBRTE_BBDEV_DEBUG +#define rte_bbdev_log_debug(fmt, ...) \ + rte_bbdev_log(DEBUG, "acc200_pmd: " fmt, \ + ##__VA_ARGS__) +#else +#define rte_bbdev_log_debug(fmt, ...) +#endif + +/* ACC200 PF and VF driver names */ +#define ACC200PF_DRIVER_NAME intel_acc200_pf +#define ACC200VF_DRIVER_NAME intel_acc200_vf + +/* ACC200 PCI vendor & device IDs */ +#define RTE_ACC200_VENDOR_ID (0x8086) +#define RTE_ACC200_PF_DEVICE_ID (0x57C0) +#define RTE_ACC200_VF_DEVICE_ID (0x57C1) + +#endif /* _RTE_ACC200_PMD_H_ */ diff --git a/drivers/baseband/acc/meson.build b/drivers/baseband/acc/meson.build index 9a1a3b8..63912f0 100644 --- a/drivers/baseband/acc/meson.build +++ b/drivers/baseband/acc/meson.build @@ -3,6 +3,6 @@ deps += ['bbdev', 'bus_vdev', 'ring', 'pci', 'bus_pci'] -sources = files('rte_acc100_pmd.c') +sources = files('rte_acc100_pmd.c', 'rte_acc200_pmd.c') headers = files('rte_acc100_cfg.h') diff --git a/drivers/baseband/acc/rte_acc200_pmd.c b/drivers/baseband/acc/rte_acc200_pmd.c new file mode 100644 index 0000000..db8b641 --- /dev/null +++ b/drivers/baseband/acc/rte_acc200_pmd.c @@ -0,0 +1,142 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#ifdef RTE_BBDEV_OFFLOAD_COST +#include +#endif + +#include +#include +#include "acc200_pmd.h" + +#ifdef RTE_LIBRTE_BBDEV_DEBUG +RTE_LOG_REGISTER_DEFAULT(acc200_logtype, DEBUG); +#else +RTE_LOG_REGISTER_DEFAULT(acc200_logtype, NOTICE); +#endif + +/* Free memory used for software rings */ +static int +acc200_dev_close(struct rte_bbdev *dev) +{ + RTE_SET_USED(dev); + return 0; +} + + +static const struct rte_bbdev_ops acc200_bbdev_ops = { + .close = acc200_dev_close, +}; + +/* ACC200 PCI PF address map */ +static struct rte_pci_id pci_id_acc200_pf_map[] = { + { + RTE_PCI_DEVICE(RTE_ACC200_VENDOR_ID, RTE_ACC200_PF_DEVICE_ID) + }, + {.device_id = 0}, +}; + +/* ACC200 PCI VF address map */ +static struct rte_pci_id pci_id_acc200_vf_map[] = { + { + RTE_PCI_DEVICE(RTE_ACC200_VENDOR_ID, RTE_ACC200_VF_DEVICE_ID) + }, + {.device_id = 0}, +}; + +/* Initialization Function */ +static void +acc200_bbdev_init(struct rte_bbdev *dev, struct rte_pci_driver *drv) +{ + struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev->device); + + dev->dev_ops = &acc200_bbdev_ops; + + ((struct acc_device *) dev->data->dev_private)->pf_device = + !strcmp(drv->driver.name, + RTE_STR(ACC200PF_DRIVER_NAME)); + ((struct acc_device *) dev->data->dev_private)->mmio_base = + pci_dev->mem_resource[0].addr; + + rte_bbdev_log_debug("Init device %s [%s] @ vaddr %p paddr %#"PRIx64"", + drv->driver.name, dev->data->name, + (void *)pci_dev->mem_resource[0].addr, + pci_dev->mem_resource[0].phys_addr); +} + +static int acc200_pci_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct rte_bbdev *bbdev = NULL; + char dev_name[RTE_BBDEV_NAME_MAX_LEN]; + + if (pci_dev == NULL) { + rte_bbdev_log(ERR, "NULL PCI device"); + return -EINVAL; + } + + rte_pci_device_name(&pci_dev->addr, dev_name, sizeof(dev_name)); + + /* Allocate memory to be used privately by drivers */ + bbdev = rte_bbdev_allocate(pci_dev->device.name); + if (bbdev == NULL) + return -ENODEV; + + /* allocate device private memory */ + bbdev->data->dev_private = rte_zmalloc_socket(dev_name, + sizeof(struct acc_device), RTE_CACHE_LINE_SIZE, + pci_dev->device.numa_node); + + if (bbdev->data->dev_private == NULL) { + rte_bbdev_log(CRIT, + "Allocate of %zu bytes for device \"%s\" failed", + sizeof(struct acc_device), dev_name); + rte_bbdev_release(bbdev); + return -ENOMEM; + } + + /* Fill HW specific part of device structure */ + bbdev->device = &pci_dev->device; + bbdev->intr_handle = pci_dev->intr_handle; + bbdev->data->socket_id = pci_dev->device.numa_node; + + /* Invoke ACC200 device initialization function */ + acc200_bbdev_init(bbdev, pci_drv); + + rte_bbdev_log_debug("Initialised bbdev %s (id = %u)", + dev_name, bbdev->data->dev_id); + return 0; +} + +static struct rte_pci_driver acc200_pci_pf_driver = { + .probe = acc200_pci_probe, + .remove = acc_pci_remove, + .id_table = pci_id_acc200_pf_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING +}; + +static struct rte_pci_driver acc200_pci_vf_driver = { + .probe = acc200_pci_probe, + .remove = acc_pci_remove, + .id_table = pci_id_acc200_vf_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING +}; + +RTE_PMD_REGISTER_PCI(ACC200PF_DRIVER_NAME, acc200_pci_pf_driver); +RTE_PMD_REGISTER_PCI_TABLE(ACC200PF_DRIVER_NAME, pci_id_acc200_pf_map); +RTE_PMD_REGISTER_PCI(ACC200VF_DRIVER_NAME, acc200_pci_vf_driver); +RTE_PMD_REGISTER_PCI_TABLE(ACC200VF_DRIVER_NAME, pci_id_acc200_vf_map); -- 1.8.3.1