From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id BB3CD1BB16 for ; Thu, 10 May 2018 16:24:18 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 May 2018 07:24:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,385,1520924400"; d="scan'208";a="38893930" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga008.fm.intel.com with ESMTP; 10 May 2018 07:24:17 -0700 Received: from fmsmsx124.amr.corp.intel.com (10.18.125.39) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 10 May 2018 07:24:16 -0700 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx124.amr.corp.intel.com (10.18.125.39) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 10 May 2018 07:24:16 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.210]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.40]) with mapi id 14.03.0319.002; Thu, 10 May 2018 22:24:14 +0800 From: "Zhang, Qi Z" To: "Xu, Rosen" , "dev@dpdk.org" , "thomas@monjalon.net" CC: "Xu, Rosen" , "Zhang, Roy Fan" , "Doherty, Declan" , "Richardson, Bruce" , "shreyansh.jain@nxp.com" , "Yigit, Ferruh" , "Ananyev, Konstantin" , "Zhang, Tianfei" , "Liu, Song" , "Wu, Hao" , "gaetan.rivet@6wind.com" , "Wu, Yanglong" Thread-Topic: [dpdk-dev] [PATCH v10 3/3] iFPGA: Add Intel FPGA BUS Rawdev Driver Thread-Index: AQHT52lcB+SNdw0oS06NzLx3e/BcUaQo8YRA Date: Thu, 10 May 2018 14:24:13 +0000 Message-ID: <039ED4275CED7440929022BC67E70611531B0C78@SHSMSX103.ccr.corp.intel.com> References: <1521553556-62982-1-git-send-email-rosen.xu@intel.com> <1525851801-16101-1-git-send-email-rosen.xu@intel.com> <1525851801-16101-4-git-send-email-rosen.xu@intel.com> In-Reply-To: <1525851801-16101-4-git-send-email-rosen.xu@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZGEyYjg1MzItYTM1ZS00Y2I1LWI4NTItNDRmOWM5MTExYmVhIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjIuNS4xOCIsIlRydXN0ZWRMYWJlbEhhc2giOiJ6bGQrb2Z4cGxiT1NXZXN5U3hZc0NMK1wvWmp0bllmY2FDbTdGcURrQU41OE1SeEhcL3pRQnZaXC9Gc0VMMjZ0cmFSIn0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.200.100 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v10 3/3] iFPGA: Add Intel FPGA BUS Rawdev Driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 May 2018 14:24:20 -0000 Hi Rosen: After read this patch, I have below suggestion 1. rte_ifgpa_device is not necessary, since we already have ifgpa bus, it = manages all devices on that bus, we don't need another abstraction to manag= e afu devices, and user can add to or remove from the bus. 2. rename rte_afu_device to rte_ifpga_device , we follow the name rule. 3. data structure looks like below. Struct rte_ifpga_deivce { rte_device device /* backing device */ Rte_raw_device *device /* link to raw pci device */ Struct Rte_afu_info info; /* afu information */ } struct rte_afu_info { struct rte_ifpga_device *ifpga_dev; /**< Point ifpga device */ struct rte_afu_id id; /**< AFU id within FPGA. */ uint32_t num_region; /**< number of regions found */ .... ... } Regards Qi > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Xu, Rosen > Sent: Wednesday, May 9, 2018 3:43 PM > To: dev@dpdk.org; thomas@monjalon.net > Cc: Xu, Rosen ; Zhang, Roy Fan > ; Doherty, Declan ; > Richardson, Bruce ; shreyansh.jain@nxp.com; > Yigit, Ferruh ; Ananyev, Konstantin > ; Zhang, Tianfei ; > Liu, Song ; Wu, Hao ; > gaetan.rivet@6wind.com; Wu, Yanglong > Subject: [dpdk-dev] [PATCH v10 3/3] iFPGA: Add Intel FPGA BUS Rawdev Driv= er >=20 > From: Rosen Xu >=20 > Add Intel FPGA BUS Rawdev Driver which is based on librte_rawdev library. >=20 > Signed-off-by: Rosen Xu > Signed-off-by: Yanglong Wu > Signed-off-by: Tianfei Zhang > Acked-by: Shreyansh Jain > --- > MAINTAINERS | 3 + > config/common_base | 5 + > doc/guides/rawdevs/ifpga_rawdev.rst | 112 ++++ > doc/guides/rawdevs/index.rst | 1 + > doc/guides/rel_notes/release_18_05.rst | 8 + > drivers/raw/Makefile | 1 + > drivers/raw/ifpga_rawdev/Makefile | 36 ++ > drivers/raw/ifpga_rawdev/ifpga_rawdev.c | 608 > +++++++++++++++++++++ > drivers/raw/ifpga_rawdev/ifpga_rawdev.h | 37 ++ > drivers/raw/ifpga_rawdev/meson.build | 15 + > .../ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map | 4 + > drivers/raw/meson.build | 2 +- > mk/rte.app.mk | 4 +- > 13 files changed, 834 insertions(+), 2 deletions(-) create mode 100644 > doc/guides/rawdevs/ifpga_rawdev.rst > create mode 100644 drivers/raw/ifpga_rawdev/Makefile create mode > 100644 drivers/raw/ifpga_rawdev/ifpga_rawdev.c > create mode 100644 drivers/raw/ifpga_rawdev/ifpga_rawdev.h > create mode 100644 drivers/raw/ifpga_rawdev/meson.build > create mode 100644 > drivers/raw/ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map >=20 > diff --git a/MAINTAINERS b/MAINTAINERS > index 9c27d5a..a93ce9f 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -851,8 +851,11 @@ Rawdev Drivers > -------------- >=20 > Intel FPGA > +M: Rosen Xu > M: Tianfei zhang > +F: drivers/raw/ifpga_rawdev/ > F: drivers/raw/ifpga_rawdev/base/ > +F: doc/guides/rawdevs/ifpga_rawdev.rst >=20 > NXP DPAA2 QDMA > M: Nipun Gupta > diff --git a/config/common_base b/config/common_base index > 1440316..017a15a 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -633,6 +633,11 @@ > CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV=3Dn > CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV=3Dn >=20 > # > +# Compile PMD for Intel FPGA raw device # > +CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV=3Dy > + > +# > # Compile librte_ring > # > CONFIG_RTE_LIBRTE_RING=3Dy > diff --git a/doc/guides/rawdevs/ifpga_rawdev.rst > b/doc/guides/rawdevs/ifpga_rawdev.rst > new file mode 100644 > index 0000000..37ae4cc > --- /dev/null > +++ b/doc/guides/rawdevs/ifpga_rawdev.rst > @@ -0,0 +1,112 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2018 Intel Corporation. > + > +IFPGA Rawdev Driver > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +FPGA is used more and more widely in Cloud and NFV, one primary reason > +is that FPGA not only provides ASIC performance but also it's more > +flexible than ASIC. > + > +FPGA uses Partial Reconfigure (PR) Parts of Bit Stream to achieve its > +flexibility. That means one FPGA Device Bit Stream is divided into many > +Parts of Bit Stream(each Part of Bit Stream is defined as > +AFU-Accelerated Function Unit), and each AFU is a hardware acceleration > +unit which can be dynamically reloaded respectively. > + > +By PR (Partial Reconfiguration) AFUs, one FPGA resources can be > +time-shared by different users. FPGA hot upgrade and fault tolerance can= be > provided easily. > + > +The SW IFPGA Rawdev Driver (**ifpga_rawdev**) provides a Rawdev driver > +that utilizes Intel FPGA Software Stack OPAE(Open Programmable > +Acceleration > +Engine) for FPGA management. > + > +Implementation details > +---------------------- > + > +Each instance of IFPGA Rawdev Driver is probed by Intel FpgaDev. In > +coordination with OPAE share code IFPGA Rawdev Driver provides common > +FPGA management ops for FPGA operation, OPAE provides all following > operations: > +- FPGA PR (Partial Reconfiguration) management > +- FPGA AFUs Identifying > +- FPGA Thermal Management > +- FPGA Power Management > +- FPGA Performance reporting > +- FPGA Remote Debug > + > +All configuration parameters are taken by vdev_ifpga_cfg driver. > +Besides configuration, vdev_ifpga_cfg driver also hot plugs in IFPGA Bus= . > + > +All of the AFUs of one FPGA may share same PCI BDF and AFUs scan depend > +on IFPGA Rawdev Driver so IFPGA Bus takes AFU device scan and AFU driver= s > probe. > +All AFU device driver bind to AFU device by its UUID (Universally > +Unique Identifier). > + > +To avoid unnecessary code duplication and ensure maximum performance, > +handling of AFU devices is left to different PMDs; all the design as > +summarized by the following block diagram:: > + > + +---------------------------------------------------------------+ > + | Application(s) > | > + +----------------------------.----------------------------------+ > + | > + | > + +----------------------------'----------------------------------+ > + | DPDK Framework (APIs) > | > + +----------|------------|--------.---------------------|--------+ > + / \ | > + / \ | > + +-------'-------+ +-------'-------+ +--------'--------+ > + | Eth PMD | | Crypto PMD | | > | > + +-------.-------+ +-------.-------+ | | > + | | | > | > + | | | > | > + +-------'-------+ +-------'-------+ | IFPGA | > + | Eth AFU Dev | |Crypto AFU Dev | | Rawdev Driver > | > + +-------.-------+ +-------.-------+ |(OPAE Share Code)| > + | | | > | > + | | Rawdev | > | > + +-------'------------------'-------+ Ops | | > + | IFPGA Bus | -------->| > | > + +-----------------.----------------+ +--------.--------+ > + | | > + Hot-plugin -->| | > + | | > + +-----------------'------------------+ +--------'--------+ > + | vdev_ifpga_cfg driver | | Intel FpgaDev > | > + +------------------------------------+ +-----------------+ > + > +Build options > +------------- > + > +- ``CONFIG_RTE_LIBRTE_IFPGA_BUS`` (default ``y``) > + > + Toggle compilation of IFPGA Bus library. > + > +- ``CONFIG_RTE_LIBRTE_IFPGA_RAWDEV`` (default ``y``) > + > + Toggle compilation of the ``ifpga_rawdev`` driver. > + > +Run-time parameters > +------------------- > + > +This driver is invoked automatically in systems added with Intel FPGA, > +but PR and IFPGA Bus scan is trigged by command line using ``--vdev > +'net_ifpga_cfg`` EAL option. > + > +The following device parameters are supported: > + > +- ``ifpga`` [string] > + > + Provide a specific Intel FPGA device PCI BDF. Can be provided > + multiple times for additional instances. > + > +- ``port`` [int] > + > + Each FPGA can provide many channels to PR AFU by software, each > + channels is identified by this parameter. > + > +- ``afu_bts`` [string] > + > + If null, the AFU Bit Stream has been PR in FPGA, if not forces PR and > + identifies AFU Bit Stream file. > diff --git a/doc/guides/rawdevs/index.rst b/doc/guides/rawdevs/index.rst > index 7769083..7c3bd95 100644 > --- a/doc/guides/rawdevs/index.rst > +++ b/doc/guides/rawdevs/index.rst > @@ -13,3 +13,4 @@ application through rawdev API. >=20 > dpaa2_cmdif > dpaa2_qdma > + ifpga_rawdev > diff --git a/doc/guides/rel_notes/release_18_05.rst > b/doc/guides/rel_notes/release_18_05.rst > index 265950a..f5241a1 100644 > --- a/doc/guides/rel_notes/release_18_05.rst > +++ b/doc/guides/rel_notes/release_18_05.rst > @@ -189,6 +189,14 @@ New Features > the DPDK framework. It provides Intel FPGA Partial Bit Stream > AFU(Accelerated > Function Unit) scan and drivers prove. >=20 > +* **Added IFPGA(Intel FPGA) Rawdev Driver.** > + > + Added a new Rawdev driver called IFPGA(Intel FPGA) Rawdev Driver, > + which cooperates with OPAE(Open Programmable Acceleration Engine) > + share code provides common FPGA management ops for FPGA operation. > + > + See the :doc:`../rawdevs/ifpga_rawdev` programmer's guide for more > details. > + > API Changes > ----------- >=20 > diff --git a/drivers/raw/Makefile b/drivers/raw/Makefile index > 2eb2787..8e29b4a 100644 > --- a/drivers/raw/Makefile > +++ b/drivers/raw/Makefile > @@ -9,5 +9,6 @@ ifeq > ($(CONFIG_RTE_EAL_VFIO)$(CONFIG_RTE_LIBRTE_FSLMC_BUS),yy) > DIRS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV) +=3D dpaa2_cmdif > DIRS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV) +=3D dpaa2_qdma > endif > +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV) +=3D ifpga_rawdev >=20 > include $(RTE_SDK)/mk/rte.subdir.mk > diff --git a/drivers/raw/ifpga_rawdev/Makefile > b/drivers/raw/ifpga_rawdev/Makefile > new file mode 100644 > index 0000000..f3b9d5e > --- /dev/null > +++ b/drivers/raw/ifpga_rawdev/Makefile > @@ -0,0 +1,36 @@ > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel > +Corporation > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +# > +# library name > +# > +LIB =3D librte_pmd_ifpga_rawdev.a > + > +CFLAGS +=3D -DALLOW_EXPERIMENTAL_API > +CFLAGS +=3D -O3 > +CFLAGS +=3D $(WERROR_FLAGS) > +CFLAGS +=3D -I$(RTE_SDK)/drivers/bus/ifpga CFLAGS +=3D > +-I$(RTE_SDK)/drivers/raw/ifpga_rawdev > +LDLIBS +=3D -lrte_eal > +LDLIBS +=3D -lrte_rawdev > +LDLIBS +=3D -lrte_bus_vdev > +LDLIBS +=3D -lrte_kvargs > +LDLIBS +=3D -lrte_bus_pci > +LDLIBS +=3D -lrte_bus_ifpga > + > +EXPORT_MAP :=3D rte_pmd_ifpga_rawdev_version.map > + > +LIBABIVER :=3D 1 > + > +VPATH +=3D $(SRCDIR)/base > + > +include $(RTE_SDK)/drivers/raw/ifpga_rawdev/base/Makefile > + > +# > +# all source are stored in SRCS-y > +# > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV) +=3D ifpga_rawdev.c > + > +include $(RTE_SDK)/mk/rte.lib.mk > diff --git a/drivers/raw/ifpga_rawdev/ifpga_rawdev.c > b/drivers/raw/ifpga_rawdev/ifpga_rawdev.c > new file mode 100644 > index 0000000..32e811c > --- /dev/null > +++ b/drivers/raw/ifpga_rawdev/ifpga_rawdev.c > @@ -0,0 +1,608 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2010-2018 Intel Corporation */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "base/opae_hw_api.h" > +#include "rte_rawdev.h" > +#include "rte_rawdev_pmd.h" > +#include "rte_bus_ifpga.h" > +#include "ifpga_common.h" > +#include "ifpga_logs.h" > +#include "ifpga_rawdev.h" > + > +int ifpga_rawdev_logtype; > + > +#define PCI_VENDOR_ID_INTEL 0x8086 > +/* PCI Device ID */ > +#define PCIE_DEVICE_ID_PF_INT_5_X 0xBCBD > +#define PCIE_DEVICE_ID_PF_INT_6_X 0xBCC0 > +#define PCIE_DEVICE_ID_PF_DSC_1_X 0x09C4 > +/* VF Device */ > +#define PCIE_DEVICE_ID_VF_INT_5_X 0xBCBF > +#define PCIE_DEVICE_ID_VF_INT_6_X 0xBCC1 > +#define PCIE_DEVICE_ID_VF_DSC_1_X 0x09C5 > +#define RTE_MAX_RAW_DEVICE 10 > + > +static const struct rte_pci_id pci_ifpga_map[] =3D { > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_PF_INT_5_X) }, > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_VF_INT_5_X) }, > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_PF_INT_6_X) }, > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_VF_INT_6_X) }, > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_PF_DSC_1_X) }, > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_VF_DSC_1_X) }, > + { .vendor_id =3D 0, /* sentinel */ }, > +}; > + > +static int > +ifpga_fill_afu_dev(struct opae_accelerator *acc, > + struct rte_afu_device *afu_dev) > +{ > + struct rte_mem_resource *res =3D afu_dev->mem_resource; > + struct opae_acc_region_info region_info; > + struct opae_acc_info info; > + unsigned long i; > + int ret; > + > + ret =3D opae_acc_get_info(acc, &info); > + if (ret) > + return ret; > + > + if (info.num_regions > PCI_MAX_RESOURCE) > + return -EFAULT; > + > + afu_dev->num_region =3D info.num_regions; > + > + for (i =3D 0; i < info.num_regions; i++) { > + region_info.index =3D i; > + ret =3D opae_acc_get_region_info(acc, ®ion_info); > + if (ret) > + return ret; > + > + if ((region_info.flags & ACC_REGION_MMIO) && > + (region_info.flags & ACC_REGION_READ) && > + (region_info.flags & ACC_REGION_WRITE)) { > + res[i].phys_addr =3D region_info.phys_addr; > + res[i].len =3D region_info.len; > + res[i].addr =3D region_info.addr; > + } else > + return -EFAULT; > + } > + > + return 0; > +} > + > +static void > +ifpga_rawdev_info_get(struct rte_rawdev *dev, > + rte_rawdev_obj_t dev_info) > +{ > + struct opae_adapter *adapter; > + struct opae_accelerator *acc; > + struct rte_afu_device *afu_dev; > + > + IFPGA_RAWDEV_PMD_FUNC_TRACE(); > + > + if (!dev_info) { > + IFPGA_RAWDEV_PMD_ERR("Invalid request"); > + return; > + } > + > + adapter =3D ifpga_rawdev_get_priv(dev); > + if (!adapter) > + return; > + > + afu_dev =3D dev_info; > + afu_dev->rawdev =3D dev; > + > + /* find opae_accelerator and fill info into afu_device */ > + opae_adapter_for_each_acc(adapter, acc) { > + if (acc->index !=3D afu_dev->id.port) > + continue; > + > + if (ifpga_fill_afu_dev(acc, afu_dev)) { > + IFPGA_RAWDEV_PMD_ERR("cannot get info\n"); > + return; > + } > + } > +} > + > +static int > +ifpga_rawdev_start(struct rte_rawdev *dev) { > + int ret =3D 0; > + struct opae_adapter *adapter; > + > + IFPGA_RAWDEV_PMD_FUNC_TRACE(); > + > + RTE_FUNC_PTR_OR_ERR_RET(dev, -EINVAL); > + > + adapter =3D ifpga_rawdev_get_priv(dev); > + if (!adapter) > + return -ENODEV; > + > + return ret; > +} > + > +static void > +ifpga_rawdev_stop(struct rte_rawdev *dev) { > + dev->started =3D 0; > +} > + > +static int > +ifpga_rawdev_close(struct rte_rawdev *dev) { > + return dev ? 0:1; > +} > + > +static int > +ifpga_rawdev_reset(struct rte_rawdev *dev) { > + return dev ? 0:1; > +} > + > +static int > +fpga_pr(struct rte_rawdev *raw_dev, u32 port_id, u64 *buffer, u32 size, > + u64 *status) > +{ > + > + struct opae_adapter *adapter; > + struct opae_manager *mgr; > + struct opae_accelerator *acc; > + struct opae_bridge *br; > + int ret; > + > + adapter =3D ifpga_rawdev_get_priv(raw_dev); > + if (!adapter) > + return -ENODEV; > + > + mgr =3D opae_adapter_get_mgr(adapter); > + if (!mgr) > + return -ENODEV; > + > + acc =3D opae_adapter_get_acc(adapter, port_id); > + if (!acc) > + return -ENODEV; > + > + br =3D opae_acc_get_br(acc); > + if (!br) > + return -ENODEV; > + > + ret =3D opae_manager_flash(mgr, port_id, buffer, size, status); > + if (ret) { > + IFPGA_RAWDEV_PMD_ERR("%s pr error %d\n", __func__, ret); > + return ret; > + } > + > + ret =3D opae_bridge_reset(br); > + if (ret) { > + IFPGA_RAWDEV_PMD_ERR("%s reset port:%d error %d\n", > + __func__, port_id, ret); > + return ret; > + } > + > + return ret; > +} > + > +static int > +rte_fpga_do_pr(struct rte_rawdev *rawdev, int port_id, > + const char *file_name) > +{ > + struct stat file_stat; > + int file_fd; > + int ret =3D 0; > + u32 buffer_size; > + void *buffer; > + u64 pr_error; > + > + if (!file_name) > + return -EINVAL; > + > + file_fd =3D open(file_name, O_RDONLY); > + if (file_fd < 0) { > + IFPGA_RAWDEV_PMD_ERR("%s: open file error: %s\n", > + __func__, file_name); > + IFPGA_RAWDEV_PMD_ERR("Message : %s\n", strerror(errno)); > + return -EINVAL; > + } > + ret =3D stat(file_name, &file_stat); > + if (ret) { > + IFPGA_RAWDEV_PMD_ERR("stat on bitstream file failed: %s\n", > + file_name); > + return -EINVAL; > + } > + buffer_size =3D file_stat.st_size; > + IFPGA_RAWDEV_PMD_INFO("bitstream file size: %u\n", buffer_size); > + buffer =3D rte_malloc(NULL, buffer_size, 0); > + if (!buffer) { > + ret =3D -ENOMEM; > + goto close_fd; > + } > + > + /*read the raw data*/ > + if (buffer_size !=3D read(file_fd, (void *)buffer, buffer_size)) { > + ret =3D -EINVAL; > + goto free_buffer; > + } > + > + /*do PR now*/ > + ret =3D fpga_pr(rawdev, port_id, buffer, buffer_size, &pr_error); > + IFPGA_RAWDEV_PMD_INFO("downloading to device port %d....%s.\n", > port_id, > + ret ? "failed" : "success"); > + if (ret) { > + ret =3D -EINVAL; > + goto free_buffer; > + } > + > +free_buffer: > + if (buffer) > + rte_free(buffer); > +close_fd: > + close(file_fd); > + file_fd =3D 0; > + return ret; > +} > + > +static int > +ifpga_rawdev_pr(struct rte_rawdev *dev, > + rte_rawdev_obj_t pr_conf) > +{ > + struct opae_adapter *adapter; > + struct rte_afu_pr_conf *afu_pr_conf; > + int ret; > + struct uuid uuid; > + struct opae_accelerator *acc; > + > + IFPGA_RAWDEV_PMD_FUNC_TRACE(); > + > + adapter =3D ifpga_rawdev_get_priv(dev); > + if (!adapter) > + return -ENODEV; > + > + if (!pr_conf) > + return -EINVAL; > + > + afu_pr_conf =3D pr_conf; > + > + if (afu_pr_conf->pr_enable) { > + ret =3D rte_fpga_do_pr(dev, > + afu_pr_conf->afu_id.port, > + afu_pr_conf->bs_path); > + if (ret) { > + IFPGA_RAWDEV_PMD_ERR("do pr error %d\n", ret); > + return ret; > + } > + } > + > + acc =3D opae_adapter_get_acc(adapter, afu_pr_conf->afu_id.port); > + if (!acc) > + return -ENODEV; > + > + ret =3D opae_acc_get_uuid(acc, &uuid); > + if (ret) > + return ret; > + > + memcpy(&afu_pr_conf->afu_id.uuid.uuid_low, uuid.b, sizeof(u64)); > + memcpy(&afu_pr_conf->afu_id.uuid.uuid_high, uuid.b + 8, sizeof(u64)); > + > + IFPGA_RAWDEV_PMD_INFO("%s: uuid_l=3D0x%lx, uuid_h=3D0x%lx\n", > __func__, > + (u64)afu_pr_conf->afu_id.uuid.uuid_low, > + (u64)afu_pr_conf->afu_id.uuid.uuid_high); > + > + return 0; > +} > + > +static const struct rte_rawdev_ops ifpga_rawdev_ops =3D { > + .dev_info_get =3D ifpga_rawdev_info_get, > + .dev_configure =3D NULL, > + .dev_start =3D ifpga_rawdev_start, > + .dev_stop =3D ifpga_rawdev_stop, > + .dev_close =3D ifpga_rawdev_close, > + .dev_reset =3D ifpga_rawdev_reset, > + > + .queue_def_conf =3D NULL, > + .queue_setup =3D NULL, > + .queue_release =3D NULL, > + > + .attr_get =3D NULL, > + .attr_set =3D NULL, > + > + .enqueue_bufs =3D NULL, > + .dequeue_bufs =3D NULL, > + > + .dump =3D NULL, > + > + .xstats_get =3D NULL, > + .xstats_get_names =3D NULL, > + .xstats_get_by_name =3D NULL, > + .xstats_reset =3D NULL, > + > + .firmware_status_get =3D NULL, > + .firmware_version_get =3D NULL, > + .firmware_load =3D ifpga_rawdev_pr, > + .firmware_unload =3D NULL, > + > + .dev_selftest =3D NULL, > +}; > + > +static int > +ifpga_rawdev_create(struct rte_pci_device *pci_dev, > + int socket_id) > +{ > + int ret =3D 0; > + struct rte_rawdev *rawdev =3D NULL; > + struct opae_adapter *adapter =3D NULL; > + struct opae_manager *mgr =3D NULL; > + struct opae_adapter_data_pci *data =3D NULL; > + char name[RTE_RAWDEV_NAME_MAX_LEN]; > + int i; > + > + if (!pci_dev) { > + IFPGA_RAWDEV_PMD_ERR("Invalid pci_dev of the device!"); > + ret =3D -EINVAL; > + goto cleanup; > + } > + > + memset(name, 0, sizeof(name)); > + snprintf(name, RTE_RAWDEV_NAME_MAX_LEN, "IFPGA:%x:%02x.%x", > + pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); > + > + IFPGA_RAWDEV_PMD_INFO("Init %s on NUMA node %d", name, > +rte_socket_id()); > + > + /* Allocate device structure */ > + rawdev =3D rte_rawdev_pmd_allocate(name, sizeof(struct opae_adapter), > + socket_id); > + if (rawdev =3D=3D NULL) { > + IFPGA_RAWDEV_PMD_ERR("Unable to allocate rawdevice"); > + ret =3D -EINVAL; > + goto cleanup; > + } > + > + /* alloc OPAE_FPGA_PCI data to register to OPAE hardware level API */ > + data =3D opae_adapter_data_alloc(OPAE_FPGA_PCI); > + if (!data) { > + ret =3D -ENOMEM; > + goto cleanup; > + } > + > + /* init opae_adapter_data_pci for device specific information */ > + for (i =3D 0; i < PCI_MAX_RESOURCE; i++) { > + data->region[i].phys_addr =3D pci_dev->mem_resource[i].phys_addr; > + data->region[i].len =3D pci_dev->mem_resource[i].len; > + data->region[i].addr =3D pci_dev->mem_resource[i].addr; > + } > + data->device_id =3D pci_dev->id.device_id; > + data->vendor_id =3D pci_dev->id.vendor_id; > + > + /* create a opae_adapter based on above device data */ > + adapter =3D opae_adapter_alloc(pci_dev->device.name, data); > + if (!adapter) { > + ret =3D -ENOMEM; > + goto free_adapter_data; > + } > + > + rawdev->dev_ops =3D &ifpga_rawdev_ops; > + rawdev->device =3D &pci_dev->device; > + rawdev->driver_name =3D pci_dev->device.driver->name; > + > + rawdev->dev_private =3D adapter; > + > + /* must enumerate the adapter before use it */ > + ret =3D opae_adapter_enumerate(adapter); > + if (ret) > + goto free_adapter; > + > + /* get opae_manager to rawdev */ > + mgr =3D opae_adapter_get_mgr(adapter); > + if (mgr) { > + /* PF function */ > + IFPGA_RAWDEV_PMD_INFO("this is a PF function"); > + } > + > + return ret; > + > +free_adapter: > + if (adapter) > + opae_adapter_free(adapter); > +free_adapter_data: > + if (data) > + opae_adapter_data_free(data); > +cleanup: > + if (rawdev) > + rte_rawdev_pmd_release(rawdev); > + > + return ret; > +} > + > +static int > +ifpga_rawdev_destroy(struct rte_pci_device *pci_dev) { > + int ret; > + struct rte_rawdev *rawdev; > + char name[RTE_RAWDEV_NAME_MAX_LEN]; > + struct opae_adapter *adapter; > + > + if (!pci_dev) { > + IFPGA_RAWDEV_PMD_ERR("Invalid pci_dev of the device!"); > + ret =3D -EINVAL; > + return ret; > + } > + > + memset(name, 0, sizeof(name)); > + snprintf(name, RTE_RAWDEV_NAME_MAX_LEN, "IFPGA:%x:%02x.%x", > + pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); > + > + IFPGA_RAWDEV_PMD_INFO("Closing %s on NUMA node %d", > + name, rte_socket_id()); > + > + rawdev =3D rte_rawdev_pmd_get_named_dev(name); > + if (!rawdev) { > + IFPGA_RAWDEV_PMD_ERR("Invalid device name (%s)", name); > + return -EINVAL; > + } > + > + adapter =3D ifpga_rawdev_get_priv(rawdev); > + if (!adapter) > + return -ENODEV; > + > + opae_adapter_data_free(adapter->data); > + opae_adapter_free(adapter); > + > + /* rte_rawdev_close is called by pmd_release */ > + ret =3D rte_rawdev_pmd_release(rawdev); > + if (ret) > + IFPGA_RAWDEV_PMD_DEBUG("Device cleanup failed"); > + > + return ret; > +} > + > +static int > +ifpga_rawdev_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, > + struct rte_pci_device *pci_dev) > +{ > + > + IFPGA_RAWDEV_PMD_FUNC_TRACE(); > + return ifpga_rawdev_create(pci_dev, rte_socket_id()); } > + > +static int > +ifpga_rawdev_pci_remove(struct rte_pci_device *pci_dev) { > + return ifpga_rawdev_destroy(pci_dev); > +} > + > +static struct rte_pci_driver rte_ifpga_rawdev_pmd =3D { > + .id_table =3D pci_ifpga_map, > + .drv_flags =3D RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, > + .probe =3D ifpga_rawdev_pci_probe, > + .remove =3D ifpga_rawdev_pci_remove, > +}; > + > +RTE_PMD_REGISTER_PCI(ifpga_rawdev_pci_driver, rte_ifpga_rawdev_pmd); > +RTE_PMD_REGISTER_PCI_TABLE(ifpga_rawdev_pci_driver, > +rte_ifpga_rawdev_pmd); > +RTE_PMD_REGISTER_KMOD_DEP(ifpga_rawdev_pci_driver, "* igb_uio | > +uio_pci_generic | vfio-pci"); > + > +RTE_INIT(ifpga_rawdev_init_log); > +static void > +ifpga_rawdev_init_log(void) > +{ > + ifpga_rawdev_logtype =3D rte_log_register("driver.raw.init"); > + if (ifpga_rawdev_logtype >=3D 0) > + rte_log_set_level(ifpga_rawdev_logtype, RTE_LOG_NOTICE); } > + > +static const char * const valid_args[] =3D { > +#define IFPGA_ARG_NAME "ifpga" > + IFPGA_ARG_NAME, > +#define IFPGA_ARG_PORT "port" > + IFPGA_ARG_PORT, > +#define IFPGA_AFU_BTS "afu_bts" > + IFPGA_AFU_BTS, > + NULL > +}; > + > +static int > +ifpga_cfg_probe(struct rte_vdev_device *dev) { > + struct rte_devargs *devargs; > + struct rte_kvargs *kvlist =3D NULL; > + int port; > + char *name =3D NULL; > + char dev_name[RTE_RAWDEV_NAME_MAX_LEN]; > + > + devargs =3D dev->device.devargs; > + > + kvlist =3D rte_kvargs_parse(devargs->args, valid_args); > + if (!kvlist) { > + IFPGA_RAWDEV_PMD_LOG(ERR, "error when parsing param"); > + goto end; > + } > + > + if (rte_kvargs_count(kvlist, IFPGA_ARG_NAME) =3D=3D 1) { > + if (rte_kvargs_process(kvlist, IFPGA_ARG_NAME, > + &rte_ifpga_get_string_arg, &name) < 0) { > + IFPGA_RAWDEV_PMD_ERR("error to parse %s", > + IFPGA_ARG_NAME); > + goto end; > + } > + } else { > + IFPGA_RAWDEV_PMD_ERR("arg %s is mandatory for ifpga bus", > + IFPGA_ARG_NAME); > + goto end; > + } > + > + if (rte_kvargs_count(kvlist, IFPGA_ARG_PORT) =3D=3D 1) { > + if (rte_kvargs_process(kvlist, > + IFPGA_ARG_PORT, > + &rte_ifpga_get_integer32_arg, > + &port) < 0) { > + IFPGA_RAWDEV_PMD_ERR("error to parse %s", > + IFPGA_ARG_PORT); > + goto end; > + } > + } else { > + IFPGA_RAWDEV_PMD_ERR("arg %s is mandatory for ifpga bus", > + IFPGA_ARG_PORT); > + goto end; > + } > + > + memset(dev_name, 0, sizeof(dev_name)); > + snprintf(dev_name, RTE_RAWDEV_NAME_MAX_LEN, "%d|%s", > + port, name); > + > + rte_eal_hotplug_add(RTE_STR(IFPGA_BUS_NAME), > + dev_name, devargs->args); > +end: > + if (kvlist) > + rte_kvargs_free(kvlist); > + if (name) > + free(name); > + > + return 0; > +} > + > +static int > +ifpga_cfg_remove(struct rte_vdev_device *vdev) { > + IFPGA_RAWDEV_PMD_INFO("Remove ifpga_cfg %p", > + vdev); > + > + return 0; > +} > + > +static struct rte_vdev_driver ifpga_cfg_driver =3D { > + .probe =3D ifpga_cfg_probe, > + .remove =3D ifpga_cfg_remove, > +}; > + > +RTE_PMD_REGISTER_VDEV(net_ifpga_cfg, ifpga_cfg_driver); > +RTE_PMD_REGISTER_ALIAS(net_ifpga_cfg, ifpga_cfg); > +RTE_PMD_REGISTER_PARAM_STRING(net_ifpga_cfg, > + "bdf=3D " > + "port=3D " > + "afu_bts=3D"); > + > diff --git a/drivers/raw/ifpga_rawdev/ifpga_rawdev.h > b/drivers/raw/ifpga_rawdev/ifpga_rawdev.h > new file mode 100644 > index 0000000..c7759b8 > --- /dev/null > +++ b/drivers/raw/ifpga_rawdev/ifpga_rawdev.h > @@ -0,0 +1,37 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2010-2018 Intel Corporation */ > + > +#ifndef _IFPGA_RAWDEV_H_ > +#define _IFPGA_RAWDEV_H_ > + > +extern int ifpga_rawdev_logtype; > + > +#define IFPGA_RAWDEV_PMD_LOG(level, fmt, args...) \ > + rte_log(RTE_LOG_ ## level, ifpga_rawdev_logtype, "ifgpa: " fmt, \ > + ##args) > + > +#define IFPGA_RAWDEV_PMD_FUNC_TRACE() > IFPGA_RAWDEV_PMD_LOG(DEBUG, ">>") > + > +#define IFPGA_RAWDEV_PMD_DEBUG(fmt, args...) \ > + IFPGA_RAWDEV_PMD_LOG(DEBUG, fmt, ## args) #define > +IFPGA_RAWDEV_PMD_INFO(fmt, args...) \ > + IFPGA_RAWDEV_PMD_LOG(INFO, fmt, ## args) #define > +IFPGA_RAWDEV_PMD_ERR(fmt, args...) \ > + IFPGA_RAWDEV_PMD_LOG(ERR, fmt, ## args) #define > +IFPGA_RAWDEV_PMD_WARN(fmt, args...) \ > + IFPGA_RAWDEV_PMD_LOG(WARNING, fmt, ## args) > + > +enum ifpga_rawdev_device_state { > + IFPGA_IDLE, > + IFPGA_READY, > + IFPGA_ERROR > +}; > + > +static inline struct opae_adapter * > +ifpga_rawdev_get_priv(const struct rte_rawdev *rawdev) { > + return rawdev->dev_private; > +} > + > +#endif /* _IFPGA_RAWDEV_H_ */ > diff --git a/drivers/raw/ifpga_rawdev/meson.build > b/drivers/raw/ifpga_rawdev/meson.build > new file mode 100644 > index 0000000..6725687 > --- /dev/null > +++ b/drivers/raw/ifpga_rawdev/meson.build > @@ -0,0 +1,15 @@ > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel > +Corporation > + > +version =3D 1 > + > +subdir('base') > +objs =3D [base_objs] > + > +deps +=3D ['rawdev', 'pci', 'bus_pci', 'kvargs', > + 'bus_vdev', 'bus_ifpga'] > +sources =3D files('ifpga_rawdev.c') > + > +includes +=3D include_directories('base') > + > +allow_experimental_apis =3D true > diff --git a/drivers/raw/ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map > b/drivers/raw/ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map > new file mode 100644 > index 0000000..9b9ab1a > --- /dev/null > +++ b/drivers/raw/ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map > @@ -0,0 +1,4 @@ > +DPDK_18.05 { > + > + local: *; > +}; > diff --git a/drivers/raw/meson.build b/drivers/raw/meson.build index > 34dfac4..a61cdcc 100644 > --- a/drivers/raw/meson.build > +++ b/drivers/raw/meson.build > @@ -1,7 +1,7 @@ > # SPDX-License-Identifier: BSD-3-Clause # Copyright 2018 NXP >=20 > -drivers =3D ['skeleton_rawdev', 'dpaa2_cmdif', 'dpaa2_qdma'] > +drivers =3D ['skeleton_rawdev', 'dpaa2_cmdif', 'dpaa2_qdma', > +'ifpga_rawdev'] > std_deps =3D ['rawdev'] > config_flag_fmt =3D 'RTE_LIBRTE_PMD_@0@_RAWDEV' > driver_name_fmt =3D 'rte_pmd_@0@' > diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 3861e1a..acc0ec3 100644 > --- a/mk/rte.app.mk > +++ b/mk/rte.app.mk > @@ -256,9 +256,11 @@ > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV) +=3D > -lrte_pmd_dpaa2_cmdif > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV) +=3D > -lrte_pmd_dpaa2_qdma endif # CONFIG_RTE_LIBRTE_FSLMC_BUS > _LDLIBS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) +=3D -lrte_bus_ifpga > +ifeq ($(CONFIG_RTE_LIBRTE_IFPGA_BUS),y) > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV) +=3D > -lrte_pmd_ifpga_rawdev > +endif # CONFIG_RTE_LIBRTE_IFPGA_BUS > endif # CONFIG_RTE_LIBRTE_RAWDEV >=20 > - > endif # !CONFIG_RTE_BUILD_SHARED_LIBS >=20 > _LDLIBS-y +=3D --no-whole-archive > -- > 1.8.3.1