From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 394AA1BDA7 for ; Fri, 11 May 2018 05:17:04 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 May 2018 20:17:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,387,1520924400"; d="scan'208";a="223347694" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga005.jf.intel.com with ESMTP; 10 May 2018 20:17:03 -0700 Received: from fmsmsx116.amr.corp.intel.com (10.18.116.20) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 10 May 2018 20:17:03 -0700 Received: from shsmsx102.ccr.corp.intel.com (10.239.4.154) by fmsmsx116.amr.corp.intel.com (10.18.116.20) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 10 May 2018 20:17:02 -0700 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.240]) by shsmsx102.ccr.corp.intel.com ([169.254.2.79]) with mapi id 14.03.0319.002; Fri, 11 May 2018 11:16:59 +0800 From: "Xu, Rosen" To: "Zhang, Qi Z" , "dev@dpdk.org" , "thomas@monjalon.net" CC: "Zhang, Roy Fan" , "Doherty, Declan" , "Richardson, Bruce" , "shreyansh.jain@nxp.com" , "Yigit, Ferruh" , "Ananyev, Konstantin" , "Zhang, Tianfei" , "Liu, Song" , "Wu, Hao" , "gaetan.rivet@6wind.com" , "Wu, Yanglong" Thread-Topic: [dpdk-dev] [PATCH v10 3/3] iFPGA: Add Intel FPGA BUS Rawdev Driver Thread-Index: AQHT52lcqqdO46jUqUu/f0QOmr1npqQo8YRAgADq7XA= Date: Fri, 11 May 2018 03:16:58 +0000 Message-ID: <0E78D399C70DA940A335608C6ED296D739FD3D62@SHSMSX104.ccr.corp.intel.com> References: <1521553556-62982-1-git-send-email-rosen.xu@intel.com> <1525851801-16101-1-git-send-email-rosen.xu@intel.com> <1525851801-16101-4-git-send-email-rosen.xu@intel.com> <039ED4275CED7440929022BC67E70611531B0C78@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <039ED4275CED7440929022BC67E70611531B0C78@SHSMSX103.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZGEyYjg1MzItYTM1ZS00Y2I1LWI4NTItNDRmOWM5MTExYmVhIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjIuNS4xOCIsIlRydXN0ZWRMYWJlbEhhc2giOiJ6bGQrb2Z4cGxiT1NXZXN5U3hZc0NMK1wvWmp0bllmY2FDbTdGcURrQU41OE1SeEhcL3pRQnZaXC9Gc0VMMjZ0cmFSIn0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.200.100 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v10 3/3] iFPGA: Add Intel FPGA BUS Rawdev Driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 May 2018 03:17:06 -0000 Hi Qi, > -----Original Message----- > From: Zhang, Qi Z > Sent: Thursday, May 10, 2018 22:24 > To: Xu, Rosen ; dev@dpdk.org; thomas@monjalon.net > Cc: Xu, Rosen ; Zhang, Roy Fan > ; Doherty, Declan ; > Richardson, Bruce ; shreyansh.jain@nxp.com; > Yigit, Ferruh ; Ananyev, Konstantin > ; Zhang, Tianfei ; > Liu, Song ; Wu, Hao ; > gaetan.rivet@6wind.com; Wu, Yanglong > Subject: RE: [dpdk-dev] [PATCH v10 3/3] iFPGA: Add Intel FPGA BUS Rawdev > Driver >=20 > Hi Rosen: >=20 > After read this patch, I have below suggestion >=20 > 1. rte_ifgpa_device is not necessary, since we already have ifgpa bus, > it manages all devices on that bus, we don't need another abstraction to > manage afu devices, and user can add to or remove from the bus. Remove no used definition of rte_ifgpa_device > 2. rename rte_afu_device to rte_ifpga_device , we follow the name > rule. AFU is partial bit stream of FPGA, so AFU and FPGA are different definition= . But we define global variables: ifpga_afu_dev_list > 3. data structure looks like below. >=20 > Struct rte_ifpga_deivce { > rte_device device /* backing device */ > Rte_raw_device *device /* link to raw pci device */ > Struct Rte_afu_info info; /* afu information */ > } >=20 > struct rte_afu_info { > struct rte_ifpga_device *ifpga_dev; /**< Point ifpga device */ > struct rte_afu_id id; /**< AFU id within FPGA. = */ > uint32_t num_region; /**< number of regions found */ > .... > ... > } For there is not ifpga definition in our patch, so I still keep the definit= ion of rte_afu_dev. Is it ok? > Regards > Qi >=20 > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Xu, Rosen > > Sent: Wednesday, May 9, 2018 3:43 PM > > To: dev@dpdk.org; thomas@monjalon.net > > Cc: Xu, Rosen ; Zhang, Roy Fan > > ; Doherty, Declan ; > > Richardson, Bruce ; > > shreyansh.jain@nxp.com; Yigit, Ferruh ; > > Ananyev, Konstantin ; Zhang, Tianfei > > ; Liu, Song ; Wu, Hao > > ; gaetan.rivet@6wind.com; Wu, Yanglong > > > > Subject: [dpdk-dev] [PATCH v10 3/3] iFPGA: Add Intel FPGA BUS Rawdev > > Driver > > > > From: Rosen Xu > > > > Add Intel FPGA BUS Rawdev Driver which is based on librte_rawdev librar= y. > > > > Signed-off-by: Rosen Xu > > Signed-off-by: Yanglong Wu > > Signed-off-by: Tianfei Zhang > > Acked-by: Shreyansh Jain > > --- > > MAINTAINERS | 3 + > > config/common_base | 5 + > > doc/guides/rawdevs/ifpga_rawdev.rst | 112 ++++ > > doc/guides/rawdevs/index.rst | 1 + > > doc/guides/rel_notes/release_18_05.rst | 8 + > > drivers/raw/Makefile | 1 + > > drivers/raw/ifpga_rawdev/Makefile | 36 ++ > > drivers/raw/ifpga_rawdev/ifpga_rawdev.c | 608 > > +++++++++++++++++++++ > > drivers/raw/ifpga_rawdev/ifpga_rawdev.h | 37 ++ > > drivers/raw/ifpga_rawdev/meson.build | 15 + > > .../ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map | 4 + > > drivers/raw/meson.build | 2 +- > > mk/rte.app.mk | 4 +- > > 13 files changed, 834 insertions(+), 2 deletions(-) create mode > > 100644 doc/guides/rawdevs/ifpga_rawdev.rst > > create mode 100644 drivers/raw/ifpga_rawdev/Makefile create mode > > 100644 drivers/raw/ifpga_rawdev/ifpga_rawdev.c > > create mode 100644 drivers/raw/ifpga_rawdev/ifpga_rawdev.h > > create mode 100644 drivers/raw/ifpga_rawdev/meson.build > > create mode 100644 > > drivers/raw/ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map > > > > diff --git a/MAINTAINERS b/MAINTAINERS index 9c27d5a..a93ce9f 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -851,8 +851,11 @@ Rawdev Drivers > > -------------- > > > > Intel FPGA > > +M: Rosen Xu > > M: Tianfei zhang > > +F: drivers/raw/ifpga_rawdev/ > > F: drivers/raw/ifpga_rawdev/base/ > > +F: doc/guides/rawdevs/ifpga_rawdev.rst > > > > NXP DPAA2 QDMA > > M: Nipun Gupta > > diff --git a/config/common_base b/config/common_base index > > 1440316..017a15a 100644 > > --- a/config/common_base > > +++ b/config/common_base > > @@ -633,6 +633,11 @@ > > CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV=3Dn > > CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV=3Dn > > > > # > > +# Compile PMD for Intel FPGA raw device # > > +CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV=3Dy > > + > > +# > > # Compile librte_ring > > # > > CONFIG_RTE_LIBRTE_RING=3Dy > > diff --git a/doc/guides/rawdevs/ifpga_rawdev.rst > > b/doc/guides/rawdevs/ifpga_rawdev.rst > > new file mode 100644 > > index 0000000..37ae4cc > > --- /dev/null > > +++ b/doc/guides/rawdevs/ifpga_rawdev.rst > > @@ -0,0 +1,112 @@ > > +.. SPDX-License-Identifier: BSD-3-Clause > > + Copyright(c) 2018 Intel Corporation. > > + > > +IFPGA Rawdev Driver > > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > + > > +FPGA is used more and more widely in Cloud and NFV, one primary > > +reason is that FPGA not only provides ASIC performance but also it's > > +more flexible than ASIC. > > + > > +FPGA uses Partial Reconfigure (PR) Parts of Bit Stream to achieve its > > +flexibility. That means one FPGA Device Bit Stream is divided into > > +many Parts of Bit Stream(each Part of Bit Stream is defined as > > +AFU-Accelerated Function Unit), and each AFU is a hardware > > +acceleration unit which can be dynamically reloaded respectively. > > + > > +By PR (Partial Reconfiguration) AFUs, one FPGA resources can be > > +time-shared by different users. FPGA hot upgrade and fault tolerance > > +can be > > provided easily. > > + > > +The SW IFPGA Rawdev Driver (**ifpga_rawdev**) provides a Rawdev > > +driver that utilizes Intel FPGA Software Stack OPAE(Open Programmable > > +Acceleration > > +Engine) for FPGA management. > > + > > +Implementation details > > +---------------------- > > + > > +Each instance of IFPGA Rawdev Driver is probed by Intel FpgaDev. In > > +coordination with OPAE share code IFPGA Rawdev Driver provides > common > > +FPGA management ops for FPGA operation, OPAE provides all following > > operations: > > +- FPGA PR (Partial Reconfiguration) management > > +- FPGA AFUs Identifying > > +- FPGA Thermal Management > > +- FPGA Power Management > > +- FPGA Performance reporting > > +- FPGA Remote Debug > > + > > +All configuration parameters are taken by vdev_ifpga_cfg driver. > > +Besides configuration, vdev_ifpga_cfg driver also hot plugs in IFPGA B= us. > > + > > +All of the AFUs of one FPGA may share same PCI BDF and AFUs scan > > +depend on IFPGA Rawdev Driver so IFPGA Bus takes AFU device scan and > > +AFU drivers > > probe. > > +All AFU device driver bind to AFU device by its UUID (Universally > > +Unique Identifier). > > + > > +To avoid unnecessary code duplication and ensure maximum > performance, > > +handling of AFU devices is left to different PMDs; all the design as > > +summarized by the following block diagram:: > > + > > + +---------------------------------------------------------------+ > > + | Application(s) > > | > > + +----------------------------.----------------------------------+ > > + | > > + | > > + +----------------------------'----------------------------------+ > > + | DPDK Framework (APIs) > > | > > + +----------|------------|--------.---------------------|--------+ > > + / \ | > > + / \ | > > + +-------'-------+ +-------'-------+ +--------'--------+ > > + | Eth PMD | | Crypto PMD | | > > | > > + +-------.-------+ +-------.-------+ | | > > + | | | > > | > > + | | | > > | > > + +-------'-------+ +-------'-------+ | IFPGA | > > + | Eth AFU Dev | |Crypto AFU Dev | | Rawdev Driver > > | > > + +-------.-------+ +-------.-------+ |(OPAE Share Code)| > > + | | | > > | > > + | | Rawdev | > > | > > + +-------'------------------'-------+ Ops | | > > + | IFPGA Bus | -------->| > > | > > + +-----------------.----------------+ +--------.--------+ > > + | | > > + Hot-plugin -->| | > > + | | > > + +-----------------'------------------+ +--------'--------+ > > + | vdev_ifpga_cfg driver | | Intel FpgaDev > > | > > + +------------------------------------+ +-----------------+ > > + > > +Build options > > +------------- > > + > > +- ``CONFIG_RTE_LIBRTE_IFPGA_BUS`` (default ``y``) > > + > > + Toggle compilation of IFPGA Bus library. > > + > > +- ``CONFIG_RTE_LIBRTE_IFPGA_RAWDEV`` (default ``y``) > > + > > + Toggle compilation of the ``ifpga_rawdev`` driver. > > + > > +Run-time parameters > > +------------------- > > + > > +This driver is invoked automatically in systems added with Intel > > +FPGA, but PR and IFPGA Bus scan is trigged by command line using > > +``--vdev 'net_ifpga_cfg`` EAL option. > > + > > +The following device parameters are supported: > > + > > +- ``ifpga`` [string] > > + > > + Provide a specific Intel FPGA device PCI BDF. Can be provided > > + multiple times for additional instances. > > + > > +- ``port`` [int] > > + > > + Each FPGA can provide many channels to PR AFU by software, each > > + channels is identified by this parameter. > > + > > +- ``afu_bts`` [string] > > + > > + If null, the AFU Bit Stream has been PR in FPGA, if not forces PR > > + and identifies AFU Bit Stream file. > > diff --git a/doc/guides/rawdevs/index.rst > > b/doc/guides/rawdevs/index.rst index 7769083..7c3bd95 100644 > > --- a/doc/guides/rawdevs/index.rst > > +++ b/doc/guides/rawdevs/index.rst > > @@ -13,3 +13,4 @@ application through rawdev API. > > > > dpaa2_cmdif > > dpaa2_qdma > > + ifpga_rawdev > > diff --git a/doc/guides/rel_notes/release_18_05.rst > > b/doc/guides/rel_notes/release_18_05.rst > > index 265950a..f5241a1 100644 > > --- a/doc/guides/rel_notes/release_18_05.rst > > +++ b/doc/guides/rel_notes/release_18_05.rst > > @@ -189,6 +189,14 @@ New Features > > the DPDK framework. It provides Intel FPGA Partial Bit Stream > > AFU(Accelerated > > Function Unit) scan and drivers prove. > > > > +* **Added IFPGA(Intel FPGA) Rawdev Driver.** > > + > > + Added a new Rawdev driver called IFPGA(Intel FPGA) Rawdev Driver, > > + which cooperates with OPAE(Open Programmable Acceleration Engine) > > + share code provides common FPGA management ops for FPGA operation. > > + > > + See the :doc:`../rawdevs/ifpga_rawdev` programmer's guide for more > > details. > > + > > API Changes > > ----------- > > > > diff --git a/drivers/raw/Makefile b/drivers/raw/Makefile index > > 2eb2787..8e29b4a 100644 > > --- a/drivers/raw/Makefile > > +++ b/drivers/raw/Makefile > > @@ -9,5 +9,6 @@ ifeq > > ($(CONFIG_RTE_EAL_VFIO)$(CONFIG_RTE_LIBRTE_FSLMC_BUS),yy) > > DIRS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV) +=3D > dpaa2_cmdif > > DIRS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV) +=3D > dpaa2_qdma endif > > +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV) +=3D ifpga_rawdev > > > > include $(RTE_SDK)/mk/rte.subdir.mk > > diff --git a/drivers/raw/ifpga_rawdev/Makefile > > b/drivers/raw/ifpga_rawdev/Makefile > > new file mode 100644 > > index 0000000..f3b9d5e > > --- /dev/null > > +++ b/drivers/raw/ifpga_rawdev/Makefile > > @@ -0,0 +1,36 @@ > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel > > +Corporation > > + > > +include $(RTE_SDK)/mk/rte.vars.mk > > + > > +# > > +# library name > > +# > > +LIB =3D librte_pmd_ifpga_rawdev.a > > + > > +CFLAGS +=3D -DALLOW_EXPERIMENTAL_API > > +CFLAGS +=3D -O3 > > +CFLAGS +=3D $(WERROR_FLAGS) > > +CFLAGS +=3D -I$(RTE_SDK)/drivers/bus/ifpga CFLAGS +=3D > > +-I$(RTE_SDK)/drivers/raw/ifpga_rawdev > > +LDLIBS +=3D -lrte_eal > > +LDLIBS +=3D -lrte_rawdev > > +LDLIBS +=3D -lrte_bus_vdev > > +LDLIBS +=3D -lrte_kvargs > > +LDLIBS +=3D -lrte_bus_pci > > +LDLIBS +=3D -lrte_bus_ifpga > > + > > +EXPORT_MAP :=3D rte_pmd_ifpga_rawdev_version.map > > + > > +LIBABIVER :=3D 1 > > + > > +VPATH +=3D $(SRCDIR)/base > > + > > +include $(RTE_SDK)/drivers/raw/ifpga_rawdev/base/Makefile > > + > > +# > > +# all source are stored in SRCS-y > > +# > > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV) +=3D ifpga_rawdev.c > > + > > +include $(RTE_SDK)/mk/rte.lib.mk > > diff --git a/drivers/raw/ifpga_rawdev/ifpga_rawdev.c > > b/drivers/raw/ifpga_rawdev/ifpga_rawdev.c > > new file mode 100644 > > index 0000000..32e811c > > --- /dev/null > > +++ b/drivers/raw/ifpga_rawdev/ifpga_rawdev.c > > @@ -0,0 +1,608 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(c) 2010-2018 Intel Corporation */ > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#include "base/opae_hw_api.h" > > +#include "rte_rawdev.h" > > +#include "rte_rawdev_pmd.h" > > +#include "rte_bus_ifpga.h" > > +#include "ifpga_common.h" > > +#include "ifpga_logs.h" > > +#include "ifpga_rawdev.h" > > + > > +int ifpga_rawdev_logtype; > > + > > +#define PCI_VENDOR_ID_INTEL 0x8086 > > +/* PCI Device ID */ > > +#define PCIE_DEVICE_ID_PF_INT_5_X 0xBCBD > > +#define PCIE_DEVICE_ID_PF_INT_6_X 0xBCC0 > > +#define PCIE_DEVICE_ID_PF_DSC_1_X 0x09C4 > > +/* VF Device */ > > +#define PCIE_DEVICE_ID_VF_INT_5_X 0xBCBF > > +#define PCIE_DEVICE_ID_VF_INT_6_X 0xBCC1 > > +#define PCIE_DEVICE_ID_VF_DSC_1_X 0x09C5 > > +#define RTE_MAX_RAW_DEVICE 10 > > + > > +static const struct rte_pci_id pci_ifpga_map[] =3D { > > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > > PCIE_DEVICE_ID_PF_INT_5_X) }, > > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > > PCIE_DEVICE_ID_VF_INT_5_X) }, > > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > > PCIE_DEVICE_ID_PF_INT_6_X) }, > > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > > PCIE_DEVICE_ID_VF_INT_6_X) }, > > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > > PCIE_DEVICE_ID_PF_DSC_1_X) }, > > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > > PCIE_DEVICE_ID_VF_DSC_1_X) }, > > + { .vendor_id =3D 0, /* sentinel */ }, > > +}; > > + > > +static int > > +ifpga_fill_afu_dev(struct opae_accelerator *acc, > > + struct rte_afu_device *afu_dev) > > +{ > > + struct rte_mem_resource *res =3D afu_dev->mem_resource; > > + struct opae_acc_region_info region_info; > > + struct opae_acc_info info; > > + unsigned long i; > > + int ret; > > + > > + ret =3D opae_acc_get_info(acc, &info); > > + if (ret) > > + return ret; > > + > > + if (info.num_regions > PCI_MAX_RESOURCE) > > + return -EFAULT; > > + > > + afu_dev->num_region =3D info.num_regions; > > + > > + for (i =3D 0; i < info.num_regions; i++) { > > + region_info.index =3D i; > > + ret =3D opae_acc_get_region_info(acc, ®ion_info); > > + if (ret) > > + return ret; > > + > > + if ((region_info.flags & ACC_REGION_MMIO) && > > + (region_info.flags & ACC_REGION_READ) && > > + (region_info.flags & ACC_REGION_WRITE)) { > > + res[i].phys_addr =3D region_info.phys_addr; > > + res[i].len =3D region_info.len; > > + res[i].addr =3D region_info.addr; > > + } else > > + return -EFAULT; > > + } > > + > > + return 0; > > +} > > + > > +static void > > +ifpga_rawdev_info_get(struct rte_rawdev *dev, > > + rte_rawdev_obj_t dev_info) > > +{ > > + struct opae_adapter *adapter; > > + struct opae_accelerator *acc; > > + struct rte_afu_device *afu_dev; > > + > > + IFPGA_RAWDEV_PMD_FUNC_TRACE(); > > + > > + if (!dev_info) { > > + IFPGA_RAWDEV_PMD_ERR("Invalid request"); > > + return; > > + } > > + > > + adapter =3D ifpga_rawdev_get_priv(dev); > > + if (!adapter) > > + return; > > + > > + afu_dev =3D dev_info; > > + afu_dev->rawdev =3D dev; > > + > > + /* find opae_accelerator and fill info into afu_device */ > > + opae_adapter_for_each_acc(adapter, acc) { > > + if (acc->index !=3D afu_dev->id.port) > > + continue; > > + > > + if (ifpga_fill_afu_dev(acc, afu_dev)) { > > + IFPGA_RAWDEV_PMD_ERR("cannot get info\n"); > > + return; > > + } > > + } > > +} > > + > > +static int > > +ifpga_rawdev_start(struct rte_rawdev *dev) { > > + int ret =3D 0; > > + struct opae_adapter *adapter; > > + > > + IFPGA_RAWDEV_PMD_FUNC_TRACE(); > > + > > + RTE_FUNC_PTR_OR_ERR_RET(dev, -EINVAL); > > + > > + adapter =3D ifpga_rawdev_get_priv(dev); > > + if (!adapter) > > + return -ENODEV; > > + > > + return ret; > > +} > > + > > +static void > > +ifpga_rawdev_stop(struct rte_rawdev *dev) { > > + dev->started =3D 0; > > +} > > + > > +static int > > +ifpga_rawdev_close(struct rte_rawdev *dev) { > > + return dev ? 0:1; > > +} > > + > > +static int > > +ifpga_rawdev_reset(struct rte_rawdev *dev) { > > + return dev ? 0:1; > > +} > > + > > +static int > > +fpga_pr(struct rte_rawdev *raw_dev, u32 port_id, u64 *buffer, u32 size= , > > + u64 *status) > > +{ > > + > > + struct opae_adapter *adapter; > > + struct opae_manager *mgr; > > + struct opae_accelerator *acc; > > + struct opae_bridge *br; > > + int ret; > > + > > + adapter =3D ifpga_rawdev_get_priv(raw_dev); > > + if (!adapter) > > + return -ENODEV; > > + > > + mgr =3D opae_adapter_get_mgr(adapter); > > + if (!mgr) > > + return -ENODEV; > > + > > + acc =3D opae_adapter_get_acc(adapter, port_id); > > + if (!acc) > > + return -ENODEV; > > + > > + br =3D opae_acc_get_br(acc); > > + if (!br) > > + return -ENODEV; > > + > > + ret =3D opae_manager_flash(mgr, port_id, buffer, size, status); > > + if (ret) { > > + IFPGA_RAWDEV_PMD_ERR("%s pr error %d\n", __func__, > ret); > > + return ret; > > + } > > + > > + ret =3D opae_bridge_reset(br); > > + if (ret) { > > + IFPGA_RAWDEV_PMD_ERR("%s reset port:%d error %d\n", > > + __func__, port_id, ret); > > + return ret; > > + } > > + > > + return ret; > > +} > > + > > +static int > > +rte_fpga_do_pr(struct rte_rawdev *rawdev, int port_id, > > + const char *file_name) > > +{ > > + struct stat file_stat; > > + int file_fd; > > + int ret =3D 0; > > + u32 buffer_size; > > + void *buffer; > > + u64 pr_error; > > + > > + if (!file_name) > > + return -EINVAL; > > + > > + file_fd =3D open(file_name, O_RDONLY); > > + if (file_fd < 0) { > > + IFPGA_RAWDEV_PMD_ERR("%s: open file error: %s\n", > > + __func__, file_name); > > + IFPGA_RAWDEV_PMD_ERR("Message : %s\n", > strerror(errno)); > > + return -EINVAL; > > + } > > + ret =3D stat(file_name, &file_stat); > > + if (ret) { > > + IFPGA_RAWDEV_PMD_ERR("stat on bitstream file > failed: %s\n", > > + file_name); > > + return -EINVAL; > > + } > > + buffer_size =3D file_stat.st_size; > > + IFPGA_RAWDEV_PMD_INFO("bitstream file size: %u\n", buffer_size); > > + buffer =3D rte_malloc(NULL, buffer_size, 0); > > + if (!buffer) { > > + ret =3D -ENOMEM; > > + goto close_fd; > > + } > > + > > + /*read the raw data*/ > > + if (buffer_size !=3D read(file_fd, (void *)buffer, buffer_size)) { > > + ret =3D -EINVAL; > > + goto free_buffer; > > + } > > + > > + /*do PR now*/ > > + ret =3D fpga_pr(rawdev, port_id, buffer, buffer_size, &pr_error); > > + IFPGA_RAWDEV_PMD_INFO("downloading to device > port %d....%s.\n", > > port_id, > > + ret ? "failed" : "success"); > > + if (ret) { > > + ret =3D -EINVAL; > > + goto free_buffer; > > + } > > + > > +free_buffer: > > + if (buffer) > > + rte_free(buffer); > > +close_fd: > > + close(file_fd); > > + file_fd =3D 0; > > + return ret; > > +} > > + > > +static int > > +ifpga_rawdev_pr(struct rte_rawdev *dev, > > + rte_rawdev_obj_t pr_conf) > > +{ > > + struct opae_adapter *adapter; > > + struct rte_afu_pr_conf *afu_pr_conf; > > + int ret; > > + struct uuid uuid; > > + struct opae_accelerator *acc; > > + > > + IFPGA_RAWDEV_PMD_FUNC_TRACE(); > > + > > + adapter =3D ifpga_rawdev_get_priv(dev); > > + if (!adapter) > > + return -ENODEV; > > + > > + if (!pr_conf) > > + return -EINVAL; > > + > > + afu_pr_conf =3D pr_conf; > > + > > + if (afu_pr_conf->pr_enable) { > > + ret =3D rte_fpga_do_pr(dev, > > + afu_pr_conf->afu_id.port, > > + afu_pr_conf->bs_path); > > + if (ret) { > > + IFPGA_RAWDEV_PMD_ERR("do pr error %d\n", ret); > > + return ret; > > + } > > + } > > + > > + acc =3D opae_adapter_get_acc(adapter, afu_pr_conf->afu_id.port); > > + if (!acc) > > + return -ENODEV; > > + > > + ret =3D opae_acc_get_uuid(acc, &uuid); > > + if (ret) > > + return ret; > > + > > + memcpy(&afu_pr_conf->afu_id.uuid.uuid_low, uuid.b, sizeof(u64)); > > + memcpy(&afu_pr_conf->afu_id.uuid.uuid_high, uuid.b + 8, > > +sizeof(u64)); > > + > > + IFPGA_RAWDEV_PMD_INFO("%s: uuid_l=3D0x%lx, uuid_h=3D0x%lx\n", > > __func__, > > + (u64)afu_pr_conf->afu_id.uuid.uuid_low, > > + (u64)afu_pr_conf->afu_id.uuid.uuid_high); > > + > > + return 0; > > +} > > + > > +static const struct rte_rawdev_ops ifpga_rawdev_ops =3D { > > + .dev_info_get =3D ifpga_rawdev_info_get, > > + .dev_configure =3D NULL, > > + .dev_start =3D ifpga_rawdev_start, > > + .dev_stop =3D ifpga_rawdev_stop, > > + .dev_close =3D ifpga_rawdev_close, > > + .dev_reset =3D ifpga_rawdev_reset, > > + > > + .queue_def_conf =3D NULL, > > + .queue_setup =3D NULL, > > + .queue_release =3D NULL, > > + > > + .attr_get =3D NULL, > > + .attr_set =3D NULL, > > + > > + .enqueue_bufs =3D NULL, > > + .dequeue_bufs =3D NULL, > > + > > + .dump =3D NULL, > > + > > + .xstats_get =3D NULL, > > + .xstats_get_names =3D NULL, > > + .xstats_get_by_name =3D NULL, > > + .xstats_reset =3D NULL, > > + > > + .firmware_status_get =3D NULL, > > + .firmware_version_get =3D NULL, > > + .firmware_load =3D ifpga_rawdev_pr, > > + .firmware_unload =3D NULL, > > + > > + .dev_selftest =3D NULL, > > +}; > > + > > +static int > > +ifpga_rawdev_create(struct rte_pci_device *pci_dev, > > + int socket_id) > > +{ > > + int ret =3D 0; > > + struct rte_rawdev *rawdev =3D NULL; > > + struct opae_adapter *adapter =3D NULL; > > + struct opae_manager *mgr =3D NULL; > > + struct opae_adapter_data_pci *data =3D NULL; > > + char name[RTE_RAWDEV_NAME_MAX_LEN]; > > + int i; > > + > > + if (!pci_dev) { > > + IFPGA_RAWDEV_PMD_ERR("Invalid pci_dev of the device!"); > > + ret =3D -EINVAL; > > + goto cleanup; > > + } > > + > > + memset(name, 0, sizeof(name)); > > + snprintf(name, RTE_RAWDEV_NAME_MAX_LEN, > "IFPGA:%x:%02x.%x", > > + pci_dev->addr.bus, pci_dev->addr.devid, pci_dev- > >addr.function); > > + > > + IFPGA_RAWDEV_PMD_INFO("Init %s on NUMA node %d", name, > > +rte_socket_id()); > > + > > + /* Allocate device structure */ > > + rawdev =3D rte_rawdev_pmd_allocate(name, sizeof(struct > opae_adapter), > > + socket_id); > > + if (rawdev =3D=3D NULL) { > > + IFPGA_RAWDEV_PMD_ERR("Unable to allocate rawdevice"); > > + ret =3D -EINVAL; > > + goto cleanup; > > + } > > + > > + /* alloc OPAE_FPGA_PCI data to register to OPAE hardware level API > */ > > + data =3D opae_adapter_data_alloc(OPAE_FPGA_PCI); > > + if (!data) { > > + ret =3D -ENOMEM; > > + goto cleanup; > > + } > > + > > + /* init opae_adapter_data_pci for device specific information */ > > + for (i =3D 0; i < PCI_MAX_RESOURCE; i++) { > > + data->region[i].phys_addr =3D pci_dev- > >mem_resource[i].phys_addr; > > + data->region[i].len =3D pci_dev->mem_resource[i].len; > > + data->region[i].addr =3D pci_dev->mem_resource[i].addr; > > + } > > + data->device_id =3D pci_dev->id.device_id; > > + data->vendor_id =3D pci_dev->id.vendor_id; > > + > > + /* create a opae_adapter based on above device data */ > > + adapter =3D opae_adapter_alloc(pci_dev->device.name, data); > > + if (!adapter) { > > + ret =3D -ENOMEM; > > + goto free_adapter_data; > > + } > > + > > + rawdev->dev_ops =3D &ifpga_rawdev_ops; > > + rawdev->device =3D &pci_dev->device; > > + rawdev->driver_name =3D pci_dev->device.driver->name; > > + > > + rawdev->dev_private =3D adapter; > > + > > + /* must enumerate the adapter before use it */ > > + ret =3D opae_adapter_enumerate(adapter); > > + if (ret) > > + goto free_adapter; > > + > > + /* get opae_manager to rawdev */ > > + mgr =3D opae_adapter_get_mgr(adapter); > > + if (mgr) { > > + /* PF function */ > > + IFPGA_RAWDEV_PMD_INFO("this is a PF function"); > > + } > > + > > + return ret; > > + > > +free_adapter: > > + if (adapter) > > + opae_adapter_free(adapter); > > +free_adapter_data: > > + if (data) > > + opae_adapter_data_free(data); > > +cleanup: > > + if (rawdev) > > + rte_rawdev_pmd_release(rawdev); > > + > > + return ret; > > +} > > + > > +static int > > +ifpga_rawdev_destroy(struct rte_pci_device *pci_dev) { > > + int ret; > > + struct rte_rawdev *rawdev; > > + char name[RTE_RAWDEV_NAME_MAX_LEN]; > > + struct opae_adapter *adapter; > > + > > + if (!pci_dev) { > > + IFPGA_RAWDEV_PMD_ERR("Invalid pci_dev of the device!"); > > + ret =3D -EINVAL; > > + return ret; > > + } > > + > > + memset(name, 0, sizeof(name)); > > + snprintf(name, RTE_RAWDEV_NAME_MAX_LEN, > "IFPGA:%x:%02x.%x", > > + pci_dev->addr.bus, pci_dev->addr.devid, pci_dev- > >addr.function); > > + > > + IFPGA_RAWDEV_PMD_INFO("Closing %s on NUMA node %d", > > + name, rte_socket_id()); > > + > > + rawdev =3D rte_rawdev_pmd_get_named_dev(name); > > + if (!rawdev) { > > + IFPGA_RAWDEV_PMD_ERR("Invalid device name (%s)", > name); > > + return -EINVAL; > > + } > > + > > + adapter =3D ifpga_rawdev_get_priv(rawdev); > > + if (!adapter) > > + return -ENODEV; > > + > > + opae_adapter_data_free(adapter->data); > > + opae_adapter_free(adapter); > > + > > + /* rte_rawdev_close is called by pmd_release */ > > + ret =3D rte_rawdev_pmd_release(rawdev); > > + if (ret) > > + IFPGA_RAWDEV_PMD_DEBUG("Device cleanup failed"); > > + > > + return ret; > > +} > > + > > +static int > > +ifpga_rawdev_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, > > + struct rte_pci_device *pci_dev) > > +{ > > + > > + IFPGA_RAWDEV_PMD_FUNC_TRACE(); > > + return ifpga_rawdev_create(pci_dev, rte_socket_id()); } > > + > > +static int > > +ifpga_rawdev_pci_remove(struct rte_pci_device *pci_dev) { > > + return ifpga_rawdev_destroy(pci_dev); } > > + > > +static struct rte_pci_driver rte_ifpga_rawdev_pmd =3D { > > + .id_table =3D pci_ifpga_map, > > + .drv_flags =3D RTE_PCI_DRV_NEED_MAPPING | > RTE_PCI_DRV_INTR_LSC, > > + .probe =3D ifpga_rawdev_pci_probe, > > + .remove =3D ifpga_rawdev_pci_remove, > > +}; > > + > > +RTE_PMD_REGISTER_PCI(ifpga_rawdev_pci_driver, > rte_ifpga_rawdev_pmd); > > +RTE_PMD_REGISTER_PCI_TABLE(ifpga_rawdev_pci_driver, > > +rte_ifpga_rawdev_pmd); > > +RTE_PMD_REGISTER_KMOD_DEP(ifpga_rawdev_pci_driver, "* igb_uio | > > +uio_pci_generic | vfio-pci"); > > + > > +RTE_INIT(ifpga_rawdev_init_log); > > +static void > > +ifpga_rawdev_init_log(void) > > +{ > > + ifpga_rawdev_logtype =3D rte_log_register("driver.raw.init"); > > + if (ifpga_rawdev_logtype >=3D 0) > > + rte_log_set_level(ifpga_rawdev_logtype, RTE_LOG_NOTICE); } > > + > > +static const char * const valid_args[] =3D { > > +#define IFPGA_ARG_NAME "ifpga" > > + IFPGA_ARG_NAME, > > +#define IFPGA_ARG_PORT "port" > > + IFPGA_ARG_PORT, > > +#define IFPGA_AFU_BTS "afu_bts" > > + IFPGA_AFU_BTS, > > + NULL > > +}; > > + > > +static int > > +ifpga_cfg_probe(struct rte_vdev_device *dev) { > > + struct rte_devargs *devargs; > > + struct rte_kvargs *kvlist =3D NULL; > > + int port; > > + char *name =3D NULL; > > + char dev_name[RTE_RAWDEV_NAME_MAX_LEN]; > > + > > + devargs =3D dev->device.devargs; > > + > > + kvlist =3D rte_kvargs_parse(devargs->args, valid_args); > > + if (!kvlist) { > > + IFPGA_RAWDEV_PMD_LOG(ERR, "error when parsing > param"); > > + goto end; > > + } > > + > > + if (rte_kvargs_count(kvlist, IFPGA_ARG_NAME) =3D=3D 1) { > > + if (rte_kvargs_process(kvlist, IFPGA_ARG_NAME, > > + &rte_ifpga_get_string_arg, &name) < 0) { > > + IFPGA_RAWDEV_PMD_ERR("error to parse %s", > > + IFPGA_ARG_NAME); > > + goto end; > > + } > > + } else { > > + IFPGA_RAWDEV_PMD_ERR("arg %s is mandatory for ifpga > bus", > > + IFPGA_ARG_NAME); > > + goto end; > > + } > > + > > + if (rte_kvargs_count(kvlist, IFPGA_ARG_PORT) =3D=3D 1) { > > + if (rte_kvargs_process(kvlist, > > + IFPGA_ARG_PORT, > > + &rte_ifpga_get_integer32_arg, > > + &port) < 0) { > > + IFPGA_RAWDEV_PMD_ERR("error to parse %s", > > + IFPGA_ARG_PORT); > > + goto end; > > + } > > + } else { > > + IFPGA_RAWDEV_PMD_ERR("arg %s is mandatory for ifpga > bus", > > + IFPGA_ARG_PORT); > > + goto end; > > + } > > + > > + memset(dev_name, 0, sizeof(dev_name)); > > + snprintf(dev_name, RTE_RAWDEV_NAME_MAX_LEN, "%d|%s", > > + port, name); > > + > > + rte_eal_hotplug_add(RTE_STR(IFPGA_BUS_NAME), > > + dev_name, devargs->args); > > +end: > > + if (kvlist) > > + rte_kvargs_free(kvlist); > > + if (name) > > + free(name); > > + > > + return 0; > > +} > > + > > +static int > > +ifpga_cfg_remove(struct rte_vdev_device *vdev) { > > + IFPGA_RAWDEV_PMD_INFO("Remove ifpga_cfg %p", > > + vdev); > > + > > + return 0; > > +} > > + > > +static struct rte_vdev_driver ifpga_cfg_driver =3D { > > + .probe =3D ifpga_cfg_probe, > > + .remove =3D ifpga_cfg_remove, > > +}; > > + > > +RTE_PMD_REGISTER_VDEV(net_ifpga_cfg, ifpga_cfg_driver); > > +RTE_PMD_REGISTER_ALIAS(net_ifpga_cfg, ifpga_cfg); > > +RTE_PMD_REGISTER_PARAM_STRING(net_ifpga_cfg, > > + "bdf=3D " > > + "port=3D " > > + "afu_bts=3D"); > > + > > diff --git a/drivers/raw/ifpga_rawdev/ifpga_rawdev.h > > b/drivers/raw/ifpga_rawdev/ifpga_rawdev.h > > new file mode 100644 > > index 0000000..c7759b8 > > --- /dev/null > > +++ b/drivers/raw/ifpga_rawdev/ifpga_rawdev.h > > @@ -0,0 +1,37 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(c) 2010-2018 Intel Corporation */ > > + > > +#ifndef _IFPGA_RAWDEV_H_ > > +#define _IFPGA_RAWDEV_H_ > > + > > +extern int ifpga_rawdev_logtype; > > + > > +#define IFPGA_RAWDEV_PMD_LOG(level, fmt, args...) \ > > + rte_log(RTE_LOG_ ## level, ifpga_rawdev_logtype, "ifgpa: " fmt, \ > > + ##args) > > + > > +#define IFPGA_RAWDEV_PMD_FUNC_TRACE() > > IFPGA_RAWDEV_PMD_LOG(DEBUG, ">>") > > + > > +#define IFPGA_RAWDEV_PMD_DEBUG(fmt, args...) \ > > + IFPGA_RAWDEV_PMD_LOG(DEBUG, fmt, ## args) #define > > +IFPGA_RAWDEV_PMD_INFO(fmt, args...) \ > > + IFPGA_RAWDEV_PMD_LOG(INFO, fmt, ## args) #define > > +IFPGA_RAWDEV_PMD_ERR(fmt, args...) \ > > + IFPGA_RAWDEV_PMD_LOG(ERR, fmt, ## args) #define > > +IFPGA_RAWDEV_PMD_WARN(fmt, args...) \ > > + IFPGA_RAWDEV_PMD_LOG(WARNING, fmt, ## args) > > + > > +enum ifpga_rawdev_device_state { > > + IFPGA_IDLE, > > + IFPGA_READY, > > + IFPGA_ERROR > > +}; > > + > > +static inline struct opae_adapter * > > +ifpga_rawdev_get_priv(const struct rte_rawdev *rawdev) { > > + return rawdev->dev_private; > > +} > > + > > +#endif /* _IFPGA_RAWDEV_H_ */ > > diff --git a/drivers/raw/ifpga_rawdev/meson.build > > b/drivers/raw/ifpga_rawdev/meson.build > > new file mode 100644 > > index 0000000..6725687 > > --- /dev/null > > +++ b/drivers/raw/ifpga_rawdev/meson.build > > @@ -0,0 +1,15 @@ > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel > > +Corporation > > + > > +version =3D 1 > > + > > +subdir('base') > > +objs =3D [base_objs] > > + > > +deps +=3D ['rawdev', 'pci', 'bus_pci', 'kvargs', > > + 'bus_vdev', 'bus_ifpga'] > > +sources =3D files('ifpga_rawdev.c') > > + > > +includes +=3D include_directories('base') > > + > > +allow_experimental_apis =3D true > > diff --git a/drivers/raw/ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map > > b/drivers/raw/ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map > > new file mode 100644 > > index 0000000..9b9ab1a > > --- /dev/null > > +++ b/drivers/raw/ifpga_rawdev/rte_pmd_ifpga_rawdev_version.map > > @@ -0,0 +1,4 @@ > > +DPDK_18.05 { > > + > > + local: *; > > +}; > > diff --git a/drivers/raw/meson.build b/drivers/raw/meson.build index > > 34dfac4..a61cdcc 100644 > > --- a/drivers/raw/meson.build > > +++ b/drivers/raw/meson.build > > @@ -1,7 +1,7 @@ > > # SPDX-License-Identifier: BSD-3-Clause # Copyright 2018 NXP > > > > -drivers =3D ['skeleton_rawdev', 'dpaa2_cmdif', 'dpaa2_qdma'] > > +drivers =3D ['skeleton_rawdev', 'dpaa2_cmdif', 'dpaa2_qdma', > > +'ifpga_rawdev'] > > std_deps =3D ['rawdev'] > > config_flag_fmt =3D 'RTE_LIBRTE_PMD_@0@_RAWDEV' > > driver_name_fmt =3D 'rte_pmd_@0@' > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 3861e1a..acc0ec3 > > 100644 > > --- a/mk/rte.app.mk > > +++ b/mk/rte.app.mk > > @@ -256,9 +256,11 @@ > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV) +=3D > > -lrte_pmd_dpaa2_cmdif > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV) +=3D > > -lrte_pmd_dpaa2_qdma endif # CONFIG_RTE_LIBRTE_FSLMC_BUS > > _LDLIBS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) +=3D -lrte_bus_ifpga > > +ifeq ($(CONFIG_RTE_LIBRTE_IFPGA_BUS),y) > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV) +=3D > > -lrte_pmd_ifpga_rawdev > > +endif # CONFIG_RTE_LIBRTE_IFPGA_BUS > > endif # CONFIG_RTE_LIBRTE_RAWDEV > > > > - > > endif # !CONFIG_RTE_BUILD_SHARED_LIBS > > > > _LDLIBS-y +=3D --no-whole-archive > > -- > > 1.8.3.1