From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <qi.z.zhang@intel.com>
Received: from mga03.intel.com (mga03.intel.com [134.134.136.65])
by dpdk.org (Postfix) with ESMTP id 2AB861BAA9
for <dev@dpdk.org>; Thu, 10 May 2018 14:26:55 +0200 (CEST)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga005.jf.intel.com ([10.7.209.41])
by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
10 May 2018 05:26:55 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.49,385,1520924400"; d="scan'208";a="223157601"
Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202])
by orsmga005.jf.intel.com with ESMTP; 10 May 2018 05:26:54 -0700
Received: from shsmsx102.ccr.corp.intel.com (10.239.4.154) by
fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS)
id 14.3.319.2; Thu, 10 May 2018 05:26:54 -0700
Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.210]) by
shsmsx102.ccr.corp.intel.com ([169.254.2.79]) with mapi id 14.03.0319.002;
Thu, 10 May 2018 20:26:51 +0800
From: "Zhang, Qi Z" <qi.z.zhang@intel.com>
To: "Xu, Rosen" <rosen.xu@intel.com>, "dev@dpdk.org" <dev@dpdk.org>,
"thomas@monjalon.net" <thomas@monjalon.net>
CC: "Xu, Rosen" <rosen.xu@intel.com>, "Zhang, Roy Fan"
<roy.fan.zhang@intel.com>, "Doherty, Declan" <declan.doherty@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>, "shreyansh.jain@nxp.com"
<shreyansh.jain@nxp.com>, "Yigit, Ferruh" <ferruh.yigit@intel.com>,
"Ananyev, Konstantin" <konstantin.ananyev@intel.com>, "Zhang, Tianfei"
<tianfei.zhang@intel.com>, "Liu, Song" <song.liu@intel.com>, "Wu, Hao"
<hao.wu@intel.com>, "gaetan.rivet@6wind.com" <gaetan.rivet@6wind.com>
Thread-Topic: [dpdk-dev] [PATCH v10 1/3] bus/ifpga: Add Intel FPGA BUS Library
Thread-Index: AQHT52lR8tV3Dl4hPk2mcWMK8PTcOqQo3qzw
Date: Thu, 10 May 2018 12:26:50 +0000
Message-ID: <039ED4275CED7440929022BC67E70611531B09E7@SHSMSX103.ccr.corp.intel.com>
References: <1521553556-62982-1-git-send-email-rosen.xu@intel.com>
<1525851801-16101-1-git-send-email-rosen.xu@intel.com>
<1525851801-16101-2-git-send-email-rosen.xu@intel.com>
In-Reply-To: <1525851801-16101-2-git-send-email-rosen.xu@intel.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNDVjNTcyNDEtMDAwNS00OWRkLWEwYmUtZmU5YmE2Njc3NzJjIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjIuNS4xOCIsIlRydXN0ZWRMYWJlbEhhc2giOiJ2K2V4NlFyR1BZR1RkYmpOdDFMY3haN3B0ajB2KzJIbkFwd1c2UFlTSXNlK2FwclpvcmdsMHhpYXlGM05GRDRYIn0=
x-ctpclassification: CTP_NT
dlp-product: dlpe-windows
dlp-version: 11.0.200.100
dlp-reaction: no-action
x-originating-ip: [10.239.127.40]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [dpdk-dev] [PATCH v10 1/3] bus/ifpga: Add Intel FPGA BUS Library
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 10 May 2018 12:26:56 -0000
Hi Rosen:
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Xu, Rosen
> Sent: Wednesday, May 9, 2018 3:43 PM
> To: dev@dpdk.org; thomas@monjalon.net
> Cc: Xu, Rosen <rosen.xu@intel.com>; Zhang, Roy Fan
> <roy.fan.zhang@intel.com>; Doherty, Declan <declan.doherty@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; shreyansh.jain@nxp.com;
> Yigit, Ferruh <ferruh.yigit@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Tianfei <tianfei.zhang@intel.com>;
> Liu, Song <song.liu@intel.com>; Wu, Hao <hao.wu@intel.com>;
> gaetan.rivet@6wind.com
> Subject: [dpdk-dev] [PATCH v10 1/3] bus/ifpga: Add Intel FPGA BUS Library
>=20
> From: Rosen Xu <rosen.xu@intel.com>
>=20
> Defined FPGA-BUS for Acceleration Drivers of AFUs
>=20
> 1. FPGA PCI Scan (1st Scan) follows DPDK UIO/VFIO PCI Scan Process, probe
> Intel FPGA Rawdev Driver, it will be covered in following patches.
>=20
> 2. AFU Scan(2nd Scan) bind DPDK driver to FPGA Partial-Bitstream.
> This scan is trigged by hotplug of IFPGA Rawdev probe, in this scan the A=
FUs
> will be created and their drivers are also probed.
>=20
> This patch will introduce rte_afu_device which describe the AFU device li=
sted in
> the FPGA-BUS.
>=20
> Signed-off-by: Rosen Xu <rosen.xu@intel.com>
> Signed-off-by: Tianfei Zhang <tianfei.zhang@intel.com>
> ---
> MAINTAINERS | 4 +
> config/common_base | 5 +
> doc/guides/rel_notes/release_18_05.rst | 5 +
> drivers/bus/Makefile | 1 +
> drivers/bus/ifpga/Makefile | 32 ++
> drivers/bus/ifpga/ifpga_bus.c | 501
> ++++++++++++++++++++++++++++
> drivers/bus/ifpga/ifpga_common.c | 88 +++++
> drivers/bus/ifpga/ifpga_common.h | 18 +
> drivers/bus/ifpga/ifpga_logs.h | 31 ++
> drivers/bus/ifpga/meson.build | 8 +
> drivers/bus/ifpga/rte_bus_ifpga.h | 160 +++++++++
> drivers/bus/ifpga/rte_bus_ifpga_version.map | 10 +
> drivers/bus/meson.build | 2 +-
> mk/rte.app.mk | 1 +
> 14 files changed, 865 insertions(+), 1 deletion(-) create mode 100644
> drivers/bus/ifpga/Makefile create mode 100644
> drivers/bus/ifpga/ifpga_bus.c create mode 100644
> drivers/bus/ifpga/ifpga_common.c create mode 100644
> drivers/bus/ifpga/ifpga_common.h create mode 100644
> drivers/bus/ifpga/ifpga_logs.h create mode 100644
> drivers/bus/ifpga/meson.build create mode 100644
> drivers/bus/ifpga/rte_bus_ifpga.h create mode 100644
> drivers/bus/ifpga/rte_bus_ifpga_version.map
>=20
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 7105920..fa0c5b1 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -383,6 +383,10 @@ F: drivers/mempool/bucket/ Bus Drivers
> -----------
>=20
> +Intel FPGA buses
> +M: Rosen Xu <rosen.xu@intel.com>
> +F: drivers/bus/ifpga/
> +
> NXP buses
> M: Hemant Agrawal <hemant.agrawal@nxp.com>
> M: Shreyansh Jain <shreyansh.jain@nxp.com> diff --git
> a/config/common_base b/config/common_base index 0d181ac..1440316
> 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -139,6 +139,11 @@
> CONFIG_RTE_ETHDEV_PROFILE_ITT_WASTED_RX_ITERATIONS=3Dn
> CONFIG_RTE_ETHDEV_TX_PREPARE_NOOP=3Dn
>=20
> #
> +# Compile the Intel FPGA bus
> +#
> +CONFIG_RTE_LIBRTE_IFPGA_BUS=3Dy
> +
> +#
> # Compile PCI bus driver
> #
> CONFIG_RTE_LIBRTE_PCI_BUS=3Dy
> diff --git a/doc/guides/rel_notes/release_18_05.rst
> b/doc/guides/rel_notes/release_18_05.rst
> index 7187348..265950a 100644
> --- a/doc/guides/rel_notes/release_18_05.rst
> +++ b/doc/guides/rel_notes/release_18_05.rst
> @@ -183,6 +183,11 @@ New Features
> stats/xstats on shared memory from secondary process, and also pdump
> packets on
> those virtual devices.
>=20
> +* **Added Ifpga Bus, a generic Intel FPGA Bus library.**
> +
> + The Ifpga Bus library provides support for integrating any Intel FPGA
> + device with the DPDK framework. It provides Intel FPGA Partial Bit
> + Stream AFU(Accelerated Function Unit) scan and drivers prove.
>=20
> API Changes
> -----------
> diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile index
> c251b65..ef7f247 100644
> --- a/drivers/bus/Makefile
> +++ b/drivers/bus/Makefile
> @@ -7,6 +7,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_DPAA_BUS) +=3D dpaa ifeq
> ($(CONFIG_RTE_EAL_VFIO),y)
> DIRS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) +=3D fslmc endif
> +DIRS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) +=3D ifpga
> DIRS-$(CONFIG_RTE_LIBRTE_PCI_BUS) +=3D pci
> DIRS-$(CONFIG_RTE_LIBRTE_VDEV_BUS) +=3D vdev
>=20
> diff --git a/drivers/bus/ifpga/Makefile b/drivers/bus/ifpga/Makefile new =
file
> mode 100644 index 0000000..3ff3bdb
> --- /dev/null
> +++ b/drivers/bus/ifpga/Makefile
> @@ -0,0 +1,32 @@
> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel
> +Corporation
> +
> +include $(RTE_SDK)/mk/rte.vars.mk
> +
> +#
> +# library name
> +#
> +LIB =3D librte_bus_ifpga.a
> +
> +CFLAGS +=3D -DALLOW_EXPERIMENTAL_API
> +CFLAGS +=3D -O3
> +CFLAGS +=3D $(WERROR_FLAGS)
> +LDLIBS +=3D -lrte_eal
> +LDLIBS +=3D -lrte_rawdev
> +LDLIBS +=3D -lrte_kvargs
> +
> +# versioning export map
> +EXPORT_MAP :=3D rte_bus_ifpga_version.map
> +
> +# library version
> +LIBABIVER :=3D 1
> +
> +SRCS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) +=3D ifpga_bus.c
> +SRCS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) +=3D ifpga_common.c
> +
> +#
> +# Export include files
> +#
> +SYMLINK-$(CONFIG_RTE_LIBRTE_IFPGA_BUS)-include +=3D rte_bus_ifpga.h
> +
> +include $(RTE_SDK)/mk/rte.lib.mk
> diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.=
c new
> file mode 100644 index 0000000..e144c01
> --- /dev/null
> +++ b/drivers/bus/ifpga/ifpga_bus.c
> @@ -0,0 +1,501 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2018 Intel Corporation */
> +
> +#include <string.h>
> +#include <inttypes.h>
> +#include <stdint.h>
> +#include <stdlib.h>
> +#include <stdio.h>
> +#include <sys/queue.h>
> +#include <sys/mman.h>
> +#include <sys/types.h>
> +#include <unistd.h>
> +#include <fcntl.h>
> +
> +#include <rte_errno.h>
> +#include <rte_bus.h>
> +#include <rte_per_lcore.h>
> +#include <rte_memory.h>
> +#include <rte_memzone.h>
> +#include <rte_eal.h>
> +#include <rte_common.h>
> +
> +#include <rte_devargs.h>
> +#include <rte_kvargs.h>
> +#include <rte_alarm.h>
> +
> +#include "rte_rawdev.h"
> +#include "rte_rawdev_pmd.h"
> +#include "rte_bus_ifpga.h"
> +#include "ifpga_logs.h"
> +#include "ifpga_common.h"
> +
> +int ifpga_bus_logtype;
> +
> +/* Forward declaration to access Intel FPGA bus
> + * on which iFPGA devices are connected */ static struct rte_bus
> +rte_ifpga_bus;
> +
> +/** Double linked list of IFPGA device. */
> +TAILQ_HEAD(ifpga_device_list, rte_ifpga_device);
> +
> +static struct ifpga_device_list ifpga_device_list =3D
> + TAILQ_HEAD_INITIALIZER(ifpga_device_list);
> +static struct afu_driver_list afu_driver_list =3D
> + TAILQ_HEAD_INITIALIZER(afu_driver_list);
> +
> +
> +/* register a ifpga bus based driver */ void
> +rte_ifpga_driver_register(struct rte_afu_driver *driver) {
> + RTE_VERIFY(driver);
> +
> + TAILQ_INSERT_TAIL(&afu_driver_list, driver, next); }
> +
> +/* un-register a fpga bus based driver */ void
> +rte_ifpga_driver_unregister(struct rte_afu_driver *driver) {
> + TAILQ_REMOVE(&afu_driver_list, driver, next); }
> +
> +static struct rte_ifpga_device *
> +ifpga_find_ifpga_dev(const struct rte_rawdev *rdev) {
> + struct rte_ifpga_device *ifpga_dev =3D NULL;
> +
> + TAILQ_FOREACH(ifpga_dev, &ifpga_device_list, next) {
> + if (rdev &&
> + ifpga_dev->rdev &&
> + ifpga_dev->rdev =3D=3D rdev)
> + return ifpga_dev;
> + }
> + return NULL;
> +}
> +
> +static struct rte_afu_device *
> +ifpga_find_afu_dev(const struct rte_ifpga_device *ifpga_dev,
> + const struct rte_afu_id *afu_id)
> +{
> + struct rte_afu_device *afu_dev =3D NULL;
> +
> + TAILQ_FOREACH(afu_dev, &ifpga_dev->afu_list, next) {
> + if (!ifpga_afu_id_cmp(&afu_dev->id, afu_id))
> + return afu_dev;
> + }
> + return NULL;
> +}
> +
> +static const char * const valid_args[] =3D {
> +#define IFPGA_ARG_NAME "ifpga"
> + IFPGA_ARG_NAME,
> +#define IFPGA_ARG_PORT "port"
> + IFPGA_ARG_PORT,
> +#define IFPGA_AFU_BTS "afu_bts"
> + IFPGA_AFU_BTS,
> + NULL
> +};
> +
> +/*
> + * Scan the content of the FPGA bus, and the devices in the devices
> + * list
> + */
> +static struct rte_afu_device *
> +ifpga_scan_one(struct rte_devargs *devargs,
> + struct rte_ifpga_device *ifpga_dev)
usually ifpag_dev should be the first parameter here, and devargs follows
> +{
> + struct rte_kvargs *kvlist =3D NULL;
> + struct rte_rawdev *rawdev =3D NULL;
> + struct rte_afu_device *afu_dev =3D NULL;
> + struct rte_afu_pr_conf afu_pr_conf;
> + int ret =3D 0;
> + char *path =3D NULL;
> +
> + memset(&afu_pr_conf, 0, sizeof(struct rte_afu_pr_conf));
> +
> + kvlist =3D rte_kvargs_parse(devargs->args, valid_args);
> + if (!kvlist) {
> + IFPGA_BUS_ERR("error when parsing param");
> + goto end;
> + }
> +
> + if (rte_kvargs_count(kvlist, IFPGA_ARG_PORT) =3D=3D 1) {
> + if (rte_kvargs_process(kvlist, IFPGA_ARG_PORT,
> + &rte_ifpga_get_integer32_arg, &afu_pr_conf.afu_id.port) < 0) {
> + IFPGA_BUS_ERR("error to parse %s",
> + IFPGA_ARG_PORT);
> + goto end;
> + }
> + } else {
> + IFPGA_BUS_ERR("arg %s is mandatory for ifpga bus",
> + IFPGA_ARG_PORT);
> + goto end;
> + }
> +
> + if (rte_kvargs_count(kvlist, IFPGA_AFU_BTS) =3D=3D 1) {
> + if (rte_kvargs_process(kvlist, IFPGA_AFU_BTS,
> + &rte_ifpga_get_string_arg, &path) < 0) {
> + IFPGA_BUS_ERR("Failed to parse %s",
> + IFPGA_AFU_BTS);
> + goto end;
> + }
> + } else {
> + IFPGA_BUS_ERR("arg %s is mandatory for ifpga bus",
> + IFPGA_AFU_BTS);
> + goto end;
> + }
> +
> + afu_pr_conf.afu_id.uuid.uuid_low =3D 0;
> + afu_pr_conf.afu_id.uuid.uuid_high =3D 0;
> + afu_pr_conf.pr_enable =3D path?1:0;
> +
> + rawdev =3D ifpga_dev->rdev;
> + if (ifpga_find_afu_dev(ifpga_dev, &afu_pr_conf.afu_id))
> + goto end;
> +
> + afu_dev =3D calloc(1, sizeof(*afu_dev));
> + if (!afu_dev)
> + goto end;
> +
> + afu_dev->device.devargs =3D devargs;
> + afu_dev->device.numa_node =3D SOCKET_ID_ANY;
> + afu_dev->device.name =3D devargs->name;
> + afu_dev->rawdev =3D rawdev;
> + afu_dev->id.uuid.uuid_low =3D 0;
> + afu_dev->id.uuid.uuid_high =3D 0;
> + afu_dev->id.port =3D afu_pr_conf.afu_id.port;
> + afu_dev->ifpga_dev =3D ifpga_dev;
> +
> + if (rawdev->dev_ops && rawdev->dev_ops->dev_info_get)
> + rawdev->dev_ops->dev_info_get(rawdev, afu_dev);
> +
> + if (rawdev->dev_ops &&
> + rawdev->dev_ops->dev_start &&
> + rawdev->dev_ops->dev_start(rawdev))
> + goto free_dev;
> +
> + strncpy(afu_pr_conf.bs_path, path, sizeof(afu_pr_conf.bs_path));
> + if (rawdev->dev_ops->firmware_load &&
> + rawdev->dev_ops->firmware_load(rawdev,
> + &afu_pr_conf)){
> + IFPGA_BUS_ERR("firmware load error %d\n", ret);
> + goto free_dev;
> + }
> + afu_dev->id.uuid.uuid_low =3D afu_pr_conf.afu_id.uuid.uuid_low;
> + afu_dev->id.uuid.uuid_high =3D afu_pr_conf.afu_id.uuid.uuid_high;
> +
> + return afu_dev;
> +
> +free_dev:
> + free(afu_dev);
> +end:
> + if (kvlist)
> + rte_kvargs_free(kvlist);
> + if (path)
> + free(path);
> +
> + return NULL;
> +}
> +
> +/*
> + * Scan the content of the FPGA bus, and the devices in the devices
> + * list
> + */
> +static int
> +ifpga_scan(void)
> +{
> + struct rte_ifpga_device *ifpga_dev;
> + struct rte_devargs *devargs;
> + struct rte_kvargs *kvlist =3D NULL;
> + struct rte_rawdev *rawdev =3D NULL;
> + char *name =3D NULL;
> + char name1[RTE_RAWDEV_NAME_MAX_LEN];
> + struct rte_afu_device *afu_dev =3D NULL;
> +
> + /* for FPGA devices we scan the devargs_list populated via cmdline */
I didn't see "--ifpga" is supported by cmdline, either you need to add corr=
esponding parser or correct the comment here.
> + RTE_EAL_DEVARGS_FOREACH("ifpga", devargs) {
IFPGA_BUS_NAME to replace "ifpga" here.
> + if (devargs->bus !=3D &rte_ifpga_bus)
> + continue;
> +
> + kvlist =3D rte_kvargs_parse(devargs->args, valid_args);
> + if (!kvlist) {
> + IFPGA_BUS_ERR("error when parsing param");
> + goto end;
> + }
> +
> + if (rte_kvargs_count(kvlist, IFPGA_ARG_NAME) =3D=3D 1) {
> + if (rte_kvargs_process(kvlist, IFPGA_ARG_NAME,
> + &rte_ifpga_get_string_arg, &name) < 0) {
> + IFPGA_BUS_ERR("error to parse %s",
> + IFPGA_ARG_NAME);
> + goto end;
> + }
> + } else {
> + IFPGA_BUS_ERR("arg %s is mandatory for ifpga bus",
> + IFPGA_ARG_NAME);
> + goto end;
> + }
> +
> + memset(name1, 0, sizeof(name1));
> + snprintf(name1, RTE_RAWDEV_NAME_MAX_LEN, "IFPGA:%s",
> name);
> +
> + rawdev =3D rte_rawdev_pmd_get_named_dev(name1);
> + if (!rawdev)
> + goto end;
> +
> + if (ifpga_find_ifpga_dev(rawdev))
> + continue;
> +
> + ifpga_dev =3D calloc(1, sizeof(*ifpga_dev));
> + if (!ifpga_dev)
> + goto end;
> +
> + ifpga_dev->rdev =3D rawdev;
> + TAILQ_INIT(&ifpga_dev->afu_list);
> +
> + TAILQ_INSERT_TAIL(&ifpga_device_list, ifpga_dev, next);
> + afu_dev =3D ifpga_scan_one(devargs, ifpga_dev);
> + if (afu_dev !=3D NULL)
> + TAILQ_INSERT_TAIL(&ifpga_dev->afu_list, afu_dev, next);
I don't understand why we need afu_list?, seems we only add a new afu_dev i=
nto the list after create a new ifpga_dev,=20
Is there any another place that we add one to list to make it reasonable? =
it looks like either a 1:1 or 1:0 for ifpga_dev:afu_dev?
> + }
> +
> +end:
> + if (kvlist)
> + rte_kvargs_free(kvlist);
> + if (name)
> + free(name);
> +
> + return 0;
> +}
> +
> +/*
> + * Match the AFU Driver and AFU Device using the ID Table */ static
> +int rte_afu_match(const struct rte_afu_driver *afu_drv,
> + const struct rte_afu_device *afu_dev) {
> + const struct rte_afu_uuid *id_table;
> +
> + for (id_table =3D afu_drv->id_table;
> + ((id_table->uuid_low !=3D 0) && (id_table->uuid_high !=3D 0));
> + id_table++) {
> + /* check if device's identifiers match the driver's ones */
> + if ((id_table->uuid_low !=3D afu_dev->id.uuid.uuid_low) ||
> + id_table->uuid_high !=3D
> + afu_dev->id.uuid.uuid_high)
> + continue;
> +
> + return 1;
> + }
> +
> + return 0;
> +}
> +
> +static int
> +ifpga_probe_one_driver(struct rte_afu_driver *drv,
> + struct rte_afu_device *afu_dev)
> +{
> + int ret;
> +
> + if (!rte_afu_match(drv, afu_dev))
> + /* Match of device and driver failed */
> + return 1;
> +
> + /* reference driver structure */
> + afu_dev->driver =3D drv;
> + afu_dev->device.driver =3D &drv->driver;
> +
> + /* call the driver probe() function */
> + ret =3D drv->probe(afu_dev);
> + if (ret) {
> + afu_dev->driver =3D NULL;
> + afu_dev->device.driver =3D NULL;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +ifpga_probe_all_drivers(struct rte_afu_device *afu_dev) {
> + struct rte_afu_driver *drv =3D NULL;
> + int ret =3D 0;
> +
> + if (afu_dev =3D=3D NULL)
> + return -1;
> +
> + /* Check if a driver is already loaded */
> + if (afu_dev->driver !=3D NULL)
> + return 0;
> +
> + TAILQ_FOREACH(drv, &afu_driver_list, next) {
> + if (ifpga_probe_one_driver(drv, afu_dev)) {
> + ret =3D -1;
> + break;
> + }
> + }
> + return ret;
> +}
> +
> +/*
> + * Scan the content of the Intel FPGA bus, and call the probe()
> +function for
> + * all registered drivers that have a matching entry in its id_table
> + * for discovered devices.
> + */
> +static int
> +ifpga_probe(void)
> +{
> + struct rte_ifpga_device *ifpga_dev;
> + struct rte_afu_device *afu_dev =3D NULL;
> + int ret =3D 0;
> +
> + TAILQ_FOREACH(ifpga_dev, &ifpga_device_list, next) {
> + TAILQ_FOREACH(afu_dev, &ifpga_dev->afu_list, next) {
> +
> + if (afu_dev->device.driver)
> + continue;
> +
> + ret =3D ifpga_probe_all_drivers(afu_dev);
> + if (ret < 0)
> + IFPGA_BUS_ERR("failed to initialize %s device\n",
> + rte_ifpga_device_name(afu_dev));
> + }
> + }
> +
> + return ret;
> +}
> +
> +static int
> +ifpga_plug(struct rte_device *dev)
> +{
> + return ifpga_probe_all_drivers(RTE_DEV_TO_AFU(dev));
> +}
> +
> +static int
> +ifpga_remove_driver(struct rte_afu_device *afu_dev) {
> + const char *name;
> + const struct rte_afu_driver *driver;
> +
> + name =3D rte_ifpga_device_name(afu_dev);
> + if (!afu_dev->device.driver) {
> + IFPGA_BUS_DEBUG("no driver attach to device %s\n", name);
> + return 1;
> + }
> +
> + driver =3D RTE_DRV_TO_AFU_CONST(afu_dev->device.driver);
> + return driver->remove(afu_dev);
> +}
> +
> +static int
> +ifpga_unplug(struct rte_device *dev)
> +{
> + struct rte_ifpga_device *ifpga_dev =3D NULL;
> + struct rte_afu_device *afu_dev =3D NULL;
> + struct rte_devargs *devargs =3D NULL;
> + int ret;
> +
> + if (dev =3D=3D NULL)
> + return -EINVAL;
> +
> + afu_dev =3D RTE_DEV_TO_AFU(dev);
> + if (!dev)
> + return -ENOENT;
> +
> + ifpga_dev =3D afu_dev->ifpga_dev;
> + devargs =3D dev->devargs;
> +
> + ret =3D ifpga_remove_driver(afu_dev);
So what is the device type that be plugged into the ifgpa bus?
ifpga_dev or afu_dev? If its afu_dev, why we need ifpga_dev?
If it is ifpga_dev, we need remove ifpga_dev from ifpga_device_list here.=20
> + if (ret)
> + return ret;
> +
> + TAILQ_REMOVE(&ifpga_dev->afu_list, afu_dev, next);
> +
> + rte_devargs_remove(devargs->bus->name, devargs->name);
> + free(afu_dev);
> + return 0;
> +
> +}
> +
> +static struct rte_device *
> +ifpga_find_device(const struct rte_device *start,
> + rte_dev_cmp_t cmp, const void *data)
> +{
> + struct rte_ifpga_device *ifpga_dev;
> + struct rte_afu_device *afu_dev;
> +
> + TAILQ_FOREACH(ifpga_dev, &ifpga_device_list, next) {
> + TAILQ_FOREACH(afu_dev, &ifpga_dev->afu_list, next) {
> + if (start && &afu_dev->device =3D=3D start) {
> + start =3D NULL;
> + continue;
> + }
> + if (cmp(&afu_dev->device, data) =3D=3D 0)
> + return &afu_dev->device;
> + }
> + }
> + return NULL;
> +}
> +static int
> +ifpga_parse(const char *name, void *addr) {
> + int *out =3D addr;
> + struct rte_rawdev *rawdev =3D NULL;
> + char rawdev_name[RTE_RAWDEV_NAME_MAX_LEN];
> + char *c1 =3D NULL, *c2 =3D NULL;
> + int port =3D IFPGA_BUS_DEV_PORT_MAX;
> + char str_port[8];
> + int str_port_len =3D 0;
> + int ret;
> +
> + memset(str_port, 0, 8);
> + c1 =3D strchr(name, '|');
> + if (c1 !=3D NULL) {
> + str_port_len =3D c1-name;
> + c2 =3D c1+1;
> + }
> +
> + if (str_port_len < 8 &&
> + str_port_len > 0) {
> + memcpy(str_port, name, str_port_len);
> + ret =3D sscanf(str_port, "%d", &port);
> + if (ret =3D=3D -1)
> + return 0;
> + }
> +
> + memset(rawdev_name, 0, sizeof(rawdev_name));
> + snprintf(rawdev_name, RTE_RAWDEV_NAME_MAX_LEN, "IFPGA:%s", c2);
> + rawdev =3D rte_rawdev_pmd_get_named_dev(rawdev_name);
> +
> + if ((port < IFPGA_BUS_DEV_PORT_MAX) &&
> + rawdev &&
> + (addr !=3D NULL))
> + *out =3D port;
> +
> + if ((port < IFPGA_BUS_DEV_PORT_MAX) &&
> + rawdev)
> + return 0;
> + else
> + return 1;
> +}
> +
> +static struct rte_bus rte_ifpga_bus =3D {
> + .scan =3D ifpga_scan,
> + .probe =3D ifpga_probe,
> + .find_device =3D ifpga_find_device,
> + .plug =3D ifpga_plug,
> + .unplug =3D ifpga_unplug,
> + .parse =3D ifpga_parse,
> +};
> +
> +RTE_REGISTER_BUS(IFPGA_BUS_NAME, rte_ifpga_bus);
> +
> +RTE_INIT(ifpga_init_log)
> +{
> + ifpga_bus_logtype =3D rte_log_register("bus.ifpga");
> + if (ifpga_bus_logtype >=3D 0)
> + rte_log_set_level(ifpga_bus_logtype, RTE_LOG_NOTICE); }
> diff --git a/drivers/bus/ifpga/ifpga_common.c
> b/drivers/bus/ifpga/ifpga_common.c
> new file mode 100644
> index 0000000..78e2eae
> --- /dev/null
> +++ b/drivers/bus/ifpga/ifpga_common.c
> @@ -0,0 +1,88 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2018 Intel Corporation */
> +
> +#include <string.h>
> +#include <inttypes.h>
> +#include <stdint.h>
> +#include <stdlib.h>
> +#include <stdio.h>
> +#include <sys/queue.h>
> +#include <sys/mman.h>
> +#include <sys/types.h>
> +#include <unistd.h>
> +#include <fcntl.h>
> +
> +#include <rte_errno.h>
> +#include <rte_bus.h>
> +#include <rte_per_lcore.h>
> +#include <rte_memory.h>
> +#include <rte_memzone.h>
> +#include <rte_eal.h>
> +#include <rte_common.h>
> +
> +#include <rte_devargs.h>
> +#include <rte_kvargs.h>
> +#include <rte_alarm.h>
> +
> +#include "rte_bus_ifpga.h"
> +#include "ifpga_logs.h"
> +#include "ifpga_common.h"
> +
> +int rte_ifpga_get_string_arg(const char *key __rte_unused,
> + const char *value, void *extra_args)
> +{
> + if (!value || !extra_args)
> + return -EINVAL;
> +
> + *(char **)extra_args =3D strdup(value);
> +
> + if (!*(char **)extra_args)
> + return -ENOMEM;
> +
> + return 0;
> +}
> +int rte_ifpga_get_integer32_arg(const char *key __rte_unused,
> + const char *value, void *extra_args)
> +{
> + if (!value || !extra_args)
> + return -EINVAL;
> +
> + *(int *)extra_args =3D strtoull(value, NULL, 0);
> +
> + return 0;
> +}
> +int ifpga_get_integer64_arg(const char *key __rte_unused,
> + const char *value, void *extra_args)
> +{
> + if (!value || !extra_args)
> + return -EINVAL;
> +
> + *(uint64_t *)extra_args =3D strtoull(value, NULL, 0);
> +
> + return 0;
> +}
> +int ifpga_get_unsigned_long(const char *str, int base) {
> + unsigned long num;
> + char *end =3D NULL;
> +
> + errno =3D 0;
> +
> + num =3D strtoul(str, &end, base);
> + if ((str[0] =3D=3D '\0') || (end =3D=3D NULL) || (*end !=3D '\0') || (e=
rrno !=3D 0))
> + return -1;
> +
> + return num;
> +}
> +
> +int ifpga_afu_id_cmp(const struct rte_afu_id *afu_id0,
> + const struct rte_afu_id *afu_id1)
> +{
> + if ((afu_id0->uuid.uuid_low =3D=3D afu_id1->uuid.uuid_low) &&
> + (afu_id0->uuid.uuid_high =3D=3D afu_id1->uuid.uuid_high) &&
> + (afu_id0->port =3D=3D afu_id1->port)) {
> + return 0;
> + } else
> + return 1;
> +}
> diff --git a/drivers/bus/ifpga/ifpga_common.h
> b/drivers/bus/ifpga/ifpga_common.h
> new file mode 100644
> index 0000000..f9254b9
> --- /dev/null
> +++ b/drivers/bus/ifpga/ifpga_common.h
> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2018 Intel Corporation */
> +
> +#ifndef _IFPGA_COMMON_H_
> +#define _IFPGA_COMMON_H_
> +
> +int rte_ifpga_get_string_arg(const char *key __rte_unused,
> + const char *value, void *extra_args);
> +int rte_ifpga_get_integer32_arg(const char *key __rte_unused,
> + const char *value, void *extra_args);
> +int ifpga_get_integer64_arg(const char *key __rte_unused,
> + const char *value, void *extra_args);
> +int ifpga_get_unsigned_long(const char *str, int base); int
> +ifpga_afu_id_cmp(const struct rte_afu_id *afu_id0,
> + const struct rte_afu_id *afu_id1);
> +
> +#endif /* _IFPGA_COMMON_H_ */
> diff --git a/drivers/bus/ifpga/ifpga_logs.h b/drivers/bus/ifpga/ifpga_log=
s.h
> new file mode 100644 index 0000000..873e0a4
> --- /dev/null
> +++ b/drivers/bus/ifpga/ifpga_logs.h
> @@ -0,0 +1,31 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2018 Intel Corporation */
> +
> +#ifndef _IFPGA_LOGS_H_
> +#define _IFPGA_LOGS_H_
> +
> +#include <rte_log.h>
> +
> +extern int ifpga_bus_logtype;
> +
> +#define IFPGA_LOG(level, fmt, args...) \
> + rte_log(RTE_LOG_ ## level, ifpga_bus_logtype, "%s(): " fmt "\n", \
> + __func__, ##args)
> +
> +#define IFPGA_BUS_LOG(level, fmt, args...) \
> + rte_log(RTE_LOG_ ## level, ifpga_bus_logtype, "%s(): " fmt "\n", \
> + __func__, ##args)
> +
> +#define IFPGA_BUS_FUNC_TRACE() IFPGA_BUS_LOG(DEBUG, ">>")
> +
> +#define IFPGA_BUS_DEBUG(fmt, args...) \
> + IFPGA_BUS_LOG(DEBUG, fmt, ## args)
> +#define IFPGA_BUS_INFO(fmt, args...) \
> + IFPGA_BUS_LOG(INFO, fmt, ## args)
> +#define IFPGA_BUS_ERR(fmt, args...) \
> + IFPGA_BUS_LOG(ERR, fmt, ## args)
> +#define IFPGA_BUS_WARN(fmt, args...) \
> + IFPGA_BUS_LOG(WARNING, fmt, ## args)
> +
> +#endif /* _IFPGA_BUS_LOGS_H_ */
> diff --git a/drivers/bus/ifpga/meson.build b/drivers/bus/ifpga/meson.buil=
d
> new file mode 100644 index 0000000..c9b08c8
> --- /dev/null
> +++ b/drivers/bus/ifpga/meson.build
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2018 Intel
> +Corporation
> +
> +deps +=3D ['pci', 'kvargs', 'rawdev']
> +install_headers('rte_bus_ifpga.h')
> +sources =3D files('ifpga_common.c', 'ifpga_bus.c')
> +
> +allow_experimental_apis =3D true
> diff --git a/drivers/bus/ifpga/rte_bus_ifpga.h
> b/drivers/bus/ifpga/rte_bus_ifpga.h
> new file mode 100644
> index 0000000..5c559e1
> --- /dev/null
> +++ b/drivers/bus/ifpga/rte_bus_ifpga.h
> @@ -0,0 +1,160 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2018 Intel Corporation */
> +
> +#ifndef _RTE_BUS_IFPGA_H_
> +#define _RTE_BUS_IFPGA_H_
> +
> +/**
> + * @file
> + *
> + * RTE Intel FPGA Bus Interface
> + */
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +#include <rte_bus.h>
> +#include <rte_pci.h>
> +
> +/** Name of Intel FPGA Bus */
> +#define IFPGA_BUS_NAME ifpga
> +
> +/* Forward declarations */
> +struct rte_afu_device;
> +
> +/** List of Intel AFU devices */
> +TAILQ_HEAD(afu_device_list, rte_afu_device);
> +/** Double linked list of AFU device drivers. */
> +TAILQ_HEAD(afu_driver_list, rte_afu_driver);
> +
> +#define IFPGA_BUS_BITSTREAM_PATH_MAX_LEN 256
> +
> +struct rte_afu_uuid {
> + uint64_t uuid_low;
> + uint64_t uuid_high;
> +} __attribute__ ((packed));
> +
> +#define IFPGA_BUS_DEV_PORT_MAX 4
> +
> +/**
> + * A structure describing an ID for a AFU driver. Each driver provides
> +a
> + * table of these IDs for each device that it supports.
> + */
> +struct rte_afu_id {
> + struct rte_afu_uuid uuid;
> + int port; /**< port number */
> +} __attribute__ ((packed));
> +
> +/**
> + * A structure PR (Partial Reconfiguration) configuration AFU driver.
> + */
> +
> +struct rte_afu_pr_conf {
> + struct rte_afu_id afu_id;
> + int pr_enable;
> + char bs_path[IFPGA_BUS_BITSTREAM_PATH_MAX_LEN];
> +};
> +
> +#define AFU_PRI_STR_SIZE (PCI_PRI_STR_SIZE + 8)
> +
> +/**
> + * A structure describing a fpga device.
> + */
> +struct rte_ifpga_device {
> + TAILQ_ENTRY(rte_ifpga_device) next; /**< Next in device list. */
> + struct rte_rawdev *rdev;
> + struct afu_device_list afu_list; /**< List of AFU devices */ };
> +
> +/**
> + * A structure describing a AFU device.
> + */
> +struct rte_afu_device {
> + TAILQ_ENTRY(rte_afu_device) next; /**< Next in device list. */
> + struct rte_device device; /**< Inherit core device */
> + struct rte_rawdev *rawdev; /**< Point Rawdev */
> + struct rte_ifpga_device *ifpga_dev; /**< Point ifpga device */
> + struct rte_afu_id id; /**< AFU id within FPGA. */
> + uint32_t num_region; /**< number of regions found */
> + struct rte_mem_resource mem_resource[PCI_MAX_RESOURCE];
> + /**< AFU Memory Resource */
> + struct rte_intr_handle intr_handle; /**< Interrupt handle */
> + struct rte_afu_driver *driver; /**< Associated driver */
> + char path[IFPGA_BUS_BITSTREAM_PATH_MAX_LEN];
> +} __attribute__ ((packed));
> +
> +/**
> + * @internal
> + * Helper macro for drivers that need to convert to struct rte_afu_devic=
e.
> + */
> +#define RTE_DEV_TO_AFU(ptr) \
> + container_of(ptr, struct rte_afu_device, device)
> +
> +#define RTE_DRV_TO_AFU_CONST(ptr) \
> + container_of(ptr, const struct rte_afu_driver, driver)
> +
> +/**
> + * Initialisation function for the driver called during FPGA BUS probing=
.
typo: Initialization
> + */
> +typedef int (afu_probe_t)(struct rte_afu_device *);
> +
> +/**
> + * Uninitialisation function for the driver called during hotplugging.
typo: Unintialization
> + */
> +typedef int (afu_remove_t)(struct rte_afu_device *);
> +
> +/**
> + * A structure describing a AFU device.
> + */
> +struct rte_afu_driver {
> + TAILQ_ENTRY(rte_afu_driver) next; /**< Next afu driver. */
> + struct rte_driver driver; /**< Inherit core driver. */
> + afu_probe_t *probe; /**< Device Probe function.
> */
> + afu_remove_t *remove; /**< Device Remove
> function. */
> + const struct rte_afu_uuid *id_table; /**< AFU uuid within FPGA. */
> + uint32_t drv_flags; /**< Flags contolling handling of device. *=
/
Typo Controlling
And what is "flags controlling handling of device?"
> +};
> +
> +static inline const char *
> +rte_ifpga_device_name(const struct rte_afu_device *afu) {
> + if (afu && afu->device.name)
> + return afu->device.name;
> + return NULL;
> +}
> +
> +/**
> + * Register a ifpga afu device driver.
> + *
> + * @param driver
> + * A pointer to a rte_afu_driver structure describing the driver
> + * to be registered.
> + */
> +void rte_ifpga_driver_register(struct rte_afu_driver *driver);
> +
> +/**
> + * Unregister a ifpga afu device driver.
> + *
> + * @param driver
> + * A pointer to a rte_afu_driver structure describing the driver
> + * to be unregistered.
> + */
> +void rte_ifpga_driver_unregister(struct rte_afu_driver *driver);
> +
> +#define RTE_PMD_REGISTER_AFU(nm, afudrv)\ RTE_INIT(afudrvinitfn_
> +##afudrv);\ static const char *afudrvinit_ ## nm ## _alias;\ static
> +void afudrvinitfn_ ##afudrv(void)\ {\
> + (afudrv).driver.name =3D RTE_STR(nm);\
> + (afudrv).driver.alias =3D afudrvinit_ ## nm ## _alias;\
> + rte_ifpga_driver_register(&afudrv);\
> +} \
> +RTE_PMD_EXPORT_NAME(nm, __COUNTER__)
> +
> +#define RTE_PMD_REGISTER_AFU_ALIAS(nm, alias)\ static const char
> +*afudrvinit_ ## nm ## _alias =3D RTE_STR(alias)
> +
> +#endif /* _RTE_BUS_IFPGA_H_ */
> diff --git a/drivers/bus/ifpga/rte_bus_ifpga_version.map
> b/drivers/bus/ifpga/rte_bus_ifpga_version.map
> new file mode 100644
> index 0000000..a027979
> --- /dev/null
> +++ b/drivers/bus/ifpga/rte_bus_ifpga_version.map
> @@ -0,0 +1,10 @@
> +DPDK_18.05 {
> + global:
> +
> + rte_ifpga_get_integer32_arg;
> + rte_ifpga_get_string_arg;
> + rte_ifpga_driver_register;
> + rte_ifpga_driver_unregister;
> +
> + local: *;
> +};
> diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build index
> 58dfbe2..52c755d 100644
> --- a/drivers/bus/meson.build
> +++ b/drivers/bus/meson.build
> @@ -1,7 +1,7 @@
> # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel
> Corporation
>=20
> -drivers =3D ['dpaa', 'fslmc', 'pci', 'vdev']
> +drivers =3D ['dpaa', 'fslmc', 'ifpga', 'pci', 'vdev']
> std_deps =3D ['eal']
> config_flag_fmt =3D 'RTE_LIBRTE_@0@_BUS'
> driver_name_fmt =3D 'rte_bus_@0@'
> diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 26f3563..3861e1a 100644
> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -255,6 +255,7 @@ ifeq
> ($(CONFIG_RTE_EAL_VFIO)$(CONFIG_RTE_LIBRTE_FSLMC_BUS),yy)
> _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV) +=3D
> -lrte_pmd_dpaa2_cmdif
> _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV) +=3D
> -lrte_pmd_dpaa2_qdma endif # CONFIG_RTE_LIBRTE_FSLMC_BUS
> +_LDLIBS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) +=3D -lrte_bus_ifpga
> endif # CONFIG_RTE_LIBRTE_RAWDEV
>=20
>=20
> --
> 1.8.3.1
Regards
Qi