From: "Xia, Chenbo" <chenbo.xia@intel.com>
To: Nipun Gupta <nipun.gupta@amd.com>, "dev@dpdk.org" <dev@dpdk.org>,
"thomas@monjalon.net" <thomas@monjalon.net>,
"david.marchand@redhat.com" <david.marchand@redhat.com>
Cc: "ferruh.yigit@amd.com" <ferruh.yigit@amd.com>,
"harpreet.anand@amd.com" <harpreet.anand@amd.com>,
"nikhil.agarwal@amd.com" <nikhil.agarwal@amd.com>
Subject: RE: [PATCH v4 1/4] bus/cdx: introduce cdx bus
Date: Tue, 9 May 2023 06:54:44 +0000 [thread overview]
Message-ID: <SN6PR11MB35044EF94E0A695E503E420F9C769@SN6PR11MB3504.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20230508111812.2655-2-nipun.gupta@amd.com>
> -----Original Message-----
> From: Nipun Gupta <nipun.gupta@amd.com>
> Sent: Monday, May 8, 2023 7:18 PM
> To: dev@dpdk.org; thomas@monjalon.net; david.marchand@redhat.com
> Cc: ferruh.yigit@amd.com; harpreet.anand@amd.com; nikhil.agarwal@amd.com;
> Nipun Gupta <nipun.gupta@amd.com>
> Subject: [PATCH v4 1/4] bus/cdx: introduce cdx bus
>
> CDX bus supports multiple type of devices, which can be
> exposed to user-space via vfio-cdx.
>
> vfio-cdx provides the MMIO IO_MEMORY regions as well as the
> DMA interface for the device (IOMMU).
>
> This support aims to enable the DPDK to support the cdx
> devices in user-space using VFIO interface.
>
> Signed-off-by: Nipun Gupta <nipun.gupta@amd.com>
> ---
> MAINTAINERS | 5 +
> doc/guides/rel_notes/release_23_07.rst | 6 +
> drivers/bus/cdx/bus_cdx_driver.h | 201 ++++++++++
> drivers/bus/cdx/cdx.c | 520 +++++++++++++++++++++++++
> drivers/bus/cdx/cdx_logs.h | 37 ++
> drivers/bus/cdx/cdx_vfio.c | 437 +++++++++++++++++++++
> drivers/bus/cdx/meson.build | 13 +
> drivers/bus/cdx/private.h | 49 +++
> drivers/bus/cdx/version.map | 11 +
> drivers/bus/meson.build | 1 +
> 10 files changed, 1280 insertions(+)
> create mode 100644 drivers/bus/cdx/bus_cdx_driver.h
> create mode 100644 drivers/bus/cdx/cdx.c
> create mode 100644 drivers/bus/cdx/cdx_logs.h
> create mode 100644 drivers/bus/cdx/cdx_vfio.c
> create mode 100644 drivers/bus/cdx/meson.build
> create mode 100644 drivers/bus/cdx/private.h
> create mode 100644 drivers/bus/cdx/version.map
>
...
> --- /dev/null
> +++ b/drivers/bus/cdx/cdx.c
> @@ -0,0 +1,520 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
> + */
> +
> +/*
> + * Architecture Overview
> + * =====================
> + * CDX is a Hardware Architecture designed for AMD FPGA devices. It
> + * consists of sophisticated mechanism for interaction between FPGA,
> + * Firmware and the APUs (Application CPUs).
> + *
> + * Firmware resides on RPU (Realtime CPUs) which interacts with
> + * the FPGA program manager and the APUs. The RPU provides memory-mapped
> + * interface (RPU if) which is used to communicate with APUs.
> + *
> + * The diagram below shows an overview of the CDX architecture:
> + *
> + * +--------------------------------------+
> + * | DPDK |
> + * | DPDK CDX drivers |
> + * | | |
> + * | DPDK CDX bus |
> + * | | |
> + * +-----------------------------|--------+
> + * |
> + * +-----------------------------|--------+
> + * | Application CPUs (APU) | |
> + * | | |
> + * | VFIO CDX driver |
> + * | Linux OS | |
> + * | Linux CDX bus |
> + * | | |
> + * +-----------------------------|--------+
> + * |
> + * |
> + * +------------------------| RPU if |----+
> + * | | |
> + * | V |
> + * | Realtime CPUs (RPU) |
> + * | |
> + * +--------------------------------------+
> + * |
> + * +---------------------|----------------+
> + * | FPGA | |
> + * | +-----------------------+ |
> + * | | | | |
> + * | +-------+ +-------+ +-------+ |
> + * | | dev 1 | | dev 2 | | dev 3 | |
> + * | +-------+ +-------+ +-------+ |
> + * +--------------------------------------+
> + *
> + * The RPU firmware extracts the device information from the loaded FPGA
> + * image and implements a mechanism that allows the APU drivers to
> + * enumerate such devices (device personality and resource details) via
> + * a dedicated communication channel.
What is APU? CPU resources that application uses? Then why cpu resources that DPDK
Uses are not part of it?
I also notice that there are more and more vfio-XXX devices in DPDK now: pci/platform/cdx.
Some vfio related code are very similar. In the future we may need to think about make them
a common lib or something.
Thanks,
Chenbo
> + *
> + * VFIO CDX driver provides the CDX device resources like MMIO and
> interrupts
> + * to map to user-space. DPDK CDX bus uses sysfs interface and the vfio-
> cdx
> + * driver to discover and initialize the CDX devices for user-space
> + * applications.
> + */
> +
next prev parent reply other threads:[~2023-05-09 6:54 UTC|newest]
Thread overview: 102+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-24 14:07 [RFC PATCH 0/6] add support for CDX bus Nipun Gupta
2023-01-24 14:07 ` [RFC PATCH 1/6] bus/cdx: introduce cdx bus Nipun Gupta
2023-01-24 14:07 ` [RFC PATCH 2/6] bus/cdx: add dma map and unmap support Nipun Gupta
2023-01-24 14:07 ` [RFC PATCH 3/6] bus/cdx: add support for MSI Nipun Gupta
2023-01-24 14:07 ` [RFC PATCH 4/6] bus/cdx: support plug unplug and dev iterator Nipun Gupta
2023-01-24 14:07 ` [RFC PATCH 5/6] bus: enable cdx bus Nipun Gupta
2023-01-24 14:07 ` [RFC PATCH 6/6] config/arm: add AMD CDX Nipun Gupta
2023-04-07 6:01 ` [PATCH 0/6] add support for CDX bus Nipun Gupta
2023-04-07 6:01 ` [PATCH 1/6] bus/cdx: introduce cdx bus Nipun Gupta
2023-04-07 6:01 ` [PATCH 2/6] bus/cdx: add dma map and unmap support Nipun Gupta
2023-04-07 6:01 ` [PATCH 3/6] bus/cdx: add support for MSI Nipun Gupta
2023-04-07 6:01 ` [PATCH 4/6] bus/cdx: support plug unplug and dev iterator Nipun Gupta
2023-04-07 6:01 ` [PATCH 5/6] bus: enable cdx bus Nipun Gupta
2023-04-07 6:01 ` [PATCH 6/6] config/arm: add AMD CDX Nipun Gupta
2023-04-07 7:18 ` [PATCH 0/6] add support for CDX bus David Marchand
2023-04-07 7:29 ` Nipun Gupta
2023-04-13 13:25 ` Gupta, Nipun
2023-04-13 13:26 ` [PATCH v2 " Nipun Gupta
2023-04-13 13:26 ` [PATCH v2 1/6] bus/cdx: introduce cdx bus Nipun Gupta
2023-04-14 16:45 ` Ferruh Yigit
2023-04-16 9:20 ` Gupta, Nipun
2023-04-13 13:27 ` [PATCH v2 2/6] bus/cdx: add dma map and unmap support Nipun Gupta
2023-04-13 13:27 ` [PATCH v2 3/6] bus/cdx: add support for MSI Nipun Gupta
2023-04-14 16:45 ` Ferruh Yigit
2023-04-16 9:20 ` Gupta, Nipun
2023-04-13 13:27 ` [PATCH v2 4/6] bus/cdx: support plug unplug and dev iterator Nipun Gupta
2023-04-13 13:27 ` [PATCH v2 5/6] bus: enable cdx bus Nipun Gupta
2023-04-14 16:45 ` Ferruh Yigit
2023-04-16 9:21 ` Gupta, Nipun
2023-04-13 13:27 ` [PATCH v2 6/6] config/arm: add AMD CDX Nipun Gupta
2023-04-14 16:45 ` [PATCH v2 0/6] add support for CDX bus Ferruh Yigit
2023-04-21 14:54 ` [PATCH v3 0/5] Support AMD CDX bus, for FPGA based CDX devices. The CDX Nipun Gupta
2023-04-21 14:54 ` [PATCH v3 1/5] bus/cdx: introduce cdx bus Nipun Gupta
2023-04-21 14:54 ` [PATCH v3 2/5] bus/cdx: add DMA map and unmap support Nipun Gupta
2023-04-21 14:54 ` [PATCH v3 3/5] bus/cdx: add support for MSI Nipun Gupta
2023-04-21 14:54 ` [PATCH v3 4/5] bus/cdx: support plug unplug and dev iterator Nipun Gupta
2023-04-21 14:54 ` [PATCH v3 5/5] config/arm: add AMD CDX Nipun Gupta
2023-05-04 15:28 ` Ferruh Yigit
2023-05-08 10:24 ` Gupta, Nipun
2023-05-08 10:44 ` Thomas Monjalon
2023-05-08 10:48 ` Gupta, Nipun
2023-05-08 11:26 ` Ferruh Yigit
2023-05-08 17:16 ` Honnappa Nagarahalli
2023-05-08 17:47 ` Ferruh Yigit
2023-05-09 4:35 ` Gupta, Nipun
2023-05-09 5:55 ` Ruifeng Wang
2023-06-14 10:30 ` Ferruh Yigit
2023-06-15 7:00 ` Ruifeng Wang
2023-05-08 11:18 ` [PATCH v4 0/4] Support AMD CDX bus, for FPGA based CDX devices. The CDX Nipun Gupta
2023-05-08 11:18 ` [PATCH v4 1/4] bus/cdx: introduce cdx bus Nipun Gupta
2023-05-09 6:54 ` Xia, Chenbo [this message]
2023-05-09 11:09 ` Gupta, Nipun
2023-05-09 11:25 ` Ferruh Yigit
2023-05-10 1:29 ` Xia, Chenbo
2023-05-24 11:14 ` Thomas Monjalon
2023-05-24 17:04 ` Gupta, Nipun
2023-05-08 11:18 ` [PATCH v4 2/4] bus/cdx: add DMA map and unmap support Nipun Gupta
2023-05-08 11:18 ` [PATCH v4 3/4] bus/cdx: add support for MSI Nipun Gupta
2023-05-24 11:06 ` Thomas Monjalon
2023-05-24 17:06 ` Gupta, Nipun
2023-05-08 11:18 ` [PATCH v4 4/4] bus/cdx: support plug unplug and dev iterator Nipun Gupta
2023-05-12 10:25 ` [PATCH v4 0/4] Support AMD CDX bus, for FPGA based CDX devices. The CDX Ferruh Yigit
2023-05-25 10:08 ` [PATCH v5 0/5] Support AMD CDX bus Nipun Gupta
2023-05-25 10:08 ` [PATCH v5 1/5] bus/cdx: introduce " Nipun Gupta
2023-06-01 15:00 ` David Marchand
2023-06-01 18:10 ` Nipun Gupta
2023-06-05 8:04 ` Nipun Gupta
2023-05-25 10:08 ` [PATCH v5 2/5] bus/cdx: add DMA map and unmap support Nipun Gupta
2023-06-01 15:07 ` David Marchand
2023-05-25 10:08 ` [PATCH v5 3/5] eal/interrupts: add IRQ count in interrupt handle Nipun Gupta
2023-06-01 15:25 ` David Marchand
2023-06-01 18:04 ` Nipun Gupta
2023-06-01 18:18 ` Gupta, Nipun
2023-06-06 7:18 ` [EXT] " Harman Kalra
2023-06-06 7:27 ` Nipun Gupta
2023-06-07 5:45 ` Harman Kalra
2023-05-25 10:08 ` [PATCH v5 4/5] bus/cdx: add support for MSI Nipun Gupta
2023-06-01 15:09 ` David Marchand
2023-06-05 8:05 ` Nipun Gupta
2023-05-25 10:08 ` [PATCH v5 5/5] bus/cdx: support plug unplug and dev iterator Nipun Gupta
2023-06-05 13:26 ` [PATCH v6 0/4] Support AMD CDX bus Nipun Gupta
2023-06-05 13:26 ` [PATCH v6 1/4] bus/cdx: introduce " Nipun Gupta
2023-06-06 8:55 ` Thomas Monjalon
2023-06-05 13:26 ` [PATCH v6 2/4] bus/cdx: add DMA map and unmap support Nipun Gupta
2023-06-05 13:26 ` [PATCH v6 3/4] bus/cdx: add support for MSI Nipun Gupta
2023-06-06 9:30 ` David Marchand
2023-06-06 9:42 ` Nipun Gupta
2023-06-05 13:26 ` [PATCH v6 4/4] bus/cdx: support plug unplug and dev iterator Nipun Gupta
2023-06-06 10:02 ` [PATCH v7 0/4] Support AMD CDX bus Nipun Gupta
2023-06-06 10:02 ` [PATCH v7 1/4] bus/cdx: introduce " Nipun Gupta
2023-06-06 13:00 ` Thomas Monjalon
2023-06-06 13:38 ` Nipun Gupta
2023-06-06 13:43 ` Thomas Monjalon
2023-06-06 10:02 ` [PATCH v7 2/4] bus/cdx: add DMA map and unmap support Nipun Gupta
2023-06-06 10:02 ` [PATCH v7 3/4] bus/cdx: add support for MSI Nipun Gupta
2023-06-06 10:02 ` [PATCH v7 4/4] bus/cdx: support plug unplug and dev iterator Nipun Gupta
2023-06-07 4:24 ` [PATCH v8 0/4] Support AMD CDX bus Nipun Gupta
2023-06-07 4:24 ` [PATCH v8 1/4] bus/cdx: introduce " Nipun Gupta
2023-06-07 4:24 ` [PATCH v8 2/4] bus/cdx: add DMA map and unmap support Nipun Gupta
2023-06-07 4:24 ` [PATCH v8 3/4] bus/cdx: add support for MSI Nipun Gupta
2023-06-07 4:24 ` [PATCH v8 4/4] bus/cdx: support plug unplug and dev iterator Nipun Gupta
2023-06-07 13:36 ` [PATCH v8 0/4] Support AMD CDX bus Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=SN6PR11MB35044EF94E0A695E503E420F9C769@SN6PR11MB3504.namprd11.prod.outlook.com \
--to=chenbo.xia@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=harpreet.anand@amd.com \
--cc=nikhil.agarwal@amd.com \
--cc=nipun.gupta@amd.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).