From: "Xia, Chenbo" <chenbo.xia@intel.com>
To: David Marchand <david.marchand@redhat.com>,
"dev@dpdk.org" <dev@dpdk.org>
Cc: "thomas@monjalon.net" <thomas@monjalon.net>,
"ferruh.yigit@amd.com" <ferruh.yigit@amd.com>,
"nipun.gupta@amd.com" <nipun.gupta@amd.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>,
"Burakov, Anatoly" <anatoly.burakov@intel.com>
Subject: RE: [PATCH v2 03/15] bus/pci: rework MSIX discovery with VFIO
Date: Wed, 6 Sep 2023 13:03:19 +0000 [thread overview]
Message-ID: <CH3PR11MB8362088359D8AF2F603DC0149CEFA@CH3PR11MB8362.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20230821113549.3191921-4-david.marchand@redhat.com>
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Monday, August 21, 2023 7:36 PM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; ferruh.yigit@amd.com; Xia, Chenbo
> <chenbo.xia@intel.com>; nipun.gupta@amd.com; Richardson, Bruce
> <bruce.richardson@intel.com>; Burakov, Anatoly <anatoly.burakov@intel.com>
> Subject: [PATCH v2 03/15] bus/pci: rework MSIX discovery with VFIO
>
> This is a preparatory step before using new helpers for finding PCI
> capabilities.
> In the code querying PCI capabilities for checking MSIX availability,
> replace direct calls to VFIO fd with the existing helpers for reading
> PCI configuration space: this requires setting VFIO fd in the PCI
> device object than was done before this change and removes the need to
> pass around this vfio_dev_fd variable.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> drivers/bus/pci/linux/pci_vfio.c | 27 ++++++++++-----------------
> 1 file changed, 10 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/bus/pci/linux/pci_vfio.c
> b/drivers/bus/pci/linux/pci_vfio.c
> index 8fa7fa458f..958f8b3b52 100644
> --- a/drivers/bus/pci/linux/pci_vfio.c
> +++ b/drivers/bus/pci/linux/pci_vfio.c
> @@ -107,23 +107,16 @@ pci_vfio_write_config(const struct rte_pci_device
> *dev,
>
> /* get PCI BAR number where MSI-X interrupts are */
> static int
> -pci_vfio_get_msix_bar(const struct rte_pci_device *dev, int fd,
> +pci_vfio_get_msix_bar(const struct rte_pci_device *dev,
> struct pci_msix_table *msix_table)
> {
> int ret;
> uint32_t reg;
> uint16_t flags;
> uint8_t cap_id, cap_offset;
> - uint64_t size, offset;
> -
> - if (pci_vfio_get_region(dev, VFIO_PCI_CONFIG_REGION_INDEX,
> - &size, &offset) != 0) {
> - RTE_LOG(ERR, EAL, "Cannot get offset of CONFIG region.\n");
> - return -1;
> - }
>
> /* read PCI capability pointer from config space */
> - ret = pread64(fd, ®, sizeof(reg), offset + PCI_CAPABILITY_LIST);
> + ret = rte_pci_read_config(dev, ®, sizeof(reg),
> PCI_CAPABILITY_LIST);
> if (ret != sizeof(reg)) {
> RTE_LOG(ERR, EAL,
> "Cannot read capability pointer from PCI config
> space!\n");
> @@ -136,7 +129,7 @@ pci_vfio_get_msix_bar(const struct rte_pci_device *dev,
> int fd,
> while (cap_offset) {
>
> /* read PCI capability ID */
> - ret = pread64(fd, ®, sizeof(reg), offset + cap_offset);
> + ret = rte_pci_read_config(dev, ®, sizeof(reg), cap_offset);
> if (ret != sizeof(reg)) {
> RTE_LOG(ERR, EAL,
> "Cannot read capability ID from PCI config
> space!\n");
> @@ -148,7 +141,7 @@ pci_vfio_get_msix_bar(const struct rte_pci_device *dev,
> int fd,
>
> /* if we haven't reached MSI-X, check next capability */
> if (cap_id != PCI_CAP_ID_MSIX) {
> - ret = pread64(fd, ®, sizeof(reg), offset +
> cap_offset);
> + ret = rte_pci_read_config(dev, ®, sizeof(reg),
> cap_offset);
> if (ret != sizeof(reg)) {
> RTE_LOG(ERR, EAL,
> "Cannot read capability pointer from PCI
> config space!\n");
> @@ -163,14 +156,14 @@ pci_vfio_get_msix_bar(const struct rte_pci_device
> *dev, int fd,
> /* else, read table offset */
> else {
> /* table offset resides in the next 4 bytes */
> - ret = pread64(fd, ®, sizeof(reg), offset + cap_offset
> + 4);
> + ret = rte_pci_read_config(dev, ®, sizeof(reg),
> cap_offset + 4);
> if (ret != sizeof(reg)) {
> RTE_LOG(ERR, EAL,
> "Cannot read table offset from PCI config
> space!\n");
> return -1;
> }
>
> - ret = pread64(fd, &flags, sizeof(flags), offset +
> cap_offset + 2);
> + ret = rte_pci_read_config(dev, &flags, sizeof(flags),
> cap_offset + 2);
> if (ret != sizeof(flags)) {
> RTE_LOG(ERR, EAL,
> "Cannot read table flags from PCI config
> space!\n");
> @@ -306,9 +299,6 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev,
> int vfio_dev_fd)
> if (rte_intr_fd_set(dev->intr_handle, fd))
> return -1;
>
> - if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd))
> - return -1;
> -
> switch (i) {
> case VFIO_PCI_MSIX_IRQ_INDEX:
> intr_mode = RTE_INTR_MODE_MSIX;
> @@ -838,6 +828,9 @@ pci_vfio_map_resource_primary(struct rte_pci_device
> *dev)
> if (ret)
> return ret;
>
> + if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd))
> + goto err_vfio_dev_fd;
> +
> /* allocate vfio_res and get region info */
> vfio_res = rte_zmalloc("VFIO_RES", sizeof(*vfio_res), 0);
> if (vfio_res == NULL) {
> @@ -869,7 +862,7 @@ pci_vfio_map_resource_primary(struct rte_pci_device
> *dev)
> /* get MSI-X BAR, if any (we have to know where it is because we
> can't
> * easily mmap it when using VFIO)
> */
> - ret = pci_vfio_get_msix_bar(dev, vfio_dev_fd, &vfio_res->msix_table);
> + ret = pci_vfio_get_msix_bar(dev, &vfio_res->msix_table);
> if (ret < 0) {
> RTE_LOG(ERR, EAL, "%s cannot get MSI-X BAR number!\n",
> pci_addr);
> --
> 2.41.0
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
next prev parent reply other threads:[~2023-09-06 13:04 UTC|newest]
Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-03 7:50 [PATCH 00/14] Cleanup PCI(e) drivers David Marchand
2023-08-03 7:50 ` [PATCH 01/14] drivers: remove duplicated PCI master control David Marchand
2023-08-03 9:45 ` Bruce Richardson
2023-10-07 2:53 ` fengchengwen
2023-08-03 7:50 ` [PATCH 02/14] bus/pci: add const to some experimental API David Marchand
2023-08-03 9:46 ` Bruce Richardson
2023-08-03 11:50 ` David Marchand
2023-08-03 7:50 ` [PATCH 03/14] bus/pci: find PCI capability David Marchand
2023-08-03 9:49 ` Bruce Richardson
2023-08-03 9:52 ` Bruce Richardson
2023-08-03 11:49 ` David Marchand
2023-08-03 7:50 ` [PATCH 04/14] pci: define some capability constants David Marchand
2023-08-03 9:51 ` Bruce Richardson
2023-08-03 7:50 ` [PATCH 05/14] pci: define some MSIX constants David Marchand
2023-08-03 9:53 ` Bruce Richardson
2023-08-03 7:50 ` [PATCH 06/14] pci: define some command constants David Marchand
2023-08-03 9:57 ` Bruce Richardson
2023-08-03 11:51 ` David Marchand
2023-08-08 9:20 ` David Marchand
2023-08-08 10:08 ` Bruce Richardson
2023-08-22 19:23 ` Adam Hassick
2023-08-03 7:50 ` [PATCH 07/14] pci: define some BAR constants David Marchand
2023-08-03 9:58 ` Bruce Richardson
2023-08-03 7:50 ` [PATCH 08/14] pci: define some PM constants David Marchand
2023-08-03 9:59 ` Bruce Richardson
2023-08-03 7:50 ` [PATCH 09/14] pci: define some PCIe constants David Marchand
2023-08-03 10:01 ` Bruce Richardson
2023-08-03 7:50 ` [PATCH 10/14] pci: define some extended capability constants David Marchand
2023-08-03 7:50 ` [PATCH 11/14] pci: define some ACS constants David Marchand
2023-08-03 7:50 ` [PATCH 12/14] pci: define some PRI constants David Marchand
2023-08-03 7:50 ` [PATCH 13/14] pci: define some AER constants David Marchand
2023-08-03 7:50 ` [PATCH 14/14] devtools: forbid inclusion of Linux header for PCI David Marchand
2023-08-03 10:03 ` [PATCH 00/14] Cleanup PCI(e) drivers Bruce Richardson
2023-08-21 11:35 ` [PATCH v2 00/15] " David Marchand
2023-08-21 11:35 ` [PATCH v2 01/15] drivers: remove duplicated PCI master control David Marchand
2023-09-06 13:02 ` Xia, Chenbo
2023-08-21 11:35 ` [PATCH v2 02/15] bus/pci: add const to some experimental API David Marchand
2023-08-21 16:14 ` Tyler Retzlaff
2023-09-06 13:02 ` Xia, Chenbo
2023-08-21 11:35 ` [PATCH v2 03/15] bus/pci: rework MSIX discovery with VFIO David Marchand
2023-09-06 13:03 ` Xia, Chenbo [this message]
2023-08-21 11:35 ` [PATCH v2 04/15] bus/pci: find PCI capability David Marchand
2023-09-07 12:43 ` Xia, Chenbo
2023-09-14 12:29 ` David Marchand
2023-09-19 2:19 ` Xia, Chenbo
2023-09-19 9:00 ` David Marchand
2023-08-21 11:35 ` [PATCH v2 05/15] pci: define some capability constants David Marchand
2023-09-07 13:15 ` Xia, Chenbo
2023-08-21 11:35 ` [PATCH v2 06/15] pci: define some MSIX constants David Marchand
2023-09-07 13:15 ` Xia, Chenbo
2023-08-21 11:35 ` [PATCH v2 07/15] pci: define some command constants David Marchand
2023-09-07 13:15 ` Xia, Chenbo
2023-08-21 11:35 ` [PATCH v2 08/15] pci: define some BAR constants David Marchand
2023-09-07 13:16 ` Xia, Chenbo
2023-08-21 11:35 ` [PATCH v2 09/15] pci: define some PM constants David Marchand
2023-09-07 13:16 ` Xia, Chenbo
2023-08-21 11:35 ` [PATCH v2 10/15] pci: define some PCIe constants David Marchand
2023-09-07 13:16 ` Xia, Chenbo
2023-08-21 11:35 ` [PATCH v2 11/15] pci: define some extended capability constants David Marchand
2023-09-07 13:23 ` Xia, Chenbo
2023-08-21 11:35 ` [PATCH v2 12/15] pci: define some ACS constants David Marchand
2023-08-21 11:35 ` [PATCH v2 13/15] pci: define some PRI constants David Marchand
2023-08-21 11:35 ` [PATCH v2 14/15] pci: define some AER constants David Marchand
2023-08-21 11:35 ` [PATCH v2 15/15] devtools: forbid inclusion of Linux header for PCI David Marchand
2023-08-21 16:24 ` Tyler Retzlaff
2023-09-07 13:33 ` Xia, Chenbo
2023-08-22 15:30 ` [PATCH v2 00/15] Cleanup PCI(e) drivers Patrick Robb
2023-08-22 16:09 ` [PATCH 00/14] " Adam Hassick
2023-08-22 16:48 ` Adam Hassick
2023-08-24 15:44 ` Adam Hassick
2023-09-14 12:35 ` [PATCH v3 00/15] " David Marchand
2023-09-14 12:36 ` [PATCH v3 01/15] drivers: remove duplicated PCI master control David Marchand
2023-09-14 12:36 ` [PATCH v3 02/15] bus/pci: add const to some experimental API David Marchand
2023-09-14 12:36 ` [PATCH v3 03/15] bus/pci: rework MSIX discovery with VFIO David Marchand
2023-09-14 12:36 ` [PATCH v3 04/15] bus/pci: find PCI capability David Marchand
2023-09-19 2:33 ` Xia, Chenbo
2023-09-14 12:36 ` [PATCH v3 05/15] pci: define some capability constants David Marchand
2023-09-15 16:27 ` Sevincer, Abdullah
2023-09-14 12:36 ` [PATCH v3 06/15] pci: define some MSIX constants David Marchand
2023-09-14 12:36 ` [PATCH v3 07/15] pci: define some command constants David Marchand
2023-09-14 12:36 ` [PATCH v3 08/15] pci: define some BAR constants David Marchand
2023-09-14 12:36 ` [PATCH v3 09/15] pci: define some PM constants David Marchand
2023-09-14 12:36 ` [PATCH v3 10/15] pci: define some PCIe constants David Marchand
2023-09-15 16:26 ` Sevincer, Abdullah
2023-09-14 12:36 ` [PATCH v3 11/15] pci: define some extended capability constants David Marchand
2023-09-15 16:27 ` Sevincer, Abdullah
2023-09-14 12:36 ` [PATCH v3 12/15] pci: define some ACS constants David Marchand
2023-09-15 16:25 ` Sevincer, Abdullah
2023-09-19 2:35 ` Xia, Chenbo
2023-09-14 12:36 ` [PATCH v3 13/15] pci: define some PRI constants David Marchand
2023-09-15 16:21 ` Sevincer, Abdullah
2023-09-19 2:36 ` Xia, Chenbo
2023-09-14 12:36 ` [PATCH v3 14/15] pci: define some AER constants David Marchand
2023-09-15 16:26 ` Sevincer, Abdullah
2023-09-19 2:36 ` Xia, Chenbo
2023-09-14 12:36 ` [PATCH v3 15/15] devtools: forbid inclusion of Linux header for PCI David Marchand
2023-09-15 15:14 ` [PATCH v3 00/15] Cleanup PCI(e) drivers Stephen Hemminger
2023-09-19 12:41 ` David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CH3PR11MB8362088359D8AF2F603DC0149CEFA@CH3PR11MB8362.namprd11.prod.outlook.com \
--to=chenbo.xia@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=bruce.richardson@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=nipun.gupta@amd.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).