From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0EA52430C1; Mon, 21 Aug 2023 13:37:24 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AB4E04327C; Mon, 21 Aug 2023 13:36:10 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 804D843279 for ; Mon, 21 Aug 2023 13:36:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692617768; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RtOCmrC/mGL2gwKPK9UczjKzovXsOxc8CU0W4dk+byQ=; b=ECX9V5XBy3SVvMXrcDv4ePxkkuEAFdgzkH7bsu6+6WRsOrlZuahc6Rc2eWBlAcHmbziXpy q+9dUQtowRvCr0o90ZF3CCoTUL6nexqfdB+b5y/SY2TK17PT+pGeLMMfWtKjJmCzzHNz91 MLOYUR6dP/6PgGKohXcFbtYpo9ayIhA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-31-PhMY_1bhNu6vgFa1zV2rgQ-1; Mon, 21 Aug 2023 07:36:05 -0400 X-MC-Unique: PhMY_1bhNu6vgFa1zV2rgQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 83A14185A794; Mon, 21 Aug 2023 11:36:04 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.226.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 387D62166B25; Mon, 21 Aug 2023 11:36:03 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, ferruh.yigit@amd.com, chenbo.xia@intel.com, nipun.gupta@amd.com, bruce.richardson@intel.com, Anatoly Burakov Subject: [PATCH v2 03/15] bus/pci: rework MSIX discovery with VFIO Date: Mon, 21 Aug 2023 13:35:36 +0200 Message-ID: <20230821113549.3191921-4-david.marchand@redhat.com> In-Reply-To: <20230821113549.3191921-1-david.marchand@redhat.com> References: <20230803075038.307012-1-david.marchand@redhat.com> <20230821113549.3191921-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This is a preparatory step before using new helpers for finding PCI capabilities. In the code querying PCI capabilities for checking MSIX availability, replace direct calls to VFIO fd with the existing helpers for reading PCI configuration space: this requires setting VFIO fd in the PCI device object than was done before this change and removes the need to pass around this vfio_dev_fd variable. Signed-off-by: David Marchand --- drivers/bus/pci/linux/pci_vfio.c | 27 ++++++++++----------------- 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c index 8fa7fa458f..958f8b3b52 100644 --- a/drivers/bus/pci/linux/pci_vfio.c +++ b/drivers/bus/pci/linux/pci_vfio.c @@ -107,23 +107,16 @@ pci_vfio_write_config(const struct rte_pci_device *dev, /* get PCI BAR number where MSI-X interrupts are */ static int -pci_vfio_get_msix_bar(const struct rte_pci_device *dev, int fd, +pci_vfio_get_msix_bar(const struct rte_pci_device *dev, struct pci_msix_table *msix_table) { int ret; uint32_t reg; uint16_t flags; uint8_t cap_id, cap_offset; - uint64_t size, offset; - - if (pci_vfio_get_region(dev, VFIO_PCI_CONFIG_REGION_INDEX, - &size, &offset) != 0) { - RTE_LOG(ERR, EAL, "Cannot get offset of CONFIG region.\n"); - return -1; - } /* read PCI capability pointer from config space */ - ret = pread64(fd, ®, sizeof(reg), offset + PCI_CAPABILITY_LIST); + ret = rte_pci_read_config(dev, ®, sizeof(reg), PCI_CAPABILITY_LIST); if (ret != sizeof(reg)) { RTE_LOG(ERR, EAL, "Cannot read capability pointer from PCI config space!\n"); @@ -136,7 +129,7 @@ pci_vfio_get_msix_bar(const struct rte_pci_device *dev, int fd, while (cap_offset) { /* read PCI capability ID */ - ret = pread64(fd, ®, sizeof(reg), offset + cap_offset); + ret = rte_pci_read_config(dev, ®, sizeof(reg), cap_offset); if (ret != sizeof(reg)) { RTE_LOG(ERR, EAL, "Cannot read capability ID from PCI config space!\n"); @@ -148,7 +141,7 @@ pci_vfio_get_msix_bar(const struct rte_pci_device *dev, int fd, /* if we haven't reached MSI-X, check next capability */ if (cap_id != PCI_CAP_ID_MSIX) { - ret = pread64(fd, ®, sizeof(reg), offset + cap_offset); + ret = rte_pci_read_config(dev, ®, sizeof(reg), cap_offset); if (ret != sizeof(reg)) { RTE_LOG(ERR, EAL, "Cannot read capability pointer from PCI config space!\n"); @@ -163,14 +156,14 @@ pci_vfio_get_msix_bar(const struct rte_pci_device *dev, int fd, /* else, read table offset */ else { /* table offset resides in the next 4 bytes */ - ret = pread64(fd, ®, sizeof(reg), offset + cap_offset + 4); + ret = rte_pci_read_config(dev, ®, sizeof(reg), cap_offset + 4); if (ret != sizeof(reg)) { RTE_LOG(ERR, EAL, "Cannot read table offset from PCI config space!\n"); return -1; } - ret = pread64(fd, &flags, sizeof(flags), offset + cap_offset + 2); + ret = rte_pci_read_config(dev, &flags, sizeof(flags), cap_offset + 2); if (ret != sizeof(flags)) { RTE_LOG(ERR, EAL, "Cannot read table flags from PCI config space!\n"); @@ -306,9 +299,6 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) if (rte_intr_fd_set(dev->intr_handle, fd)) return -1; - if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd)) - return -1; - switch (i) { case VFIO_PCI_MSIX_IRQ_INDEX: intr_mode = RTE_INTR_MODE_MSIX; @@ -838,6 +828,9 @@ pci_vfio_map_resource_primary(struct rte_pci_device *dev) if (ret) return ret; + if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd)) + goto err_vfio_dev_fd; + /* allocate vfio_res and get region info */ vfio_res = rte_zmalloc("VFIO_RES", sizeof(*vfio_res), 0); if (vfio_res == NULL) { @@ -869,7 +862,7 @@ pci_vfio_map_resource_primary(struct rte_pci_device *dev) /* get MSI-X BAR, if any (we have to know where it is because we can't * easily mmap it when using VFIO) */ - ret = pci_vfio_get_msix_bar(dev, vfio_dev_fd, &vfio_res->msix_table); + ret = pci_vfio_get_msix_bar(dev, &vfio_res->msix_table); if (ret < 0) { RTE_LOG(ERR, EAL, "%s cannot get MSI-X BAR number!\n", pci_addr); -- 2.41.0