From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 35F6EA0A03;
Tue, 19 Jan 2021 08:30:53 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 2DE1F140DBA;
Tue, 19 Jan 2021 08:30:05 +0100 (CET)
Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129])
by mails.dpdk.org (Postfix) with ESMTP id 0DB5A140D45
for <dev@dpdk.org>; Tue, 19 Jan 2021 08:29:53 +0100 (CET)
Received: from Internal Mail-Server by MTLPINE1 (envelope-from
xuemingl@nvidia.com) with SMTP; 19 Jan 2021 09:29:52 +0200
Received: from nvidia.com (pegasus05.mtr.labs.mlnx [10.210.16.100])
by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 10J7Tpqo014531;
Tue, 19 Jan 2021 09:29:52 +0200
From: Xueming Li <xuemingl@nvidia.com>
To: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Cc: dev@dpdk.org, xuemingl@nvidia.com, Asaf Penso <asafp@nvidia.com>
Date: Tue, 19 Jan 2021 07:28:11 +0000
Message-Id: <1611041295-12797-6-git-send-email-xuemingl@nvidia.com>
X-Mailer: git-send-email 1.8.3.1
In-Reply-To: <1611041295-12797-1-git-send-email-xuemingl@nvidia.com>
References: <1611041295-12797-1-git-send-email-xuemingl@nvidia.com>
In-Reply-To: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com>
References: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com>
Subject: [dpdk-dev] [PATCH v4 5/8] net/mlx5: support representor from
multiple PFs
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
To probe representors from different kernel bonding PFs, had to specify
2 separate devargs like this:
-a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3]
This patch supports range or list of PF section in devargs, so the
alternative short devargs of above is:
-a 03:00.0,representor=pf[0-1]vf[0-3]
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
doc/guides/nics/mlx5.rst | 4 ++
drivers/net/mlx5/linux/mlx5_os.c | 100 +++++++++++++++++++++----------
2 files changed, 72 insertions(+), 32 deletions(-)
diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index eaca4fc058..480c9d3fc1 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -884,6 +884,10 @@ Driver options
<PCI_BDF>,representor=sf[0-2]
+ To probe VF port representors 0 through 2 on both PFs of bonding device::
+
+ <Primary_PCI_BDF>,representor=pf[0,1]vf[0-2]
+
- ``max_dump_files_num`` parameter [int]
The maximum number of files per PMD entity that may be created for debug information.
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 9ae5910f46..521a0a5789 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1788,21 +1788,25 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev,
}
/**
- * DPDK callback to register a PCI device.
+ * Register a PCI device within bonding.
*
- * This function spawns Ethernet devices out of a given PCI device.
+ * This function spawns Ethernet devices out of a given PCI device and
+ * bonding owner PF index.
*
- * @param[in] pci_drv
- * PCI driver structure (mlx5_driver).
* @param[in] pci_dev
* PCI device information.
+ * @param[in] req_eth_da
+ * Requested ethdev device argument.
+ * @param[in] owner_id
+ * Requested owner PF port ID within bonding device, default to 0.
*
* @return
* 0 on success, a negative errno value otherwise and rte_errno is set.
*/
-int
-mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
- struct rte_pci_device *pci_dev)
+static int
+mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev,
+ struct rte_eth_devargs *req_eth_da,
+ uint16_t owner_id)
{
struct ibv_device **ibv_list;
/*
@@ -1832,7 +1836,7 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
struct mlx5_dev_spawn_data *list = NULL;
struct mlx5_dev_config dev_config;
unsigned int dev_config_vf;
- struct rte_eth_devargs eth_da = { .type = RTE_ETH_REPRESENTOR_NONE };
+ struct rte_eth_devargs eth_da = *req_eth_da;
struct rte_pci_addr owner_pci = pci_dev->addr; /* Owner PF. */
int ret = -1;
@@ -1844,27 +1848,6 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
strerror(rte_errno));
return -rte_errno;
}
- if (pci_dev->device.devargs) {
- /* Parse representor information from device argument. */
- if (pci_dev->device.devargs->cls_str)
- ret = rte_eth_devargs_parse(
- pci_dev->device.devargs->cls_str, ð_da);
- if (ret) {
- DRV_LOG(ERR, "failed to parse device arguments: %s",
- pci_dev->device.devargs->cls_str);
- return -rte_errno;
- }
- if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) {
- /* Support legacy device argument */
- ret = rte_eth_devargs_parse(
- pci_dev->device.devargs->args, ð_da);
- if (ret) {
- DRV_LOG(ERR, "failed to parse device arguments: %s",
- pci_dev->device.devargs->args);
- return -rte_errno;
- }
- }
- }
errno = 0;
ibv_list = mlx5_glue->get_device_list(&ret);
if (!ibv_list) {
@@ -1886,8 +1869,7 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
DRV_LOG(DEBUG, "checking device \"%s\"", ibv_list[ret]->name);
bd = mlx5_device_bond_pci_match
- (ibv_list[ret], &owner_pci, nl_rdma,
- eth_da.ports[0]);
+ (ibv_list[ret], &owner_pci, nl_rdma, owner_id);
if (bd >= 0) {
/*
* Bonding device detected. Only one match is allowed,
@@ -1906,7 +1888,7 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
}
/* Amend owner pci address if owner PF ID specified. */
if (eth_da.nb_representor_ports)
- owner_pci.function += eth_da.ports[0];
+ owner_pci.function += owner_id;
DRV_LOG(INFO, "PCI information matches for"
" slave %d bonding device \"%s\"",
bd, ibv_list[ret]->name);
@@ -2294,6 +2276,60 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
return ret;
}
+/**
+ * DPDK callback to register a PCI device.
+ *
+ * This function spawns Ethernet devices out of a given PCI device.
+ *
+ * @param[in] pci_drv
+ * PCI driver structure (mlx5_driver).
+ * @param[in] pci_dev
+ * PCI device information.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct rte_eth_devargs eth_da = { .type = RTE_ETH_REPRESENTOR_NONE };
+ int ret = 0;
+ uint16_t p;
+
+ if (pci_dev->device.devargs) {
+ /* Parse representor information from device argument. */
+ if (pci_dev->device.devargs->cls_str)
+ ret = rte_eth_devargs_parse(
+ pci_dev->device.devargs->cls_str, ð_da);
+ if (ret) {
+ DRV_LOG(ERR, "failed to parse device arguments: %s",
+ pci_dev->device.devargs->cls_str);
+ return -rte_errno;
+ }
+ if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) {
+ /* Support legacy device argument */
+ ret = rte_eth_devargs_parse(
+ pci_dev->device.devargs->args, ð_da);
+ if (ret) {
+ DRV_LOG(ERR, "failed to parse device arguments: %s",
+ pci_dev->device.devargs->args);
+ return -rte_errno;
+ }
+ }
+ }
+
+ if (eth_da.nb_ports > 0) {
+ /* Iterate all port if devargs pf is range: "pf[0-1]vf[...]". */
+ for (p = 0; p < eth_da.nb_ports; p++)
+ ret = mlx5_os_pci_probe_pf(pci_dev, ð_da,
+ eth_da.ports[p]);
+ } else {
+ ret = mlx5_os_pci_probe_pf(pci_dev, ð_da, 0);
+ }
+ return ret;
+}
+
static int
mlx5_config_doorbell_mapping_env(const struct mlx5_dev_config *config)
{
--
2.25.1