From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <adrien.mazarguil@6wind.com>
Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68])
by dpdk.org (Postfix) with ESMTP id 3ED8F1B451
for <dev@dpdk.org>; Tue, 10 Jul 2018 18:05:13 +0200 (CEST)
Received: by mail-wm0-f68.google.com with SMTP id w16-v6so25076292wmc.2
for <dev@dpdk.org>; Tue, 10 Jul 2018 09:05:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=6wind-com.20150623.gappssmtp.com; s=20150623;
h=date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:in-reply-to;
bh=VXghhL2Z7+VjFqQkx0R/BCV4LUBo98GUAsAKz1Dsfx8=;
b=OmHvo6RLnUaNd4Sjyh3IZbSD0DPZ4X0BTUeC/oEEqTCmtsaLdPH755/0hJ0DsUVer0
ZLKW1yeq77i1kFUHkFQ/pQyYnXEu+l0PlEhEmt4qiq5BZuLdeTriGmCRo0ma2OM+dL5z
W0xMItesVVe21JyMtrniEFy7HWbpe6T2M6AxoKvVDjFA4Gqm9NRvSKs0RbU37ZkUpyb2
K94kxn8rYeVP7DUm76Xc6ltl6bQTOSka9MFgxDpgeVN43DsnJsYSR7EW+kMTjrDbDjHO
avVEVxSzXrYExvuVMTAu3gIUy8aemAkXPLaalyoIDVCZbvKcrsfToW1Z4AIs7feBzhgT
gozQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:date:from:to:cc:subject:message-id:references
:mime-version:content-disposition:in-reply-to;
bh=VXghhL2Z7+VjFqQkx0R/BCV4LUBo98GUAsAKz1Dsfx8=;
b=DYUvg+KvseV1JBBz+0Zr/k/GVpYs06qM8L+3d2+IkJJd2N6NODbAiQlrNsJEPJZFKo
HoPuhbVzA4+mbK5RMTybrv/3RDIGLif5enpqJzTI/P80OYuvmifWnZFmFbUFz00qS4zw
OLJwjK+kyynIYsTKGvWB03TNtuhkKIs/GkP+s71hLjiSvPxAry0Lfj7QpwWEJt0+dazN
V86nSzDWVcOYBVE/cKZLNpASPL5pGHHkEDuOHdepfYmHlDp9fVDI53G445EAq+BOQDKF
2CBE7GHdU6SjCmwWSu8IyuvVuS13FCykJiwSDtUQPqIY6jI+I/ibIoMkXhdplELF8fcX
MJzQ==
X-Gm-Message-State: APt69E2uk/xpcKMxqAFKaWCoGFoqtr9RS6H4W40ySA8dLcHkRWG9m2Rt
rtNv5JxJljPA7vAyOIhrPPKriA==
X-Google-Smtp-Source: AAOMgpco+PC6wh315ozjEhpr4Zm02//xY99hHEfPr3+drSZA2F58NqaiH130mI+0QyssUl/rTT6G4Q==
X-Received: by 2002:a1c:a341:: with SMTP id
m62-v6mr17216787wme.3.1531238712859;
Tue, 10 Jul 2018 09:05:12 -0700 (PDT)
Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78])
by smtp.gmail.com with ESMTPSA id r194-v6sm28424052wmd.36.2018.07.10.09.05.12
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Tue, 10 Jul 2018 09:05:12 -0700 (PDT)
Date: Tue, 10 Jul 2018 18:04:56 +0200
From: Adrien Mazarguil <adrien.mazarguil@6wind.com>
To: Shahaf Shuler <shahafs@mellanox.com>
Cc: dev@dpdk.org
Message-ID: <20180710155743.14448-9-adrien.mazarguil@6wind.com>
References: <20180705083934.5535-1-adrien.mazarguil@6wind.com>
<20180710155743.14448-1-adrien.mazarguil@6wind.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20180710155743.14448-1-adrien.mazarguil@6wind.com>
X-Mailer: git-send-email 2.11.0
Subject: [dpdk-dev] [PATCH v5 08/10] net/mlx5: probe port representors in
natural order
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Jul 2018 16:05:13 -0000
Port representors are probed in whatever unspecified order
ibv_get_device_list() returns them.
This is counterintuitive to users since DPDK port IDs assignment almost
never follows the same sequence as representor IDs. Additionally, the
master device does not necessarily inherit the lowest DPDK port ID.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
--
v3 changes:
- This patch was not present in prior revisions.
---
drivers/net/mlx5/mlx5.c | 95 ++++++++++++++++++++++++++++++++++----------
1 file changed, 74 insertions(+), 21 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index c02afbb82..6592480bf 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1168,6 +1168,52 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
return NULL;
}
+/** Data associated with devices to spawn. */
+struct mlx5_dev_spawn_data {
+ unsigned int ifindex; /**< Network interface index. */
+ struct mlx5_switch_info info; /**< Switch information. */
+ struct ibv_device *ibv_dev; /**< Associated IB device. */
+ struct rte_eth_dev *eth_dev; /**< Associated Ethernet device. */
+};
+
+/**
+ * Comparison callback to sort device data.
+ *
+ * This is meant to be used with qsort().
+ *
+ * @param a[in]
+ * Pointer to pointer to first data object.
+ * @param b[in]
+ * Pointer to pointer to second data object.
+ *
+ * @return
+ * 0 if both objects are equal, less than 0 if the first argument is less
+ * than the second, greater than 0 otherwise.
+ */
+static int
+mlx5_dev_spawn_data_cmp(const void *a, const void *b)
+{
+ const struct mlx5_switch_info *si_a =
+ &((const struct mlx5_dev_spawn_data *)a)->info;
+ const struct mlx5_switch_info *si_b =
+ &((const struct mlx5_dev_spawn_data *)b)->info;
+ int ret;
+
+ /* Master device first. */
+ ret = si_b->master - si_a->master;
+ if (ret)
+ return ret;
+ /* Then representor devices. */
+ ret = si_b->representor - si_a->representor;
+ if (ret)
+ return ret;
+ /* Unidentified devices come last in no specific order. */
+ if (!si_a->representor)
+ return 0;
+ /* Order representors by name. */
+ return si_a->port_name - si_b->port_name;
+}
+
/**
* DPDK callback to register a PCI device.
*
@@ -1218,9 +1264,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
}
ibv_match[n] = NULL;
- unsigned int ifindex[n];
- struct mlx5_switch_info info[n];
- struct rte_eth_dev *eth_list[n];
+ struct mlx5_dev_spawn_data list[n];
int nl_route = n ? mlx5_nl_init(0, NETLINK_ROUTE) : -1;
int nl_rdma = n ? mlx5_nl_init(0, NETLINK_RDMA) : -1;
unsigned int i;
@@ -1242,16 +1286,19 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
* bail out.
*/
for (i = 0; i != n; ++i) {
+ list[i].ibv_dev = ibv_match[i];
+ list[i].eth_dev = NULL;
if (nl_rdma < 0)
- ifindex[i] = 0;
+ list[i].ifindex = 0;
else
- ifindex[i] = mlx5_nl_ifindex(nl_rdma,
- ibv_match[i]->name);
+ list[i].ifindex = mlx5_nl_ifindex
+ (nl_rdma, list[i].ibv_dev->name);
if (nl_route < 0 ||
- !ifindex[i] ||
- mlx5_nl_switch_info(nl_route, ifindex[i], &info[i])) {
- ifindex[i] = 0;
- memset(&info[i], 0, sizeof(info[i]));
+ !list[i].ifindex ||
+ mlx5_nl_switch_info(nl_route, list[i].ifindex,
+ &list[i].info)) {
+ list[i].ifindex = 0;
+ memset(&list[i].info, 0, sizeof(list[i].info));
continue;
}
}
@@ -1261,7 +1308,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
close(nl_route);
/* Count unidentified devices. */
for (u = 0, i = 0; i != n; ++i)
- if (!info[i].master && !info[i].representor)
+ if (!list[i].info.master && !list[i].info.representor)
++u;
if (u) {
if (n == 1 && u == 1) {
@@ -1275,6 +1322,12 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
n = 0;
}
}
+ /*
+ * Sort list to probe devices in natural order for users convenience
+ * (i.e. master first, then representors from lowest to highest ID).
+ */
+ if (n)
+ qsort(list, n, sizeof(*list), mlx5_dev_spawn_data_cmp);
switch (pci_dev->id.device_id) {
case PCI_DEVICE_ID_MELLANOX_CONNECTX4VF:
case PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF:
@@ -1288,15 +1341,15 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
for (i = 0; i != n; ++i) {
uint32_t restore;
- eth_list[i] = mlx5_dev_spawn(&pci_dev->device, ibv_match[i],
- vf, &info[i]);
- if (!eth_list[i])
+ list[i].eth_dev = mlx5_dev_spawn
+ (&pci_dev->device, list[i].ibv_dev, vf, &list[i].info);
+ if (!list[i].eth_dev)
break;
- restore = eth_list[i]->data->dev_flags;
- rte_eth_copy_pci_info(eth_list[i], pci_dev);
+ restore = list[i].eth_dev->data->dev_flags;
+ rte_eth_copy_pci_info(list[i].eth_dev, pci_dev);
/* Restore non-PCI flags cleared by the above call. */
- eth_list[i]->data->dev_flags |= restore;
- rte_eth_dev_probing_finish(eth_list[i]);
+ list[i].eth_dev->data->dev_flags |= restore;
+ rte_eth_dev_probing_finish(list[i].eth_dev);
}
mlx5_glue->free_device_list(ibv_list);
if (!n) {
@@ -1317,10 +1370,10 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
ret = -rte_errno;
/* Roll back. */
while (i--) {
- mlx5_dev_close(eth_list[i]);
+ mlx5_dev_close(list[i].eth_dev);
if (rte_eal_process_type() == RTE_PROC_PRIMARY)
- rte_free(eth_list[i]->data->dev_private);
- claim_zero(rte_eth_dev_release_port(eth_list[i]));
+ rte_free(list[i].eth_dev->data->dev_private);
+ claim_zero(rte_eth_dev_release_port(list[i].eth_dev));
}
/* Restore original error. */
rte_errno = -ret;
--
2.11.0