From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id 8215D44BE for ; Fri, 25 May 2018 18:35:38 +0200 (CEST) Received: by mail-wm0-f67.google.com with SMTP id t11-v6so16007409wmt.0 for ; Fri, 25 May 2018 09:35:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=6cB5/idg/MUzzreE8pwzYRHfg4vGHyGAxOFetQrf4SY=; b=rTi9YXzQtJeqXWiOFENPvvcyp6EKYeddzN+wCNhFigixHNTn4ZDfP/NS2lnOlRHPAn W6q3Kjpm9Dl02UZfLVlBQgN2cEjtiOG4qFS8Hvz0OFdcY4tO2B/Q+JkroFLNSGbYJ+LL 4Ob/0Pl9hxiv38c+coLELNju22csVbCH6PfrJNKRHkv73+4MV291UsrVkkwdL5s7v91b TlvbUb3HRwzQS06u32yAfh4/hMJfG2nnjZNtFnAvjFYKZ7PaQ+40ucnCLscwhqmEAX7G +3cUhiAqEg6SFHGF1HYPJRFU6smSfQ7U35o1J4GzWVHQHSXq6AJQE3AEYPKWGz9VpWWG ADGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=6cB5/idg/MUzzreE8pwzYRHfg4vGHyGAxOFetQrf4SY=; b=BCWLACQrV1ivbvkMFSTurzJ0PNXtB5HVi3/iMPWHl3YgV8LSh6hwSgmQlOgWXuh0g4 20M0cYgjSjKjXeA6BZJTr8XgTD3r5AxCe1ohmeW5zhnaSNSVk8A+7gp6uT0pa+yHOBET KAkzkTYlZYbXiiJb1U3m55cjnQ4cRVN+vXVQNAOAIMT7fUhN6mUSk+NCt88vs0Zb4g5B PNr9ADwasvgr+CJoOSXXMZFYBgso3j2BX89vvB0+aF8gjOapv4EuvShiuHrOjrOih4SV MWTJN7IVZ7hU7w2ePgoKWISkLmwY+wAy3V0UTZBd5m+ElDUuQ3I3WKpmI4preH8UnlOk chaw== X-Gm-Message-State: ALKqPwdR69mfuyVCuF4Of9aNB0VTOrEnOFmGu+aXPA0mw7h2CInuFe9v hHWGyubWHAY5T8iGmXCrNHxDPg== X-Google-Smtp-Source: ADUXVKLn8c2P+AdsuwucikFhdAOd26TXU+jOknBxemVXgkmuRqvE0ZiHhotVL1v+gMEEO0D0TZ08Lw== X-Received: by 2002:a1c:b45:: with SMTP id 66-v6mr2091888wml.155.1527266138253; Fri, 25 May 2018 09:35:38 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id 19-v6sm30510477wrz.7.2018.05.25.09.35.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 25 May 2018 09:35:37 -0700 (PDT) Date: Fri, 25 May 2018 18:35:23 +0200 From: Adrien Mazarguil To: Shahaf Shuler Cc: dev@dpdk.org Message-ID: <20180525161814.13873-6-adrien.mazarguil@6wind.com> References: <20180525161814.13873-1-adrien.mazarguil@6wind.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180525161814.13873-1-adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.11.0 Subject: [dpdk-dev] [PATCH 5/7] net/mlx5: add port representor awareness X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2018 16:35:39 -0000 The current PCI probing method is not aware of Verbs port representors, which appear as standard Verbs devices bound to the same PCI address and cannot be distinguished. Problem is that more often than not, the wrong Verbs device is used, resulting in unexpected traffic. This patch adds necessary heuristics to bind affected driver instances to the intended (i.e. non-representor) device. (Patch based on prior work from Yuanhan Liu) Signed-off-by: Adrien Mazarguil --- drivers/net/mlx5/mlx5.c | 61 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 57 insertions(+), 4 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index d57e8118c..d3a298332 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1155,6 +1155,32 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, } /** + * Comparison callback to sort Verbs device names. + * + * This is meant to be used with qsort(). + * + * @param a[in] + * Pointer to pointer to first Verbs device. + * @param b[in] + * Pointer to pointer to second Verbs device. + * + * @return + * 0 if both names are equal, less than 0 if the first argument is less + * than the second, greater than 0 otherwise. + */ +static int +mlx5_cmp_ibv_name(const void *a, const void *b) +{ + const char *name_a = (*(const struct ibv_device *const *)a)->name; + const char *name_b = (*(const struct ibv_device *const *)b)->name; + size_t i = 0; + + while (name_a[i] && name_a[i] == name_b[i]) + ++i; + return atoi(name_a + i) - atoi(name_b + i); +} + +/** * DPDK callback to register a PCI device. * * This function creates an Ethernet device for each port of a given @@ -1174,6 +1200,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, { struct ibv_device **ibv_list; struct rte_eth_dev **eth_list = NULL; + int n = 0; int vf; int ret; @@ -1195,6 +1222,9 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, DRV_LOG(ERR, "cannot list devices, is ib_uverbs loaded?"); return -rte_errno; } + + struct ibv_device *ibv_match[ret + 1]; + while (ret-- > 0) { struct rte_pci_addr pci_addr; @@ -1206,12 +1236,35 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, pci_dev->addr.devid != pci_addr.devid || pci_dev->addr.function != pci_addr.function) continue; - DRV_LOG(INFO, "PCI information matches, using device \"%s\"", + DRV_LOG(INFO, "PCI information matches for device \"%s\"", ibv_list[ret]->name); - break; + ibv_match[n++] = ibv_list[ret]; + } + ibv_match[n] = NULL; + if (n > 1) { + /* + * The existence of several matching entries means port + * representors have been instantiated. No existing Verbs + * call nor /sys entries can tell them apart at this point. + * + * While definitely hackish, assume their names are numbered + * based on order of creation with master device first, + * followed by first port representor, followed by the + * second one and so on. + */ + DRV_LOG(WARNING, + "probing device with port representors involves" + " heuristics with uncertain outcome"); + qsort(ibv_match, n, sizeof(*ibv_match), mlx5_cmp_ibv_name); + DRV_LOG(WARNING, "assuming \"%s\" is the master device", + ibv_match[0]->name); + for (ret = 1; ret < n; ++ret) + DRV_LOG(WARNING, + "assuming \"%s\" is port representor #%d", + ibv_match[ret]->name, ret - 1); } - if (ret >= 0) - eth_list = mlx5_dev_spawn(&pci_dev->device, ibv_list[ret], vf); + if (n) + eth_list = mlx5_dev_spawn(&pci_dev->device, ibv_match[0], vf); mlx5_glue->free_device_list(ibv_list); if (!eth_list || !*eth_list) { DRV_LOG(WARNING, -- 2.11.0