From: Sunil Kumar Kori <skori@marvell.com>
To: "Gaëtan Rivet" <grive@u256.net>
Cc: "stephen@networkplumber.org" <stephen@networkplumber.org>,
"david.marchand@redhat.com" <david.marchand@redhat.com>,
"Jerin Jacob Kollanukkaran" <jerinj@marvell.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v4 1/1] bus/pci: optimise scanning with whitelist/blacklist
Date: Sat, 2 May 2020 07:20:46 +0000 [thread overview]
Message-ID: <BY5PR18MB310538B2DA5D7F2968954F47B4A80@BY5PR18MB3105.namprd18.prod.outlook.com> (raw)
In-Reply-To: <20200501210013.hbkcm6pcsyxscyvc@u256.net>
>-----Original Message-----
>From: Gaëtan Rivet <grive@u256.net>
>Sent: Saturday, May 2, 2020 2:30 AM
>To: Sunil Kumar Kori <skori@marvell.com>
>Cc: stephen@networkplumber.org; david.marchand@redhat.com; Jerin Jacob
>Kollanukkaran <jerinj@marvell.com>; dev@dpdk.org
>Subject: [EXT] Re: [PATCH v4 1/1] bus/pci: optimise scanning with
>whitelist/blacklist
>
>External Email
>
>----------------------------------------------------------------------
>Hello Sunil,
>
>It's pretty close, thanks. Unfortunately, I just have a few nits remaining.
>
>On 01/05/20 17:09 +0530, Sunil Kumar Kori wrote:
>> rte_bus_scan API scans all the available PCI devices irrespective of
>> white or black listing parameters then further devices are probed
>> based on white or black listing parameters. So unnecessary CPU cycles
>> are wasted during rte_pci_scan.
>>
>> For Octeontx2 platform with core frequency 2.4 Ghz, rte_bus_scan
>> consumes around 26ms to scan around 90 PCI devices but all may not be
>> used by the application. So for the application which uses 2 NICs,
>> rte_bus_scan consumes few microseconds and rest time is saved with this
>patch.
>>
>> Patch restricts devices to be scanned as per below mentioned conditions:
>> - All devices will be scanned if no parameters are passed.
>> - Only white listed devices will be scanned if white list is available.
>> - All devices, except black listed, will be scanned if black list is
>> available.
>>
>> Signed-off-by: Sunil Kumar Kori <skori@marvell.com>
>> ---
>> v4:
>> - Review comments incorporated (Gaeten and David).
>> - Rebased on top of tree.
>> v3:
>> - Remove __rte_experimental from private function.
>> - Remove entry from map file too.
>> v2:
>> - Added function to validate ignorance of device based on PCI address.
>> - Marked device validation function as experimental.
>>
>> drivers/bus/pci/bsd/pci.c | 12 +++++++++++-
>> drivers/bus/pci/linux/pci.c | 3 +++ drivers/bus/pci/pci_common.c |
>> 33 ++++++++++++---------------------
>> drivers/bus/pci/private.h | 11 +++++++++++
>> 4 files changed, 37 insertions(+), 22 deletions(-)
>>
>> diff --git a/drivers/bus/pci/bsd/pci.c b/drivers/bus/pci/bsd/pci.c
>> index ebbfeb13a..709a1e7e5 100644
>> --- a/drivers/bus/pci/bsd/pci.c
>> +++ b/drivers/bus/pci/bsd/pci.c
>> @@ -338,6 +338,7 @@ rte_pci_scan(void)
>> .match_buf_len = sizeof(matches),
>> .matches = &matches[0],
>> };
>> + struct rte_pci_addr pci_addr;
>>
>> /* for debug purposes, PCI can be disabled */
>> if (!rte_eal_has_pci())
>> @@ -357,9 +358,18 @@ rte_pci_scan(void)
>> goto error;
>> }
>>
>> - for (i = 0; i < conf_io.num_matches; i++)
>> + for (i = 0; i < conf_io.num_matches; i++) {
>> + pci_addr.domain = matches[i].pc_sel.pc_domain;
>> + pci_addr.bus = matches[i].pc_sel.pc_bus;
>> + pci_addr.devid = matches[i].pc_sel.pc_dev;
>> + pci_addr.function = matches[i].pc_sel.pc_func;
>> +
>> + if (rte_pci_ignore_device_addr(&pci_addr))
>> + continue;
>> +
>> if (pci_scan_one(fd, &matches[i]) < 0)
>> goto error;
>> + }
>>
>> dev_count += conf_io.num_matches;
>> } while(conf_io.status == PCI_GETCONF_MORE_DEVS); diff --git
>> a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c index
>> ca783b157..ec347eff3 100644
>> --- a/drivers/bus/pci/linux/pci.c
>> +++ b/drivers/bus/pci/linux/pci.c
>> @@ -487,6 +487,9 @@ rte_pci_scan(void)
>> if (parse_pci_addr_format(e->d_name, sizeof(e->d_name),
>&addr) != 0)
>> continue;
>>
>> + if (rte_pci_ignore_device_addr(&addr))
>> + continue;
>> +
>> snprintf(dirname, sizeof(dirname), "%s/%s",
>> rte_pci_get_sysfs_path(), e->d_name);
>>
>> diff --git a/drivers/bus/pci/pci_common.c
>> b/drivers/bus/pci/pci_common.c index 3f5542076..d34e59536 100644
>> --- a/drivers/bus/pci/pci_common.c
>> +++ b/drivers/bus/pci/pci_common.c
>> @@ -42,14 +42,17 @@ const char *rte_pci_get_sysfs_path(void)
>> return path;
>> }
>>
>> -static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device
>> *dev)
>> +static struct rte_devargs *
>> +pci_devargs_lookup(const struct rte_pci_addr *pci_addr)
>> {
>> struct rte_devargs *devargs;
>> struct rte_pci_addr addr;
>>
>> RTE_EAL_DEVARGS_FOREACH("pci", devargs) {
>> - devargs->bus->parse(devargs->name, &addr);
>> - if (!rte_pci_addr_cmp(&dev->addr, &addr))
>> + if (rte_pci_addr_parse(devargs->name, &addr) < 0)
>> + continue;
>> +
>> + if (!rte_pci_addr_cmp(pci_addr, &addr))
>
>I'm really sorry, I was overzealous on your v4. I thought using
>devargs->bus->parse() was from your patch, but it was already there.
>
>It's even more shameful as it was me who wrote this.
>
>Anyway, it's much better looking this way, but it should be a separate patch.
>
No problem. I will revert this change to its original state and later will share separate
Patch to replace devargs->bus->parse.
>> return devargs;
>> }
>> return NULL;
>> @@ -63,7 +66,7 @@ pci_name_set(struct rte_pci_device *dev)
>> /* Each device has its internal, canonical name set. */
>> rte_pci_device_name(&dev->addr,
>> dev->name, sizeof(dev->name));
>> - devargs = pci_devargs_lookup(dev);
>> + devargs = pci_devargs_lookup(&dev->addr);
>> dev->device.devargs = devargs;
>> /* In blacklist mode, if the device is not blacklisted, no
>> * rte_devargs exists for it.
>> @@ -293,23 +296,12 @@ rte_pci_probe(void) {
>> struct rte_pci_device *dev = NULL;
>> size_t probed = 0, failed = 0;
>> - struct rte_devargs *devargs;
>> - int probe_all = 0;
>> int ret = 0;
>>
>> - if (rte_pci_bus.bus.conf.scan_mode != RTE_BUS_SCAN_WHITELIST)
>> - probe_all = 1;
>> -
>> FOREACH_DEVICE_ON_PCIBUS(dev) {
>> probed++;
>>
>> - devargs = dev->device.devargs;
>> - /* probe all or only whitelisted devices */
>> - if (probe_all)
>> - ret = pci_probe_all_drivers(dev);
>> - else if (devargs != NULL &&
>> - devargs->policy == RTE_DEV_WHITELISTED)
>> - ret = pci_probe_all_drivers(dev);
>> + ret = pci_probe_all_drivers(dev);
>> if (ret < 0) {
>> if (ret != -EEXIST) {
>> RTE_LOG(ERR, EAL, "Requested device "
>> @@ -589,10 +581,10 @@ pci_dma_unmap(struct rte_device *dev, void
>*addr, uint64_t iova, size_t len)
>> return -1;
>> }
>>
>> -static bool
>> -pci_ignore_device(const struct rte_pci_device *dev)
>> +bool
>> +rte_pci_ignore_device_addr(const struct rte_pci_addr *pci_addr)
>
>I'd prefer naming this function rte_pci_ignore_device(), given that
>pci_ignore_device() disappears.
>
Ack.
>> {
>> - struct rte_devargs *devargs = dev->device.devargs;
>> + struct rte_devargs *devargs = pci_devargs_lookup(pci_addr);
>>
>> switch (rte_pci_bus.bus.conf.scan_mode) {
>> case RTE_BUS_SCAN_WHITELIST:
>> @@ -627,8 +619,7 @@ rte_pci_get_iommu_class(void)
>> if (iommu_no_va == -1)
>> iommu_no_va = pci_device_iommu_support_va(dev)
>> ? 0 : 1;
>> - if (pci_ignore_device(dev))
>> - continue;
>> +
>> if (dev->kdrv == RTE_KDRV_UNKNOWN ||
>> dev->kdrv == RTE_KDRV_NONE)
>> continue;
>> diff --git a/drivers/bus/pci/private.h b/drivers/bus/pci/private.h
>> index a205d4d9f..3874219ba 100644
>> --- a/drivers/bus/pci/private.h
>> +++ b/drivers/bus/pci/private.h
>> @@ -42,6 +42,17 @@ int rte_pci_scan(void); void pci_name_set(struct
>> rte_pci_device *dev);
>>
>> +/**
>> + * Validate whether a device with given pci address should be ignored or
>not.
>> + *
>> + * @param pci_addr
>> + * PCI address of device to be validated
>> + * @return
>> + * 1: if device is to be ignored,
>> + * 0: if device is to be scanned,
>> + */
>> +bool rte_pci_ignore_device_addr(const struct rte_pci_addr *pci_addr);
>> +
>> /**
>> * Add a PCI device to the PCI Bus (append to PCI Device list). This function
>> * also updates the bus references of the PCI Device (and the generic
>> device
>> --
>> 2.17.1
>>
>
>Otherwise it looks good to me, almost finished!
>
>--
>Gaëtan
next prev parent reply other threads:[~2020-05-02 7:20 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-16 7:55 [dpdk-dev] [PATCH] bus/pci: restricted bus scanning to allowed devices Sunil Kumar Kori
2019-12-16 16:13 ` Stephen Hemminger
2019-12-17 11:00 ` [dpdk-dev] [EXT] " Sunil Kumar Kori
2020-01-21 8:39 ` Sunil Kumar Kori
[not found] ` <MN2PR18MB327936807460D9F2AE4894F3B40F0@MN2PR18MB3279.namprd18.prod.outlook.com>
2020-02-27 8:30 ` Sunil Kumar Kori
2020-03-09 6:06 ` Sunil Kumar Kori
2020-04-06 9:32 ` Sunil Kumar Kori
2020-04-06 13:21 ` David Marchand
2020-04-07 9:21 ` Sunil Kumar Kori
2020-04-07 9:28 ` [dpdk-dev] [PATCH v2 1/1] bus/pci: optimise scanning with whitelist/blacklist Sunil Kumar Kori
2020-04-17 8:30 ` Sunil Kumar Kori
2020-04-17 8:44 ` David Marchand
2020-04-17 11:15 ` [dpdk-dev] [EXT] " Sunil Kumar Kori
2020-04-17 13:25 ` David Marchand
2020-04-17 15:12 ` Sunil Kumar Kori
2020-04-17 15:35 ` David Marchand
2020-04-17 16:00 ` Sunil Kumar Kori
2020-04-20 6:59 ` Sunil Kumar Kori
2020-04-20 6:55 ` [dpdk-dev] [PATCH v3 " Sunil Kumar Kori
2020-04-21 15:18 ` Gaëtan Rivet
2020-04-22 6:17 ` [dpdk-dev] [EXT] " Sunil Kumar Kori
2020-04-22 9:38 ` Gaëtan Rivet
2020-04-23 7:47 ` Sunil Kumar Kori
2020-04-27 18:43 ` [dpdk-dev] " Gaëtan Rivet
2020-04-28 13:52 ` [dpdk-dev] [EXT] " Sunil Kumar Kori
2020-05-01 11:39 ` [dpdk-dev] [PATCH v4 " Sunil Kumar Kori
2020-05-01 12:40 ` Sunil Kumar Kori
2020-05-01 21:00 ` Gaëtan Rivet
2020-05-02 7:20 ` Sunil Kumar Kori [this message]
2020-05-02 7:42 ` [dpdk-dev] [PATCH v5 " Sunil Kumar Kori
2020-05-02 11:27 ` Gaëtan Rivet
2020-05-04 14:17 ` David Marchand
2020-05-05 5:57 ` [dpdk-dev] [EXT] " Sunil Kumar Kori
2020-05-06 12:54 ` [dpdk-dev] " David Marchand
2020-05-11 14:59 ` David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BY5PR18MB310538B2DA5D7F2968954F47B4A80@BY5PR18MB3105.namprd18.prod.outlook.com \
--to=skori@marvell.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=grive@u256.net \
--cc=jerinj@marvell.com \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).