Re: [dpdk-dev] [EXT] Re: [PATCH v3 1/1] bus/pci: optimise scanning with whitelist/blacklist

[Sunil Kumar Kori <skori@marvell.com>]

>-----Original Message-----
>From: Gaëtan Rivet <grive@u256.net>
>Sent: Wednesday, April 22, 2020 3:09 PM
>To: Sunil Kumar Kori <skori@marvell.com>
>Cc: stephen@networkplumber.org; david.marchand@redhat.com; Jerin Jacob
>Kollanukkaran <jerinj@marvell.com>; dev@dpdk.org
>Subject: Re: [EXT] Re: [dpdk-dev] [PATCH v3 1/1] bus/pci: optimise scanning
>with whitelist/blacklist
>
>On 22/04/20 06:17 +0000, Sunil Kumar Kori wrote:
>> >-----Original Message-----
>> >From: Gaëtan Rivet <grive@u256.net>
>> >Sent: Tuesday, April 21, 2020 8:48 PM
>> >To: Sunil Kumar Kori <skori@marvell.com>
>> >Cc: stephen@networkplumber.org; david.marchand@redhat.com; Jerin
>> >Jacob Kollanukkaran <jerinj@marvell.com>; dev@dpdk.org
>> >Subject: [EXT] Re: [dpdk-dev] [PATCH v3 1/1] bus/pci: optimise
>> >scanning with whitelist/blacklist
>> >
>> >External Email
>> >
>> >---------------------------------------------------------------------
>> >- On 20/04/20 12:25 +0530, Sunil Kumar Kori wrote:
>> >> rte_bus_scan API scans all the available PCI devices irrespective
>> >> of white or black listing parameters then further devices are
>> >> probed based on white or black listing parameters. So unnecessary
>> >> CPU cycles are wasted during rte_pci_scan.
>> >>
>> >> For Octeontx2 platform with core frequency 2.4 Ghz, rte_bus_scan
>> >> consumes around 26ms to scan around 90 PCI devices but all may not
>> >> be used by the application. So for the application which uses 2
>> >> NICs, rte_bus_scan consumes few microseconds and rest time is saved
>> >> with this
>> >patch.
>> >>
>> >
>> >Hi Sunil,
>> >
>> >The PCI bus was written at first with the understanding that all PCI
>> >devices were scanned and made available on the bus -- the probe will filter
>afterward.
>> >
>> >Device hotplug and iteration were written with this in mind. Changing
>> >this principle might have unintended consequences in other EAL parts.
>> >I'm not fundamentally against it, but it is not how buses are
>> >currently designed in DPDK.
>> >
>> I am also not sure about this. I would request you provide suggestion
>> to ensure that there won't be any negative consequences if any.  So that I
>can handle those too.
>>
>
>I would like also to hear from other stakeholders for the PCI bus.
>
>Generally, as long as the blacklist mode is the default, behavior should not
>change, but devil is in the details.
>
>I would have some comments on the patch itself if everyone agrees with this
>direction.
>
>If the principle of the patch is accepted, it would be great for you to test
>hotplug and device listing with testpmd:
>
>   hotplug:
>    * You can spawn VMs with virtual e1000 ports on PCI using QEMU for this,
>      and within testpmd `port attach <pci-id>` -- of course, the
>      port(s) should not be attached when starting testpmd. You might
>      have to either blacklist them, or you could hotplug them in QEMU using the
>      monitor. I don't recall the QEMU commands to do that, sorry.
>
>   device list:
>    * `show device info all` in testpmd. I thought I had added a command to
>      test the device iterator, taking an arbitrary device string, but
>      the patch has been dropped it seems.
>
>If you have no segfault and no surprise, it is a good start.
>
Okay but before verification I would appreciate to have more comments and
closure on principle from other PCI stakeholders. If there is no objection on
principle then I will invest energy in testing.
I will wait for inputs by next week and if there are no inputs then
 assuming it fundamentally correct, I will start verification of above test cases. 

Also If anyone has already validated above mentioned test cases then
Suggestions, about the impact of this patch on PCI bus design, will be very
helpful to understand the real issues with this. 

>> >To me, a one-time 26ms gain is not enough justification to change
>> >this principle. How problematic is this for you? Do you encounter
>> >specific issues due to this delay?
>> >
>> >Thanks,
>>
>> Recently we observed this requirement to cater a use of having lowest
>bootup time for DPDK application.
>> One of the use-case for this to reduce the downtime as part of DPDK SW
>> upgrade in the field. i.e after the SW update, time to close the application
>and restart it again for packet processing.
>> Having this solution application will be up soon and lesser traffic impact will
>be there in a deployed system.
>
>DPDK startup was not written with low latency in mind. You will find here and
>there minute improvements, but I think it is a pipedream to reduce service
>disruption on binary upgrade.
>
>People in the field would be better served with HA, not relying on a critical
>apps restarting as fast as possible.
>
Recently we had a requirement to have bootup time <= 500ms and find
it as one of the candidate to be improved. So thought of to upstream it. 
Also having mechanism to improve bootup time is  good. I think, there is
no harm in this.

>Cheers,
>--
>Gaëtan