From: "Wiles, Keith" <keith.wiles@intel.com>
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Cc: Shreyansh Jain <shreyansh.jain@nxp.com>,
Jan Blunck <jblunck@infradead.org>, "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH 3/7] eal: move virtual device probing into a bus
Date: Wed, 15 Feb 2017 18:09:06 +0000 [thread overview]
Message-ID: <9AE1640E-5F5D-446F-A304-F94D6667FC02@intel.com> (raw)
In-Reply-To: <20170215172530.GA8119@localhost.localdomain>
> On Feb 15, 2017, at 11:25 AM, Jerin Jacob <jerin.jacob@caviumnetworks.com> wrote:
>
> On Wed, Feb 15, 2017 at 02:27:47PM +0000, Shreyansh Jain wrote:
>>> -----Original Message-----
>>> From: Wiles, Keith [mailto:keith.wiles@intel.com]
>>> Sent: Wednesday, February 15, 2017 7:53 PM
>>> To: Shreyansh Jain <shreyansh.jain@nxp.com>
>>> Cc: Jan Blunck <jblunck@infradead.org>; dev@dpdk.org
>>> Subject: Re: [dpdk-dev] [PATCH 3/7] eal: move virtual device probing into a
>>> bus
>>>
>>>
>>>> On Feb 15, 2017, at 8:15 AM, Shreyansh Jain <shreyansh.jain@nxp.com> wrote:
>>>>
>>>> On Wednesday 15 February 2017 07:41 PM, Shreyansh Jain wrote:
>>>>> On Wednesday 15 February 2017 03:32 PM, Jan Blunck wrote:
>>>>>> This is a refactoring of the virtual device probing which moves into into
>>>>>> a proper bus structure.
>>>>>>
>>>>>> Signed-off-by: Jan Blunck <jblunck@infradead.org>
>>>>>> ---
>>>>>> lib/librte_eal/common/eal_common_dev.c | 22 -----------------
>>>>>> lib/librte_eal/common/eal_common_vdev.c | 44
>>>>>> +++++++++++++++++++++++++++++++++
>>>>>> 2 files changed, 44 insertions(+), 22 deletions(-)
>>>>>>
>>>>>
>>>>> [...]
>>>>>
>>>>>>
>>>>>> diff --git a/lib/librte_eal/common/eal_common_vdev.c
>>>>>> b/lib/librte_eal/common/eal_common_vdev.c
>>>>>> index 7d6e54f..523a3d6 100644
>>>>>> --- a/lib/librte_eal/common/eal_common_vdev.c
>>>>>> +++ b/lib/librte_eal/common/eal_common_vdev.c
>>>>>> @@ -37,8 +37,10 @@
>>>>>> #include <stdint.h>
>>>>>> #include <sys/queue.h>
>>>>>>
>>>>> [...]
>>>>>
>>>>>> +
>>>>>> +static struct rte_bus rte_vdev_bus = {
>>>>>> + .scan = vdev_scan,
>>>>>> + .probe = vdev_probe,
>>>>>> +};
>>>>>> +
>>>>>> +RTE_REGISTER_BUS_LATE(virtual, rte_vdev_bus);
>>>>>>
>>>>>
>>>>> Does it matter if VDEV buses are registered before or after other
>>>>> buses? Either way, the callbacks would be called in the order specified
>>>>> in EAL.
>>>>>
>>>>>
>>>>
>>>> Just ignore this comment - I am misunderstood something.
>>>>
>>>> But another question: Is there specific reason VDEV should be
>>> registered/scanned *after* other devices? Is there some specific problem if
>>> we do otherwise? (I think this is should be done, but I don't have a specific
>>> reason).
>>>
>>> Does the bonding driver which uses physical devices need to be registered
>>> after physical ones? In Pktgen I noticed the vdev after the physical ports
>>> and I could not blacklist them as the bonding driver needed them, which
>>> caused the bonding ports to have a greater port number. In the case of pktgen
>>> the bonding ports were up around 8 or 10 and caused the display to not show
>>> the bonding ports. This is really just a usability problem for the developer
>>> using Pktgen. I would like to see the vdev devices first, but as long as the
>>> drivers (like bonding) are fine with them being first.
>>
>> Ah, now I remember - there was a patch from Jerin for this.
>> Probably he is the best person to comment here.
>> (I don't have much insight here).
>
> commit f4ce209a8ce5f416b61c76cee773bc54749e2048
> Author: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Date: Sun Nov 20 13:30:50 2016 +0530
>
> eal: postpone vdev initialization
>
> Some platform like octeontx may use pci and
> vdev based combined device to represent a logical
> dpdk functional device.In such case, postponing the
> vdev initialization after pci device
> initialization will provide the better view of
> the pci device resources in the system in
> vdev's probe function, and it allows better
> functional subsystem registration in vdev probe
> function.
>
> As a bonus, This patch fixes a bond device
> initialization use case.
>
> example command to reproduce the issue:
> ./testpmd -c 0x2 --vdev 'eth_bond0,mode=0,
> slave=0000:02:00.0,slave=0000:03:00.0' --
> --port-topology=chained
>
> root cause:
> In existing case(vdev initialization and then pci
> initialization), creates three Ethernet ports with
> following port ids
> 0 - Bond device
> 1 - PCI device 0
> 2 - PCI devive 1
>
> Since testpmd, calls the configure/start on all the ports on
> start up,it will translate to following illegal setup sequence
I guess I see this differently, meaning we modified the system to put vdev devices last only because we do not have clean way to startup the system for pdev/vdev devices. The application should be agnostic to the devices being started and the system needs to determine the correct order without a chicken and egg problem. The test-pmd application just starts from 0 to n to initialize devices, which he should be able to do in any order. It is possible the application could initialize the devices (pdev/vdev) in any order, which the current design would break if they tried to init the bonding driver first.
What happens if a vdev needs to be initialized before a pdev device?
Not saying we need to solve this problem now, but need to figure this out some how. Maybe we need a priority for pdev/vdev devices to determine init order????
>
> 1)bond device configure/start
> 1.1) pci device0 stop/configure/start
> 1.2) pci device1 stop/configure/start
> 2)pci device 0 configure(illegal setup case,
> as device in start state)
>
> The fix changes the initialization sequence and
> allow initialization in following valid setup order
> 1) pcie device 0 configure/start
> 2) pcie device 1 configure/start
> 3) bond device 2 configure/start
> 3.1) pcie device 0/stop/configure/start
> 3.2) pcie device 1/stop/configure/start
Regards,
Keith
next prev parent reply other threads:[~2017-02-15 18:09 UTC|newest]
Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-02-15 10:02 [dpdk-dev] [PATCH 0/7] Rework vdev probing to use rte_bus infrastructure Jan Blunck
2017-02-15 10:02 ` [dpdk-dev] [PATCH 1/7] eal: use different constructor priorities for initcalls Jan Blunck
2017-02-15 14:37 ` Shreyansh Jain
2017-02-15 15:05 ` Jan Blunck
2017-02-16 5:59 ` Shreyansh Jain
2017-02-15 10:02 ` [dpdk-dev] [PATCH 2/7] eal: probe legacy PCI devices before other bus devices Jan Blunck
2017-02-15 14:03 ` Shreyansh Jain
2017-02-15 10:02 ` [dpdk-dev] [PATCH 3/7] eal: move virtual device probing into a bus Jan Blunck
2017-02-15 14:11 ` Shreyansh Jain
2017-02-15 14:13 ` Jan Blunck
2017-02-15 14:20 ` Shreyansh Jain
2017-02-15 14:15 ` Shreyansh Jain
2017-02-15 14:22 ` Wiles, Keith
2017-02-15 14:27 ` Shreyansh Jain
2017-02-15 17:25 ` Jerin Jacob
2017-02-15 18:09 ` Wiles, Keith [this message]
2017-02-15 20:06 ` Jan Blunck
2017-02-15 21:56 ` Wiles, Keith
2017-02-15 17:06 ` Jan Blunck
2017-02-15 17:10 ` Wiles, Keith
2017-02-15 17:22 ` Wiles, Keith
2017-02-15 10:02 ` [dpdk-dev] [PATCH 4/7] eal: remove unused rte_eal_dev_init() Jan Blunck
2017-02-15 17:11 ` Ferruh Yigit
2017-02-15 10:02 ` [dpdk-dev] [PATCH 5/7] eal: Refactor vdev driver probe/remove Jan Blunck
2017-02-15 10:02 ` [dpdk-dev] [PATCH 6/7] eal: add struct rte_vdev_device Jan Blunck
2017-02-15 17:11 ` Ferruh Yigit
2017-02-16 15:55 ` Jan Blunck
2017-02-15 10:02 ` [dpdk-dev] [PATCH 7/7] eal: make virtual bus use rte_vdev_device Jan Blunck
2017-02-15 17:11 ` Ferruh Yigit
2017-02-15 17:11 ` [dpdk-dev] [PATCH 0/7] Rework vdev probing to use rte_bus infrastructure Ferruh Yigit
2017-02-20 14:17 ` [dpdk-dev] [PATCH v2 0/8] " Jan Blunck
2017-02-21 6:44 ` Shreyansh Jain
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 00/10] " Jan Blunck
2017-02-27 13:09 ` Jan Blunck
2017-02-28 8:48 ` Shreyansh Jain
2017-02-28 9:19 ` Jan Blunck
2017-02-28 9:28 ` Shreyansh Jain
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 " Jan Blunck
2017-03-13 17:55 ` Thomas Monjalon
2017-03-27 7:47 ` Jan Blunck
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 00/12] " Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 01/12] eal: probe new virtual bus after other bus devices Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 02/12] eal: move virtual device probing into a bus Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 03/12] eal: remove unused rte_eal_dev_init() Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 04/12] eal: Refactor vdev driver probe/remove Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 05/12] eal: add struct rte_vdev_device Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 06/12] eal: add virtual device name helper function Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 07/12] eal: add virtual device arguments " Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 08/12] eal: make virtual bus use rte_vdev_device Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 09/12] eal: make virtual driver probe and remove take rte_vdev_device Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 10/12] net/kni: use generic vdev for probe and remove Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 11/12] crypto: " Gaetan Rivet
2017-04-11 15:44 ` [dpdk-dev] [PATCH v5 12/12] event: " Gaetan Rivet
2017-04-14 12:21 ` [dpdk-dev] [PATCH v5 00/12] Rework vdev probing to use rte_bus infrastructure Thomas Monjalon
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 01/10] eal: probe legacy PCI devices before other bus devices Jan Blunck
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 02/10] eal: probe new virtual bus after " Jan Blunck
2017-03-13 17:42 ` Thomas Monjalon
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 03/10] eal: move virtual device probing into a bus Jan Blunck
2017-03-13 17:44 ` Thomas Monjalon
2017-03-27 7:46 ` Jan Blunck
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 04/10] eal: remove unused rte_eal_dev_init() Jan Blunck
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 05/10] eal: Refactor vdev driver probe/remove Jan Blunck
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 06/10] eal: add struct rte_vdev_device Jan Blunck
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 07/10] eal: add virtual device name helper function Jan Blunck
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 08/10] eal: add virtual device arguments " Jan Blunck
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 09/10] eal: make virtual bus use rte_vdev_device Jan Blunck
2017-03-13 17:51 ` Thomas Monjalon
2017-03-27 7:43 ` Jan Blunck
2017-03-06 10:56 ` [dpdk-dev] [PATCH v4 10/10] eal: make virtual driver probe and remove take rte_vdev_device Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 01/10] eal: probe legacy PCI devices before other bus devices Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 02/10] eal: probe new virtual bus after " Jan Blunck
2017-02-27 8:59 ` Shreyansh Jain
2017-02-27 9:09 ` Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 03/10] eal: move virtual device probing into a bus Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 04/10] eal: remove unused rte_eal_dev_init() Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 05/10] eal: Refactor vdev driver probe/remove Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 06/10] eal: add struct rte_vdev_device Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 07/10] eal: add virtual device name helper function Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 08/10] eal: add virtual device arguments " Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 09/10] eal: make virtual bus use rte_vdev_device Jan Blunck
2017-02-25 10:28 ` [dpdk-dev] [PATCH v3 10/10] eal: make virtual driver probe and remove take rte_vdev_device Jan Blunck
2017-02-20 14:17 ` [dpdk-dev] [PATCH v2 1/8] eal: use different constructor priorities for initcalls Jan Blunck
2017-02-21 12:30 ` Ferruh Yigit
2017-02-20 14:17 ` [dpdk-dev] [PATCH v2 2/8] eal: probe legacy PCI devices before other bus devices Jan Blunck
2017-02-20 14:17 ` [dpdk-dev] [PATCH v2 3/8] eal: move virtual device probing into a bus Jan Blunck
2017-02-20 14:17 ` [dpdk-dev] [PATCH v2 4/8] eal: remove unused rte_eal_dev_init() Jan Blunck
2017-02-20 14:17 ` [dpdk-dev] [PATCH v2 5/8] eal: Refactor vdev driver probe/remove Jan Blunck
2017-02-20 14:17 ` [dpdk-dev] [PATCH v2 6/8] eal: add struct rte_vdev_device Jan Blunck
2017-02-20 14:17 ` [dpdk-dev] [PATCH v2 7/8] eal: add virtual device name helper function Jan Blunck
2017-02-21 12:25 ` Ferruh Yigit
2017-02-20 14:17 ` [dpdk-dev] [PATCH v2 8/8] eal: make virtual bus use rte_vdev_device Jan Blunck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9AE1640E-5F5D-446F-A304-F94D6667FC02@intel.com \
--to=keith.wiles@intel.com \
--cc=dev@dpdk.org \
--cc=jblunck@infradead.org \
--cc=jerin.jacob@caviumnetworks.com \
--cc=shreyansh.jain@nxp.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).