From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id C7E22952 for ; Wed, 15 Feb 2017 19:09:08 +0100 (CET) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Feb 2017 10:09:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,166,1484035200"; d="scan'208";a="65573810" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga005.fm.intel.com with ESMTP; 15 Feb 2017 10:09:07 -0800 Received: from fmsmsx113.amr.corp.intel.com ([169.254.13.230]) by FMSMSX108.amr.corp.intel.com ([169.254.9.130]) with mapi id 14.03.0248.002; Wed, 15 Feb 2017 10:09:07 -0800 From: "Wiles, Keith" To: Jerin Jacob CC: Shreyansh Jain , Jan Blunck , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH 3/7] eal: move virtual device probing into a bus Thread-Index: AQHSh5fH4K3Z7eBhQkKQQ7+2ormWKKFq1/CAgAAMKwA= Date: Wed, 15 Feb 2017 18:09:06 +0000 Message-ID: <9AE1640E-5F5D-446F-A304-F94D6667FC02@intel.com> References: <1487152929-23627-1-git-send-email-jblunck@infradead.org> <1487152929-23627-4-git-send-email-jblunck@infradead.org> <485202cf-3644-5ee9-45b5-d443569bf5b5@nxp.com> <7843A545-14B6-4CD9-9689-ABD123B85890@intel.com> <20170215172530.GA8119@localhost.localdomain> In-Reply-To: <20170215172530.GA8119@localhost.localdomain> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.254.102.97] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH 3/7] eal: move virtual device probing into a bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Feb 2017 18:09:09 -0000 > On Feb 15, 2017, at 11:25 AM, Jerin Jacob wrote: >=20 > On Wed, Feb 15, 2017 at 02:27:47PM +0000, Shreyansh Jain wrote: >>> -----Original Message----- >>> From: Wiles, Keith [mailto:keith.wiles@intel.com] >>> Sent: Wednesday, February 15, 2017 7:53 PM >>> To: Shreyansh Jain >>> Cc: Jan Blunck ; dev@dpdk.org >>> Subject: Re: [dpdk-dev] [PATCH 3/7] eal: move virtual device probing in= to a >>> bus >>>=20 >>>=20 >>>> On Feb 15, 2017, at 8:15 AM, Shreyansh Jain w= rote: >>>>=20 >>>> On Wednesday 15 February 2017 07:41 PM, Shreyansh Jain wrote: >>>>> On Wednesday 15 February 2017 03:32 PM, Jan Blunck wrote: >>>>>> This is a refactoring of the virtual device probing which moves into= into >>>>>> a proper bus structure. >>>>>>=20 >>>>>> Signed-off-by: Jan Blunck >>>>>> --- >>>>>> lib/librte_eal/common/eal_common_dev.c | 22 ----------------- >>>>>> lib/librte_eal/common/eal_common_vdev.c | 44 >>>>>> +++++++++++++++++++++++++++++++++ >>>>>> 2 files changed, 44 insertions(+), 22 deletions(-) >>>>>>=20 >>>>>=20 >>>>> [...] >>>>>=20 >>>>>>=20 >>>>>> diff --git a/lib/librte_eal/common/eal_common_vdev.c >>>>>> b/lib/librte_eal/common/eal_common_vdev.c >>>>>> index 7d6e54f..523a3d6 100644 >>>>>> --- a/lib/librte_eal/common/eal_common_vdev.c >>>>>> +++ b/lib/librte_eal/common/eal_common_vdev.c >>>>>> @@ -37,8 +37,10 @@ >>>>>> #include >>>>>> #include >>>>>>=20 >>>>> [...] >>>>>=20 >>>>>> + >>>>>> +static struct rte_bus rte_vdev_bus =3D { >>>>>> + .scan =3D vdev_scan, >>>>>> + .probe =3D vdev_probe, >>>>>> +}; >>>>>> + >>>>>> +RTE_REGISTER_BUS_LATE(virtual, rte_vdev_bus); >>>>>>=20 >>>>>=20 >>>>> Does it matter if VDEV buses are registered before or after other >>>>> buses? Either way, the callbacks would be called in the order specifi= ed >>>>> in EAL. >>>>>=20 >>>>>=20 >>>>=20 >>>> Just ignore this comment - I am misunderstood something. >>>>=20 >>>> But another question: Is there specific reason VDEV should be >>> registered/scanned *after* other devices? Is there some specific proble= m if >>> we do otherwise? (I think this is should be done, but I don't have a sp= ecific >>> reason). >>>=20 >>> Does the bonding driver which uses physical devices need to be register= ed >>> after physical ones? In Pktgen I noticed the vdev after the physical po= rts >>> and I could not blacklist them as the bonding driver needed them, which >>> caused the bonding ports to have a greater port number. In the case of = pktgen >>> the bonding ports were up around 8 or 10 and caused the display to not = show >>> the bonding ports. This is really just a usability problem for the deve= loper >>> using Pktgen. I would like to see the vdev devices first, but as long a= s the >>> drivers (like bonding) are fine with them being first. >>=20 >> Ah, now I remember - there was a patch from Jerin for this. >> Probably he is the best person to comment here. >> (I don't have much insight here). >=20 > commit f4ce209a8ce5f416b61c76cee773bc54749e2048 > Author: Jerin Jacob > Date: Sun Nov 20 13:30:50 2016 +0530 >=20 > eal: postpone vdev initialization >=20 > Some platform like octeontx may use pci and > vdev based combined device to represent a logical > dpdk functional device.In such case, postponing the > vdev initialization after pci device > initialization will provide the better view of > the pci device resources in the system in > vdev's probe function, and it allows better > functional subsystem registration in vdev probe > function. >=20 > As a bonus, This patch fixes a bond device > initialization use case. >=20 > example command to reproduce the issue: > ./testpmd -c 0x2 --vdev 'eth_bond0,mode=3D0, > slave=3D0000:02:00.0,slave=3D0000:03:00.0' -- > --port-topology=3Dchained >=20 > root cause: > In existing case(vdev initialization and then pci > initialization), creates three Ethernet ports with > following port ids > 0 - Bond device > 1 - PCI device 0 > 2 - PCI devive 1 >=20 > Since testpmd, calls the configure/start on all the ports on > start up,it will translate to following illegal setup sequence I guess I see this differently, meaning we modified the system to put vdev = devices last only because we do not have clean way to startup the system fo= r pdev/vdev devices. The application should be agnostic to the devices bein= g started and the system needs to determine the correct order without a chi= cken and egg problem. The test-pmd application just starts from 0 to n to i= nitialize devices, which he should be able to do in any order. It is possib= le the application could initialize the devices (pdev/vdev) in any order, w= hich the current design would break if they tried to init the bonding drive= r first. What happens if a vdev needs to be initialized before a pdev device? Not saying we need to solve this problem now, but need to figure this out s= ome how. Maybe we need a priority for pdev/vdev devices to determine init o= rder???? >=20 > 1)bond device configure/start > 1.1) pci device0 stop/configure/start > 1.2) pci device1 stop/configure/start > 2)pci device 0 configure(illegal setup case, > as device in start state) >=20 > The fix changes the initialization sequence and > allow initialization in following valid setup order > 1) pcie device 0 configure/start > 2) pcie device 1 configure/start > 3) bond device 2 configure/start > 3.1) pcie device 0/stop/configure/start > 3.2) pcie device 1/stop/configure/start Regards, Keith