From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 5CB5D325F for ; Thu, 15 Mar 2018 02:17:49 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Mar 2018 18:17:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,308,1517904000"; d="scan'208";a="208354958" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga005.jf.intel.com with ESMTP; 14 Mar 2018 18:17:47 -0700 Received: from fmsmsx158.amr.corp.intel.com (10.18.116.75) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 14 Mar 2018 18:17:33 -0700 Received: from shsmsx102.ccr.corp.intel.com (10.239.4.154) by fmsmsx158.amr.corp.intel.com (10.18.116.75) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 14 Mar 2018 18:17:33 -0700 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.226]) by shsmsx102.ccr.corp.intel.com ([169.254.2.80]) with mapi id 14.03.0319.002; Thu, 15 Mar 2018 09:17:28 +0800 From: "Xu, Rosen" To: "gaetan.rivet@6wind.com" , "Richardson, Bruce" CC: Shreyansh Jain , "dev@dpdk.org" , "Doherty, Declan" , "Zhang, Tianfei" Thread-Topic: [dpdk-dev] [RFC 3/4] lib/librte_eal/common: Add Intel FPGA Bus Second Scan, it should be scanned after PCI Bus Thread-Index: AQHTtRNvFIaxRKZ880CIUiXuZapUgKPC8v8ggAAoJJiADW/VsA== Date: Thu, 15 Mar 2018 01:17:28 +0000 Message-ID: <0E78D399C70DA940A335608C6ED296D739F1E61B@SHSMSX104.ccr.corp.intel.com> References: <1520300638-134954-1-git-send-email-rosen.xu@intel.com> <1520300638-134954-4-git-send-email-rosen.xu@intel.com> <0E78D399C70DA940A335608C6ED296D739F0ED23@SHSMSX104.ccr.corp.intel.com> <20180306104622.3ngsn6syickaphbm@bidouze.vm.6wind.com> <20180306113616.GA7644@bricha3-MOBL3.ger.corp.intel.com> <20180306115937.7zxzhzihbgcnhgaz@bidouze.vm.6wind.com> In-Reply-To: <20180306115937.7zxzhzihbgcnhgaz@bidouze.vm.6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYzNmYjA2MmQtOGM2My00NjZlLWE0NGItZWNmMjQ2ZDM2NTFkIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IlRLYVRURFpUMFM3bWd0bVpoOVA2NmRMQUhaMDh2YnhQYVAwTkswREFSRlk9In0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [RFC 3/4] lib/librte_eal/common: Add Intel FPGA Bus Second Scan, it should be scanned after PCI Bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Mar 2018 01:17:51 -0000 -----Original Message----- From: Ga=EBtan Rivet [mailto:gaetan.rivet@6wind.com]=20 Sent: Tuesday, March 06, 2018 20:00 To: Richardson, Bruce Cc: Xu, Rosen ; Shreyansh Jain = ; dev@dpdk.org; Doherty, Declan ; Zhang, Tianfei = Subject: Re: [dpdk-dev] [RFC 3/4] lib/librte_eal/common: Add Intel FPGA Bus= Second Scan, it should be scanned after PCI Bus On Tue, Mar 06, 2018 at 11:36:17AM +0000, Bruce Richardson wrote: > On Tue, Mar 06, 2018 at 11:46:22AM +0100, Ga=EBtan Rivet wrote: > > On Tue, Mar 06, 2018 at 10:42:14AM +0000, Xu, Rosen wrote: > > >=20 > > >=20 > > > -----Original Message----- > > > From: Shreyansh Jain [mailto:shreyansh.jain@nxp.com] > > > Sent: Tuesday, March 06, 2018 14:20 > > > To: Xu, Rosen > > > Cc: dev@dpdk.org; Doherty, Declan ;=20 > > > Zhang, Tianfei > > > Subject: Re: [dpdk-dev] [RFC 3/4] lib/librte_eal/common: Add Intel=20 > > > FPGA Bus Second Scan, it should be scanned after PCI Bus > > >=20 > > > On Tue, Mar 6, 2018 at 7:13 AM, Rosen Xu wrote: > > > > Signed-off-by: Rosen Xu > > > > --- > > > > lib/librte_eal/common/eal_common_bus.c | 14 +++++++++++++- > > > > 1 file changed, 13 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/lib/librte_eal/common/eal_common_bus.c > > > > b/lib/librte_eal/common/eal_common_bus.c > > > > index 3e022d5..74bfa15 100644 > > > > --- a/lib/librte_eal/common/eal_common_bus.c > > > > +++ b/lib/librte_eal/common/eal_common_bus.c > > > > @@ -70,15 +70,27 @@ struct rte_bus_list rte_bus_list =3D > > > > rte_bus_scan(void) > > > > { > > > > int ret; > > > > - struct rte_bus *bus =3D NULL; > > > > + struct rte_bus *bus =3D NULL, *ifpga_bus =3D NULL; > > > > > > > > TAILQ_FOREACH(bus, &rte_bus_list, next) { > > > > + if (!strcmp(bus->name, "ifpga")) { > > > > + ifpga_bus =3D bus; > > > > + continue; > > > > + } > > > > + > > > > ret =3D bus->scan(); > > > > if (ret) > > > > RTE_LOG(ERR, EAL, "Scan for (%s) bus failed= .\n", > > > > bus->name); > > > > } > > > > > > > > + if (ifpga_bus) { > > > > + ret =3D ifpga_bus->scan(); > > > > + if (ret) > > > > + RTE_LOG(ERR, EAL, "Scan for (%s) bus failed= .\n", > > > > + ifpga_bus->name); > > > > + } > > > > + > > >=20 > > > You are doing this just so that PCI scans are completed *before* ifpg= a scans? > > > Rosen: yes > > > Well, I understand that this certainly is an issue that we can't yet = define a priority ordering of bus scans. > > >=20 > > > But, I think what you are require is a simpler: > > >=20 > > > In the file ifpga_bus.c: > > >=20 > > > +RTE_REGISTER_BUS(IFPGA_BUS_NAME, rte_ifpga_bus.bus); <=3D=3D this > > > ... > > > ... > > > #define RTE_REGISTER_BUS(nm, bus) \ RTE_INIT_PRIO(businitfn_ ##nm,=20 > > > 110); \ > > >=20 > > > If you define your own version of RTE_REGISTER_BUS with the priority = number higher, it would be inserted later in the bus list. > > > rte_register_bus doesn't do any inherent ordering. > > > This would save the changes you are doing in the lib/librte_eal/commo= n/eal_common_bus.c file. > > >=20 > > > But I think there has to be a better provision of defining priority o= f bus scans - I am sure when new devices come in, there would be possibilit= y of dependencies as in your case. > > > Rosen: is the priority scan of bus is implemented? > >=20 > > No, there is no priority set for scanning order. > > However, the order in which buses are registered, will modify the=20 > > order in which scans are done. > >=20 > > Thus, if you change the priority of your registration, you should be=20 > > able to ensure that your scan comes last. > >=20 >=20 > Can we register the bus only when a PCI device match is found at=20 > runtime, e.g. as part of the PCI driver instance initialization? >=20 > /Bruce Technically, yes. You would append a new bus during rte_bus_probe, so the l= inked list would simply have a new node and you would then probe it. You wo= uld need to make sure you scan your bus first, so you would have some weird= conditions (whether you are loaded during probe or naturally, you'd have t= o do your scan or not). However, this seems like a terrible idea. You introduce an edge case that w= ill need to be carried over in most of the bus API implementation. This new bus seems like a specialization of the PCI bus. Why not directly u= se the PCI bus and have your driver linked to either a rawdev or a vdev, wh= ere you could store your metadata and expose a specialized interface? Rosen: pls see my v1 patch, in that patch we don't need to modify rte_bus_s= can(), the IFPGA Bus Scen is probed by FPGA PCI Driver. The reason wo don't directly use PCI bus is that: 1. One FPGA PCI Device has more than one AFU bitstream; 2. Each AFU is a hardware device viewed by DPDK; 3. Acceleration Driver(like Eth/Crpt) bind to AFU dirver; 4. We also need to hotplug AFU bitstream in runtime; -- Ga=EBtan Rivet 6WIND