From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f44.google.com (mail-wm0-f44.google.com [74.125.82.44]) by dpdk.org (Postfix) with ESMTP id 3CDCD2C58 for ; Wed, 21 Mar 2018 17:21:45 +0100 (CET) Received: by mail-wm0-f44.google.com with SMTP id h21so10758689wmd.1 for ; Wed, 21 Mar 2018 09:21:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=YI1zEv5GFtwERlrQpMIOUH1j5UheUEjYSFKQctQTQZY=; b=jilPUyG9iUnmRuceRRLqPT+dA9ydl6Zu3M3gdcwxFwPt6mvHzmWWbbH8BkEM1N0fd1 fkB0FM0eCz58f+RHV7i7Jc+CR48c7WN2F4Q0lYoS86ZtrD4UMmfN7G1h25X5lxvl+K0X MHD5eai3q0tpVg0TxFW+0NGuPUq7BRz+ATN7/C5gckYXsb1ZNRZSQdZ7B0wEBZFEQhk/ V3L7i9ZbDP6Ed2Le0+U+0wEYzzu4BFrAEJchu4nLfkN/orvyu/9K1skZYQPXSdYkY0ZQ hjzigUREBpRjl08lFtFDub91Lag2ti9R1AWDZEOmQxsm49gxwexCfgrYStM8VtUEvXZP EEiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=YI1zEv5GFtwERlrQpMIOUH1j5UheUEjYSFKQctQTQZY=; b=h3P7lHH8qVG6SdqmIiPqEAG6gsQJdGUUePS1kxzOBSjoVN3Q1c1q49Hbrm7xzb2sRv sGZtLOl2QRQ8IcH6Q3Y2i0JSBwhqoP1rodAHEOd3TeCzvaUYleS0TVKkYjKS0FJTmxRW MZYFeoJv2IcjHVPUNsXVsH2Wm3bp/BZVW7oUH0O4luYWl1RYYu+rwTg3Yb2+GpMZyZaa sc++1LysXBczA1jW72rD5ylMRZEsAbst2eTRqvYK8kn0FE38pTSe/tXqogLVFKj7mRN5 VjWbYDWnKNik7/3xiFbZI2sWDUDedcFsVMPSblLU0KPt7FbiMvKTSHNvssqVxhq/9okg jUwQ== X-Gm-Message-State: AElRT7ELv+1t8sX1At4tka20+g5IsNq7ezOT4jjQcSVlzKKN2uyVWR6Y l0gmyM1TFOFEj4QvzRc7nRKd2w== X-Google-Smtp-Source: AG47ELv9WPBola9x8fEIypQnpRLCgMMw9n4ouU1W/gRZjBIB1lCjfMXmdiTlGLykW6waMWES91IJyg== X-Received: by 10.28.105.19 with SMTP id e19mr3298372wmc.3.1521649304638; Wed, 21 Mar 2018 09:21:44 -0700 (PDT) Received: from bidouze.vm.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id b13sm3375271wmi.42.2018.03.21.09.21.43 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 21 Mar 2018 09:21:43 -0700 (PDT) Date: Wed, 21 Mar 2018 17:21:29 +0100 From: =?iso-8859-1?Q?Ga=EBtan?= Rivet To: Bruce Richardson Cc: Rosen Xu , dev@dpdk.org, declan.doherty@intel.com, shreyansh.jain@nxp.com, tianfei.zhang@intel.com, hao.wu@intel.com Message-ID: <20180321162129.4r5k2r5g4ftt2i23@bidouze.vm.6wind.com> References: <1521618694-140757-1-git-send-email-rosen.xu@intel.com> <1521618694-140757-4-git-send-email-rosen.xu@intel.com> <20180321102025.v2zeijxrsauuvkss@bidouze.vm.6wind.com> <20180321133509.GA15364@bricha3-MOBL.ger.corp.intel.com> <20180321141405.mdtibaq75o7dbx4o@bidouze.vm.6wind.com> <20180321143131.5yptsqcsypmgdtwj@bidouze.vm.6wind.com> <20180321154156.GB11100@bricha3-MOBL.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180321154156.GB11100@bricha3-MOBL.ger.corp.intel.com> User-Agent: NeoMutt/20170113 (1.7.2) Subject: Re: [dpdk-dev] [PATCH V2 3/5] Add Intel FPGA BUS Lib Code X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2018 16:21:45 -0000 On Wed, Mar 21, 2018 at 03:41:56PM +0000, Bruce Richardson wrote: > On Wed, Mar 21, 2018 at 03:31:31PM +0100, Gaëtan Rivet wrote: > > On Wed, Mar 21, 2018 at 03:14:05PM +0100, Gaëtan Rivet wrote: > > > Hi Bruce, > > > > > > On Wed, Mar 21, 2018 at 01:35:09PM +0000, Bruce Richardson wrote: > > > > On Wed, Mar 21, 2018 at 11:20:25AM +0100, Gaëtan Rivet wrote: > > > > > Hi, > > > > > > > > > > I have had issues compiling a few things here, have you checked > > > > > build status before submitting? > > > > > > > > > > On Wed, Mar 21, 2018 at 03:51:32PM +0800, Rosen Xu wrote: > > > > > > Signed-off-by: Rosen Xu > > > > > > --- > > > > > > > > > + > > > > > > +/* > > > > > > + * Scan the content of the FPGA bus, and the devices in the devices > > > > > > + * list > > > > > > + */ > > > > > > > > > > So you seem to scan your bus by reading parameters > > > > > given to the --ifpga EAL option. > > > > > > > > > > Can you justify why you cannot use the PCI bus, have your FPGA be probed > > > > > by a PCI driver, that would take those parameters as driver parameters, > > > > > and spawn raw devices (one per bitstream) as needed as a result? > > > > > > > > > > I see no reason this is not feasible. Unless you duly justify this > > > > > approach, it seems unacceptable to me. You are subverting generic EAL > > > > > code to bend things to your approach, without clear rationale. > > > > > > > > > > > > > While I agree with the comments in other emails about avoiding > > > > special-cases in the code that makes things not-scalable, I would take the > > > > view that using a bus-type is the correct choice for this. While you could > > > > have a single device that creates other devices, that is also true for all > > > > other buses as well. [Furthermore, I think it is incorrect assume that all > > > > devices on the FPGA bus would be raw devices, it's entirely possible to > > > > have cryptodevs, bbdevs or compress devs implemented in the AFUs]. > > > > > > > > Consider what a bus driver provides: it's a generic mechanism for scanning > > > > for devices - which all use a common connection method - for DPDK use, and > > > > mapping device drivers to those devices. For an FPGA device which presents > > > > multiple AFUs, this seems to be exactly what is required - a device driver > > > > to scan for devices and present them to DPDK. The FPGA bus driver will have > > > > to check each AFU and match it against the set of registered AFU device > > > > drivers to ensure that the crypto AFU gets the cryptodev driver, etc. > > > > > > > > Logically, therefore, it is a bus - which just happens to be a sub-bus of > > > > PCI, i.e. presented as a PCI device. Consider also that it may be possible > > > > or even desirable, to use blacklisting and whitelisting for those AFU > > > > devices so that some AFUs could be used by one app, while others by > > > > another. If we just have a single PCI device, I think we'll find ourselves > > > > duplicating a lot of bus-related functionality inside the driver in that > > > > case. > > > > > > > > Regards, > > > > /Bruce > > > > > > It makes sense indeed if you need to specialize several drivers specific > > > to AFU mappings. > > > > > > So, taking the bus approach, it seems we almost all agree that the > > > current probe is not good enough. > > > > > > I think you will run into issues if you registered your bus upon probing > > > a PCI driver. Instead, would it be possible to: > > > > > > * not add the EAL option --ifpga. > > > * register the ifpga bus like any other. > > > * have a PCI driver that hotplug an IFPGA device, triggering a > > > scan and probe on the ifpga bus using the PCI device parameters. > > > Essentially you would find there the parameters that could be > > > given previously to the --ifpga option. > > > > > > There should not be any EAL modification necessary. > > > > > > The only downside I see would be that having both IFPGA device (for > > > triggering bus ops) and afterward using AFU devices on this bus could be > > > confusing. However, it seems safer and still cleaner overall. > > > > Actually, just to build upon this remark, and thinking a little more > > about it: > > > > You would not need an additional "special" type of ifpga devices. > > Upon attempting to hotplug an ifpga device, you would have a new > > rte_devargs having the parameters within inserted, addressed to your > > bus. > > > > During scan, you would find this devargs, read the relevant parameters > > and parse them (exactly like it is now implemented). You would then be > > able to spawn as needed any number of rte_afu_device within your bus and > > the scan would be complete. > > > > The only trickery would be afterward, because you'd need at least one > > device that would be detached and having the name given in the devargs. > > So you would have to name your rte_afu_device after the devargs passed > > as parameter to the plug, essentially having all of them using the same name. > > > > Nothing prevents you from renaming your AFU devices after bus->plug, but > > the EAL will rely on it for matching and knowing the scan went ok. > > > > -- > Thanks, Gaëtan. > > So, just to confirm I understand the suggestion correctly, you are > proposing that we create and use the FPGA bus as normal, except that the > scan will not report any devices. Then when an FPGA PCI device is > discovered, we use the hotplug infrastructure to trigger a real scan of the > FPGA for AFU devices, and do the whole driver mapping etc. process then. > Is that the basic idea you suggest? Yes. -- Gaëtan Rivet 6WIND