From: Jerin Jacob <jerin.jacob@caviumnetworks.com>
To: Declan Doherty <declan.doherty@intel.com>
Cc: Thomas Monjalon <thomas@monjalon.net>,
Radu Nicolau <radu.nicolau@intel.com>,
dev@dpdk.org
Subject: Re: [dpdk-dev] [RFC][PATCH 2/5] pci: allow shared device instances.
Date: Wed, 10 May 2017 17:48:10 +0530 [thread overview]
Message-ID: <20170510121809.GA25209@jerin> (raw)
In-Reply-To: <73b94d12-f1cf-295f-284f-0ee1bd5f0d15@intel.com>
-----Original Message-----
> Date: Wed, 10 May 2017 12:31:48 +0100
> From: Declan Doherty <declan.doherty@intel.com>
> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> CC: Thomas Monjalon <thomas@monjalon.net>, Radu Nicolau
> <radu.nicolau@intel.com>, dev@dpdk.org
> Subject: Re: [dpdk-dev] [RFC][PATCH 2/5] pci: allow shared device instances.
> User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
> Thunderbird/45.8.0
>
> On 10/05/2017 12:08 PM, Jerin Jacob wrote:
> > -----Original Message-----
> > > Date: Wed, 10 May 2017 11:52:45 +0100
> > > From: Declan Doherty <declan.doherty@intel.com>
> > > To: Thomas Monjalon <thomas@monjalon.net>, Radu Nicolau
> > > <radu.nicolau@intel.com>
> > > CC: dev@dpdk.org
> > > Subject: Re: [dpdk-dev] [RFC][PATCH 2/5] pci: allow shared device instances.
> > > User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
> > > Thunderbird/45.8.0
> > >
> > > Hey Thomas, I've been working on this with Radu, so see my take below
> > >
> > > On 10/05/2017 11:28 AM, Thomas Monjalon wrote:
> > > > 10/05/2017 12:11, Radu Nicolau:
> > > > > Hi
> > > > >
> > > > >
> > > > > On 5/10/2017 10:09 AM, Thomas Monjalon wrote:
> > > > > > Hi,
> > > > > >
> > > > > > 09/05/2017 16:57, Radu Nicolau:
> > > > > > > Updated PCI initialization code to allow devices to be shared across multiple PMDs.
> > > > > > >
> > > > > > > Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> > > > > > I was waiting the day when we have a device shared
> > > > > > by two different interfaces.
> > > > > > Note that some Mellanox and Chelsio devices already instantiate
> > > > > > two ethdev ports per PCI device.
> > > > > >
> > > > > > Please explain your idea behind this "shared" flag.
> > > > > > What is your exact need?
> > > > >
> > > > > Currently for each pci device a look-up into a list of PMDs is
> > > > > performed, and when a match is found the system moves to the next
> > > > > device. Having this flag will allow a PMD to inform the system that
> > > > > there may be more matches, more PMDs that can be used for this
> > > > > particular device.
> > > > > There is a difference when comparing to the devices you mentioned above,
> > > > > in this case the PMDs are totally different types, one network and one
> > > > > cryptodev PMD for each IXGBE network card.
> > > >
> > > > Yes I know it is a lack in DPDK.
> > > > Linux introduced MultiFunction Device in 2005:
> > > > http://events.linuxfoundation.org/sites/events/files/slides/belloni-mfd-regmap-syscon_0.pdf
> > > >
> > >
> > > So at the most basic level the intention is to allow more than one device of
> > > different types, in our case a net PMD and a crypto PMD, to be instantiated
> > > on a single PCI bar, in essence to share the bar. I'm not familiar with the
> > > approaches taken in the Mellanox and Chelsio devices but I assume they are
> > > handled with the driver probe/create functions independently from the EAL
> > > infrastructure?
> > >
> > > For the initial proto-typing of this RFC we only implemented the
> > > multi-device creation but I envisage that there will be a requirement for
> > > sharing state between drivers, or at a minimum implementing locking around
> > > shared resources, registers etc. And I would like to see this done in a
> > > generic fashion that can me leverage by any driver and not have each driver
> > > having to solve this independently.
> >
> > Cavium's next generation PCI based NW devices has similar scheme where we
> > need to share the same BAR with multiple DPDK subsystems(ethdev,
> > eventdev etc) unlike current generation(OcteonTX).
> >
>
> Have you done investigation into how you would like to support this, and are
> you trending to any particular approach. The rte_bus approach as you outline
> below does sound like it would suit this multi-function device.
Not much investigation has been done as its for next generation.
It is no PCIe multi function device.
There will be a lot shared functions between these shared DPDK devices and there
should be place holder for this in code.I thought driver/bus/foo may a
option. In additional to this, If we expose new function pointer based
interfaces in bus for the shared device register access and other shared resource
alloc/free between these two DPDK devices, it can be centralized to one
place(driver/bus/foo) and generalized.
Just 2c. We haven't done any prototype.
>
> > I think, Another possible way to handle this in generic way is to:
> > Register a new rte_bus for the shared PCI access which sits on top PCIe bus.
> > With new bus's scan and probe scheme, it can probe the two devices.
> >
> >
>
> Yes, this would work and I think it makes a lot of sense in the case where
> you have logically independent hardware functional blocks on a shared bus.
> In our particular case, we only have a single physical device, which we are
> presenting as 2 logical devices purely to improve the sw model through DPDK
> existing infrastructure. We may also need to implement some shared context
> for protecting access to shared resources such as register and to
> synchronized exposure of capabilities. In the case of the IXBGE family of
> devices they can support MACsec or IPsec functionality but not both at the
> same time, so some mechanism of passing this state between the net and
> crypto PMDs will be required. I guess it should be possible to do this
> through the bus model as well but we'll need to have another look, although
> my initial feeling is they are slightly different problems.
next prev parent reply other threads:[~2017-05-10 12:18 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-09 14:57 [dpdk-dev] [RFC][PATCH 0/5] cryptodev: Adding support for inline crypto processing of IPsec flows Radu Nicolau
2017-05-09 14:57 ` [dpdk-dev] [RFC][PATCH 1/5] cryptodev: Updated API to add suport for inline IPSec Radu Nicolau
2017-05-09 14:57 ` [dpdk-dev] [RFC][PATCH 2/5] pci: allow shared device instances Radu Nicolau
2017-05-10 9:09 ` Thomas Monjalon
2017-05-10 10:11 ` Radu Nicolau
2017-05-10 10:28 ` Thomas Monjalon
2017-05-10 10:47 ` Radu Nicolau
2017-05-10 10:52 ` Declan Doherty
2017-05-10 11:08 ` Jerin Jacob
2017-05-10 11:31 ` Declan Doherty
2017-05-10 12:18 ` Jerin Jacob [this message]
2017-05-10 11:37 ` Thomas Monjalon
2017-05-09 14:57 ` [dpdk-dev] [RFC][PATCH 3/5] mbuff: added inline IPSec flags and metadata Radu Nicolau
2017-05-09 14:57 ` [dpdk-dev] [RFC][PATCH 4/5] cryptodev: added new crypto PMD supporting inline IPSec for IXGBE Radu Nicolau
2017-05-09 14:57 ` [dpdk-dev] [RFC][PATCH 5/5] examples: updated IPSec sample app to support inline IPSec Radu Nicolau
2017-05-10 16:07 ` [dpdk-dev] [RFC][PATCH 0/5] cryptodev: Adding support for inline crypto processing of IPsec flows Boris Pismenny
2017-05-10 17:21 ` Declan Doherty
2017-05-11 5:27 ` Boris Pismenny
2017-05-11 9:05 ` Radu Nicolau
2017-05-16 21:46 ` Thomas Monjalon
2017-05-24 10:06 ` Declan Doherty
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170510121809.GA25209@jerin \
--to=jerin.jacob@caviumnetworks.com \
--cc=declan.doherty@intel.com \
--cc=dev@dpdk.org \
--cc=radu.nicolau@intel.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).