DPDK usage discussions
 help / color / mirror / Atom feed
From: devendra rawat <devendra.rawat.singh@gmail.com>
To: "Loftus, Ciara" <ciara.loftus@intel.com>
Cc: Thomas Monjalon <thomas@monjalon.net>,
	Kevin Traynor <ktraynor@redhat.com>,
	"ovs-dev@openvswitch.org" <ovs-dev@openvswitch.org>,
	Adrien Mazarguil <adrien.mazarguil@6wind.com>,
	 "nelio.laranjeiro@6wind.com" <nelio.laranjeiro@6wind.com>,
	"users@dpdk.org" <users@dpdk.org>,
	 Yuanhan Liu <yliu@fridaylinux.org>,
	"olgas@mellanox.com" <olgas@mellanox.com>
Subject: Re: [dpdk-users] [ovs-dev] adding dpdk ports sharing same pci address to ovs-dpdk bridge
Date: Tue, 10 Oct 2017 18:26:51 +0530	[thread overview]
Message-ID: <CACsJYqqNCChjQyA3csMAzxfcuGg2+-n-QPq1b9dgCr4eEOz-kg@mail.gmail.com> (raw)
In-Reply-To: <74F120C019F4A64C9B78E802F6AD4CC278E1B981@IRSMSX106.ger.corp.intel.com>

Hi Ciara,

On Fri, Oct 6, 2017 at 7:30 PM, Loftus, Ciara <ciara.loftus@intel.com>
wrote:

> >
> > On Thu, Sep 21, 2017 at 1:58 PM, Loftus, Ciara <ciara.loftus@intel.com>
> > wrote:
> > > 21/09/2017 10:04, Loftus, Ciara:
> > > > > 20/09/2017 19:33, Kevin Traynor:
> > > > > > On 09/08/2017 10:56 AM, Loftus, Ciara wrote:
> > > > > > > It seems the DPDK function rte_eth_dev_get_port_by_name() will
> > > > > > > always return the port ID of the first port on your NIC,
> > > > > > > when you specify the single PCI address and that's where the
> > > > > > > problem is. There doesn't seem to be a way currently to
> > > > > > > indicate to the calling application that in fact two
> > > > > > > (or more) port IDs are associated with the one PCI address.
> > > > >
> > > > > We have two ports (with the same PCI address) so we should have
> > > > > two different names.
> > > > > Where the names passed to rte_eth_dev_get_port_by_name() come
> > > from?
> > > > > It is the user parameter from options:dpdk-devargs=0002:01:00.0,
> > right?
> > > >
> > > > Yes, we're using the PCI address specified by the user in
> dpdk-devargs.
> > > >
> > > > > > > I am cc-ing DPDK users mailing list for hopefully some input.
> > > > > > > Are there any plans for the rte_eth_dev_get_port_by_name
> > > function
> > > > > > > to be compatible with NICs with multiple ports under the same
> PCI
> > > address?
> > > > >
> > > > > We cannot return two different ports for the same name.
> > > > > There are two issues here:
> > > > >   - the input should not be the PCI address
> > > > >   - the ethdev function should look at ethdev name, not rte_device
> > > > > one
> > > >
> > > > This would require the user having to "guess" the DPDK ethdev name
> > > > which is something we'd like to avoid.
> > >
> > > Yes, but you can provide a way to list the ports with their names
> > > and characteristics.
> > Ok, I see. Maybe something like this could be considered:
> >
> > port A = dpdk-devargs=xx:yy:zz 0
> > port B = dpdk-devargs=xx:yy:zz 1
> >
> > If we detect a value after the PCI address we iterate through the
> > rte_eth_dev_info
> > (http://dpdk.org/doc/api/structrte__eth__dev__info.html) for all valid
> port
> > IDs and assign port A to the first ethdev encountered with the provided
> PCI
> > address, and port B to the second, etc.
> >
> > If we don't detect a value, then we operate as normal. Thoughts?
> >
> > Hi Everyone,
> >
> > Anything finalized for sorting out this issue, do you need any more
> > information regarding this issue ?
>
> Hi,
>
> I put together a very rough RFC that aims to work-around the issue:
> https://mail.openvswitch.org/pipermail/ovs-dev/2017-October/339496.html
>
> It hasn't been tested as I don't have access to the type of card that has
> one PCI for multiple ports.
> If anybody does have access to such a device, I welcome you to try the
> patch although I'm not hopeful it will succeed first -pass.
> Looking for feedback on implementation, interface, etc.
>
>  I applied the patch on top of OVS v2.8.1 and used DPDK v17.08. I used
Mellanox ConnectX-3 pro NIC for testing, this NIC provides two 10G ports
that share a single PCI address.

The patch is working fine, I was able to add both the 10G ports to OVS
bridge by specifying the port no. (0 or 1) in the dpdk-devargs.

# ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
options:dpdk-devargs=0002:01:00.0,0
# ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
options:dpdk-devargs=0002:01:00.0,1

the port no. (0 and 1) are bound to actual physical port, i.e if I keep on
adding and deleting port no. 0 to bridge br0 multiple times,
every time I add the port back to br0, the same physical port is added to
bridge br0.

Thanks,
Devendra


> >
> > Thanks,
> > Devendra
> >
> >
> > >
> > > > We had the same problem using DPDK port IDs and decided not to use
> > > > them anymore, and use the PCI instead as it took the guesswork out.
> > > > Ethdev names and port IDs can change between tests, unlike the PCI
> > > > address which tends to remain constant for a device.
> > >
> > > We can add a requirement on ethdev names and make sure they remain
> > > constant for a given port.
> > >
> > > > > The idea is that we have only one rte_device object and we
> instantiate
> > > > > two rte_eth_dev ports.
> > > > > An ethdev port can be identified with its id (a number) or its
> unique
> > > name.
> > > > > Unfortunately, the user cannot guess the port id or the name set
> by the
> > > > > PMD.
> > > >
> > > > Exactly. Thanks for clarifying what's going on under the hood.
> > > >
> > > > Ciara
> > > >
> > > > >
> > > > > > Hi Adrien/Nelio,
> > > > > >
> > > > > > Is this something you can answer? We're wondering how to handle
> > this
> > > in
> > > > > > OVS and whether a temporary or long term solution is needed.
> > > > >
> > > > > I suggest to rely on ethdev name.
> > > > > You will need to show to the user the mapping between the bus
> > > information
> > > > > (PCI id here) and the device names.
> > > > >
> > > > > Another alternative is to add a new function returning all ethdev
> ports
> > > > > associated to a given rte_device resource.
> > > > > So you would get two ports and you could pick one on the first
> "add-
> > > port",
> > > > > and the other one for the second "add-port" command.
> > > > > It means the user would be forced to add them in the right order
> if he
> > > > > wants a reproducible result.
>
>

  reply	other threads:[~2017-10-10 12:56 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CACsJYqps0gM6NTUspVBcvjw1-VM4wj_UV2SbAhWGd-Y6WDBOzg@mail.gmail.com>
2017-09-08  9:56 ` Loftus, Ciara
2017-09-20 17:33   ` Kevin Traynor
2017-09-20 22:08     ` Thomas Monjalon
2017-09-21  8:04       ` Loftus, Ciara
2017-09-21  8:19         ` Thomas Monjalon
2017-09-21  8:28           ` Loftus, Ciara
2017-10-05 16:19             ` devendra rawat
2017-10-06 14:00               ` Loftus, Ciara
2017-10-10 12:56                 ` devendra rawat [this message]
2017-11-07  9:08                   ` Yuanhan Liu
2017-09-21  9:17     ` Adrien Mazarguil
2017-09-21  9:45       ` Adrien Mazarguil

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CACsJYqqNCChjQyA3csMAzxfcuGg2+-n-QPq1b9dgCr4eEOz-kg@mail.gmail.com \
    --to=devendra.rawat.singh@gmail.com \
    --cc=adrien.mazarguil@6wind.com \
    --cc=ciara.loftus@intel.com \
    --cc=ktraynor@redhat.com \
    --cc=nelio.laranjeiro@6wind.com \
    --cc=olgas@mellanox.com \
    --cc=ovs-dev@openvswitch.org \
    --cc=thomas@monjalon.net \
    --cc=users@dpdk.org \
    --cc=yliu@fridaylinux.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).