From: "Doherty, Declan" <declan.doherty@intel.com>
To: Neil Horman <nhorman@tuxdriver.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3 2/5] Link Bonding PMD Library (librte_eal/librte_ether link bonding support changes)
Date: Mon, 16 Jun 2014 08:59:25 +0000 [thread overview]
Message-ID: <345C63BAECC1AD42A2EC8C63AFFC3ADC13D39383@IRSMSX101.ger.corp.intel.com> (raw)
In-Reply-To: <20140613193815.GE22451@hmsreliant.think-freely.org>
> -----Original Message-----
> From: Neil Horman [mailto:nhorman@tuxdriver.com]
> Sent: Friday, June 13, 2014 8:38 PM
> To: Doherty, Declan
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v3 2/5] Link Bonding PMD Library
> (librte_eal/librte_ether link bonding support changes)
>
> On Fri, Jun 13, 2014 at 06:34:04PM +0000, Doherty, Declan wrote:
> > > -----Original Message-----
> > > From: Neil Horman [mailto:nhorman@tuxdriver.com]
> > > Sent: Friday, June 13, 2014 5:08 PM
> > > To: Doherty, Declan
> > > Cc: dev@dpdk.org
> > > Subject: Re: [dpdk-dev] [PATCH v3 2/5] Link Bonding PMD Library
> > > (librte_eal/librte_ether link bonding support changes)
> > >
> > > On Fri, Jun 13, 2014 at 03:41:59PM +0100, Declan Doherty wrote:
> > > > Updating functionality in EAL to support adding link bonding
> > > > devices via –vdev option. Link bonding devices will be
> > > > initialized after all physical devices have been probed and
> > > > initialized.
> > > >
> > > > Signed-off-by: Declan Doherty <declan.doherty@intel.com>
> > > > ---
> > > > lib/librte_eal/common/eal_common_dev.c | 66
> > > +++++++++++++++++++++++++++--
> > > > lib/librte_eal/common/eal_common_pci.c | 6 +++
> > > > lib/librte_eal/common/include/eal_private.h | 7 +++
> > > > lib/librte_eal/common/include/rte_dev.h | 1 +
> > > > lib/librte_ether/rte_ethdev.c | 34 +++++++++++++--
> > > > lib/librte_ether/rte_ethdev.h | 7 ++-
> > > > lib/librte_pmd_pcap/rte_eth_pcap.c | 22 +++++-----
> > > > lib/librte_pmd_ring/rte_eth_ring.c | 32 +++++++-------
> > > > lib/librte_pmd_ring/rte_eth_ring.h | 3 +-
> > > > lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +-
> > > > 10 files changed, 144 insertions(+), 36 deletions(-)
> > > >
> > > > diff --git a/lib/librte_eal/common/eal_common_dev.c
> > > b/lib/librte_eal/common/eal_common_dev.c
> > > > index eae5656..b50c908 100644
> > > > --- a/lib/librte_eal/common/eal_common_dev.c
> > > > +++ b/lib/librte_eal/common/eal_common_dev.c
> > > > @@ -75,14 +75,28 @@ rte_eal_dev_init(void)
> > > >
> > > > /* call the init function for each virtual device */
> > > > TAILQ_FOREACH(devargs, &devargs_list, next) {
> > > > + uint8_t bdev = 0;
> > > >
> > > > if (devargs->type != RTE_DEVTYPE_VIRTUAL)
> > > > continue;
> > > >
> > > > TAILQ_FOREACH(driver, &dev_driver_list, next) {
> > > > - if (driver->type != PMD_VDEV)
> > > > + /* RTE_DEVTYPE_VIRTUAL can only be a virtual or bonded
> > > device*/
> > > > + if (driver->type != PMD_VDEV && driver->type !=
> > > PMD_BDEV)
> > > > continue;
> > > >
> > > > + /*
> > > > + * Bonded devices are not initialize here, we do it later in
> > > > + * rte_eal_bonded_dev_init() after all physical devices
> > > have been
> > > > + * probed and initialized
> > > > + */
> > > > + if (driver->type == PMD_BDEV &&
> > > > + !strncmp(driver->name, devargs-
> > > >virtual.drv_name,
> > > > + strlen(driver->name))) {
> > > > + bdev = 1;
> > > > + break;
> > > > + }
> > > > +
> > > I really don't think you need to add a new device type for bonded devs. Its got
> > > no specific hardware that it drives, and you configure it with a --vdev
> command,
> > > so treat it as one here. I understand that you need to pass additional
> > > information about slaves to a bonded device, which is fine, but you can do that
> > > with kvargs pretty easily, at which point its just another vdev. The only other
> > > requirement is that you initilize the bonded vdev after the slave vdevs have
> > > been created, which you can do by any of several methods (a priority field to
> > > indicate that bonded drivers should be initilized last/later, a deferral return
> > > code from the init routine, or by dead reckoning via the careful construction
> of
> > > the application command line (placed the bonded --vdev option last on the
> > > command line argument list at run time).
> > >
> > It was my initial intent to do as you have describe above, but the physical devices
> > cause a real issue here, physical devices don't call through to
> rte_eth_dev_allocate until
> > during rte_eal_pci_probe call, so it's not possible to initialize the bonded device
> from
> > within rte_eal_dev_init as the physical devices have not been fully initialized at
> this
> > point, as a port_id has not been allocated and can't be added as bonding slaves. I
> don't
> > see away around this without changing the EAL API, which I've tried to avoid with
> this solution.
> >
> > Ordering isn't an issue, and can easily be solved if the above problem didn't
> exist, and although
> > a new device type isn't technically required, I think it's a cleaner solution than
> doing
> > string comparisons.
> >
>
>
> Ugh, you're right. That stinks. We still shouldn't just drop the init code in
> with the pci probe code though, especially if dpdk starts supporting more
> stacked network devices. A notification system for interface creation that you
> could monitor for pci_device addresses (B:D:F tupples) would be nice, but might
> be overkill since dpdk doesn't really support device freeing yet. what might be
> useful is a generic ordering mechanism for rte_eal_dev_init. If you added a
> priority field to the rte_driver structure, you could create an enumeration for
> the field to define when the init routine for that driver was called something
> like:
> typdef enum {
> PMD_INIT_FIRST = 0,
> PMD_INIT_POST_PROBE = 1,
> }
>
> Then you could make multiple calls to rte_eal_dev_init from rte_eal_init at the
> appropriate places in the initalization process, each time passing one of the
> above values. rte_eal_dev_init would only initalize those drivers whos priority
> matched the passed in priority. That would be helpful for interfaces like
> vlans, vxlans, tunnels, etc.
>
I'm not sure I understand how you intend this to work, are you proposing that
rte_eal_dev_init is call multiple times by the user pre and post rte_eal_pci_probe
this doesn't seem like an idea solution either. I'm not 100% clear why
rte_eal_pci_probe is currently called by the application code and not initiated
from within rte_eal_dev_init, if this was the case we would be able to figure out
a dependency tree for initialization of devices, based on the parsed input
arguments without having extra user input to or multiple calls of rte_eal_dev_init.
Either way I don't think that any of these solution are trivial and they would
need to be thoroughly tested, which I don't think will be possible in the time
frame for 1.7. If my current implementation calling rte_eal_bonded_dev_init
from within rte_eal_pci_probe is unacceptable, then I propose that I remove
the EAL changes for --vdev initialization of bonding devices from this release and
only support the C API, and that an overall strategy is discussed for static device
initialization that is developed for 1.7.1 or 1.8.
> > > >
> > > > +static int
> > > > +rte_eth_dev_name_unique(const char* name)
> > > > +{
> > > > + unsigned i;
> > > > +
> > > > + for (i = 0; i < nb_ports; i++) {
> > > > + if (strcmp(rte_eth_devices[i].data->name, name) == 0)
> > > > + return -1;
> > > > + }
> > > > +
> > > > + return 0;
> > > > +}
> > > > +
> > > > struct rte_eth_dev *
> > > > -rte_eth_dev_allocate(void)
> > > > +rte_eth_dev_allocate(const char* name)
> > > > {
> > > > struct rte_eth_dev *eth_dev;
> > > >
> > > > @@ -165,23 +179,37 @@ rte_eth_dev_allocate(void)
> > > > if (rte_eth_dev_data == NULL)
> > > > rte_eth_dev_data_alloc();
> > > >
> > > > + if (rte_eth_dev_name_unique(name)) {
> > > > + PMD_DEBUG_TRACE("Ethernet Device with name %s already
> > > allocated!\n");
> > > > + return NULL;
> > > > + }
> > > > +
> > > This seems fairly racy if you allow dynamic device creation at run time from
> the
> > > application, if multiple threads attempt to create bonds in parallel.
> >
> > True but if this is an issue then there probably should be some locking around
> this allocation anyway.
> Agreed the allocation of the port id is also racy.
>
> > >
> > > > /**
> > > > diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c
> > > b/lib/librte_pmd_pcap/rte_eth_pcap.c
> > > Hmm, we're modifying other pmds for the naming feature, I think it would be
> best
> > > split out into a separate patch. Something entitled "support unique interface
> > > naming for virtual pmds" or something.
> >
> > True, but if I do that, I will need to break support for virtual devices in this patch
> set, and
> > I would prefer to keep this functionality working initial release of link bonding.
> Also another
> > patchset will be required at some point in the near future to tidy up/rationalize
> the current
> > handling of virtual devices identification and these changes do not change
> functionally how
> > these devices work.
>
> I'm not sure I'm following what you're saying here. If you split this into
> another patch set, and order earlier in this series, then you won't break
> anything. I'm not suggesting a functional change here, just a organizational
> one, so that this change doesn't do more than the changelog indicates.
>
Sorry I misunderstood, I thought that you where proposing that I create a completely new
patchset separate to the link bonding release set for these changes.
> Neil
Declan
next prev parent reply other threads:[~2014-06-16 9:00 UTC|newest]
Thread overview: 127+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-05-28 15:32 [dpdk-dev] [PATCH 0/4] Link Bonding Library declan.doherty
2014-05-28 15:32 ` [dpdk-dev] [PATCH 1/4] " declan.doherty
2014-05-28 16:54 ` Shaw, Jeffrey B
2014-05-29 13:32 ` Doherty, Declan
2014-05-28 15:32 ` [dpdk-dev] [PATCH 2/4] Link bonding unit tests declan.doherty
2014-05-28 15:32 ` [dpdk-dev] [PATCH 3/4] Link bonding integration into testpmd declan.doherty
2014-05-28 15:32 ` [dpdk-dev] [PATCH 4/4] Add Link Bonding Library to Doxygen declan.doherty
2014-05-28 17:49 ` [dpdk-dev] [PATCH 0/4] Link Bonding Library Neil Horman
2014-05-29 10:33 ` Doherty, Declan
2014-05-29 11:33 ` Neil Horman
2014-05-29 3:23 ` Cao, Waterman
2014-05-29 10:35 ` Doherty, Declan
2014-06-04 15:18 ` [dpdk-dev] [PATCH v2 " declan.doherty
2014-06-04 15:18 ` [dpdk-dev] [PATCH v2 1/4] " declan.doherty
2014-06-04 15:18 ` declan.doherty
2014-06-05 15:15 ` Stephen Hemminger
2014-06-06 9:07 ` Doherty, Declan
2014-06-06 15:13 ` Stephen Hemminger
2014-06-09 21:11 ` Eric Kinzie
2014-06-13 14:03 ` Doherty, Declan
2014-06-04 15:18 ` [dpdk-dev] [PATCH v2 2/4] Link bonding unit tests, including: - code to generate packet bursts for testing rx and tx functionality of bonded device - virtual/stubbed out ethdev for use as slave ethdev in testing - checkpack fixes declan.doherty
2014-06-04 15:18 ` declan.doherty
2014-06-04 15:18 ` [dpdk-dev] [PATCH v2 0/4] Link Bonding Library declan.doherty
2014-06-04 15:18 ` [dpdk-dev] [PATCH v2 3/4] Adding link bonding support to testpmd. - Includes the ability to create new bonded devices. - Add /remove bonding slave devices. - Interogate bonded device stats/configuration - Change bonding modes and select balance transmit polices declan.doherty
2014-06-04 15:18 ` [dpdk-dev] [PATCH v2 4/4] Add Link Bonding Library to Doxygen declan.doherty
2014-06-04 16:10 ` [dpdk-dev] [PATCH v2 0/4] Link Bonding Library Doherty, Declan
2014-06-05 8:03 ` De Lara Guarch, Pablo
2014-06-05 11:03 ` Neil Horman
2014-06-06 8:23 ` Doherty, Declan
2014-06-06 14:54 ` Neil Horman
2014-06-13 14:56 ` Doherty, Declan
2014-06-13 15:11 ` Neil Horman
2014-06-06 3:26 ` Cao, Waterman
2014-06-11 16:33 ` Thomas Monjalon
2014-06-13 14:08 ` Doherty, Declan
2014-06-13 15:15 ` Thomas Monjalon
2014-06-13 14:41 ` [dpdk-dev] [PATCH v3 0/5] Link Bonding PMD Library Declan Doherty
2014-06-13 14:41 ` [dpdk-dev] [PATCH v3 1/5] " Declan Doherty
2014-06-13 14:41 ` [dpdk-dev] [PATCH v3 2/5] Link Bonding PMD Library (librte_eal/librte_ether link bonding support changes) Declan Doherty
2014-06-13 16:08 ` Neil Horman
2014-06-13 18:34 ` Doherty, Declan
2014-06-13 19:38 ` Neil Horman
2014-06-16 8:59 ` Doherty, Declan [this message]
2014-06-16 11:07 ` Neil Horman
2014-06-16 16:17 ` Richardson, Bruce
2014-06-16 17:47 ` Neil Horman
2014-06-16 18:07 ` Richardson, Bruce
2014-06-16 18:09 ` Thomas Monjalon
2014-06-13 21:59 ` Stephen Hemminger
2014-06-16 7:59 ` Doherty, Declan
2014-06-13 14:42 ` [dpdk-dev] [PATCH v3 3/5] Link Bonding PMD Library (Unit Test Suite) Declan Doherty
2014-06-13 14:42 ` [dpdk-dev] [PATCH v3 4/5] Link Bonding PMD Library (testpmd link bonding API support) Declan Doherty
2014-06-13 14:42 ` [dpdk-dev] [PATCH v3 5/5] Link Bonding PMD Library (Doxygen Additions) Declan Doherty
2014-06-13 15:20 ` [dpdk-dev] [PATCH v3 0/5] Link Bonding PMD Library Neil Horman
2014-06-16 11:18 ` [dpdk-dev] [PATCH v4 0/6] Link Bonding Library Declan Doherty
2014-06-18 16:14 ` [dpdk-dev] [PATCH v5 " Declan Doherty
2014-06-18 16:18 ` Neil Horman
2014-06-24 14:52 ` [dpdk-dev] [PATCH v6 " Declan Doherty
2014-06-24 16:03 ` [dpdk-dev] [PATCH v7 " Declan Doherty
2014-06-25 20:07 ` [dpdk-dev] [PATCH v8 " Declan Doherty
2014-06-26 16:02 ` De Lara Guarch, Pablo
2014-06-26 23:57 ` [dpdk-dev] [PATCH v9 0/5] link bonding Thomas Monjalon
2014-06-26 23:57 ` [dpdk-dev] [PATCH v9 1/5] bond: new link bonding library Thomas Monjalon
2014-06-27 0:45 ` Thomas Monjalon
2014-06-26 23:57 ` [dpdk-dev] [PATCH v9 2/5] ethdev: add unique name to devices Thomas Monjalon
2014-06-26 23:57 ` [dpdk-dev] [PATCH v9 3/5] eal: support link bonding device initialization Thomas Monjalon
2014-06-26 23:57 ` [dpdk-dev] [PATCH v9 4/5] bond: unit tests Thomas Monjalon
2014-06-26 23:57 ` [dpdk-dev] [PATCH v9 5/5] bond: testpmd support Thomas Monjalon
2014-06-27 10:18 ` [dpdk-dev] [PATCH v10 0/5] link bonding Declan Doherty
2014-06-27 20:58 ` Thomas Monjalon
2014-06-29 17:49 ` [dpdk-dev] [PATCH v11 0/5] link bonding library Declan Doherty
2014-06-30 9:21 ` Thomas Monjalon
2014-06-30 9:28 ` Doherty, Declan
2014-07-01 22:01 ` Thomas Monjalon
2014-06-29 17:49 ` [dpdk-dev] [PATCH v11 1/5] bond: new " Declan Doherty
2014-06-30 9:13 ` Thomas Monjalon
2014-06-30 22:29 ` Robert Sanford
2014-07-01 14:16 ` Thomas Monjalon
2014-07-01 14:19 ` Doherty, Declan
2014-07-01 14:26 ` Thomas Monjalon
2014-06-29 17:49 ` [dpdk-dev] [PATCH v11 2/5] ethdev: add unique name to devices Declan Doherty
2014-06-29 17:49 ` [dpdk-dev] [PATCH v11 3/5] eal: support link bonding device initialization Declan Doherty
2014-06-29 17:49 ` [dpdk-dev] [PATCH v11 4/5] bond: unit tests Declan Doherty
2014-06-30 8:56 ` Thomas Monjalon
2014-06-29 17:49 ` [dpdk-dev] [PATCH v11 5/5] bond: testpmd support Declan Doherty
2014-06-27 10:18 ` [dpdk-dev] [PATCH v10 1/5] bond: new link bonding library Declan Doherty
2014-06-27 10:18 ` [dpdk-dev] [PATCH v10 2/5] ethdev: add unique name to devices Declan Doherty
2014-06-27 10:18 ` [dpdk-dev] [PATCH v10 3/5] eal: support link bonding device initialization Declan Doherty
2014-06-27 10:18 ` [dpdk-dev] [PATCH v10 4/5] bond: unit tests Declan Doherty
2014-06-27 10:18 ` [dpdk-dev] [PATCH v10 5/5] bond: testpmd support Declan Doherty
2014-06-25 20:07 ` [dpdk-dev] [PATCH v8 1/6] Link Bonding Library (lib/librte_pmd_bond) Declan Doherty
2014-06-25 20:07 ` [dpdk-dev] [PATCH v8 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-25 20:07 ` [dpdk-dev] [PATCH v8 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-25 20:07 ` [dpdk-dev] [PATCH v8 4/6] Link bonding Unit Tests Declan Doherty
2014-06-25 20:07 ` [dpdk-dev] [PATCH v8 5/6] testpmd link bonding additions Declan Doherty
2014-06-25 20:07 ` [dpdk-dev] [PATCH v8 6/6] Link Bonding Library doxygen additions Declan Doherty
2014-06-24 16:03 ` [dpdk-dev] [PATCH v7 1/6] Link Bonding Library (lib/librte_pmd_bond) Declan Doherty
2014-06-24 16:03 ` [dpdk-dev] [PATCH v7 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-24 16:03 ` [dpdk-dev] [PATCH v7 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-25 13:54 ` Thomas Monjalon
2014-06-25 14:41 ` Doherty, Declan
2014-06-25 16:00 ` Thomas Monjalon
2014-06-25 16:15 ` Richardson, Bruce
2014-06-24 16:03 ` [dpdk-dev] [PATCH v7 4/6] Link bonding Unit Tests Declan Doherty
2014-06-24 16:03 ` [dpdk-dev] [PATCH v7 5/6] testpmd link bonding additions Declan Doherty
2014-06-24 16:03 ` [dpdk-dev] [PATCH v7 6/6] Link Bonding Library doxygen additions Declan Doherty
2014-06-25 13:43 ` Thomas Monjalon
2014-06-25 14:19 ` Doherty, Declan
2014-06-25 14:23 ` Thomas Monjalon
2014-06-24 14:52 ` [dpdk-dev] [PATCH v6 1/6] Link Bonding Library (lib/librte_pmd_bond) Declan Doherty
2014-06-24 14:52 ` [dpdk-dev] [PATCH v6 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-24 14:52 ` [dpdk-dev] [PATCH v6 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-24 14:52 ` [dpdk-dev] [PATCH v6 4/6] Link bonding Unit Tests Declan Doherty
2014-06-24 14:52 ` [dpdk-dev] [PATCH v6 5/6] testpmd link bonding additions Declan Doherty
2014-06-24 14:52 ` [dpdk-dev] [PATCH v6 6/6] Link Bonding Library doxygen additions Declan Doherty
2014-06-18 16:14 ` [dpdk-dev] [PATCH v5 1/6] Link Bonding Library (lib/librte_pmd_bond) Declan Doherty
2014-06-18 16:14 ` [dpdk-dev] [PATCH v5 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-18 16:14 ` [dpdk-dev] [PATCH v5 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-18 16:14 ` [dpdk-dev] [PATCH v5 4/6] Link bonding Unit Tests Declan Doherty
2014-06-18 16:14 ` [dpdk-dev] [PATCH v5 5/6] testpmd link bonding additions Declan Doherty
2014-06-18 16:14 ` [dpdk-dev] [PATCH v5 6/6] Link Bonding Library doxygen additions Declan Doherty
2014-06-16 11:18 ` [dpdk-dev] [PATCH v4 1/6] Link Bonding Library (lib/librte_pmd_bond) initial release with support for Mode 0 - Round Robin Mode 1 - Active Backup Mode 2 - Balance -> Supports 3 transmit polices (layer 2, layer 2+3, la Mode 3 - Broadcast Declan Doherty
2014-06-16 11:18 ` [dpdk-dev] [PATCH v4 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-16 11:18 ` [dpdk-dev] [PATCH v4 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-16 11:18 ` [dpdk-dev] [PATCH v4 4/6] Link bonding Unit Tests Declan Doherty
2014-06-16 11:18 ` [dpdk-dev] [PATCH v4 5/6] testpmd link bonding additions Declan Doherty
2014-06-16 11:18 ` [dpdk-dev] [PATCH v4 6/6] Link Bonding Library doxygen additions Declan Doherty
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=345C63BAECC1AD42A2EC8C63AFFC3ADC13D39383@IRSMSX101.ger.corp.intel.com \
--to=declan.doherty@intel.com \
--cc=dev@dpdk.org \
--cc=nhorman@tuxdriver.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).