DPDK patches and discussions
 help / color / mirror / Atom feed
From: Neil Horman <nhorman@tuxdriver.com>
To: "Doherty, Declan" <declan.doherty@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3 2/5] Link Bonding PMD Library (librte_eal/librte_ether link bonding support changes)
Date: Fri, 13 Jun 2014 15:38:15 -0400
Message-ID: <20140613193815.GE22451@hmsreliant.think-freely.org> (raw)
In-Reply-To: <345C63BAECC1AD42A2EC8C63AFFC3ADC13D38EE9@IRSMSX101.ger.corp.intel.com>

On Fri, Jun 13, 2014 at 06:34:04PM +0000, Doherty, Declan wrote:
> > -----Original Message-----
> > From: Neil Horman [mailto:nhorman@tuxdriver.com]
> > Sent: Friday, June 13, 2014 5:08 PM
> > To: Doherty, Declan
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH v3 2/5] Link Bonding PMD Library
> > (librte_eal/librte_ether link bonding support changes)
> > 
> > On Fri, Jun 13, 2014 at 03:41:59PM +0100, Declan Doherty wrote:
> > > Updating functionality in EAL to support adding link bonding
> > > devices via –vdev option. Link bonding devices will be
> > > initialized after all physical devices have been probed and
> > > initialized.
> > >
> > > Signed-off-by: Declan Doherty <declan.doherty@intel.com>
> > > ---
> > >  lib/librte_eal/common/eal_common_dev.c      | 66
> > +++++++++++++++++++++++++++--
> > >  lib/librte_eal/common/eal_common_pci.c      |  6 +++
> > >  lib/librte_eal/common/include/eal_private.h |  7 +++
> > >  lib/librte_eal/common/include/rte_dev.h     |  1 +
> > >  lib/librte_ether/rte_ethdev.c               | 34 +++++++++++++--
> > >  lib/librte_ether/rte_ethdev.h               |  7 ++-
> > >  lib/librte_pmd_pcap/rte_eth_pcap.c          | 22 +++++-----
> > >  lib/librte_pmd_ring/rte_eth_ring.c          | 32 +++++++-------
> > >  lib/librte_pmd_ring/rte_eth_ring.h          |  3 +-
> > >  lib/librte_pmd_xenvirt/rte_eth_xenvirt.c    |  2 +-
> > >  10 files changed, 144 insertions(+), 36 deletions(-)
> > >
> > > diff --git a/lib/librte_eal/common/eal_common_dev.c
> > b/lib/librte_eal/common/eal_common_dev.c
> > > index eae5656..b50c908 100644
> > > --- a/lib/librte_eal/common/eal_common_dev.c
> > > +++ b/lib/librte_eal/common/eal_common_dev.c
> > > @@ -75,14 +75,28 @@ rte_eal_dev_init(void)
> > >
> > >  	/* call the init function for each virtual device */
> > >  	TAILQ_FOREACH(devargs, &devargs_list, next) {
> > > +		uint8_t bdev = 0;
> > >
> > >  		if (devargs->type != RTE_DEVTYPE_VIRTUAL)
> > >  			continue;
> > >
> > >  		TAILQ_FOREACH(driver, &dev_driver_list, next) {
> > > -			if (driver->type != PMD_VDEV)
> > > +			/* RTE_DEVTYPE_VIRTUAL can only be a virtual or bonded
> > device*/
> > > +			if (driver->type != PMD_VDEV && driver->type !=
> > PMD_BDEV)
> > >  				continue;
> > >
> > > +			/*
> > > +			 * Bonded devices are not initialize here, we do it later in
> > > +			 * rte_eal_bonded_dev_init() after all physical devices
> > have been
> > > +			 * probed and initialized
> > > +			 */
> > > +			if (driver->type == PMD_BDEV &&
> > > +					!strncmp(driver->name, devargs-
> > >virtual.drv_name,
> > > +							strlen(driver->name))) {
> > > +				bdev = 1;
> > > +				break;
> > > +			}
> > > +
> > I really don't think you need to add a new device type for bonded devs.  Its got
> > no specific hardware that it drives, and you configure it with a --vdev command,
> > so treat it as one here.  I understand that you need to pass additional
> > information about slaves to a bonded device, which is fine, but you can do that
> > with kvargs pretty easily, at which point its just another vdev. The only other
> > requirement is that you initilize the bonded vdev after the slave vdevs have
> > been created, which you can do by any of several methods (a priority field to
> > indicate that bonded drivers should be initilized last/later, a deferral return
> > code from the init routine, or by dead reckoning via the careful construction of
> > the application command line (placed the bonded --vdev option last on the
> > command line argument list at run time).
> > 
> It was my initial intent to do as you have describe above, but the physical devices
> cause a real issue here, physical devices don't call through to rte_eth_dev_allocate until 
> during rte_eal_pci_probe call, so it's not possible to initialize the bonded device from
> within rte_eal_dev_init as the physical devices have not been fully initialized at this
> point, as a port_id has not been allocated and can't be added as bonding slaves. I don't 
> see away around this without changing the EAL API, which I've tried to avoid with this solution.
> 
> Ordering isn't an issue, and can easily be solved if the above problem didn't exist, and although
> a new device type isn't technically required, I think it's a cleaner solution than doing
> string comparisons.
> 


Ugh, you're right. That stinks.  We still shouldn't just drop the init code in
with the pci probe code though, especially if dpdk starts supporting more
stacked network devices. A notification system for interface creation that you
could monitor for pci_device addresses (B:D:F tupples) would be nice, but might
be overkill since dpdk doesn't really support device freeing yet.  what might be
useful is a generic ordering mechanism for rte_eal_dev_init.  If you added a
priority field to the rte_driver structure, you could create an enumeration for
the field to define when the init routine for that driver was called something
like:
typdef enum {
	PMD_INIT_FIRST = 0,
	PMD_INIT_POST_PROBE = 1,
}

Then you could make multiple calls to rte_eal_dev_init from rte_eal_init at the
appropriate places in the initalization process, each time passing one of the
above values.  rte_eal_dev_init would only initalize those drivers whos priority
matched the passed in priority.  That would be helpful for interfaces like
vlans, vxlans, tunnels, etc.


> > >
> > > +static int
> > > +rte_eth_dev_name_unique(const char* name)
> > > +{
> > > +	unsigned i;
> > > +
> > > +	for (i = 0; i < nb_ports; i++) {
> > > +		if (strcmp(rte_eth_devices[i].data->name, name) == 0)
> > > +			return -1;
> > > +	}
> > > +
> > > +	return 0;
> > > +}
> > > +
> > >  struct rte_eth_dev *
> > > -rte_eth_dev_allocate(void)
> > > +rte_eth_dev_allocate(const char* name)
> > >  {
> > >  	struct rte_eth_dev *eth_dev;
> > >
> > > @@ -165,23 +179,37 @@ rte_eth_dev_allocate(void)
> > >  	if (rte_eth_dev_data == NULL)
> > >  		rte_eth_dev_data_alloc();
> > >
> > > +	if (rte_eth_dev_name_unique(name)) {
> > > +		PMD_DEBUG_TRACE("Ethernet Device with name %s already
> > allocated!\n");
> > > +		return NULL;
> > > +	}
> > > +
> > This seems fairly racy if you allow dynamic device creation at run time from the
> > application, if multiple threads attempt to create bonds in parallel.
> 
> True but if this is an issue then there probably should be some locking around this allocation anyway.
Agreed the allocation of the port id is also racy.

> > 
> > >  /**
> > > diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c
> > b/lib/librte_pmd_pcap/rte_eth_pcap.c
> > Hmm, we're modifying other pmds for the naming feature, I think it would be best
> > split out into a separate patch.  Something entitled "support unique interface
> > naming for virtual pmds" or something.
> 
> True, but if I do that, I will need to break support for virtual devices in this patch set, and
> I would prefer to keep this functionality working initial release of link bonding. Also another
> patchset will be required at some point in the near future to tidy up/rationalize the current 
> handling of virtual devices identification and these changes do not change functionally how
> these devices work.

I'm not sure I'm following what you're saying here.  If you split this into
another patch set, and order earlier in this series, then you won't break
anything.  I'm not suggesting a functional change here, just a organizational
one, so that this change doesn't do more than the changelog indicates.

Neil

  reply	other threads:[~2014-06-13 19:38 UTC|newest]

Thread overview: 127+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-05-28 15:32 [dpdk-dev] [PATCH 0/4] Link Bonding Library declan.doherty
2014-05-28 15:32 ` [dpdk-dev] [PATCH 1/4] " declan.doherty
2014-05-28 16:54   ` Shaw, Jeffrey B
2014-05-29 13:32     ` Doherty, Declan
2014-05-28 15:32 ` [dpdk-dev] [PATCH 2/4] Link bonding unit tests declan.doherty
2014-05-28 15:32 ` [dpdk-dev] [PATCH 3/4] Link bonding integration into testpmd declan.doherty
2014-05-28 15:32 ` [dpdk-dev] [PATCH 4/4] Add Link Bonding Library to Doxygen declan.doherty
2014-05-28 17:49 ` [dpdk-dev] [PATCH 0/4] Link Bonding Library Neil Horman
2014-05-29 10:33   ` Doherty, Declan
2014-05-29 11:33     ` Neil Horman
2014-05-29  3:23 ` Cao, Waterman
2014-05-29 10:35   ` Doherty, Declan
2014-06-04 15:18 ` [dpdk-dev] [PATCH v2 " declan.doherty
2014-06-04 15:18   ` [dpdk-dev] [PATCH v2 1/4] " declan.doherty
2014-06-04 15:18     ` declan.doherty
2014-06-05 15:15     ` Stephen Hemminger
2014-06-06  9:07       ` Doherty, Declan
2014-06-06 15:13         ` Stephen Hemminger
2014-06-09 21:11     ` Eric Kinzie
2014-06-13 14:03       ` Doherty, Declan
2014-06-04 15:18   ` [dpdk-dev] [PATCH v2 2/4] Link bonding unit tests, including: - code to generate packet bursts for testing rx and tx functionality of bonded device - virtual/stubbed out ethdev for use as slave ethdev in testing - checkpack fixes declan.doherty
2014-06-04 15:18     ` declan.doherty
2014-06-04 15:18   ` [dpdk-dev] [PATCH v2 0/4] Link Bonding Library declan.doherty
2014-06-04 15:18   ` [dpdk-dev] [PATCH v2 3/4] Adding link bonding support to testpmd. - Includes the ability to create new bonded devices. - Add /remove bonding slave devices. - Interogate bonded device stats/configuration - Change bonding modes and select balance transmit polices declan.doherty
2014-06-04 15:18   ` [dpdk-dev] [PATCH v2 4/4] Add Link Bonding Library to Doxygen declan.doherty
2014-06-04 16:10   ` [dpdk-dev] [PATCH v2 0/4] Link Bonding Library Doherty, Declan
2014-06-05  8:03   ` De Lara Guarch, Pablo
2014-06-05 11:03   ` Neil Horman
2014-06-06  8:23     ` Doherty, Declan
2014-06-06 14:54       ` Neil Horman
2014-06-13 14:56         ` Doherty, Declan
2014-06-13 15:11           ` Neil Horman
2014-06-06  3:26   ` Cao, Waterman
2014-06-11 16:33   ` Thomas Monjalon
2014-06-13 14:08     ` Doherty, Declan
2014-06-13 15:15       ` Thomas Monjalon
2014-06-13 14:41 ` [dpdk-dev] [PATCH v3 0/5] Link Bonding PMD Library Declan Doherty
2014-06-13 14:41   ` [dpdk-dev] [PATCH v3 1/5] " Declan Doherty
2014-06-13 14:41   ` [dpdk-dev] [PATCH v3 2/5] Link Bonding PMD Library (librte_eal/librte_ether link bonding support changes) Declan Doherty
2014-06-13 16:08     ` Neil Horman
2014-06-13 18:34       ` Doherty, Declan
2014-06-13 19:38         ` Neil Horman [this message]
2014-06-16  8:59           ` Doherty, Declan
2014-06-16 11:07             ` Neil Horman
2014-06-16 16:17               ` Richardson, Bruce
2014-06-16 17:47                 ` Neil Horman
2014-06-16 18:07                   ` Richardson, Bruce
2014-06-16 18:09                   ` Thomas Monjalon
2014-06-13 21:59     ` Stephen Hemminger
2014-06-16  7:59       ` Doherty, Declan
2014-06-13 14:42   ` [dpdk-dev] [PATCH v3 3/5] Link Bonding PMD Library (Unit Test Suite) Declan Doherty
2014-06-13 14:42   ` [dpdk-dev] [PATCH v3 4/5] Link Bonding PMD Library (testpmd link bonding API support) Declan Doherty
2014-06-13 14:42   ` [dpdk-dev] [PATCH v3 5/5] Link Bonding PMD Library (Doxygen Additions) Declan Doherty
2014-06-13 15:20   ` [dpdk-dev] [PATCH v3 0/5] Link Bonding PMD Library Neil Horman
2014-06-16 11:18   ` [dpdk-dev] [PATCH v4 0/6] Link Bonding Library Declan Doherty
2014-06-18 16:14     ` [dpdk-dev] [PATCH v5 " Declan Doherty
2014-06-18 16:18       ` Neil Horman
2014-06-24 14:52       ` [dpdk-dev] [PATCH v6 " Declan Doherty
2014-06-24 16:03         ` [dpdk-dev] [PATCH v7 " Declan Doherty
2014-06-25 20:07           ` [dpdk-dev] [PATCH v8 " Declan Doherty
2014-06-26 16:02             ` De Lara Guarch, Pablo
2014-06-26 23:57             ` [dpdk-dev] [PATCH v9 0/5] link bonding Thomas Monjalon
2014-06-26 23:57               ` [dpdk-dev] [PATCH v9 1/5] bond: new link bonding library Thomas Monjalon
2014-06-27  0:45                 ` Thomas Monjalon
2014-06-26 23:57               ` [dpdk-dev] [PATCH v9 2/5] ethdev: add unique name to devices Thomas Monjalon
2014-06-26 23:57               ` [dpdk-dev] [PATCH v9 3/5] eal: support link bonding device initialization Thomas Monjalon
2014-06-26 23:57               ` [dpdk-dev] [PATCH v9 4/5] bond: unit tests Thomas Monjalon
2014-06-26 23:57               ` [dpdk-dev] [PATCH v9 5/5] bond: testpmd support Thomas Monjalon
2014-06-27 10:18               ` [dpdk-dev] [PATCH v10 0/5] link bonding Declan Doherty
2014-06-27 20:58                 ` Thomas Monjalon
2014-06-29 17:49                 ` [dpdk-dev] [PATCH v11 0/5] link bonding library Declan Doherty
2014-06-30  9:21                   ` Thomas Monjalon
2014-06-30  9:28                     ` Doherty, Declan
2014-07-01 22:01                       ` Thomas Monjalon
2014-06-29 17:49                 ` [dpdk-dev] [PATCH v11 1/5] bond: new " Declan Doherty
2014-06-30  9:13                   ` Thomas Monjalon
2014-06-30 22:29                   ` Robert Sanford
2014-07-01 14:16                     ` Thomas Monjalon
2014-07-01 14:19                     ` Doherty, Declan
2014-07-01 14:26                       ` Thomas Monjalon
2014-06-29 17:49                 ` [dpdk-dev] [PATCH v11 2/5] ethdev: add unique name to devices Declan Doherty
2014-06-29 17:49                 ` [dpdk-dev] [PATCH v11 3/5] eal: support link bonding device initialization Declan Doherty
2014-06-29 17:49                 ` [dpdk-dev] [PATCH v11 4/5] bond: unit tests Declan Doherty
2014-06-30  8:56                   ` Thomas Monjalon
2014-06-29 17:49                 ` [dpdk-dev] [PATCH v11 5/5] bond: testpmd support Declan Doherty
2014-06-27 10:18               ` [dpdk-dev] [PATCH v10 1/5] bond: new link bonding library Declan Doherty
2014-06-27 10:18               ` [dpdk-dev] [PATCH v10 2/5] ethdev: add unique name to devices Declan Doherty
2014-06-27 10:18               ` [dpdk-dev] [PATCH v10 3/5] eal: support link bonding device initialization Declan Doherty
2014-06-27 10:18               ` [dpdk-dev] [PATCH v10 4/5] bond: unit tests Declan Doherty
2014-06-27 10:18               ` [dpdk-dev] [PATCH v10 5/5] bond: testpmd support Declan Doherty
2014-06-25 20:07           ` [dpdk-dev] [PATCH v8 1/6] Link Bonding Library (lib/librte_pmd_bond) Declan Doherty
2014-06-25 20:07           ` [dpdk-dev] [PATCH v8 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-25 20:07           ` [dpdk-dev] [PATCH v8 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-25 20:07           ` [dpdk-dev] [PATCH v8 4/6] Link bonding Unit Tests Declan Doherty
2014-06-25 20:07           ` [dpdk-dev] [PATCH v8 5/6] testpmd link bonding additions Declan Doherty
2014-06-25 20:07           ` [dpdk-dev] [PATCH v8 6/6] Link Bonding Library doxygen additions Declan Doherty
2014-06-24 16:03         ` [dpdk-dev] [PATCH v7 1/6] Link Bonding Library (lib/librte_pmd_bond) Declan Doherty
2014-06-24 16:03         ` [dpdk-dev] [PATCH v7 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-24 16:03         ` [dpdk-dev] [PATCH v7 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-25 13:54           ` Thomas Monjalon
2014-06-25 14:41             ` Doherty, Declan
2014-06-25 16:00               ` Thomas Monjalon
2014-06-25 16:15                 ` Richardson, Bruce
2014-06-24 16:03         ` [dpdk-dev] [PATCH v7 4/6] Link bonding Unit Tests Declan Doherty
2014-06-24 16:03         ` [dpdk-dev] [PATCH v7 5/6] testpmd link bonding additions Declan Doherty
2014-06-24 16:03         ` [dpdk-dev] [PATCH v7 6/6] Link Bonding Library doxygen additions Declan Doherty
2014-06-25 13:43           ` Thomas Monjalon
2014-06-25 14:19             ` Doherty, Declan
2014-06-25 14:23               ` Thomas Monjalon
2014-06-24 14:52       ` [dpdk-dev] [PATCH v6 1/6] Link Bonding Library (lib/librte_pmd_bond) Declan Doherty
2014-06-24 14:52       ` [dpdk-dev] [PATCH v6 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-24 14:52       ` [dpdk-dev] [PATCH v6 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-24 14:52       ` [dpdk-dev] [PATCH v6 4/6] Link bonding Unit Tests Declan Doherty
2014-06-24 14:52       ` [dpdk-dev] [PATCH v6 5/6] testpmd link bonding additions Declan Doherty
2014-06-24 14:52       ` [dpdk-dev] [PATCH v6 6/6] Link Bonding Library doxygen additions Declan Doherty
2014-06-18 16:14     ` [dpdk-dev] [PATCH v5 1/6] Link Bonding Library (lib/librte_pmd_bond) Declan Doherty
2014-06-18 16:14     ` [dpdk-dev] [PATCH v5 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-18 16:14     ` [dpdk-dev] [PATCH v5 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-18 16:14     ` [dpdk-dev] [PATCH v5 4/6] Link bonding Unit Tests Declan Doherty
2014-06-18 16:14     ` [dpdk-dev] [PATCH v5 5/6] testpmd link bonding additions Declan Doherty
2014-06-18 16:14     ` [dpdk-dev] [PATCH v5 6/6] Link Bonding Library doxygen additions Declan Doherty
2014-06-16 11:18   ` [dpdk-dev] [PATCH v4 1/6] Link Bonding Library (lib/librte_pmd_bond) initial release with support for Mode 0 - Round Robin Mode 1 - Active Backup Mode 2 - Balance -> Supports 3 transmit polices (layer 2, layer 2+3, la Mode 3 - Broadcast Declan Doherty
2014-06-16 11:18   ` [dpdk-dev] [PATCH v4 2/6] Support for unique interface naming of pmds Declan Doherty
2014-06-16 11:18   ` [dpdk-dev] [PATCH v4 3/6] EAL support for link bonding device initialization Declan Doherty
2014-06-16 11:18   ` [dpdk-dev] [PATCH v4 4/6] Link bonding Unit Tests Declan Doherty
2014-06-16 11:18   ` [dpdk-dev] [PATCH v4 5/6] testpmd link bonding additions Declan Doherty
2014-06-16 11:18   ` [dpdk-dev] [PATCH v4 6/6] Link Bonding Library doxygen additions Declan Doherty

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20140613193815.GE22451@hmsreliant.think-freely.org \
    --to=nhorman@tuxdriver.com \
    --cc=declan.doherty@intel.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git