From: Matan Azrad <matan@mellanox.com>
To: "Matan Azrad" <matan@mellanox.com>,
"Gaëtan Rivet" <gaetan.rivet@6wind.com>,
"Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
"Thomas Monjalon" <thomas@monjalon.net>
Cc: "Wu, Jingjing" <jingjing.wu@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
Neil Horman <nhorman@tuxdriver.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>
Subject: Re: [dpdk-dev] [PATCH v3 7/7] app/testpmd: adjust ethdev port ownership
Date: Thu, 25 Jan 2018 09:36:39 +0000 [thread overview]
Message-ID: <AM6PR0502MB37974A3305A09994C1CAB06DD2E10@AM6PR0502MB3797.eurprd05.prod.outlook.com> (raw)
In-Reply-To: <AM6PR0502MB37979FF971BEAE971C4714D6D2E30@AM6PR0502MB3797.eurprd05.prod.outlook.com>
Gaetan, Konstantin, Thomas
Any response to my suggestion below?
From: Matan Azrad
> Hi
>
> From: Gaëtan Rivet [mailto:gaetan.rivet@6wind.com] <snip>
> > > > > > > Look,
> > > > > > > > Testpmd initiates some of its internal databases depends
> > > > > > > > on specific port iteration, In some time someone may take
> > > > > > > > ownership of Testpmd ports and testpmd will continue to
> > > > > > > > touch
> > them.
> > > > > >
> > > > > > But if someone will take the ownership (assign new owner_id)
> > > > > > that port will not appear in RTE_ETH_FOREACH_DEV() any more.
> > > > > >
> > > > >
> > > > > Yes, but testpmd sometimes depends on previous iteration using
> > internal database.
> > > > > So it uses internal database that was updated by old iteration.
> > > >
> > > > That sounds like just a bug in testpmd that need to be fixed, no?
> > >
> > > If Testpmd already took ownership for these ports(like I did), it is ok.
> > >
> >
> > Have you tested using the default iterator (NO_OWNER)?
> > It worked until now with the bare minimal device tagging using
> > DEV_DEFERRED. Testpmd did not seem to mind having to skip this port.
> >
> > I'm sure there were places where this was overlooked, but overall, I'd
> > think everything should be fixable using only the NO_OWNER iteration.
>
> I don't think so.
>
> > Can you point to a specific scenario (command line, chain of event)
> > that would lead to a problem?
> >
>
> I didn't construct a race test to catch testpmd issue, but I think without this
> patch, there is a lot of issues.
> Go to the testpmd code (before ownership) and find usage of the old
> iterator(after the first iteration in main), Ask yourself what should happen if
> exactly in this time, a new port is created by fail-safe(plug in event).
>
> > > > Any particular places where outdated device info is used?
> > >
> > > For example, look for the stream management in testpmd(I think I saw
> > > it
> > there).
> > >
> >
> > The stream management is certainly shaky, but it happens after the EAL
> > initial port creation, and is not able to update itself for new
> > hotplugged ports (unless something changed).
> >
>
> Yes, but conceptually someone in the future may take the port(because it
> ownerless).
>
> > > > > > > If I look back on the fail-safe, its sole purpose is to have
> > > > > > > seamless hotplug with existing applications.
> > > > > > >
> > > > > > > Port ownership is a genericization of some functions
> > > > > > > introduced by the fail-safe, that could structure DPDK
> > > > > > > further. It should allow applications to have a seamless
> > > > > > > integration with subsystems using port ownership. Without
> > > > > > > this,
> > port ownership cannot be used.
> > > > > > >
> > > > > > > Testpmd should be fixed, but follow the most common design
> > > > > > > patterns of DPDK applications. Going with port ownership
> > > > > > > seems like a paradigm shift.
> > > > > > >
> > > > > > > > In addition
> > > > > > > > Using the old iterator in some places in testpmd will
> > > > > > > > cause a race for run-
> > > > > > time new ports(can be created by failsafe or any hotplug code):
> > > > > > > > - testpmd finds an ownerless port(just now created) by the
> > > > > > > > old iterator and start traffic there,
> >
> > How does testpmd start traffic there? Testpmd has only a callback for
> > displaying that it received an event for a new port. It has no concept
> > of hotplugging beyond that.
> >
>
> Yes, so no traffic just some control command.
>
> > Testpmd will not start using any new port probed using the hotplug API
> > on its own, again, unless something has drastically changed.
> >
>
> Every iterator using in testpmd is exposed to race.
>
> > > > > > > > - failsafe takes ownership of this new port and start traffic there.
> > > > > > > > Problem!
> > > > > >
> > > > > > Could you shed a bit more light here - it would be race
> > > > > > condition between whom and whom?
> > > > >
> > > > > Sure.
> > > > >
> > > > > > As I remember in testpmd all control ops are done within one
> > > > > > thread (main lcore).
> > > > >
> > > > > But other dpdk entity can use another thread, for example:
> > > > > Failsafe uses the host thread(using alarm callback) to create a
> > > > > new port and
> > > > to take ownership of a port.
> > > >
> > > > Hm, and you create new ports inside failsafe PMD, right and then
> > > > set new owner_id for it?
> > >
> > > Yes.
> > >
> > > > And all this in alarm in interrupt thread?
> > >
> > > Yes.
> > >
> > > > If so I wonder how you can guarantee that no-one else will set
> > > > different owner_id between
> > > > rte_eth_dev_allocate() and rte_eth_dev_owner_set()?
> > >
> > > I check it (see failsafe patch to this series - V5).
> > > Function: fs_bus_init.
> > >
> > > > Could you point me to that place (I am not really familiar with
> > > > familiar with failsafe code)?
> > > >
> > > > >
> > > > > The race:
> > > > > Testpmd iterates over all ports by the master thread.
> > > > > Failsafe takes ownership of a port by the host thread and start using
> it.
> > > > > => The two dpdk entities may use the device at same time!
> > > >
> >
> > When can this happen? Fail-safe creates its initial pool of ports
> > during EAL init, before testpmd scans eth_dev ports and configure its
> streams.
> > At that point, it has taken ownership, from the master lcore context.
> >
> > After this point, new ports could be detected and hotplugged by fail-safe.
> > However, even if testpmd had a callback to capture those new ports and
> > reconfigure its streams, it would be executed from within the
> > intr-thread, same as failsafe. If the thread was interrupted, by a
> > dataplane-lcore for example, streams would not have been reconfigured.
> > The fail-safe would execute its callback and set the owner-id before
> > the callback chains goes to the application.
> >
>
> Some iterator may be invoked in plug out process by other thread in testpmd
> and causes to control command
>
> > And that would only be if testpmd had any callback for hotplugging
> > ports and reconfiguring its streams, which it hasn't, as far as I know.
> >
>
> We don't need to implement it in testpmd.
>
> > > > Ok, if failsafe really assigns its owner_id(s) to ports that are
> > > > already in use by the app, then how such scheme supposed to work
> > > > at
> > all?
> > >
> > > If the app works well (with the new rules) it already took ownership
> > > and
> > failsafe will see it and will wait until the application release it.
> > > Every dpdk entity should know which port it wants to manage, If 2
> > > entities want to manage the same device - it can be ok and port
> > > ownership
> > can synchronize the usage.
> > >
> > > Probably, application which will run fail-safe wants to manage only
> > > the fail-
> > safe port and therefor to take ownership only for it.
> > >
> > > > I.E. application has a port - it assigns some owner_id != 0 to it,
> > > > then PMD tries to set its owner_id tot the same port.
> > > > Obviously failsafe's set_owner() will always fail in such case.
> > > >
> > > Yes, and will try again after some time.
> > >
> > > > From what I hear we need to introduce a concept of 'default owner id'.
> > > > I.E. when failsafe PMD is created - user assigns some owner_id to
> > > > it
> > (default).
> > > > Then failsafe PMD generates it's own owner_id and assigns it only
> > > > to the ports whose current owner_id is equal either 0 or 'default'
> owner_id.
> > > >
> > >
> > > It is a suggestion and we need to think about it more (I'm talking
> > > about it
> > with Gaetan in another thread).
> > > Actually I think, if we want a generic solution to the generic
> > > problem the
> > current solution is ok.
> > >
> >
> > We could as well conclude this other thread there.
> >
> > The only solution would be to have a default relationship between
> > owners, something that goes beyond the scope assigned by Thomas to
> > your evolution, but would be necessary for this API to be properly
> > used by existing applications.
> >
> > I think it's the only way to have a sane default behavior with your
> > API, but I also think this goes beyong the scope of the DPDK altogether.
> >
> > But even with those considerations that could be ironed out later (API
> > is still experimental anyway), in the meantime, I think we should
> > strive not to break "userland" as much as possible. Meaning that
> > unless you have a specific situation creating a bug, you shouldn't
> > have to modify testpmd, and if an issues arises, you need to try to
> > improve your API before resorting to changing the resource management
> model of all existing applications.
> >
>
> I understand it.
> Suggestion:
>
> 2 system owners.
> APP_OWNER - 1.
> NO_OWNER - 0.
>
> And allowing for more owners as now.
>
> 1. Every port creation will set the owner for NO_OWNER (as now).
> 2. There is option for all dpdk entities to take owner of NO_OWNER ports all
> the time(as now).
> 3. In some point in the end of EAL init: set all the NO_OWNER to
> APP_OWNER(for V6).
> 4. Change the old iterator to iterate over APP_OWNER ports(for V6).
>
> What do you think?
>
> <snip>
next prev parent reply other threads:[~2018-01-25 9:36 UTC|newest]
Thread overview: 212+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-11-28 11:57 [dpdk-dev] [PATCH 0/5] ethdev: Port ownership Matan Azrad
2017-11-28 11:57 ` [dpdk-dev] [PATCH 1/5] ethdev: free a port by a dedicated API Matan Azrad
2017-11-28 11:57 ` [dpdk-dev] [PATCH 2/5] ethdev: add port ownership Matan Azrad
2017-11-30 12:36 ` Neil Horman
2017-11-30 13:24 ` Gaëtan Rivet
2017-11-30 14:30 ` Matan Azrad
2017-11-30 15:09 ` Gaëtan Rivet
2017-11-30 15:43 ` Matan Azrad
2017-12-01 12:09 ` Neil Horman
2017-12-03 8:04 ` Matan Azrad
2017-12-03 11:10 ` Ananyev, Konstantin
2017-12-03 13:46 ` Matan Azrad
2017-12-04 16:01 ` Neil Horman
2017-12-04 18:10 ` Matan Azrad
2017-12-04 22:30 ` Neil Horman
2017-12-05 6:08 ` Matan Azrad
2017-12-05 10:05 ` Bruce Richardson
2017-12-08 11:35 ` Thomas Monjalon
2017-12-08 12:31 ` Neil Horman
2017-12-21 17:06 ` Thomas Monjalon
2017-12-21 17:43 ` Neil Horman
2017-12-21 19:37 ` Matan Azrad
2017-12-21 20:14 ` Neil Horman
2017-12-21 21:57 ` Matan Azrad
2017-12-22 14:26 ` Neil Horman
2017-12-23 22:36 ` Matan Azrad
2017-12-29 16:56 ` Neil Horman
2017-12-05 19:26 ` Neil Horman
2017-12-08 11:06 ` Thomas Monjalon
2017-12-05 11:12 ` Ananyev, Konstantin
2017-12-05 11:44 ` Ananyev, Konstantin
2017-12-05 11:53 ` Thomas Monjalon
2017-12-05 14:56 ` Bruce Richardson
2017-12-05 14:57 ` Ananyev, Konstantin
2017-12-05 11:47 ` Thomas Monjalon
2017-12-05 15:13 ` Ananyev, Konstantin
2017-12-05 15:49 ` Thomas Monjalon
2017-11-28 11:57 ` [dpdk-dev] [PATCH 3/5] net/failsafe: free an eth port by a dedicated API Matan Azrad
2017-11-28 11:58 ` [dpdk-dev] [PATCH 4/5] net/failsafe: use ownership mechanism to own ports Matan Azrad
2017-11-28 11:58 ` [dpdk-dev] [PATCH 5/5] app/testpmd: adjust ethdev port ownership Matan Azrad
2018-01-07 9:45 ` [dpdk-dev] [PATCH v2 0/6] ethdev: " Matan Azrad
2018-01-07 9:45 ` [dpdk-dev] [PATCH v2 1/6] ethdev: fix port data reset timing Matan Azrad
2018-01-07 9:45 ` [dpdk-dev] [PATCH v2 2/6] ethdev: add port ownership Matan Azrad
2018-01-10 13:36 ` Ananyev, Konstantin
2018-01-10 16:58 ` Matan Azrad
2018-01-11 12:40 ` Ananyev, Konstantin
2018-01-11 14:51 ` Matan Azrad
2018-01-12 0:02 ` Ananyev, Konstantin
2018-01-12 7:24 ` Matan Azrad
2018-01-15 11:45 ` Ananyev, Konstantin
2018-01-15 13:09 ` Matan Azrad
2018-01-15 18:43 ` Ananyev, Konstantin
2018-01-16 8:04 ` Matan Azrad
2018-01-16 19:11 ` Ananyev, Konstantin
2018-01-16 20:32 ` Matan Azrad
2018-01-17 11:24 ` Ananyev, Konstantin
2018-01-17 12:05 ` Matan Azrad
2018-01-17 12:54 ` Ananyev, Konstantin
2018-01-17 13:10 ` Matan Azrad
2018-01-17 16:52 ` Ananyev, Konstantin
2018-01-17 18:02 ` Matan Azrad
2018-01-17 20:34 ` Matan Azrad
2018-01-18 14:17 ` Ananyev, Konstantin
2018-01-18 14:26 ` Matan Azrad
2018-01-18 14:41 ` Ananyev, Konstantin
2018-01-18 14:45 ` Matan Azrad
2018-01-18 14:51 ` Ananyev, Konstantin
2018-01-18 15:00 ` Matan Azrad
2018-01-17 14:00 ` Neil Horman
2018-01-17 17:01 ` Ananyev, Konstantin
2018-01-18 13:10 ` Neil Horman
2018-01-18 14:00 ` Matan Azrad
2018-01-18 16:54 ` Neil Horman
2018-01-18 17:20 ` Matan Azrad
2018-01-18 18:41 ` Neil Horman
2018-01-18 20:21 ` Matan Azrad
2018-01-19 1:41 ` Neil Horman
2018-01-19 7:14 ` Matan Azrad
2018-01-19 9:30 ` Bruce Richardson
2018-01-19 10:44 ` Matan Azrad
2018-01-19 13:30 ` Neil Horman
2018-01-19 13:57 ` Matan Azrad
2018-01-19 14:13 ` Thomas Monjalon
2018-01-19 15:27 ` Neil Horman
2018-01-19 17:17 ` Thomas Monjalon
2018-01-19 17:43 ` Neil Horman
2018-01-19 18:12 ` Thomas Monjalon
2018-01-19 19:47 ` Neil Horman
2018-01-19 20:19 ` Thomas Monjalon
2018-01-19 22:52 ` Neil Horman
2018-01-20 3:38 ` Tuxdriver
2018-01-20 12:54 ` Ananyev, Konstantin
2018-01-20 14:02 ` Thomas Monjalon
2018-01-19 12:55 ` Neil Horman
2018-01-19 13:52 ` Neil Horman
2018-01-18 16:27 ` Neil Horman
2018-01-17 17:58 ` Matan Azrad
2018-01-18 13:20 ` Neil Horman
2018-01-18 14:52 ` Matan Azrad
2018-01-19 13:57 ` Neil Horman
2018-01-19 14:07 ` Thomas Monjalon
2018-01-19 14:32 ` Neil Horman
2018-01-19 17:09 ` Thomas Monjalon
2018-01-19 17:37 ` Neil Horman
2018-01-19 18:10 ` Thomas Monjalon
2018-01-21 22:12 ` Ferruh Yigit
2018-01-07 9:45 ` [dpdk-dev] [PATCH v2 3/6] ethdev: synchronize port allocation Matan Azrad
2018-01-07 9:58 ` Matan Azrad
2018-01-07 9:45 ` [dpdk-dev] [PATCH v2 4/6] net/failsafe: free an eth port by a dedicated API Matan Azrad
2018-01-07 9:45 ` [dpdk-dev] [PATCH v2 5/6] net/failsafe: use ownership mechanism to own ports Matan Azrad
2018-01-08 10:32 ` Gaëtan Rivet
2018-01-08 11:16 ` Matan Azrad
2018-01-08 11:35 ` Gaëtan Rivet
2018-01-07 9:45 ` [dpdk-dev] [PATCH v2 6/6] app/testpmd: adjust ethdev port ownership Matan Azrad
2018-01-08 11:39 ` Gaëtan Rivet
2018-01-08 12:30 ` Matan Azrad
2018-01-08 13:30 ` Gaëtan Rivet
2018-01-08 13:55 ` Matan Azrad
2018-01-08 14:21 ` Gaëtan Rivet
2018-01-08 14:42 ` Matan Azrad
2018-01-16 5:53 ` Lu, Wenzhuo
2018-01-16 8:15 ` Matan Azrad
2018-01-17 0:46 ` Lu, Wenzhuo
2018-01-17 8:51 ` Matan Azrad
2018-01-18 0:53 ` Lu, Wenzhuo
2018-01-18 16:35 ` [dpdk-dev] [PATCH v3 0/7] Port ownership and syncronization Matan Azrad
2018-01-18 16:35 ` [dpdk-dev] [PATCH v3 1/7] ethdev: fix port data reset timing Matan Azrad
2018-01-18 17:00 ` Thomas Monjalon
2018-01-19 12:38 ` Ananyev, Konstantin
2018-03-05 11:24 ` [dpdk-dev] [dpdk-stable] " Ferruh Yigit
2018-03-05 14:52 ` Matan Azrad
2018-03-05 15:06 ` Ferruh Yigit
2018-03-05 15:12 ` Matan Azrad
2018-03-27 22:37 ` Ferruh Yigit
2018-03-28 12:07 ` Matan Azrad
2018-03-30 10:39 ` Ferruh Yigit
2018-04-19 11:07 ` Ferruh Yigit
2018-04-25 12:16 ` Matan Azrad
2018-04-25 12:30 ` Ori Kam
2018-04-25 12:54 ` Ferruh Yigit
2018-04-25 14:01 ` Matan Azrad
2018-01-18 16:35 ` [dpdk-dev] [PATCH v3 2/7] ethdev: fix used portid allocation Matan Azrad
2018-01-18 17:00 ` Thomas Monjalon
2018-01-19 12:40 ` Ananyev, Konstantin
2018-01-20 16:48 ` Matan Azrad
2018-01-20 17:26 ` Ananyev, Konstantin
2018-01-18 16:35 ` [dpdk-dev] [PATCH v3 3/7] ethdev: add port ownership Matan Azrad
2018-01-18 21:11 ` Thomas Monjalon
2018-01-19 12:41 ` Ananyev, Konstantin
2018-01-18 16:35 ` [dpdk-dev] [PATCH v3 4/7] ethdev: synchronize port allocation Matan Azrad
2018-01-18 20:43 ` Thomas Monjalon
2018-01-18 20:52 ` Matan Azrad
2018-01-18 21:17 ` Thomas Monjalon
2018-01-19 12:47 ` Ananyev, Konstantin
2018-01-18 16:35 ` [dpdk-dev] [PATCH v3 5/7] net/failsafe: free an eth port by a dedicated API Matan Azrad
2018-01-18 16:35 ` [dpdk-dev] [PATCH v3 6/7] net/failsafe: use ownership mechanism to own ports Matan Azrad
2018-01-18 16:35 ` [dpdk-dev] [PATCH v3 7/7] app/testpmd: adjust ethdev port ownership Matan Azrad
2018-01-19 12:37 ` Ananyev, Konstantin
2018-01-19 12:51 ` Matan Azrad
2018-01-19 13:08 ` Ananyev, Konstantin
2018-01-19 13:35 ` Matan Azrad
2018-01-19 15:00 ` Gaëtan Rivet
2018-01-20 18:14 ` Matan Azrad
2018-01-22 10:17 ` Gaëtan Rivet
2018-01-22 11:22 ` Matan Azrad
2018-01-22 12:28 ` Ananyev, Konstantin
2018-01-22 13:22 ` Matan Azrad
2018-01-22 20:48 ` Ananyev, Konstantin
2018-01-23 8:54 ` Matan Azrad
2018-01-23 12:56 ` Gaëtan Rivet
2018-01-23 14:30 ` Matan Azrad
2018-01-25 9:36 ` Matan Azrad [this message]
2018-01-25 10:05 ` Thomas Monjalon
2018-01-25 11:15 ` Ananyev, Konstantin
2018-01-25 11:33 ` Thomas Monjalon
2018-01-25 11:55 ` Ananyev, Konstantin
2018-01-23 13:34 ` Ananyev, Konstantin
2018-01-23 14:18 ` Thomas Monjalon
2018-01-23 15:12 ` Ananyev, Konstantin
2018-01-23 15:18 ` Ananyev, Konstantin
2018-01-23 17:33 ` Thomas Monjalon
2018-01-23 21:18 ` Ananyev, Konstantin
2018-01-24 8:10 ` Thomas Monjalon
2018-01-24 18:30 ` Ananyev, Konstantin
2018-01-25 10:55 ` Thomas Monjalon
2018-01-25 11:09 ` Ananyev, Konstantin
2018-01-25 11:27 ` Thomas Monjalon
2018-01-23 14:43 ` Matan Azrad
2018-01-20 21:24 ` [dpdk-dev] [PATCH v4 0/7] Port ownership and syncronization Matan Azrad
2018-01-20 21:24 ` [dpdk-dev] [PATCH v4 1/7] ethdev: fix port data reset timing Matan Azrad
2018-01-20 21:24 ` [dpdk-dev] [PATCH v4 2/7] ethdev: fix used portid allocation Matan Azrad
2018-01-20 21:24 ` [dpdk-dev] [PATCH v4 3/7] ethdev: add port ownership Matan Azrad
2018-01-21 20:43 ` Ferruh Yigit
2018-01-21 20:46 ` Ferruh Yigit
2018-01-20 21:24 ` [dpdk-dev] [PATCH v4 4/7] ethdev: synchronize port allocation Matan Azrad
2018-01-20 21:24 ` [dpdk-dev] [PATCH v4 5/7] net/failsafe: free an eth port by a dedicated API Matan Azrad
2018-01-20 21:24 ` [dpdk-dev] [PATCH v4 6/7] net/failsafe: use ownership mechanism to own ports Matan Azrad
2018-01-20 21:24 ` [dpdk-dev] [PATCH v4 7/7] app/testpmd: adjust ethdev port ownership Matan Azrad
2018-01-22 16:38 ` [dpdk-dev] [PATCH v5 0/7] Port ownership and synchronization Matan Azrad
2018-01-22 16:38 ` [dpdk-dev] [PATCH v5 1/7] ethdev: fix port data reset timing Matan Azrad
2018-01-22 16:38 ` [dpdk-dev] [PATCH v5 2/7] ethdev: fix used portid allocation Matan Azrad
2018-01-22 16:38 ` [dpdk-dev] [PATCH v5 3/7] ethdev: add port ownership Matan Azrad
2018-01-22 16:38 ` [dpdk-dev] [PATCH v5 4/7] ethdev: synchronize port allocation Matan Azrad
2018-01-22 16:38 ` [dpdk-dev] [PATCH v5 5/7] net/failsafe: free an eth port by a dedicated API Matan Azrad
2018-01-22 16:38 ` [dpdk-dev] [PATCH v5 6/7] net/failsafe: use ownership mechanism to own ports Matan Azrad
2018-01-22 16:38 ` [dpdk-dev] [PATCH v5 7/7] app/testpmd: adjust ethdev port ownership Matan Azrad
2018-01-25 1:47 ` Lu, Wenzhuo
2018-01-25 8:30 ` Matan Azrad
2018-01-26 0:50 ` Lu, Wenzhuo
2018-01-29 11:21 ` [dpdk-dev] [PATCH v5 0/7] Port ownership and synchronization Matan Azrad
2018-01-31 19:53 ` Thomas Monjalon
2018-01-25 14:35 ` [dpdk-dev] [PATCH v3 0/7] Port ownership and syncronization Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=AM6PR0502MB37974A3305A09994C1CAB06DD2E10@AM6PR0502MB3797.eurprd05.prod.outlook.com \
--to=matan@mellanox.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=gaetan.rivet@6wind.com \
--cc=jingjing.wu@intel.com \
--cc=konstantin.ananyev@intel.com \
--cc=nhorman@tuxdriver.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).