[dpdk-dev] new ethdev API proposal for port ownership

[dpdk-dev] new ethdev API proposal for port ownership

[Thomas Monjalon]

Hi all,

A DPDK queue is designed to be polled by only one thread.
And the port settings must be managed by only one entity/thread.
These assumptions come from lockless design and management logic.
	- thank you Captain Obvious ;)
That's why we can say that an ethdev port is owned by one entity.
Note that this discussion can also apply to cryptodev or eventdev.

The ownership of a port is implicit in DPDK.
I propose to make it explicit for several reasons I explain below.

0/ Explicit is better than implicit

1/ It may be convenient for multi-process applications to know which
process is in charge of a port.

2/ A library could work on top of a port.

3/ A port can work on top of another port (failsafe design).
In the failsafe case, an issue has been met in testpmd.
We need to check that the user is not trying to use a port
which is already managed by failsafe:
	http://dpdk.org/ml/archives/dev/2017-September/074253.html

Now let's discuss how to implement the port ownership.

When failsafe has been introduced, the need of skipping owned ports
has been adressed with a new "deferred" state:
	http://dpdk.org/commit/cb894d99ec
	http://dpdk.org/commit/5588909af2
It allows to iterate over "free" ports but not to check one port.
Moreover this new state is weak. It does not say who owns the port.

I suggest to replace RTE_ETH_DEV_DEFERRED state by an owner string.
The owner could be printed or compared with known values.
The failsafe PMD would set "failsafe" as owner of its sub-devices.
It could also be used by applications to record which ports are used.
For instance, testpmd would set the owner as "testpmd" for polled ports.
It would bring a new information to DPDK entities: some ports are free.
And in the case of multi-process, the PID could be part of the string.

About iterating over ports with the macro RTE_ETH_FOREACH_DEV,
we should add a string parameter to compare the owner of the ports.
The empty string would be an acceptable value to find ports not owned.
Note that applications are not forced to make explicit their ownership.
We could use the NULL pointer to iterate over all existing ports if needed.

I suggest to add two new functions:
	- rte_eth_dev_owner_set(id, owner)
	- rte_eth_dev_owner_get(id)

Opinions, comments?

Re: [dpdk-dev] new ethdev API proposal for port ownership

[Adrien Mazarguil]

On Wed, Sep 06, 2017 at 12:17:48PM +0200, Thomas Monjalon wrote:
> Hi all,
> 
> A DPDK queue is designed to be polled by only one thread.
> And the port settings must be managed by only one entity/thread.
> These assumptions come from lockless design and management logic.
> 	- thank you Captain Obvious ;)
> That's why we can say that an ethdev port is owned by one entity.
> Note that this discussion can also apply to cryptodev or eventdev.
> 
> The ownership of a port is implicit in DPDK.
> I propose to make it explicit for several reasons I explain below.
> 
> 0/ Explicit is better than implicit
> 
> 1/ It may be convenient for multi-process applications to know which
> process is in charge of a port.
> 
> 2/ A library could work on top of a port.
> 
> 3/ A port can work on top of another port (failsafe design).
> In the failsafe case, an issue has been met in testpmd.
> We need to check that the user is not trying to use a port
> which is already managed by failsafe:
> 	http://dpdk.org/ml/archives/dev/2017-September/074253.html
> 
> Now let's discuss how to implement the port ownership.
> 
> When failsafe has been introduced, the need of skipping owned ports
> has been adressed with a new "deferred" state:
> 	http://dpdk.org/commit/cb894d99ec
> 	http://dpdk.org/commit/5588909af2
> It allows to iterate over "free" ports but not to check one port.
> Moreover this new state is weak. It does not say who owns the port.
> 
> I suggest to replace RTE_ETH_DEV_DEFERRED state by an owner string.
> The owner could be printed or compared with known values.
> The failsafe PMD would set "failsafe" as owner of its sub-devices.
> It could also be used by applications to record which ports are used.
> For instance, testpmd would set the owner as "testpmd" for polled ports.
> It would bring a new information to DPDK entities: some ports are free.
> And in the case of multi-process, the PID could be part of the string.
> 
> About iterating over ports with the macro RTE_ETH_FOREACH_DEV,
> we should add a string parameter to compare the owner of the ports.
> The empty string would be an acceptable value to find ports not owned.
> Note that applications are not forced to make explicit their ownership.
> We could use the NULL pointer to iterate over all existing ports if needed.
> 
> I suggest to add two new functions:
> 	- rte_eth_dev_owner_set(id, owner)
> 	- rte_eth_dev_owner_get(id)
> 
> Opinions, comments?

*raises hand*

I think the owner in the case of fail-safe (and probably others) is one
instance of the PMD, not the PMD itself, e.g. "failsafe0001". Instances need
to uniquely identify their stuff to avoid conflicts with others. This
proposal does not describe any kind of central authority to guarantee unique
owners.

Given that strings are also not super efficient for comparisons, how about
using an opaque integer value as the owner, managed by the following calls:

 rte_eth_dev_owner_new(void) /* returns a unique owner ID */
 rte_eth_dev_owner_delete(owner) /* releases it */

Another idea to avoid these two extra functions would be to make the owner
an opaque pointer to some internal data of the owner instance instead of a
string. NULL would mean "not owned", and any other value must correspond to
a local object the real owner must recognize.

In short, this would be a kind of dual-purpose private data pointer
applications could use to tag devices with.

-- 
Adrien Mazarguil
6WIND