[dpdk-dev] [PATCH] examples/client_server_mp: check port ownership

[dpdk-dev] [PATCH] examples/client_server_mp: check port ownership

[Stephen Hemminger]

The mp_server would accept a port mask that included hidden (owned)
ports and which later caused either lost packets or failed initialization.

This fixes explictly checks for ownership when parsing the port mask.

Fixes: 5b7ba31148a8 ("ethdev: add port ownership")
Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>
---
 .../client_server_mp/mp_server/args.c         | 32 ++++++++++++-------
 1 file changed, 21 insertions(+), 11 deletions(-)

diff --git a/examples/multi_process/client_server_mp/mp_server/args.c b/examples/multi_process/client_server_mp/mp_server/args.c
index b0d8d7665c85..933f69a54296 100644
--- a/examples/multi_process/client_server_mp/mp_server/args.c
+++ b/examples/multi_process/client_server_mp/mp_server/args.c
@@ -10,6 +10,7 @@
 #include <errno.h>
 
 #include <rte_memory.h>
+#include <rte_ethdev.h>
 #include <rte_string_fns.h>
 
 #include "common.h"
@@ -45,27 +46,36 @@ parse_portmask(uint8_t max_ports, const char *portmask)
 {
 	char *end = NULL;
 	unsigned long pm;
-	uint16_t count = 0;
+	uint16_t count;
 
 	if (portmask == NULL || *portmask == '\0')
 		return -1;
 
 	/* convert parameter to a number and verify */
 	pm = strtoul(portmask, &end, 16);
-	if (end == NULL || *end != '\0' || pm == 0)
+	if (end == NULL || *end != '\0' || pm > UINT16_MAX || pm == 0)
 		return -1;
 
 	/* loop through bits of the mask and mark ports */
-	while (pm != 0){
-		if (pm & 0x01){ /* bit is set in mask, use port */
-			if (count >= max_ports)
-				printf("WARNING: requested port %u not present"
-				" - ignoring\n", (unsigned)count);
-			else
-			    ports->id[ports->num_ports++] = count;
+	for (count = 0; pm != 0; pm >>= 1, ++count) {
+		struct rte_eth_dev_owner owner;
+
+		if ((pm & 0x1) == 0)
+			continue;
+
+		if (count >= max_ports) {
+			printf("WARNING: requested port %u not present - ignoring\n",
+				count);
+		} else if (rte_eth_dev_owner_get(count, &owner) < 0) {
+			printf("ERROR: can not find port %u owner\n", count);
+			return -1;
+		} else if (owner.id != RTE_ETH_DEV_NO_OWNER) {
+			printf("ERROR: requested port %u is owned by device %s\n",
+					count, owner.name);
+			return -1;
+		} else {
+			ports->id[ports->num_ports++] = count;
 		}
-		pm = (pm >> 1);
-		count++;
 	}
 
 	return 0;
-- 
2.20.1

[dpdk-dev] [PATCH v2] examples/client_server_mp: check port ownership

[Stephen Hemminger]

From: Stephen Hemminger <sthemmin@microsoft.com>

The mp_server would accept a port mask that included hidden (owned)
ports and which later caused either lost packets or failed initialization.

This fixes explicitly checks for ownership when parsing the port mask.

Fixes: 5b7ba31148a8 ("ethdev: add port ownership")
Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>
---
v2 - fix checkpatch complains about return and else

 .../client_server_mp/mp_server/args.c         | 35 +++++++++++++------
 1 file changed, 24 insertions(+), 11 deletions(-)

diff --git a/examples/multi_process/client_server_mp/mp_server/args.c b/examples/multi_process/client_server_mp/mp_server/args.c
index b0d8d7665c85..72cb85008c39 100644
--- a/examples/multi_process/client_server_mp/mp_server/args.c
+++ b/examples/multi_process/client_server_mp/mp_server/args.c
@@ -10,6 +10,7 @@
 #include <errno.h>
 
 #include <rte_memory.h>
+#include <rte_ethdev.h>
 #include <rte_string_fns.h>
 
 #include "common.h"
@@ -45,27 +46,39 @@ parse_portmask(uint8_t max_ports, const char *portmask)
 {
 	char *end = NULL;
 	unsigned long pm;
-	uint16_t count = 0;
+	uint16_t count;
 
 	if (portmask == NULL || *portmask == '\0')
 		return -1;
 
 	/* convert parameter to a number and verify */
 	pm = strtoul(portmask, &end, 16);
-	if (end == NULL || *end != '\0' || pm == 0)
+	if (end == NULL || *end != '\0' || pm > UINT16_MAX || pm == 0)
 		return -1;
 
 	/* loop through bits of the mask and mark ports */
-	while (pm != 0){
-		if (pm & 0x01){ /* bit is set in mask, use port */
-			if (count >= max_ports)
-				printf("WARNING: requested port %u not present"
-				" - ignoring\n", (unsigned)count);
-			else
-			    ports->id[ports->num_ports++] = count;
+	for (count = 0; pm != 0; pm >>= 1, ++count) {
+		struct rte_eth_dev_owner owner;
+
+		if ((pm & 0x1) == 0)
+			continue;
+
+		if (count >= max_ports) {
+			printf("WARNING: requested port %u not present - ignoring\n",
+				count);
+			continue;
+		}
+		if (rte_eth_dev_owner_get(count, &owner) < 0) {
+			printf("ERROR: can not find port %u owner\n", count);
+			return -1;
 		}
-		pm = (pm >> 1);
-		count++;
+		if (owner.id != RTE_ETH_DEV_NO_OWNER) {
+			printf("ERROR: requested port %u is owned by device %s\n",
+					count, owner.name);
+			return -1;
+		}
+
+		ports->id[ports->num_ports++] = count;
 	}
 
 	return 0;
-- 
2.20.1

Re: [dpdk-dev] [PATCH v2] examples/client_server_mp: check port ownership

[Thomas Monjalon]

We need a review here.

04/06/2019 00:28, Stephen Hemminger:
> From: Stephen Hemminger <sthemmin@microsoft.com>
> 
> The mp_server would accept a port mask that included hidden (owned)
> ports and which later caused either lost packets or failed initialization.
> 
> This fixes explicitly checks for ownership when parsing the port mask.
> 
> Fixes: 5b7ba31148a8 ("ethdev: add port ownership")
> Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>

Re: [dpdk-dev] [PATCH v2] examples/client_server_mp: check port ownership

[Matan Azrad]

Hi Stephen

From: Stephen Hemminger
> From: Stephen Hemminger <sthemmin@microsoft.com>
> 
> The mp_server would accept a port mask that included hidden (owned)
> ports and which later caused either lost packets or failed initialization.
> 
> This fixes explicitly checks for ownership when parsing the port mask.
> 
> Fixes: 5b7ba31148a8 ("ethdev: add port ownership")
> Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>
> ---
> v2 - fix checkpatch complains about return and else
> 
>  .../client_server_mp/mp_server/args.c         | 35 +++++++++++++------
>  1 file changed, 24 insertions(+), 11 deletions(-)
> 
> diff --git a/examples/multi_process/client_server_mp/mp_server/args.c
> b/examples/multi_process/client_server_mp/mp_server/args.c
> index b0d8d7665c85..72cb85008c39 100644
> --- a/examples/multi_process/client_server_mp/mp_server/args.c
> +++ b/examples/multi_process/client_server_mp/mp_server/args.c
> @@ -10,6 +10,7 @@
>  #include <errno.h>
> 
>  #include <rte_memory.h>
> +#include <rte_ethdev.h>
>  #include <rte_string_fns.h>
> 
>  #include "common.h"
> @@ -45,27 +46,39 @@ parse_portmask(uint8_t max_ports, const char
> *portmask)  {
>  	char *end = NULL;
>  	unsigned long pm;
> -	uint16_t count = 0;
> +	uint16_t count;
> 
>  	if (portmask == NULL || *portmask == '\0')
>  		return -1;
> 
>  	/* convert parameter to a number and verify */
>  	pm = strtoul(portmask, &end, 16);
> -	if (end == NULL || *end != '\0' || pm == 0)
> +	if (end == NULL || *end != '\0' || pm > UINT16_MAX || pm == 0)
>  		return -1;
> 
>  	/* loop through bits of the mask and mark ports */
> -	while (pm != 0){
> -		if (pm & 0x01){ /* bit is set in mask, use port */
> -			if (count >= max_ports)
> -				printf("WARNING: requested port %u not
> present"
> -				" - ignoring\n", (unsigned)count);
> -			else
> -			    ports->id[ports->num_ports++] = count;
> +	for (count = 0; pm != 0; pm >>= 1, ++count) {
> +		struct rte_eth_dev_owner owner;
> +
> +		if ((pm & 0x1) == 0)
> +			continue;
> +
> +		if (count >= max_ports) {
> +			printf("WARNING: requested port %u not present -
> ignoring\n",
> +				count);
> +			continue;
> +		}
> +		if (rte_eth_dev_owner_get(count, &owner) < 0) {
> +			printf("ERROR: can not find port %u owner\n",
> count);

What if some entity will take ownership later?
If you want the app will be ownership aware:
	if you sure that you want this port to be owned by this application you need to take ownership on it.
else:
the port is hidden by RTE_ETH_FOREACH_DEV if it is owned by some entity. 
see how it was done in testpmd function: port_id_is_invalid().


> +			return -1;
>  		}
> -		pm = (pm >> 1);
> -		count++;
> +		if (owner.id != RTE_ETH_DEV_NO_OWNER) {
> +			printf("ERROR: requested port %u is owned by
> device %s\n",
> +					count, owner.name);
> +			return -1;
> +		}
> +
> +		ports->id[ports->num_ports++] = count;
>  	}
> 
>  	return 0;
> --

Re: [dpdk-dev] [PATCH v2] examples/client_server_mp: check port ownership

[Stephen Hemminger]

On Sun, 7 Jul 2019 05:44:55 +0000
Matan Azrad <matan@mellanox.com> wrote:

> > +	for (count = 0; pm != 0; pm >>= 1, ++count) {
> > +		struct rte_eth_dev_owner owner;
> > +
> > +		if ((pm & 0x1) == 0)
> > +			continue;
> > +
> > +		if (count >= max_ports) {
> > +			printf("WARNING: requested port %u not present -
> > ignoring\n",
> > +				count);
> > +			continue;
> > +		}
> > +		if (rte_eth_dev_owner_get(count, &owner) < 0) {
> > +			printf("ERROR: can not find port %u owner\n",
> > count);  
> 
> What if some entity will take ownership later?
> If you want the app will be ownership aware:
> 	if you sure that you want this port to be owned by this application you need to take ownership on it.
> else:
> the port is hidden by RTE_ETH_FOREACH_DEV if it is owned by some entity. 
> see how it was done in testpmd function: port_id_is_invalid().

There are no mysterious entities in DPDK.
The only thing that can happen later is hotplug, and that will not change state
of existing port.

This model is used for all applications.  The application does not
take ownership, only device drivers do.

The whole portmask as command-line parameter is a bad user experience
now, but that is a different problem.

Re: [dpdk-dev] [PATCH v2] examples/client_server_mp: check port ownership

[Matan Azrad]

Hi Stephen

From: Stephen Hemminger 
> Sent: Sunday, July 7, 2019 7:47 PM
> To: Matan Azrad <matan@mellanox.com>
> Cc: anatoly.burakov@intel.com; dev@dpdk.org; Stephen Hemminger
> <sthemmin@microsoft.com>
> Subject: Re: [dpdk-dev] [PATCH v2] examples/client_server_mp: check port
> ownership
> 
> On Sun, 7 Jul 2019 05:44:55 +0000
> Matan Azrad <matan@mellanox.com> wrote:
> 
> > > +	for (count = 0; pm != 0; pm >>= 1, ++count) {
> > > +		struct rte_eth_dev_owner owner;
> > > +
> > > +		if ((pm & 0x1) == 0)
> > > +			continue;
> > > +
> > > +		if (count >= max_ports) {
> > > +			printf("WARNING: requested port %u not present -
> > > ignoring\n",
> > > +				count);
> > > +			continue;
> > > +		}
> > > +		if (rte_eth_dev_owner_get(count, &owner) < 0) {
> > > +			printf("ERROR: can not find port %u owner\n",
> > > count);
> >
> > What if some entity will take ownership later?
> > If you want the app will be ownership aware:
> > 	if you sure that you want this port to be owned by this application
> you need to take ownership on it.
> > else:
> > the port is hidden by RTE_ETH_FOREACH_DEV if it is owned by some entity.
> > see how it was done in testpmd function: port_id_is_invalid().
> 
> There are no mysterious entities in DPDK.
> The only thing that can happen later is hotplug, and that will not change state
> of existing port.
> This model is used for all applications.  The application does not take
> ownership, only device drivers do.

A long discussions were done on it.
There is an application model to take ownership as I wrote you above.
You chose in the second option - not to be ownership aware.
 
From docs:
"10.4.2. Port Ownership
The Ethernet devices ports can be owned by a single DPDK entity (application, library, PMD, process, etc). The ownership mechanism is controlled by ethdev APIs and allows to set/remove/get a port owner by DPDK entities. Allowing this should prevent any multiple management of Ethernet port by different entities.

Note

It is the DPDK entity responsibility to set the port owner before using it and to manage the port usage synchronization between different threads or processes."

> The whole portmask as command-line parameter is a bad user experience
> now, but that is a different problem.

I think, this is the problem you should solve here.


Matan

Re: [dpdk-dev] [PATCH v2] examples/client_server_mp: check port ownership

[Stephen Hemminger]

On Mon, 8 Jul 2019 06:37:14 +0000
Matan Azrad <matan@mellanox.com> wrote:

> Hi Stephen
> 
> From: Stephen Hemminger 
> > Sent: Sunday, July 7, 2019 7:47 PM
> > To: Matan Azrad <matan@mellanox.com>
> > Cc: anatoly.burakov@intel.com; dev@dpdk.org; Stephen Hemminger
> > <sthemmin@microsoft.com>
> > Subject: Re: [dpdk-dev] [PATCH v2] examples/client_server_mp: check port
> > ownership
> > 
> > On Sun, 7 Jul 2019 05:44:55 +0000
> > Matan Azrad <matan@mellanox.com> wrote:
> >   
> > > > +	for (count = 0; pm != 0; pm >>= 1, ++count) {
> > > > +		struct rte_eth_dev_owner owner;
> > > > +
> > > > +		if ((pm & 0x1) == 0)
> > > > +			continue;
> > > > +
> > > > +		if (count >= max_ports) {
> > > > +			printf("WARNING: requested port %u not present -
> > > > ignoring\n",
> > > > +				count);
> > > > +			continue;
> > > > +		}
> > > > +		if (rte_eth_dev_owner_get(count, &owner) < 0) {
> > > > +			printf("ERROR: can not find port %u owner\n",
> > > > count);  
> > >
> > > What if some entity will take ownership later?
> > > If you want the app will be ownership aware:
> > > 	if you sure that you want this port to be owned by this application  
> > you need to take ownership on it.  
> > > else:
> > > the port is hidden by RTE_ETH_FOREACH_DEV if it is owned by some entity.
> > > see how it was done in testpmd function: port_id_is_invalid().  
> > 
> > There are no mysterious entities in DPDK.
> > The only thing that can happen later is hotplug, and that will not change state
> > of existing port.
> > This model is used for all applications.  The application does not take
> > ownership, only device drivers do.  
> 
> A long discussions were done on it.
> There is an application model to take ownership as I wrote you above.
> You chose in the second option - not to be ownership aware.
>  
> From docs:
> "10.4.2. Port Ownership
> The Ethernet devices ports can be owned by a single DPDK entity (application, library, PMD, process, etc). The ownership mechanism is controlled by ethdev APIs and allows to set/remove/get a port owner by DPDK entities. Allowing this should prevent any multiple management of Ethernet port by different entities.
> 
> Note
> 
> It is the DPDK entity responsibility to set the port owner before using it and to manage the port usage synchronization between different threads or processes."
> 
> > The whole portmask as command-line parameter is a bad user experience
> > now, but that is a different problem.  
> 
> I think, this is the problem you should solve here.

No other application is doing this: testpmd, l3fwd, ... why do you think
this application is different. Plus you would be putting a new requirement on
all the user applications. That would be a user hostile change.

Re: [dpdk-dev] [PATCH v2] examples/client_server_mp: check port ownership

[Matan Azrad]

Hi Stephen

From: Stephen Hemminger
> On Mon, 8 Jul 2019 06:37:14 +0000
> Matan Azrad <matan@mellanox.com> wrote:
> 
> > Hi Stephen
> >
> > From: Stephen Hemminger
> > > Sent: Sunday, July 7, 2019 7:47 PM
> > > To: Matan Azrad <matan@mellanox.com>
> > > Cc: anatoly.burakov@intel.com; dev@dpdk.org; Stephen Hemminger
> > > <sthemmin@microsoft.com>
> > > Subject: Re: [dpdk-dev] [PATCH v2] examples/client_server_mp: check
> > > port ownership
> > >
> > > On Sun, 7 Jul 2019 05:44:55 +0000
> > > Matan Azrad <matan@mellanox.com> wrote:
> > >
> > > > > +	for (count = 0; pm != 0; pm >>= 1, ++count) {
> > > > > +		struct rte_eth_dev_owner owner;
> > > > > +
> > > > > +		if ((pm & 0x1) == 0)
> > > > > +			continue;
> > > > > +
> > > > > +		if (count >= max_ports) {
> > > > > +			printf("WARNING: requested port %u not
> present -
> > > > > ignoring\n",
> > > > > +				count);
> > > > > +			continue;
> > > > > +		}
> > > > > +		if (rte_eth_dev_owner_get(count, &owner) < 0) {
> > > > > +			printf("ERROR: can not find port %u
> owner\n",
> > > > > count);
> > > >
> > > > What if some entity will take ownership later?
> > > > If you want the app will be ownership aware:
> > > > 	if you sure that you want this port to be owned by this
> > > > application
> > > you need to take ownership on it.
> > > > else:
> > > > the port is hidden by RTE_ETH_FOREACH_DEV if it is owned by some
> entity.
> > > > see how it was done in testpmd function: port_id_is_invalid().
> > >
> > > There are no mysterious entities in DPDK.
> > > The only thing that can happen later is hotplug, and that will not
> > > change state of existing port.
> > > This model is used for all applications.  The application does not
> > > take ownership, only device drivers do.
> >
> > A long discussions were done on it.
> > There is an application model to take ownership as I wrote you above.
> > You chose in the second option - not to be ownership aware.
> >
> > From docs:
> > "10.4.2. Port Ownership
> > The Ethernet devices ports can be owned by a single DPDK entity
> (application, library, PMD, process, etc). The ownership mechanism is
> controlled by ethdev APIs and allows to set/remove/get a port owner by
> DPDK entities. Allowing this should prevent any multiple management of
> Ethernet port by different entities.
> >
> > Note
> >
> > It is the DPDK entity responsibility to set the port owner before using it and
> to manage the port usage synchronization between different threads or
> processes."
> >
> > > The whole portmask as command-line parameter is a bad user
> > > experience now, but that is a different problem.
> >
> > I think, this is the problem you should solve here.
> 
> No other application is doing this: testpmd, l3fwd, ... why do you think this
> application is different. Plus you would be putting a new requirement on all
> the user applications. That would be a user hostile change.

As I said above:
Apps have 2 options:
1. be ownership aware and to take ownership on its ports.
2. not be ownership aware and use by RTE_ETH_FOREACH_DEV to detect the application ports.

Most of the apps should use the second option. (see testpmd).

Matan.