[dpdk-dev] [RFC 0/7] support SubFunction representor

[dpdk-dev] [RFC 0/7] support SubFunction representor

[Xueming Li]

SubFunction [1] is a portion of the PCI device, a SF netdev has its own
dedicated queues(txq, rxq). A SF netdev supports eswitch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to existing
VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For flexibility, this patch also introduces new netdev capability
to indicate the capability to support new representor types.

[1]:
https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/



Xueming Li (7):
  ethdev: support sub function representor
  ethdev: support multi-host representor
  devarg: change reprsentor ID to bitmap
  ethdev: capability for new representor syntax
  kvargs: update parser for new representor syntax
  common/mlx5: update representor name parsing
  net/mlx5: support representor of sub function

 0000-cover-letter.patch                    |  44 ++++++
 config/rte_config.h                        |   1 +
 drivers/common/mlx5/linux/mlx5_common_os.c |  32 ++--
 drivers/common/mlx5/linux/mlx5_nl.c        |   2 +
 drivers/common/mlx5/mlx5_common.h          |   2 +
 drivers/net/mlx5/linux/mlx5_ethdev_os.c    |   5 +
 drivers/net/mlx5/linux/mlx5_os.c           |  69 ++++++++-
 drivers/net/mlx5/mlx5_ethdev.c             |   2 +
 lib/librte_ethdev/ethdev_private.c         | 164 ++++++++++++++-------
 lib/librte_ethdev/ethdev_private.h         |   3 -
 lib/librte_ethdev/rte_class_eth.c          |   7 +-
 lib/librte_ethdev/rte_ethdev.c             |   5 +-
 lib/librte_ethdev/rte_ethdev.h             |   2 +
 lib/librte_ethdev/rte_ethdev_driver.h      |  35 +++++
 lib/librte_kvargs/rte_kvargs.c             |  82 +++++++----
 15 files changed, 351 insertions(+), 104 deletions(-)
 create mode 100644 0000-cover-letter.patch

-- 
2.25.1

[dpdk-dev] [RFC 1/7] ethdev: support sub function representor

[Xueming Li]

To support SF representor, this patch enhances devargs syntax:

[pf#]vf#: new VF port representor/s, example: vf[0-3], pf0vf3
[pf#]sf#: new SF port representor/s, example: sf[0-3], pf1sf2

pf# is optional for SF and VF representor. If not present, representor
is detected from PCI BDF of device argument. In case of bonding, pf#
indicates owner PF device of SF or VF.

For backwards compatible, # default to vf#.

Enhances rte_eth_devargs_parse_representor_port() function to parse new
representor syntax, replace rte_eth_devargs_parse_list() function which
only parse simple list.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_ethdev/ethdev_private.c    | 115 ++++++++++++++------------
 lib/librte_ethdev/ethdev_private.h    |   3 -
 lib/librte_ethdev/rte_class_eth.c     |   7 +-
 lib/librte_ethdev/rte_ethdev.c        |   5 +-
 lib/librte_ethdev/rte_ethdev_driver.h |   9 ++
 5 files changed, 77 insertions(+), 62 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 162a502fe7..2a057c5f36 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -38,60 +38,13 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
 	return NULL;
 }
 
-int
-rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data)
-{
-	char *str_start;
-	int state;
-	int result;
-
-	if (*str != '[')
-		/* Single element, not a list */
-		return callback(str, data);
-
-	/* Sanity check, then strip the brackets */
-	str_start = &str[strlen(str) - 1];
-	if (*str_start != ']') {
-		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
-		return -EINVAL;
-	}
-	str++;
-	*str_start = '\0';
-
-	/* Process list elements */
-	state = 0;
-	while (1) {
-		if (state == 0) {
-			if (*str == '\0')
-				break;
-			if (*str != ',') {
-				str_start = str;
-				state = 1;
-			}
-		} else if (state == 1) {
-			if (*str == ',' || *str == '\0') {
-				if (str > str_start) {
-					/* Non-empty string fragment */
-					*str = '\0';
-					result = callback(str_start, data);
-					if (result < 0)
-						return result;
-				}
-				state = 0;
-			}
-		}
-		str++;
-	}
-	return 0;
-}
-
 static int
 rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	const uint16_t max_list)
 {
 	uint16_t lo, hi, val;
 	int result;
+	char *pos = str;
 
 	result = sscanf(str, "%hu-%hu", &lo, &hi);
 	if (result == 1) {
@@ -99,7 +52,7 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 			return -ENOMEM;
 		list[(*len_list)++] = lo;
 	} else if (result == 2) {
-		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
+		if (lo >= hi)
 			return -EINVAL;
 		for (val = lo; val <= hi; val++) {
 			if (*len_list >= max_list)
@@ -108,14 +61,74 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 		}
 	} else
 		return -EINVAL;
-	return 0;
+	while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
+		pos++;
+	return pos - str;
 }
 
+static int
+rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
+	const uint16_t max_list)
+{
+	char *pos = str;
+	int ret;
+
+	if (*pos == '[')
+		pos++;
+	while (1) {
+		ret = rte_eth_devargs_process_range(pos, list, len_list,
+						    max_list);
+		if (ret < 0)
+			return ret;
+		pos += ret;
+		if (*pos != ',') /* end of list */
+			break;
+		pos++;
+	}
+	if (*str == '[' && *pos != ']')
+		return -EINVAL;
+	if (*pos == ']')
+		pos++;
+	return pos - str;
+}
+
+/*
+ * representor format:
+ *   #: range or single number of VF representor - legacy
+ *   [pf#]vf#: VF port representor/s
+ *   [pf#]sf#: SF port representor/s
+ */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
+	int ret;
 
-	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
+	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
+	/* parse pf# */
+	if (str[0] == 'p' && str[1] == 'f') {
+		str += 2;
+		eth_da->type = RTE_ETH_REPRESENTOR_PF;
+		ret = rte_eth_devargs_process_list(str, eth_da->ports,
+				&eth_da->nb_ports, RTE_MAX_ETHPORTS);
+		if (ret < 0)
+			goto err;
+		str += ret;
+	}
+	/* parse vf# and sf#, number # alone implies VF */
+	if (str[0] == 'v'  && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+		str += 2;
+	} else if (str[0] == 's'  && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_SF;
+		str += 2;
+	} else {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	}
+	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+err:
+	if (ret < 0)
+		RTE_LOG(ERR, EAL, "wrong representor format: %s", str);
+	return ret < 0 ? ret : 0;
 }
diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
index 905a45c337..220ddd4408 100644
--- a/lib/librte_ethdev/ethdev_private.h
+++ b/lib/librte_ethdev/ethdev_private.h
@@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
 		const void *data);
 
 /* Parse devargs value for representor parameter. */
-typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
-int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data);
 int rte_eth_devargs_parse_representor_ports(char *str, void *data);
 
 #ifdef __cplusplus
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index 6338355e25..7da06808ea 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -66,7 +66,7 @@ eth_representor_cmp(const char *key __rte_unused,
 	int ret;
 	char *values;
 	const struct rte_eth_dev_data *data = opaque;
-	struct rte_eth_devargs representors;
+	struct rte_eth_devargs representors = { 0 };
 	uint16_t index;
 
 	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
@@ -76,10 +76,7 @@ eth_representor_cmp(const char *key __rte_unused,
 	values = strdup(value);
 	if (values == NULL)
 		return -1;
-	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_list(values,
-			rte_eth_devargs_parse_representor_ports,
-			&representors);
+	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 17ddacc78d..2ac51ac149 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5542,9 +5542,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
-			result = rte_eth_devargs_parse_list(pair->value,
-				rte_eth_devargs_parse_representor_ports,
-				eth_da);
+			result = rte_eth_devargs_parse_representor_ports(
+					pair->value, eth_da);
 			if (result < 0)
 				goto parse_cleanup;
 		}
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 0eacfd8425..edb000cbd4 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1193,6 +1193,14 @@ __rte_internal
 int
 rte_eth_switch_domain_free(uint16_t domain_id);
 
+/** Ethernet device representor type */
+enum rte_eth_representor_type {
+	RTE_ETH_REPRESENTOR_NONE, /* not a representor */
+	RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
+	RTE_ETH_REPRESENTOR_SF,   /* representor of SF */
+	RTE_ETH_REPRESENTOR_PF,   /* representor of host PF */
+};
+
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
 	uint16_t ports[RTE_MAX_ETHPORTS];
@@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
 	/** representor port/s identifier to enable on device */
 	uint16_t nb_representor_ports;
 	/** number of ports in representor port field */
+	enum rte_eth_representor_type type; /* type of representor */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [RFC 2/7] ethdev: support multi-host representor

[Xueming Li]

This patch introduce multi-host controller for ethdev representor
syntax, examples:

[[c#]pf#]vf#: VF port representor/s, example: pf0vf1
[[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]

c# is controller ID/range in case of multi-host, optional. It is mostly
for SmartNIC attached to multiple hosts in the same rack to allow
routing the packets between PF/SF/VF running on these hosts.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 config/rte_config.h                   |  1 +
 lib/librte_ethdev/ethdev_private.c    | 13 +++++++++++--
 lib/librte_ethdev/rte_ethdev_driver.h |  4 ++++
 3 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index a0b5160ff2..23d02d51ef 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -58,6 +58,7 @@
 #define RTE_MAX_QUEUES_PER_PORT 1024
 #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
 #define RTE_ETHDEV_RXTX_CALLBACKS 1
+#define RTE_MAX_MULTI_HOST_CTRLS 4
 
 /* cryptodev defines */
 #define RTE_CRYPTO_MAX_DEVS 64
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 2a057c5f36..3e455acea9 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -95,8 +95,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 /*
  * representor format:
  *   #: range or single number of VF representor - legacy
- *   [pf#]vf#: VF port representor/s
- *   [pf#]sf#: SF port representor/s
+ *   [[c#]pf#]vf#: VF port representor/s
+ *   [[c#]pf#]sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -105,6 +105,15 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	int ret;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
+	if (str[0] == 'c') {
+		str += 1;
+		ret = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
+				&eth_da->nb_mh_controllers,
+				RTE_DIM(eth_da->mh_controllers));
+		if (ret < 0)
+			goto err;
+		str += ret;
+	}
 	/* parse pf# */
 	if (str[0] == 'p' && str[1] == 'f') {
 		str += 2;
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index edb000cbd4..a7969c9408 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1203,6 +1203,10 @@ enum rte_eth_representor_type {
 
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
+	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
+	/** controller/s number in case of multi-host */
+	uint16_t nb_mh_controllers;
+	/** number of controllers in multi-host contollerss field */
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
 	uint16_t nb_ports;
-- 
2.25.1

[dpdk-dev] [RFC 3/7] devarg: change reprsentor ID to bitmap

[Xueming Li]

In eth representor comparer callback, ethdev was compared with devarg.
Since ethdev representor port didn't contain controller(host) and owner
port information, callback only compared representor port and returned
representor port on other PF port.

This patch changes representor port to bitmap encoding, expands and
updates representor port ID after parsing, when device representor ID
uses the same bitmap encoding, the eth representor comparer callback
returns correct ethdev.

Representor port ID bitmap definition:
 Representor ID bitmap:
 xxxx xxxx xxxx xxxx
 |||| |LLL LLLL LLLL vf/sf id
 |||| L 1:sf, 0:vf
 ||LL pf id
 LL controller(host) id

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 0000-cover-letter.patch               | 44 +++++++++++++++++++++++++++
 lib/librte_ethdev/ethdev_private.c    | 42 ++++++++++++++++++++++++-
 lib/librte_ethdev/rte_ethdev_driver.h | 22 ++++++++++++++
 3 files changed, 107 insertions(+), 1 deletion(-)
 create mode 100644 0000-cover-letter.patch

diff --git a/0000-cover-letter.patch b/0000-cover-letter.patch
new file mode 100644
index 0000000000..3f8ce2be72
--- /dev/null
+++ b/0000-cover-letter.patch
@@ -0,0 +1,44 @@
+From 4e1f8fc062fa6813e0b57f78ad72760601ca1d98 Mon Sep 17 00:00:00 2001
+From: Xueming Li <xuemingl@nvidia.com>
+Date: Fri, 18 Dec 2020 22:31:53 +0800
+Subject: [RFC 0/7] *** SUBJECT HERE ***
+To: Viacheslav Ovsiienko <viacheslavo@nvidia.com>,
+    Thomas Monjalon <thomas@monjalon.net>,
+    Ferruh Yigit <ferruh.yigit@intel.com>,
+    Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
+    Olivier Matz <olivier.matz@6wind.com>,
+    Matan Azrad <matan@nvidia.com>
+Cc: dev@dpdk.org,
+    xuemingl@nvidia.com,
+    Asaf Penso <asafp@nvidia.com>
+
+*** BLURB HERE ***
+
+Xueming Li (7):
+  ethdev: support sub function representor
+  ethdev: support multi-host representor
+  devarg: change reprsentor ID to bitmap
+  ethdev: capability for new representor syntax
+  kvargs: update parser for new representor syntax
+  common/mlx5: update representor name parsing
+  net/mlx5: support representor of sub function
+
+ config/rte_config.h                        |   1 +
+ drivers/common/mlx5/linux/mlx5_common_os.c |  32 ++--
+ drivers/common/mlx5/linux/mlx5_nl.c        |   2 +
+ drivers/common/mlx5/mlx5_common.h          |   2 +
+ drivers/net/mlx5/linux/mlx5_ethdev_os.c    |   5 +
+ drivers/net/mlx5/linux/mlx5_os.c           |  69 ++++++++-
+ drivers/net/mlx5/mlx5_ethdev.c             |   2 +
+ lib/librte_ethdev/ethdev_private.c         | 163 ++++++++++++++-------
+ lib/librte_ethdev/ethdev_private.h         |   3 -
+ lib/librte_ethdev/rte_class_eth.c          |   7 +-
+ lib/librte_ethdev/rte_ethdev.c             |   5 +-
+ lib/librte_ethdev/rte_ethdev.h             |   2 +
+ lib/librte_ethdev/rte_ethdev_driver.h      |  35 +++++
+ lib/librte_kvargs/rte_kvargs.c             |  82 +++++++----
+ 14 files changed, 306 insertions(+), 104 deletions(-)
+
+-- 
+2.25.1
+
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 3e455acea9..a0fc187378 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -93,16 +93,20 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 }
 
 /*
- * representor format:
+ * Parse representor ports, expand and update representor port ID.
+ * Representor format:
  *   #: range or single number of VF representor - legacy
  *   [[c#]pf#]vf#: VF port representor/s
  *   [[c#]pf#]sf#: SF port representor/s
+ *
+ * See RTE_ETH_REPR() for representor ID format.
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 	int ret;
+	uint32_t c, p, f, i = 0;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
 	if (str[0] == 'c') {
@@ -136,6 +140,42 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	}
 	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+	if (ret < 0)
+		goto err;
+
+	/* Set default values, expand and update representor ID. */
+	if (!eth_da->nb_mh_controllers) {
+		eth_da->nb_mh_controllers = 1;
+		eth_da->mh_controllers[0] = 0;
+	}
+	if (!eth_da->nb_ports) {
+		eth_da->nb_ports = 1;
+		eth_da->ports[0] = 0;
+	}
+	if (!eth_da->nb_representor_ports) {
+		eth_da->nb_representor_ports = 1;
+		eth_da->representor_ports[0] = 0;
+	}
+	for (c = 0; c < eth_da->nb_mh_controllers; ++c) {
+		for (p = 0; p < eth_da->nb_ports; ++p) {
+			for (f = 0; f < eth_da->nb_representor_ports; ++f) {
+				i = c * eth_da->nb_ports *
+					eth_da->nb_representor_ports +
+				    p * eth_da->nb_representor_ports + f;
+				if (i >= RTE_DIM(eth_da->representor_ports)) {
+					RTE_LOG(ERR, EAL, "too many representor specified: %s",
+						str);
+					return -EINVAL;
+				}
+				eth_da->representor_ports[i] = RTE_ETH_REPR(
+					eth_da->mh_controllers[c],
+					eth_da->ports[p],
+					eth_da->type == RTE_ETH_REPRESENTOR_SF,
+					eth_da->representor_ports[f]);
+			}
+		}
+	}
+	eth_da->nb_representor_ports = i + 1;
 err:
 	if (ret < 0)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s", str);
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index a7969c9408..dbad55c704 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1218,6 +1218,28 @@ struct rte_eth_devargs {
 	enum rte_eth_representor_type type; /* type of representor */
 };
 
+/**
+ * Encoding representor port ID.
+ *
+ * The compact format is used for device iterator that comparing
+ * ethdev representor ID with target devargs.
+ *
+ * xxxx xxxx xxxx xxxx
+ * |||| |LLL LLLL LLLL vf/sf id
+ * |||| L 1:sf, 0:vf
+ * ||LL pf id
+ * LL controller(host) id
+ */
+#define RTE_ETH_REPR(c, pf, sf, port) \
+	((((c) & 3) << 14) |     \
+	(((pf) & 3) << 12) |     \
+	(!!(sf) << 11) |         \
+	((port) & 0x7ff))
+/** Get 'pf' port id from representor ID */
+#define RTE_ETH_REPR_PF(repr) (((repr) >> 12) & 3)
+/** Get 'vf' or 'sf' port from representor ID */
+#define RTE_ETH_REPR_PORT(repr) ((repr) & 0x7ff)
+
 /**
  * PMD helper function to parse ethdev arguments
  *
-- 
2.25.1

[dpdk-dev] [RFC 4/7] ethdev: capability for new representor syntax

[Xueming Li]

This patch introduces new eth device capability to support SubFunction
representor device.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_ethdev/rte_ethdev.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index f5f8919186..3c087bec67 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1432,6 +1432,8 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/**< Device supports SubFunction representor. */
+#define RTE_ETH_DEV_CAPA_REPRESENTOR_SF         0x00000004
 /**@}*/
 
 /*
-- 
2.25.1

[dpdk-dev] [RFC 5/7] kvargs: update parser for new representor syntax

[Xueming Li]

This patch updates kvargs parser to allow comma in list value:
  k1=a[1,2]b[3-5]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_kvargs/rte_kvargs.c | 82 +++++++++++++++++++++++-----------
 1 file changed, 56 insertions(+), 26 deletions(-)

diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
index 285081c86c..6193109a17 100644
--- a/lib/librte_kvargs/rte_kvargs.c
+++ b/lib/librte_kvargs/rte_kvargs.c
@@ -13,15 +13,19 @@
 /*
  * Receive a string with a list of arguments following the pattern
  * key=value,key=value,... and insert them into the list.
- * strtok() is used so the params string will be copied to be modified.
+ * Params string will be copied to be modified.
+ * list "[]" and list element splitter ",", "-" is treated as value.
+ * Supported examples:
+ *   k1=v1,k2=v2
+ *   v1
+ *   k1=x[0-1]y[1,3-5,9]z
  */
 static int
 rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 {
 	unsigned i;
-	char *str;
-	char *ctx1 = NULL;
-	char *ctx2 = NULL;
+	char *str, *start;
+	int in_list = 0, end_k = 0, end_v = 0;
 
 	/* Copy the const char *params to a modifiable string
 	 * to pass to rte_strsplit
@@ -32,36 +36,62 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 
 	/* browse each key/value pair and add it in kvlist */
 	str = kvlist->str;
-	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
+	start = str; /* start of current key or value */
+	while (1) {
+		switch (*str) {
+		case '=': /* End of key. */
+			end_k = 1;
+			break;
+		case ',':
+			/* End of value, skip comma in middle of range */
+			if (!in_list)
+				end_v = 1;
+			break;
+		case '[': /* Start of list. */
+			in_list++;
+			break;
+		case ']': /* End of list.  */
+			if (in_list)
+				in_list--;
+			break;
+		case 0: /* End of string */
+			end_v = 1;
+			break;
+		default:
+			break;
+		}
+
+		if (!end_k && !end_v) {
+			/* Continue if not end of key or value. */
+			str++;
+			continue;
+		}
 
 		i = kvlist->count;
 		if (i >= RTE_KVARGS_MAX)
 			return -1;
 
-		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
-		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
-		if (kvlist->pairs[i].key == NULL ||
-		    kvlist->pairs[i].value == NULL)
-			return -1;
-
-		/* Detect list [a,b] to skip comma delimiter in list. */
-		str = kvlist->pairs[i].value;
-		if (str[0] == '[') {
-			/* Find the end of the list. */
-			while (str[strlen(str) - 1] != ']') {
-				/* Restore the comma erased by strtok_r(). */
-				if (ctx1 == NULL || ctx1[0] == '\0')
-					return -1; /* no closing bracket */
-				str[strlen(str)] = ',';
-				/* Parse until next comma. */
-				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
-				if (str == NULL)
-					return -1; /* no closing bracket */
+		if (start == str) /* Empty key  or value. */
+			start = NULL;
+
+		if (end_k) {
+			/* Key parsed. */
+			kvlist->pairs[i].key = start;
+			end_k = 0;
+		} else if (end_v) {
+			/* Allow single key or single value. */
+			if (kvlist->pairs[i].key || start) {
+				kvlist->pairs[i].value = start;
+				kvlist->count++;
 			}
+			end_v = 0;
 		}
 
-		kvlist->count++;
-		str = NULL;
+		if (!*str) /* End of string. */
+			break;
+		*str = 0;
+		str++;
+		start = str;
 	}
 
 	return 0;
-- 
2.25.1

[dpdk-dev] [RFC 6/7] common/mlx5: update representor name parsing

[Xueming Li]

This patch updates representor name parsing for SF.
In sysfs, representor name stored in from 'phys_port_name file', similar
to VF representor, switch port name of SF representor is "pf<x>sf<y>".

For netlink message, net SF type is supported.

Examples:

pf0sf1
pf0sf[0-3]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.c | 32 +++++++++++++++-------
 drivers/common/mlx5/linux/mlx5_nl.c        |  2 ++
 drivers/common/mlx5/mlx5_common.h          |  2 ++
 drivers/net/mlx5/linux/mlx5_ethdev_os.c    |  3 ++
 4 files changed, 29 insertions(+), 10 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c
index 0edd78ea6d..5cf9576921 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.c
+++ b/drivers/common/mlx5/linux/mlx5_common_os.c
@@ -97,22 +97,34 @@ void
 mlx5_translate_port_name(const char *port_name_in,
 			 struct mlx5_switch_info *port_info_out)
 {
-	char pf_c1, pf_c2, vf_c1, vf_c2, eol;
+	char ctrl = 0, pf_c1, pf_c2, vf_c1, vf_c2, eol;
 	char *end;
 	int sc_items;
 
-	/*
-	 * Check for port-name as a string of the form pf0vf0
-	 * (support kernel ver >= 5.0 or OFED ver >= 4.6).
-	 */
+	sc_items = sscanf(port_name_in, "%c%d",
+			  &ctrl, &port_info_out->ctrl_num);
+	if (sc_items == 2 && ctrl == 'c') {
+		port_name_in++; /* 'c' */
+		port_name_in += snprintf(NULL, 0, "%d",
+					  port_info_out->ctrl_num);
+	}
+	/* Check for port-name as a string of the form pf0vf0 or pf0sf0 */
 	sc_items = sscanf(port_name_in, "%c%c%d%c%c%d%c",
 			  &pf_c1, &pf_c2, &port_info_out->pf_num,
 			  &vf_c1, &vf_c2, &port_info_out->port_name, &eol);
-	if (sc_items == 6 &&
-	    pf_c1 == 'p' && pf_c2 == 'f' &&
-	    vf_c1 == 'v' && vf_c2 == 'f') {
-		port_info_out->name_type = MLX5_PHYS_PORT_NAME_TYPE_PFVF;
-		return;
+	if (sc_items == 6 && pf_c1 == 'p' && pf_c2 == 'f') {
+		if (vf_c1 == 'v' && vf_c2 == 'f') {
+			/* Kernel ver >= 5.0 or OFED ver >= 4.6 */
+			port_info_out->name_type =
+					MLX5_PHYS_PORT_NAME_TYPE_PFVF;
+			return;
+		}
+		if (vf_c1 == 's' && vf_c2 == 'f') {
+			/* Kernel ver >= 5.11 or OFED ver >= 5.1 */
+			port_info_out->name_type =
+					MLX5_PHYS_PORT_NAME_TYPE_PFSF;
+			return;
+		}
 	}
 	/*
 	 * Check for port-name as a string of the form p0
diff --git a/drivers/common/mlx5/linux/mlx5_nl.c b/drivers/common/mlx5/linux/mlx5_nl.c
index 40d8620300..3d55cc98b4 100644
--- a/drivers/common/mlx5/linux/mlx5_nl.c
+++ b/drivers/common/mlx5/linux/mlx5_nl.c
@@ -1148,6 +1148,8 @@ mlx5_nl_check_switch_info(bool num_vf_set,
 	case MLX5_PHYS_PORT_NAME_TYPE_PFHPF:
 		/* Fallthrough */
 	case MLX5_PHYS_PORT_NAME_TYPE_PFVF:
+		/* Fallthrough */
+	case MLX5_PHYS_PORT_NAME_TYPE_PFSF:
 		/* New representors naming schema. */
 		switch_info->representor = 1;
 		break;
diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h
index a484b74b9c..4c75addd08 100644
--- a/drivers/common/mlx5/mlx5_common.h
+++ b/drivers/common/mlx5/mlx5_common.h
@@ -153,6 +153,7 @@ enum mlx5_nl_phys_port_name_type {
 	MLX5_PHYS_PORT_NAME_TYPE_UPLINK, /* p0, kernel ver >= 5.0 */
 	MLX5_PHYS_PORT_NAME_TYPE_PFVF, /* pf0vf0, kernel ver >= 5.0 */
 	MLX5_PHYS_PORT_NAME_TYPE_PFHPF, /* pf0, kernel ver >= 5.7, HPF rep */
+	MLX5_PHYS_PORT_NAME_TYPE_PFSF, /* pf0sf0, kernel ver >= 5.0 */
 	MLX5_PHYS_PORT_NAME_TYPE_UNKNOWN, /* Unrecognized. */
 };
 
@@ -161,6 +162,7 @@ struct mlx5_switch_info {
 	uint32_t master:1; /**< Master device. */
 	uint32_t representor:1; /**< Representor device. */
 	enum mlx5_nl_phys_port_name_type name_type; /** < Port name type. */
+	int32_t ctrl_num; /**< Controller number (valid for c#pf#vf# format). */
 	int32_t pf_num; /**< PF number (valid for pfxvfx format only). */
 	int32_t port_name; /**< Representor port name. */
 	uint64_t switch_id; /**< Switch identifier. */
diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
index 128845cb52..d4eed11fbc 100644
--- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
@@ -1013,6 +1013,9 @@ mlx5_sysfs_check_switch_info(bool device_dir,
 		/* New representors naming schema. */
 		switch_info->representor = 1;
 		break;
+	default:
+		switch_info->master = device_dir;
+		break;
 	}
 }
 
-- 
2.25.1

[dpdk-dev] [RFC 7/7] net/mlx5: support representor of sub function

[Xueming Li]

This patch supports of SF representor. Similar to VF representor, switch
port name of SF representor in sysfs phys_port_name is "pf<x>sf<y>".

Device representor argumnt is "representors=sf[list]", list member could
be mix of instance and range. Example:
  representors=sf[0,2,4,8-12,-1]

To probe VF representor and SF representor, need to separate into 2
devices:
  -a <BDF>,representor=vf[list] -a <BDF>,representor=sf[list]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_ethdev_os.c |  2 +
 drivers/net/mlx5/linux/mlx5_os.c        | 69 ++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_ethdev.c          |  2 +
 3 files changed, 67 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
index d4eed11fbc..11c30d587c 100644
--- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
@@ -1010,6 +1010,8 @@ mlx5_sysfs_check_switch_info(bool device_dir,
 	case MLX5_PHYS_PORT_NAME_TYPE_PFHPF:
 		/* Fallthrough */
 	case MLX5_PHYS_PORT_NAME_TYPE_PFVF:
+		/* Fallthrough */
+	case MLX5_PHYS_PORT_NAME_TYPE_PFSF:
 		/* New representors naming schema. */
 		switch_info->representor = 1;
 		break;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 917d6be7b8..aeac1abb2c 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -692,20 +692,70 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 
 	/* Determine if this port representor is supposed to be spawned. */
 	if (switch_info->representor && dpdk_dev->devargs) {
-		struct rte_eth_devargs eth_da;
+		struct rte_eth_devargs eth_da = { .nb_ports = 0 };
 
 		err = rte_eth_devargs_parse(dpdk_dev->devargs->args, &eth_da);
 		if (err) {
 			rte_errno = -err;
 			DRV_LOG(ERR, "failed to process device arguments: %s",
-				strerror(rte_errno));
+				dpdk_dev->devargs->args);
 			return NULL;
 		}
+		switch (eth_da.type) {
+		case RTE_ETH_REPRESENTOR_PF:
+			if (switch_info->name_type !=
+					MLX5_PHYS_PORT_NAME_TYPE_PFHPF) {
+				rte_errno = EBUSY;
+				return NULL;
+			}
+			break;
+		case RTE_ETH_REPRESENTOR_SF:
+			if (switch_info->name_type !=
+					MLX5_PHYS_PORT_NAME_TYPE_PFSF) {
+				rte_errno = EBUSY;
+				return NULL;
+			}
+			break;
+		case RTE_ETH_REPRESENTOR_VF:
+			/* Allows HPF representor index -1 as exception. */
+			if (!(spawn->info.port_name == -1 &&
+			      switch_info->name_type ==
+					MLX5_PHYS_PORT_NAME_TYPE_PFHPF) &&
+			    switch_info->name_type !=
+					MLX5_PHYS_PORT_NAME_TYPE_PFVF) {
+				rte_errno = EBUSY;
+				return NULL;
+			}
+			break;
+		default:
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		/* Check controller ID: */
+		for (i = 0; i < eth_da.nb_mh_controllers; ++i)
+			if (eth_da.mh_controllers[i] ==
+			    (uint16_t)switch_info->ctrl_num)
+				break;
+		if (eth_da.nb_mh_controllers && i == eth_da.nb_mh_controllers) {
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		/* Check PF ID: */
+		for (i = 0; i < eth_da.nb_ports; ++i)
+			if (eth_da.ports[i] ==
+			    (uint16_t)switch_info->pf_num)
+				break;
+		if (eth_da.nb_ports && i == eth_da.nb_ports) {
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		/* Check SF/VF ID: */
 		for (i = 0; i < eth_da.nb_representor_ports; ++i)
 			if (eth_da.representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
 				break;
-		if (i == eth_da.nb_representor_ports) {
+		if (eth_da.type != RTE_ETH_REPRESENTOR_PF &&
+		    i == eth_da.nb_representor_ports) {
 			rte_errno = EBUSY;
 			return NULL;
 		}
@@ -716,8 +766,11 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 		if (!switch_info->representor)
 			strlcpy(name, dpdk_dev->name, sizeof(name));
 		else
-			snprintf(name, sizeof(name), "%s_representor_%u",
-				 dpdk_dev->name, switch_info->port_name);
+			snprintf(name, sizeof(name), "%s_representor_%s%u",
+				 dpdk_dev->name,
+				 switch_info->name_type ==
+				 MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf",
+				 switch_info->port_name);
 	} else {
 		/* Bonding device. */
 		if (!switch_info->representor)
@@ -725,9 +778,11 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				 dpdk_dev->name,
 				 mlx5_os_get_dev_device_name(spawn->phys_dev));
 		else
-			snprintf(name, sizeof(name), "%s_%s_representor_%u",
+			snprintf(name, sizeof(name), "%s_%s_representor_%s%u",
 				 dpdk_dev->name,
 				 mlx5_os_get_dev_device_name(spawn->phys_dev),
+				 switch_info->name_type ==
+				 MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf",
 				 switch_info->port_name);
 	}
 	/* check if the device is already spawned */
@@ -1966,6 +2021,8 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 				case MLX5_PHYS_PORT_NAME_TYPE_PFHPF:
 					/* Fallthrough */
 				case MLX5_PHYS_PORT_NAME_TYPE_PFVF:
+					/* Fallthrough */
+				case MLX5_PHYS_PORT_NAME_TYPE_PFSF:
 					if (list[ns].info.pf_num == bd)
 						ns++;
 					break;
diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
index 45ee7e4488..ad6aacc329 100644
--- a/drivers/net/mlx5/mlx5_ethdev.c
+++ b/drivers/net/mlx5/mlx5_ethdev.c
@@ -374,6 +374,8 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info)
 			break;
 		}
 	}
+	if (priv->master)
+		info->dev_capa = RTE_ETH_DEV_CAPA_REPRESENTOR_SF;
 	return 0;
 }
 
-- 
2.25.1

Re: [dpdk-dev] [RFC 1/7] ethdev: support sub function representor

[Andrew Rybchenko]

On 12/18/20 5:55 PM, Xueming Li wrote:
> To support SF representor, this patch enhances devargs syntax:
> 
> [pf#]vf#: new VF port representor/s, example: vf[0-3], pf0vf3
> [pf#]sf#: new SF port representor/s, example: sf[0-3], pf1sf2
> 
> pf# is optional for SF and VF representor. If not present, representor
> is detected from PCI BDF of device argument. In case of bonding, pf#
> indicates owner PF device of SF or VF.
> 
> For backwards compatible, # default to vf#.
> 
> Enhances rte_eth_devargs_parse_representor_port() function to parse new
> representor syntax, replace rte_eth_devargs_parse_list() function which
> only parse simple list.

Consider to split it into few patches:
 1. rework infrastructure/functions, introduce enum, support old syntax only
 2. add new syntax for VFs
 3. add SubFunctions support, add enum member (don't forget to
    update existing drivers to reject if SF is requested)
 4. add PFs support, add enum member (also don't forget to
    update existing drivers as well)

> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> ---
>  lib/librte_ethdev/ethdev_private.c    | 115 ++++++++++++++------------
>  lib/librte_ethdev/ethdev_private.h    |   3 -
>  lib/librte_ethdev/rte_class_eth.c     |   7 +-
>  lib/librte_ethdev/rte_ethdev.c        |   5 +-
>  lib/librte_ethdev/rte_ethdev_driver.h |   9 ++
>  5 files changed, 77 insertions(+), 62 deletions(-)
> 
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index 162a502fe7..2a057c5f36 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -38,60 +38,13 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
>  	return NULL;
>  }
>  
> -int
> -rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
> -	void *data)
> -{
> -	char *str_start;
> -	int state;
> -	int result;
> -
> -	if (*str != '[')
> -		/* Single element, not a list */
> -		return callback(str, data);
> -
> -	/* Sanity check, then strip the brackets */
> -	str_start = &str[strlen(str) - 1];
> -	if (*str_start != ']') {
> -		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
> -		return -EINVAL;
> -	}
> -	str++;
> -	*str_start = '\0';
> -
> -	/* Process list elements */
> -	state = 0;
> -	while (1) {
> -		if (state == 0) {
> -			if (*str == '\0')
> -				break;
> -			if (*str != ',') {
> -				str_start = str;
> -				state = 1;
> -			}
> -		} else if (state == 1) {
> -			if (*str == ',' || *str == '\0') {
> -				if (str > str_start) {
> -					/* Non-empty string fragment */
> -					*str = '\0';
> -					result = callback(str_start, data);
> -					if (result < 0)
> -						return result;
> -				}
> -				state = 0;
> -			}
> -		}
> -		str++;
> -	}
> -	return 0;
> -}
> -
>  static int
>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>  	const uint16_t max_list)
>  {
>  	uint16_t lo, hi, val;
>  	int result;
> +	char *pos = str;
>  
>  	result = sscanf(str, "%hu-%hu", &lo, &hi);
>  	if (result == 1) {
> @@ -99,7 +52,7 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>  			return -ENOMEM;
>  		list[(*len_list)++] = lo;
>  	} else if (result == 2) {
> -		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
> +		if (lo >= hi)
>  			return -EINVAL;
>  		for (val = lo; val <= hi; val++) {
>  			if (*len_list >= max_list)
> @@ -108,14 +61,74 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>  		}
>  	} else
>  		return -EINVAL;
> -	return 0;
> +	while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
> +		pos++;
> +	return pos - str;
>  }
>  
> +static int
> +rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
> +	const uint16_t max_list)
> +{
> +	char *pos = str;
> +	int ret;
> +
> +	if (*pos == '[')
> +		pos++;
> +	while (1) {
> +		ret = rte_eth_devargs_process_range(pos, list, len_list,
> +						    max_list);
> +		if (ret < 0)
> +			return ret;
> +		pos += ret;
> +		if (*pos != ',') /* end of list */
> +			break;
> +		pos++;
> +	}
> +	if (*str == '[' && *pos != ']')
> +		return -EINVAL;
> +	if (*pos == ']')
> +		pos++;
> +	return pos - str;
> +}
> +
> +/*
> + * representor format:
> + *   #: range or single number of VF representor - legacy
> + *   [pf#]vf#: VF port representor/s
> + *   [pf#]sf#: SF port representor/s
> + */
>  int
>  rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  {
>  	struct rte_eth_devargs *eth_da = data;
> +	int ret;
>  
> -	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
> +	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
> +	/* parse pf# */
> +	if (str[0] == 'p' && str[1] == 'f') {
> +		str += 2;
> +		eth_da->type = RTE_ETH_REPRESENTOR_PF;

Just to be consistent with similar lines below, please,
swap these two lines.

> +		ret = rte_eth_devargs_process_list(str, eth_da->ports,
> +				&eth_da->nb_ports, RTE_MAX_ETHPORTS);

I'm not sure that it matches ports and nb_ports fields purpose.

In any case it should update existing drivers which support
representors to reject devargs with PFs specified.

> +		if (ret < 0)
> +			goto err;
> +		str += ret;
> +	}
> +	/* parse vf# and sf#, number # alone implies VF */
> +	if (str[0] == 'v'  && str[1] == 'f') {
> +		eth_da->type = RTE_ETH_REPRESENTOR_VF;
> +		str += 2;
> +	} else if (str[0] == 's'  && str[1] == 'f') {
> +		eth_da->type = RTE_ETH_REPRESENTOR_SF;
> +		str += 2;
> +	} else {
> +		eth_da->type = RTE_ETH_REPRESENTOR_VF;
> +	}
> +	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
> +err:
> +	if (ret < 0)
> +		RTE_LOG(ERR, EAL, "wrong representor format: %s", str);

Missing \n

> +	return ret < 0 ? ret : 0;
>  }
> diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
> index 905a45c337..220ddd4408 100644
> --- a/lib/librte_ethdev/ethdev_private.h
> +++ b/lib/librte_ethdev/ethdev_private.h
> @@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
>  		const void *data);
>  
>  /* Parse devargs value for representor parameter. */
> -typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
> -int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
> -	void *data);
>  int rte_eth_devargs_parse_representor_ports(char *str, void *data);
>  
>  #ifdef __cplusplus
> diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
> index 6338355e25..7da06808ea 100644
> --- a/lib/librte_ethdev/rte_class_eth.c
> +++ b/lib/librte_ethdev/rte_class_eth.c
> @@ -66,7 +66,7 @@ eth_representor_cmp(const char *key __rte_unused,
>  	int ret;
>  	char *values;
>  	const struct rte_eth_dev_data *data = opaque;
> -	struct rte_eth_devargs representors;
> +	struct rte_eth_devargs representors = { 0 };
>  	uint16_t index;
>  
>  	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
> @@ -76,10 +76,7 @@ eth_representor_cmp(const char *key __rte_unused,
>  	values = strdup(value);
>  	if (values == NULL)
>  		return -1;
> -	memset(&representors, 0, sizeof(representors));
> -	ret = rte_eth_devargs_parse_list(values,
> -			rte_eth_devargs_parse_representor_ports,
> -			&representors);
> +	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
>  	free(values);
>  	if (ret != 0)
>  		return -1; /* invalid devargs value */
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 17ddacc78d..2ac51ac149 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -5542,9 +5542,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>  	for (i = 0; i < args.count; i++) {
>  		pair = &args.pairs[i];
>  		if (strcmp("representor", pair->key) == 0) {
> -			result = rte_eth_devargs_parse_list(pair->value,
> -				rte_eth_devargs_parse_representor_ports,
> -				eth_da);
> +			result = rte_eth_devargs_parse_representor_ports(
> +					pair->value, eth_da);
>  			if (result < 0)
>  				goto parse_cleanup;
>  		}
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index 0eacfd8425..edb000cbd4 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -1193,6 +1193,14 @@ __rte_internal
>  int
>  rte_eth_switch_domain_free(uint16_t domain_id);
>  
> +/** Ethernet device representor type */
> +enum rte_eth_representor_type {
> +	RTE_ETH_REPRESENTOR_NONE, /* not a representor */
> +	RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
> +	RTE_ETH_REPRESENTOR_SF,   /* representor of SF */
> +	RTE_ETH_REPRESENTOR_PF,   /* representor of host PF */
> +};
> +
>  /** Generic Ethernet device arguments  */
>  struct rte_eth_devargs {
>  	uint16_t ports[RTE_MAX_ETHPORTS];
> @@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
>  	/** representor port/s identifier to enable on device */
>  	uint16_t nb_representor_ports;
>  	/** number of ports in representor port field */
> +	enum rte_eth_representor_type type; /* type of representor */
>  };
>  
>  /**
> 

Re: [dpdk-dev] [RFC 4/7] ethdev: capability for new representor syntax

[Andrew Rybchenko]

On 12/18/20 5:55 PM, Xueming Li wrote:
> This patch introduces new eth device capability to support SubFunction
> representor device.
>
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>

Motivation why the flag is required is missing.

Re: [dpdk-dev] [RFC 5/7] kvargs: update parser for new representor syntax

[Andrew Rybchenko]

On 12/18/20 5:55 PM, Xueming Li wrote:
> This patch updates kvargs parser to allow comma in list value:
>   k1=a[1,2]b[3-5]
>
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> ---
>  lib/librte_kvargs/rte_kvargs.c | 82 +++++++++++++++++++++++-----------
>  1 file changed, 56 insertions(+), 26 deletions(-)
>
> diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
> index 285081c86c..6193109a17 100644
> --- a/lib/librte_kvargs/rte_kvargs.c
> +++ b/lib/librte_kvargs/rte_kvargs.c
> @@ -13,15 +13,19 @@
>  /*
>   * Receive a string with a list of arguments following the pattern
>   * key=value,key=value,... and insert them into the list.
> - * strtok() is used so the params string will be copied to be modified.
> + * Params string will be copied to be modified.
> + * list "[]" and list element splitter ",", "-" is treated as value.
> + * Supported examples:
> + *   k1=v1,k2=v2
> + *   v1
> + *   k1=x[0-1]y[1,3-5,9]z
>   */
>  static int
>  rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
>  {
>  	unsigned i;
> -	char *str;
> -	char *ctx1 = NULL;
> -	char *ctx2 = NULL;
> +	char *str, *start;
> +	int in_list = 0, end_k = 0, end_v = 0;
>  
>  	/* Copy the const char *params to a modifiable string
>  	 * to pass to rte_strsplit
> @@ -32,36 +36,62 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
>  
>  	/* browse each key/value pair and add it in kvlist */
>  	str = kvlist->str;
> -	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
> +	start = str; /* start of current key or value */
> +	while (1) {
> +		switch (*str) {
> +		case '=': /* End of key. */
> +			end_k = 1;
> +			break;
> +		case ',':
> +			/* End of value, skip comma in middle of range */
> +			if (!in_list)
> +				end_v = 1;
> +			break;
> +		case '[': /* Start of list. */
> +			in_list++;
> +			break;
> +		case ']': /* End of list.  */
> +			if (in_list)
> +				in_list--;
> +			break;
> +		case 0: /* End of string */
> +			end_v = 1;
> +			break;
> +		default:
> +			break;
> +		}
> +
> +		if (!end_k && !end_v) {

DPDK coding style requires to compare integers vs 0 explicitly.
Otherwise you can define it as bool and use as bool this way.

> +			/* Continue if not end of key or value. */
> +			str++;
> +			continue;
> +		}
>  
>  		i = kvlist->count;
>  		if (i >= RTE_KVARGS_MAX)
>  			return -1;
>  
> -		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
> -		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
> -		if (kvlist->pairs[i].key == NULL ||
> -		    kvlist->pairs[i].value == NULL)
> -			return -1;
> -
> -		/* Detect list [a,b] to skip comma delimiter in list. */
> -		str = kvlist->pairs[i].value;
> -		if (str[0] == '[') {
> -			/* Find the end of the list. */
> -			while (str[strlen(str) - 1] != ']') {
> -				/* Restore the comma erased by strtok_r(). */
> -				if (ctx1 == NULL || ctx1[0] == '\0')
> -					return -1; /* no closing bracket */
> -				str[strlen(str)] = ',';
> -				/* Parse until next comma. */
> -				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
> -				if (str == NULL)
> -					return -1; /* no closing bracket */
> +		if (start == str) /* Empty key  or value. */
> +			start = NULL;
> +
> +		if (end_k) {

same

> +			/* Key parsed. */
> +			kvlist->pairs[i].key = start;
> +			end_k = 0;
> +		} else if (end_v) {

same

> +			/* Allow single key or single value. */
> +			if (kvlist->pairs[i].key || start) {
> +				kvlist->pairs[i].value = start;
> +				kvlist->count++;
>  			}
> +			end_v = 0;
>  		}
>  
> -		kvlist->count++;
> -		str = NULL;
> +		if (!*str) /* End of string. */

Compare with '\0'.

> +			break;
> +		*str = 0;
> +		str++;
> +		start = str;
>  	}
>  
>  	return 0;

Re: [dpdk-dev] [RFC 3/7] devarg: change reprsentor ID to bitmap

[Andrew Rybchenko]

On 12/18/20 5:55 PM, Xueming Li wrote:
> In eth representor comparer callback, ethdev was compared with devarg.

comparer -> comparator?

> Since ethdev representor port didn't contain controller(host) and owner
> port information, callback only compared representor port and returned
> representor port on other PF port.
> 
> This patch changes representor port to bitmap encoding, expands and
> updates representor port ID after parsing, when device representor ID
> uses the same bitmap encoding, the eth representor comparer callback
> returns correct ethdev.
> 
> Representor port ID bitmap definition:
>  Representor ID bitmap:
>  xxxx xxxx xxxx xxxx
>  |||| |LLL LLLL LLLL vf/sf id
>  |||| L 1:sf, 0:vf
>  ||LL pf id

Just 2 bits for PF ID is definitely not future proof.

>  LL controller(host) id

Same here.

In general, I'm not sure that such approch with bitmap makes
sense. I think we need a new API which returns information
about available functions which could be represented and
IDs there could be used as representor IDs.

> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> ---
>  0000-cover-letter.patch               | 44 +++++++++++++++++++++++++++

I guess it should not be added to the changeset.

>  lib/librte_ethdev/ethdev_private.c    | 42 ++++++++++++++++++++++++-
>  lib/librte_ethdev/rte_ethdev_driver.h | 22 ++++++++++++++
>  3 files changed, 107 insertions(+), 1 deletion(-)
>  create mode 100644 0000-cover-letter.patch
> 
> diff --git a/0000-cover-letter.patch b/0000-cover-letter.patch
> new file mode 100644
> index 0000000000..3f8ce2be72
> --- /dev/null
> +++ b/0000-cover-letter.patch
> @@ -0,0 +1,44 @@
> +From 4e1f8fc062fa6813e0b57f78ad72760601ca1d98 Mon Sep 17 00:00:00 2001
> +From: Xueming Li <xuemingl@nvidia.com>
> +Date: Fri, 18 Dec 2020 22:31:53 +0800
> +Subject: [RFC 0/7] *** SUBJECT HERE ***
> +To: Viacheslav Ovsiienko <viacheslavo@nvidia.com>,
> +    Thomas Monjalon <thomas@monjalon.net>,
> +    Ferruh Yigit <ferruh.yigit@intel.com>,
> +    Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
> +    Olivier Matz <olivier.matz@6wind.com>,
> +    Matan Azrad <matan@nvidia.com>
> +Cc: dev@dpdk.org,
> +    xuemingl@nvidia.com,
> +    Asaf Penso <asafp@nvidia.com>
> +
> +*** BLURB HERE ***
> +
> +Xueming Li (7):
> +  ethdev: support sub function representor
> +  ethdev: support multi-host representor
> +  devarg: change reprsentor ID to bitmap
> +  ethdev: capability for new representor syntax
> +  kvargs: update parser for new representor syntax
> +  common/mlx5: update representor name parsing
> +  net/mlx5: support representor of sub function
> +
> + config/rte_config.h                        |   1 +
> + drivers/common/mlx5/linux/mlx5_common_os.c |  32 ++--
> + drivers/common/mlx5/linux/mlx5_nl.c        |   2 +
> + drivers/common/mlx5/mlx5_common.h          |   2 +
> + drivers/net/mlx5/linux/mlx5_ethdev_os.c    |   5 +
> + drivers/net/mlx5/linux/mlx5_os.c           |  69 ++++++++-
> + drivers/net/mlx5/mlx5_ethdev.c             |   2 +
> + lib/librte_ethdev/ethdev_private.c         | 163 ++++++++++++++-------
> + lib/librte_ethdev/ethdev_private.h         |   3 -
> + lib/librte_ethdev/rte_class_eth.c          |   7 +-
> + lib/librte_ethdev/rte_ethdev.c             |   5 +-
> + lib/librte_ethdev/rte_ethdev.h             |   2 +
> + lib/librte_ethdev/rte_ethdev_driver.h      |  35 +++++
> + lib/librte_kvargs/rte_kvargs.c             |  82 +++++++----
> + 14 files changed, 306 insertions(+), 104 deletions(-)
> +
> +-- 
> +2.25.1
> +
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index 3e455acea9..a0fc187378 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -93,16 +93,20 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>  }
>  
>  /*
> - * representor format:
> + * Parse representor ports, expand and update representor port ID.
> + * Representor format:
>   *   #: range or single number of VF representor - legacy
>   *   [[c#]pf#]vf#: VF port representor/s
>   *   [[c#]pf#]sf#: SF port representor/s
> + *
> + * See RTE_ETH_REPR() for representor ID format.
>   */
>  int
>  rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  {
>  	struct rte_eth_devargs *eth_da = data;
>  	int ret;
> +	uint32_t c, p, f, i = 0;
>  
>  	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
>  	if (str[0] == 'c') {
> @@ -136,6 +140,42 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  	}
>  	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
> +	if (ret < 0)
> +		goto err;
> +
> +	/* Set default values, expand and update representor ID. */
> +	if (!eth_da->nb_mh_controllers) {

DPDK coding style requires to compare vs 0 expliticly.

> +		eth_da->nb_mh_controllers = 1;
> +		eth_da->mh_controllers[0] = 0;
> +	}
> +	if (!eth_da->nb_ports) {

DPDK coding style requires to compare vs 0 expliticly.

> +		eth_da->nb_ports = 1;
> +		eth_da->ports[0] = 0;
> +	}
> +	if (!eth_da->nb_representor_ports) {

DPDK coding style requires to compare vs 0 expliticly.

> +		eth_da->nb_representor_ports = 1;
> +		eth_da->representor_ports[0] = 0;
> +	}
> +	for (c = 0; c < eth_da->nb_mh_controllers; ++c) {
> +		for (p = 0; p < eth_da->nb_ports; ++p) {
> +			for (f = 0; f < eth_da->nb_representor_ports; ++f) {
> +				i = c * eth_da->nb_ports *
> +					eth_da->nb_representor_ports +
> +				    p * eth_da->nb_representor_ports + f;
> +				if (i >= RTE_DIM(eth_da->representor_ports)) {
> +					RTE_LOG(ERR, EAL, "too many representor specified: %s",

Missing \n

> +						str);
> +					return -EINVAL;
> +				}
> +				eth_da->representor_ports[i] = RTE_ETH_REPR(
> +					eth_da->mh_controllers[c],
> +					eth_da->ports[p],
> +					eth_da->type == RTE_ETH_REPRESENTOR_SF,
> +					eth_da->representor_ports[f]);
> +			}
> +		}
> +	}
> +	eth_da->nb_representor_ports = i + 1;
>  err:
>  	if (ret < 0)
>  		RTE_LOG(ERR, EAL, "wrong representor format: %s", str);
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index a7969c9408..dbad55c704 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -1218,6 +1218,28 @@ struct rte_eth_devargs {
>  	enum rte_eth_representor_type type; /* type of representor */
>  };
>  
> +/**
> + * Encoding representor port ID.
> + *
> + * The compact format is used for device iterator that comparing
> + * ethdev representor ID with target devargs.
> + *
> + * xxxx xxxx xxxx xxxx
> + * |||| |LLL LLLL LLLL vf/sf id
> + * |||| L 1:sf, 0:vf
> + * ||LL pf id
> + * LL controller(host) id
> + */
> +#define RTE_ETH_REPR(c, pf, sf, port) \
> +	((((c) & 3) << 14) |     \
> +	(((pf) & 3) << 12) |     \
> +	(!!(sf) << 11) |         \
> +	((port) & 0x7ff))
> +/** Get 'pf' port id from representor ID */
> +#define RTE_ETH_REPR_PF(repr) (((repr) >> 12) & 3)
> +/** Get 'vf' or 'sf' port from representor ID */
> +#define RTE_ETH_REPR_PORT(repr) ((repr) & 0x7ff)
> +
>  /**
>   * PMD helper function to parse ethdev arguments
>   *
> 

Re: [dpdk-dev] [RFC 2/7] ethdev: support multi-host representor

[Andrew Rybchenko]

On 12/18/20 5:55 PM, Xueming Li wrote:
> This patch introduce multi-host controller for ethdev representor
> syntax, examples:
> 
> [[c#]pf#]vf#: VF port representor/s, example: pf0vf1
> [[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]
> 
> c# is controller ID/range in case of multi-host, optional. It is mostly
> for SmartNIC attached to multiple hosts in the same rack to allow
> routing the packets between PF/SF/VF running on these hosts.

There is a big question here what is the controller ID and
how to explain it to end-user (e.g. OvS admin) which one is 0
and which one is 1, etc.

> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> ---
>  config/rte_config.h                   |  1 +
>  lib/librte_ethdev/ethdev_private.c    | 13 +++++++++++--
>  lib/librte_ethdev/rte_ethdev_driver.h |  4 ++++
>  3 files changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/config/rte_config.h b/config/rte_config.h
> index a0b5160ff2..23d02d51ef 100644
> --- a/config/rte_config.h
> +++ b/config/rte_config.h
> @@ -58,6 +58,7 @@
>  #define RTE_MAX_QUEUES_PER_PORT 1024
>  #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
>  #define RTE_ETHDEV_RXTX_CALLBACKS 1
> +#define RTE_MAX_MULTI_HOST_CTRLS 4
>  
>  /* cryptodev defines */
>  #define RTE_CRYPTO_MAX_DEVS 64
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index 2a057c5f36..3e455acea9 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -95,8 +95,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>  /*
>   * representor format:
>   *   #: range or single number of VF representor - legacy
> - *   [pf#]vf#: VF port representor/s
> - *   [pf#]sf#: SF port representor/s
> + *   [[c#]pf#]vf#: VF port representor/s
> + *   [[c#]pf#]sf#: SF port representor/s
>   */
>  int
>  rte_eth_devargs_parse_representor_ports(char *str, void *data)
> @@ -105,6 +105,15 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  	int ret;
>  
>  	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
> +	if (str[0] == 'c') {
> +		str += 1;
> +		ret = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
> +				&eth_da->nb_mh_controllers,
> +				RTE_DIM(eth_da->mh_controllers));
> +		if (ret < 0)
> +			goto err;
> +		str += ret;
> +	}
>  	/* parse pf# */
>  	if (str[0] == 'p' && str[1] == 'f') {
>  		str += 2;
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index edb000cbd4..a7969c9408 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -1203,6 +1203,10 @@ enum rte_eth_representor_type {
>  
>  /** Generic Ethernet device arguments  */
>  struct rte_eth_devargs {
> +	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
> +	/** controller/s number in case of multi-host */
> +	uint16_t nb_mh_controllers;
> +	/** number of controllers in multi-host contollerss field */

Typo: contollerss

>  	uint16_t ports[RTE_MAX_ETHPORTS];
>  	/** port/s number to enable on a multi-port single function */
>  	uint16_t nb_ports;
> 

Re: [dpdk-dev] [RFC 0/7] support SubFunction representor

[Andrew Rybchenko]

On 12/18/20 5:55 PM, Xueming Li wrote:
> SubFunction [1] is a portion of the PCI device, a SF netdev has its own
> dedicated queues(txq, rxq). A SF netdev supports eswitch representation
> offload similar to existing PF and VF representors. A SF shares PCI
> level resources with other SFs and/or with its parent PCI function.
>
> >From SmartNIC perspective, when PCI device is shared for multi-host,
> representors for host controller and host PF is required.
>
> This patch set introduces new representor types in addtion to existing
> VF representor. Syntax:
>
> [[c#]pf#]vf#: VF port representor/s from controller/pf
> [[c#]pf#]sf#: SF port representor/s from controller/pf
> #: VF representor - for backwards compatibility
>
> "#" is number instance, list or range, valid examples:
>   1, [1,3,5], [0-3], [0,2-4,6]
>
> For flexibility, this patch also introduces new netdev capability
> to indicate the capability to support new representor types.

Many thanks for sharing the patchset. Looks very interesting.
I've already sent my comments and questions for individual patches.

> [1]:
> https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/

[snip]

Re: [dpdk-dev] [RFC 0/7] support SubFunction representor

[Xueming(Steven) Li]

Hi Andrew,

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Monday, December 28, 2020 9:45 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Slava Ovsiienko
><viacheslavo@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Olivier Matz
><olivier.matz@6wind.com>; Matan Azrad <matan@nvidia.com>
>Cc: dev@dpdk.org; Asaf Penso <asafp@nvidia.com>
>Subject: Re: [RFC 0/7] support SubFunction representor
>
>On 12/18/20 5:55 PM, Xueming Li wrote:
>> SubFunction [1] is a portion of the PCI device, a SF netdev has its
>> own dedicated queues(txq, rxq). A SF netdev supports eswitch
>> representation offload similar to existing PF and VF representors. A
>> SF shares PCI level resources with other SFs and/or with its parent PCI
>function.
>>
>> >From SmartNIC perspective, when PCI device is shared for multi-host,
>> representors for host controller and host PF is required.
>>
>> This patch set introduces new representor types in addtion to existing
>> VF representor. Syntax:
>>
>> [[c#]pf#]vf#: VF port representor/s from controller/pf
>> [[c#]pf#]sf#: SF port representor/s from controller/pf
>> #: VF representor - for backwards compatibility
>>
>> "#" is number instance, list or range, valid examples:
>>   1, [1,3,5], [0-3], [0,2-4,6]
>>
>> For flexibility, this patch also introduces new netdev capability to
>> indicate the capability to support new representor types.
>
>Many thanks for sharing the patchset. Looks very interesting.
>I've already sent my comments and questions for individual patches.
Appreciate for the wonderful review, will address them next week.

>
>> [1]:
>> https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/
>
>[snip]

Re: [dpdk-dev] [RFC 0/7] support SubFunction representor

[Thomas Monjalon]

30/12/2020 09:54, Xueming(Steven) Li:
> Hi Andrew,
> 
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> >On 12/18/20 5:55 PM, Xueming Li wrote:
> >> SubFunction [1] is a portion of the PCI device, a SF netdev has its
> >> own dedicated queues(txq, rxq). A SF netdev supports eswitch
> >> representation offload similar to existing PF and VF representors. A
> >> SF shares PCI level resources with other SFs and/or with its parent PCI
> >function.
> >>
> >> >From SmartNIC perspective, when PCI device is shared for multi-host,
> >> representors for host controller and host PF is required.
> >>
> >> This patch set introduces new representor types in addtion to existing
> >> VF representor. Syntax:
> >>
> >> [[c#]pf#]vf#: VF port representor/s from controller/pf
> >> [[c#]pf#]sf#: SF port representor/s from controller/pf
> >> #: VF representor - for backwards compatibility
> >>
> >> "#" is number instance, list or range, valid examples:
> >>   1, [1,3,5], [0-3], [0,2-4,6]
> >>
> >> For flexibility, this patch also introduces new netdev capability to
> >> indicate the capability to support new representor types.
> >
> >Many thanks for sharing the patchset. Looks very interesting.
> >I've already sent my comments and questions for individual patches.
> 
> Appreciate for the wonderful review, will address them next week.

I agree, good review, thanks.
I am waiting for a v2.

Re: [dpdk-dev] [RFC 3/7] devarg: change reprsentor ID to bitmap

[Xueming(Steven) Li]

Hi Andrew,

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Monday, December 28, 2020 9:37 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Slava Ovsiienko
><viacheslavo@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Olivier Matz
><olivier.matz@6wind.com>; Matan Azrad <matan@nvidia.com>
>Cc: dev@dpdk.org; Asaf Penso <asafp@nvidia.com>
>Subject: Re: [RFC 3/7] devarg: change reprsentor ID to bitmap
>
>On 12/18/20 5:55 PM, Xueming Li wrote:
>> In eth representor comparer callback, ethdev was compared with devarg.
>
>comparer -> comparator?
>
>> Since ethdev representor port didn't contain controller(host) and
>> owner port information, callback only compared representor port and
>> returned representor port on other PF port.
>>
>> This patch changes representor port to bitmap encoding, expands and
>> updates representor port ID after parsing, when device representor ID
>> uses the same bitmap encoding, the eth representor comparer callback
>> returns correct ethdev.
>>
>> Representor port ID bitmap definition:
>>  Representor ID bitmap:
>>  xxxx xxxx xxxx xxxx
>>  |||| |LLL LLLL LLLL vf/sf id
>>  |||| L 1:sf, 0:vf
>>  ||LL pf id
>
>Just 2 bits for PF ID is definitely not future proof.

Yes, this is a valid concern, to keep ABI compatibility, need to wait next LTS to
change it to u32 or u64.

>
>>  LL controller(host) id
>
>Same here.
>
>In general, I'm not sure that such approch with bitmap makes sense. I think
>we need a new API which returns information about available functions which
>could be represented and IDs there could be used as representor IDs.

Agree, will introduce rte_eth_representor_id_encode() and rte_eth_representor_id_parse()
in next vesion.
>
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> ---
>>  0000-cover-letter.patch               | 44 +++++++++++++++++++++++++++
>
>I guess it should not be added to the changeset.
>
>>  lib/librte_ethdev/ethdev_private.c    | 42 ++++++++++++++++++++++++-
>>  lib/librte_ethdev/rte_ethdev_driver.h | 22 ++++++++++++++
>>  3 files changed, 107 insertions(+), 1 deletion(-)  create mode 100644
>> 0000-cover-letter.patch
>>
>> diff --git a/0000-cover-letter.patch b/0000-cover-letter.patch new
>> file mode 100644 index 0000000000..3f8ce2be72
>> --- /dev/null
>> +++ b/0000-cover-letter.patch
>> @@ -0,0 +1,44 @@
>> +From 4e1f8fc062fa6813e0b57f78ad72760601ca1d98 Mon Sep 17 00:00:00
>> +2001
>> +From: Xueming Li <xuemingl@nvidia.com>
>> +Date: Fri, 18 Dec 2020 22:31:53 +0800
>> +Subject: [RFC 0/7] *** SUBJECT HERE ***
>> +To: Viacheslav Ovsiienko <viacheslavo@nvidia.com>,
>> +    Thomas Monjalon <thomas@monjalon.net>,
>> +    Ferruh Yigit <ferruh.yigit@intel.com>,
>> +    Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
>> +    Olivier Matz <olivier.matz@6wind.com>,
>> +    Matan Azrad <matan@nvidia.com>
>> +Cc: dev@dpdk.org,
>> +    xuemingl@nvidia.com,
>> +    Asaf Penso <asafp@nvidia.com>
>> +
>> +*** BLURB HERE ***
>> +
>> +Xueming Li (7):
>> +  ethdev: support sub function representor
>> +  ethdev: support multi-host representor
>> +  devarg: change reprsentor ID to bitmap
>> +  ethdev: capability for new representor syntax
>> +  kvargs: update parser for new representor syntax
>> +  common/mlx5: update representor name parsing
>> +  net/mlx5: support representor of sub function
>> +
>> + config/rte_config.h                        |   1 +
>> + drivers/common/mlx5/linux/mlx5_common_os.c |  32 ++--
>> + drivers/common/mlx5/linux/mlx5_nl.c        |   2 +
>> + drivers/common/mlx5/mlx5_common.h          |   2 +
>> + drivers/net/mlx5/linux/mlx5_ethdev_os.c    |   5 +
>> + drivers/net/mlx5/linux/mlx5_os.c           |  69 ++++++++-
>> + drivers/net/mlx5/mlx5_ethdev.c             |   2 +
>> + lib/librte_ethdev/ethdev_private.c         | 163 ++++++++++++++-------
>> + lib/librte_ethdev/ethdev_private.h         |   3 -
>> + lib/librte_ethdev/rte_class_eth.c          |   7 +-
>> + lib/librte_ethdev/rte_ethdev.c             |   5 +-
>> + lib/librte_ethdev/rte_ethdev.h             |   2 +
>> + lib/librte_ethdev/rte_ethdev_driver.h      |  35 +++++
>> + lib/librte_kvargs/rte_kvargs.c             |  82 +++++++----
>> + 14 files changed, 306 insertions(+), 104 deletions(-)
>> +
>> +--
>> +2.25.1
>> +
>> diff --git a/lib/librte_ethdev/ethdev_private.c
>> b/lib/librte_ethdev/ethdev_private.c
>> index 3e455acea9..a0fc187378 100644
>> --- a/lib/librte_ethdev/ethdev_private.c
>> +++ b/lib/librte_ethdev/ethdev_private.c
>> @@ -93,16 +93,20 @@ rte_eth_devargs_process_list(char *str, uint16_t
>> *list, uint16_t *len_list,  }
>>
>>  /*
>> - * representor format:
>> + * Parse representor ports, expand and update representor port ID.
>> + * Representor format:
>>   *   #: range or single number of VF representor - legacy
>>   *   [[c#]pf#]vf#: VF port representor/s
>>   *   [[c#]pf#]sf#: SF port representor/s
>> + *
>> + * See RTE_ETH_REPR() for representor ID format.
>>   */
>>  int
>>  rte_eth_devargs_parse_representor_ports(char *str, void *data)  {
>>  	struct rte_eth_devargs *eth_da = data;
>>  	int ret;
>> +	uint32_t c, p, f, i = 0;
>>
>>  	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
>>  	if (str[0] == 'c') {
>> @@ -136,6 +140,42 @@ rte_eth_devargs_parse_representor_ports(char
>*str, void *data)
>>  	}
>>  	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
>> +	if (ret < 0)
>> +		goto err;
>> +
>> +	/* Set default values, expand and update representor ID. */
>> +	if (!eth_da->nb_mh_controllers) {
>
>DPDK coding style requires to compare vs 0 expliticly.
>
>> +		eth_da->nb_mh_controllers = 1;
>> +		eth_da->mh_controllers[0] = 0;
>> +	}
>> +	if (!eth_da->nb_ports) {
>
>DPDK coding style requires to compare vs 0 expliticly.
>
>> +		eth_da->nb_ports = 1;
>> +		eth_da->ports[0] = 0;
>> +	}
>> +	if (!eth_da->nb_representor_ports) {
>
>DPDK coding style requires to compare vs 0 expliticly.
>
>> +		eth_da->nb_representor_ports = 1;
>> +		eth_da->representor_ports[0] = 0;
>> +	}
>> +	for (c = 0; c < eth_da->nb_mh_controllers; ++c) {
>> +		for (p = 0; p < eth_da->nb_ports; ++p) {
>> +			for (f = 0; f < eth_da->nb_representor_ports; ++f) {
>> +				i = c * eth_da->nb_ports *
>> +					eth_da->nb_representor_ports +
>> +				    p * eth_da->nb_representor_ports + f;
>> +				if (i >= RTE_DIM(eth_da->representor_ports))
>{
>> +					RTE_LOG(ERR, EAL, "too many
>representor specified: %s",
>
>Missing \n
>
>> +						str);
>> +					return -EINVAL;
>> +				}
>> +				eth_da->representor_ports[i] =
>RTE_ETH_REPR(
>> +					eth_da->mh_controllers[c],
>> +					eth_da->ports[p],
>> +					eth_da->type ==
>RTE_ETH_REPRESENTOR_SF,
>> +					eth_da->representor_ports[f]);
>> +			}
>> +		}
>> +	}
>> +	eth_da->nb_representor_ports = i + 1;
>>  err:
>>  	if (ret < 0)
>>  		RTE_LOG(ERR, EAL, "wrong representor format: %s", str); diff -
>-git
>> a/lib/librte_ethdev/rte_ethdev_driver.h
>> b/lib/librte_ethdev/rte_ethdev_driver.h
>> index a7969c9408..dbad55c704 100644
>> --- a/lib/librte_ethdev/rte_ethdev_driver.h
>> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
>> @@ -1218,6 +1218,28 @@ struct rte_eth_devargs {
>>  	enum rte_eth_representor_type type; /* type of representor */  };
>>
>> +/**
>> + * Encoding representor port ID.
>> + *
>> + * The compact format is used for device iterator that comparing
>> + * ethdev representor ID with target devargs.
>> + *
>> + * xxxx xxxx xxxx xxxx
>> + * |||| |LLL LLLL LLLL vf/sf id
>> + * |||| L 1:sf, 0:vf
>> + * ||LL pf id
>> + * LL controller(host) id
>> + */
>> +#define RTE_ETH_REPR(c, pf, sf, port) \
>> +	((((c) & 3) << 14) |     \
>> +	(((pf) & 3) << 12) |     \
>> +	(!!(sf) << 11) |         \
>> +	((port) & 0x7ff))
>> +/** Get 'pf' port id from representor ID */ #define
>> +RTE_ETH_REPR_PF(repr) (((repr) >> 12) & 3)
>> +/** Get 'vf' or 'sf' port from representor ID */ #define
>> +RTE_ETH_REPR_PORT(repr) ((repr) & 0x7ff)
>> +
>>  /**
>>   * PMD helper function to parse ethdev arguments
>>   *
>>

[dpdk-dev] [PATCH v2 0/9] support SubFunction representor

[Xueming Li]

SubFunction [1] is a portion of the PCI device, a SF netdev has its own
dedicated queues(txq, rxq). A SF netdev supports eswitch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to existing
VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For backward compatibility, this patch also introduces new netdev
capability
to indicate the capability to support SF representor.

Version history:
 RFC:
 	initial version [2]
 V2:
    - separate patch for represnetor infrastructure, controller, pf and
      sf.
    - replace representor ID macro with functions:
      rte_eth_representor_id_encode()
      rte_eth_representor_id_parse()
    - new patch to allow devargs with same PCI BDF but different
      representors.
    - other minor code updates according to comments, thanks Andrew!
    - update document

[1] SubFunction in kernel:
https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/

[2] RFC:
http://patchwork.dpdk.org/project/dpdk/list/?series=14376

Xueming Li (9):
  ethdev: refactor representor infrastructure
  ethdev: support new VF representor syntax
  ethdev: support sub function representor
  ethdev: support PF index in representor
  ethdev: support multi-host representor
  devarg: change reprsentor ID to bitmap
  ethdev: capability of new representor syntax
  kvargs: update parser for new representor syntax
  eal: allow PCI device with different representors

 config/rte_config.h                      |   1 +
 doc/guides/prog_guide/poll_mode_drv.rst  |   8 +-
 drivers/net/bnxt/bnxt_ethdev.c           |  12 +++
 drivers/net/bnxt/bnxt_reps.c             |   3 +-
 drivers/net/enic/enic_ethdev.c           |   7 ++
 drivers/net/enic/enic_vf_representor.c   |   3 +-
 drivers/net/i40e/i40e_ethdev.c           |   8 ++
 drivers/net/i40e/i40e_vf_representor.c   |   3 +-
 drivers/net/ixgbe/ixgbe_ethdev.c         |   8 ++
 drivers/net/ixgbe/ixgbe_vf_representor.c |   3 +-
 drivers/net/mlx5/linux/mlx5_os.c         |  15 ++-
 lib/librte_eal/common/eal_common_bus.c   |  25 +++++
 lib/librte_ethdev/ethdev_private.c       | 128 ++++++++++++++---------
 lib/librte_ethdev/ethdev_private.h       |   3 -
 lib/librte_ethdev/rte_class_eth.c        |  40 +++++--
 lib/librte_ethdev/rte_ethdev.c           |  31 +++++-
 lib/librte_ethdev/rte_ethdev.h           |   2 +
 lib/librte_ethdev/rte_ethdev_driver.h    |  64 ++++++++++++
 lib/librte_ethdev/version.map            |   2 +
 lib/librte_kvargs/rte_kvargs.c           |  83 ++++++++++-----
 20 files changed, 348 insertions(+), 101 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH v2 1/9] ethdev: refactor representor infrastructure

[Xueming Li]

To support extended representor syntax, this patch refactor represntor
infrastructure:
1. introduces representor type enum
2. devargs representor port range extraction from partial value

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/net/bnxt/bnxt_ethdev.c        | 12 ++++
 drivers/net/enic/enic_ethdev.c        |  7 ++
 drivers/net/i40e/i40e_ethdev.c        |  8 +++
 drivers/net/ixgbe/ixgbe_ethdev.c      |  8 +++
 drivers/net/mlx5/linux/mlx5_os.c      | 11 ++++
 lib/librte_ethdev/ethdev_private.c    | 93 ++++++++++++---------------
 lib/librte_ethdev/ethdev_private.h    |  3 -
 lib/librte_ethdev/rte_class_eth.c     |  4 +-
 lib/librte_ethdev/rte_ethdev.c        |  5 +-
 lib/librte_ethdev/rte_ethdev_driver.h |  7 ++
 10 files changed, 98 insertions(+), 60 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 81c8f8d79d..844a6c3c66 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5520,6 +5520,18 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
 	int i, ret = 0;
 	struct rte_kvargs *kvlist = NULL;
 
+	if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
+			    eth_da->type);
+		return -ENOTSUP;
+	}
+	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
+			    eth_da->type);
+		return -EINVAL;
+	}
 	num_rep = eth_da->nb_representor_ports;
 	if (num_rep > BNXT_MAX_VF_REPS) {
 		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index d041a6bee9..dd085caa93 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1303,6 +1303,13 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (retval)
 			return retval;
 	}
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct enic),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index f54769c29d..05ed2e1079 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -640,6 +640,14 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 			return retval;
 	}
 
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct i40e_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 9a47a8b262..9ea0139197 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1717,6 +1717,14 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	} else
 		memset(&eth_da, 0, sizeof(eth_da));
 
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct ixgbe_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 6812a1f215..6981ba1f41 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -706,6 +706,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				strerror(rte_errno));
 			return NULL;
 		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
+			/* Representor not specified. */
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+			rte_errno = ENOTSUP;
+			DRV_LOG(ERR, "unsupported representor type: %s",
+				dpdk_dev->devargs->args);
+			return NULL;
+		}
 		for (i = 0; i < eth_da.nb_representor_ports; ++i)
 			if (eth_da.representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 162a502fe7..c219164a4a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -38,60 +38,13 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
 	return NULL;
 }
 
-int
-rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data)
-{
-	char *str_start;
-	int state;
-	int result;
-
-	if (*str != '[')
-		/* Single element, not a list */
-		return callback(str, data);
-
-	/* Sanity check, then strip the brackets */
-	str_start = &str[strlen(str) - 1];
-	if (*str_start != ']') {
-		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
-		return -EINVAL;
-	}
-	str++;
-	*str_start = '\0';
-
-	/* Process list elements */
-	state = 0;
-	while (1) {
-		if (state == 0) {
-			if (*str == '\0')
-				break;
-			if (*str != ',') {
-				str_start = str;
-				state = 1;
-			}
-		} else if (state == 1) {
-			if (*str == ',' || *str == '\0') {
-				if (str > str_start) {
-					/* Non-empty string fragment */
-					*str = '\0';
-					result = callback(str_start, data);
-					if (result < 0)
-						return result;
-				}
-				state = 0;
-			}
-		}
-		str++;
-	}
-	return 0;
-}
-
 static int
 rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	const uint16_t max_list)
 {
 	uint16_t lo, hi, val;
 	int result;
+	char *pos = str;
 
 	result = sscanf(str, "%hu-%hu", &lo, &hi);
 	if (result == 1) {
@@ -99,7 +52,7 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 			return -ENOMEM;
 		list[(*len_list)++] = lo;
 	} else if (result == 2) {
-		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
+		if (lo >= hi)
 			return -EINVAL;
 		for (val = lo; val <= hi; val++) {
 			if (*len_list >= max_list)
@@ -108,14 +61,52 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 		}
 	} else
 		return -EINVAL;
-	return 0;
+	while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
+		pos++;
+	return pos - str;
 }
 
+static int
+rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
+	const uint16_t max_list)
+{
+	char *pos = str;
+	int ret;
+
+	if (*pos == '[')
+		pos++;
+	while (1) {
+		ret = rte_eth_devargs_process_range(pos, list, len_list,
+						    max_list);
+		if (ret < 0)
+			return ret;
+		pos += ret;
+		if (*pos != ',') /* end of list */
+			break;
+		pos++;
+	}
+	if (*str == '[' && *pos != ']')
+		return -EINVAL;
+	if (*pos == ']')
+		pos++;
+	return pos - str;
+}
+
+/*
+ * representor format:
+ *   #: range or single number of VF representor - legacy
+ */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
+	int ret;
 
-	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
+	/* Number # alone implies VF */
+	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+	if (ret < 0)
+		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
+	return ret < 0 ? ret : 0;
 }
diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
index 905a45c337..220ddd4408 100644
--- a/lib/librte_ethdev/ethdev_private.h
+++ b/lib/librte_ethdev/ethdev_private.h
@@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
 		const void *data);
 
 /* Parse devargs value for representor parameter. */
-typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
-int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data);
 int rte_eth_devargs_parse_representor_ports(char *str, void *data);
 
 #ifdef __cplusplus
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index 6338355e25..efe6149df5 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
 	if (values == NULL)
 		return -1;
 	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_list(values,
-			rte_eth_devargs_parse_representor_ports,
-			&representors);
+	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 17ddacc78d..2ac51ac149 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5542,9 +5542,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
-			result = rte_eth_devargs_parse_list(pair->value,
-				rte_eth_devargs_parse_representor_ports,
-				eth_da);
+			result = rte_eth_devargs_parse_representor_ports(
+					pair->value, eth_da);
 			if (result < 0)
 				goto parse_cleanup;
 		}
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 0eacfd8425..b66a955b18 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1193,6 +1193,12 @@ __rte_internal
 int
 rte_eth_switch_domain_free(uint16_t domain_id);
 
+/** Ethernet device representor type */
+enum rte_eth_representor_type {
+	RTE_ETH_REPRESENTOR_NONE, /* not a representor */
+	RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
+};
+
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
 	uint16_t ports[RTE_MAX_ETHPORTS];
@@ -1203,6 +1209,7 @@ struct rte_eth_devargs {
 	/** representor port/s identifier to enable on device */
 	uint16_t nb_representor_ports;
 	/** number of ports in representor port field */
+	enum rte_eth_representor_type type; /* type of representor */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v2 2/9] ethdev: support new VF representor syntax

[Xueming Li]

To prepare for more representor types, this patch adds compatible VF
representor devargs syntax:

vf#: new VF port representor/s, example: vf[0-3], vf2

For backwards compatibility, representor "#" is deemed as "vf#".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 6 +++---
 lib/librte_ethdev/ethdev_private.c      | 5 ++++-
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 239ec820ea..979a2bab9d 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -374,9 +374,9 @@ parameters to those ports.
   this argument allows user to specify which switch ports to enable port
   representors for.::
 
-   -a DBDF,representor=0
-   -a DBDF,representor=[0,4,6,9]
-   -a DBDF,representor=[0-31]
+   -a DBDF,representor=vf0
+   -a DBDF,representor=vf[0,4,6,9]
+   -a DBDF,representor=vf[0-31]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index c219164a4a..54f565ca37 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -95,6 +95,7 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 /*
  * representor format:
  *   #: range or single number of VF representor - legacy
+ *   vf#: VF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -102,8 +103,10 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 	int ret;
 
-	/* Number # alone implies VF */
+	/* Parse vf# or number # alone implies VF */
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	if (str[0] == 'v'  && str[1] == 'f')
+		str += 2;
 	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 	if (ret < 0)
-- 
2.25.1

[dpdk-dev] [PATCH v2 3/9] ethdev: support sub function representor

[Xueming Li]

SubFunction is a portion of the PCI device, created on demand, a SF
netdev has its own dedicated queues(txq, rxq). A SF netdev supports
eswitch representation offload similar to existing PF and VF
representors.

To support SF representor, this patch introduces new devargs syntax:

sf#: new SF port representor/s, example: sf[0-3], sf2

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  1 +
 lib/librte_ethdev/ethdev_private.c      | 14 +++++++++++---
 lib/librte_ethdev/rte_ethdev_driver.h   |  1 +
 3 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 979a2bab9d..831d620c8d 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -377,6 +377,7 @@ parameters to those ports.
    -a DBDF,representor=vf0
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
+   -a DBDF,representor=sf[0-1023]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 54f565ca37..551a43738a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -96,6 +96,7 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * representor format:
  *   #: range or single number of VF representor - legacy
  *   vf#: VF port representor/s
+ *   sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -103,10 +104,17 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 	int ret;
 
-	/* Parse vf# or number # alone implies VF */
-	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	if (str[0] == 'v'  && str[1] == 'f')
+	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
+	/* Parse vf# and sf#, number # alone implies VF */
+	if (str[0] == 'v'  && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
+	} else if (str[0] == 's'  && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_SF;
+		str += 2;
+	} else {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	}
 	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 	if (ret < 0)
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index b66a955b18..086d64223a 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1197,6 +1197,7 @@ rte_eth_switch_domain_free(uint16_t domain_id);
 enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /* not a representor */
 	RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
+	RTE_ETH_REPRESENTOR_SF,   /* representor of SF */
 };
 
 /** Generic Ethernet device arguments  */
-- 
2.25.1

[dpdk-dev] [PATCH v2 4/9] ethdev: support PF index in representor

[Xueming Li]

To support representor in bonding, this patch introduces 'pf' section
to representor devargs syntax:

[pf#]vf#: VF port representor/s, example: vf3, pf0vf3
[pf#]sf#: SF port representor/s, example: sf2, pf1sf2

pf# is optional for SF and VF representor, indicates owner PF device of
SF or VF.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  1 +
 lib/librte_ethdev/ethdev_private.c      | 15 +++++++++++++--
 lib/librte_ethdev/rte_ethdev_driver.h   |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 831d620c8d..87f23c4f49 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -378,6 +378,7 @@ parameters to those ports.
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
    -a DBDF,representor=sf[0-1023]
+   -a DBDF,representor=pf[0-1]sf[0-1023]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 551a43738a..ccc638ec49 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -95,8 +95,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 /*
  * representor format:
  *   #: range or single number of VF representor - legacy
- *   vf#: VF port representor/s
- *   sf#: SF port representor/s
+ *   [pf#]vf#: VF port representor/s
+ *   [pf#]sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -105,6 +105,16 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	int ret;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
+	/* Parse pf# */
+	if (str[0] == 'p' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_PF;
+		str += 2;
+		ret = rte_eth_devargs_process_list(str, eth_da->ports,
+				&eth_da->nb_ports, RTE_MAX_ETHPORTS);
+		if (ret < 0)
+			goto err;
+		str += ret;
+	}
 	/* Parse vf# and sf#, number # alone implies VF */
 	if (str[0] == 'v'  && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
@@ -117,6 +127,7 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	}
 	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+err:
 	if (ret < 0)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
 	return ret < 0 ? ret : 0;
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 086d64223a..edb000cbd4 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1198,6 +1198,7 @@ enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /* not a representor */
 	RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
 	RTE_ETH_REPRESENTOR_SF,   /* representor of SF */
+	RTE_ETH_REPRESENTOR_PF,   /* representor of host PF */
 };
 
 /** Generic Ethernet device arguments  */
-- 
2.25.1

[dpdk-dev] [PATCH v2 5/9] ethdev: support multi-host representor

[Xueming Li]

This patch introduces multi-host controller for ethdev representor
syntax, examples:

[[c#]pf#]vf#: VF port representor/s, example: pf0vf1
[[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]

c# is controller(host) ID/range in case of multi-host, optional. It is
mostly for SmartNIC attached to multiple hosts in the same rack to allow
routing the packets between PF/SF/VF running on these hosts.

Controller ID is physical host ID in multi-host, for end-user (e.g. OVS)
it is transparent as part of representor syntax, sam as representor
ID, interpreted by PMD.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 config/rte_config.h                   |  1 +
 lib/librte_ethdev/ethdev_private.c    | 14 ++++++++++++--
 lib/librte_ethdev/rte_ethdev_driver.h |  4 ++++
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index a0b5160ff2..23d02d51ef 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -58,6 +58,7 @@
 #define RTE_MAX_QUEUES_PER_PORT 1024
 #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
 #define RTE_ETHDEV_RXTX_CALLBACKS 1
+#define RTE_MAX_MULTI_HOST_CTRLS 4
 
 /* cryptodev defines */
 #define RTE_CRYPTO_MAX_DEVS 64
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index ccc638ec49..95f1ab847a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -95,8 +95,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 /*
  * representor format:
  *   #: range or single number of VF representor - legacy
- *   [pf#]vf#: VF port representor/s
- *   [pf#]sf#: SF port representor/s
+ *   [[c#]pf#]vf#: VF port representor/s
+ *   [[c#]pf#]sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -105,6 +105,16 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	int ret;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
+	/* Parse c# */
+	if (str[0] == 'c') {
+		str += 1;
+		ret = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
+				&eth_da->nb_mh_controllers,
+				RTE_DIM(eth_da->mh_controllers));
+		if (ret < 0)
+			goto err;
+		str += ret;
+	}
 	/* Parse pf# */
 	if (str[0] == 'p' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_PF;
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index edb000cbd4..b9e4a0b9ba 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1203,6 +1203,10 @@ enum rte_eth_representor_type {
 
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
+	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
+	/** controller/s number in case of multi-host */
+	uint16_t nb_mh_controllers;
+	/** number of controllers in multi-host controllers field */
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
 	uint16_t nb_ports;
-- 
2.25.1

[dpdk-dev] [PATCH v2 6/9] devarg: change reprsentor ID to bitmap

[Xueming Li]

In eth representor comparator callback, ethdev was compared with devarg.
Since ethdev representor port didn't contain controller(host) and owner
port information, callback only compared representor port and returned
representor port on other PF port.

This patch changes representor port to bitmap encoding, expands and
updates representor port ID after parsing, when device representor ID
uses the same bitmap encoding, the eth representor comparer callback
returns correct ethdev.

Representor port ID bitmap definition:
 Representor ID bitmap:
 xxxx xxxx xxxx xxxx
 |||| |LLL LLLL LLLL vf/sf id
 |||| L 1:sf, 0:vf
 ||LL pf id
 LL controller(host) id

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/net/bnxt/bnxt_reps.c             |  3 +-
 drivers/net/enic/enic_vf_representor.c   |  3 +-
 drivers/net/i40e/i40e_vf_representor.c   |  3 +-
 drivers/net/ixgbe/ixgbe_vf_representor.c |  3 +-
 drivers/net/mlx5/linux/mlx5_os.c         |  4 +-
 lib/librte_ethdev/ethdev_private.c       |  5 ++-
 lib/librte_ethdev/rte_class_eth.c        | 38 ++++++++++++++----
 lib/librte_ethdev/rte_ethdev.c           | 26 ++++++++++++
 lib/librte_ethdev/rte_ethdev_driver.h    | 51 ++++++++++++++++++++++++
 lib/librte_ethdev/version.map            |  2 +
 10 files changed, 124 insertions(+), 14 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index e5ba0909b9..bc3b564d06 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -186,7 +186,8 @@ int bnxt_representor_init(struct rte_eth_dev *eth_dev, void *params)
 
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	eth_dev->data->representor_id = rep_params->vf_id;
+	eth_dev->data->representor_id = rte_eth_representor_id_encode(
+		0, 0, RTE_ETH_REPRESENTOR_VF, rep_params->vf_id);
 
 	rte_eth_random_addr(vf_rep_bp->dflt_mac_addr);
 	memcpy(vf_rep_bp->mac_addr, vf_rep_bp->dflt_mac_addr,
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index c2c03c0281..632317af15 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -674,7 +674,8 @@ int enic_vf_representor_init(struct rte_eth_dev *eth_dev, void *init_params)
 	eth_dev->dev_ops = &enic_vf_representor_dev_ops;
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	eth_dev->data->representor_id = vf->vf_id;
+	eth_dev->data->representor_id = rte_eth_representor_id_encode(
+		0, 0, RTE_ETH_REPRESENTOR_VF, vf->vf_id);
 	eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
 		sizeof(struct rte_ether_addr) *
 		ENIC_UNICAST_PERFECT_FILTERS, 0);
diff --git a/drivers/net/i40e/i40e_vf_representor.c b/drivers/net/i40e/i40e_vf_representor.c
index 9e40406a3d..d90d0fdb9d 100644
--- a/drivers/net/i40e/i40e_vf_representor.c
+++ b/drivers/net/i40e/i40e_vf_representor.c
@@ -510,7 +510,8 @@ i40e_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
 
 	ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	ethdev->data->representor_id = representor->vf_id;
+	ethdev->data->representor_id = rte_eth_representor_id_encode(
+			0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
 
 	/* Setting the number queues allocated to the VF */
 	ethdev->data->nb_rx_queues = vf->vsi->nb_qps;
diff --git a/drivers/net/ixgbe/ixgbe_vf_representor.c b/drivers/net/ixgbe/ixgbe_vf_representor.c
index 8185f0d3bb..e15b794761 100644
--- a/drivers/net/ixgbe/ixgbe_vf_representor.c
+++ b/drivers/net/ixgbe/ixgbe_vf_representor.c
@@ -196,7 +196,8 @@ ixgbe_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
 		return -ENODEV;
 
 	ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
-	ethdev->data->representor_id = representor->vf_id;
+	ethdev->data->representor_id = rte_eth_representor_id_encode(
+			0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
 
 	/* Set representor device ops */
 	ethdev->dev_ops = &ixgbe_vf_representor_dev_ops;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 6981ba1f41..7739a1b0a5 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1031,7 +1031,9 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 #endif
 	/* representor_id field keeps the unmodified VF index. */
 	priv->representor_id = switch_info->representor ?
-			       switch_info->port_name : -1;
+		rte_eth_representor_id_encode(0, 0, RTE_ETH_REPRESENTOR_VF,
+					      switch_info->port_name) :
+		-1;
 	/*
 	 * Look for sibling devices in order to reuse their switch domain
 	 * if any, otherwise allocate one.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 95f1ab847a..57e84a34af 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -93,10 +93,13 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 }
 
 /*
- * representor format:
+ * Parse representor ports, expand and update representor port ID.
+ * Representor format:
  *   #: range or single number of VF representor - legacy
  *   [[c#]pf#]vf#: VF port representor/s
  *   [[c#]pf#]sf#: SF port representor/s
+ *
+ * See RTE_ETH_REPR() for representor ID format.
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index efe6149df5..994db96960 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -66,8 +66,8 @@ eth_representor_cmp(const char *key __rte_unused,
 	int ret;
 	char *values;
 	const struct rte_eth_dev_data *data = opaque;
-	struct rte_eth_devargs representors;
-	uint16_t index;
+	struct rte_eth_devargs eth_da;
+	uint16_t index, c, p, f;
 
 	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
 		return -1; /* not a representor port */
@@ -76,17 +76,39 @@ eth_representor_cmp(const char *key __rte_unused,
 	values = strdup(value);
 	if (values == NULL)
 		return -1;
-	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
+	memset(&eth_da, 0, sizeof(eth_da));
+	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
 
+	/* Set default values. */
+	if (eth_da.nb_mh_controllers == 0) {
+		eth_da.nb_mh_controllers = 1;
+		eth_da.mh_controllers[0] = 0;
+	}
+	if (eth_da.nb_ports == 0) {
+		eth_da.nb_ports = 1;
+		eth_da.ports[0] = 0;
+	}
+	if (eth_da.nb_representor_ports == 0) {
+		eth_da.nb_representor_ports = 1;
+		eth_da.representor_ports[0] = 0;
+	}
 	/* Return 0 if representor id is matching one of the values. */
-	for (index = 0; index < representors.nb_representor_ports; index++)
-		if (data->representor_id ==
-				representors.representor_ports[index])
-			return 0;
+	for (c = 0; c < eth_da.nb_mh_controllers; ++c) {
+		for (p = 0; p < eth_da.nb_ports; ++p) {
+			for (f = 0; f < eth_da.nb_representor_ports; ++f) {
+				index = rte_eth_representor_id_encode(
+					eth_da.mh_controllers[c],
+					eth_da.ports[p],
+					eth_da.type,
+					eth_da.representor_ports[f]);
+				if (data->representor_id == index)
+					return 0;
+			}
+		}
+	}
 	return -1; /* no match */
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 2ac51ac149..276f42e54b 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5556,6 +5556,32 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	return result;
 }
 
+uint16_t
+rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
+			      enum rte_eth_representor_type type,
+			      uint16_t representor_port)
+{
+	return (((controller & 3) << 14) |
+		((pf & 3) << 12) |
+		(!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
+		(representor_port & 0x7ff));
+}
+
+uint16_t
+rte_eth_representor_id_parse(const uint16_t representor_id,
+			     uint16_t *controller, uint16_t *pf,
+			     enum rte_eth_representor_type *type)
+{
+	if (controller)
+		*controller = (representor_id >> 14) & 3;
+	if (pf)
+		*pf = (representor_id >> 12) & 3;
+	if (type)
+		*type = ((representor_id >> 11) & 1) ?
+			RTE_ETH_REPRESENTOR_SF : RTE_ETH_REPRESENTOR_VF;
+	return representor_id & 0x7ff;
+}
+
 static int
 eth_dev_handle_port_list(const char *cmd __rte_unused,
 		const char *params __rte_unused,
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index b9e4a0b9ba..d557b763b6 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1218,6 +1218,57 @@ struct rte_eth_devargs {
 	enum rte_eth_representor_type type; /* type of representor */
 };
 
+/**
+ * PMD helper function to encode representor ID
+ *
+ * The compact format is used for device iterator that comparing
+ * ethdev representor ID with target devargs.
+ *
+ * xxxx xxxx xxxx xxxx
+ * |||| |LLL LLLL LLLL vf/sf id
+ * |||| L 1:sf, 0:vf
+ * ||LL pf id
+ * LL controller(host) id
+ *
+ * @param controller
+ *  Controller ID.
+ * @param pf
+ *  PF port ID.
+ * @param type
+ *  Representor type.
+ * @param representor_port
+ *  Representor port ID.
+ *
+ * @return
+ *   Encoded representor ID.
+ */
+__rte_internal
+uint16_t
+rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
+			      enum rte_eth_representor_type type,
+			      uint16_t representor_port);
+
+/**
+ * PMD helper function to parse representor ID
+ *
+ * @param representor_id
+ *  Representor ID.
+ * @param controller
+ *  Parsed controller ID.
+ * @param pf
+ *  Parsed PF port ID.
+ * @param type
+ *  Parsed representor type.
+ *
+ * @return
+ *   Parsed representor port ID.
+ */
+__rte_internal
+uint16_t
+rte_eth_representor_id_parse(const uint16_t representor_id,
+			     uint16_t *controller, uint16_t *pf,
+			     enum rte_eth_representor_type *type);
+
 /**
  * PMD helper function to parse ethdev arguments
  *
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index d3f5410806..44edaed507 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -257,6 +257,8 @@ INTERNAL {
 	rte_eth_dev_release_port;
 	rte_eth_dev_internal_reset;
 	rte_eth_devargs_parse;
+	rte_eth_representor_id_encode;
+	rte_eth_representor_id_parse;
 	rte_eth_dma_zone_free;
 	rte_eth_dma_zone_reserve;
 	rte_eth_hairpin_queue_peer_bind;
-- 
2.25.1

[dpdk-dev] [PATCH v2 7/9] ethdev: capability of new representor syntax

[Xueming Li]

New representor devargs syntax can't be recognized correctly by previous
DPDK version. For application to adapt different DPDK version
automatically, this patch introduces new eth device capability of
supportting SubFunction representor device.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_ethdev/rte_ethdev.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index f5f8919186..3c087bec67 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1432,6 +1432,8 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/**< Device supports SubFunction representor. */
+#define RTE_ETH_DEV_CAPA_REPRESENTOR_SF         0x00000004
 /**@}*/
 
 /*
-- 
2.25.1

[dpdk-dev] [PATCH v2 8/9] kvargs: update parser for new representor syntax

[Xueming Li]

This patch updates kvargs parser to allow comma in list value:
  k1=a[1,2]b[3-5]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_kvargs/rte_kvargs.c | 83 +++++++++++++++++++++++-----------
 1 file changed, 57 insertions(+), 26 deletions(-)

diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
index 285081c86c..4053989157 100644
--- a/lib/librte_kvargs/rte_kvargs.c
+++ b/lib/librte_kvargs/rte_kvargs.c
@@ -5,6 +5,7 @@
 
 #include <string.h>
 #include <stdlib.h>
+#include <stdbool.h>
 
 #include <rte_string_fns.h>
 
@@ -13,15 +14,19 @@
 /*
  * Receive a string with a list of arguments following the pattern
  * key=value,key=value,... and insert them into the list.
- * strtok() is used so the params string will be copied to be modified.
+ * Params string will be copied to be modified.
+ * list "[]" and list element splitter ",", "-" is treated as value.
+ * Supported examples:
+ *   k1=v1,k2=v2
+ *   v1
+ *   k1=x[0-1]y[1,3-5,9]z
  */
 static int
 rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 {
 	unsigned i;
-	char *str;
-	char *ctx1 = NULL;
-	char *ctx2 = NULL;
+	char *str, *start;
+	bool in_list = false, end_k = false, end_v = false;
 
 	/* Copy the const char *params to a modifiable string
 	 * to pass to rte_strsplit
@@ -32,36 +37,62 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 
 	/* browse each key/value pair and add it in kvlist */
 	str = kvlist->str;
-	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
+	start = str; /* start of current key or value */
+	while (1) {
+		switch (*str) {
+		case '=': /* End of key. */
+			end_k = true;
+			break;
+		case ',':
+			/* End of value, skip comma in middle of range */
+			if (!in_list)
+				end_v = true;
+			break;
+		case '[': /* Start of list. */
+			in_list = true;
+			break;
+		case ']': /* End of list.  */
+			if (in_list)
+				in_list = false;
+			break;
+		case 0: /* End of string */
+			end_v = true;
+			break;
+		default:
+			break;
+		}
+
+		if (!end_k && !end_v) {
+			/* Continue if not end of key or value. */
+			str++;
+			continue;
+		}
 
 		i = kvlist->count;
 		if (i >= RTE_KVARGS_MAX)
 			return -1;
 
-		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
-		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
-		if (kvlist->pairs[i].key == NULL ||
-		    kvlist->pairs[i].value == NULL)
-			return -1;
-
-		/* Detect list [a,b] to skip comma delimiter in list. */
-		str = kvlist->pairs[i].value;
-		if (str[0] == '[') {
-			/* Find the end of the list. */
-			while (str[strlen(str) - 1] != ']') {
-				/* Restore the comma erased by strtok_r(). */
-				if (ctx1 == NULL || ctx1[0] == '\0')
-					return -1; /* no closing bracket */
-				str[strlen(str)] = ',';
-				/* Parse until next comma. */
-				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
-				if (str == NULL)
-					return -1; /* no closing bracket */
+		if (start == str) /* Empty key  or value. */
+			start = NULL;
+
+		if (end_k) {
+			/* Key parsed. */
+			kvlist->pairs[i].key = start;
+			end_k = false;
+		} else if (end_v) {
+			/* Allow single key or single value. */
+			if (kvlist->pairs[i].key || start) {
+				kvlist->pairs[i].value = start;
+				kvlist->count++;
 			}
+			end_v = false;
 		}
 
-		kvlist->count++;
-		str = NULL;
+		if (*str == '\0') /* End of string. */
+			break;
+		*str = '\0';
+		str++;
+		start = str;
 	}
 
 	return 0;
-- 
2.25.1

[dpdk-dev] [PATCH v2 9/9] eal: allow PCI device with different representors

[Xueming Li]

When probing same PCI device with different representor arguments, PCI
bus on probed first devargs, ignored other allowed devices with
different representor arguments.

This patch iterates all devargs and try them all after PCI bus scan.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_eal/common/eal_common_bus.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index baa5b532af..47c0243647 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -10,6 +10,7 @@
 #include <rte_debug.h>
 #include <rte_string_fns.h>
 #include <rte_errno.h>
+#include <rte_devargs.h>
 
 #include "eal_private.h"
 
@@ -56,12 +57,22 @@ rte_bus_scan(void)
 	return 0;
 }
 
+static int
+cmp_dev_name(const struct rte_device *dev, const void *_name)
+{
+	const char *name = _name;
+
+	return strcmp(dev->name, name);
+}
+
 /* Probe all devices of all buses */
 int
 rte_bus_probe(void)
 {
 	int ret;
 	struct rte_bus *bus, *vbus = NULL;
+	struct rte_devargs *da;
+	struct rte_device *dev;
 
 	TAILQ_FOREACH(bus, &rte_bus_list, next) {
 		if (!strcmp(bus->name, "vdev")) {
@@ -82,6 +93,20 @@ rte_bus_probe(void)
 				vbus->name);
 	}
 
+	/* For devargs with same name but different arguments, try them all. */
+	RTE_EAL_DEVARGS_FOREACH("pci", da) {
+		dev = da->bus->find_device(NULL, cmp_dev_name, da->name);
+		if (!dev || !rte_dev_is_probed(dev) || dev->devargs == da)
+			continue;
+		dev->devargs = da;
+		ret = dev->bus->plug(dev);
+		if (ret > 0)
+			ret = -ENOTSUP;
+		if (!ret && rte_dev_is_probed(dev))
+			RTE_LOG(ERR, EAL, "device probed %s %s", da->name,
+				da->args);
+	}
+
 	return 0;
 }
 
-- 
2.25.1

[dpdk-dev] [PATCH v2 0/4] net/mlx5: support SubFunction representor

[Xueming Li]

SubFunction [1] is a portion of the PCI device, a SF netdev has its own
dedicated queues(txq, rxq). A SF netdev supports eswitch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

This patch set introduces SubFunction representor support for mlx5
PMD driver.

Depends-on: series-14559 ("support SubFunction representor")

Version history:
 RFC:
 	initial version [2]
 V2:
    - support bonding representor probe with new pf#vf# devargs
    - adapt EAL api V2 [3] changes
    - update document

[1] SubFunction in kernel:
https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/

[2] RFC:
http://patchwork.dpdk.org/project/dpdk/list/?series=14376

[3] EAL part to support SF representor:
http://patchwork.dpdk.org/project/dpdk/list/?series=14559


Xueming Li (4):
  common/mlx5: update representor name parsing
  net/mlx5: support representor of sub function
  net/mlx5: revert setting representor to first PF
  net/mlx5: improve bonding representor probe

 doc/guides/nics/mlx5.rst                   |  62 +++++-
 drivers/common/mlx5/linux/mlx5_common_os.c |  32 ++-
 drivers/common/mlx5/linux/mlx5_nl.c        |   2 +
 drivers/common/mlx5/mlx5_common.h          |   2 +
 drivers/net/mlx5/linux/mlx5_ethdev_os.c    |   5 +
 drivers/net/mlx5/linux/mlx5_os.c           | 215 ++++++++++++++-------
 drivers/net/mlx5/mlx5.c                    |  23 ++-
 drivers/net/mlx5/mlx5.h                    |   4 +-
 drivers/net/mlx5/mlx5_defs.h               |   4 -
 drivers/net/mlx5/mlx5_ethdev.c             |  29 +--
 drivers/net/mlx5/mlx5_mac.c                |   8 +-
 11 files changed, 261 insertions(+), 125 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH v2 1/4] common/mlx5: update representor name parsing

[Xueming Li]

This patch updates representor name parsing for SF.
In sysfs, representor name stored in from 'phys_port_name file', similar
to VF representor, switch port name of SF representor is "pf<x>sf<y>".

For netlink message, net SF type is supported.

Examples:

pf0sf1
pf0sf[0-3]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.c | 32 +++++++++++++++-------
 drivers/common/mlx5/linux/mlx5_nl.c        |  2 ++
 drivers/common/mlx5/mlx5_common.h          |  2 ++
 drivers/net/mlx5/linux/mlx5_ethdev_os.c    |  3 ++
 4 files changed, 29 insertions(+), 10 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c
index 0edd78ea6d..5cf9576921 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.c
+++ b/drivers/common/mlx5/linux/mlx5_common_os.c
@@ -97,22 +97,34 @@ void
 mlx5_translate_port_name(const char *port_name_in,
 			 struct mlx5_switch_info *port_info_out)
 {
-	char pf_c1, pf_c2, vf_c1, vf_c2, eol;
+	char ctrl = 0, pf_c1, pf_c2, vf_c1, vf_c2, eol;
 	char *end;
 	int sc_items;
 
-	/*
-	 * Check for port-name as a string of the form pf0vf0
-	 * (support kernel ver >= 5.0 or OFED ver >= 4.6).
-	 */
+	sc_items = sscanf(port_name_in, "%c%d",
+			  &ctrl, &port_info_out->ctrl_num);
+	if (sc_items == 2 && ctrl == 'c') {
+		port_name_in++; /* 'c' */
+		port_name_in += snprintf(NULL, 0, "%d",
+					  port_info_out->ctrl_num);
+	}
+	/* Check for port-name as a string of the form pf0vf0 or pf0sf0 */
 	sc_items = sscanf(port_name_in, "%c%c%d%c%c%d%c",
 			  &pf_c1, &pf_c2, &port_info_out->pf_num,
 			  &vf_c1, &vf_c2, &port_info_out->port_name, &eol);
-	if (sc_items == 6 &&
-	    pf_c1 == 'p' && pf_c2 == 'f' &&
-	    vf_c1 == 'v' && vf_c2 == 'f') {
-		port_info_out->name_type = MLX5_PHYS_PORT_NAME_TYPE_PFVF;
-		return;
+	if (sc_items == 6 && pf_c1 == 'p' && pf_c2 == 'f') {
+		if (vf_c1 == 'v' && vf_c2 == 'f') {
+			/* Kernel ver >= 5.0 or OFED ver >= 4.6 */
+			port_info_out->name_type =
+					MLX5_PHYS_PORT_NAME_TYPE_PFVF;
+			return;
+		}
+		if (vf_c1 == 's' && vf_c2 == 'f') {
+			/* Kernel ver >= 5.11 or OFED ver >= 5.1 */
+			port_info_out->name_type =
+					MLX5_PHYS_PORT_NAME_TYPE_PFSF;
+			return;
+		}
 	}
 	/*
 	 * Check for port-name as a string of the form p0
diff --git a/drivers/common/mlx5/linux/mlx5_nl.c b/drivers/common/mlx5/linux/mlx5_nl.c
index 40d8620300..3d55cc98b4 100644
--- a/drivers/common/mlx5/linux/mlx5_nl.c
+++ b/drivers/common/mlx5/linux/mlx5_nl.c
@@ -1148,6 +1148,8 @@ mlx5_nl_check_switch_info(bool num_vf_set,
 	case MLX5_PHYS_PORT_NAME_TYPE_PFHPF:
 		/* Fallthrough */
 	case MLX5_PHYS_PORT_NAME_TYPE_PFVF:
+		/* Fallthrough */
+	case MLX5_PHYS_PORT_NAME_TYPE_PFSF:
 		/* New representors naming schema. */
 		switch_info->representor = 1;
 		break;
diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h
index a484b74b9c..4c75addd08 100644
--- a/drivers/common/mlx5/mlx5_common.h
+++ b/drivers/common/mlx5/mlx5_common.h
@@ -153,6 +153,7 @@ enum mlx5_nl_phys_port_name_type {
 	MLX5_PHYS_PORT_NAME_TYPE_UPLINK, /* p0, kernel ver >= 5.0 */
 	MLX5_PHYS_PORT_NAME_TYPE_PFVF, /* pf0vf0, kernel ver >= 5.0 */
 	MLX5_PHYS_PORT_NAME_TYPE_PFHPF, /* pf0, kernel ver >= 5.7, HPF rep */
+	MLX5_PHYS_PORT_NAME_TYPE_PFSF, /* pf0sf0, kernel ver >= 5.0 */
 	MLX5_PHYS_PORT_NAME_TYPE_UNKNOWN, /* Unrecognized. */
 };
 
@@ -161,6 +162,7 @@ struct mlx5_switch_info {
 	uint32_t master:1; /**< Master device. */
 	uint32_t representor:1; /**< Representor device. */
 	enum mlx5_nl_phys_port_name_type name_type; /** < Port name type. */
+	int32_t ctrl_num; /**< Controller number (valid for c#pf#vf# format). */
 	int32_t pf_num; /**< PF number (valid for pfxvfx format only). */
 	int32_t port_name; /**< Representor port name. */
 	uint64_t switch_id; /**< Switch identifier. */
diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
index e36a78091c..1b37970c21 100644
--- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
@@ -1013,6 +1013,9 @@ mlx5_sysfs_check_switch_info(bool device_dir,
 		/* New representors naming schema. */
 		switch_info->representor = 1;
 		break;
+	default:
+		switch_info->master = device_dir;
+		break;
 	}
 }
 
-- 
2.25.1

[dpdk-dev] [PATCH v2 2/4] net/mlx5: support representor of sub function

[Xueming Li]

This patch supports of SF representor. Similar to VF representor, switch
port name of SF representor in sysfs phys_port_name is "pf<x>sf<y>".

Device representor argumnt is "representors=sf[list]", list member could
be mix of instance and range. Example:
  representors=sf[0,2,4,8-12,-1]

To probe VF representor and SF representor, need to separate into 2
devices:
  -a <BDF>,representor=vf[list] -a <BDF>,representor=sf[list]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 doc/guides/nics/mlx5.rst                |  58 +++++++++--
 drivers/net/mlx5/linux/mlx5_ethdev_os.c |   2 +
 drivers/net/mlx5/linux/mlx5_os.c        | 123 ++++++++++++++++++++----
 drivers/net/mlx5/mlx5_ethdev.c          |   2 +
 4 files changed, 154 insertions(+), 31 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 3bda0f8417..fc7e93842f 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -874,14 +874,18 @@ Driver options
 - ``representor`` parameter [list]
 
   This parameter can be used to instantiate DPDK Ethernet devices from
-  existing port (or VF) representors configured on the device.
+  existing port (PF, VF or SF) representors configured on the device.
 
   It is a standard parameter whose format is described in
   :ref:`ethernet_device_standard_device_arguments`.
 
-  For instance, to probe port representors 0 through 2::
+  For instance, to probe VF port representors 0 through 2::
 
-    representor=[0-2]
+    representor=vf[0-2]
+
+  To probe SF port representors 0 through 2::
+
+    representor=sf[0-2]
 
 - ``max_dump_files_num`` parameter [int]
 
@@ -1253,15 +1257,15 @@ Quick Start Guide on OFED/EN
 Enable switchdev mode
 ---------------------
 
-Switchdev mode is a mode in E-Switch, that binds between representor and VF.
-Representor is a port in DPDK that is connected to a VF in such a way
-that assuming there are no offload flows, each packet that is sent from the VF
-will be received by the corresponding representor. While each packet that is
-sent to a representor will be received by the VF.
+Switchdev mode is a mode in E-Switch, that binds between representor and VF or SF.
+Representor is a port in DPDK that is connected to a VF or SF in such a way
+that assuming there are no offload flows, each packet that is sent from the VF or SF
+will be received by the corresponding representor. While each packet that is or SF
+sent to a representor will be received by the VF or SF.
 This is very useful in case of SRIOV mode, where the first packet that is sent
-by the VF will be received by the DPDK application which will decide if this
+by the VF or SF will be received by the DPDK application which will decide if this
 flow should be offloaded to the E-Switch. After offloading the flow packet
-that the VF that are matching the flow will not be received any more by
+that the VF or SF that are matching the flow will not be received any more by
 the DPDK application.
 
 1. Enable SRIOV mode::
@@ -1288,6 +1292,40 @@ the DPDK application.
 
         echo switchdev > /sys/class/net/<net device>/compat/devlink/mode
 
+SubFunction representor support
+-------------------------------
+SubFunction is a portion of the PCI device, a SF netdev has its own
+dedicated queues(txq, rxq). A SF netdev supports eswitch representation
+offload similar to existing PF and VF representors. A SF shares PCI
+level resources with other SFs and/or with its parent PCI function.
+
+1. Configure SF feature::
+
+        mlxconfig -d <mst device> set PF_BAR2_SIZE=<0/1/2/3> PF_BAR2_ENABLE=1
+
+        Value of PF_BAR2_SIZE:
+
+            0: 8 SFs
+            1: 16 SFs
+            2: 32 SFs
+            3: 64 SFs
+
+2. Reset the FW::
+
+        mlxfwreset -d <mst device> reset
+
+3. Enable switchdev mode::
+
+        echo switchdev > /sys/class/net/<net device>/compat/devlink/mode
+
+4. Create SF::
+
+        mlnx-sf -d <PCI_BDF> -a create
+
+5. Probe SF representor::
+
+        testpmd> port attach <PCI_BDF>,representor=sf0,dv_flow_en=1
+
 Performance tuning
 ------------------
 
diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
index 1b37970c21..ac311de46d 100644
--- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
@@ -1010,6 +1010,8 @@ mlx5_sysfs_check_switch_info(bool device_dir,
 	case MLX5_PHYS_PORT_NAME_TYPE_PFHPF:
 		/* Fallthrough */
 	case MLX5_PHYS_PORT_NAME_TYPE_PFVF:
+		/* Fallthrough */
+	case MLX5_PHYS_PORT_NAME_TYPE_PFSF:
 		/* New representors naming schema. */
 		switch_info->representor = 1;
 		break;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 7739a1b0a5..6f667dd747 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -654,6 +654,8 @@ mlx5_flow_counter_mode_config(struct rte_eth_dev *dev __rte_unused)
  *   Verbs device parameters (name, port, switch_info) to spawn.
  * @param config
  *   Device configuration parameters.
+ * @param config
+ *   Device arguments.
  *
  * @return
  *   A valid Ethernet device object on success, NULL otherwise and rte_errno
@@ -665,7 +667,8 @@ mlx5_flow_counter_mode_config(struct rte_eth_dev *dev __rte_unused)
 static struct rte_eth_dev *
 mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	       struct mlx5_dev_spawn_data *spawn,
-	       struct mlx5_dev_config *config)
+	       struct mlx5_dev_config *config,
+	       struct rte_eth_devargs *eth_da)
 {
 	const struct mlx5_switch_info *switch_info = &spawn->info;
 	struct mlx5_dev_ctx_shared *sh = NULL;
@@ -697,34 +700,82 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 
 	/* Determine if this port representor is supposed to be spawned. */
 	if (switch_info->representor && dpdk_dev->devargs) {
-		struct rte_eth_devargs eth_da;
-
-		err = rte_eth_devargs_parse(dpdk_dev->devargs->args, &eth_da);
-		if (err) {
-			rte_errno = -err;
-			DRV_LOG(ERR, "failed to process device arguments: %s",
-				strerror(rte_errno));
-			return NULL;
-		}
-		if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
-			/* Representor not specified. */
+		switch (eth_da->type) {
+		case RTE_ETH_REPRESENTOR_SF:
+			if (switch_info->name_type !=
+					MLX5_PHYS_PORT_NAME_TYPE_PFSF) {
+				rte_errno = EBUSY;
+				return NULL;
+			}
+			break;
+		case RTE_ETH_REPRESENTOR_VF:
+			/* Allows HPF representor index -1 as exception. */
+			if (!(spawn->info.port_name == -1 &&
+			      switch_info->name_type ==
+					MLX5_PHYS_PORT_NAME_TYPE_PFHPF) &&
+			    switch_info->name_type !=
+					MLX5_PHYS_PORT_NAME_TYPE_PFVF) {
+				rte_errno = EBUSY;
+				return NULL;
+			}
+			break;
+		case RTE_ETH_REPRESENTOR_NONE:
 			rte_errno = EBUSY;
 			return NULL;
-		}
-		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+			break;
+		default:
 			rte_errno = ENOTSUP;
 			DRV_LOG(ERR, "unsupported representor type: %s",
 				dpdk_dev->devargs->args);
 			return NULL;
 		}
-		for (i = 0; i < eth_da.nb_representor_ports; ++i)
-			if (eth_da.representor_ports[i] ==
+		/* Check controller ID: */
+		for (i = 0; i < eth_da->nb_mh_controllers; ++i)
+			if (eth_da->mh_controllers[i] ==
+			    (uint16_t)switch_info->ctrl_num)
+				break;
+		if (eth_da->nb_mh_controllers &&
+		    i == eth_da->nb_mh_controllers) {
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		/* Check SF/VF ID: */
+		for (i = 0; i < eth_da->nb_representor_ports; ++i)
+			if (eth_da->representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
 				break;
-		if (i == eth_da.nb_representor_ports) {
+		if (eth_da->type != RTE_ETH_REPRESENTOR_PF &&
+		    i == eth_da->nb_representor_ports) {
 			rte_errno = EBUSY;
 			return NULL;
 		}
+		/* Check PF ID. Check after repr port to avoid warning flood. */
+		if (spawn->pf_bond >= 0) {
+			for (i = 0; i < eth_da->nb_ports; ++i)
+				if (eth_da->ports[i] ==
+				    (uint16_t)switch_info->pf_num)
+					break;
+			if (eth_da->nb_ports && i == eth_da->nb_ports) {
+				/* For backward compatibility, bonding
+				 * representor syntax supported with limitation,
+				 * device iterator won't find it:
+				 *    <PF1_BDF>,representor=#
+				 */
+				if (switch_info->pf_num > 0 &&
+				    eth_da->ports[0] == 0) {
+					DRV_LOG(WARNING, "Representor on Bonding PF should use pf#vf# format: %s",
+						dpdk_dev->devargs->args);
+				} else {
+					rte_errno = EBUSY;
+					return NULL;
+				}
+			}
+		} else if (eth_da->nb_ports > 1 || eth_da->ports[0]) {
+			rte_errno = EINVAL;
+			DRV_LOG(ERR, "PF id not supported by non-bond device: %s",
+				dpdk_dev->devargs->args);
+			return NULL;
+		}
 	}
 	/* Build device name. */
 	if (spawn->pf_bond <  0) {
@@ -732,8 +783,11 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 		if (!switch_info->representor)
 			strlcpy(name, dpdk_dev->name, sizeof(name));
 		else
-			snprintf(name, sizeof(name), "%s_representor_%u",
-				 dpdk_dev->name, switch_info->port_name);
+			snprintf(name, sizeof(name), "%s_representor_%s%u",
+				 dpdk_dev->name,
+				 switch_info->name_type ==
+				 MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf",
+				 switch_info->port_name);
 	} else {
 		/* Bonding device. */
 		if (!switch_info->representor)
@@ -741,9 +795,11 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				 dpdk_dev->name,
 				 mlx5_os_get_dev_device_name(spawn->phys_dev));
 		else
-			snprintf(name, sizeof(name), "%s_%s_representor_%u",
+			snprintf(name, sizeof(name), "%s_%s_representor_%s%u",
 				 dpdk_dev->name,
 				 mlx5_os_get_dev_device_name(spawn->phys_dev),
+				 switch_info->name_type ==
+				 MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf",
 				 switch_info->port_name);
 	}
 	/* check if the device is already spawned */
@@ -1802,6 +1858,7 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	struct mlx5_dev_spawn_data *list = NULL;
 	struct mlx5_dev_config dev_config;
 	unsigned int dev_config_vf;
+	struct rte_eth_devargs eth_da = { .type = RTE_ETH_REPRESENTOR_NONE };
 	int ret;
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
@@ -1812,6 +1869,27 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 			strerror(rte_errno));
 		return -rte_errno;
 	}
+	if (pci_dev->device.devargs) {
+		/* Parse representor information from device argument. */
+		if (pci_dev->device.devargs->cls_str)
+			ret = rte_eth_devargs_parse(
+				pci_dev->device.devargs->cls_str, &eth_da);
+		if (ret) {
+			DRV_LOG(ERR, "failed to parse device arguments: %s",
+				pci_dev->device.devargs->cls_str);
+			return -rte_errno;
+		}
+		if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) {
+			/* Support legacy device argument */
+			ret = rte_eth_devargs_parse(
+				pci_dev->device.devargs->args, &eth_da);
+			if (ret) {
+				DRV_LOG(ERR, "failed to parse device arguments: %s",
+					pci_dev->device.devargs->args);
+				return -rte_errno;
+			}
+		}
+	}
 	errno = 0;
 	ibv_list = mlx5_glue->get_device_list(&ret);
 	if (!ibv_list) {
@@ -1984,6 +2062,8 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 				case MLX5_PHYS_PORT_NAME_TYPE_PFHPF:
 					/* Fallthrough */
 				case MLX5_PHYS_PORT_NAME_TYPE_PFVF:
+					/* Fallthrough */
+				case MLX5_PHYS_PORT_NAME_TYPE_PFSF:
 					if (list[ns].info.pf_num == bd)
 						ns++;
 					break;
@@ -2161,7 +2241,8 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		dev_config.log_hp_size = MLX5_ARG_UNSET;
 		list[i].eth_dev = mlx5_dev_spawn(&pci_dev->device,
 						 &list[i],
-						 &dev_config);
+						 &dev_config,
+						 &eth_da);
 		if (!list[i].eth_dev) {
 			if (rte_errno != EBUSY && rte_errno != EEXIST)
 				break;
diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
index 45ee7e4488..ad6aacc329 100644
--- a/drivers/net/mlx5/mlx5_ethdev.c
+++ b/drivers/net/mlx5/mlx5_ethdev.c
@@ -374,6 +374,8 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info)
 			break;
 		}
 	}
+	if (priv->master)
+		info->dev_capa = RTE_ETH_DEV_CAPA_REPRESENTOR_SF;
 	return 0;
 }
 
-- 
2.25.1

[dpdk-dev] [PATCH v2 3/4] net/mlx5: revert setting representor to first PF

[Xueming Li]

Representors on second PF are being probed by devargs:
	<primary_bdf>,representor=pf1vf<N>
No need to save primary PF port ID and lookup when probing sibling
ports, revert patch [1]

[1]:
commit e6818853c022 ("net/mlx5: set representor to first PF in bonding
mode")

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 20 ++------------------
 drivers/net/mlx5/mlx5.c          |  1 -
 drivers/net/mlx5/mlx5.h          |  1 -
 3 files changed, 2 insertions(+), 20 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 6f667dd747..1209bfed5b 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -817,13 +817,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 			rte_errno = ENOMEM;
 			return NULL;
 		}
-		priv = eth_dev->data->dev_private;
-		if (priv->sh->bond_dev != UINT16_MAX)
-			/* For bonding port, use primary PCI device. */
-			eth_dev->device =
-				rte_eth_devices[priv->sh->bond_dev].device;
-		else
-			eth_dev->device = dpdk_dev;
+		eth_dev->device = dpdk_dev;
 		eth_dev->dev_ops = &mlx5_dev_sec_ops;
 		eth_dev->rx_descriptor_status = mlx5_rx_descriptor_status;
 		eth_dev->tx_descriptor_status = mlx5_tx_descriptor_status;
@@ -1451,17 +1445,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	eth_dev->data->dev_private = priv;
 	priv->dev_data = eth_dev->data;
 	eth_dev->data->mac_addrs = priv->mac;
-	if (spawn->pf_bond < 0) {
-		eth_dev->device = dpdk_dev;
-	} else {
-		/* Use primary bond PCI as device. */
-		if (sh->bond_dev == UINT16_MAX) {
-			sh->bond_dev = eth_dev->data->port_id;
-			eth_dev->device = dpdk_dev;
-		} else {
-			eth_dev->device = rte_eth_devices[sh->bond_dev].device;
-		}
-	}
+	eth_dev->device = dpdk_dev;
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
 	/* Configure the first MAC address by default. */
 	if (mlx5_get_mac(eth_dev, &mac.addr_bytes)) {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 023ef50a77..628587faac 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -917,7 +917,6 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 		goto error;
 	}
 	sh->refcnt = 1;
-	sh->bond_dev = UINT16_MAX;
 	sh->max_port = spawn->max_port;
 	strncpy(sh->ibdev_name, mlx5_os_get_ctx_device_name(sh->ctx),
 		sizeof(sh->ibdev_name) - 1);
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 41034f5d19..a77a1600d5 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -692,7 +692,6 @@ struct mlx5_flex_parser_profiles {
 struct mlx5_dev_ctx_shared {
 	LIST_ENTRY(mlx5_dev_ctx_shared) next;
 	uint32_t refcnt;
-	uint16_t bond_dev; /* Bond primary device id. */
 	uint32_t devx:1; /* Opened with DV. */
 	uint32_t flow_hit_aso_en:1; /* Flow Hit ASO is supported. */
 	uint32_t eqn; /* Event Queue number. */
-- 
2.25.1

[dpdk-dev] [PATCH v2 4/4] net/mlx5: improve bonding representor probe

[Xueming Li]

To probe representor on 2nd PF of bonding device, had to specify PF1 BDF
in devarg:
  <PF1_BDF>,representor=0
When closing bonding device, all representors had to be closed together
and this implies all representors have to use master PF of bonding
device. So after probing representor port on 2nd PF, when locating new
probed device using device argument, the filter used 2nd PF as PCI
address and failed to locate new device.

Conflict happened by using current representor devargs:
 - Use PCI BDF to specify representor owner PF
 - Use PCI BDF to locate probed representor device.
 - PMD use master PCI BDF as PCI device.

To resolve such conflicts, new representor syntax is introduced here:
  <master BDF>,representor=pfXvfY
All representors must use master PF as owner PCI device, PMD internally
locate owner PCI address by checking representor "pfX" part. To EAL, all
representor are registered to master PCI device, 2nd PF is hidden to
EAL, thus all search should be consistent.

This patch also add pf index into bonding mode representor port name:
	<BDF>_<ib_name>_representor_pf<X>vf<Y>

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 doc/guides/nics/mlx5.rst         |   8 +-
 drivers/net/mlx5/linux/mlx5_os.c | 160 +++++++++++++++++--------------
 drivers/net/mlx5/mlx5.c          |  22 +++++
 drivers/net/mlx5/mlx5.h          |   3 +-
 drivers/net/mlx5/mlx5_defs.h     |   4 -
 drivers/net/mlx5/mlx5_ethdev.c   |  27 ------
 drivers/net/mlx5/mlx5_mac.c      |   8 +-
 7 files changed, 122 insertions(+), 110 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index fc7e93842f..8a142e1d59 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -881,11 +881,15 @@ Driver options
 
   For instance, to probe VF port representors 0 through 2::
 
-    representor=vf[0-2]
+    <PCI_BDF>,representor=vf[0-2]
 
   To probe SF port representors 0 through 2::
 
-    representor=sf[0-2]
+    <PCI_BDF>,representor=sf[0-2]
+
+  To probe VF port representors 0 through 2 on both PFs of bonding device::
+
+    <Primary_PCI_BDF>,representor=pf[0,1]vf[0-2]
 
 - ``max_dump_files_num`` parameter [int]
 
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 1209bfed5b..6866381298 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -676,6 +676,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	struct mlx5dv_context dv_attr = { .comp_mask = 0 };
 	struct rte_eth_dev *eth_dev = NULL;
 	struct mlx5_priv *priv = NULL;
+	uint16_t repr_id = -1;
 	int err = 0;
 	unsigned int hw_padding = 0;
 	unsigned int mps;
@@ -693,11 +694,19 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	char name[RTE_ETH_NAME_MAX_LEN];
 	int own_domain_id = 0;
 	uint16_t port_id;
-	unsigned int i;
+	unsigned int c = 0, p = 0, f = 0;
 #ifdef HAVE_MLX5DV_DR_DEVX_PORT
 	struct mlx5dv_devx_port devx_port = { .comp_mask = 0 };
 #endif
 
+	if (switch_info->representor)
+		repr_id = rte_eth_representor_id_encode(
+			switch_info->ctrl_num,
+			spawn->pf_bond >= 0 ? switch_info->pf_num : 0,
+			switch_info->name_type ==
+				MLX5_PHYS_PORT_NAME_TYPE_PFSF ?
+				RTE_ETH_REPRESENTOR_SF : RTE_ETH_REPRESENTOR_VF,
+			switch_info->port_name);
 	/* Determine if this port representor is supposed to be spawned. */
 	if (switch_info->representor && dpdk_dev->devargs) {
 		switch (eth_da->type) {
@@ -729,51 +738,27 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				dpdk_dev->devargs->args);
 			return NULL;
 		}
-		/* Check controller ID: */
-		for (i = 0; i < eth_da->nb_mh_controllers; ++i)
-			if (eth_da->mh_controllers[i] ==
-			    (uint16_t)switch_info->ctrl_num)
-				break;
-		if (eth_da->nb_mh_controllers &&
-		    i == eth_da->nb_mh_controllers) {
-			rte_errno = EBUSY;
-			return NULL;
-		}
-		/* Check SF/VF ID: */
-		for (i = 0; i < eth_da->nb_representor_ports; ++i)
-			if (eth_da->representor_ports[i] ==
-			    (uint16_t)switch_info->port_name)
-				break;
-		if (eth_da->type != RTE_ETH_REPRESENTOR_PF &&
-		    i == eth_da->nb_representor_ports) {
-			rte_errno = EBUSY;
-			return NULL;
-		}
-		/* Check PF ID. Check after repr port to avoid warning flood. */
-		if (spawn->pf_bond >= 0) {
-			for (i = 0; i < eth_da->nb_ports; ++i)
-				if (eth_da->ports[i] ==
-				    (uint16_t)switch_info->pf_num)
-					break;
-			if (eth_da->nb_ports && i == eth_da->nb_ports) {
-				/* For backward compatibility, bonding
-				 * representor syntax supported with limitation,
-				 * device iterator won't find it:
-				 *    <PF1_BDF>,representor=#
-				 */
-				if (switch_info->pf_num > 0 &&
-				    eth_da->ports[0] == 0) {
-					DRV_LOG(WARNING, "Representor on Bonding PF should use pf#vf# format: %s",
-						dpdk_dev->devargs->args);
-				} else {
-					rte_errno = EBUSY;
-					return NULL;
+		/* Check representor ID: */
+		for (c = 0; c < eth_da->nb_mh_controllers; ++c) {
+			for (p = 0; p < eth_da->nb_ports; ++p) {
+				for (f = 0; f < eth_da->nb_representor_ports;
+				     ++f) {
+					uint16_t repr;
+
+					repr = rte_eth_representor_id_encode(
+						eth_da->mh_controllers[c],
+						eth_da->ports[p],
+						eth_da->type,
+						eth_da->representor_ports[f]);
+
+					if (repr_id == repr)
+						break;
 				}
 			}
-		} else if (eth_da->nb_ports > 1 || eth_da->ports[0]) {
-			rte_errno = EINVAL;
-			DRV_LOG(ERR, "PF id not supported by non-bond device: %s",
-				dpdk_dev->devargs->args);
+		}
+		if (c == eth_da->nb_mh_controllers && p == eth_da->nb_ports &&
+		    f == eth_da->nb_representor_ports) {
+			rte_errno = EBUSY;
 			return NULL;
 		}
 	}
@@ -790,17 +775,23 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				 switch_info->port_name);
 	} else {
 		/* Bonding device. */
-		if (!switch_info->representor)
+		if (!switch_info->representor) {
 			snprintf(name, sizeof(name), "%s_%s",
 				 dpdk_dev->name,
 				 mlx5_os_get_dev_device_name(spawn->phys_dev));
-		else
-			snprintf(name, sizeof(name), "%s_%s_representor_%s%u",
-				 dpdk_dev->name,
-				 mlx5_os_get_dev_device_name(spawn->phys_dev),
-				 switch_info->name_type ==
-				 MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf",
-				 switch_info->port_name);
+		} else {
+			err = snprintf(name, sizeof(name), "%s_%s_representor_c%dpf%d%s%u",
+				dpdk_dev->name,
+				mlx5_os_get_dev_device_name(spawn->phys_dev),
+				switch_info->ctrl_num,
+				switch_info->pf_num,
+				switch_info->name_type ==
+				MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf",
+				switch_info->port_name);
+			if (err >= (int)sizeof(name))
+				DRV_LOG(WARNING, "representor name overflow %s",
+					name);
+		}
 	}
 	/* check if the device is already spawned */
 	if (rte_eth_dev_get_port_by_name(name, &port_id) == 0) {
@@ -1079,11 +1070,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	priv->vport_id = switch_info->representor ?
 			 switch_info->port_name + 1 : -1;
 #endif
-	/* representor_id field keeps the unmodified VF index. */
-	priv->representor_id = switch_info->representor ?
-		rte_eth_representor_id_encode(0, 0, RTE_ETH_REPRESENTOR_VF,
-					      switch_info->port_name) :
-		-1;
+	priv->representor_id = repr_id;
 	/*
 	 * Look for sibling devices in order to reuse their switch domain
 	 * if any, otherwise allocate one.
@@ -1704,9 +1691,11 @@ mlx5_dev_spawn_data_cmp(const void *a, const void *b)
  * @param[in] ibv_dev
  *   Pointer to Infiniband device structure.
  * @param[in] pci_dev
- *   Pointer to PCI device structure to match PCI address.
+ *   Pointer to master PCI Address structure to match PCI address.
  * @param[in] nl_rdma
  *   Netlink RDMA group socket handle.
+ * @param[in] owner
+ *   Rerepsentor owner PF index.
  *
  * @return
  *   negative value if no bonding device found, otherwise
@@ -1714,8 +1703,8 @@ mlx5_dev_spawn_data_cmp(const void *a, const void *b)
  */
 static int
 mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev,
-			   const struct rte_pci_device *pci_dev,
-			   int nl_rdma)
+			   const struct rte_pci_addr *pci_dev,
+			   int nl_rdma, uint16_t owner)
 {
 	char ifname[IF_NAMESIZE + 1];
 	unsigned int ifindex;
@@ -1772,10 +1761,10 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev,
 					 " for netdev \"%s\"", ifname);
 			continue;
 		}
-		if (pci_dev->addr.domain != pci_addr.domain ||
-		    pci_dev->addr.bus != pci_addr.bus ||
-		    pci_dev->addr.devid != pci_addr.devid ||
-		    pci_dev->addr.function != pci_addr.function)
+		if (pci_dev->domain != pci_addr.domain ||
+		    pci_dev->bus != pci_addr.bus ||
+		    pci_dev->devid != pci_addr.devid ||
+		    pci_dev->function + owner != pci_addr.function)
 			continue;
 		/* Slave interface PCI address match found. */
 		fclose(file);
@@ -1843,7 +1832,8 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	struct mlx5_dev_config dev_config;
 	unsigned int dev_config_vf;
 	struct rte_eth_devargs eth_da = { .type = RTE_ETH_REPRESENTOR_NONE };
-	int ret;
+	struct rte_pci_addr probe_addr = pci_dev->addr;
+	int ret = -1;
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
 		mlx5_pmd_socket_init();
@@ -1895,7 +1885,8 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 
 		DRV_LOG(DEBUG, "checking device \"%s\"", ibv_list[ret]->name);
 		bd = mlx5_device_bond_pci_match
-				(ibv_list[ret], pci_dev, nl_rdma);
+				(ibv_list[ret], &probe_addr, nl_rdma,
+				 eth_da.ports[0]);
 		if (bd >= 0) {
 			/*
 			 * Bonding device detected. Only one match is allowed,
@@ -1912,6 +1903,9 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 				ret = -rte_errno;
 				goto exit;
 			}
+			/* Amend master pci address if owner PF specified. */
+			if (eth_da.nb_representor_ports)
+				probe_addr.function += eth_da.ports[0];
 			DRV_LOG(INFO, "PCI information matches for"
 				      " slave %d bonding device \"%s\"",
 				      bd, ibv_list[ret]->name);
@@ -1921,10 +1915,10 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (mlx5_dev_to_pci_addr
 			(ibv_list[ret]->ibdev_path, &pci_addr))
 			continue;
-		if (pci_dev->addr.domain != pci_addr.domain ||
-		    pci_dev->addr.bus != pci_addr.bus ||
-		    pci_dev->addr.devid != pci_addr.devid ||
-		    pci_dev->addr.function != pci_addr.function)
+		if (probe_addr.domain != pci_addr.domain ||
+		    probe_addr.bus != pci_addr.bus ||
+		    probe_addr.devid != pci_addr.devid ||
+		    probe_addr.function != pci_addr.function)
 			continue;
 		DRV_LOG(INFO, "PCI information matches for device \"%s\"",
 			ibv_list[ret]->name);
@@ -1936,8 +1930,8 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		DRV_LOG(WARNING,
 			"no Verbs device matches PCI device " PCI_PRI_FMT ","
 			" are kernel drivers loaded?",
-			pci_dev->addr.domain, pci_dev->addr.bus,
-			pci_dev->addr.devid, pci_dev->addr.function);
+			probe_addr.domain, probe_addr.bus,
+			probe_addr.devid, probe_addr.function);
 		rte_errno = ENOENT;
 		ret = -rte_errno;
 		goto exit;
@@ -2202,6 +2196,24 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		dev_config_vf = 0;
 		break;
 	}
+	if (pci_dev->device.devargs) {
+		/* Set devargs default values. */
+		if (eth_da.nb_mh_controllers == 0) {
+			eth_da.nb_mh_controllers = 1;
+			eth_da.mh_controllers[0] = 0;
+		}
+		if (eth_da.nb_ports == 0 && ns > 0) {
+			if (list[0].pf_bond >= 0 && list[0].info.representor)
+				DRV_LOG(WARNING, "Representor on Bonding device should use pf#vf# syntax: %s",
+					pci_dev->device.devargs->args);
+			eth_da.nb_ports = 1;
+			eth_da.ports[0] = list[0].info.pf_num;
+		}
+		if (eth_da.nb_representor_ports == 0) {
+			eth_da.nb_representor_ports = 1;
+			eth_da.representor_ports[0] = 0;
+		}
+	}
 	for (i = 0; i != ns; ++i) {
 		uint32_t restore;
 
@@ -2243,8 +2255,8 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		DRV_LOG(ERR,
 			"probe of PCI device " PCI_PRI_FMT " aborted after"
 			" encountering an error: %s",
-			pci_dev->addr.domain, pci_dev->addr.bus,
-			pci_dev->addr.devid, pci_dev->addr.function,
+			probe_addr.domain, probe_addr.bus,
+			probe_addr.devid, probe_addr.function,
 			strerror(rte_errno));
 		ret = -rte_errno;
 		/* Roll back. */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 628587faac..a8de42ff14 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -358,6 +358,28 @@ static const struct mlx5_indexed_pool_config mlx5_ipool_cfg[] = {
 
 #define MLX5_FLOW_TABLE_HLIST_ARRAY_SIZE 4096
 
+/**
+ * Decide whether representor ID is a HPF(host PF) port on BF2.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ * @return
+ *   Non-zero if HPF, otherwise 0.
+ */
+int
+mlx5_is_hpf(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	enum rte_eth_representor_type type;
+	uint16_t port;
+
+	port = rte_eth_representor_id_parse(priv->representor_id,
+					    NULL, NULL, &type);
+	return priv->representor && type == RTE_ETH_REPRESENTOR_VF &&
+	       port == rte_eth_representor_id_parse(-1, NULL, NULL, NULL);
+}
+
 /**
  * Initialize the ASO aging management structure.
  *
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index a77a1600d5..767dcfdc6d 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -945,7 +945,7 @@ struct mlx5_priv {
 	uint16_t vport_id; /* Associated VF vport index (if any). */
 	uint32_t vport_meta_tag; /* Used for vport index match ove VF LAG. */
 	uint32_t vport_meta_mask; /* Used for vport index field match mask. */
-	int32_t representor_id; /* Port representor identifier. */
+	int32_t representor_id; /* RTE_ETH_REPR(), -1 if not a representor. */
 	int32_t pf_bond; /* >=0 means PF index in bonding configuration. */
 	unsigned int if_index; /* Associated kernel network device index. */
 	uint32_t bond_ifindex; /**< Bond interface index. */
@@ -1019,6 +1019,7 @@ int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev,
 			      struct rte_eth_udp_tunnel *udp_tunnel);
 uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev);
 int mlx5_dev_close(struct rte_eth_dev *dev);
+int mlx5_is_hpf(struct rte_eth_dev *dev);
 void mlx5_age_event_prepare(struct mlx5_dev_ctx_shared *sh);
 
 /* Macro to iterate over all valid ports for mlx5 driver. */
diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h
index 85a0979653..4648196550 100644
--- a/drivers/net/mlx5/mlx5_defs.h
+++ b/drivers/net/mlx5/mlx5_defs.h
@@ -48,10 +48,6 @@
 #define MLX5_PMD_SOFT_COUNTERS 1
 #endif
 
-/* Switch port ID parameters for bonding configurations. */
-#define MLX5_PORT_ID_BONDING_PF_MASK 0xf
-#define MLX5_PORT_ID_BONDING_PF_SHIFT 12
-
 /* Alarm timeout. */
 #define MLX5_ALARM_TIMEOUT_US 100000
 
diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
index ad6aacc329..5341eb16c9 100644
--- a/drivers/net/mlx5/mlx5_ethdev.c
+++ b/drivers/net/mlx5/mlx5_ethdev.c
@@ -330,33 +330,6 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info)
 	if (priv->representor) {
 		uint16_t port_id;
 
-		if (priv->pf_bond >= 0) {
-			/*
-			 * Switch port ID is opaque value with driver defined
-			 * format. Push the PF index in bonding configurations
-			 * in upper four bits of port ID. If we get too many
-			 * representors (more than 4K) or PFs (more than 15)
-			 * this approach must be reconsidered.
-			 */
-			/* Switch port ID for VF representors: 0 - 0xFFE */
-			if ((info->switch_info.port_id != 0xffff &&
-				info->switch_info.port_id >=
-				((1 << MLX5_PORT_ID_BONDING_PF_SHIFT) - 1)) ||
-			    priv->pf_bond > MLX5_PORT_ID_BONDING_PF_MASK) {
-				DRV_LOG(ERR, "can't update switch port ID"
-					     " for bonding device");
-				MLX5_ASSERT(false);
-				return -ENODEV;
-			}
-			/*
-			 * Switch port ID for Host PF representor
-			 * (representor_id is -1) , set to 0xFFF
-			 */
-			if (info->switch_info.port_id == 0xffff)
-				info->switch_info.port_id = 0xfff;
-			info->switch_info.port_id |=
-				priv->pf_bond << MLX5_PORT_ID_BONDING_PF_SHIFT;
-		}
 		MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) {
 			struct mlx5_priv *opriv =
 				rte_eth_devices[port_id].data->dev_private;
diff --git a/drivers/net/mlx5/mlx5_mac.c b/drivers/net/mlx5/mlx5_mac.c
index bd786fd638..b5b810b508 100644
--- a/drivers/net/mlx5/mlx5_mac.c
+++ b/drivers/net/mlx5/mlx5_mac.c
@@ -159,7 +159,7 @@ mlx5_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr)
 	 * Configuring the VF instead of its representor,
 	 * need to skip the special case of HPF on Bluefield.
 	 */
-	if (priv->representor && priv->representor_id >= 0) {
+	if (priv->representor && !mlx5_is_hpf(dev)) {
 		DRV_LOG(DEBUG, "VF represented by port %u setting primary MAC address",
 			dev->data->port_id);
 		RTE_ETH_FOREACH_DEV_SIBLING(port_id, dev->data->port_id) {
@@ -169,7 +169,11 @@ mlx5_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr)
 				return mlx5_os_vf_mac_addr_modify
 				       (priv,
 					mlx5_ifindex(&rte_eth_devices[port_id]),
-					mac_addr, priv->representor_id);
+					mac_addr,
+					rte_eth_representor_id_parse(
+							priv->representor_id,
+							NULL, NULL, NULL)
+					);
 			}
 		}
 		rte_errno = -ENOTSUP;
-- 
2.25.1

Re: [dpdk-dev] [PATCH v2 1/9] ethdev: refactor representor infrastructure

[Somnath Kotur]

[-- Attachment #1: Type: text/plain, Size: 14450 bytes --]

On Wed, Jan 6, 2021 at 9:48 PM Xueming Li <xuemingl@nvidia.com> wrote:
>
> To support extended representor syntax, this patch refactor represntor
Typo in 'representor'
> infrastructure:
> 1. introduces representor type enum
> 2. devargs representor port range extraction from partial value
>
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> ---
>  drivers/net/bnxt/bnxt_ethdev.c        | 12 ++++
>  drivers/net/enic/enic_ethdev.c        |  7 ++
>  drivers/net/i40e/i40e_ethdev.c        |  8 +++
>  drivers/net/ixgbe/ixgbe_ethdev.c      |  8 +++
>  drivers/net/mlx5/linux/mlx5_os.c      | 11 ++++
>  lib/librte_ethdev/ethdev_private.c    | 93 ++++++++++++---------------
>  lib/librte_ethdev/ethdev_private.h    |  3 -
>  lib/librte_ethdev/rte_class_eth.c     |  4 +-
>  lib/librte_ethdev/rte_ethdev.c        |  5 +-
>  lib/librte_ethdev/rte_ethdev_driver.h |  7 ++
>  10 files changed, 98 insertions(+), 60 deletions(-)
>
> diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
> index 81c8f8d79d..844a6c3c66 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -5520,6 +5520,18 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
>         int i, ret = 0;
>         struct rte_kvargs *kvlist = NULL;
>
> +       if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
> +               return 0;
> +       if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
> +               PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
> +                           eth_da->type);
> +               return -ENOTSUP;
> +       }
> +       if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
Seems like an extra 'if ' condition by mistake? Otherwise there is no
diff b/n this 'if' condition and the one few lines above?
> +               PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
> +                           eth_da->type);
> +               return -EINVAL;
> +       }
>         num_rep = eth_da->nb_representor_ports;
>         if (num_rep > BNXT_MAX_VF_REPS) {
>                 PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
> diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
> index d041a6bee9..dd085caa93 100644
> --- a/drivers/net/enic/enic_ethdev.c
> +++ b/drivers/net/enic/enic_ethdev.c
> @@ -1303,6 +1303,13 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>                 if (retval)
>                         return retval;
>         }
> +       if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
> +               return 0;
> +       if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> +               ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
> +                           pci_dev->device.devargs->args);
> +               return -ENOTSUP;
> +       }
>         retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
>                 sizeof(struct enic),
>                 eth_dev_pci_specific_init, pci_dev,
> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index f54769c29d..05ed2e1079 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -640,6 +640,14 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>                         return retval;
>         }
>
> +       if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
> +               return 0;
> +       if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> +               PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
> +                           pci_dev->device.devargs->args);
> +               return -ENOTSUP;
> +       }
> +
>         retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
>                 sizeof(struct i40e_adapter),
>                 eth_dev_pci_specific_init, pci_dev,
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
> index 9a47a8b262..9ea0139197 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -1717,6 +1717,14 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>         } else
>                 memset(&eth_da, 0, sizeof(eth_da));
>
> +       if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
> +               return 0;
> +       if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> +               PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
> +                           pci_dev->device.devargs->args);
> +               return -ENOTSUP;
> +       }
> +
>         retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
>                 sizeof(struct ixgbe_adapter),
>                 eth_dev_pci_specific_init, pci_dev,
> diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
> index 6812a1f215..6981ba1f41 100644
> --- a/drivers/net/mlx5/linux/mlx5_os.c
> +++ b/drivers/net/mlx5/linux/mlx5_os.c
> @@ -706,6 +706,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
>                                 strerror(rte_errno));
>                         return NULL;
>                 }
> +               if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
> +                       /* Representor not specified. */
> +                       rte_errno = EBUSY;
> +                       return NULL;
> +               }
> +               if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> +                       rte_errno = ENOTSUP;
> +                       DRV_LOG(ERR, "unsupported representor type: %s",
> +                               dpdk_dev->devargs->args);
> +                       return NULL;
> +               }
>                 for (i = 0; i < eth_da.nb_representor_ports; ++i)
>                         if (eth_da.representor_ports[i] ==
>                             (uint16_t)switch_info->port_name)
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index 162a502fe7..c219164a4a 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -38,60 +38,13 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
>         return NULL;
>  }
>
> -int
> -rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
> -       void *data)
> -{
> -       char *str_start;
> -       int state;
> -       int result;
> -
> -       if (*str != '[')
> -               /* Single element, not a list */
> -               return callback(str, data);
> -
> -       /* Sanity check, then strip the brackets */
> -       str_start = &str[strlen(str) - 1];
> -       if (*str_start != ']') {
> -               RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
> -               return -EINVAL;
> -       }
> -       str++;
> -       *str_start = '\0';
> -
> -       /* Process list elements */
> -       state = 0;
> -       while (1) {
> -               if (state == 0) {
> -                       if (*str == '\0')
> -                               break;
> -                       if (*str != ',') {
> -                               str_start = str;
> -                               state = 1;
> -                       }
> -               } else if (state == 1) {
> -                       if (*str == ',' || *str == '\0') {
> -                               if (str > str_start) {
> -                                       /* Non-empty string fragment */
> -                                       *str = '\0';
> -                                       result = callback(str_start, data);
> -                                       if (result < 0)
> -                                               return result;
> -                               }
> -                               state = 0;
> -                       }
> -               }
> -               str++;
> -       }
> -       return 0;
> -}
> -
>  static int
>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>         const uint16_t max_list)
>  {
>         uint16_t lo, hi, val;
>         int result;
> +       char *pos = str;
>
>         result = sscanf(str, "%hu-%hu", &lo, &hi);
>         if (result == 1) {
> @@ -99,7 +52,7 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>                         return -ENOMEM;
>                 list[(*len_list)++] = lo;
>         } else if (result == 2) {
> -               if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
> +               if (lo >= hi)
>                         return -EINVAL;
>                 for (val = lo; val <= hi; val++) {
>                         if (*len_list >= max_list)
> @@ -108,14 +61,52 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>                 }
>         } else
>                 return -EINVAL;
> -       return 0;
> +       while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
> +               pos++;
> +       return pos - str;
>  }
>
> +static int
> +rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
> +       const uint16_t max_list)
> +{
> +       char *pos = str;
> +       int ret;
> +
> +       if (*pos == '[')
> +               pos++;
> +       while (1) {
> +               ret = rte_eth_devargs_process_range(pos, list, len_list,
> +                                                   max_list);
> +               if (ret < 0)
> +                       return ret;
> +               pos += ret;
> +               if (*pos != ',') /* end of list */
> +                       break;
> +               pos++;
> +       }
> +       if (*str == '[' && *pos != ']')
> +               return -EINVAL;
> +       if (*pos == ']')
> +               pos++;
> +       return pos - str;
> +}
> +
> +/*
> + * representor format:
> + *   #: range or single number of VF representor - legacy
> + */
>  int
>  rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  {
>         struct rte_eth_devargs *eth_da = data;
> +       int ret;
>
> -       return rte_eth_devargs_process_range(str, eth_da->representor_ports,
> +       /* Number # alone implies VF */
> +       eth_da->type = RTE_ETH_REPRESENTOR_VF;
> +       ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>                 &eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
> +       if (ret < 0)
> +               RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
> +       return ret < 0 ? ret : 0;
>  }
> diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
> index 905a45c337..220ddd4408 100644
> --- a/lib/librte_ethdev/ethdev_private.h
> +++ b/lib/librte_ethdev/ethdev_private.h
> @@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
>                 const void *data);
>
>  /* Parse devargs value for representor parameter. */
> -typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
> -int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
> -       void *data);
>  int rte_eth_devargs_parse_representor_ports(char *str, void *data);
>
>  #ifdef __cplusplus
> diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
> index 6338355e25..efe6149df5 100644
> --- a/lib/librte_ethdev/rte_class_eth.c
> +++ b/lib/librte_ethdev/rte_class_eth.c
> @@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
>         if (values == NULL)
>                 return -1;
>         memset(&representors, 0, sizeof(representors));
> -       ret = rte_eth_devargs_parse_list(values,
> -                       rte_eth_devargs_parse_representor_ports,
> -                       &representors);
> +       ret = rte_eth_devargs_parse_representor_ports(values, &representors);
>         free(values);
>         if (ret != 0)
>                 return -1; /* invalid devargs value */
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 17ddacc78d..2ac51ac149 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -5542,9 +5542,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>         for (i = 0; i < args.count; i++) {
>                 pair = &args.pairs[i];
>                 if (strcmp("representor", pair->key) == 0) {
> -                       result = rte_eth_devargs_parse_list(pair->value,
> -                               rte_eth_devargs_parse_representor_ports,
> -                               eth_da);
> +                       result = rte_eth_devargs_parse_representor_ports(
> +                                       pair->value, eth_da);
>                         if (result < 0)
>                                 goto parse_cleanup;
>                 }
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index 0eacfd8425..b66a955b18 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -1193,6 +1193,12 @@ __rte_internal
>  int
>  rte_eth_switch_domain_free(uint16_t domain_id);
>
> +/** Ethernet device representor type */
> +enum rte_eth_representor_type {
> +       RTE_ETH_REPRESENTOR_NONE, /* not a representor */
> +       RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
> +};
> +
>  /** Generic Ethernet device arguments  */
>  struct rte_eth_devargs {
>         uint16_t ports[RTE_MAX_ETHPORTS];
> @@ -1203,6 +1209,7 @@ struct rte_eth_devargs {
>         /** representor port/s identifier to enable on device */
>         uint16_t nb_representor_ports;
>         /** number of ports in representor port field */
> +       enum rte_eth_representor_type type; /* type of representor */
>  };
>
>  /**
> --
> 2.25.1
>

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

Re: [dpdk-dev] [PATCH v2 1/9] ethdev: refactor representor infrastructure

[Xueming(Steven) Li]

>-----Original Message-----
>From: Somnath Kotur <somnath.kotur@broadcom.com>
>Sent: Thursday, January 7, 2021 2:32 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
><ferruh.yigit@intel.com>; Andrew Rybchenko
><andrew.rybchenko@oktetlabs.ru>; Olivier Matz <olivier.matz@6wind.com>;
>Slava Ovsiienko <viacheslavo@nvidia.com>; dev <dev@dpdk.org>; Asaf Penso
><asafp@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v2 1/9] ethdev: refactor representor
>infrastructure
>
>On Wed, Jan 6, 2021 at 9:48 PM Xueming Li <xuemingl@nvidia.com> wrote:
>>
>> To support extended representor syntax, this patch refactor represntor
>Typo in 'representor'
>> infrastructure:
>> 1. introduces representor type enum
>> 2. devargs representor port range extraction from partial value
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> ---
>>  drivers/net/bnxt/bnxt_ethdev.c        | 12 ++++
>>  drivers/net/enic/enic_ethdev.c        |  7 ++
>>  drivers/net/i40e/i40e_ethdev.c        |  8 +++
>>  drivers/net/ixgbe/ixgbe_ethdev.c      |  8 +++
>>  drivers/net/mlx5/linux/mlx5_os.c      | 11 ++++
>>  lib/librte_ethdev/ethdev_private.c    | 93 ++++++++++++---------------
>>  lib/librte_ethdev/ethdev_private.h    |  3 -
>>  lib/librte_ethdev/rte_class_eth.c     |  4 +-
>>  lib/librte_ethdev/rte_ethdev.c        |  5 +-
>>  lib/librte_ethdev/rte_ethdev_driver.h |  7 ++
>>  10 files changed, 98 insertions(+), 60 deletions(-)
>>
>> diff --git a/drivers/net/bnxt/bnxt_ethdev.c
>b/drivers/net/bnxt/bnxt_ethdev.c
>> index 81c8f8d79d..844a6c3c66 100644
>> --- a/drivers/net/bnxt/bnxt_ethdev.c
>> +++ b/drivers/net/bnxt/bnxt_ethdev.c
>> @@ -5520,6 +5520,18 @@ static int bnxt_rep_port_probe(struct
>rte_pci_device *pci_dev,
>>         int i, ret = 0;
>>         struct rte_kvargs *kvlist = NULL;
>>
>> +       if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
>> +               return 0;
>> +       if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
>> +               PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
>> +                           eth_da->type);
>> +               return -ENOTSUP;
>> +       }
>> +       if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
>Seems like an extra 'if ' condition by mistake? Otherwise there is no
>diff b/n this 'if' condition and the one few lines above?

Thanks, good catch!

>> +               PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
>> +                           eth_da->type);
>> +               return -EINVAL;
>> +       }
>>         num_rep = eth_da->nb_representor_ports;
>>         if (num_rep > BNXT_MAX_VF_REPS) {
>>                 PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF
>REPS\n",
>> diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
>> index d041a6bee9..dd085caa93 100644
>> --- a/drivers/net/enic/enic_ethdev.c
>> +++ b/drivers/net/enic/enic_ethdev.c
>> @@ -1303,6 +1303,13 @@ static int eth_enic_pci_probe(struct
>rte_pci_driver *pci_drv __rte_unused,
>>                 if (retval)
>>                         return retval;
>>         }
>> +       if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
>> +               return 0;
>> +       if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
>> +               ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
>> +                           pci_dev->device.devargs->args);
>> +               return -ENOTSUP;
>> +       }
>>         retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
>>                 sizeof(struct enic),
>>                 eth_dev_pci_specific_init, pci_dev,
>> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
>> index f54769c29d..05ed2e1079 100644
>> --- a/drivers/net/i40e/i40e_ethdev.c
>> +++ b/drivers/net/i40e/i40e_ethdev.c
>> @@ -640,6 +640,14 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv
>__rte_unused,
>>                         return retval;
>>         }
>>
>> +       if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
>> +               return 0;
>> +       if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
>> +               PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
>> +                           pci_dev->device.devargs->args);
>> +               return -ENOTSUP;
>> +       }
>> +
>>         retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
>>                 sizeof(struct i40e_adapter),
>>                 eth_dev_pci_specific_init, pci_dev,
>> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
>b/drivers/net/ixgbe/ixgbe_ethdev.c
>> index 9a47a8b262..9ea0139197 100644
>> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
>> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
>> @@ -1717,6 +1717,14 @@ eth_ixgbe_pci_probe(struct rte_pci_driver
>*pci_drv __rte_unused,
>>         } else
>>                 memset(&eth_da, 0, sizeof(eth_da));
>>
>> +       if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
>> +               return 0;
>> +       if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
>> +               PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
>> +                           pci_dev->device.devargs->args);
>> +               return -ENOTSUP;
>> +       }
>> +
>>         retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
>>                 sizeof(struct ixgbe_adapter),
>>                 eth_dev_pci_specific_init, pci_dev,
>> diff --git a/drivers/net/mlx5/linux/mlx5_os.c
>b/drivers/net/mlx5/linux/mlx5_os.c
>> index 6812a1f215..6981ba1f41 100644
>> --- a/drivers/net/mlx5/linux/mlx5_os.c
>> +++ b/drivers/net/mlx5/linux/mlx5_os.c
>> @@ -706,6 +706,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
>>                                 strerror(rte_errno));
>>                         return NULL;
>>                 }
>> +               if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
>> +                       /* Representor not specified. */
>> +                       rte_errno = EBUSY;
>> +                       return NULL;
>> +               }
>> +               if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
>> +                       rte_errno = ENOTSUP;
>> +                       DRV_LOG(ERR, "unsupported representor type: %s",
>> +                               dpdk_dev->devargs->args);
>> +                       return NULL;
>> +               }
>>                 for (i = 0; i < eth_da.nb_representor_ports; ++i)
>>                         if (eth_da.representor_ports[i] ==
>>                             (uint16_t)switch_info->port_name)
>> diff --git a/lib/librte_ethdev/ethdev_private.c
>b/lib/librte_ethdev/ethdev_private.c
>> index 162a502fe7..c219164a4a 100644
>> --- a/lib/librte_ethdev/ethdev_private.c
>> +++ b/lib/librte_ethdev/ethdev_private.c
>> @@ -38,60 +38,13 @@ eth_find_device(const struct rte_eth_dev *start,
>rte_eth_cmp_t cmp,
>>         return NULL;
>>  }
>>
>> -int
>> -rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
>> -       void *data)
>> -{
>> -       char *str_start;
>> -       int state;
>> -       int result;
>> -
>> -       if (*str != '[')
>> -               /* Single element, not a list */
>> -               return callback(str, data);
>> -
>> -       /* Sanity check, then strip the brackets */
>> -       str_start = &str[strlen(str) - 1];
>> -       if (*str_start != ']') {
>> -               RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
>> -               return -EINVAL;
>> -       }
>> -       str++;
>> -       *str_start = '\0';
>> -
>> -       /* Process list elements */
>> -       state = 0;
>> -       while (1) {
>> -               if (state == 0) {
>> -                       if (*str == '\0')
>> -                               break;
>> -                       if (*str != ',') {
>> -                               str_start = str;
>> -                               state = 1;
>> -                       }
>> -               } else if (state == 1) {
>> -                       if (*str == ',' || *str == '\0') {
>> -                               if (str > str_start) {
>> -                                       /* Non-empty string fragment */
>> -                                       *str = '\0';
>> -                                       result = callback(str_start, data);
>> -                                       if (result < 0)
>> -                                               return result;
>> -                               }
>> -                               state = 0;
>> -                       }
>> -               }
>> -               str++;
>> -       }
>> -       return 0;
>> -}
>> -
>>  static int
>>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>>         const uint16_t max_list)
>>  {
>>         uint16_t lo, hi, val;
>>         int result;
>> +       char *pos = str;
>>
>>         result = sscanf(str, "%hu-%hu", &lo, &hi);
>>         if (result == 1) {
>> @@ -99,7 +52,7 @@ rte_eth_devargs_process_range(char *str, uint16_t
>*list, uint16_t *len_list,
>>                         return -ENOMEM;
>>                 list[(*len_list)++] = lo;
>>         } else if (result == 2) {
>> -               if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
>> +               if (lo >= hi)
>>                         return -EINVAL;
>>                 for (val = lo; val <= hi; val++) {
>>                         if (*len_list >= max_list)
>> @@ -108,14 +61,52 @@ rte_eth_devargs_process_range(char *str, uint16_t
>*list, uint16_t *len_list,
>>                 }
>>         } else
>>                 return -EINVAL;
>> -       return 0;
>> +       while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
>> +               pos++;
>> +       return pos - str;
>>  }
>>
>> +static int
>> +rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>> +       const uint16_t max_list)
>> +{
>> +       char *pos = str;
>> +       int ret;
>> +
>> +       if (*pos == '[')
>> +               pos++;
>> +       while (1) {
>> +               ret = rte_eth_devargs_process_range(pos, list, len_list,
>> +                                                   max_list);
>> +               if (ret < 0)
>> +                       return ret;
>> +               pos += ret;
>> +               if (*pos != ',') /* end of list */
>> +                       break;
>> +               pos++;
>> +       }
>> +       if (*str == '[' && *pos != ']')
>> +               return -EINVAL;
>> +       if (*pos == ']')
>> +               pos++;
>> +       return pos - str;
>> +}
>> +
>> +/*
>> + * representor format:
>> + *   #: range or single number of VF representor - legacy
>> + */
>>  int
>>  rte_eth_devargs_parse_representor_ports(char *str, void *data)
>>  {
>>         struct rte_eth_devargs *eth_da = data;
>> +       int ret;
>>
>> -       return rte_eth_devargs_process_range(str, eth_da->representor_ports,
>> +       /* Number # alone implies VF */
>> +       eth_da->type = RTE_ETH_REPRESENTOR_VF;
>> +       ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>>                 &eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
>> +       if (ret < 0)
>> +               RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
>> +       return ret < 0 ? ret : 0;
>>  }
>> diff --git a/lib/librte_ethdev/ethdev_private.h
>b/lib/librte_ethdev/ethdev_private.h
>> index 905a45c337..220ddd4408 100644
>> --- a/lib/librte_ethdev/ethdev_private.h
>> +++ b/lib/librte_ethdev/ethdev_private.h
>> @@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start,
>rte_eth_cmp_t cmp,
>>                 const void *data);
>>
>>  /* Parse devargs value for representor parameter. */
>> -typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
>> -int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t
>callback,
>> -       void *data);
>>  int rte_eth_devargs_parse_representor_ports(char *str, void *data);
>>
>>  #ifdef __cplusplus
>> diff --git a/lib/librte_ethdev/rte_class_eth.c
>b/lib/librte_ethdev/rte_class_eth.c
>> index 6338355e25..efe6149df5 100644
>> --- a/lib/librte_ethdev/rte_class_eth.c
>> +++ b/lib/librte_ethdev/rte_class_eth.c
>> @@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
>>         if (values == NULL)
>>                 return -1;
>>         memset(&representors, 0, sizeof(representors));
>> -       ret = rte_eth_devargs_parse_list(values,
>> -                       rte_eth_devargs_parse_representor_ports,
>> -                       &representors);
>> +       ret = rte_eth_devargs_parse_representor_ports(values, &representors);
>>         free(values);
>>         if (ret != 0)
>>                 return -1; /* invalid devargs value */
>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>> index 17ddacc78d..2ac51ac149 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -5542,9 +5542,8 @@ rte_eth_devargs_parse(const char *dargs, struct
>rte_eth_devargs *eth_da)
>>         for (i = 0; i < args.count; i++) {
>>                 pair = &args.pairs[i];
>>                 if (strcmp("representor", pair->key) == 0) {
>> -                       result = rte_eth_devargs_parse_list(pair->value,
>> -                               rte_eth_devargs_parse_representor_ports,
>> -                               eth_da);
>> +                       result = rte_eth_devargs_parse_representor_ports(
>> +                                       pair->value, eth_da);
>>                         if (result < 0)
>>                                 goto parse_cleanup;
>>                 }
>> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
>b/lib/librte_ethdev/rte_ethdev_driver.h
>> index 0eacfd8425..b66a955b18 100644
>> --- a/lib/librte_ethdev/rte_ethdev_driver.h
>> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
>> @@ -1193,6 +1193,12 @@ __rte_internal
>>  int
>>  rte_eth_switch_domain_free(uint16_t domain_id);
>>
>> +/** Ethernet device representor type */
>> +enum rte_eth_representor_type {
>> +       RTE_ETH_REPRESENTOR_NONE, /* not a representor */
>> +       RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
>> +};
>> +
>>  /** Generic Ethernet device arguments  */
>>  struct rte_eth_devargs {
>>         uint16_t ports[RTE_MAX_ETHPORTS];
>> @@ -1203,6 +1209,7 @@ struct rte_eth_devargs {
>>         /** representor port/s identifier to enable on device */
>>         uint16_t nb_representor_ports;
>>         /** number of ports in representor port field */
>> +       enum rte_eth_representor_type type; /* type of representor */
>>  };
>>
>>  /**
>> --
>> 2.25.1
>>
>
>--
>This electronic communication and the information and any files transmitted
>with it, or attached to it, are confidential and are intended solely for
>the use of the individual or entity to whom it is addressed and may contain
>information that is confidential, legally privileged, protected by privacy
>laws, or otherwise restricted from disclosure to anyone else. If you are
>not the intended recipient or the person responsible for delivering the
>e-mail to the intended recipient, you are hereby notified that any use,
>copying, distributing, dissemination, forwarding, printing, or copying of
>this e-mail is strictly prohibited. If you received this e-mail in error,
>please return the e-mail to the sender, delete it from your computer, and
>destroy any printed copy of it.

Re: [dpdk-dev] [PATCH v2 1/9] ethdev: refactor representor infrastructure

[Somnath Kotur]

[-- Attachment #1: Type: text/plain, Size: 17001 bytes --]

On Thu, Jan 7, 2021 at 12:08 PM Xueming(Steven) Li <xuemingl@nvidia.com> wrote:
>
>
>
> >-----Original Message-----
> >From: Somnath Kotur <somnath.kotur@broadcom.com>
> >Sent: Thursday, January 7, 2021 2:32 PM
> >To: Xueming(Steven) Li <xuemingl@nvidia.com>
> >Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
> ><ferruh.yigit@intel.com>; Andrew Rybchenko
> ><andrew.rybchenko@oktetlabs.ru>; Olivier Matz <olivier.matz@6wind.com>;
> >Slava Ovsiienko <viacheslavo@nvidia.com>; dev <dev@dpdk.org>; Asaf Penso
> ><asafp@nvidia.com>
> >Subject: Re: [dpdk-dev] [PATCH v2 1/9] ethdev: refactor representor
> >infrastructure
> >
> >On Wed, Jan 6, 2021 at 9:48 PM Xueming Li <xuemingl@nvidia.com> wrote:
> >>
> >> To support extended representor syntax, this patch refactor represntor
Please fix this Typo in 'representor' as well ...Thanks
> >> infrastructure:
> >> 1. introduces representor type enum
> >> 2. devargs representor port range extraction from partial value
> >>
> >> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> >> ---
> >>  drivers/net/bnxt/bnxt_ethdev.c        | 12 ++++
> >>  drivers/net/enic/enic_ethdev.c        |  7 ++
> >>  drivers/net/i40e/i40e_ethdev.c        |  8 +++
> >>  drivers/net/ixgbe/ixgbe_ethdev.c      |  8 +++
> >>  drivers/net/mlx5/linux/mlx5_os.c      | 11 ++++
> >>  lib/librte_ethdev/ethdev_private.c    | 93 ++++++++++++---------------
> >>  lib/librte_ethdev/ethdev_private.h    |  3 -
> >>  lib/librte_ethdev/rte_class_eth.c     |  4 +-
> >>  lib/librte_ethdev/rte_ethdev.c        |  5 +-
> >>  lib/librte_ethdev/rte_ethdev_driver.h |  7 ++
> >>  10 files changed, 98 insertions(+), 60 deletions(-)
> >>
> >> diff --git a/drivers/net/bnxt/bnxt_ethdev.c
> >b/drivers/net/bnxt/bnxt_ethdev.c
> >> index 81c8f8d79d..844a6c3c66 100644
> >> --- a/drivers/net/bnxt/bnxt_ethdev.c
> >> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> >> @@ -5520,6 +5520,18 @@ static int bnxt_rep_port_probe(struct
> >rte_pci_device *pci_dev,
> >>         int i, ret = 0;
> >>         struct rte_kvargs *kvlist = NULL;
> >>
> >> +       if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
> >> +               return 0;
> >> +       if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
> >> +               PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
> >> +                           eth_da->type);
> >> +               return -ENOTSUP;
> >> +       }
> >> +       if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
> >Seems like an extra 'if ' condition by mistake? Otherwise there is no
> >diff b/n this 'if' condition and the one few lines above?
>
> Thanks, good catch!
>
> >> +               PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
> >> +                           eth_da->type);
> >> +               return -EINVAL;
> >> +       }
> >>         num_rep = eth_da->nb_representor_ports;
> >>         if (num_rep > BNXT_MAX_VF_REPS) {
> >>                 PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF
> >REPS\n",
> >> diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
> >> index d041a6bee9..dd085caa93 100644
> >> --- a/drivers/net/enic/enic_ethdev.c
> >> +++ b/drivers/net/enic/enic_ethdev.c
> >> @@ -1303,6 +1303,13 @@ static int eth_enic_pci_probe(struct
> >rte_pci_driver *pci_drv __rte_unused,
> >>                 if (retval)
> >>                         return retval;
> >>         }
> >> +       if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
> >> +               return 0;
> >> +       if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> >> +               ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
> >> +                           pci_dev->device.devargs->args);
> >> +               return -ENOTSUP;
> >> +       }
> >>         retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
> >>                 sizeof(struct enic),
> >>                 eth_dev_pci_specific_init, pci_dev,
> >> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> >> index f54769c29d..05ed2e1079 100644
> >> --- a/drivers/net/i40e/i40e_ethdev.c
> >> +++ b/drivers/net/i40e/i40e_ethdev.c
> >> @@ -640,6 +640,14 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv
> >__rte_unused,
> >>                         return retval;
> >>         }
> >>
> >> +       if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
> >> +               return 0;
> >> +       if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> >> +               PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
> >> +                           pci_dev->device.devargs->args);
> >> +               return -ENOTSUP;
> >> +       }
> >> +
> >>         retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
> >>                 sizeof(struct i40e_adapter),
> >>                 eth_dev_pci_specific_init, pci_dev,
> >> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> >b/drivers/net/ixgbe/ixgbe_ethdev.c
> >> index 9a47a8b262..9ea0139197 100644
> >> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> >> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> >> @@ -1717,6 +1717,14 @@ eth_ixgbe_pci_probe(struct rte_pci_driver
> >*pci_drv __rte_unused,
> >>         } else
> >>                 memset(&eth_da, 0, sizeof(eth_da));
> >>
> >> +       if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
> >> +               return 0;
> >> +       if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> >> +               PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
> >> +                           pci_dev->device.devargs->args);
> >> +               return -ENOTSUP;
> >> +       }
> >> +
> >>         retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
> >>                 sizeof(struct ixgbe_adapter),
> >>                 eth_dev_pci_specific_init, pci_dev,
> >> diff --git a/drivers/net/mlx5/linux/mlx5_os.c
> >b/drivers/net/mlx5/linux/mlx5_os.c
> >> index 6812a1f215..6981ba1f41 100644
> >> --- a/drivers/net/mlx5/linux/mlx5_os.c
> >> +++ b/drivers/net/mlx5/linux/mlx5_os.c
> >> @@ -706,6 +706,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
> >>                                 strerror(rte_errno));
> >>                         return NULL;
> >>                 }
> >> +               if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
> >> +                       /* Representor not specified. */
> >> +                       rte_errno = EBUSY;
> >> +                       return NULL;
> >> +               }
> >> +               if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> >> +                       rte_errno = ENOTSUP;
> >> +                       DRV_LOG(ERR, "unsupported representor type: %s",
> >> +                               dpdk_dev->devargs->args);
> >> +                       return NULL;
> >> +               }
> >>                 for (i = 0; i < eth_da.nb_representor_ports; ++i)
> >>                         if (eth_da.representor_ports[i] ==
> >>                             (uint16_t)switch_info->port_name)
> >> diff --git a/lib/librte_ethdev/ethdev_private.c
> >b/lib/librte_ethdev/ethdev_private.c
> >> index 162a502fe7..c219164a4a 100644
> >> --- a/lib/librte_ethdev/ethdev_private.c
> >> +++ b/lib/librte_ethdev/ethdev_private.c
> >> @@ -38,60 +38,13 @@ eth_find_device(const struct rte_eth_dev *start,
> >rte_eth_cmp_t cmp,
> >>         return NULL;
> >>  }
> >>
> >> -int
> >> -rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
> >> -       void *data)
> >> -{
> >> -       char *str_start;
> >> -       int state;
> >> -       int result;
> >> -
> >> -       if (*str != '[')
> >> -               /* Single element, not a list */
> >> -               return callback(str, data);
> >> -
> >> -       /* Sanity check, then strip the brackets */
> >> -       str_start = &str[strlen(str) - 1];
> >> -       if (*str_start != ']') {
> >> -               RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
> >> -               return -EINVAL;
> >> -       }
> >> -       str++;
> >> -       *str_start = '\0';
> >> -
> >> -       /* Process list elements */
> >> -       state = 0;
> >> -       while (1) {
> >> -               if (state == 0) {
> >> -                       if (*str == '\0')
> >> -                               break;
> >> -                       if (*str != ',') {
> >> -                               str_start = str;
> >> -                               state = 1;
> >> -                       }
> >> -               } else if (state == 1) {
> >> -                       if (*str == ',' || *str == '\0') {
> >> -                               if (str > str_start) {
> >> -                                       /* Non-empty string fragment */
> >> -                                       *str = '\0';
> >> -                                       result = callback(str_start, data);
> >> -                                       if (result < 0)
> >> -                                               return result;
> >> -                               }
> >> -                               state = 0;
> >> -                       }
> >> -               }
> >> -               str++;
> >> -       }
> >> -       return 0;
> >> -}
> >> -
> >>  static int
> >>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
> >>         const uint16_t max_list)
> >>  {
> >>         uint16_t lo, hi, val;
> >>         int result;
> >> +       char *pos = str;
> >>
> >>         result = sscanf(str, "%hu-%hu", &lo, &hi);
> >>         if (result == 1) {
> >> @@ -99,7 +52,7 @@ rte_eth_devargs_process_range(char *str, uint16_t
> >*list, uint16_t *len_list,
> >>                         return -ENOMEM;
> >>                 list[(*len_list)++] = lo;
> >>         } else if (result == 2) {
> >> -               if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
> >> +               if (lo >= hi)
> >>                         return -EINVAL;
> >>                 for (val = lo; val <= hi; val++) {
> >>                         if (*len_list >= max_list)
> >> @@ -108,14 +61,52 @@ rte_eth_devargs_process_range(char *str, uint16_t
> >*list, uint16_t *len_list,
> >>                 }
> >>         } else
> >>                 return -EINVAL;
> >> -       return 0;
> >> +       while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
> >> +               pos++;
> >> +       return pos - str;
> >>  }
> >>
> >> +static int
> >> +rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
> >> +       const uint16_t max_list)
> >> +{
> >> +       char *pos = str;
> >> +       int ret;
> >> +
> >> +       if (*pos == '[')
> >> +               pos++;
> >> +       while (1) {
> >> +               ret = rte_eth_devargs_process_range(pos, list, len_list,
> >> +                                                   max_list);
> >> +               if (ret < 0)
> >> +                       return ret;
> >> +               pos += ret;
> >> +               if (*pos != ',') /* end of list */
> >> +                       break;
> >> +               pos++;
> >> +       }
> >> +       if (*str == '[' && *pos != ']')
> >> +               return -EINVAL;
> >> +       if (*pos == ']')
> >> +               pos++;
> >> +       return pos - str;
> >> +}
> >> +
> >> +/*
> >> + * representor format:
> >> + *   #: range or single number of VF representor - legacy
> >> + */
> >>  int
> >>  rte_eth_devargs_parse_representor_ports(char *str, void *data)
> >>  {
> >>         struct rte_eth_devargs *eth_da = data;
> >> +       int ret;
> >>
> >> -       return rte_eth_devargs_process_range(str, eth_da->representor_ports,
> >> +       /* Number # alone implies VF */
> >> +       eth_da->type = RTE_ETH_REPRESENTOR_VF;
> >> +       ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
> >>                 &eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
> >> +       if (ret < 0)
> >> +               RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
> >> +       return ret < 0 ? ret : 0;
> >>  }
> >> diff --git a/lib/librte_ethdev/ethdev_private.h
> >b/lib/librte_ethdev/ethdev_private.h
> >> index 905a45c337..220ddd4408 100644
> >> --- a/lib/librte_ethdev/ethdev_private.h
> >> +++ b/lib/librte_ethdev/ethdev_private.h
> >> @@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start,
> >rte_eth_cmp_t cmp,
> >>                 const void *data);
> >>
> >>  /* Parse devargs value for representor parameter. */
> >> -typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
> >> -int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t
> >callback,
> >> -       void *data);
> >>  int rte_eth_devargs_parse_representor_ports(char *str, void *data);
> >>
> >>  #ifdef __cplusplus
> >> diff --git a/lib/librte_ethdev/rte_class_eth.c
> >b/lib/librte_ethdev/rte_class_eth.c
> >> index 6338355e25..efe6149df5 100644
> >> --- a/lib/librte_ethdev/rte_class_eth.c
> >> +++ b/lib/librte_ethdev/rte_class_eth.c
> >> @@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
> >>         if (values == NULL)
> >>                 return -1;
> >>         memset(&representors, 0, sizeof(representors));
> >> -       ret = rte_eth_devargs_parse_list(values,
> >> -                       rte_eth_devargs_parse_representor_ports,
> >> -                       &representors);
> >> +       ret = rte_eth_devargs_parse_representor_ports(values, &representors);
> >>         free(values);
> >>         if (ret != 0)
> >>                 return -1; /* invalid devargs value */
> >> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> >> index 17ddacc78d..2ac51ac149 100644
> >> --- a/lib/librte_ethdev/rte_ethdev.c
> >> +++ b/lib/librte_ethdev/rte_ethdev.c
> >> @@ -5542,9 +5542,8 @@ rte_eth_devargs_parse(const char *dargs, struct
> >rte_eth_devargs *eth_da)
> >>         for (i = 0; i < args.count; i++) {
> >>                 pair = &args.pairs[i];
> >>                 if (strcmp("representor", pair->key) == 0) {
> >> -                       result = rte_eth_devargs_parse_list(pair->value,
> >> -                               rte_eth_devargs_parse_representor_ports,
> >> -                               eth_da);
> >> +                       result = rte_eth_devargs_parse_representor_ports(
> >> +                                       pair->value, eth_da);
> >>                         if (result < 0)
> >>                                 goto parse_cleanup;
> >>                 }
> >> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
> >b/lib/librte_ethdev/rte_ethdev_driver.h
> >> index 0eacfd8425..b66a955b18 100644
> >> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> >> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> >> @@ -1193,6 +1193,12 @@ __rte_internal
> >>  int
> >>  rte_eth_switch_domain_free(uint16_t domain_id);
> >>
> >> +/** Ethernet device representor type */
> >> +enum rte_eth_representor_type {
> >> +       RTE_ETH_REPRESENTOR_NONE, /* not a representor */
> >> +       RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
> >> +};
> >> +
> >>  /** Generic Ethernet device arguments  */
> >>  struct rte_eth_devargs {
> >>         uint16_t ports[RTE_MAX_ETHPORTS];
> >> @@ -1203,6 +1209,7 @@ struct rte_eth_devargs {
> >>         /** representor port/s identifier to enable on device */
> >>         uint16_t nb_representor_ports;
> >>         /** number of ports in representor port field */
> >> +       enum rte_eth_representor_type type; /* type of representor */
> >>  };
> >>
> >>  /**
> >> --
> >> 2.25.1
> >>
> >
> >--
> >This electronic communication and the information and any files transmitted
> >with it, or attached to it, are confidential and are intended solely for
> >the use of the individual or entity to whom it is addressed and may contain
> >information that is confidential, legally privileged, protected by privacy
> >laws, or otherwise restricted from disclosure to anyone else. If you are
> >not the intended recipient or the person responsible for delivering the
> >e-mail to the intended recipient, you are hereby notified that any use,
> >copying, distributing, dissemination, forwarding, printing, or copying of
> >this e-mail is strictly prohibited. If you received this e-mail in error,
> >please return the e-mail to the sender, delete it from your computer, and
> >destroy any printed copy of it.

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

[dpdk-dev] [PATCH v3 0/9] ethdev: support SubFunction representor

[Xueming Li]

SubFunction [1] is a portion of the PCI device, a SF netdev has its own
dedicated queues(txq, rxq). A SF netdev supports eswitch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to existing
VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For backward compatibility, this patch also introduces new netdev
capability to indicate the capability of supportting SF representor.

Version history:
 RFC:
 	initial version [2]
 V2:
    - separate patch for represnetor infrastructure, controller, pf and
      sf.
    - replace representor ID macro with functions:
      rte_eth_representor_id_encode()
      rte_eth_representor_id_parse()
    - new patch to allow devargs with same PCI BDF but different
      representors.
    - other minor code updates according to comments, thanks Andrew!
    - update document
 V3:
    - improve probing of allowed devargs with same name.
    - parse single word of kvargs as key. 
    - update kvargs test cases.


[1] SubFunction in kernel:
https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/

[2] RFC:
http://patchwork.dpdk.org/project/dpdk/list/?series=14376

[3] V2:
http://patchwork.dpdk.org/project/dpdk/list/?series=14559


Xueming Li (9):
  ethdev: refactor representor infrastructure
  ethdev: support new VF representor syntax
  ethdev: support sub function representor
  ethdev: support PF index in representor
  ethdev: support multi-host representor
  devarg: change reprsentor ID to bitmap
  ethdev: capability of new representor syntax
  kvargs: update parser for new representor syntax
  eal: probe devices of same PCI but different devargs

 app/test/test_kvargs.c                   |  51 +++++++--
 config/rte_config.h                      |   1 +
 doc/guides/prog_guide/poll_mode_drv.rst  |   8 +-
 drivers/net/bnxt/bnxt_ethdev.c           |   7 ++
 drivers/net/bnxt/bnxt_reps.c             |   3 +-
 drivers/net/enic/enic_ethdev.c           |   7 ++
 drivers/net/enic/enic_vf_representor.c   |   3 +-
 drivers/net/i40e/i40e_ethdev.c           |   8 ++
 drivers/net/i40e/i40e_vf_representor.c   |   3 +-
 drivers/net/ixgbe/ixgbe_ethdev.c         |   8 ++
 drivers/net/ixgbe/ixgbe_vf_representor.c |   3 +-
 drivers/net/mlx5/linux/mlx5_os.c         |  15 ++-
 lib/librte_eal/common/eal_common_bus.c   |  23 ++++
 lib/librte_ethdev/ethdev_private.c       | 128 ++++++++++++++---------
 lib/librte_ethdev/ethdev_private.h       |   3 -
 lib/librte_ethdev/rte_class_eth.c        |  40 +++++--
 lib/librte_ethdev/rte_ethdev.c           |  31 +++++-
 lib/librte_ethdev/rte_ethdev.h           |   2 +
 lib/librte_ethdev/rte_ethdev_driver.h    |  64 ++++++++++++
 lib/librte_ethdev/version.map            |   2 +
 lib/librte_kvargs/rte_kvargs.c           | 101 +++++++++++++-----
 21 files changed, 401 insertions(+), 110 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH v3 1/9] ethdev: refactor representor infrastructure

[Xueming Li]

To support extended representor syntax, this patch refactor representor
infrastructure:
1. introduces representor type enum
2. devargs representor port range extraction from partial value

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/net/bnxt/bnxt_ethdev.c        |  7 ++
 drivers/net/enic/enic_ethdev.c        |  7 ++
 drivers/net/i40e/i40e_ethdev.c        |  8 +++
 drivers/net/ixgbe/ixgbe_ethdev.c      |  8 +++
 drivers/net/mlx5/linux/mlx5_os.c      | 11 ++++
 lib/librte_ethdev/ethdev_private.c    | 93 ++++++++++++---------------
 lib/librte_ethdev/ethdev_private.h    |  3 -
 lib/librte_ethdev/rte_class_eth.c     |  4 +-
 lib/librte_ethdev/rte_ethdev.c        |  5 +-
 lib/librte_ethdev/rte_ethdev_driver.h |  7 ++
 10 files changed, 93 insertions(+), 60 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 0788d263d8..fe4d4d4728 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5513,6 +5513,13 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
 	int i, ret = 0;
 	struct rte_kvargs *kvlist = NULL;
 
+	if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
+			    eth_da->type);
+		return -ENOTSUP;
+	}
 	num_rep = eth_da->nb_representor_ports;
 	if (num_rep > BNXT_MAX_VF_REPS) {
 		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index d041a6bee9..dd085caa93 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1303,6 +1303,13 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (retval)
 			return retval;
 	}
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct enic),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 14622484a0..6b74b514fc 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -640,6 +640,14 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 			return retval;
 	}
 
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct i40e_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index d7a1806ab8..eb2c4929e2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1717,6 +1717,14 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	} else
 		memset(&eth_da, 0, sizeof(eth_da));
 
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct ixgbe_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 6812a1f215..6981ba1f41 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -706,6 +706,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				strerror(rte_errno));
 			return NULL;
 		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
+			/* Representor not specified. */
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+			rte_errno = ENOTSUP;
+			DRV_LOG(ERR, "unsupported representor type: %s",
+				dpdk_dev->devargs->args);
+			return NULL;
+		}
 		for (i = 0; i < eth_da.nb_representor_ports; ++i)
 			if (eth_da.representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 162a502fe7..c219164a4a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -38,60 +38,13 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
 	return NULL;
 }
 
-int
-rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data)
-{
-	char *str_start;
-	int state;
-	int result;
-
-	if (*str != '[')
-		/* Single element, not a list */
-		return callback(str, data);
-
-	/* Sanity check, then strip the brackets */
-	str_start = &str[strlen(str) - 1];
-	if (*str_start != ']') {
-		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
-		return -EINVAL;
-	}
-	str++;
-	*str_start = '\0';
-
-	/* Process list elements */
-	state = 0;
-	while (1) {
-		if (state == 0) {
-			if (*str == '\0')
-				break;
-			if (*str != ',') {
-				str_start = str;
-				state = 1;
-			}
-		} else if (state == 1) {
-			if (*str == ',' || *str == '\0') {
-				if (str > str_start) {
-					/* Non-empty string fragment */
-					*str = '\0';
-					result = callback(str_start, data);
-					if (result < 0)
-						return result;
-				}
-				state = 0;
-			}
-		}
-		str++;
-	}
-	return 0;
-}
-
 static int
 rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	const uint16_t max_list)
 {
 	uint16_t lo, hi, val;
 	int result;
+	char *pos = str;
 
 	result = sscanf(str, "%hu-%hu", &lo, &hi);
 	if (result == 1) {
@@ -99,7 +52,7 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 			return -ENOMEM;
 		list[(*len_list)++] = lo;
 	} else if (result == 2) {
-		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
+		if (lo >= hi)
 			return -EINVAL;
 		for (val = lo; val <= hi; val++) {
 			if (*len_list >= max_list)
@@ -108,14 +61,52 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 		}
 	} else
 		return -EINVAL;
-	return 0;
+	while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
+		pos++;
+	return pos - str;
 }
 
+static int
+rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
+	const uint16_t max_list)
+{
+	char *pos = str;
+	int ret;
+
+	if (*pos == '[')
+		pos++;
+	while (1) {
+		ret = rte_eth_devargs_process_range(pos, list, len_list,
+						    max_list);
+		if (ret < 0)
+			return ret;
+		pos += ret;
+		if (*pos != ',') /* end of list */
+			break;
+		pos++;
+	}
+	if (*str == '[' && *pos != ']')
+		return -EINVAL;
+	if (*pos == ']')
+		pos++;
+	return pos - str;
+}
+
+/*
+ * representor format:
+ *   #: range or single number of VF representor - legacy
+ */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
+	int ret;
 
-	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
+	/* Number # alone implies VF */
+	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+	if (ret < 0)
+		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
+	return ret < 0 ? ret : 0;
 }
diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
index 905a45c337..220ddd4408 100644
--- a/lib/librte_ethdev/ethdev_private.h
+++ b/lib/librte_ethdev/ethdev_private.h
@@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
 		const void *data);
 
 /* Parse devargs value for representor parameter. */
-typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
-int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data);
 int rte_eth_devargs_parse_representor_ports(char *str, void *data);
 
 #ifdef __cplusplus
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index 6338355e25..efe6149df5 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
 	if (values == NULL)
 		return -1;
 	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_list(values,
-			rte_eth_devargs_parse_representor_ports,
-			&representors);
+	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 17ddacc78d..2ac51ac149 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5542,9 +5542,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
-			result = rte_eth_devargs_parse_list(pair->value,
-				rte_eth_devargs_parse_representor_ports,
-				eth_da);
+			result = rte_eth_devargs_parse_representor_ports(
+					pair->value, eth_da);
 			if (result < 0)
 				goto parse_cleanup;
 		}
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 0eacfd8425..b66a955b18 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1193,6 +1193,12 @@ __rte_internal
 int
 rte_eth_switch_domain_free(uint16_t domain_id);
 
+/** Ethernet device representor type */
+enum rte_eth_representor_type {
+	RTE_ETH_REPRESENTOR_NONE, /* not a representor */
+	RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
+};
+
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
 	uint16_t ports[RTE_MAX_ETHPORTS];
@@ -1203,6 +1209,7 @@ struct rte_eth_devargs {
 	/** representor port/s identifier to enable on device */
 	uint16_t nb_representor_ports;
 	/** number of ports in representor port field */
+	enum rte_eth_representor_type type; /* type of representor */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v3 2/9] ethdev: support new VF representor syntax

[Xueming Li]

To prepare for more representor types, this patch adds compatible VF
representor devargs syntax:

vf#: new VF port representor/s, example: vf[0-3], vf2

For backwards compatibility, representor "#" is deemed as "vf#".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 6 +++---
 lib/librte_ethdev/ethdev_private.c      | 5 ++++-
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 239ec820ea..979a2bab9d 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -374,9 +374,9 @@ parameters to those ports.
   this argument allows user to specify which switch ports to enable port
   representors for.::
 
-   -a DBDF,representor=0
-   -a DBDF,representor=[0,4,6,9]
-   -a DBDF,representor=[0-31]
+   -a DBDF,representor=vf0
+   -a DBDF,representor=vf[0,4,6,9]
+   -a DBDF,representor=vf[0-31]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index c219164a4a..54f565ca37 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -95,6 +95,7 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 /*
  * representor format:
  *   #: range or single number of VF representor - legacy
+ *   vf#: VF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -102,8 +103,10 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 	int ret;
 
-	/* Number # alone implies VF */
+	/* Parse vf# or number # alone implies VF */
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	if (str[0] == 'v'  && str[1] == 'f')
+		str += 2;
 	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 	if (ret < 0)
-- 
2.25.1

[dpdk-dev] [PATCH v3 3/9] ethdev: support sub function representor

[Xueming Li]

SubFunction is a portion of the PCI device, created on demand, a SF
netdev has its own dedicated queues(txq, rxq). A SF netdev supports
eswitch representation offload similar to existing PF and VF
representors.

To support SF representor, this patch introduces new devargs syntax:

sf#: new SF port representor/s, example: sf[0-3], sf2

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  1 +
 lib/librte_ethdev/ethdev_private.c      | 14 +++++++++++---
 lib/librte_ethdev/rte_ethdev_driver.h   |  1 +
 3 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 979a2bab9d..831d620c8d 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -377,6 +377,7 @@ parameters to those ports.
    -a DBDF,representor=vf0
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
+   -a DBDF,representor=sf[0-1023]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 54f565ca37..551a43738a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -96,6 +96,7 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * representor format:
  *   #: range or single number of VF representor - legacy
  *   vf#: VF port representor/s
+ *   sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -103,10 +104,17 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 	int ret;
 
-	/* Parse vf# or number # alone implies VF */
-	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	if (str[0] == 'v'  && str[1] == 'f')
+	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
+	/* Parse vf# and sf#, number # alone implies VF */
+	if (str[0] == 'v'  && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
+	} else if (str[0] == 's'  && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_SF;
+		str += 2;
+	} else {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	}
 	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 	if (ret < 0)
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index b66a955b18..086d64223a 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1197,6 +1197,7 @@ rte_eth_switch_domain_free(uint16_t domain_id);
 enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /* not a representor */
 	RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
+	RTE_ETH_REPRESENTOR_SF,   /* representor of SF */
 };
 
 /** Generic Ethernet device arguments  */
-- 
2.25.1

[dpdk-dev] [PATCH v3 4/9] ethdev: support PF index in representor

[Xueming Li]

To support representor in bonding, this patch introduces 'pf' section
to representor devargs syntax:

[pf#]vf#: VF port representor/s, example: vf3, pf0vf3
[pf#]sf#: SF port representor/s, example: sf2, pf1sf2

pf# is optional for SF and VF representor, indicates owner PF device of
SF or VF.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  1 +
 lib/librte_ethdev/ethdev_private.c      | 15 +++++++++++++--
 lib/librte_ethdev/rte_ethdev_driver.h   |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 831d620c8d..87f23c4f49 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -378,6 +378,7 @@ parameters to those ports.
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
    -a DBDF,representor=sf[0-1023]
+   -a DBDF,representor=pf[0-1]sf[0-1023]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 551a43738a..ccc638ec49 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -95,8 +95,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 /*
  * representor format:
  *   #: range or single number of VF representor - legacy
- *   vf#: VF port representor/s
- *   sf#: SF port representor/s
+ *   [pf#]vf#: VF port representor/s
+ *   [pf#]sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -105,6 +105,16 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	int ret;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
+	/* Parse pf# */
+	if (str[0] == 'p' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_PF;
+		str += 2;
+		ret = rte_eth_devargs_process_list(str, eth_da->ports,
+				&eth_da->nb_ports, RTE_MAX_ETHPORTS);
+		if (ret < 0)
+			goto err;
+		str += ret;
+	}
 	/* Parse vf# and sf#, number # alone implies VF */
 	if (str[0] == 'v'  && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
@@ -117,6 +127,7 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	}
 	ret = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+err:
 	if (ret < 0)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
 	return ret < 0 ? ret : 0;
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 086d64223a..edb000cbd4 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1198,6 +1198,7 @@ enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /* not a representor */
 	RTE_ETH_REPRESENTOR_VF,   /* representor of VF */
 	RTE_ETH_REPRESENTOR_SF,   /* representor of SF */
+	RTE_ETH_REPRESENTOR_PF,   /* representor of host PF */
 };
 
 /** Generic Ethernet device arguments  */
-- 
2.25.1

[dpdk-dev] [PATCH v3 5/9] ethdev: support multi-host representor

[Xueming Li]

This patch introduces multi-host controller for ethdev representor
syntax, examples:

[[c#]pf#]vf#: VF port representor/s, example: pf0vf1
[[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]

c# is controller(host) ID/range in case of multi-host, optional. It is
mostly for SmartNIC attached to multiple hosts in the same rack to allow
routing the packets between PF/SF/VF running on these hosts.

Controller ID is physical host ID in multi-host, for end-user (e.g. OVS)
it is transparent as part of representor syntax, sam as representor
ID, interpreted by PMD.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 config/rte_config.h                   |  1 +
 lib/librte_ethdev/ethdev_private.c    | 14 ++++++++++++--
 lib/librte_ethdev/rte_ethdev_driver.h |  4 ++++
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index a0b5160ff2..23d02d51ef 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -58,6 +58,7 @@
 #define RTE_MAX_QUEUES_PER_PORT 1024
 #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
 #define RTE_ETHDEV_RXTX_CALLBACKS 1
+#define RTE_MAX_MULTI_HOST_CTRLS 4
 
 /* cryptodev defines */
 #define RTE_CRYPTO_MAX_DEVS 64
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index ccc638ec49..95f1ab847a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -95,8 +95,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 /*
  * representor format:
  *   #: range or single number of VF representor - legacy
- *   [pf#]vf#: VF port representor/s
- *   [pf#]sf#: SF port representor/s
+ *   [[c#]pf#]vf#: VF port representor/s
+ *   [[c#]pf#]sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -105,6 +105,16 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	int ret;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_NONE;
+	/* Parse c# */
+	if (str[0] == 'c') {
+		str += 1;
+		ret = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
+				&eth_da->nb_mh_controllers,
+				RTE_DIM(eth_da->mh_controllers));
+		if (ret < 0)
+			goto err;
+		str += ret;
+	}
 	/* Parse pf# */
 	if (str[0] == 'p' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_PF;
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index edb000cbd4..b9e4a0b9ba 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1203,6 +1203,10 @@ enum rte_eth_representor_type {
 
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
+	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
+	/** controller/s number in case of multi-host */
+	uint16_t nb_mh_controllers;
+	/** number of controllers in multi-host controllers field */
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
 	uint16_t nb_ports;
-- 
2.25.1

[dpdk-dev] [PATCH v3 6/9] devarg: change reprsentor ID to bitmap

[Xueming Li]

In eth representor comparator callback, ethdev was compared with devarg.
Since ethdev representor port didn't contain controller(host) and owner
port information, callback only compared representor port and returned
representor port on other PF port.

This patch changes representor port to bitmap encoding, expands and
updates representor port ID after parsing, when device representor ID
uses the same bitmap encoding, the eth representor comparer callback
returns correct ethdev.

Representor port ID bitmap definition:
 Representor ID bitmap:
 xxxx xxxx xxxx xxxx
 |||| |LLL LLLL LLLL vf/sf id
 |||| L 1:sf, 0:vf
 ||LL pf id
 LL controller(host) id

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/net/bnxt/bnxt_reps.c             |  3 +-
 drivers/net/enic/enic_vf_representor.c   |  3 +-
 drivers/net/i40e/i40e_vf_representor.c   |  3 +-
 drivers/net/ixgbe/ixgbe_vf_representor.c |  3 +-
 drivers/net/mlx5/linux/mlx5_os.c         |  4 +-
 lib/librte_ethdev/ethdev_private.c       |  5 ++-
 lib/librte_ethdev/rte_class_eth.c        | 38 ++++++++++++++----
 lib/librte_ethdev/rte_ethdev.c           | 26 ++++++++++++
 lib/librte_ethdev/rte_ethdev_driver.h    | 51 ++++++++++++++++++++++++
 lib/librte_ethdev/version.map            |  2 +
 10 files changed, 124 insertions(+), 14 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index f7bbf77d3f..34febc2d1e 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -186,7 +186,8 @@ int bnxt_representor_init(struct rte_eth_dev *eth_dev, void *params)
 
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	eth_dev->data->representor_id = rep_params->vf_id;
+	eth_dev->data->representor_id = rte_eth_representor_id_encode(
+		0, 0, RTE_ETH_REPRESENTOR_VF, rep_params->vf_id);
 
 	rte_eth_random_addr(vf_rep_bp->dflt_mac_addr);
 	memcpy(vf_rep_bp->mac_addr, vf_rep_bp->dflt_mac_addr,
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index c2c03c0281..632317af15 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -674,7 +674,8 @@ int enic_vf_representor_init(struct rte_eth_dev *eth_dev, void *init_params)
 	eth_dev->dev_ops = &enic_vf_representor_dev_ops;
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	eth_dev->data->representor_id = vf->vf_id;
+	eth_dev->data->representor_id = rte_eth_representor_id_encode(
+		0, 0, RTE_ETH_REPRESENTOR_VF, vf->vf_id);
 	eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
 		sizeof(struct rte_ether_addr) *
 		ENIC_UNICAST_PERFECT_FILTERS, 0);
diff --git a/drivers/net/i40e/i40e_vf_representor.c b/drivers/net/i40e/i40e_vf_representor.c
index 9e40406a3d..d90d0fdb9d 100644
--- a/drivers/net/i40e/i40e_vf_representor.c
+++ b/drivers/net/i40e/i40e_vf_representor.c
@@ -510,7 +510,8 @@ i40e_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
 
 	ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	ethdev->data->representor_id = representor->vf_id;
+	ethdev->data->representor_id = rte_eth_representor_id_encode(
+			0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
 
 	/* Setting the number queues allocated to the VF */
 	ethdev->data->nb_rx_queues = vf->vsi->nb_qps;
diff --git a/drivers/net/ixgbe/ixgbe_vf_representor.c b/drivers/net/ixgbe/ixgbe_vf_representor.c
index 8185f0d3bb..e15b794761 100644
--- a/drivers/net/ixgbe/ixgbe_vf_representor.c
+++ b/drivers/net/ixgbe/ixgbe_vf_representor.c
@@ -196,7 +196,8 @@ ixgbe_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
 		return -ENODEV;
 
 	ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
-	ethdev->data->representor_id = representor->vf_id;
+	ethdev->data->representor_id = rte_eth_representor_id_encode(
+			0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
 
 	/* Set representor device ops */
 	ethdev->dev_ops = &ixgbe_vf_representor_dev_ops;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 6981ba1f41..7739a1b0a5 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1031,7 +1031,9 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 #endif
 	/* representor_id field keeps the unmodified VF index. */
 	priv->representor_id = switch_info->representor ?
-			       switch_info->port_name : -1;
+		rte_eth_representor_id_encode(0, 0, RTE_ETH_REPRESENTOR_VF,
+					      switch_info->port_name) :
+		-1;
 	/*
 	 * Look for sibling devices in order to reuse their switch domain
 	 * if any, otherwise allocate one.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 95f1ab847a..57e84a34af 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -93,10 +93,13 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 }
 
 /*
- * representor format:
+ * Parse representor ports, expand and update representor port ID.
+ * Representor format:
  *   #: range or single number of VF representor - legacy
  *   [[c#]pf#]vf#: VF port representor/s
  *   [[c#]pf#]sf#: SF port representor/s
+ *
+ * See RTE_ETH_REPR() for representor ID format.
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index efe6149df5..994db96960 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -66,8 +66,8 @@ eth_representor_cmp(const char *key __rte_unused,
 	int ret;
 	char *values;
 	const struct rte_eth_dev_data *data = opaque;
-	struct rte_eth_devargs representors;
-	uint16_t index;
+	struct rte_eth_devargs eth_da;
+	uint16_t index, c, p, f;
 
 	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
 		return -1; /* not a representor port */
@@ -76,17 +76,39 @@ eth_representor_cmp(const char *key __rte_unused,
 	values = strdup(value);
 	if (values == NULL)
 		return -1;
-	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
+	memset(&eth_da, 0, sizeof(eth_da));
+	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
 
+	/* Set default values. */
+	if (eth_da.nb_mh_controllers == 0) {
+		eth_da.nb_mh_controllers = 1;
+		eth_da.mh_controllers[0] = 0;
+	}
+	if (eth_da.nb_ports == 0) {
+		eth_da.nb_ports = 1;
+		eth_da.ports[0] = 0;
+	}
+	if (eth_da.nb_representor_ports == 0) {
+		eth_da.nb_representor_ports = 1;
+		eth_da.representor_ports[0] = 0;
+	}
 	/* Return 0 if representor id is matching one of the values. */
-	for (index = 0; index < representors.nb_representor_ports; index++)
-		if (data->representor_id ==
-				representors.representor_ports[index])
-			return 0;
+	for (c = 0; c < eth_da.nb_mh_controllers; ++c) {
+		for (p = 0; p < eth_da.nb_ports; ++p) {
+			for (f = 0; f < eth_da.nb_representor_ports; ++f) {
+				index = rte_eth_representor_id_encode(
+					eth_da.mh_controllers[c],
+					eth_da.ports[p],
+					eth_da.type,
+					eth_da.representor_ports[f]);
+				if (data->representor_id == index)
+					return 0;
+			}
+		}
+	}
 	return -1; /* no match */
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 2ac51ac149..276f42e54b 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5556,6 +5556,32 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	return result;
 }
 
+uint16_t
+rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
+			      enum rte_eth_representor_type type,
+			      uint16_t representor_port)
+{
+	return (((controller & 3) << 14) |
+		((pf & 3) << 12) |
+		(!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
+		(representor_port & 0x7ff));
+}
+
+uint16_t
+rte_eth_representor_id_parse(const uint16_t representor_id,
+			     uint16_t *controller, uint16_t *pf,
+			     enum rte_eth_representor_type *type)
+{
+	if (controller)
+		*controller = (representor_id >> 14) & 3;
+	if (pf)
+		*pf = (representor_id >> 12) & 3;
+	if (type)
+		*type = ((representor_id >> 11) & 1) ?
+			RTE_ETH_REPRESENTOR_SF : RTE_ETH_REPRESENTOR_VF;
+	return representor_id & 0x7ff;
+}
+
 static int
 eth_dev_handle_port_list(const char *cmd __rte_unused,
 		const char *params __rte_unused,
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index b9e4a0b9ba..d557b763b6 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1218,6 +1218,57 @@ struct rte_eth_devargs {
 	enum rte_eth_representor_type type; /* type of representor */
 };
 
+/**
+ * PMD helper function to encode representor ID
+ *
+ * The compact format is used for device iterator that comparing
+ * ethdev representor ID with target devargs.
+ *
+ * xxxx xxxx xxxx xxxx
+ * |||| |LLL LLLL LLLL vf/sf id
+ * |||| L 1:sf, 0:vf
+ * ||LL pf id
+ * LL controller(host) id
+ *
+ * @param controller
+ *  Controller ID.
+ * @param pf
+ *  PF port ID.
+ * @param type
+ *  Representor type.
+ * @param representor_port
+ *  Representor port ID.
+ *
+ * @return
+ *   Encoded representor ID.
+ */
+__rte_internal
+uint16_t
+rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
+			      enum rte_eth_representor_type type,
+			      uint16_t representor_port);
+
+/**
+ * PMD helper function to parse representor ID
+ *
+ * @param representor_id
+ *  Representor ID.
+ * @param controller
+ *  Parsed controller ID.
+ * @param pf
+ *  Parsed PF port ID.
+ * @param type
+ *  Parsed representor type.
+ *
+ * @return
+ *   Parsed representor port ID.
+ */
+__rte_internal
+uint16_t
+rte_eth_representor_id_parse(const uint16_t representor_id,
+			     uint16_t *controller, uint16_t *pf,
+			     enum rte_eth_representor_type *type);
+
 /**
  * PMD helper function to parse ethdev arguments
  *
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index d3f5410806..44edaed507 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -257,6 +257,8 @@ INTERNAL {
 	rte_eth_dev_release_port;
 	rte_eth_dev_internal_reset;
 	rte_eth_devargs_parse;
+	rte_eth_representor_id_encode;
+	rte_eth_representor_id_parse;
 	rte_eth_dma_zone_free;
 	rte_eth_dma_zone_reserve;
 	rte_eth_hairpin_queue_peer_bind;
-- 
2.25.1

[dpdk-dev] [PATCH v3 7/9] ethdev: capability of new representor syntax

[Xueming Li]

New representor devargs syntax can't be recognized correctly by previous
DPDK version. For application to adapt different DPDK version
automatically, this patch introduces new eth device capability of
supportting SubFunction representor device.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_ethdev/rte_ethdev.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index f5f8919186..3c087bec67 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1432,6 +1432,8 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/**< Device supports SubFunction representor. */
+#define RTE_ETH_DEV_CAPA_REPRESENTOR_SF         0x00000004
 /**@}*/
 
 /*
-- 
2.25.1

[dpdk-dev] [PATCH v3 8/9] kvargs: update parser for new representor syntax

[Xueming Li]

This patch updates kvargs parser to allow comma in list value:
  k1=a[1,2]b[3-5]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 app/test/test_kvargs.c         |  51 +++++++++++++++--
 lib/librte_kvargs/rte_kvargs.c | 101 +++++++++++++++++++++++----------
 2 files changed, 117 insertions(+), 35 deletions(-)

diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index 2a2dae43a0..b42cdcb9f5 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -35,6 +35,25 @@ static int check_handler(const char *key, const char *value,
 	return 0;
 }
 
+/* test parsing. */
+static int test_kvargs_parsing(const char *args, unsigned int n)
+{
+	struct rte_kvargs *kvlist;
+
+	kvlist = rte_kvargs_parse(args, NULL);
+	if (kvlist == NULL) {
+		printf("rte_kvargs_parse() error: %s\n", args);
+		return -1;
+	}
+	if (kvlist->count != n) {
+		printf("invalid count value %d: %s\n", kvlist->count, args);
+		rte_kvargs_free(kvlist);
+		return -1;
+	}
+	rte_kvargs_free(kvlist);
+	return 0;
+}
+
 /* test a valid case */
 static int test_valid_kvargs(void)
 {
@@ -191,6 +210,32 @@ static int test_valid_kvargs(void)
 	}
 	rte_kvargs_free(kvlist);
 
+	valid_keys = NULL;
+
+	args = "foo=1,foo=";
+	if (test_kvargs_parsing(args, 2))
+		goto fail;
+
+	args = "foo=1,foo";
+	if (test_kvargs_parsing(args, 2))
+		goto fail;
+
+	args = "foo=1,=2";
+	if (test_kvargs_parsing(args, 2))
+		goto fail;
+
+	args = "foo=[1,2";
+	if (test_kvargs_parsing(args, 1))
+		goto fail;
+
+	args = ",=";
+	if (test_kvargs_parsing(args, 1))
+		goto fail;
+
+	args = "foo=[";
+	if (test_kvargs_parsing(args, 1))
+		goto fail;
+
 	return 0;
 
  fail:
@@ -212,12 +257,6 @@ static int test_invalid_kvargs(void)
 	/* list of argument that should fail */
 	const char *args_list[] = {
 		"wrong-key=x",     /* key not in valid_keys_list */
-		"foo=1,foo=",      /* empty value */
-		"foo=1,foo",       /* no value */
-		"foo=1,=2",        /* no key */
-		"foo=[1,2",        /* no closing bracket in value */
-		",=",              /* also test with a smiley */
-		"foo=[",           /* no value in list and no closing bracket */
 		NULL };
 	const char **args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
index 285081c86c..ffae8914cf 100644
--- a/lib/librte_kvargs/rte_kvargs.c
+++ b/lib/librte_kvargs/rte_kvargs.c
@@ -5,6 +5,7 @@
 
 #include <string.h>
 #include <stdlib.h>
+#include <stdbool.h>
 
 #include <rte_string_fns.h>
 
@@ -13,15 +14,19 @@
 /*
  * Receive a string with a list of arguments following the pattern
  * key=value,key=value,... and insert them into the list.
- * strtok() is used so the params string will be copied to be modified.
+ * Params string will be copied to be modified.
+ * list "[]" and list element splitter ",", "-" is treated as value.
+ * Supported examples:
+ *   k1=v1,k2=v2
+ *   k1
+ *   k1=x[0-1]y[1,3-5,9]z
  */
 static int
 rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 {
-	unsigned i;
-	char *str;
-	char *ctx1 = NULL;
-	char *ctx2 = NULL;
+	char *str, *start;
+	bool in_list = false, end_key = false, end_value = false;
+	bool save = false, end_pair = false;
 
 	/* Copy the const char *params to a modifiable string
 	 * to pass to rte_strsplit
@@ -32,36 +37,74 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 
 	/* browse each key/value pair and add it in kvlist */
 	str = kvlist->str;
-	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
+	start = str; /* start of current key or value */
+	while (1) {
+		switch (*str) {
+		case '=': /* End of key. */
+			end_key = true;
+			save = true;
+			break;
+		case ',':
+			/* End of value, skip comma in middle of range */
+			if (!in_list) {
+				if (end_key)
+					end_value = true;
+				else
+					end_key = true;
+				save = true;
+				end_pair = true;
+			}
+			break;
+		case '[': /* Start of list. */
+			in_list = true;
+			break;
+		case ']': /* End of list.  */
+			if (in_list)
+				in_list = false;
+			break;
+		case '\0': /* End of string */
+			if (end_key)
+				end_value = true;
+			else
+				end_key = true;
+			save = true;
+			end_pair = true;
+			break;
+		default:
+			break;
+		}
 
-		i = kvlist->count;
-		if (i >= RTE_KVARGS_MAX)
-			return -1;
+		if (!save) {
+			/* Continue if not end of key or value. */
+			str++;
+			continue;
+		}
 
-		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
-		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
-		if (kvlist->pairs[i].key == NULL ||
-		    kvlist->pairs[i].value == NULL)
+		if (kvlist->count >= RTE_KVARGS_MAX)
 			return -1;
 
-		/* Detect list [a,b] to skip comma delimiter in list. */
-		str = kvlist->pairs[i].value;
-		if (str[0] == '[') {
-			/* Find the end of the list. */
-			while (str[strlen(str) - 1] != ']') {
-				/* Restore the comma erased by strtok_r(). */
-				if (ctx1 == NULL || ctx1[0] == '\0')
-					return -1; /* no closing bracket */
-				str[strlen(str)] = ',';
-				/* Parse until next comma. */
-				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
-				if (str == NULL)
-					return -1; /* no closing bracket */
-			}
+		if (end_value)
+			/* Value parsed */
+			kvlist->pairs[kvlist->count].value = start;
+		else if (end_key)
+			/* Key parsed. */
+			kvlist->pairs[kvlist->count].key = start;
+
+		if (end_pair) {
+			if (end_value || str != start)
+				/* Ignore empty pair. */
+				kvlist->count++;
+			end_key = false;
+			end_value = false;
+			end_pair = false;
 		}
 
-		kvlist->count++;
-		str = NULL;
+		if (*str == '\0') /* End of string. */
+			break;
+		*str = '\0';
+		str++;
+		start = str;
+		save = false;
 	}
 
 	return 0;
-- 
2.25.1

[dpdk-dev] [PATCH v3 9/9] eal: probe devices of same PCI but different devargs

[Xueming Li]

When probing same PCI device with different representor arguments, PCI
bus only probed first devargs, ignored other allowed devices with
different representor arguments.

This patch iterates all devargs and try them all after bus scan.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_eal/common/eal_common_bus.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index baa5b532af..9f88a0fe6e 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -10,6 +10,7 @@
 #include <rte_debug.h>
 #include <rte_string_fns.h>
 #include <rte_errno.h>
+#include <rte_devargs.h>
 
 #include "eal_private.h"
 
@@ -56,12 +57,22 @@ rte_bus_scan(void)
 	return 0;
 }
 
+static int
+cmp_dev_name(const struct rte_device *dev, const void *_name)
+{
+	const char *name = _name;
+
+	return strcmp(dev->name, name);
+}
+
 /* Probe all devices of all buses */
 int
 rte_bus_probe(void)
 {
 	int ret;
 	struct rte_bus *bus, *vbus = NULL;
+	struct rte_devargs *da;
+	struct rte_device *dev;
 
 	TAILQ_FOREACH(bus, &rte_bus_list, next) {
 		if (!strcmp(bus->name, "vdev")) {
@@ -82,6 +93,18 @@ rte_bus_probe(void)
 				vbus->name);
 	}
 
+	/* For devargs with same name but different arguments, try them all. */
+	RTE_EAL_DEVARGS_FOREACH(NULL, da) {
+		dev = da->bus->find_device(NULL, cmp_dev_name, da->name);
+		if (!dev || !rte_dev_is_probed(dev) || dev->devargs == da)
+			continue;
+		dev->devargs = da;
+		ret = dev->bus->plug(dev);
+		if (ret == 0)
+			RTE_LOG(DEBUG, EAL, "device probed %s %s", da->name,
+				da->args);
+	}
+
 	return 0;
 }
 
-- 
2.25.1

[dpdk-dev] [PATCH v4 0/9] ethdev: support SubFunction representor

[Xueming Li]

SubFunction [1] is a portion of the PCI device, a SF netdev has its own
dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to existing
VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For backward compatibility, this patch also introduces new netdev
capability to indicate the capability of supportting SF representor.

Version history:
 RFC:
 	initial version [2]
 V2:
    - separate patch for represnetor infrastructure, controller, pf and
      sf.
    - replace representor ID macro with functions:
      rte_eth_representor_id_encode()
      rte_eth_representor_id_parse()
    - new patch to allow devargs with same PCI BDF but different
      representors.
    - other minor code updates according to comments, thanks Andrew!
    - update document
 V3:
    - improve probing of allowed devargs with same name.
    - parse single word of kvargs as key. 
    - update kvargs test cases.
 V4:
    - split first representor refactor patch into
      1: add representor type
      2: refector representor list parsing
    - push the patch supporting multi-devargs for same device.


[1] SubFunction in kernel:
https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/

[2] RFC:
http://patchwork.dpdk.org/project/dpdk/list/?series=14376

[3] V2:
http://patchwork.dpdk.org/project/dpdk/list/?series=14559

[4] v3:
1 http://patchwork.dpdk.org/patch/86460/
2 http://patchwork.dpdk.org/patch/86461/
3 http://patchwork.dpdk.org/patch/86462/
4 http://patchwork.dpdk.org/patch/86463/
5 http://patchwork.dpdk.org/patch/86464/
6 http://patchwork.dpdk.org/patch/86465/
7 http://patchwork.dpdk.org/patch/86467/
8 http://patchwork.dpdk.org/patch/86466/
9 http://patchwork.dpdk.org/patch/86468/


Xueming Li (9):
  ethdev: introduce representor type
  ethdev: support representor port list
  ethdev: support new VF representor syntax
  ethdev: support sub function representor
  ethdev: support PF index in representor
  ethdev: support multi-host in representor
  devarg: change representor ID to bitmap
  ethdev: add capability of sub-function representor
  kvargs: update parser to support lists

 app/test/test_kvargs.c                   |  46 +++++++-
 config/rte_config.h                      |   1 +
 doc/guides/prog_guide/poll_mode_drv.rst  |  13 ++-
 drivers/net/bnxt/bnxt_ethdev.c           |   7 ++
 drivers/net/bnxt/bnxt_reps.c             |   3 +-
 drivers/net/enic/enic_ethdev.c           |   7 ++
 drivers/net/enic/enic_vf_representor.c   |   3 +-
 drivers/net/i40e/i40e_ethdev.c           |   8 ++
 drivers/net/i40e/i40e_vf_representor.c   |   3 +-
 drivers/net/ixgbe/ixgbe_ethdev.c         |   8 ++
 drivers/net/ixgbe/ixgbe_vf_representor.c |   3 +-
 drivers/net/mlx5/linux/mlx5_os.c         |  15 ++-
 lib/librte_ethdev/ethdev_private.c       | 140 ++++++++++++++---------
 lib/librte_ethdev/ethdev_private.h       |   3 -
 lib/librte_ethdev/rte_class_eth.c        |  40 +++++--
 lib/librte_ethdev/rte_ethdev.c           |  31 ++++-
 lib/librte_ethdev/rte_ethdev.h           |   2 +
 lib/librte_ethdev/rte_ethdev_driver.h    |  66 +++++++++++
 lib/librte_ethdev/version.map            |   2 +
 lib/librte_kvargs/rte_kvargs.c           | 101 +++++++++++-----
 20 files changed, 389 insertions(+), 113 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Xueming Li]

To support more representor type, this patch introduces representor type
enum. The enum is subject to extend for new types upcoming.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/bnxt/bnxt_ethdev.c        |  7 +++++++
 drivers/net/enic/enic_ethdev.c        |  7 +++++++
 drivers/net/i40e/i40e_ethdev.c        |  8 ++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c      |  8 ++++++++
 drivers/net/mlx5/linux/mlx5_os.c      | 11 +++++++++++
 lib/librte_ethdev/ethdev_private.c    |  5 +++++
 lib/librte_ethdev/rte_ethdev_driver.h |  9 +++++++++
 7 files changed, 55 insertions(+)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 74b0f3d1dc..d7c8b3ec07 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5586,6 +5586,13 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
 	int i, ret = 0;
 	struct rte_kvargs *kvlist = NULL;
 
+	if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
+			    eth_da->type);
+		return -ENOTSUP;
+	}
 	num_rep = eth_da->nb_representor_ports;
 	if (num_rep > BNXT_MAX_VF_REPS) {
 		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index d041a6bee9..dd085caa93 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1303,6 +1303,13 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (retval)
 			return retval;
 	}
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct enic),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index c1c2327b3f..3cea6de2bd 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -638,6 +638,14 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 			return retval;
 	}
 
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct i40e_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index d7a1806ab8..eb2c4929e2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1717,6 +1717,14 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	} else
 		memset(&eth_da, 0, sizeof(eth_da));
 
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct ixgbe_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 9ac1d46b1b..caead107b0 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -705,6 +705,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				strerror(rte_errno));
 			return NULL;
 		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
+			/* Representor not specified. */
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+			rte_errno = ENOTSUP;
+			DRV_LOG(ERR, "unsupported representor type: %s",
+				dpdk_dev->devargs->args);
+			return NULL;
+		}
 		for (i = 0; i < eth_da.nb_representor_ports; ++i)
 			if (eth_da.representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 162a502fe7..c1a411dba4 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -111,11 +111,16 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	return 0;
 }
 
+/*
+ * representor format:
+ *   #: range or single number of VF representor
+ */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 }
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 0eacfd8425..3bc5c5bbbb 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1193,6 +1193,14 @@ __rte_internal
 int
 rte_eth_switch_domain_free(uint16_t domain_id);
 
+/** Ethernet device representor type */
+enum rte_eth_representor_type {
+	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
+	RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
+	RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
+	RTE_ETH_REPRESENTOR_PF,   /**< representor of host PF. */
+};
+
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
 	uint16_t ports[RTE_MAX_ETHPORTS];
@@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
 	/** representor port/s identifier to enable on device */
 	uint16_t nb_representor_ports;
 	/** number of ports in representor port field */
+	enum rte_eth_representor_type type; /* type of representor */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v4 2/9] ethdev: support representor port list

[Xueming Li]

To support extended representor syntax, this patch extends the
representor list parsing to support for representor port range in
devargs, examples:
   representor=[1,2,3]         - single list
   representor=[1,3-5,7,9-11]  - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 lib/librte_ethdev/ethdev_private.c | 105 ++++++++++++++---------------
 lib/librte_ethdev/ethdev_private.h |   3 -
 lib/librte_ethdev/rte_class_eth.c  |   4 +-
 lib/librte_ethdev/rte_ethdev.c     |   5 +-
 4 files changed, 54 insertions(+), 63 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index c1a411dba4..12bcc7e98d 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -38,77 +38,71 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
 	return NULL;
 }
 
-int
-rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data)
+static int
+rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
+		       const uint16_t max_list, uint16_t val)
 {
-	char *str_start;
-	int state;
-	int result;
-
-	if (*str != '[')
-		/* Single element, not a list */
-		return callback(str, data);
-
-	/* Sanity check, then strip the brackets */
-	str_start = &str[strlen(str) - 1];
-	if (*str_start != ']') {
-		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
-		return -EINVAL;
-	}
-	str++;
-	*str_start = '\0';
+	uint16_t i;
 
-	/* Process list elements */
-	state = 0;
-	while (1) {
-		if (state == 0) {
-			if (*str == '\0')
-				break;
-			if (*str != ',') {
-				str_start = str;
-				state = 1;
-			}
-		} else if (state == 1) {
-			if (*str == ',' || *str == '\0') {
-				if (str > str_start) {
-					/* Non-empty string fragment */
-					*str = '\0';
-					result = callback(str_start, data);
-					if (result < 0)
-						return result;
-				}
-				state = 0;
-			}
-		}
-		str++;
+	if (*len_list >= max_list)
+		return -1;
+	for (i = 0; i < *len_list; i++) {
+		if (list[i] == val)
+			return 0;
 	}
+	list[(*len_list)++] = val;
 	return 0;
 }
 
-static int
+static char *
 rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	const uint16_t max_list)
 {
 	uint16_t lo, hi, val;
 	int result;
+	char *pos = str;
 
 	result = sscanf(str, "%hu-%hu", &lo, &hi);
 	if (result == 1) {
-		if (*len_list >= max_list)
-			return -ENOMEM;
-		list[(*len_list)++] = lo;
+		if (rte_eth_devargs_enlist(list, len_list, max_list, lo) != 0)
+			return NULL;
 	} else if (result == 2) {
-		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
-			return -EINVAL;
+		if (lo >= hi)
+			return NULL;
 		for (val = lo; val <= hi; val++) {
-			if (*len_list >= max_list)
-				return -ENOMEM;
-			list[(*len_list)++] = val;
+			if (rte_eth_devargs_enlist(list, len_list, max_list,
+						   val) != 0)
+				return NULL;
 		}
 	} else
-		return -EINVAL;
-	return 0;
+		return NULL;
+	while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
+		pos++;
+	return pos;
+}
+
+static char *
+rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
+	const uint16_t max_list)
+{
+	char *pos = str;
+
+	if (*pos == '[')
+		pos++;
+	while (1) {
+		pos = rte_eth_devargs_process_range(pos, list, len_list,
+						    max_list);
+		if (pos == NULL)
+			return NULL;
+		if (*pos != ',') /* end of list */
+			break;
+		pos++;
+	}
+	if (*str == '[' && *pos != ']')
+		return NULL;
+	if (*pos == ']')
+		pos++;
+	return pos;
 }
 
 /*
@@ -121,6 +115,9 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
+	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+	if (str == NULL)
+		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
+	return str == NULL ? -1 : 0;
 }
diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
index 905a45c337..220ddd4408 100644
--- a/lib/librte_ethdev/ethdev_private.h
+++ b/lib/librte_ethdev/ethdev_private.h
@@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
 		const void *data);
 
 /* Parse devargs value for representor parameter. */
-typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
-int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data);
 int rte_eth_devargs_parse_representor_ports(char *str, void *data);
 
 #ifdef __cplusplus
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index 6338355e25..efe6149df5 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
 	if (values == NULL)
 		return -1;
 	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_list(values,
-			rte_eth_devargs_parse_representor_ports,
-			&representors);
+	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 17ddacc78d..2ac51ac149 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5542,9 +5542,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
-			result = rte_eth_devargs_parse_list(pair->value,
-				rte_eth_devargs_parse_representor_ports,
-				eth_da);
+			result = rte_eth_devargs_parse_representor_ports(
+					pair->value, eth_da);
 			if (result < 0)
 				goto parse_cleanup;
 		}
-- 
2.25.1

[dpdk-dev] [PATCH v4 3/9] ethdev: support new VF representor syntax

[Xueming Li]

To prepare for more representor types, this patch adds compatible VF
representor devargs syntax:

vf#: new VF port representor/s, examples:
 representor=vf2          - single representor
 representor=vf[1,3,5]    - single list
 representor=vf[0-3]      - single range
 representor=vf[0,1,4-7]  - list with singles and range

For backwards compatibility, representor "#" is deemed as "vf#".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 7 ++++---
 lib/librte_ethdev/ethdev_private.c      | 5 ++++-
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 239ec820ea..0117c2af07 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -374,9 +374,10 @@ parameters to those ports.
   this argument allows user to specify which switch ports to enable port
   representors for.::
 
-   -a DBDF,representor=0
-   -a DBDF,representor=[0,4,6,9]
-   -a DBDF,representor=[0-31]
+   -a DBDF,representor=vf0
+   -a DBDF,representor=vf[0,4,6,9]
+   -a DBDF,representor=vf[0-31]
+   -a DBDF,representor=vf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 12bcc7e98d..ee60b1259a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -107,7 +107,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 
 /*
  * representor format:
- *   #: range or single number of VF representor
+ *   #: range or single number of VF representor - legacy
+ *   vf#: VF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -115,6 +116,8 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	if (str[0] == 'v' && str[1] == 'f')
+		str += 2;
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 	if (str == NULL)
-- 
2.25.1

[dpdk-dev] [PATCH v4 4/9] ethdev: support sub function representor

[Xueming Li]

SubFunction is a portion of the PCI device, created on demand, a SF
netdev has its own dedicated queues(txq, rxq). A SF netdev supports
eswitch representation offload similar to existing PF and VF
representors.

To support SF representor, this patch introduces new devargs syntax,
examples:
 representor=sf0               - single SubFunction representor
 representor=sf[1,3,5]         - single list
 representor=sf[0-3],          - single range
 representor=sf[0,2-6,8,10-12] - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  4 ++++
 lib/librte_ethdev/ethdev_private.c      | 11 +++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 0117c2af07..86e5867f1b 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -378,6 +378,10 @@ parameters to those ports.
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
    -a DBDF,representor=vf[0,2-4,7,9-11]
+   -a DBDF,representor=sf0
+   -a DBDF,representor=sf[1,3,5]
+   -a DBDF,representor=sf[0-1023]
+   -a DBDF,representor=sf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index ee60b1259a..170b3ef067 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -109,15 +109,22 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * representor format:
  *   #: range or single number of VF representor - legacy
  *   vf#: VF port representor/s
+ *   sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
-	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	if (str[0] == 'v' && str[1] == 'f')
+	if (str[0] == 'v' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
+	} else if (str[0] == 's' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_SF;
+		str += 2;
+	} else {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 	if (str == NULL)
-- 
2.25.1

[dpdk-dev] [PATCH v4 5/9] ethdev: support PF index in representor

[Xueming Li]

With Kernel bonding, multiple underlying PFs are bonded, VFs come
from different PF, need to identify representor of VFs unambiguously by
adding PF index.

This patch introduces optional 'pf' section to representor devargs
syntax, examples:
 representor=pf0vf0             - single VF representor
 representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs


Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  2 ++
 lib/librte_ethdev/ethdev_private.c      | 13 +++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 86e5867f1b..b2147aad30 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -382,6 +382,8 @@ parameters to those ports.
    -a DBDF,representor=sf[1,3,5]
    -a DBDF,representor=sf[0-1023]
    -a DBDF,representor=sf[0,2-4,7,9-11]
+   -a DBDF,representor=pf1vf0
+   -a DBDF,representor=pf[0-1]sf[0-127]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 170b3ef067..f81423ceb8 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -108,14 +108,22 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 /*
  * representor format:
  *   #: range or single number of VF representor - legacy
- *   vf#: VF port representor/s
- *   sf#: SF port representor/s
+ *   [pf#]vf#: VF port representor/s
+ *   [pf#]sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'p' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_PF;
+		str += 2;
+		str = rte_eth_devargs_process_list(str, eth_da->ports,
+				&eth_da->nb_ports, RTE_MAX_ETHPORTS);
+		if (str == NULL)
+			goto err;
+	}
 	if (str[0] == 'v' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
@@ -127,6 +135,7 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+err:
 	if (str == NULL)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
 	return str == NULL ? -1 : 0;
-- 
2.25.1

[dpdk-dev] [PATCH v4 6/9] ethdev: support multi-host in representor

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to the multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. To provide the unambiguous identification
of the PCIe function the controller index is added. The full representor
identifier consists of three indices - controller index, PF index, and
VF or SF index (if any).

This patch introduces controller index to ethdev representor syntax,
examples:

[[c#]pf#]vf#: VF port representor/s, example: pf0vf1
[[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]

c# is controller(host) ID/range in case of multi-host, optional.

For user application (e.g. OVS), PMD is responsible to interpret and
locate representor device based on controller ID, PF ID and VF/SF ID in
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 config/rte_config.h                   |  1 +
 lib/librte_ethdev/ethdev_private.c    | 12 ++++++++++--
 lib/librte_ethdev/rte_ethdev_driver.h |  4 ++++
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index a0b5160ff2..23d02d51ef 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -58,6 +58,7 @@
 #define RTE_MAX_QUEUES_PER_PORT 1024
 #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
 #define RTE_ETHDEV_RXTX_CALLBACKS 1
+#define RTE_MAX_MULTI_HOST_CTRLS 4
 
 /* cryptodev defines */
 #define RTE_CRYPTO_MAX_DEVS 64
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index f81423ceb8..57473b5a39 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -108,14 +108,22 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 /*
  * representor format:
  *   #: range or single number of VF representor - legacy
- *   [pf#]vf#: VF port representor/s
- *   [pf#]sf#: SF port representor/s
+ *   [[c#]pf#]vf#: VF port representor/s
+ *   [[c#]pf#]sf#: SF port representor/s
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'c') {
+		str += 1;
+		str = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
+				&eth_da->nb_mh_controllers,
+				RTE_DIM(eth_da->mh_controllers));
+		if (str == NULL)
+			goto err;
+	}
 	if (str[0] == 'p' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_PF;
 		str += 2;
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 3bc5c5bbbb..8e04634660 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1203,6 +1203,10 @@ enum rte_eth_representor_type {
 
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
+	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
+	/** controller/s number in case of multi-host */
+	uint16_t nb_mh_controllers;
+	/** number of controllers in multi-host controllers field */
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
 	uint16_t nb_ports;
-- 
2.25.1

[dpdk-dev] [PATCH v4 7/9] devarg: change representor ID to bitmap

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

SR-IOV and SubFunction are created on top of PF. PF index is introduced
because there might be multiple PFs in the bonding configuration and
only bonding device is probed.

In eth representor comparator callback, ethdev was compared with devarg.
Since ethdev representor port didn't contain controller index and PF
index information, callback returned representor from other PF or
controller.

This patch changes representor ID to bitmap so that the ethdev
representor comparer callback returns correct ethdev by comparing full
representor information including: controller index, PF index,
representor type, SF or VF index.

Representor ID bitmap definition:
 xxxx xxxx xxxx xxxx
 |||| |LLL LLLL LLLL vf/sf id
 |||| L 1:sf, 0:vf
 ||LL pf id
 LL controller(host) id

This approach keeps binary compatibility with all drivers, VF
representor id matches with simple id for non-bonding and non-multi-host
configurations.

In the future, the representor ID field and each section should extend
to bigger width to support more devices.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/bnxt/bnxt_reps.c             |  3 +-
 drivers/net/enic/enic_vf_representor.c   |  3 +-
 drivers/net/i40e/i40e_vf_representor.c   |  3 +-
 drivers/net/ixgbe/ixgbe_vf_representor.c |  3 +-
 drivers/net/mlx5/linux/mlx5_os.c         |  4 +-
 lib/librte_ethdev/ethdev_private.c       |  5 ++-
 lib/librte_ethdev/rte_class_eth.c        | 38 +++++++++++++----
 lib/librte_ethdev/rte_ethdev.c           | 26 ++++++++++++
 lib/librte_ethdev/rte_ethdev_driver.h    | 53 ++++++++++++++++++++++++
 lib/librte_ethdev/version.map            |  2 +
 10 files changed, 126 insertions(+), 14 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index f7bbf77d3f..34febc2d1e 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -186,7 +186,8 @@ int bnxt_representor_init(struct rte_eth_dev *eth_dev, void *params)
 
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	eth_dev->data->representor_id = rep_params->vf_id;
+	eth_dev->data->representor_id = rte_eth_representor_id_encode(
+		0, 0, RTE_ETH_REPRESENTOR_VF, rep_params->vf_id);
 
 	rte_eth_random_addr(vf_rep_bp->dflt_mac_addr);
 	memcpy(vf_rep_bp->mac_addr, vf_rep_bp->dflt_mac_addr,
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index c2c03c0281..632317af15 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -674,7 +674,8 @@ int enic_vf_representor_init(struct rte_eth_dev *eth_dev, void *init_params)
 	eth_dev->dev_ops = &enic_vf_representor_dev_ops;
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	eth_dev->data->representor_id = vf->vf_id;
+	eth_dev->data->representor_id = rte_eth_representor_id_encode(
+		0, 0, RTE_ETH_REPRESENTOR_VF, vf->vf_id);
 	eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
 		sizeof(struct rte_ether_addr) *
 		ENIC_UNICAST_PERFECT_FILTERS, 0);
diff --git a/drivers/net/i40e/i40e_vf_representor.c b/drivers/net/i40e/i40e_vf_representor.c
index 9e40406a3d..d90d0fdb9d 100644
--- a/drivers/net/i40e/i40e_vf_representor.c
+++ b/drivers/net/i40e/i40e_vf_representor.c
@@ -510,7 +510,8 @@ i40e_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
 
 	ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	ethdev->data->representor_id = representor->vf_id;
+	ethdev->data->representor_id = rte_eth_representor_id_encode(
+			0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
 
 	/* Setting the number queues allocated to the VF */
 	ethdev->data->nb_rx_queues = vf->vsi->nb_qps;
diff --git a/drivers/net/ixgbe/ixgbe_vf_representor.c b/drivers/net/ixgbe/ixgbe_vf_representor.c
index 8185f0d3bb..e15b794761 100644
--- a/drivers/net/ixgbe/ixgbe_vf_representor.c
+++ b/drivers/net/ixgbe/ixgbe_vf_representor.c
@@ -196,7 +196,8 @@ ixgbe_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
 		return -ENODEV;
 
 	ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
-	ethdev->data->representor_id = representor->vf_id;
+	ethdev->data->representor_id = rte_eth_representor_id_encode(
+			0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
 
 	/* Set representor device ops */
 	ethdev->dev_ops = &ixgbe_vf_representor_dev_ops;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index caead107b0..4d7940bcca 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1025,7 +1025,9 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 #endif
 	/* representor_id field keeps the unmodified VF index. */
 	priv->representor_id = switch_info->representor ?
-			       switch_info->port_name : -1;
+		rte_eth_representor_id_encode(0, 0, RTE_ETH_REPRESENTOR_VF,
+					      switch_info->port_name) :
+		-1;
 	/*
 	 * Look for sibling devices in order to reuse their switch domain
 	 * if any, otherwise allocate one.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 57473b5a39..0b3b4aa871 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -106,10 +106,13 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
 }
 
 /*
- * representor format:
+ * Parse representor ports, expand and update representor port ID.
+ * Representor format:
  *   #: range or single number of VF representor - legacy
  *   [[c#]pf#]vf#: VF port representor/s
  *   [[c#]pf#]sf#: SF port representor/s
+ *
+ * See RTE_ETH_REPR() for representor ID format.
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index efe6149df5..994db96960 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -66,8 +66,8 @@ eth_representor_cmp(const char *key __rte_unused,
 	int ret;
 	char *values;
 	const struct rte_eth_dev_data *data = opaque;
-	struct rte_eth_devargs representors;
-	uint16_t index;
+	struct rte_eth_devargs eth_da;
+	uint16_t index, c, p, f;
 
 	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
 		return -1; /* not a representor port */
@@ -76,17 +76,39 @@ eth_representor_cmp(const char *key __rte_unused,
 	values = strdup(value);
 	if (values == NULL)
 		return -1;
-	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
+	memset(&eth_da, 0, sizeof(eth_da));
+	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
 
+	/* Set default values. */
+	if (eth_da.nb_mh_controllers == 0) {
+		eth_da.nb_mh_controllers = 1;
+		eth_da.mh_controllers[0] = 0;
+	}
+	if (eth_da.nb_ports == 0) {
+		eth_da.nb_ports = 1;
+		eth_da.ports[0] = 0;
+	}
+	if (eth_da.nb_representor_ports == 0) {
+		eth_da.nb_representor_ports = 1;
+		eth_da.representor_ports[0] = 0;
+	}
 	/* Return 0 if representor id is matching one of the values. */
-	for (index = 0; index < representors.nb_representor_ports; index++)
-		if (data->representor_id ==
-				representors.representor_ports[index])
-			return 0;
+	for (c = 0; c < eth_da.nb_mh_controllers; ++c) {
+		for (p = 0; p < eth_da.nb_ports; ++p) {
+			for (f = 0; f < eth_da.nb_representor_ports; ++f) {
+				index = rte_eth_representor_id_encode(
+					eth_da.mh_controllers[c],
+					eth_da.ports[p],
+					eth_da.type,
+					eth_da.representor_ports[f]);
+				if (data->representor_id == index)
+					return 0;
+			}
+		}
+	}
 	return -1; /* no match */
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 2ac51ac149..276f42e54b 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5556,6 +5556,32 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	return result;
 }
 
+uint16_t
+rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
+			      enum rte_eth_representor_type type,
+			      uint16_t representor_port)
+{
+	return (((controller & 3) << 14) |
+		((pf & 3) << 12) |
+		(!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
+		(representor_port & 0x7ff));
+}
+
+uint16_t
+rte_eth_representor_id_parse(const uint16_t representor_id,
+			     uint16_t *controller, uint16_t *pf,
+			     enum rte_eth_representor_type *type)
+{
+	if (controller)
+		*controller = (representor_id >> 14) & 3;
+	if (pf)
+		*pf = (representor_id >> 12) & 3;
+	if (type)
+		*type = ((representor_id >> 11) & 1) ?
+			RTE_ETH_REPRESENTOR_SF : RTE_ETH_REPRESENTOR_VF;
+	return representor_id & 0x7ff;
+}
+
 static int
 eth_dev_handle_port_list(const char *cmd __rte_unused,
 		const char *params __rte_unused,
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 8e04634660..0d8893693e 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1218,6 +1218,59 @@ struct rte_eth_devargs {
 	enum rte_eth_representor_type type; /* type of representor */
 };
 
+#define RTE_NO_REPRESENTOR_ID UINT16_MAX /**< No representor ID. */
+
+/**
+ * PMD helper function to encode representor ID
+ *
+ * The compact format is used for device iterator that comparing
+ * ethdev representor ID with target devargs.
+ *
+ * xxxx xxxx xxxx xxxx
+ * |||| |LLL LLLL LLLL vf/sf id
+ * |||| L 1:sf, 0:vf
+ * ||LL pf id
+ * LL controller(host) id
+ *
+ * @param controller
+ *  Controller ID.
+ * @param pf
+ *  PF port ID.
+ * @param type
+ *  Representor type.
+ * @param representor_port
+ *  Representor port ID.
+ *
+ * @return
+ *   Encoded representor ID.
+ */
+__rte_internal
+uint16_t
+rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
+			      enum rte_eth_representor_type type,
+			      uint16_t representor_port);
+
+/**
+ * PMD helper function to parse representor ID
+ *
+ * @param representor_id
+ *  Representor ID.
+ * @param controller
+ *  Parsed controller ID.
+ * @param pf
+ *  Parsed PF port ID.
+ * @param type
+ *  Parsed representor type.
+ *
+ * @return
+ *   Parsed representor port ID.
+ */
+__rte_internal
+uint16_t
+rte_eth_representor_id_parse(const uint16_t representor_id,
+			     uint16_t *controller, uint16_t *pf,
+			     enum rte_eth_representor_type *type);
+
 /**
  * PMD helper function to parse ethdev arguments
  *
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index d3f5410806..44edaed507 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -257,6 +257,8 @@ INTERNAL {
 	rte_eth_dev_release_port;
 	rte_eth_dev_internal_reset;
 	rte_eth_devargs_parse;
+	rte_eth_representor_id_encode;
+	rte_eth_representor_id_parse;
 	rte_eth_dma_zone_free;
 	rte_eth_dma_zone_reserve;
 	rte_eth_hairpin_queue_peer_bind;
-- 
2.25.1

[dpdk-dev] [PATCH v4 8/9] ethdev: add capability of sub-function representor

[Xueming Li]

Old DPDK version or some drivers that don't support SubFunction
representor. For application to adapt different DPDK version
automatically, or to be used for different NICs, this patch introduces
new eth device capability of supporting SubFunction representor device.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 lib/librte_ethdev/rte_ethdev.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 2cbce958cf..da65adf1ab 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1433,6 +1433,8 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/** Device supports SubFunction representor. */
+#define RTE_ETH_DEV_CAPA_REPRESENTOR_SF         0x00000004
 /**@}*/
 
 /*
-- 
2.25.1

[dpdk-dev] [PATCH v4 9/9] kvargs: update parser to support lists

[Xueming Li]

This patch updates kvargs parser to support lists on top of range, allow
multiple lists or range:
  k1=a[1,2]b[3-5]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 app/test/test_kvargs.c         |  46 +++++++++++++--
 lib/librte_kvargs/rte_kvargs.c | 101 +++++++++++++++++++++++----------
 2 files changed, 112 insertions(+), 35 deletions(-)

diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index 2a2dae43a0..a91ea8dc47 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -35,6 +35,25 @@ static int check_handler(const char *key, const char *value,
 	return 0;
 }
 
+/* test parsing. */
+static int test_kvargs_parsing(const char *args, unsigned int n)
+{
+	struct rte_kvargs *kvlist;
+
+	kvlist = rte_kvargs_parse(args, NULL);
+	if (kvlist == NULL) {
+		printf("rte_kvargs_parse() error: %s\n", args);
+		return -1;
+	}
+	if (kvlist->count != n) {
+		printf("invalid count value %d: %s\n", kvlist->count, args);
+		rte_kvargs_free(kvlist);
+		return -1;
+	}
+	rte_kvargs_free(kvlist);
+	return 0;
+}
+
 /* test a valid case */
 static int test_valid_kvargs(void)
 {
@@ -42,6 +61,19 @@ static int test_valid_kvargs(void)
 	const char *args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
 	const char **valid_keys;
+	static const struct {
+		unsigned int expected;
+		const char *input;
+	} valid_inputs[] = {
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo" },
+		{ 2, "foo=1,=2" },
+		{ 1, "foo=[1,2" },
+		{ 1, ",=" },
+		{ 1, "foo=[" },
+	};
+	unsigned int i;
 
 	/* empty args is valid */
 	args = "";
@@ -191,6 +223,14 @@ static int test_valid_kvargs(void)
 	}
 	rte_kvargs_free(kvlist);
 
+	valid_keys = NULL;
+
+	for (i = 0; i < RTE_DIM(valid_inputs); ++i) {
+		args = valid_inputs[i].input;
+		if (test_kvargs_parsing(args, valid_inputs[i].expected))
+			goto fail;
+	}
+
 	return 0;
 
  fail:
@@ -212,12 +252,6 @@ static int test_invalid_kvargs(void)
 	/* list of argument that should fail */
 	const char *args_list[] = {
 		"wrong-key=x",     /* key not in valid_keys_list */
-		"foo=1,foo=",      /* empty value */
-		"foo=1,foo",       /* no value */
-		"foo=1,=2",        /* no key */
-		"foo=[1,2",        /* no closing bracket in value */
-		",=",              /* also test with a smiley */
-		"foo=[",           /* no value in list and no closing bracket */
 		NULL };
 	const char **args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
index 285081c86c..ffae8914cf 100644
--- a/lib/librte_kvargs/rte_kvargs.c
+++ b/lib/librte_kvargs/rte_kvargs.c
@@ -5,6 +5,7 @@
 
 #include <string.h>
 #include <stdlib.h>
+#include <stdbool.h>
 
 #include <rte_string_fns.h>
 
@@ -13,15 +14,19 @@
 /*
  * Receive a string with a list of arguments following the pattern
  * key=value,key=value,... and insert them into the list.
- * strtok() is used so the params string will be copied to be modified.
+ * Params string will be copied to be modified.
+ * list "[]" and list element splitter ",", "-" is treated as value.
+ * Supported examples:
+ *   k1=v1,k2=v2
+ *   k1
+ *   k1=x[0-1]y[1,3-5,9]z
  */
 static int
 rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 {
-	unsigned i;
-	char *str;
-	char *ctx1 = NULL;
-	char *ctx2 = NULL;
+	char *str, *start;
+	bool in_list = false, end_key = false, end_value = false;
+	bool save = false, end_pair = false;
 
 	/* Copy the const char *params to a modifiable string
 	 * to pass to rte_strsplit
@@ -32,36 +37,74 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 
 	/* browse each key/value pair and add it in kvlist */
 	str = kvlist->str;
-	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
+	start = str; /* start of current key or value */
+	while (1) {
+		switch (*str) {
+		case '=': /* End of key. */
+			end_key = true;
+			save = true;
+			break;
+		case ',':
+			/* End of value, skip comma in middle of range */
+			if (!in_list) {
+				if (end_key)
+					end_value = true;
+				else
+					end_key = true;
+				save = true;
+				end_pair = true;
+			}
+			break;
+		case '[': /* Start of list. */
+			in_list = true;
+			break;
+		case ']': /* End of list.  */
+			if (in_list)
+				in_list = false;
+			break;
+		case '\0': /* End of string */
+			if (end_key)
+				end_value = true;
+			else
+				end_key = true;
+			save = true;
+			end_pair = true;
+			break;
+		default:
+			break;
+		}
 
-		i = kvlist->count;
-		if (i >= RTE_KVARGS_MAX)
-			return -1;
+		if (!save) {
+			/* Continue if not end of key or value. */
+			str++;
+			continue;
+		}
 
-		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
-		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
-		if (kvlist->pairs[i].key == NULL ||
-		    kvlist->pairs[i].value == NULL)
+		if (kvlist->count >= RTE_KVARGS_MAX)
 			return -1;
 
-		/* Detect list [a,b] to skip comma delimiter in list. */
-		str = kvlist->pairs[i].value;
-		if (str[0] == '[') {
-			/* Find the end of the list. */
-			while (str[strlen(str) - 1] != ']') {
-				/* Restore the comma erased by strtok_r(). */
-				if (ctx1 == NULL || ctx1[0] == '\0')
-					return -1; /* no closing bracket */
-				str[strlen(str)] = ',';
-				/* Parse until next comma. */
-				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
-				if (str == NULL)
-					return -1; /* no closing bracket */
-			}
+		if (end_value)
+			/* Value parsed */
+			kvlist->pairs[kvlist->count].value = start;
+		else if (end_key)
+			/* Key parsed. */
+			kvlist->pairs[kvlist->count].key = start;
+
+		if (end_pair) {
+			if (end_value || str != start)
+				/* Ignore empty pair. */
+				kvlist->count++;
+			end_key = false;
+			end_value = false;
+			end_pair = false;
 		}
 
-		kvlist->count++;
-		str = NULL;
+		if (*str == '\0') /* End of string. */
+			break;
+		*str = '\0';
+		str++;
+		start = str;
+		save = false;
 	}
 
 	return 0;
-- 
2.25.1

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Thomas Monjalon]

+Cc more maintainers

18/01/2021 12:16, Xueming Li:
> To support more representor type, this patch introduces representor type
> enum. The enum is subject to extend for new types upcoming.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> ---
>  drivers/net/bnxt/bnxt_ethdev.c        |  7 +++++++
>  drivers/net/enic/enic_ethdev.c        |  7 +++++++
>  drivers/net/i40e/i40e_ethdev.c        |  8 ++++++++
>  drivers/net/ixgbe/ixgbe_ethdev.c      |  8 ++++++++
>  drivers/net/mlx5/linux/mlx5_os.c      | 11 +++++++++++
>  lib/librte_ethdev/ethdev_private.c    |  5 +++++
>  lib/librte_ethdev/rte_ethdev_driver.h |  9 +++++++++
>  7 files changed, 55 insertions(+)
> 
> diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
> index 74b0f3d1dc..d7c8b3ec07 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -5586,6 +5586,13 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
>  	int i, ret = 0;
>  	struct rte_kvargs *kvlist = NULL;
>  
> +	if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
> +		return 0;
> +	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
> +		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
> +			    eth_da->type);
> +		return -ENOTSUP;
> +	}
>  	num_rep = eth_da->nb_representor_ports;
>  	if (num_rep > BNXT_MAX_VF_REPS) {
>  		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
> diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
> index d041a6bee9..dd085caa93 100644
> --- a/drivers/net/enic/enic_ethdev.c
> +++ b/drivers/net/enic/enic_ethdev.c
> @@ -1303,6 +1303,13 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>  		if (retval)
>  			return retval;
>  	}
> +	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
> +		return 0;
> +	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> +		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
> +			    pci_dev->device.devargs->args);
> +		return -ENOTSUP;
> +	}
>  	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
>  		sizeof(struct enic),
>  		eth_dev_pci_specific_init, pci_dev,
> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index c1c2327b3f..3cea6de2bd 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -638,6 +638,14 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>  			return retval;
>  	}
>  
> +	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
> +		return 0;
> +	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> +		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
> +			    pci_dev->device.devargs->args);
> +		return -ENOTSUP;
> +	}
> +
>  	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
>  		sizeof(struct i40e_adapter),
>  		eth_dev_pci_specific_init, pci_dev,
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
> index d7a1806ab8..eb2c4929e2 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -1717,6 +1717,14 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>  	} else
>  		memset(&eth_da, 0, sizeof(eth_da));
>  
> +	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
> +		return 0;
> +	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> +		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
> +			    pci_dev->device.devargs->args);
> +		return -ENOTSUP;
> +	}
> +
>  	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
>  		sizeof(struct ixgbe_adapter),
>  		eth_dev_pci_specific_init, pci_dev,
> diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
> index 9ac1d46b1b..caead107b0 100644
> --- a/drivers/net/mlx5/linux/mlx5_os.c
> +++ b/drivers/net/mlx5/linux/mlx5_os.c
> @@ -705,6 +705,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
>  				strerror(rte_errno));
>  			return NULL;
>  		}
> +		if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
> +			/* Representor not specified. */
> +			rte_errno = EBUSY;
> +			return NULL;
> +		}
> +		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> +			rte_errno = ENOTSUP;
> +			DRV_LOG(ERR, "unsupported representor type: %s",
> +				dpdk_dev->devargs->args);
> +			return NULL;
> +		}
>  		for (i = 0; i < eth_da.nb_representor_ports; ++i)
>  			if (eth_da.representor_ports[i] ==
>  			    (uint16_t)switch_info->port_name)
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index 162a502fe7..c1a411dba4 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -111,11 +111,16 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>  	return 0;
>  }
>  
> +/*
> + * representor format:
> + *   #: range or single number of VF representor
> + */
>  int
>  rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  {
>  	struct rte_eth_devargs *eth_da = data;
>  
> +	eth_da->type = RTE_ETH_REPRESENTOR_VF;
>  	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
>  }
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index 0eacfd8425..3bc5c5bbbb 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -1193,6 +1193,14 @@ __rte_internal
>  int
>  rte_eth_switch_domain_free(uint16_t domain_id);
>  
> +/** Ethernet device representor type */
> +enum rte_eth_representor_type {
> +	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
> +	RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
> +	RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
> +	RTE_ETH_REPRESENTOR_PF,   /**< representor of host PF. */
> +};
> +
>  /** Generic Ethernet device arguments  */
>  struct rte_eth_devargs {
>  	uint16_t ports[RTE_MAX_ETHPORTS];
> @@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
>  	/** representor port/s identifier to enable on device */
>  	uint16_t nb_representor_ports;
>  	/** number of ports in representor port field */
> +	enum rte_eth_representor_type type; /* type of representor */
>  };

Re: [dpdk-dev] [PATCH v4 2/9] ethdev: support representor port list

[Thomas Monjalon]

18/01/2021 12:16, Xueming Li:
> To support extended representor syntax, this patch extends the
> representor list parsing to support for representor port range in
> devargs, examples:
>    representor=[1,2,3]         - single list
>    representor=[1,3-5,7,9-11]  - list with singles and ranges
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

Parsing functions are usually difficult to read.
If you need to do another version, I would recommend adding
an oneline comment on top of parsing functions to give a hint
about what it is really doing.

Here,

> +static int
> +rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
> +		       const uint16_t max_list, uint16_t val)

here,

> +static char *
>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>  	const uint16_t max_list)

here,

> +static char *
> +rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
> +	const uint16_t max_list)

and also for this old one:

> @@ -121,6 +115,9 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)

Re: [dpdk-dev] [PATCH v4 0/9] ethdev: support SubFunction representor

[Thomas Monjalon]

18/01/2021 12:16, Xueming Li:
> This patch set introduces new representor types in addtion to existing
> VF representor. Syntax:
> 
> [[c#]pf#]vf#: VF port representor/s from controller/pf
> [[c#]pf#]sf#: SF port representor/s from controller/pf
> #: VF representor - for backwards compatibility
> 
> "#" is number instance, list or range, valid examples:
>   1, [1,3,5], [0-3], [0,2-4,6]
[...]
> Xueming Li (9):
>   ethdev: introduce representor type
>   ethdev: support representor port list
>   ethdev: support new VF representor syntax
>   ethdev: support sub function representor
>   ethdev: support PF index in representor
>   ethdev: support multi-host in representor
>   devarg: change representor ID to bitmap
>   ethdev: add capability of sub-function representor
>   kvargs: update parser to support lists

For the series,
Acked-by: Thomas Monjalon <thomas@monjalon.net>

Nice work, thank you.

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Ajit Khaparde]

[-- Attachment #1: Type: text/plain, Size: 2562 bytes --]

On Mon, Jan 18, 2021 at 3:17 AM Xueming Li <xuemingl@nvidia.com> wrote:
>
> To support more representor type, this patch introduces representor type
> enum. The enum is subject to extend for new types upcoming.
>
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

> ---
>  drivers/net/bnxt/bnxt_ethdev.c        |  7 +++++++
>  drivers/net/enic/enic_ethdev.c        |  7 +++++++
>  drivers/net/i40e/i40e_ethdev.c        |  8 ++++++++
>  drivers/net/ixgbe/ixgbe_ethdev.c      |  8 ++++++++
>  drivers/net/mlx5/linux/mlx5_os.c      | 11 +++++++++++
>  lib/librte_ethdev/ethdev_private.c    |  5 +++++
>  lib/librte_ethdev/rte_ethdev_driver.h |  9 +++++++++
>  7 files changed, 55 insertions(+)
>
> diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
> index 74b0f3d1dc..d7c8b3ec07 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -5586,6 +5586,13 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
>         int i, ret = 0;
>         struct rte_kvargs *kvlist = NULL;
>
> +       if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
> +               return 0;
> +       if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
> +               PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
> +                           eth_da->type);
> +               return -ENOTSUP;
> +       }
>         num_rep = eth_da->nb_representor_ports;
>         if (num_rep > BNXT_MAX_VF_REPS) {
>                 PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
Ack. Thanks

:::: snip ::::

> +/** Ethernet device representor type */
> +enum rte_eth_representor_type {
> +       RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
> +       RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
> +       RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
Till we get used to the terminology...
Can we also have SF = "Sub Function" mentioned in the docs or comments?

> +       RTE_ETH_REPRESENTOR_PF,   /**< representor of host PF. */
> +};
> +
>  /** Generic Ethernet device arguments  */
>  struct rte_eth_devargs {
>         uint16_t ports[RTE_MAX_ETHPORTS];
> @@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
>         /** representor port/s identifier to enable on device */
>         uint16_t nb_representor_ports;
>         /** number of ports in representor port field */
> +       enum rte_eth_representor_type type; /* type of representor */
>  };
>
>  /**
> --
> 2.25.1
>

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Thomas Monjalon]

18/01/2021 18:42, Ajit Khaparde:
> On Mon, Jan 18, 2021 at 3:17 AM Xueming Li <xuemingl@nvidia.com> wrote:
> > +enum rte_eth_representor_type {
> > +       RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
> > +       RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
> > +       RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
> Till we get used to the terminology...
> Can we also have SF = "Sub Function" mentioned in the docs or comments?

Are we sure about the definition?
I remember seeing SF = Scalable Function somewhere else (maybe from Intel)

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Ajit Khaparde]

[-- Attachment #1: Type: text/plain, Size: 734 bytes --]

On Mon, Jan 18, 2021 at 9:57 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 18/01/2021 18:42, Ajit Khaparde:
> > On Mon, Jan 18, 2021 at 3:17 AM Xueming Li <xuemingl@nvidia.com> wrote:
> > > +enum rte_eth_representor_type {
> > > +       RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
> > > +       RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
> > > +       RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
> > Till we get used to the terminology...
> > Can we also have SF = "Sub Function" mentioned in the docs or comments?
>
> Are we sure about the definition?
> I remember seeing SF = Scalable Function somewhere else (maybe from Intel)
That complicates it. But if they mean the same thing, let's pick one.

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Thomas Monjalon]

18/01/2021 19:00, Ajit Khaparde:
> On Mon, Jan 18, 2021 at 9:57 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > 18/01/2021 18:42, Ajit Khaparde:
> > > On Mon, Jan 18, 2021 at 3:17 AM Xueming Li <xuemingl@nvidia.com> wrote:
> > > > +enum rte_eth_representor_type {
> > > > +       RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
> > > > +       RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
> > > > +       RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
> > > Till we get used to the terminology...
> > > Can we also have SF = "Sub Function" mentioned in the docs or comments?
> >
> > Are we sure about the definition?
> > I remember seeing SF = Scalable Function somewhere else (maybe from Intel)
> That complicates it. But if they mean the same thing, let's pick one.

I think "Sub Function" and "Virtual Function" are easy to understand
for everybody.
I suggest picking these two for comments above.

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Ajit Khaparde]

[-- Attachment #1: Type: text/plain, Size: 1045 bytes --]

On Mon, Jan 18, 2021 at 10:15 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 18/01/2021 19:00, Ajit Khaparde:
> > On Mon, Jan 18, 2021 at 9:57 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > 18/01/2021 18:42, Ajit Khaparde:
> > > > On Mon, Jan 18, 2021 at 3:17 AM Xueming Li <xuemingl@nvidia.com> wrote:
> > > > > +enum rte_eth_representor_type {
> > > > > +       RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
> > > > > +       RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
> > > > > +       RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
> > > > Till we get used to the terminology...
> > > > Can we also have SF = "Sub Function" mentioned in the docs or comments?
> > >
> > > Are we sure about the definition?
> > > I remember seeing SF = Scalable Function somewhere else (maybe from Intel)
> > That complicates it. But if they mean the same thing, let's pick one.
>
> I think "Sub Function" and "Virtual Function" are easy to understand
> for everybody.
> I suggest picking these two for comments above.
+1

>

Re: [dpdk-dev] [PATCH v4 7/9] devarg: change representor ID to bitmap

[Ajit Khaparde]

[-- Attachment #1: Type: text/plain, Size: 13660 bytes --]

On Mon, Jan 18, 2021 at 3:18 AM Xueming Li <xuemingl@nvidia.com> wrote:
>
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
>
> SR-IOV and SubFunction are created on top of PF. PF index is introduced
> because there might be multiple PFs in the bonding configuration and
> only bonding device is probed.
>
> In eth representor comparator callback, ethdev was compared with devarg.
> Since ethdev representor port didn't contain controller index and PF
> index information, callback returned representor from other PF or
> controller.
>
> This patch changes representor ID to bitmap so that the ethdev
> representor comparer callback returns correct ethdev by comparing full
> representor information including: controller index, PF index,
> representor type, SF or VF index.
>
> Representor ID bitmap definition:
>  xxxx xxxx xxxx xxxx
>  |||| |LLL LLLL LLLL vf/sf id
>  |||| L 1:sf, 0:vf
>  ||LL pf id
>  LL controller(host) id
>
> This approach keeps binary compatibility with all drivers, VF
> representor id matches with simple id for non-bonding and non-multi-host
> configurations.
>
> In the future, the representor ID field and each section should extend
> to bigger width to support more devices.
>
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Not just in this patch, there is a lot of info in the commit logs.
Unless I missed it, I don't see an update to
doc/guides/prog_guide/switch_representation.rst
Can you please update that.

> ---
>  drivers/net/bnxt/bnxt_reps.c             |  3 +-
>  drivers/net/enic/enic_vf_representor.c   |  3 +-
>  drivers/net/i40e/i40e_vf_representor.c   |  3 +-
>  drivers/net/ixgbe/ixgbe_vf_representor.c |  3 +-
>  drivers/net/mlx5/linux/mlx5_os.c         |  4 +-
>  lib/librte_ethdev/ethdev_private.c       |  5 ++-
>  lib/librte_ethdev/rte_class_eth.c        | 38 +++++++++++++----
>  lib/librte_ethdev/rte_ethdev.c           | 26 ++++++++++++
>  lib/librte_ethdev/rte_ethdev_driver.h    | 53 ++++++++++++++++++++++++
>  lib/librte_ethdev/version.map            |  2 +
>  10 files changed, 126 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
> index f7bbf77d3f..34febc2d1e 100644
> --- a/drivers/net/bnxt/bnxt_reps.c
> +++ b/drivers/net/bnxt/bnxt_reps.c
> @@ -186,7 +186,8 @@ int bnxt_representor_init(struct rte_eth_dev *eth_dev, void *params)
>
>         eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
>                                         RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> -       eth_dev->data->representor_id = rep_params->vf_id;
> +       eth_dev->data->representor_id = rte_eth_representor_id_encode(
> +               0, 0, RTE_ETH_REPRESENTOR_VF, rep_params->vf_id);
>
>         rte_eth_random_addr(vf_rep_bp->dflt_mac_addr);
>         memcpy(vf_rep_bp->mac_addr, vf_rep_bp->dflt_mac_addr,
> diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
> index c2c03c0281..632317af15 100644
> --- a/drivers/net/enic/enic_vf_representor.c
> +++ b/drivers/net/enic/enic_vf_representor.c
> @@ -674,7 +674,8 @@ int enic_vf_representor_init(struct rte_eth_dev *eth_dev, void *init_params)
>         eth_dev->dev_ops = &enic_vf_representor_dev_ops;
>         eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
>                                         RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> -       eth_dev->data->representor_id = vf->vf_id;
> +       eth_dev->data->representor_id = rte_eth_representor_id_encode(
> +               0, 0, RTE_ETH_REPRESENTOR_VF, vf->vf_id);
>         eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
>                 sizeof(struct rte_ether_addr) *
>                 ENIC_UNICAST_PERFECT_FILTERS, 0);
> diff --git a/drivers/net/i40e/i40e_vf_representor.c b/drivers/net/i40e/i40e_vf_representor.c
> index 9e40406a3d..d90d0fdb9d 100644
> --- a/drivers/net/i40e/i40e_vf_representor.c
> +++ b/drivers/net/i40e/i40e_vf_representor.c
> @@ -510,7 +510,8 @@ i40e_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
>
>         ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
>                                         RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> -       ethdev->data->representor_id = representor->vf_id;
> +       ethdev->data->representor_id = rte_eth_representor_id_encode(
> +                       0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
>
>         /* Setting the number queues allocated to the VF */
>         ethdev->data->nb_rx_queues = vf->vsi->nb_qps;
> diff --git a/drivers/net/ixgbe/ixgbe_vf_representor.c b/drivers/net/ixgbe/ixgbe_vf_representor.c
> index 8185f0d3bb..e15b794761 100644
> --- a/drivers/net/ixgbe/ixgbe_vf_representor.c
> +++ b/drivers/net/ixgbe/ixgbe_vf_representor.c
> @@ -196,7 +196,8 @@ ixgbe_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
>                 return -ENODEV;
>
>         ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
> -       ethdev->data->representor_id = representor->vf_id;
> +       ethdev->data->representor_id = rte_eth_representor_id_encode(
> +                       0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
>
>         /* Set representor device ops */
>         ethdev->dev_ops = &ixgbe_vf_representor_dev_ops;
> diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
> index caead107b0..4d7940bcca 100644
> --- a/drivers/net/mlx5/linux/mlx5_os.c
> +++ b/drivers/net/mlx5/linux/mlx5_os.c
> @@ -1025,7 +1025,9 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
>  #endif
>         /* representor_id field keeps the unmodified VF index. */
>         priv->representor_id = switch_info->representor ?
> -                              switch_info->port_name : -1;
> +               rte_eth_representor_id_encode(0, 0, RTE_ETH_REPRESENTOR_VF,
> +                                             switch_info->port_name) :
> +               -1;
>         /*
>          * Look for sibling devices in order to reuse their switch domain
>          * if any, otherwise allocate one.
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index 57473b5a39..0b3b4aa871 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -106,10 +106,13 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>  }
>
>  /*
> - * representor format:
> + * Parse representor ports, expand and update representor port ID.
> + * Representor format:
>   *   #: range or single number of VF representor - legacy
>   *   [[c#]pf#]vf#: VF port representor/s
>   *   [[c#]pf#]sf#: SF port representor/s
> + *
> + * See RTE_ETH_REPR() for representor ID format.
>   */
>  int
>  rte_eth_devargs_parse_representor_ports(char *str, void *data)
> diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
> index efe6149df5..994db96960 100644
> --- a/lib/librte_ethdev/rte_class_eth.c
> +++ b/lib/librte_ethdev/rte_class_eth.c
> @@ -66,8 +66,8 @@ eth_representor_cmp(const char *key __rte_unused,
>         int ret;
>         char *values;
>         const struct rte_eth_dev_data *data = opaque;
> -       struct rte_eth_devargs representors;
> -       uint16_t index;
> +       struct rte_eth_devargs eth_da;
> +       uint16_t index, c, p, f;
>
>         if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
>                 return -1; /* not a representor port */
> @@ -76,17 +76,39 @@ eth_representor_cmp(const char *key __rte_unused,
>         values = strdup(value);
>         if (values == NULL)
>                 return -1;
> -       memset(&representors, 0, sizeof(representors));
> -       ret = rte_eth_devargs_parse_representor_ports(values, &representors);
> +       memset(&eth_da, 0, sizeof(eth_da));
> +       ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
>         free(values);
>         if (ret != 0)
>                 return -1; /* invalid devargs value */
>
> +       /* Set default values. */
> +       if (eth_da.nb_mh_controllers == 0) {
> +               eth_da.nb_mh_controllers = 1;
> +               eth_da.mh_controllers[0] = 0;
> +       }
> +       if (eth_da.nb_ports == 0) {
> +               eth_da.nb_ports = 1;
> +               eth_da.ports[0] = 0;
> +       }
> +       if (eth_da.nb_representor_ports == 0) {
> +               eth_da.nb_representor_ports = 1;
> +               eth_da.representor_ports[0] = 0;
> +       }
>         /* Return 0 if representor id is matching one of the values. */
> -       for (index = 0; index < representors.nb_representor_ports; index++)
> -               if (data->representor_id ==
> -                               representors.representor_ports[index])
> -                       return 0;
> +       for (c = 0; c < eth_da.nb_mh_controllers; ++c) {
> +               for (p = 0; p < eth_da.nb_ports; ++p) {
> +                       for (f = 0; f < eth_da.nb_representor_ports; ++f) {
> +                               index = rte_eth_representor_id_encode(
> +                                       eth_da.mh_controllers[c],
> +                                       eth_da.ports[p],
> +                                       eth_da.type,
> +                                       eth_da.representor_ports[f]);
> +                               if (data->representor_id == index)
> +                                       return 0;
> +                       }
> +               }
> +       }
>         return -1; /* no match */
>  }
>
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 2ac51ac149..276f42e54b 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -5556,6 +5556,32 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>         return result;
>  }
>
> +uint16_t
> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
> +                             enum rte_eth_representor_type type,
> +                             uint16_t representor_port)
> +{
> +       return (((controller & 3) << 14) |
> +               ((pf & 3) << 12) |
> +               (!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
> +               (representor_port & 0x7ff));
> +}
> +
> +uint16_t
> +rte_eth_representor_id_parse(const uint16_t representor_id,
> +                            uint16_t *controller, uint16_t *pf,
> +                            enum rte_eth_representor_type *type)
> +{
> +       if (controller)
> +               *controller = (representor_id >> 14) & 3;
> +       if (pf)
> +               *pf = (representor_id >> 12) & 3;
> +       if (type)
> +               *type = ((representor_id >> 11) & 1) ?
> +                       RTE_ETH_REPRESENTOR_SF : RTE_ETH_REPRESENTOR_VF;
> +       return representor_id & 0x7ff;
> +}
> +
>  static int
>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>                 const char *params __rte_unused,
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index 8e04634660..0d8893693e 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -1218,6 +1218,59 @@ struct rte_eth_devargs {
>         enum rte_eth_representor_type type; /* type of representor */
>  };
>
> +#define RTE_NO_REPRESENTOR_ID UINT16_MAX /**< No representor ID. */
> +
> +/**
> + * PMD helper function to encode representor ID
> + *
> + * The compact format is used for device iterator that comparing
> + * ethdev representor ID with target devargs.
> + *
> + * xxxx xxxx xxxx xxxx
> + * |||| |LLL LLLL LLLL vf/sf id
> + * |||| L 1:sf, 0:vf
> + * ||LL pf id
> + * LL controller(host) id
> + *
> + * @param controller
> + *  Controller ID.
> + * @param pf
> + *  PF port ID.
> + * @param type
> + *  Representor type.
> + * @param representor_port
> + *  Representor port ID.
> + *
> + * @return
> + *   Encoded representor ID.
> + */
> +__rte_internal
> +uint16_t
> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
> +                             enum rte_eth_representor_type type,
> +                             uint16_t representor_port);
> +
> +/**
> + * PMD helper function to parse representor ID
> + *
> + * @param representor_id
> + *  Representor ID.
> + * @param controller
> + *  Parsed controller ID.
> + * @param pf
> + *  Parsed PF port ID.
> + * @param type
> + *  Parsed representor type.
> + *
> + * @return
> + *   Parsed representor port ID.
> + */
> +__rte_internal
> +uint16_t
> +rte_eth_representor_id_parse(const uint16_t representor_id,
> +                            uint16_t *controller, uint16_t *pf,
> +                            enum rte_eth_representor_type *type);
> +
>  /**
>   * PMD helper function to parse ethdev arguments
>   *
> diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
> index d3f5410806..44edaed507 100644
> --- a/lib/librte_ethdev/version.map
> +++ b/lib/librte_ethdev/version.map
> @@ -257,6 +257,8 @@ INTERNAL {
>         rte_eth_dev_release_port;
>         rte_eth_dev_internal_reset;
>         rte_eth_devargs_parse;
> +       rte_eth_representor_id_encode;
> +       rte_eth_representor_id_parse;
>         rte_eth_dma_zone_free;
>         rte_eth_dma_zone_reserve;
>         rte_eth_hairpin_queue_peer_bind;
> --
> 2.25.1
>

Re: [dpdk-dev] [PATCH v4 2/9] ethdev: support representor port list

[Xueming(Steven) Li]

Hi Thomas,

>-----Original Message-----
>From: Thomas Monjalon <thomas@monjalon.net>
>Sent: Tuesday, January 19, 2021 12:18 AM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: Ferruh Yigit <ferruh.yigit@intel.com>; Andrew Rybchenko
><andrew.rybchenko@oktetlabs.ru>; Olivier Matz <olivier.matz@6wind.com>;
>dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v4 2/9] ethdev: support representor port list
>
>18/01/2021 12:16, Xueming Li:
>> To support extended representor syntax, this patch extends the
>> representor list parsing to support for representor port range in
>> devargs, examples:
>>    representor=[1,2,3]         - single list
>>    representor=[1,3-5,7,9-11]  - list with singles and ranges
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>
>Parsing functions are usually difficult to read.
>If you need to do another version, I would recommend adding an oneline
>comment on top of parsing functions to give a hint about what it is really
>doing.
>
>Here,
>
>> +static int
>> +rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
>> +		       const uint16_t max_list, uint16_t val)
>
>here,
>
>> +static char *
>>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>>  	const uint16_t max_list)
>
>here,
>
>> +static char *
>> +rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>> +	const uint16_t max_list)
>
>and also for this old one:
>
>> @@ -121,6 +115,9 @@ rte_eth_devargs_parse_representor_ports(char *str,
>> void *data)
>
>

Will add them in next version, thanks.

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ajit Khaparde <ajit.khaparde@broadcom.com>
>Sent: Tuesday, January 19, 2021 2:18 AM
>To: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>
>Cc: Xueming(Steven) Li <xuemingl@nvidia.com>; Ferruh Yigit
><ferruh.yigit@intel.com>; Andrew Rybchenko
><andrew.rybchenko@oktetlabs.ru>; Olivier Matz <olivier.matz@6wind.com>;
>dpdk-dev <dev@dpdk.org>; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>Penso <asafp@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type
>
>On Mon, Jan 18, 2021 at 10:15 AM Thomas Monjalon <thomas@monjalon.net>
>wrote:
>>
>> 18/01/2021 19:00, Ajit Khaparde:
>> > On Mon, Jan 18, 2021 at 9:57 AM Thomas Monjalon
><thomas@monjalon.net> wrote:
>> > > 18/01/2021 18:42, Ajit Khaparde:
>> > > > On Mon, Jan 18, 2021 at 3:17 AM Xueming Li <xuemingl@nvidia.com>
>wrote:
>> > > > > +enum rte_eth_representor_type {
>> > > > > +       RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
>> > > > > +       RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
>> > > > > +       RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
>> > > > Till we get used to the terminology...
>> > > > Can we also have SF = "Sub Function" mentioned in the docs or
>comments?
>> > >
>> > > Are we sure about the definition?
>> > > I remember seeing SF = Scalable Function somewhere else (maybe from
>Intel)
>> > That complicates it. But if they mean the same thing, let's pick one.
>>
>> I think "Sub Function" and "Virtual Function" are easy to understand
>> for everybody.
>> I suggest picking these two for comments above.
>+1

There was an internal discussion and the conclusion is to align with kernel driver name.
Will update comment in next version, thanks!

>
>>

[dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Xueming Li]

dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to existing
VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For backward compatibility, this patch also introduces new netdev
capability to indicate the capability of supportting SF representor.

Version history:
 RFC:
 	initial version [2]
 V2:
    - separate patch for represnetor infrastructure, controller, pf and
      sf.
    - replace representor ID macro with functions:
      rte_eth_representor_id_encode()
      rte_eth_representor_id_parse()
    - new patch to allow devargs with same PCI BDF but different
      representors.
    - other minor code updates according to comments, thanks Andrew!
    - update document
 V3:
    - improve probing of allowed devargs with same name.
    - parse single word of kvargs as key. 
    - update kvargs test cases.
 V4:
    - split first representor refactor patch into
      1: add representor type
      2: refector representor list parsing
    - push the patch supporting multi-devargs for same device.
 V5:
    - add comments for parsing functions
    - update switch_representation.rst - Thanks Ajit



[1] SubFunction in kernel:
https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/

[2] RFC:
http://patchwork.dpdk.org/project/dpdk/list/?series=14376

[3] V2:
http://patchwork.dpdk.org/project/dpdk/list/?series=14559

[4] V3:
1 http://patchwork.dpdk.org/patch/86460/
2 http://patchwork.dpdk.org/patch/86461/
3 http://patchwork.dpdk.org/patch/86462/
4 http://patchwork.dpdk.org/patch/86463/
5 http://patchwork.dpdk.org/patch/86464/
6 http://patchwork.dpdk.org/patch/86465/
7 http://patchwork.dpdk.org/patch/86467/
8 http://patchwork.dpdk.org/patch/86466/
9 http://patchwork.dpdk.org/patch/86468/

[5] V4:
http://patchwork.dpdk.org/project/dpdk/list/?series=14809

Xueming Li (9):
  ethdev: introduce representor type
  ethdev: support representor port list
  ethdev: support new VF representor syntax
  ethdev: support sub function representor
  ethdev: support PF index in representor
  ethdev: support multi-host in representor
  devarg: change representor ID to bitmap
  ethdev: add capability of sub-function representor
  kvargs: update parser to support lists

 app/test/test_kvargs.c                        |  46 +++++-
 config/rte_config.h                           |   1 +
 doc/guides/prog_guide/poll_mode_drv.rst       |  13 +-
 .../prog_guide/switch_representation.rst      |  35 ++--
 drivers/net/bnxt/bnxt_ethdev.c                |   7 +
 drivers/net/bnxt/bnxt_reps.c                  |   3 +-
 drivers/net/enic/enic_ethdev.c                |   7 +
 drivers/net/enic/enic_vf_representor.c        |   3 +-
 drivers/net/i40e/i40e_ethdev.c                |   8 +
 drivers/net/i40e/i40e_vf_representor.c        |   3 +-
 drivers/net/ixgbe/ixgbe_ethdev.c              |   8 +
 drivers/net/ixgbe/ixgbe_vf_representor.c      |   3 +-
 drivers/net/mlx5/linux/mlx5_os.c              |  15 +-
 lib/librte_ethdev/ethdev_private.c            | 154 ++++++++++++------
 lib/librte_ethdev/ethdev_private.h            |   3 -
 lib/librte_ethdev/rte_class_eth.c             |  40 +++--
 lib/librte_ethdev/rte_ethdev.c                |  31 +++-
 lib/librte_ethdev/rte_ethdev.h                |   2 +
 lib/librte_ethdev/rte_ethdev_driver.h         |  66 ++++++++
 lib/librte_ethdev/version.map                 |   2 +
 lib/librte_kvargs/rte_kvargs.c                | 101 ++++++++----
 21 files changed, 428 insertions(+), 123 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH v5 1/9] ethdev: introduce representor type

[Xueming Li]

To support more representor type, this patch introduces representor type
enum. The enum is subject to extend for new types upcoming.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_ethdev.c        |  7 +++++++
 drivers/net/enic/enic_ethdev.c        |  7 +++++++
 drivers/net/i40e/i40e_ethdev.c        |  8 ++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c      |  8 ++++++++
 drivers/net/mlx5/linux/mlx5_os.c      | 11 +++++++++++
 lib/librte_ethdev/ethdev_private.c    |  5 +++++
 lib/librte_ethdev/rte_ethdev_driver.h |  9 +++++++++
 7 files changed, 55 insertions(+)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 74b0f3d1dc..d7c8b3ec07 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5586,6 +5586,13 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
 	int i, ret = 0;
 	struct rte_kvargs *kvlist = NULL;
 
+	if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
+			    eth_da->type);
+		return -ENOTSUP;
+	}
 	num_rep = eth_da->nb_representor_ports;
 	if (num_rep > BNXT_MAX_VF_REPS) {
 		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index d041a6bee9..dd085caa93 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1303,6 +1303,13 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (retval)
 			return retval;
 	}
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct enic),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index f083ac52fa..eb55d64585 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -638,6 +638,14 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 			return retval;
 	}
 
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct i40e_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index fa0f5afd03..6f1a710dae 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1717,6 +1717,14 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	} else
 		memset(&eth_da, 0, sizeof(eth_da));
 
+	if (eth_da.type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct ixgbe_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 9ac1d46b1b..caead107b0 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -705,6 +705,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				strerror(rte_errno));
 			return NULL;
 		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
+			/* Representor not specified. */
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+			rte_errno = ENOTSUP;
+			DRV_LOG(ERR, "unsupported representor type: %s",
+				dpdk_dev->devargs->args);
+			return NULL;
+		}
 		for (i = 0; i < eth_da.nb_representor_ports; ++i)
 			if (eth_da.representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 162a502fe7..c1a411dba4 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -111,11 +111,16 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	return 0;
 }
 
+/*
+ * representor format:
+ *   #: range or single number of VF representor
+ */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 }
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 0eacfd8425..cd33184f17 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1193,6 +1193,14 @@ __rte_internal
 int
 rte_eth_switch_domain_free(uint16_t domain_id);
 
+/** Ethernet device representor port type */
+enum rte_eth_representor_type {
+	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
+	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
+	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
+	RTE_ETH_REPRESENTOR_PF,   /**< representor of host Physical Function. */
+};
+
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
 	uint16_t ports[RTE_MAX_ETHPORTS];
@@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
 	/** representor port/s identifier to enable on device */
 	uint16_t nb_representor_ports;
 	/** number of ports in representor port field */
+	enum rte_eth_representor_type type; /* type of representor */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v5 2/9] ethdev: support representor port list

[Xueming Li]

To support extended representor syntax, this patch extends the
representor list parsing to support for representor port range in
devargs, examples:
   representor=[1,2,3]         - single list
   representor=[1,3-5,7,9-11]  - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 lib/librte_ethdev/ethdev_private.c | 124 ++++++++++++++++-------------
 lib/librte_ethdev/ethdev_private.h |   3 -
 lib/librte_ethdev/rte_class_eth.c  |   4 +-
 lib/librte_ethdev/rte_ethdev.c     |   5 +-
 4 files changed, 72 insertions(+), 64 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index c1a411dba4..e87636f665 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -38,82 +38,93 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
 	return NULL;
 }
 
-int
-rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data)
+/* Put new value into list. */
+static int
+rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
+		       const uint16_t max_list, uint16_t val)
 {
-	char *str_start;
-	int state;
-	int result;
+	uint16_t i;
 
-	if (*str != '[')
-		/* Single element, not a list */
-		return callback(str, data);
-
-	/* Sanity check, then strip the brackets */
-	str_start = &str[strlen(str) - 1];
-	if (*str_start != ']') {
-		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
-		return -EINVAL;
-	}
-	str++;
-	*str_start = '\0';
-
-	/* Process list elements */
-	state = 0;
-	while (1) {
-		if (state == 0) {
-			if (*str == '\0')
-				break;
-			if (*str != ',') {
-				str_start = str;
-				state = 1;
-			}
-		} else if (state == 1) {
-			if (*str == ',' || *str == '\0') {
-				if (str > str_start) {
-					/* Non-empty string fragment */
-					*str = '\0';
-					result = callback(str_start, data);
-					if (result < 0)
-						return result;
-				}
-				state = 0;
-			}
-		}
-		str++;
+	if (*len_list >= max_list)
+		return -1;
+	for (i = 0; i < *len_list; i++) {
+		if (list[i] == val)
+			return 0;
 	}
+	list[(*len_list)++] = val;
 	return 0;
 }
 
-static int
+/* Parse and enlist a range expression of "min-max" or a single value. */
+static char *
 rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	const uint16_t max_list)
 {
 	uint16_t lo, hi, val;
 	int result;
+	char *pos = str;
 
 	result = sscanf(str, "%hu-%hu", &lo, &hi);
 	if (result == 1) {
-		if (*len_list >= max_list)
-			return -ENOMEM;
-		list[(*len_list)++] = lo;
+		if (rte_eth_devargs_enlist(list, len_list, max_list, lo) != 0)
+			return NULL;
 	} else if (result == 2) {
-		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
-			return -EINVAL;
+		if (lo >= hi)
+			return NULL;
 		for (val = lo; val <= hi; val++) {
-			if (*len_list >= max_list)
-				return -ENOMEM;
-			list[(*len_list)++] = val;
+			if (rte_eth_devargs_enlist(list, len_list, max_list,
+						   val) != 0)
+				return NULL;
 		}
 	} else
-		return -EINVAL;
-	return 0;
+		return NULL;
+	while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
+		pos++;
+	return pos;
+}
+
+/*
+ * Parse list of values separated by ",".
+ * Each value could be a range [min-max] or single number.
+ * Examples:
+ *  2               - single
+ *  [1,2,3]         - single list
+ *  [1,3-5,7,9-11]  - list with singles and ranges
+ */
+static char *
+rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
+	const uint16_t max_list)
+{
+	char *pos = str;
+
+	if (*pos == '[')
+		pos++;
+	while (1) {
+		pos = rte_eth_devargs_process_range(pos, list, len_list,
+						    max_list);
+		if (pos == NULL)
+			return NULL;
+		if (*pos != ',') /* end of list */
+			break;
+		pos++;
+	}
+	if (*str == '[' && *pos != ']')
+		return NULL;
+	if (*pos == ']')
+		pos++;
+	return pos;
 }
 
 /*
- * representor format:
+ * Parse representor ports from a single value or lists.
+ *
+ * Representor format:
  *   #: range or single number of VF representor
+ *
+ * Examples of #:
+ *  2               - single
+ *  [1,2,3]         - single list
+ *  [1,3-5,7,9-11]  - list with singles and ranges
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -121,6 +132,9 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
+	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+	if (str == NULL)
+		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
+	return str == NULL ? -1 : 0;
 }
diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
index 905a45c337..220ddd4408 100644
--- a/lib/librte_ethdev/ethdev_private.h
+++ b/lib/librte_ethdev/ethdev_private.h
@@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
 		const void *data);
 
 /* Parse devargs value for representor parameter. */
-typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
-int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data);
 int rte_eth_devargs_parse_representor_ports(char *str, void *data);
 
 #ifdef __cplusplus
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index 6338355e25..efe6149df5 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
 	if (values == NULL)
 		return -1;
 	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_list(values,
-			rte_eth_devargs_parse_representor_ports,
-			&representors);
+	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 71e1e9a6db..91b3263338 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5561,9 +5561,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
-			result = rte_eth_devargs_parse_list(pair->value,
-				rte_eth_devargs_parse_representor_ports,
-				eth_da);
+			result = rte_eth_devargs_parse_representor_ports(
+					pair->value, eth_da);
 			if (result < 0)
 				goto parse_cleanup;
 		}
-- 
2.25.1

[dpdk-dev] [PATCH v5 3/9] ethdev: support new VF representor syntax

[Xueming Li]

Current VF representor syntax:
 representor=2          - single representor
 representor=[0-3]      - single range

To prepare for more representor types, this patch adds compatible VF
representor devargs syntax:

vf#:
 representor=vf2          - single representor
 representor=vf[1,3,5]    - single list
 representor=vf[0-3]      - single range
 representor=vf[0,1,4-7]  - list with singles and range

For backwards compatibility, representor "#" is interpreted as "vf#".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 7 ++++---
 lib/librte_ethdev/ethdev_private.c      | 5 ++++-
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 239ec820ea..0117c2af07 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -374,9 +374,10 @@ parameters to those ports.
   this argument allows user to specify which switch ports to enable port
   representors for.::
 
-   -a DBDF,representor=0
-   -a DBDF,representor=[0,4,6,9]
-   -a DBDF,representor=[0-31]
+   -a DBDF,representor=vf0
+   -a DBDF,representor=vf[0,4,6,9]
+   -a DBDF,representor=vf[0-31]
+   -a DBDF,representor=vf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index e87636f665..20e48a48be 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -119,7 +119,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Parse representor ports from a single value or lists.
  *
  * Representor format:
- *   #: range or single number of VF representor
+ *   #: range or single number of VF representor - legacy
+ *   vf#: VF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -132,6 +133,8 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	if (str[0] == 'v' && str[1] == 'f')
+		str += 2;
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 	if (str == NULL)
-- 
2.25.1

[dpdk-dev] [PATCH v5 4/9] ethdev: support sub function representor

[Xueming Li]

SubFunction is a portion of the PCI device, created on demand, a SF
netdev has its own dedicated queues(txq, rxq). A SF netdev supports
eswitch representation offload similar to existing PF and VF
representors.

To support SF representor, this patch introduces new devargs syntax,
examples:
 representor=sf0               - single SubFunction representor
 representor=sf[1,3,5]         - single list
 representor=sf[0-3],          - single range
 representor=sf[0,2-6,8,10-12] - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 doc/guides/prog_guide/poll_mode_drv.rst       |  4 +++
 .../prog_guide/switch_representation.rst      | 35 +++++++++++++------
 lib/librte_ethdev/ethdev_private.c            | 11 ++++--
 3 files changed, 38 insertions(+), 12 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 0117c2af07..86e5867f1b 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -378,6 +378,10 @@ parameters to those ports.
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
    -a DBDF,representor=vf[0,2-4,7,9-11]
+   -a DBDF,representor=sf0
+   -a DBDF,representor=sf[1,3,5]
+   -a DBDF,representor=sf[0-1023]
+   -a DBDF,representor=sf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/doc/guides/prog_guide/switch_representation.rst b/doc/guides/prog_guide/switch_representation.rst
index 07ba12bea6..ff6aa91c80 100644
--- a/doc/guides/prog_guide/switch_representation.rst
+++ b/doc/guides/prog_guide/switch_representation.rst
@@ -13,7 +13,7 @@ Introduction
 
 Network adapters with multiple physical ports and/or SR-IOV capabilities
 usually support the offload of traffic steering rules between their virtual
-functions (VFs), physical functions (PFs) and ports.
+functions (VFs), sub functions (SFs), physical functions (PFs) and ports.
 
 Like for standard Ethernet switches, this involves a combination of
 automatic MAC learning and manual configuration. For most purposes it is
@@ -24,7 +24,7 @@ layer 2 (L2) traffic (such as OVS) need to steer traffic themselves
 according on their own criteria.
 
 Without a standard software interface to manage traffic steering rules
-between VFs, PFs and the various physical ports of a given device,
+between VFs, SFs, PFs and the various physical ports of a given device,
 applications cannot take advantage of these offloads; software processing is
 mandatory even for traffic which ends up re-injected into the device it
 originates from.
@@ -34,6 +34,17 @@ the DPDK flow API (**rte_flow**), with emphasis on the SR-IOV use case
 (PF/VF steering) using a single physical port for clarity, however the same
 logic applies to any number of ports without necessarily involving SR-IOV.
 
+Sub Function
+------------
+Besides SR-IOV, Sub function is a portion of the PCI device, a SF netdev
+has its own dedicated queues(txq, rxq). A SF netdev supports E-Switch
+representation offload similar to existing PF and VF representors.
+A SF shares PCI level resources with other SFs and/or with its parent PCI
+function.
+
+Sub function is created on-demand, coexists with VFs. Number of SFs is
+limited by hardware resources.
+
 Port Representors
 -----------------
 
@@ -42,15 +53,16 @@ applications usually have to process a bit of traffic in software before
 thinking about offloading specific flows to hardware.
 
 Applications therefore need the ability to receive and inject traffic to
-various device endpoints (other VFs, PFs or physical ports) before
+various device endpoints (other VFs, SFs, PFs or physical ports) before
 connecting them together. Device drivers must provide means to hook the
 "other end" of these endpoints and to refer them when configuring flow
 rules.
 
 This role is left to so-called "port representors" (also known as "VF
-representors" in the specific context of VFs), which are to DPDK what the
-Ethernet switch device driver model (**switchdev**) [1]_ is to Linux, and
-which can be thought as a software "patch panel" front-end for applications.
+representors" in the specific context of VFs, "SF representors" in the
+specific context of SFs), which are to DPDK what the Ethernet switch
+device driver model (**switchdev**) [1]_ is to Linux, and which can be
+thought as a software "patch panel" front-end for applications.
 
 - DPDK port representors are implemented as additional virtual Ethernet
   device (**ethdev**) instances, spawned on an as needed basis through
@@ -59,9 +71,12 @@ which can be thought as a software "patch panel" front-end for applications.
 
 ::
 
-   -a pci:dbdf,representor=0
-   -a pci:dbdf,representor=[0-3]
-   -a pci:dbdf,representor=[0,5-11]
+   -a pci:dbdf,representor=vf0
+   -a pci:dbdf,representor=vf[0-3]
+   -a pci:dbdf,representor=vf[0,5-11]
+   -a pci:dbdf,representor=sf1
+   -a pci:dbdf,representor=sf[0-1023]
+   -a pci:dbdf,representor=sf[0,2-1023]
 
 - As virtual devices, they may be more limited than their physical
   counterparts, for instance by exposing only a subset of device
@@ -360,7 +375,7 @@ Compared to creating a brand new dedicated interface, **rte_flow** was
 deemed flexible enough to manage representor traffic only with minor
 extensions:
 
-- Using physical ports, PF, VF or port representors as targets.
+- Using physical ports, PF, SF, VF or port representors as targets.
 
 - Affecting traffic that is not necessarily addressed to the DPDK port ID a
   flow rule is associated with (e.g. forcing VF traffic redirection to PF).
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 20e48a48be..d513f035d0 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -121,6 +121,7 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Representor format:
  *   #: range or single number of VF representor - legacy
  *   vf#: VF port representor/s
+ *   sf#: SF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -132,9 +133,15 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
-	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	if (str[0] == 'v' && str[1] == 'f')
+	if (str[0] == 'v' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
+	} else if (str[0] == 's' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_SF;
+		str += 2;
+	} else {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 	if (str == NULL)
-- 
2.25.1

[dpdk-dev] [PATCH v5 5/9] ethdev: support PF index in representor

[Xueming Li]

With Kernel bonding, multiple underlying PFs are bonded, VFs come
from different PF, need to identify representor of VFs unambiguously by
adding PF index.

This patch introduces optional 'pf' section to representor devargs
syntax, examples:
 representor=pf0vf0             - single VF representor
 representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs


Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  2 ++
 lib/librte_ethdev/ethdev_private.c      | 13 +++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 86e5867f1b..b2147aad30 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -382,6 +382,8 @@ parameters to those ports.
    -a DBDF,representor=sf[1,3,5]
    -a DBDF,representor=sf[0-1023]
    -a DBDF,representor=sf[0,2-4,7,9-11]
+   -a DBDF,representor=pf1vf0
+   -a DBDF,representor=pf[0-1]sf[0-127]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index d513f035d0..b9fdbd0f72 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -120,8 +120,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   vf#: VF port representor/s
- *   sf#: SF port representor/s
+ *   [pf#]vf#: VF port representor/s
+ *   [pf#]sf#: SF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -133,6 +133,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'p' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_PF;
+		str += 2;
+		str = rte_eth_devargs_process_list(str, eth_da->ports,
+				&eth_da->nb_ports, RTE_MAX_ETHPORTS);
+		if (str == NULL)
+			goto err;
+	}
 	if (str[0] == 'v' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
@@ -144,6 +152,7 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+err:
 	if (str == NULL)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
 	return str == NULL ? -1 : 0;
-- 
2.25.1

[dpdk-dev] [PATCH v5 6/9] ethdev: support multi-host in representor

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to the multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. To provide the unambiguous identification
of the PCIe function the controller index is added. The full representor
identifier consists of three indices - controller index, PF index, and
VF or SF index (if any).

This patch introduces controller index to ethdev representor syntax,
examples:

[[c#]pf#]vf#: VF port representor/s, example: pf0vf1
[[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]

c# is controller(host) ID/range in case of multi-host, optional.

For user application (e.g. OVS), PMD is responsible to interpret and
locate representor device based on controller ID, PF ID and VF/SF ID in
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 config/rte_config.h                   |  1 +
 lib/librte_ethdev/ethdev_private.c    | 12 ++++++++++--
 lib/librte_ethdev/rte_ethdev_driver.h |  4 ++++
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index a0b5160ff2..23d02d51ef 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -58,6 +58,7 @@
 #define RTE_MAX_QUEUES_PER_PORT 1024
 #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
 #define RTE_ETHDEV_RXTX_CALLBACKS 1
+#define RTE_MAX_MULTI_HOST_CTRLS 4
 
 /* cryptodev defines */
 #define RTE_CRYPTO_MAX_DEVS 64
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index b9fdbd0f72..9a0945c22e 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -120,8 +120,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   [pf#]vf#: VF port representor/s
- *   [pf#]sf#: SF port representor/s
+ *   [[c#]pf#]vf#: VF port representor/s
+ *   [[c#]pf#]sf#: SF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -133,6 +133,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'c') {
+		str += 1;
+		str = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
+				&eth_da->nb_mh_controllers,
+				RTE_DIM(eth_da->mh_controllers));
+		if (str == NULL)
+			goto err;
+	}
 	if (str[0] == 'p' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_PF;
 		str += 2;
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index cd33184f17..b01f118965 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1203,6 +1203,10 @@ enum rte_eth_representor_type {
 
 /** Generic Ethernet device arguments  */
 struct rte_eth_devargs {
+	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
+	/** controller/s number in case of multi-host */
+	uint16_t nb_mh_controllers;
+	/** number of controllers in multi-host controllers field */
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
 	uint16_t nb_ports;
-- 
2.25.1

[dpdk-dev] [PATCH v5 7/9] devarg: change representor ID to bitmap

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

SR-IOV and SubFunction are created on top of PF. PF index is introduced
because there might be multiple PFs in the bonding configuration and
only bonding device is probed.

In eth representor comparator callback, ethdev was compared with devarg.
Since ethdev representor port didn't contain controller index and PF
index information, callback returned representor from other PF or
controller.

This patch changes representor ID to bitmap so that the ethdev
representor comparer callback returns correct ethdev by comparing full
representor information including: controller index, PF index,
representor type, SF or VF index.

Representor ID bitmap definition:
 xxxx xxxx xxxx xxxx
 |||| |LLL LLLL LLLL vf/sf id
 |||| L 1:sf, 0:vf
 ||LL pf id
 LL controller(host) id

This approach keeps binary compatibility with all drivers, VF
representor id matches with simple id for non-bonding and non-multi-host
configurations.

In the future, the representor ID field and each section should extend
to bigger width to support more devices.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/bnxt/bnxt_reps.c             |  3 +-
 drivers/net/enic/enic_vf_representor.c   |  3 +-
 drivers/net/i40e/i40e_vf_representor.c   |  3 +-
 drivers/net/ixgbe/ixgbe_vf_representor.c |  3 +-
 drivers/net/mlx5/linux/mlx5_os.c         |  4 +-
 lib/librte_ethdev/rte_class_eth.c        | 38 +++++++++++++----
 lib/librte_ethdev/rte_ethdev.c           | 26 ++++++++++++
 lib/librte_ethdev/rte_ethdev_driver.h    | 53 ++++++++++++++++++++++++
 lib/librte_ethdev/version.map            |  2 +
 9 files changed, 122 insertions(+), 13 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index f7bbf77d3f..34febc2d1e 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -186,7 +186,8 @@ int bnxt_representor_init(struct rte_eth_dev *eth_dev, void *params)
 
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	eth_dev->data->representor_id = rep_params->vf_id;
+	eth_dev->data->representor_id = rte_eth_representor_id_encode(
+		0, 0, RTE_ETH_REPRESENTOR_VF, rep_params->vf_id);
 
 	rte_eth_random_addr(vf_rep_bp->dflt_mac_addr);
 	memcpy(vf_rep_bp->mac_addr, vf_rep_bp->dflt_mac_addr,
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index c2c03c0281..632317af15 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -674,7 +674,8 @@ int enic_vf_representor_init(struct rte_eth_dev *eth_dev, void *init_params)
 	eth_dev->dev_ops = &enic_vf_representor_dev_ops;
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	eth_dev->data->representor_id = vf->vf_id;
+	eth_dev->data->representor_id = rte_eth_representor_id_encode(
+		0, 0, RTE_ETH_REPRESENTOR_VF, vf->vf_id);
 	eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
 		sizeof(struct rte_ether_addr) *
 		ENIC_UNICAST_PERFECT_FILTERS, 0);
diff --git a/drivers/net/i40e/i40e_vf_representor.c b/drivers/net/i40e/i40e_vf_representor.c
index 9e40406a3d..d90d0fdb9d 100644
--- a/drivers/net/i40e/i40e_vf_representor.c
+++ b/drivers/net/i40e/i40e_vf_representor.c
@@ -510,7 +510,8 @@ i40e_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
 
 	ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
 					RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
-	ethdev->data->representor_id = representor->vf_id;
+	ethdev->data->representor_id = rte_eth_representor_id_encode(
+			0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
 
 	/* Setting the number queues allocated to the VF */
 	ethdev->data->nb_rx_queues = vf->vsi->nb_qps;
diff --git a/drivers/net/ixgbe/ixgbe_vf_representor.c b/drivers/net/ixgbe/ixgbe_vf_representor.c
index 8185f0d3bb..e15b794761 100644
--- a/drivers/net/ixgbe/ixgbe_vf_representor.c
+++ b/drivers/net/ixgbe/ixgbe_vf_representor.c
@@ -196,7 +196,8 @@ ixgbe_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
 		return -ENODEV;
 
 	ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
-	ethdev->data->representor_id = representor->vf_id;
+	ethdev->data->representor_id = rte_eth_representor_id_encode(
+			0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
 
 	/* Set representor device ops */
 	ethdev->dev_ops = &ixgbe_vf_representor_dev_ops;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index caead107b0..4d7940bcca 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1025,7 +1025,9 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 #endif
 	/* representor_id field keeps the unmodified VF index. */
 	priv->representor_id = switch_info->representor ?
-			       switch_info->port_name : -1;
+		rte_eth_representor_id_encode(0, 0, RTE_ETH_REPRESENTOR_VF,
+					      switch_info->port_name) :
+		-1;
 	/*
 	 * Look for sibling devices in order to reuse their switch domain
 	 * if any, otherwise allocate one.
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index efe6149df5..994db96960 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -66,8 +66,8 @@ eth_representor_cmp(const char *key __rte_unused,
 	int ret;
 	char *values;
 	const struct rte_eth_dev_data *data = opaque;
-	struct rte_eth_devargs representors;
-	uint16_t index;
+	struct rte_eth_devargs eth_da;
+	uint16_t index, c, p, f;
 
 	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
 		return -1; /* not a representor port */
@@ -76,17 +76,39 @@ eth_representor_cmp(const char *key __rte_unused,
 	values = strdup(value);
 	if (values == NULL)
 		return -1;
-	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
+	memset(&eth_da, 0, sizeof(eth_da));
+	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
 
+	/* Set default values. */
+	if (eth_da.nb_mh_controllers == 0) {
+		eth_da.nb_mh_controllers = 1;
+		eth_da.mh_controllers[0] = 0;
+	}
+	if (eth_da.nb_ports == 0) {
+		eth_da.nb_ports = 1;
+		eth_da.ports[0] = 0;
+	}
+	if (eth_da.nb_representor_ports == 0) {
+		eth_da.nb_representor_ports = 1;
+		eth_da.representor_ports[0] = 0;
+	}
 	/* Return 0 if representor id is matching one of the values. */
-	for (index = 0; index < representors.nb_representor_ports; index++)
-		if (data->representor_id ==
-				representors.representor_ports[index])
-			return 0;
+	for (c = 0; c < eth_da.nb_mh_controllers; ++c) {
+		for (p = 0; p < eth_da.nb_ports; ++p) {
+			for (f = 0; f < eth_da.nb_representor_ports; ++f) {
+				index = rte_eth_representor_id_encode(
+					eth_da.mh_controllers[c],
+					eth_da.ports[p],
+					eth_da.type,
+					eth_da.representor_ports[f]);
+				if (data->representor_id == index)
+					return 0;
+			}
+		}
+	}
 	return -1; /* no match */
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 91b3263338..2cac0ccfbd 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5575,6 +5575,32 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	return result;
 }
 
+uint16_t
+rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
+			      enum rte_eth_representor_type type,
+			      uint16_t representor_port)
+{
+	return (((controller & 3) << 14) |
+		((pf & 3) << 12) |
+		(!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
+		(representor_port & 0x7ff));
+}
+
+uint16_t
+rte_eth_representor_id_parse(const uint16_t representor_id,
+			     uint16_t *controller, uint16_t *pf,
+			     enum rte_eth_representor_type *type)
+{
+	if (controller)
+		*controller = (representor_id >> 14) & 3;
+	if (pf)
+		*pf = (representor_id >> 12) & 3;
+	if (type)
+		*type = ((representor_id >> 11) & 1) ?
+			RTE_ETH_REPRESENTOR_SF : RTE_ETH_REPRESENTOR_VF;
+	return representor_id & 0x7ff;
+}
+
 static int
 eth_dev_handle_port_list(const char *cmd __rte_unused,
 		const char *params __rte_unused,
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index b01f118965..57253c8f90 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -1218,6 +1218,59 @@ struct rte_eth_devargs {
 	enum rte_eth_representor_type type; /* type of representor */
 };
 
+#define RTE_NO_REPRESENTOR_ID UINT16_MAX /**< No representor ID. */
+
+/**
+ * PMD helper function to encode representor ID
+ *
+ * The compact format is used for device iterator that comparing
+ * ethdev representor ID with target devargs.
+ *
+ * xxxx xxxx xxxx xxxx
+ * |||| |LLL LLLL LLLL vf/sf id
+ * |||| L 1:sf, 0:vf
+ * ||LL pf id
+ * LL controller(host) id
+ *
+ * @param controller
+ *  Controller ID.
+ * @param pf
+ *  PF port ID.
+ * @param type
+ *  Representor type.
+ * @param representor_port
+ *  Representor port ID.
+ *
+ * @return
+ *   Encoded representor ID.
+ */
+__rte_internal
+uint16_t
+rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
+			      enum rte_eth_representor_type type,
+			      uint16_t representor_port);
+
+/**
+ * PMD helper function to parse representor ID
+ *
+ * @param representor_id
+ *  Representor ID.
+ * @param controller
+ *  Parsed controller ID.
+ * @param pf
+ *  Parsed PF port ID.
+ * @param type
+ *  Parsed representor type.
+ *
+ * @return
+ *   Parsed representor port ID.
+ */
+__rte_internal
+uint16_t
+rte_eth_representor_id_parse(const uint16_t representor_id,
+			     uint16_t *controller, uint16_t *pf,
+			     enum rte_eth_representor_type *type);
+
 /**
  * PMD helper function to parse ethdev arguments
  *
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index d3f5410806..44edaed507 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -257,6 +257,8 @@ INTERNAL {
 	rte_eth_dev_release_port;
 	rte_eth_dev_internal_reset;
 	rte_eth_devargs_parse;
+	rte_eth_representor_id_encode;
+	rte_eth_representor_id_parse;
 	rte_eth_dma_zone_free;
 	rte_eth_dma_zone_reserve;
 	rte_eth_hairpin_queue_peer_bind;
-- 
2.25.1

[dpdk-dev] [PATCH v5 8/9] ethdev: add capability of sub-function representor

[Xueming Li]

Old DPDK version or some drivers didn't support SubFunction representor.
For application to adapt different DPDK version automatically, or to be
used for different NICs, this patch introduces new eth device capability
of supporting SubFunction representor device.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 lib/librte_ethdev/rte_ethdev.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 2cbce958cf..da65adf1ab 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1433,6 +1433,8 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/** Device supports SubFunction representor. */
+#define RTE_ETH_DEV_CAPA_REPRESENTOR_SF         0x00000004
 /**@}*/
 
 /*
-- 
2.25.1

[dpdk-dev] [PATCH v5 9/9] kvargs: update parser to support lists

[Xueming Li]

This patch updates kvargs parser to support lists on top of range, allow
multiple lists or range:
  k1=a[1,2]b[3-5]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test/test_kvargs.c         |  46 +++++++++++++--
 lib/librte_kvargs/rte_kvargs.c | 101 +++++++++++++++++++++++----------
 2 files changed, 112 insertions(+), 35 deletions(-)

diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index 2a2dae43a0..a91ea8dc47 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -35,6 +35,25 @@ static int check_handler(const char *key, const char *value,
 	return 0;
 }
 
+/* test parsing. */
+static int test_kvargs_parsing(const char *args, unsigned int n)
+{
+	struct rte_kvargs *kvlist;
+
+	kvlist = rte_kvargs_parse(args, NULL);
+	if (kvlist == NULL) {
+		printf("rte_kvargs_parse() error: %s\n", args);
+		return -1;
+	}
+	if (kvlist->count != n) {
+		printf("invalid count value %d: %s\n", kvlist->count, args);
+		rte_kvargs_free(kvlist);
+		return -1;
+	}
+	rte_kvargs_free(kvlist);
+	return 0;
+}
+
 /* test a valid case */
 static int test_valid_kvargs(void)
 {
@@ -42,6 +61,19 @@ static int test_valid_kvargs(void)
 	const char *args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
 	const char **valid_keys;
+	static const struct {
+		unsigned int expected;
+		const char *input;
+	} valid_inputs[] = {
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo" },
+		{ 2, "foo=1,=2" },
+		{ 1, "foo=[1,2" },
+		{ 1, ",=" },
+		{ 1, "foo=[" },
+	};
+	unsigned int i;
 
 	/* empty args is valid */
 	args = "";
@@ -191,6 +223,14 @@ static int test_valid_kvargs(void)
 	}
 	rte_kvargs_free(kvlist);
 
+	valid_keys = NULL;
+
+	for (i = 0; i < RTE_DIM(valid_inputs); ++i) {
+		args = valid_inputs[i].input;
+		if (test_kvargs_parsing(args, valid_inputs[i].expected))
+			goto fail;
+	}
+
 	return 0;
 
  fail:
@@ -212,12 +252,6 @@ static int test_invalid_kvargs(void)
 	/* list of argument that should fail */
 	const char *args_list[] = {
 		"wrong-key=x",     /* key not in valid_keys_list */
-		"foo=1,foo=",      /* empty value */
-		"foo=1,foo",       /* no value */
-		"foo=1,=2",        /* no key */
-		"foo=[1,2",        /* no closing bracket in value */
-		",=",              /* also test with a smiley */
-		"foo=[",           /* no value in list and no closing bracket */
 		NULL };
 	const char **args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
index 285081c86c..ffae8914cf 100644
--- a/lib/librte_kvargs/rte_kvargs.c
+++ b/lib/librte_kvargs/rte_kvargs.c
@@ -5,6 +5,7 @@
 
 #include <string.h>
 #include <stdlib.h>
+#include <stdbool.h>
 
 #include <rte_string_fns.h>
 
@@ -13,15 +14,19 @@
 /*
  * Receive a string with a list of arguments following the pattern
  * key=value,key=value,... and insert them into the list.
- * strtok() is used so the params string will be copied to be modified.
+ * Params string will be copied to be modified.
+ * list "[]" and list element splitter ",", "-" is treated as value.
+ * Supported examples:
+ *   k1=v1,k2=v2
+ *   k1
+ *   k1=x[0-1]y[1,3-5,9]z
  */
 static int
 rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 {
-	unsigned i;
-	char *str;
-	char *ctx1 = NULL;
-	char *ctx2 = NULL;
+	char *str, *start;
+	bool in_list = false, end_key = false, end_value = false;
+	bool save = false, end_pair = false;
 
 	/* Copy the const char *params to a modifiable string
 	 * to pass to rte_strsplit
@@ -32,36 +37,74 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 
 	/* browse each key/value pair and add it in kvlist */
 	str = kvlist->str;
-	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
+	start = str; /* start of current key or value */
+	while (1) {
+		switch (*str) {
+		case '=': /* End of key. */
+			end_key = true;
+			save = true;
+			break;
+		case ',':
+			/* End of value, skip comma in middle of range */
+			if (!in_list) {
+				if (end_key)
+					end_value = true;
+				else
+					end_key = true;
+				save = true;
+				end_pair = true;
+			}
+			break;
+		case '[': /* Start of list. */
+			in_list = true;
+			break;
+		case ']': /* End of list.  */
+			if (in_list)
+				in_list = false;
+			break;
+		case '\0': /* End of string */
+			if (end_key)
+				end_value = true;
+			else
+				end_key = true;
+			save = true;
+			end_pair = true;
+			break;
+		default:
+			break;
+		}
 
-		i = kvlist->count;
-		if (i >= RTE_KVARGS_MAX)
-			return -1;
+		if (!save) {
+			/* Continue if not end of key or value. */
+			str++;
+			continue;
+		}
 
-		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
-		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
-		if (kvlist->pairs[i].key == NULL ||
-		    kvlist->pairs[i].value == NULL)
+		if (kvlist->count >= RTE_KVARGS_MAX)
 			return -1;
 
-		/* Detect list [a,b] to skip comma delimiter in list. */
-		str = kvlist->pairs[i].value;
-		if (str[0] == '[') {
-			/* Find the end of the list. */
-			while (str[strlen(str) - 1] != ']') {
-				/* Restore the comma erased by strtok_r(). */
-				if (ctx1 == NULL || ctx1[0] == '\0')
-					return -1; /* no closing bracket */
-				str[strlen(str)] = ',';
-				/* Parse until next comma. */
-				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
-				if (str == NULL)
-					return -1; /* no closing bracket */
-			}
+		if (end_value)
+			/* Value parsed */
+			kvlist->pairs[kvlist->count].value = start;
+		else if (end_key)
+			/* Key parsed. */
+			kvlist->pairs[kvlist->count].key = start;
+
+		if (end_pair) {
+			if (end_value || str != start)
+				/* Ignore empty pair. */
+				kvlist->count++;
+			end_key = false;
+			end_value = false;
+			end_pair = false;
 		}
 
-		kvlist->count++;
-		str = NULL;
+		if (*str == '\0') /* End of string. */
+			break;
+		*str = '\0';
+		str++;
+		start = str;
+		save = false;
 	}
 
 	return 0;
-- 
2.25.1

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Andrew Rybchenko]

On 1/18/21 2:16 PM, Xueming Li wrote:
> To support more representor type, this patch introduces representor type
> enum. The enum is subject to extend for new types upcoming.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

One nit below and a question below.

In any case:

Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

[snip]

> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index 0eacfd8425..3bc5c5bbbb 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -1193,6 +1193,14 @@ __rte_internal
>  int
>  rte_eth_switch_domain_free(uint16_t domain_id);
>  
> +/** Ethernet device representor type */
> +enum rte_eth_representor_type {
> +	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
> +	RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
> +	RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
> +	RTE_ETH_REPRESENTOR_PF,   /**< representor of host PF. */

RTE_ETH_REPRESENTOR_SF and PF looks dead in the patch.
IMHO, addition of these members here make future patches
which add support inconsistent.

> +};
> +
>  /** Generic Ethernet device arguments  */
>  struct rte_eth_devargs {
>  	uint16_t ports[RTE_MAX_ETHPORTS];
> @@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
>  	/** representor port/s identifier to enable on device */
>  	uint16_t nb_representor_ports;
>  	/** number of ports in representor port field */
> +	enum rte_eth_representor_type type; /* type of representor */

Is it intended and documented limitation that we can't add
different type representors in one request? Or am I missing
something and it is possible?

>  };
>  
>  /**
> 

Re: [dpdk-dev] [PATCH v5 7/9] devarg: change representor ID to bitmap

[Wang, Haiyue]

> -----Original Message-----
> From: Xueming Li <xuemingl@nvidia.com>
> Sent: Tuesday, January 19, 2021 15:15
> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; xuemingl@nvidia.com; Asaf Penso
> <asafp@nvidia.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>; Somnath Kotur
> <somnath.kotur@broadcom.com>; Daley, John <johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>;
> Xing, Beilei <beilei.xing@intel.com>; Guo, Jia <jia.guo@intel.com>; Wang, Haiyue
> <haiyue.wang@intel.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Thomas
> Monjalon <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew Rybchenko
> <andrew.rybchenko@oktetlabs.ru>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
> Subject: [PATCH v5 7/9] devarg: change representor ID to bitmap
> 
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> SR-IOV and SubFunction are created on top of PF. PF index is introduced
> because there might be multiple PFs in the bonding configuration and
> only bonding device is probed.
> 
> In eth representor comparator callback, ethdev was compared with devarg.
> Since ethdev representor port didn't contain controller index and PF
> index information, callback returned representor from other PF or
> controller.
> 
> This patch changes representor ID to bitmap so that the ethdev
> representor comparer callback returns correct ethdev by comparing full
> representor information including: controller index, PF index,
> representor type, SF or VF index.
> 
> Representor ID bitmap definition:
>  xxxx xxxx xxxx xxxx
>  |||| |LLL LLLL LLLL vf/sf id
>  |||| L 1:sf, 0:vf
>  ||LL pf id
>  LL controller(host) id
> 
> This approach keeps binary compatibility with all drivers, VF
> representor id matches with simple id for non-bonding and non-multi-host
> configurations.
> 
> In the future, the representor ID field and each section should extend
> to bigger width to support more devices.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  drivers/net/bnxt/bnxt_reps.c             |  3 +-
>  drivers/net/enic/enic_vf_representor.c   |  3 +-
>  drivers/net/i40e/i40e_vf_representor.c   |  3 +-
>  drivers/net/ixgbe/ixgbe_vf_representor.c |  3 +-
>  drivers/net/mlx5/linux/mlx5_os.c         |  4 +-
>  lib/librte_ethdev/rte_class_eth.c        | 38 +++++++++++++----
>  lib/librte_ethdev/rte_ethdev.c           | 26 ++++++++++++
>  lib/librte_ethdev/rte_ethdev_driver.h    | 53 ++++++++++++++++++++++++
>  lib/librte_ethdev/version.map            |  2 +
>  9 files changed, 122 insertions(+), 13 deletions(-)
> 


> +uint16_t
> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
> +			      enum rte_eth_representor_type type,
> +			      uint16_t representor_port)
> +{
> +	return (((controller & 3) << 14) |
> +		((pf & 3) << 12) |
> +		(!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
> +		(representor_port & 0x7ff));
> +}
> +
> +uint16_t
> +rte_eth_representor_id_parse(const uint16_t representor_id,
> +			     uint16_t *controller, uint16_t *pf,
> +			     enum rte_eth_representor_type *type)
> +{
> +	if (controller)
> +		*controller = (representor_id >> 14) & 3;
> +	if (pf)
> +		*pf = (representor_id >> 12) & 3;
> +	if (type)
> +		*type = ((representor_id >> 11) & 1) ?
> +			RTE_ETH_REPRESENTOR_SF : RTE_ETH_REPRESENTOR_VF;
> +	return representor_id & 0x7ff;
> +}
> +

Rename 'parse' to 'decode' ? Since not parse from string. ;-)
The these two functions are pair style.

rte_eth_representor_id_encode / rte_eth_representor_id_decode ?

> 2.25.1

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Xueming(Steven) Li]

Hi Andrew,

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 3:25 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; NBU-Contact-Thomas
>Monjalon <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>;
>Olivier Matz <olivier.matz@6wind.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>
>Subject: Re: [PATCH v4 1/9] ethdev: introduce representor type
>
>On 1/18/21 2:16 PM, Xueming Li wrote:
>> To support more representor type, this patch introduces representor
>> type enum. The enum is subject to extend for new types upcoming.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>
>One nit below and a question below.
>
>In any case:
>
>Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>
>[snip]
>
>> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
>> b/lib/librte_ethdev/rte_ethdev_driver.h
>> index 0eacfd8425..3bc5c5bbbb 100644
>> --- a/lib/librte_ethdev/rte_ethdev_driver.h
>> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
>> @@ -1193,6 +1193,14 @@ __rte_internal
>>  int
>>  rte_eth_switch_domain_free(uint16_t domain_id);
>>
>> +/** Ethernet device representor type */ enum rte_eth_representor_type
>> +{
>> +	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
>> +	RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
>> +	RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
>> +	RTE_ETH_REPRESENTOR_PF,   /**< representor of host PF. */
>
>RTE_ETH_REPRESENTOR_SF and PF looks dead in the patch.
>IMHO, addition of these members here make future patches which add
>support inconsistent.

Yes, later patch in this patchset will support it.
>
>> +};
>> +
>>  /** Generic Ethernet device arguments  */  struct rte_eth_devargs {
>>  	uint16_t ports[RTE_MAX_ETHPORTS];
>> @@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
>>  	/** representor port/s identifier to enable on device */
>>  	uint16_t nb_representor_ports;
>>  	/** number of ports in representor port field */
>> +	enum rte_eth_representor_type type; /* type of representor */
>
>Is it intended and documented limitation that we can't add different type
>representors in one request? Or am I missing something and it is possible?

Correct, current devargs structure can't support mix of different types.
I'll update in next version if any.
>
>>  };
>>
>>  /**
>>

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Ajit Khaparde]

[-- Attachment #1: Type: text/plain, Size: 1990 bytes --]

On Mon, Jan 18, 2021 at 3:41 PM Xueming(Steven) Li <xuemingl@nvidia.com> wrote:
>
> >-----Original Message-----
> >From: Ajit Khaparde <ajit.khaparde@broadcom.com>
> >Sent: Tuesday, January 19, 2021 2:18 AM
> >To: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>
> >Cc: Xueming(Steven) Li <xuemingl@nvidia.com>; Ferruh Yigit
> ><ferruh.yigit@intel.com>; Andrew Rybchenko
> ><andrew.rybchenko@oktetlabs.ru>; Olivier Matz <olivier.matz@6wind.com>;
> >dpdk-dev <dev@dpdk.org>; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
> >Penso <asafp@nvidia.com>
> >Subject: Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type
> >
> >On Mon, Jan 18, 2021 at 10:15 AM Thomas Monjalon <thomas@monjalon.net>
> >wrote:
> >>
> >> 18/01/2021 19:00, Ajit Khaparde:
> >> > On Mon, Jan 18, 2021 at 9:57 AM Thomas Monjalon
> ><thomas@monjalon.net> wrote:
> >> > > 18/01/2021 18:42, Ajit Khaparde:
> >> > > > On Mon, Jan 18, 2021 at 3:17 AM Xueming Li <xuemingl@nvidia.com>
> >wrote:
> >> > > > > +enum rte_eth_representor_type {
> >> > > > > +       RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
> >> > > > > +       RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
> >> > > > > +       RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
> >> > > > Till we get used to the terminology...
> >> > > > Can we also have SF = "Sub Function" mentioned in the docs or
> >comments?
> >> > >
> >> > > Are we sure about the definition?
> >> > > I remember seeing SF = Scalable Function somewhere else (maybe from
> >Intel)
> >> > That complicates it. But if they mean the same thing, let's pick one.
> >>
> >> I think "Sub Function" and "Virtual Function" are easy to understand
> >> for everybody.
> >> I suggest picking these two for comments above.
> >+1
>
> There was an internal discussion and the conclusion is to align with kernel driver name.
> Will update comment in next version, thanks!
Ok. In that case for the series:

Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

>
> >
> >>

Re: [dpdk-dev] [PATCH v4 2/9] ethdev: support representor port list

[Andrew Rybchenko]

On 1/18/21 2:16 PM, Xueming Li wrote:
> To support extended representor syntax, this patch extends the
> representor list parsing to support for representor port range in
> devargs, examples:
>    representor=[1,2,3]         - single list
>    representor=[1,3-5,7,9-11]  - list with singles and ranges
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

See below

> ---
>  lib/librte_ethdev/ethdev_private.c | 105 ++++++++++++++---------------
>  lib/librte_ethdev/ethdev_private.h |   3 -
>  lib/librte_ethdev/rte_class_eth.c  |   4 +-
>  lib/librte_ethdev/rte_ethdev.c     |   5 +-
>  4 files changed, 54 insertions(+), 63 deletions(-)
> 
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index c1a411dba4..12bcc7e98d 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -38,77 +38,71 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
>  	return NULL;
>  }
>  
> -int
> -rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
> -	void *data)
> +static int
> +rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
> +		       const uint16_t max_list, uint16_t val)
>  {
> -	char *str_start;
> -	int state;
> -	int result;
> -
> -	if (*str != '[')
> -		/* Single element, not a list */
> -		return callback(str, data);
> -
> -	/* Sanity check, then strip the brackets */
> -	str_start = &str[strlen(str) - 1];
> -	if (*str_start != ']') {
> -		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
> -		return -EINVAL;
> -	}
> -	str++;
> -	*str_start = '\0';
> +	uint16_t i;
>  
> -	/* Process list elements */
> -	state = 0;
> -	while (1) {
> -		if (state == 0) {
> -			if (*str == '\0')
> -				break;
> -			if (*str != ',') {
> -				str_start = str;
> -				state = 1;
> -			}
> -		} else if (state == 1) {
> -			if (*str == ',' || *str == '\0') {
> -				if (str > str_start) {
> -					/* Non-empty string fragment */
> -					*str = '\0';
> -					result = callback(str_start, data);
> -					if (result < 0)
> -						return result;
> -				}
> -				state = 0;
> -			}
> -		}
> -		str++;
> +	if (*len_list >= max_list)
> +		return -1;

If current length is equal to max, but added value is already
is in the list, it should not be an error. So, these two lines
should be moved after below for loop.

> +	for (i = 0; i < *len_list; i++) {
> +		if (list[i] == val)
> +			return 0;
>  	}
> +	list[(*len_list)++] = val;
>  	return 0;
>  }
>  
> -static int
> +static char *
>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>  	const uint16_t max_list)
>  {
>  	uint16_t lo, hi, val;
>  	int result;
> +	char *pos = str;
>  
>  	result = sscanf(str, "%hu-%hu", &lo, &hi);
>  	if (result == 1) {
> -		if (*len_list >= max_list)
> -			return -ENOMEM;
> -		list[(*len_list)++] = lo;
> +		if (rte_eth_devargs_enlist(list, len_list, max_list, lo) != 0)
> +			return NULL;
>  	} else if (result == 2) {
> -		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)

Strictly speaking removal of comparision vs RTE_MAX_ETHPORTS is
a separate logical change with a separate motivation.

> -			return -EINVAL;
> +		if (lo >= hi)

I'd remove '=' here. It should not be a problem and handed
perfectly by below code. I see no point to deny 3-3 range
which is an equivalent for just 3. It could be convenient
in some cases.

> +			return NULL;
>  		for (val = lo; val <= hi; val++) {
> -			if (*len_list >= max_list)
> -				return -ENOMEM;
> -			list[(*len_list)++] = val;
> +			if (rte_eth_devargs_enlist(list, len_list, max_list,
> +						   val) != 0)
> +				return NULL;
>  		}
>  	} else
> -		return -EINVAL;
> -	return 0;
> +		return NULL;
> +	while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))

*post != '\0' is a bit better looking at subsequent
comparisons. Yes, it is just style. Up to you.

> +		pos++;

It looks too fragile. May I suggest to use %n in above scanf to
be able to skip only parsed characters.

> +	return pos;
> +}
> +
> +static char *
> +rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
> +	const uint16_t max_list)
> +{
> +	char *pos = str;
> +
> +	if (*pos == '[')
> +		pos++;
> +	while (1) {
> +		pos = rte_eth_devargs_process_range(pos, list, len_list,
> +						    max_list);
> +		if (pos == NULL)
> +			return NULL;
> +		if (*pos != ',') /* end of list */
> +			break;
> +		pos++;
> +	}
> +	if (*str == '[' && *pos != ']')
> +		return NULL;
> +	if (*pos == ']')
> +		pos++;
> +	return pos;
>  }
>  
>  /*
> @@ -121,6 +115,9 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  	struct rte_eth_devargs *eth_da = data;
>  
>  	eth_da->type = RTE_ETH_REPRESENTOR_VF;
> -	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
> +	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);

Not directly related to the patch, but I dislike
RTE_MAX_ETHPORTS above.
RTE_DIM(eth_da->representor_ports) would be more
readable.

> +	if (str == NULL)
> +		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
> +	return str == NULL ? -1 : 0;
>  }


[snip]

Re: [dpdk-dev] [PATCH v5 2/9] ethdev: support representor port list

[Andrew Rybchenko]

On 1/19/21 10:14 AM, Xueming Li wrote:
> To support extended representor syntax, this patch extends the
> representor list parsing to support for representor port range in
> devargs, examples:
>    representor=[1,2,3]         - single list
>    representor=[1,3-5,7,9-11]  - list with singles and ranges
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>

It looks like my comments to the previous version are
all applicable here. Sorry, I should review this one
from the very beginning.

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Andrew Rybchenko]

On 1/19/21 10:37 AM, Xueming(Steven) Li wrote:
> Hi Andrew,
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Tuesday, January 19, 2021 3:25 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; NBU-Contact-Thomas
>> Monjalon <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>;
>> Olivier Matz <olivier.matz@6wind.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
>> <asafp@nvidia.com>
>> Subject: Re: [PATCH v4 1/9] ethdev: introduce representor type
>>
>> On 1/18/21 2:16 PM, Xueming Li wrote:
>>> To support more representor type, this patch introduces representor
>>> type enum. The enum is subject to extend for new types upcoming.
>>>
>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>
>> One nit below and a question below.
>>
>> In any case:
>>
>> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>
>> [snip]
>>
>>> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
>>> b/lib/librte_ethdev/rte_ethdev_driver.h
>>> index 0eacfd8425..3bc5c5bbbb 100644
>>> --- a/lib/librte_ethdev/rte_ethdev_driver.h
>>> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
>>> @@ -1193,6 +1193,14 @@ __rte_internal
>>>  int
>>>  rte_eth_switch_domain_free(uint16_t domain_id);
>>>
>>> +/** Ethernet device representor type */ enum rte_eth_representor_type
>>> +{
>>> +	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
>>> +	RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
>>> +	RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
>>> +	RTE_ETH_REPRESENTOR_PF,   /**< representor of host PF. */
>>
>> RTE_ETH_REPRESENTOR_SF and PF looks dead in the patch.
>> IMHO, addition of these members here make future patches which add
>> support inconsistent.
> 
> Yes, later patch in this patchset will support it.


I know. The question is why it is not added in the
later patches when these types are actually supported.

>>
>>> +};
>>> +
>>>  /** Generic Ethernet device arguments  */  struct rte_eth_devargs {
>>>  	uint16_t ports[RTE_MAX_ETHPORTS];
>>> @@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
>>>  	/** representor port/s identifier to enable on device */
>>>  	uint16_t nb_representor_ports;
>>>  	/** number of ports in representor port field */
>>> +	enum rte_eth_representor_type type; /* type of representor */
>>
>> Is it intended and documented limitation that we can't add different type
>> representors in one request? Or am I missing something and it is possible?
> 
> Correct, current devargs structure can't support mix of different types.
> I'll update in next version if any.
>>
>>>  };
>>>
>>>  /**
>>>
> 

Re: [dpdk-dev] [PATCH v5 3/9] ethdev: support new VF representor syntax

[Andrew Rybchenko]

On 1/19/21 10:14 AM, Xueming Li wrote:
> Current VF representor syntax:
>  representor=2          - single representor
>  representor=[0-3]      - single range
> 
> To prepare for more representor types, this patch adds compatible VF
> representor devargs syntax:
> 
> vf#:
>  representor=vf2          - single representor
>  representor=vf[1,3,5]    - single list
>  representor=vf[0-3]      - single range
>  representor=vf[0,1,4-7]  - list with singles and range
> 
> For backwards compatibility, representor "#" is interpreted as "vf#".
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>

Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

Re: [dpdk-dev] [PATCH v5 4/9] ethdev: support sub function representor

[Andrew Rybchenko]

On 1/19/21 10:14 AM, Xueming Li wrote:
> SubFunction is a portion of the PCI device, created on demand, a SF
> netdev has its own dedicated queues(txq, rxq). A SF netdev supports
> eswitch representation offload similar to existing PF and VF
> representors.
> 
> To support SF representor, this patch introduces new devargs syntax,
> examples:
>  representor=sf0               - single SubFunction representor
>  representor=sf[1,3,5]         - single list
>  representor=sf[0-3],          - single range
>  representor=sf[0,2-6,8,10-12] - list with singles and ranges
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>

Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 3:49 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; NBU-Contact-Thomas
>Monjalon <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>;
>Olivier Matz <olivier.matz@6wind.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>
>Subject: Re: [PATCH v4 1/9] ethdev: introduce representor type
>
>On 1/19/21 10:37 AM, Xueming(Steven) Li wrote:
>> Hi Andrew,
>>
>>> -----Original Message-----
>>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>> Sent: Tuesday, January 19, 2021 3:25 PM
>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; NBU-Contact-Thomas
>>> Monjalon <thomas@monjalon.net>; Ferruh Yigit
>>> <ferruh.yigit@intel.com>; Olivier Matz <olivier.matz@6wind.com>
>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>> Penso <asafp@nvidia.com>
>>> Subject: Re: [PATCH v4 1/9] ethdev: introduce representor type
>>>
>>> On 1/18/21 2:16 PM, Xueming Li wrote:
>>>> To support more representor type, this patch introduces representor
>>>> type enum. The enum is subject to extend for new types upcoming.
>>>>
>>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>>
>>> One nit below and a question below.
>>>
>>> In any case:
>>>
>>> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>>
>>> [snip]
>>>
>>>> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
>>>> b/lib/librte_ethdev/rte_ethdev_driver.h
>>>> index 0eacfd8425..3bc5c5bbbb 100644
>>>> --- a/lib/librte_ethdev/rte_ethdev_driver.h
>>>> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
>>>> @@ -1193,6 +1193,14 @@ __rte_internal  int
>>>> rte_eth_switch_domain_free(uint16_t domain_id);
>>>>
>>>> +/** Ethernet device representor type */ enum
>>>> +rte_eth_representor_type {
>>>> +	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
>>>> +	RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
>>>> +	RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
>>>> +	RTE_ETH_REPRESENTOR_PF,   /**< representor of host PF. */
>>>
>>> RTE_ETH_REPRESENTOR_SF and PF looks dead in the patch.
>>> IMHO, addition of these members here make future patches which add
>>> support inconsistent.
>>
>> Yes, later patch in this patchset will support it.
>
>
>I know. The question is why it is not added in the later patches when these
>types are actually supported.

Good suggestion, will update
>
>>>
>>>> +};
>>>> +
>>>>  /** Generic Ethernet device arguments  */  struct rte_eth_devargs {
>>>>  	uint16_t ports[RTE_MAX_ETHPORTS];
>>>> @@ -1203,6 +1211,7 @@ struct rte_eth_devargs {
>>>>  	/** representor port/s identifier to enable on device */
>>>>  	uint16_t nb_representor_ports;
>>>>  	/** number of ports in representor port field */
>>>> +	enum rte_eth_representor_type type; /* type of representor */
>>>
>>> Is it intended and documented limitation that we can't add different
>>> type representors in one request? Or am I missing something and it is
>possible?
>>
>> Correct, current devargs structure can't support mix of different types.
>> I'll update in next version if any.
>>>
>>>>  };
>>>>
>>>>  /**
>>>>
>>

Re: [dpdk-dev] [PATCH v5 5/9] ethdev: support PF index in representor

[Andrew Rybchenko]

On 1/19/21 10:14 AM, Xueming Li wrote:
> With Kernel bonding, multiple underlying PFs are bonded, VFs come
> from different PF, need to identify representor of VFs unambiguously by
> adding PF index.
> 
> This patch introduces optional 'pf' section to representor devargs
> syntax, examples:
>  representor=pf0vf0             - single VF representor
>  representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs


Don't we need
 representor=pf3
i.e. without VF or sub-function?

> 
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  doc/guides/prog_guide/poll_mode_drv.rst |  2 ++
>  lib/librte_ethdev/ethdev_private.c      | 13 +++++++++++--
>  2 files changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
> index 86e5867f1b..b2147aad30 100644
> --- a/doc/guides/prog_guide/poll_mode_drv.rst
> +++ b/doc/guides/prog_guide/poll_mode_drv.rst
> @@ -382,6 +382,8 @@ parameters to those ports.
>     -a DBDF,representor=sf[1,3,5]
>     -a DBDF,representor=sf[0-1023]
>     -a DBDF,representor=sf[0,2-4,7,9-11]
> +   -a DBDF,representor=pf1vf0
> +   -a DBDF,representor=pf[0-1]sf[0-127]
>  
>  Note: PMDs are not required to support the standard device arguments and users
>  should consult the relevant PMD documentation to see support devargs.
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index d513f035d0..b9fdbd0f72 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -120,8 +120,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>   *
>   * Representor format:
>   *   #: range or single number of VF representor - legacy
> - *   vf#: VF port representor/s
> - *   sf#: SF port representor/s
> + *   [pf#]vf#: VF port representor/s
> + *   [pf#]sf#: SF port representor/s
>   *
>   * Examples of #:
>   *  2               - single
> @@ -133,6 +133,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  {
>  	struct rte_eth_devargs *eth_da = data;
>  
> +	if (str[0] == 'p' && str[1] == 'f') {
> +		eth_da->type = RTE_ETH_REPRESENTOR_PF;
> +		str += 2;
> +		str = rte_eth_devargs_process_list(str, eth_da->ports,
> +				&eth_da->nb_ports, RTE_MAX_ETHPORTS);

May be RTE_MAX_ETHPORTS -> RTE_DIM(eth_da->ports) ?

> +		if (str == NULL)
> +			goto err;

Below we should not allow legacy VF syntax without
"vf" prefix.

> +	}
>  	if (str[0] == 'v' && str[1] == 'f') {
>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>  		str += 2;
> @@ -144,6 +152,7 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  	}
>  	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
> +err:
>  	if (str == NULL)
>  		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
>  	return str == NULL ? -1 : 0;
> 

Re: [dpdk-dev] [PATCH v5 6/9] ethdev: support multi-host in representor

[Andrew Rybchenko]

On 1/19/21 10:14 AM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to the multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. To provide the unambiguous identification
> of the PCIe function the controller index is added. The full representor
> identifier consists of three indices - controller index, PF index, and
> VF or SF index (if any).
> 
> This patch introduces controller index to ethdev representor syntax,
> examples:
> 
> [[c#]pf#]vf#: VF port representor/s, example: pf0vf1
> [[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]
> 
> c# is controller(host) ID/range in case of multi-host, optional.
> 
> For user application (e.g. OVS), PMD is responsible to interpret and
> locate representor device based on controller ID, PF ID and VF/SF ID in
> representor syntax.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  config/rte_config.h                   |  1 +
>  lib/librte_ethdev/ethdev_private.c    | 12 ++++++++++--
>  lib/librte_ethdev/rte_ethdev_driver.h |  4 ++++
>  3 files changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/config/rte_config.h b/config/rte_config.h
> index a0b5160ff2..23d02d51ef 100644
> --- a/config/rte_config.h
> +++ b/config/rte_config.h
> @@ -58,6 +58,7 @@
>  #define RTE_MAX_QUEUES_PER_PORT 1024
>  #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
>  #define RTE_ETHDEV_RXTX_CALLBACKS 1
> +#define RTE_MAX_MULTI_HOST_CTRLS 4
>  
>  /* cryptodev defines */
>  #define RTE_CRYPTO_MAX_DEVS 64
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index b9fdbd0f72..9a0945c22e 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -120,8 +120,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>   *
>   * Representor format:
>   *   #: range or single number of VF representor - legacy
> - *   [pf#]vf#: VF port representor/s
> - *   [pf#]sf#: SF port representor/s
> + *   [[c#]pf#]vf#: VF port representor/s
> + *   [[c#]pf#]sf#: SF port representor/s
>   *
>   * Examples of #:
>   *  2               - single
> @@ -133,6 +133,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  {
>  	struct rte_eth_devargs *eth_da = data;
>  
> +	if (str[0] == 'c') {
> +		str += 1;
> +		str = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
> +				&eth_da->nb_mh_controllers,
> +				RTE_DIM(eth_da->mh_controllers));
> +		if (str == NULL)
> +			goto err;

The parser must enforce pf to follow. I.e. should not allow
c1vf3. At least above syntax description in comments says so.

> +	}
>  	if (str[0] == 'p' && str[1] == 'f') {
>  		eth_da->type = RTE_ETH_REPRESENTOR_PF;
>  		str += 2;
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index cd33184f17..b01f118965 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -1203,6 +1203,10 @@ enum rte_eth_representor_type {
>  
>  /** Generic Ethernet device arguments  */
>  struct rte_eth_devargs {
> +	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
> +	/** controller/s number in case of multi-host */
> +	uint16_t nb_mh_controllers;
> +	/** number of controllers in multi-host controllers field */
>  	uint16_t ports[RTE_MAX_ETHPORTS];
>  	/** port/s number to enable on a multi-port single function */
>  	uint16_t nb_ports;
> 

Re: [dpdk-dev] [PATCH v5 8/9] ethdev: add capability of sub-function representor

[Andrew Rybchenko]

On 1/19/21 10:15 AM, Xueming Li wrote:
> Old DPDK version or some drivers didn't support SubFunction representor.
> For application to adapt different DPDK version automatically, or to be
> used for different NICs, this patch introduces new eth device capability
> of supporting SubFunction representor device.

Sorry, it does not sound sufficient motivation to introduce the
capability. I simply need real life example why application
need to know it.

> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  lib/librte_ethdev/rte_ethdev.h | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 2cbce958cf..da65adf1ab 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -1433,6 +1433,8 @@ struct rte_eth_conf {
>  #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
>  /** Device supports Tx queue setup after device started. */
>  #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
> +/** Device supports SubFunction representor. */
> +#define RTE_ETH_DEV_CAPA_REPRESENTOR_SF         0x00000004

Will we have a PMD which supports it in the release cycle?

>  /**@}*/
>  
>  /*

Re: [dpdk-dev] [PATCH v5 7/9] devarg: change representor ID to bitmap

[Xueming(Steven) Li]

Hi Haiyue,

>-----Original Message-----
>From: Wang, Haiyue <haiyue.wang@intel.com>
>Sent: Tuesday, January 19, 2021 3:37 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>;
>Somnath Kotur <somnath.kotur@broadcom.com>; Daley, John
><johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>; Xing, Beilei
><beilei.xing@intel.com>; Guo, Jia <jia.guo@intel.com>; Matan Azrad
><matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; NBU-Contact-
>Thomas Monjalon <thomas@monjalon.net>; Yigit, Ferruh
><ferruh.yigit@intel.com>; Andrew Rybchenko
><andrew.rybchenko@oktetlabs.ru>; Ray Kinsella <mdr@ashroe.eu>; Neil
>Horman <nhorman@tuxdriver.com>
>Subject: RE: [PATCH v5 7/9] devarg: change representor ID to bitmap
>
>> -----Original Message-----
>> From: Xueming Li <xuemingl@nvidia.com>
>> Sent: Tuesday, January 19, 2021 15:15
>> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>;
>> xuemingl@nvidia.com; Asaf Penso <asafp@nvidia.com>; Ajit Khaparde
>> <ajit.khaparde@broadcom.com>; Somnath Kotur
>> <somnath.kotur@broadcom.com>; Daley, John <johndale@cisco.com>;
>Hyong
>> Youb Kim <hyonkim@cisco.com>; Xing, Beilei <beilei.xing@intel.com>;
>> Guo, Jia <jia.guo@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>;
>> Matan Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>;
>> Thomas Monjalon <thomas@monjalon.net>; Yigit, Ferruh
>> <ferruh.yigit@intel.com>; Andrew Rybchenko
>> <andrew.rybchenko@oktetlabs.ru>; Ray Kinsella <mdr@ashroe.eu>; Neil
>> Horman <nhorman@tuxdriver.com>
>> Subject: [PATCH v5 7/9] devarg: change representor ID to bitmap
>>
>> The NIC can have multiple PCIe links and can be attached to multiple
>> hosts, for example the same single NIC can be shared for multiple
>> server units in the rack. On each PCIe link NIC can provide multiple
>> PFs and VFs/SFs based on these ones. The full representor identifier
>> consists of three indices - controller index, PF index, and VF or SF index (if
>any).
>>
>> SR-IOV and SubFunction are created on top of PF. PF index is
>> introduced because there might be multiple PFs in the bonding
>> configuration and only bonding device is probed.
>>
>> In eth representor comparator callback, ethdev was compared with devarg.
>> Since ethdev representor port didn't contain controller index and PF
>> index information, callback returned representor from other PF or
>> controller.
>>
>> This patch changes representor ID to bitmap so that the ethdev
>> representor comparer callback returns correct ethdev by comparing full
>> representor information including: controller index, PF index,
>> representor type, SF or VF index.
>>
>> Representor ID bitmap definition:
>>  xxxx xxxx xxxx xxxx
>>  |||| |LLL LLLL LLLL vf/sf id
>>  |||| L 1:sf, 0:vf
>>  ||LL pf id
>>  LL controller(host) id
>>
>> This approach keeps binary compatibility with all drivers, VF
>> representor id matches with simple id for non-bonding and
>> non-multi-host configurations.
>>
>> In the future, the representor ID field and each section should extend
>> to bigger width to support more devices.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>> ---
>>  drivers/net/bnxt/bnxt_reps.c             |  3 +-
>>  drivers/net/enic/enic_vf_representor.c   |  3 +-
>>  drivers/net/i40e/i40e_vf_representor.c   |  3 +-
>>  drivers/net/ixgbe/ixgbe_vf_representor.c |  3 +-
>>  drivers/net/mlx5/linux/mlx5_os.c         |  4 +-
>>  lib/librte_ethdev/rte_class_eth.c        | 38 +++++++++++++----
>>  lib/librte_ethdev/rte_ethdev.c           | 26 ++++++++++++
>>  lib/librte_ethdev/rte_ethdev_driver.h    | 53 ++++++++++++++++++++++++
>>  lib/librte_ethdev/version.map            |  2 +
>>  9 files changed, 122 insertions(+), 13 deletions(-)
>>
>
>
>> +uint16_t
>> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
>> +			      enum rte_eth_representor_type type,
>> +			      uint16_t representor_port)
>> +{
>> +	return (((controller & 3) << 14) |
>> +		((pf & 3) << 12) |
>> +		(!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
>> +		(representor_port & 0x7ff));
>> +}
>> +
>> +uint16_t
>> +rte_eth_representor_id_parse(const uint16_t representor_id,
>> +			     uint16_t *controller, uint16_t *pf,
>> +			     enum rte_eth_representor_type *type) {
>> +	if (controller)
>> +		*controller = (representor_id >> 14) & 3;
>> +	if (pf)
>> +		*pf = (representor_id >> 12) & 3;
>> +	if (type)
>> +		*type = ((representor_id >> 11) & 1) ?
>> +			RTE_ETH_REPRESENTOR_SF :
>RTE_ETH_REPRESENTOR_VF;
>> +	return representor_id & 0x7ff;
>> +}
>> +
>
>Rename 'parse' to 'decode' ? Since not parse from string. ;-) The these two
>functions are pair style.
>
>rte_eth_representor_id_encode / rte_eth_representor_id_decode ?

Nice catch, thanks!
>
>> 2.25.1

Re: [dpdk-dev] [PATCH v5 2/9] ethdev: support representor port list

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 3:48 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>
>Subject: Re: [PATCH v5 2/9] ethdev: support representor port list
>
>On 1/19/21 10:14 AM, Xueming Li wrote:
>> To support extended representor syntax, this patch extends the
>> representor list parsing to support for representor port range in
>> devargs, examples:
>>    representor=[1,2,3]         - single list
>>    representor=[1,3-5,7,9-11]  - list with singles and ranges
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>
>It looks like my comments to the previous version are all applicable here.
>Sorry, I should review this one from the very beginning.
NP, noted and WIP.

Re: [dpdk-dev] [PATCH v5 7/9] devarg: change representor ID to bitmap

[Andrew Rybchenko]

On 1/19/21 10:14 AM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> SR-IOV and SubFunction are created on top of PF. PF index is introduced
> because there might be multiple PFs in the bonding configuration and
> only bonding device is probed.
> 
> In eth representor comparator callback, ethdev was compared with devarg.
> Since ethdev representor port didn't contain controller index and PF
> index information, callback returned representor from other PF or
> controller.
> 
> This patch changes representor ID to bitmap so that the ethdev
> representor comparer callback returns correct ethdev by comparing full
> representor information including: controller index, PF index,
> representor type, SF or VF index.
> 
> Representor ID bitmap definition:
>  xxxx xxxx xxxx xxxx
>  |||| |LLL LLLL LLLL vf/sf id
>  |||| L 1:sf, 0:vf
>  ||LL pf id
>  LL controller(host) id

What about PF representor case? I.e. representor for entire PF.

Also it implies that controller ID 0 is the caller. I.e.
special meaning. So, space for just 3 specific contoller left

Similar for PF. In fact it is worse. E.g. PMD is bound to the
second PF (PF number 1). If so, vf0 means the first VF of the
PF itself, i.e. PF 1 VF 0. But, pf0vf0 should mean PF 1 VF 1.

> 
> This approach keeps binary compatibility with all drivers, VF
> representor id matches with simple id for non-bonding and non-multi-host
> configurations.
> 
> In the future, the representor ID field and each section should extend
> to bigger width to support more devices.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>


> diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
> index efe6149df5..994db96960 100644
> --- a/lib/librte_ethdev/rte_class_eth.c
> +++ b/lib/librte_ethdev/rte_class_eth.c
> @@ -66,8 +66,8 @@ eth_representor_cmp(const char *key __rte_unused,
>  	int ret;
>  	char *values;
>  	const struct rte_eth_dev_data *data = opaque;
> -	struct rte_eth_devargs representors;
> -	uint16_t index;
> +	struct rte_eth_devargs eth_da;
> +	uint16_t index, c, p, f;
>  
>  	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
>  		return -1; /* not a representor port */
> @@ -76,17 +76,39 @@ eth_representor_cmp(const char *key __rte_unused,
>  	values = strdup(value);
>  	if (values == NULL)
>  		return -1;
> -	memset(&representors, 0, sizeof(representors));
> -	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
> +	memset(&eth_da, 0, sizeof(eth_da));
> +	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
>  	free(values);
>  	if (ret != 0)
>  		return -1; /* invalid devargs value */
>  
> +	/* Set default values. */
> +	if (eth_da.nb_mh_controllers == 0) {
> +		eth_da.nb_mh_controllers = 1;
> +		eth_da.mh_controllers[0] = 0;
> +	}
> +	if (eth_da.nb_ports == 0) {
> +		eth_da.nb_ports = 1;
> +		eth_da.ports[0] = 0;
> +	}
> +	if (eth_da.nb_representor_ports == 0) {
> +		eth_da.nb_representor_ports = 1;
> +		eth_da.representor_ports[0] = 0;
> +	}
>  	/* Return 0 if representor id is matching one of the values. */
> -	for (index = 0; index < representors.nb_representor_ports; index++)
> -		if (data->representor_id ==
> -				representors.representor_ports[index])
> -			return 0;
> +	for (c = 0; c < eth_da.nb_mh_controllers; ++c) {
> +		for (p = 0; p < eth_da.nb_ports; ++p) {
> +			for (f = 0; f < eth_da.nb_representor_ports; ++f) {
> +				index = rte_eth_representor_id_encode(
> +					eth_da.mh_controllers[c],
> +					eth_da.ports[p],
> +					eth_da.type,
> +					eth_da.representor_ports[f]);
> +				if (data->representor_id == index)
> +					return 0;
> +			}
> +		}
> +	}
>  	return -1; /* no match */
>  }
>  
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 91b3263338..2cac0ccfbd 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -5575,6 +5575,32 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>  	return result;
>  }
>  
> +uint16_t
> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
> +			      enum rte_eth_representor_type type,
> +			      uint16_t representor_port)
> +{
> +	return (((controller & 3) << 14) |
> +		((pf & 3) << 12) |
> +		(!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
> +		(representor_port & 0x7ff));
> +}
> +
> +uint16_t
> +rte_eth_representor_id_parse(const uint16_t representor_id,
> +			     uint16_t *controller, uint16_t *pf,
> +			     enum rte_eth_representor_type *type)
> +{
> +	if (controller)

Compare vs NULL in accordance with DPDK coding style guide

> +		*controller = (representor_id >> 14) & 3;
> +	if (pf)

Compare vs NULL in accordance with DPDK coding style guide

> +		*pf = (representor_id >> 12) & 3;
> +	if (type)

Compare vs NULL in accordance with DPDK coding style guide

> +		*type = ((representor_id >> 11) & 1) ?
> +			RTE_ETH_REPRESENTOR_SF : RTE_ETH_REPRESENTOR_VF;
> +	return representor_id & 0x7ff;
> +}
> +
>  static int
>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>  		const char *params __rte_unused,
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index b01f118965..57253c8f90 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -1218,6 +1218,59 @@ struct rte_eth_devargs {
>  	enum rte_eth_representor_type type; /* type of representor */
>  };
>  
> +#define RTE_NO_REPRESENTOR_ID UINT16_MAX /**< No representor ID. */
> +
> +/**
> + * PMD helper function to encode representor ID
> + *
> + * The compact format is used for device iterator that comparing
> + * ethdev representor ID with target devargs.
> + *
> + * xxxx xxxx xxxx xxxx
> + * |||| |LLL LLLL LLLL vf/sf id
> + * |||| L 1:sf, 0:vf
> + * ||LL pf id
> + * LL controller(host) id
> + *
> + * @param controller
> + *  Controller ID.
> + * @param pf
> + *  PF port ID.
> + * @param type
> + *  Representor type.
> + * @param representor_port
> + *  Representor port ID.
> + *
> + * @return
> + *   Encoded representor ID.
> + */
> +__rte_internal
> +uint16_t
> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
> +			      enum rte_eth_representor_type type,
> +			      uint16_t representor_port);
> +
> +/**
> + * PMD helper function to parse representor ID
> + *
> + * @param representor_id
> + *  Representor ID.
> + * @param controller
> + *  Parsed controller ID.
> + * @param pf
> + *  Parsed PF port ID.
> + * @param type
> + *  Parsed representor type.
> + *
> + * @return
> + *   Parsed representor port ID.
> + */
> +__rte_internal
> +uint16_t
> +rte_eth_representor_id_parse(const uint16_t representor_id,
> +			     uint16_t *controller, uint16_t *pf,
> +			     enum rte_eth_representor_type *type);
> +
>  /**
>   * PMD helper function to parse ethdev arguments
>   *
> diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
> index d3f5410806..44edaed507 100644
> --- a/lib/librte_ethdev/version.map
> +++ b/lib/librte_ethdev/version.map
> @@ -257,6 +257,8 @@ INTERNAL {
>  	rte_eth_dev_release_port;
>  	rte_eth_dev_internal_reset;
>  	rte_eth_devargs_parse;
> +	rte_eth_representor_id_encode;
> +	rte_eth_representor_id_parse;

The place looks wrong. It must be sorted or added at bottom
with the comment with version number. I'd prefer to have
it simply alphanumberically sorted.

>  	rte_eth_dma_zone_free;
>  	rte_eth_dma_zone_reserve;
>  	rte_eth_hairpin_queue_peer_bind;
> 

Re: [dpdk-dev] [PATCH v5 7/9] devarg: change representor ID to bitmap

[Thomas Monjalon]

19/01/2021 09:20, Andrew Rybchenko:
> On 1/19/21 10:14 AM, Xueming Li wrote:
> > --- a/lib/librte_ethdev/version.map
> > +++ b/lib/librte_ethdev/version.map
> > @@ -257,6 +257,8 @@ INTERNAL {
> >  	rte_eth_dev_release_port;
> >  	rte_eth_dev_internal_reset;
> >  	rte_eth_devargs_parse;
> > +	rte_eth_representor_id_encode;
> > +	rte_eth_representor_id_parse;
> 
> The place looks wrong. It must be sorted or added at bottom
> with the comment with version number. I'd prefer to have
> it simply alphanumberically sorted.

We add a version number for experimental symbols to help
moving old ones to stable.
In this case, they are internal symbols, so we don't need version,
but you're right it must be alphabetically sorted.

Re: [dpdk-dev] [PATCH v4 1/9] ethdev: introduce representor type

[Thomas Monjalon]

19/01/2021 08:56, Xueming(Steven) Li:
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> >On 1/19/21 10:37 AM, Xueming(Steven) Li wrote:
> >> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> >>> On 1/18/21 2:16 PM, Xueming Li wrote:
> >>>> +/** Ethernet device representor type */ enum
> >>>> +rte_eth_representor_type {
> >>>> +	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
> >>>> +	RTE_ETH_REPRESENTOR_VF,   /**< representor of VF. */
> >>>> +	RTE_ETH_REPRESENTOR_SF,   /**< representor of SF. */
> >>>> +	RTE_ETH_REPRESENTOR_PF,   /**< representor of host PF. */
> >>>
> >>> RTE_ETH_REPRESENTOR_SF and PF looks dead in the patch.
> >>> IMHO, addition of these members here make future patches which add
> >>> support inconsistent.
> >>
> >> Yes, later patch in this patchset will support it.
> >
> >I know. The question is why it is not added in the later patches when these
> >types are actually supported.
> 
> Good suggestion, will update

+1 (I was sure it was already the case)

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Andrew Rybchenko]

On 1/19/21 10:13 AM, Xueming Li wrote:
> dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
> offload similar to existing PF and VF representors. A SF shares PCI
> level resources with other SFs and/or with its parent PCI function.
> 
>>From SmartNIC perspective, when PCI device is shared for multi-host,
> representors for host controller and host PF is required.
> 
> This patch set introduces new representor types in addtion to existing
> VF representor. Syntax:
> 
> [[c#]pf#]vf#: VF port representor/s from controller/pf
> [[c#]pf#]sf#: SF port representor/s from controller/pf
> #: VF representor - for backwards compatibility
> 
> "#" is number instance, list or range, valid examples:
>   1, [1,3,5], [0-3], [0,2-4,6]
> 
> For backward compatibility, this patch also introduces new netdev
> capability to indicate the capability of supportting SF representor.

The patch series looks really nice. Thanks.

As before, my biggest concern is making representor ID
a bitmap. See my comments to a specific patch.
Plus absence of defined semantics of caller function.
Basically it is looks like it is assumed that
controller #0 and PF #0 is the caller. However, it
could be wrong.

The next biggest concert is the absence of capability
reporting API. How many controller are available?
How many PFs on each controller are available?
How many VFs on each controller/PF are available?
How many SFs?

From the first sight it sounds not that important right
now and an extra feature which could be added in the
follow up patches, but IMHO addition of the API
would allow to avoid making representor ID a bitmap.
Basically capabilities API can provide an array of
available functions with representor ID assigned to
each entry. Also it could make the entire patch
series optional since it would allow to interpret
numbers in representor=[....] as representor IDs
which are mapped to controller/PF/VF/SF by the
capabilities reporting API.

I realize that sometimes it could be more convenient to
use syntax suggested here. Mainly for human.

Re: [dpdk-dev] [PATCH v4 2/9] ethdev: support representor port list

[Xueming(Steven) Li]

Hi Andrew,

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 3:46 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; NBU-Contact-Thomas
>Monjalon <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>;
>Olivier Matz <olivier.matz@6wind.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>
>Subject: Re: [PATCH v4 2/9] ethdev: support representor port list
>
>On 1/18/21 2:16 PM, Xueming Li wrote:
>> To support extended representor syntax, this patch extends the
>> representor list parsing to support for representor port range in
>> devargs, examples:
>>    representor=[1,2,3]         - single list
>>    representor=[1,3-5,7,9-11]  - list with singles and ranges
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>
>See below
>
>> ---
>>  lib/librte_ethdev/ethdev_private.c | 105 ++++++++++++++---------------
>>  lib/librte_ethdev/ethdev_private.h |   3 -
>>  lib/librte_ethdev/rte_class_eth.c  |   4 +-
>>  lib/librte_ethdev/rte_ethdev.c     |   5 +-
>>  4 files changed, 54 insertions(+), 63 deletions(-)
>>
>> diff --git a/lib/librte_ethdev/ethdev_private.c
>> b/lib/librte_ethdev/ethdev_private.c
>> index c1a411dba4..12bcc7e98d 100644
>> --- a/lib/librte_ethdev/ethdev_private.c
>> +++ b/lib/librte_ethdev/ethdev_private.c
>> @@ -38,77 +38,71 @@ eth_find_device(const struct rte_eth_dev *start,
>rte_eth_cmp_t cmp,
>>  	return NULL;
>>  }
>>
>> -int
>> -rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
>> -	void *data)
>> +static int
>> +rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
>> +		       const uint16_t max_list, uint16_t val)
>>  {
>> -	char *str_start;
>> -	int state;
>> -	int result;
>> -
>> -	if (*str != '[')
>> -		/* Single element, not a list */
>> -		return callback(str, data);
>> -
>> -	/* Sanity check, then strip the brackets */
>> -	str_start = &str[strlen(str) - 1];
>> -	if (*str_start != ']') {
>> -		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
>> -		return -EINVAL;
>> -	}
>> -	str++;
>> -	*str_start = '\0';
>> +	uint16_t i;
>>
>> -	/* Process list elements */
>> -	state = 0;
>> -	while (1) {
>> -		if (state == 0) {
>> -			if (*str == '\0')
>> -				break;
>> -			if (*str != ',') {
>> -				str_start = str;
>> -				state = 1;
>> -			}
>> -		} else if (state == 1) {
>> -			if (*str == ',' || *str == '\0') {
>> -				if (str > str_start) {
>> -					/* Non-empty string fragment */
>> -					*str = '\0';
>> -					result = callback(str_start, data);
>> -					if (result < 0)
>> -						return result;
>> -				}
>> -				state = 0;
>> -			}
>> -		}
>> -		str++;
>> +	if (*len_list >= max_list)
>> +		return -1;
>
>If current length is equal to max, but added value is already is in the list, it
>should not be an error. So, these two lines should be moved after below for
>loop.
>
>> +	for (i = 0; i < *len_list; i++) {
>> +		if (list[i] == val)
>> +			return 0;
>>  	}
>> +	list[(*len_list)++] = val;
>>  	return 0;
>>  }
>>
>> -static int
>> +static char *
>>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>>  	const uint16_t max_list)
>>  {
>>  	uint16_t lo, hi, val;
>>  	int result;
>> +	char *pos = str;
>>
>>  	result = sscanf(str, "%hu-%hu", &lo, &hi);
>>  	if (result == 1) {
>> -		if (*len_list >= max_list)
>> -			return -ENOMEM;
>> -		list[(*len_list)++] = lo;
>> +		if (rte_eth_devargs_enlist(list, len_list, max_list, lo) != 0)
>> +			return NULL;
>>  	} else if (result == 2) {
>> -		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi >
>RTE_MAX_ETHPORTS)
>
>Strictly speaking removal of comparision vs RTE_MAX_ETHPORTS is a separate
>logical change with a separate motivation.

To make this function comment for controller/pf/vf/sf parsing, the max value is
passed in as parameter and checked in rte_eth_devargs_enlist().
Caller code in rte_eth_devargs_process_list() decide the max value.
>
>> -			return -EINVAL;
>> +		if (lo >= hi)
>
>I'd remove '=' here. It should not be a problem and handed perfectly by below
>code. I see no point to deny 3-3 range which is an equivalent for just 3. It
>could be convenient in some cases.
>
>> +			return NULL;
>>  		for (val = lo; val <= hi; val++) {
>> -			if (*len_list >= max_list)
>> -				return -ENOMEM;
>> -			list[(*len_list)++] = val;
>> +			if (rte_eth_devargs_enlist(list, len_list, max_list,
>> +						   val) != 0)
>> +				return NULL;
>>  		}
>>  	} else
>> -		return -EINVAL;
>> -	return 0;
>> +		return NULL;
>> +	while (*pos != 0 && ((*pos >= '0' && *pos <= '9') || *pos == '-'))
>
>*post != '\0' is a bit better looking at subsequent comparisons. Yes, it is just
>style. Up to you.
>
>> +		pos++;
>
>It looks too fragile. May I suggest to use %n in above scanf to be able to skip
>only parsed characters.
>
>> +	return pos;
>> +}
>> +
>> +static char *
>> +rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>> +	const uint16_t max_list)
>> +{
>> +	char *pos = str;
>> +
>> +	if (*pos == '[')
>> +		pos++;
>> +	while (1) {
>> +		pos = rte_eth_devargs_process_range(pos, list, len_list,
>> +						    max_list);
>> +		if (pos == NULL)
>> +			return NULL;
>> +		if (*pos != ',') /* end of list */
>> +			break;
>> +		pos++;
>> +	}
>> +	if (*str == '[' && *pos != ']')
>> +		return NULL;
>> +	if (*pos == ']')
>> +		pos++;
>> +	return pos;
>>  }
>>
>>  /*
>> @@ -121,6 +115,9 @@ rte_eth_devargs_parse_representor_ports(char *str,
>void *data)
>>  	struct rte_eth_devargs *eth_da = data;
>>
>>  	eth_da->type = RTE_ETH_REPRESENTOR_VF;
>> -	return rte_eth_devargs_process_range(str, eth_da-
>>representor_ports,
>> +	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
>
>Not directly related to the patch, but I dislike RTE_MAX_ETHPORTS above.
>RTE_DIM(eth_da->representor_ports) would be more readable.

The array dim could be different than RTE_MAX_ETHPORTS, although they are
same today 😊
>
>> +	if (str == NULL)
>> +		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
>> +	return str == NULL ? -1 : 0;
>>  }
>
>
>[snip]

Other comments looks great, will update, thanks!

Re: [dpdk-dev] [PATCH v4 2/9] ethdev: support representor port list

[Andrew Rybchenko]

On 1/19/21 11:59 AM, Xueming(Steven) Li wrote:
> Hi Andrew,
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Tuesday, January 19, 2021 3:46 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; NBU-Contact-Thomas
>> Monjalon <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>;
>> Olivier Matz <olivier.matz@6wind.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
>> <asafp@nvidia.com>
>> Subject: Re: [PATCH v4 2/9] ethdev: support representor port list
>>
>> On 1/18/21 2:16 PM, Xueming Li wrote:
>>> To support extended representor syntax, this patch extends the
>>> representor list parsing to support for representor port range in
>>> devargs, examples:
>>>    representor=[1,2,3]         - single list
>>>    representor=[1,3-5,7,9-11]  - list with singles and ranges
>>>
>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>
>> See below
>>
>>> ---
>>>  lib/librte_ethdev/ethdev_private.c | 105 ++++++++++++++---------------
>>>  lib/librte_ethdev/ethdev_private.h |   3 -
>>>  lib/librte_ethdev/rte_class_eth.c  |   4 +-
>>>  lib/librte_ethdev/rte_ethdev.c     |   5 +-
>>>  4 files changed, 54 insertions(+), 63 deletions(-)
>>>
>>> diff --git a/lib/librte_ethdev/ethdev_private.c
>>> b/lib/librte_ethdev/ethdev_private.c
>>> index c1a411dba4..12bcc7e98d 100644
>>> --- a/lib/librte_ethdev/ethdev_private.c
>>> +++ b/lib/librte_ethdev/ethdev_private.c
>>> @@ -38,77 +38,71 @@ eth_find_device(const struct rte_eth_dev *start,
>> rte_eth_cmp_t cmp,
>>>  	return NULL;
>>>  }
>>>
>>> -int
>>> -rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
>>> -	void *data)
>>> +static int
>>> +rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
>>> +		       const uint16_t max_list, uint16_t val)
>>>  {
>>> -	char *str_start;
>>> -	int state;
>>> -	int result;
>>> -
>>> -	if (*str != '[')
>>> -		/* Single element, not a list */
>>> -		return callback(str, data);
>>> -
>>> -	/* Sanity check, then strip the brackets */
>>> -	str_start = &str[strlen(str) - 1];
>>> -	if (*str_start != ']') {
>>> -		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
>>> -		return -EINVAL;
>>> -	}
>>> -	str++;
>>> -	*str_start = '\0';
>>> +	uint16_t i;
>>>
>>> -	/* Process list elements */
>>> -	state = 0;
>>> -	while (1) {
>>> -		if (state == 0) {
>>> -			if (*str == '\0')
>>> -				break;
>>> -			if (*str != ',') {
>>> -				str_start = str;
>>> -				state = 1;
>>> -			}
>>> -		} else if (state == 1) {
>>> -			if (*str == ',' || *str == '\0') {
>>> -				if (str > str_start) {
>>> -					/* Non-empty string fragment */
>>> -					*str = '\0';
>>> -					result = callback(str_start, data);
>>> -					if (result < 0)
>>> -						return result;
>>> -				}
>>> -				state = 0;
>>> -			}
>>> -		}
>>> -		str++;
>>> +	if (*len_list >= max_list)
>>> +		return -1;
>>
>> If current length is equal to max, but added value is already is in the list, it
>> should not be an error. So, these two lines should be moved after below for
>> loop.
>>
>>> +	for (i = 0; i < *len_list; i++) {
>>> +		if (list[i] == val)
>>> +			return 0;
>>>  	}
>>> +	list[(*len_list)++] = val;
>>>  	return 0;
>>>  }
>>>
>>> -static int
>>> +static char *
>>>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
>>>  	const uint16_t max_list)
>>>  {
>>>  	uint16_t lo, hi, val;
>>>  	int result;
>>> +	char *pos = str;
>>>
>>>  	result = sscanf(str, "%hu-%hu", &lo, &hi);
>>>  	if (result == 1) {
>>> -		if (*len_list >= max_list)
>>> -			return -ENOMEM;
>>> -		list[(*len_list)++] = lo;
>>> +		if (rte_eth_devargs_enlist(list, len_list, max_list, lo) != 0)
>>> +			return NULL;
>>>  	} else if (result == 2) {
>>> -		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi >
>> RTE_MAX_ETHPORTS)
>>
>> Strictly speaking removal of comparision vs RTE_MAX_ETHPORTS is a separate
>> logical change with a separate motivation.
> 
> To make this function comment for controller/pf/vf/sf parsing, the max value is
> passed in as parameter and checked in rte_eth_devargs_enlist().
> Caller code in rte_eth_devargs_process_list() decide the max value.

Which maximum? Maximum number of elements in array and
maximum element value are different things.

[snip]

Re: [dpdk-dev] [PATCH v5 5/9] ethdev: support PF index in representor

[Xueming(Steven) Li]

Hi Andrew,

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 4:01 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>
>Subject: Re: [PATCH v5 5/9] ethdev: support PF index in representor
>
>On 1/19/21 10:14 AM, Xueming Li wrote:
>> With Kernel bonding, multiple underlying PFs are bonded, VFs come from
>> different PF, need to identify representor of VFs unambiguously by
>> adding PF index.
>>
>> This patch introduces optional 'pf' section to representor devargs
>> syntax, examples:
>>  representor=pf0vf0             - single VF representor
>>  representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs
>
>
>Don't we need
> representor=pf3
>i.e. without VF or sub-function?

Standalone PF not used by Mellnaox PMD, but should be supported.
Will update.
>
>>
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>> ---
>>  doc/guides/prog_guide/poll_mode_drv.rst |  2 ++
>>  lib/librte_ethdev/ethdev_private.c      | 13 +++++++++++--
>>  2 files changed, 13 insertions(+), 2 deletions(-)
>>
>> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst
>> b/doc/guides/prog_guide/poll_mode_drv.rst
>> index 86e5867f1b..b2147aad30 100644
>> --- a/doc/guides/prog_guide/poll_mode_drv.rst
>> +++ b/doc/guides/prog_guide/poll_mode_drv.rst
>> @@ -382,6 +382,8 @@ parameters to those ports.
>>     -a DBDF,representor=sf[1,3,5]
>>     -a DBDF,representor=sf[0-1023]
>>     -a DBDF,representor=sf[0,2-4,7,9-11]
>> +   -a DBDF,representor=pf1vf0
>> +   -a DBDF,representor=pf[0-1]sf[0-127]
>>
>>  Note: PMDs are not required to support the standard device arguments
>> and users  should consult the relevant PMD documentation to see support
>devargs.
>> diff --git a/lib/librte_ethdev/ethdev_private.c
>> b/lib/librte_ethdev/ethdev_private.c
>> index d513f035d0..b9fdbd0f72 100644
>> --- a/lib/librte_ethdev/ethdev_private.c
>> +++ b/lib/librte_ethdev/ethdev_private.c
>> @@ -120,8 +120,8 @@ rte_eth_devargs_process_list(char *str, uint16_t
>*list, uint16_t *len_list,
>>   *
>>   * Representor format:
>>   *   #: range or single number of VF representor - legacy
>> - *   vf#: VF port representor/s
>> - *   sf#: SF port representor/s
>> + *   [pf#]vf#: VF port representor/s
>> + *   [pf#]sf#: SF port representor/s
>>   *
>>   * Examples of #:
>>   *  2               - single
>> @@ -133,6 +133,14 @@ rte_eth_devargs_parse_representor_ports(char
>> *str, void *data)  {
>>  	struct rte_eth_devargs *eth_da = data;
>>
>> +	if (str[0] == 'p' && str[1] == 'f') {
>> +		eth_da->type = RTE_ETH_REPRESENTOR_PF;
>> +		str += 2;
>> +		str = rte_eth_devargs_process_list(str, eth_da->ports,
>> +				&eth_da->nb_ports, RTE_MAX_ETHPORTS);
>
>May be RTE_MAX_ETHPORTS -> RTE_DIM(eth_da->ports) ?

Same here, the dim could be different than MAX value.
>
>> +		if (str == NULL)
>> +			goto err;
>
>Below we should not allow legacy VF syntax without "vf" prefix.

For backward compatibility, default numbers to "vf", otherwise some existing
app like OVS that working with VF will break.
>
>> +	}
>>  	if (str[0] == 'v' && str[1] == 'f') {
>>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>>  		str += 2;
>> @@ -144,6 +152,7 @@ rte_eth_devargs_parse_representor_ports(char *str,
>void *data)
>>  	}
>>  	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
>> +err:
>>  	if (str == NULL)
>>  		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
>>  	return str == NULL ? -1 : 0;
>>

Re: [dpdk-dev] [PATCH v5 6/9] ethdev: support multi-host in representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 4:03 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>; Bruce Richardson <bruce.richardson@intel.com>; NBU-
>Contact-Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
><ferruh.yigit@intel.com>
>Subject: Re: [PATCH v5 6/9] ethdev: support multi-host in representor
>
>On 1/19/21 10:14 AM, Xueming Li wrote:
>> The NIC can have multiple PCIe links and can be attached to the
>> multiple hosts, for example the same single NIC can be shared for
>> multiple server units in the rack. On each PCIe link NIC can provide
>> multiple PFs and VFs/SFs based on these ones. To provide the
>> unambiguous identification of the PCIe function the controller index
>> is added. The full representor identifier consists of three indices -
>> controller index, PF index, and VF or SF index (if any).
>>
>> This patch introduces controller index to ethdev representor syntax,
>> examples:
>>
>> [[c#]pf#]vf#: VF port representor/s, example: pf0vf1
>> [[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]
>>
>> c# is controller(host) ID/range in case of multi-host, optional.
>>
>> For user application (e.g. OVS), PMD is responsible to interpret and
>> locate representor device based on controller ID, PF ID and VF/SF ID
>> in representor syntax.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>> ---
>>  config/rte_config.h                   |  1 +
>>  lib/librte_ethdev/ethdev_private.c    | 12 ++++++++++--
>>  lib/librte_ethdev/rte_ethdev_driver.h |  4 ++++
>>  3 files changed, 15 insertions(+), 2 deletions(-)
>>
>> diff --git a/config/rte_config.h b/config/rte_config.h index
>> a0b5160ff2..23d02d51ef 100644
>> --- a/config/rte_config.h
>> +++ b/config/rte_config.h
>> @@ -58,6 +58,7 @@
>>  #define RTE_MAX_QUEUES_PER_PORT 1024
>>  #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */  #define
>> RTE_ETHDEV_RXTX_CALLBACKS 1
>> +#define RTE_MAX_MULTI_HOST_CTRLS 4
>>
>>  /* cryptodev defines */
>>  #define RTE_CRYPTO_MAX_DEVS 64
>> diff --git a/lib/librte_ethdev/ethdev_private.c
>> b/lib/librte_ethdev/ethdev_private.c
>> index b9fdbd0f72..9a0945c22e 100644
>> --- a/lib/librte_ethdev/ethdev_private.c
>> +++ b/lib/librte_ethdev/ethdev_private.c
>> @@ -120,8 +120,8 @@ rte_eth_devargs_process_list(char *str, uint16_t
>*list, uint16_t *len_list,
>>   *
>>   * Representor format:
>>   *   #: range or single number of VF representor - legacy
>> - *   [pf#]vf#: VF port representor/s
>> - *   [pf#]sf#: SF port representor/s
>> + *   [[c#]pf#]vf#: VF port representor/s
>> + *   [[c#]pf#]sf#: SF port representor/s
>>   *
>>   * Examples of #:
>>   *  2               - single
>> @@ -133,6 +133,14 @@ rte_eth_devargs_parse_representor_ports(char
>> *str, void *data)  {
>>  	struct rte_eth_devargs *eth_da = data;
>>
>> +	if (str[0] == 'c') {
>> +		str += 1;
>> +		str = rte_eth_devargs_process_list(str, eth_da-
>>mh_controllers,
>> +				&eth_da->nb_mh_controllers,
>> +				RTE_DIM(eth_da->mh_controllers));
>> +		if (str == NULL)
>> +			goto err;
>
>The parser must enforce pf to follow. I.e. should not allow c1vf3. At least
>above syntax description in comments says so.

Good catch, will add, thanks.
>
>> +	}
>>  	if (str[0] == 'p' && str[1] == 'f') {
>>  		eth_da->type = RTE_ETH_REPRESENTOR_PF;
>>  		str += 2;
>> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
>> b/lib/librte_ethdev/rte_ethdev_driver.h
>> index cd33184f17..b01f118965 100644
>> --- a/lib/librte_ethdev/rte_ethdev_driver.h
>> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
>> @@ -1203,6 +1203,10 @@ enum rte_eth_representor_type {
>>
>>  /** Generic Ethernet device arguments  */  struct rte_eth_devargs {
>> +	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
>> +	/** controller/s number in case of multi-host */
>> +	uint16_t nb_mh_controllers;
>> +	/** number of controllers in multi-host controllers field */
>>  	uint16_t ports[RTE_MAX_ETHPORTS];
>>  	/** port/s number to enable on a multi-port single function */
>>  	uint16_t nb_ports;
>>

Re: [dpdk-dev] [PATCH v5 5/9] ethdev: support PF index in representor

[Andrew Rybchenko]

On 1/19/21 12:30 PM, Xueming(Steven) Li wrote:
> Hi Andrew,
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Tuesday, January 19, 2021 4:01 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
>> <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
>> <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>
>> Subject: Re: [PATCH v5 5/9] ethdev: support PF index in representor
>>
>> On 1/19/21 10:14 AM, Xueming Li wrote:
>>> With Kernel bonding, multiple underlying PFs are bonded, VFs come from
>>> different PF, need to identify representor of VFs unambiguously by
>>> adding PF index.
>>>
>>> This patch introduces optional 'pf' section to representor devargs
>>> syntax, examples:
>>>  representor=pf0vf0             - single VF representor
>>>  representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs
>>
>>
>> Don't we need
>> representor=pf3
>> i.e. without VF or sub-function?
> 
> Standalone PF not used by Mellnaox PMD, but should be supported.
> Will update.
>>
>>>
>>>
>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>> ---
>>>  doc/guides/prog_guide/poll_mode_drv.rst |  2 ++
>>>  lib/librte_ethdev/ethdev_private.c      | 13 +++++++++++--
>>>  2 files changed, 13 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst
>>> b/doc/guides/prog_guide/poll_mode_drv.rst
>>> index 86e5867f1b..b2147aad30 100644
>>> --- a/doc/guides/prog_guide/poll_mode_drv.rst
>>> +++ b/doc/guides/prog_guide/poll_mode_drv.rst
>>> @@ -382,6 +382,8 @@ parameters to those ports.
>>>     -a DBDF,representor=sf[1,3,5]
>>>     -a DBDF,representor=sf[0-1023]
>>>     -a DBDF,representor=sf[0,2-4,7,9-11]
>>> +   -a DBDF,representor=pf1vf0
>>> +   -a DBDF,representor=pf[0-1]sf[0-127]
>>>
>>>  Note: PMDs are not required to support the standard device arguments
>>> and users  should consult the relevant PMD documentation to see support
>> devargs.
>>> diff --git a/lib/librte_ethdev/ethdev_private.c
>>> b/lib/librte_ethdev/ethdev_private.c
>>> index d513f035d0..b9fdbd0f72 100644
>>> --- a/lib/librte_ethdev/ethdev_private.c
>>> +++ b/lib/librte_ethdev/ethdev_private.c
>>> @@ -120,8 +120,8 @@ rte_eth_devargs_process_list(char *str, uint16_t
>> *list, uint16_t *len_list,
>>>   *
>>>   * Representor format:
>>>   *   #: range or single number of VF representor - legacy
>>> - *   vf#: VF port representor/s
>>> - *   sf#: SF port representor/s
>>> + *   [pf#]vf#: VF port representor/s
>>> + *   [pf#]sf#: SF port representor/s
>>>   *
>>>   * Examples of #:
>>>   *  2               - single
>>> @@ -133,6 +133,14 @@ rte_eth_devargs_parse_representor_ports(char
>>> *str, void *data)  {
>>>  	struct rte_eth_devargs *eth_da = data;
>>>
>>> +	if (str[0] == 'p' && str[1] == 'f') {
>>> +		eth_da->type = RTE_ETH_REPRESENTOR_PF;
>>> +		str += 2;
>>> +		str = rte_eth_devargs_process_list(str, eth_da->ports,
>>> +				&eth_da->nb_ports, RTE_MAX_ETHPORTS);
>>
>> May be RTE_MAX_ETHPORTS -> RTE_DIM(eth_da->ports) ?
> 
> Same here, the dim could be different than MAX value.

Hold on, just for my understanding. The maximum says how
many entries could be added to the array. So, why?

>>
>>> +		if (str == NULL)
>>> +			goto err;
>>
>> Below we should not allow legacy VF syntax without "vf" prefix.
> 
> For backward compatibility, default numbers to "vf", otherwise some existing
> app like OVS that working with VF will break.

I mean if new syntax is used (i.e. we have pfX prefix), we must
deny legacy syntax for VFs below.

>>
>>> +	}
>>>  	if (str[0] == 'v' && str[1] == 'f') {
>>>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>>>  		str += 2;
>>> @@ -144,6 +152,7 @@ rte_eth_devargs_parse_representor_ports(char *str,
>> void *data)
>>>  	}
>>>  	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>>>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
>>> +err:
>>>  	if (str == NULL)
>>>  		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
>>>  	return str == NULL ? -1 : 0;
>>>
> 

Re: [dpdk-dev] [PATCH v4 2/9] ethdev: support representor port list

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 5:04 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; NBU-Contact-Thomas
>Monjalon <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>;
>Olivier Matz <olivier.matz@6wind.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>
>Subject: Re: [PATCH v4 2/9] ethdev: support representor port list
>
>On 1/19/21 11:59 AM, Xueming(Steven) Li wrote:
>> Hi Andrew,
>>
>>> -----Original Message-----
>>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>> Sent: Tuesday, January 19, 2021 3:46 PM
>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; NBU-Contact-Thomas
>>> Monjalon <thomas@monjalon.net>; Ferruh Yigit
>>> <ferruh.yigit@intel.com>; Olivier Matz <olivier.matz@6wind.com>
>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>> Penso <asafp@nvidia.com>
>>> Subject: Re: [PATCH v4 2/9] ethdev: support representor port list
>>>
>>> On 1/18/21 2:16 PM, Xueming Li wrote:
>>>> To support extended representor syntax, this patch extends the
>>>> representor list parsing to support for representor port range in
>>>> devargs, examples:
>>>>    representor=[1,2,3]         - single list
>>>>    representor=[1,3-5,7,9-11]  - list with singles and ranges
>>>>
>>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>>
>>> See below
>>>
>>>> ---
>>>>  lib/librte_ethdev/ethdev_private.c | 105 ++++++++++++++---------------
>>>>  lib/librte_ethdev/ethdev_private.h |   3 -
>>>>  lib/librte_ethdev/rte_class_eth.c  |   4 +-
>>>>  lib/librte_ethdev/rte_ethdev.c     |   5 +-
>>>>  4 files changed, 54 insertions(+), 63 deletions(-)
>>>>
>>>> diff --git a/lib/librte_ethdev/ethdev_private.c
>>>> b/lib/librte_ethdev/ethdev_private.c
>>>> index c1a411dba4..12bcc7e98d 100644
>>>> --- a/lib/librte_ethdev/ethdev_private.c
>>>> +++ b/lib/librte_ethdev/ethdev_private.c
>>>> @@ -38,77 +38,71 @@ eth_find_device(const struct rte_eth_dev *start,
>>> rte_eth_cmp_t cmp,
>>>>  	return NULL;
>>>>  }
>>>>
>>>> -int
>>>> -rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t
>callback,
>>>> -	void *data)
>>>> +static int
>>>> +rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
>>>> +		       const uint16_t max_list, uint16_t val)
>>>>  {
>>>> -	char *str_start;
>>>> -	int state;
>>>> -	int result;
>>>> -
>>>> -	if (*str != '[')
>>>> -		/* Single element, not a list */
>>>> -		return callback(str, data);
>>>> -
>>>> -	/* Sanity check, then strip the brackets */
>>>> -	str_start = &str[strlen(str) - 1];
>>>> -	if (*str_start != ']') {
>>>> -		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
>>>> -		return -EINVAL;
>>>> -	}
>>>> -	str++;
>>>> -	*str_start = '\0';
>>>> +	uint16_t i;
>>>>
>>>> -	/* Process list elements */
>>>> -	state = 0;
>>>> -	while (1) {
>>>> -		if (state == 0) {
>>>> -			if (*str == '\0')
>>>> -				break;
>>>> -			if (*str != ',') {
>>>> -				str_start = str;
>>>> -				state = 1;
>>>> -			}
>>>> -		} else if (state == 1) {
>>>> -			if (*str == ',' || *str == '\0') {
>>>> -				if (str > str_start) {
>>>> -					/* Non-empty string fragment */
>>>> -					*str = '\0';
>>>> -					result = callback(str_start, data);
>>>> -					if (result < 0)
>>>> -						return result;
>>>> -				}
>>>> -				state = 0;
>>>> -			}
>>>> -		}
>>>> -		str++;
>>>> +	if (*len_list >= max_list)
>>>> +		return -1;
>>>
>>> If current length is equal to max, but added value is already is in
>>> the list, it should not be an error. So, these two lines should be
>>> moved after below for loop.
>>>
>>>> +	for (i = 0; i < *len_list; i++) {
>>>> +		if (list[i] == val)
>>>> +			return 0;
>>>>  	}
>>>> +	list[(*len_list)++] = val;
>>>>  	return 0;
>>>>  }
>>>>
>>>> -static int
>>>> +static char *
>>>>  rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t
>*len_list,
>>>>  	const uint16_t max_list)
>>>>  {
>>>>  	uint16_t lo, hi, val;
>>>>  	int result;
>>>> +	char *pos = str;
>>>>
>>>>  	result = sscanf(str, "%hu-%hu", &lo, &hi);
>>>>  	if (result == 1) {
>>>> -		if (*len_list >= max_list)
>>>> -			return -ENOMEM;
>>>> -		list[(*len_list)++] = lo;
>>>> +		if (rte_eth_devargs_enlist(list, len_list, max_list, lo) != 0)
>>>> +			return NULL;
>>>>  	} else if (result == 2) {
>>>> -		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi >
>>> RTE_MAX_ETHPORTS)
>>>
>>> Strictly speaking removal of comparision vs RTE_MAX_ETHPORTS is a
>>> separate logical change with a separate motivation.
>>
>> To make this function comment for controller/pf/vf/sf parsing, the max
>> value is passed in as parameter and checked in rte_eth_devargs_enlist().
>> Caller code in rte_eth_devargs_process_list() decide the max value.
>
>Which maximum? Maximum number of elements in array and maximum
>element value are different things.

My bad, after another check, it should be max number of elements in array,
not max element value. Will update, thanks! 
>
>[snip]

Re: [dpdk-dev] [PATCH v5 7/9] devarg: change representor ID to bitmap

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 4:21 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>;
>Somnath Kotur <somnath.kotur@broadcom.com>; John Daley
><johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>; Beilei Xing
><beilei.xing@intel.com>; Jeff Guo <jia.guo@intel.com>; Haiyue Wang
><haiyue.wang@intel.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler
><shahafs@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Ray Kinsella
><mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
>Subject: Re: [PATCH v5 7/9] devarg: change representor ID to bitmap
>
>On 1/19/21 10:14 AM, Xueming Li wrote:
>> The NIC can have multiple PCIe links and can be attached to multiple
>> hosts, for example the same single NIC can be shared for multiple
>> server units in the rack. On each PCIe link NIC can provide multiple
>> PFs and VFs/SFs based on these ones. The full representor identifier
>> consists of three indices - controller index, PF index, and VF or SF index (if
>any).
>>
>> SR-IOV and SubFunction are created on top of PF. PF index is
>> introduced because there might be multiple PFs in the bonding
>> configuration and only bonding device is probed.
>>
>> In eth representor comparator callback, ethdev was compared with devarg.
>> Since ethdev representor port didn't contain controller index and PF
>> index information, callback returned representor from other PF or
>> controller.
>>
>> This patch changes representor ID to bitmap so that the ethdev
>> representor comparer callback returns correct ethdev by comparing full
>> representor information including: controller index, PF index,
>> representor type, SF or VF index.
>>
>> Representor ID bitmap definition:
>>  xxxx xxxx xxxx xxxx
>>  |||| |LLL LLLL LLLL vf/sf id
>>  |||| L 1:sf, 0:vf
>>  ||LL pf id
>>  LL controller(host) id
>
>What about PF representor case? I.e. representor for entire PF.
>
>Also it implies that controller ID 0 is the caller. I.e.
>special meaning. So, space for just 3 specific contoller left
>
>Similar for PF. In fact it is worse. E.g. PMD is bound to the second PF (PF
>number 1). If so, vf0 means the first VF of the PF itself, i.e. PF 1 VF 0. But,
>pf0vf0 should mean PF 1 VF 1.

Agree, need to extend bits width in LTS release.

PF representor is not considered here, how about moving one bit from vf/sf id?
1k SF devices should be fine for me so far.

The controller ID and PF ID is related to the context device, how device configured
and bonding state is critical for PMD to interpret the IDs. For example:
"<BDF>,representor=pf1vf1" is valid for bonding device, invalid for standalone device.
"c#" is meaningful for multi-host scenario, invalid for normal NIC. PMD is responsible to
encode representor ID correctly according to device configuration to make Device+ReprID
unique, because the ReprID is used in device iterator. So the user app should specify
representor syntax with necessary parts to cover device configuration, PMD should 
extract required info according to device state.

>
>>
>> This approach keeps binary compatibility with all drivers, VF
>> representor id matches with simple id for non-bonding and
>> non-multi-host configurations.
>>
>> In the future, the representor ID field and each section should extend
>> to bigger width to support more devices.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>
>
>> diff --git a/lib/librte_ethdev/rte_class_eth.c
>> b/lib/librte_ethdev/rte_class_eth.c
>> index efe6149df5..994db96960 100644
>> --- a/lib/librte_ethdev/rte_class_eth.c
>> +++ b/lib/librte_ethdev/rte_class_eth.c
>> @@ -66,8 +66,8 @@ eth_representor_cmp(const char *key __rte_unused,
>>  	int ret;
>>  	char *values;
>>  	const struct rte_eth_dev_data *data = opaque;
>> -	struct rte_eth_devargs representors;
>> -	uint16_t index;
>> +	struct rte_eth_devargs eth_da;
>> +	uint16_t index, c, p, f;
>>
>>  	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
>>  		return -1; /* not a representor port */ @@ -76,17 +76,39 @@
>> eth_representor_cmp(const char *key __rte_unused,
>>  	values = strdup(value);
>>  	if (values == NULL)
>>  		return -1;
>> -	memset(&representors, 0, sizeof(representors));
>> -	ret = rte_eth_devargs_parse_representor_ports(values,
>&representors);
>> +	memset(&eth_da, 0, sizeof(eth_da));
>> +	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
>>  	free(values);
>>  	if (ret != 0)
>>  		return -1; /* invalid devargs value */
>>
>> +	/* Set default values. */
>> +	if (eth_da.nb_mh_controllers == 0) {
>> +		eth_da.nb_mh_controllers = 1;
>> +		eth_da.mh_controllers[0] = 0;
>> +	}
>> +	if (eth_da.nb_ports == 0) {
>> +		eth_da.nb_ports = 1;
>> +		eth_da.ports[0] = 0;
>> +	}
>> +	if (eth_da.nb_representor_ports == 0) {
>> +		eth_da.nb_representor_ports = 1;
>> +		eth_da.representor_ports[0] = 0;
>> +	}
>>  	/* Return 0 if representor id is matching one of the values. */
>> -	for (index = 0; index < representors.nb_representor_ports; index++)
>> -		if (data->representor_id ==
>> -				representors.representor_ports[index])
>> -			return 0;
>> +	for (c = 0; c < eth_da.nb_mh_controllers; ++c) {
>> +		for (p = 0; p < eth_da.nb_ports; ++p) {
>> +			for (f = 0; f < eth_da.nb_representor_ports; ++f) {
>> +				index = rte_eth_representor_id_encode(
>> +					eth_da.mh_controllers[c],
>> +					eth_da.ports[p],
>> +					eth_da.type,
>> +					eth_da.representor_ports[f]);
>> +				if (data->representor_id == index)
>> +					return 0;
>> +			}
>> +		}
>> +	}
>>  	return -1; /* no match */
>>  }
>>
>> diff --git a/lib/librte_ethdev/rte_ethdev.c
>> b/lib/librte_ethdev/rte_ethdev.c index 91b3263338..2cac0ccfbd 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -5575,6 +5575,32 @@ rte_eth_devargs_parse(const char *dargs, struct
>rte_eth_devargs *eth_da)
>>  	return result;
>>  }
>>
>> +uint16_t
>> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
>> +			      enum rte_eth_representor_type type,
>> +			      uint16_t representor_port)
>> +{
>> +	return (((controller & 3) << 14) |
>> +		((pf & 3) << 12) |
>> +		(!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
>> +		(representor_port & 0x7ff));
>> +}
>> +
>> +uint16_t
>> +rte_eth_representor_id_parse(const uint16_t representor_id,
>> +			     uint16_t *controller, uint16_t *pf,
>> +			     enum rte_eth_representor_type *type) {
>> +	if (controller)
>
>Compare vs NULL in accordance with DPDK coding style guide
>
>> +		*controller = (representor_id >> 14) & 3;
>> +	if (pf)
>
>Compare vs NULL in accordance with DPDK coding style guide
>
>> +		*pf = (representor_id >> 12) & 3;
>> +	if (type)
>
>Compare vs NULL in accordance with DPDK coding style guide
>
>> +		*type = ((representor_id >> 11) & 1) ?
>> +			RTE_ETH_REPRESENTOR_SF :
>RTE_ETH_REPRESENTOR_VF;
>> +	return representor_id & 0x7ff;
>> +}
>> +
>>  static int
>>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>>  		const char *params __rte_unused,
>> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
>> b/lib/librte_ethdev/rte_ethdev_driver.h
>> index b01f118965..57253c8f90 100644
>> --- a/lib/librte_ethdev/rte_ethdev_driver.h
>> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
>> @@ -1218,6 +1218,59 @@ struct rte_eth_devargs {
>>  	enum rte_eth_representor_type type; /* type of representor */  };
>>
>> +#define RTE_NO_REPRESENTOR_ID UINT16_MAX /**< No representor ID.
>*/
>> +
>> +/**
>> + * PMD helper function to encode representor ID
>> + *
>> + * The compact format is used for device iterator that comparing
>> + * ethdev representor ID with target devargs.
>> + *
>> + * xxxx xxxx xxxx xxxx
>> + * |||| |LLL LLLL LLLL vf/sf id
>> + * |||| L 1:sf, 0:vf
>> + * ||LL pf id
>> + * LL controller(host) id
>> + *
>> + * @param controller
>> + *  Controller ID.
>> + * @param pf
>> + *  PF port ID.
>> + * @param type
>> + *  Representor type.
>> + * @param representor_port
>> + *  Representor port ID.
>> + *
>> + * @return
>> + *   Encoded representor ID.
>> + */
>> +__rte_internal
>> +uint16_t
>> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
>> +			      enum rte_eth_representor_type type,
>> +			      uint16_t representor_port);
>> +
>> +/**
>> + * PMD helper function to parse representor ID
>> + *
>> + * @param representor_id
>> + *  Representor ID.
>> + * @param controller
>> + *  Parsed controller ID.
>> + * @param pf
>> + *  Parsed PF port ID.
>> + * @param type
>> + *  Parsed representor type.
>> + *
>> + * @return
>> + *   Parsed representor port ID.
>> + */
>> +__rte_internal
>> +uint16_t
>> +rte_eth_representor_id_parse(const uint16_t representor_id,
>> +			     uint16_t *controller, uint16_t *pf,
>> +			     enum rte_eth_representor_type *type);
>> +
>>  /**
>>   * PMD helper function to parse ethdev arguments
>>   *
>> diff --git a/lib/librte_ethdev/version.map
>> b/lib/librte_ethdev/version.map index d3f5410806..44edaed507 100644
>> --- a/lib/librte_ethdev/version.map
>> +++ b/lib/librte_ethdev/version.map
>> @@ -257,6 +257,8 @@ INTERNAL {
>>  	rte_eth_dev_release_port;
>>  	rte_eth_dev_internal_reset;
>>  	rte_eth_devargs_parse;
>> +	rte_eth_representor_id_encode;
>> +	rte_eth_representor_id_parse;
>
>The place looks wrong. It must be sorted or added at bottom with the
>comment with version number. I'd prefer to have it simply alphanumberically
>sorted.
>
>>  	rte_eth_dma_zone_free;
>>  	rte_eth_dma_zone_reserve;
>>  	rte_eth_hairpin_queue_peer_bind;
>>

Re: [dpdk-dev] [PATCH v5 7/9] devarg: change representor ID to bitmap

[Andrew Rybchenko]

On 1/19/21 2:04 PM, Xueming(Steven) Li wrote:
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Tuesday, January 19, 2021 4:21 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
>> <asafp@nvidia.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>;
>> Somnath Kotur <somnath.kotur@broadcom.com>; John Daley
>> <johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>; Beilei Xing
>> <beilei.xing@intel.com>; Jeff Guo <jia.guo@intel.com>; Haiyue Wang
>> <haiyue.wang@intel.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler
>> <shahafs@nvidia.com>; NBU-Contact-Thomas Monjalon
>> <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Ray Kinsella
>> <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
>> Subject: Re: [PATCH v5 7/9] devarg: change representor ID to bitmap
>>
>> On 1/19/21 10:14 AM, Xueming Li wrote:
>>> The NIC can have multiple PCIe links and can be attached to multiple
>>> hosts, for example the same single NIC can be shared for multiple
>>> server units in the rack. On each PCIe link NIC can provide multiple
>>> PFs and VFs/SFs based on these ones. The full representor identifier
>>> consists of three indices - controller index, PF index, and VF or SF index (if
>> any).
>>>
>>> SR-IOV and SubFunction are created on top of PF. PF index is
>>> introduced because there might be multiple PFs in the bonding
>>> configuration and only bonding device is probed.
>>>
>>> In eth representor comparator callback, ethdev was compared with devarg.
>>> Since ethdev representor port didn't contain controller index and PF
>>> index information, callback returned representor from other PF or
>>> controller.
>>>
>>> This patch changes representor ID to bitmap so that the ethdev
>>> representor comparer callback returns correct ethdev by comparing full
>>> representor information including: controller index, PF index,
>>> representor type, SF or VF index.
>>>
>>> Representor ID bitmap definition:
>>>  xxxx xxxx xxxx xxxx
>>>  |||| |LLL LLLL LLLL vf/sf id
>>>  |||| L 1:sf, 0:vf
>>>  ||LL pf id
>>>  LL controller(host) id
>>
>> What about PF representor case? I.e. representor for entire PF.
>>
>> Also it implies that controller ID 0 is the caller. I.e.
>> special meaning. So, space for just 3 specific contoller left
>>
>> Similar for PF. In fact it is worse. E.g. PMD is bound to the second PF (PF
>> number 1). If so, vf0 means the first VF of the PF itself, i.e. PF 1 VF 0. But,
>> pf0vf0 should mean PF 1 VF 1.
> 
> Agree, need to extend bits width in LTS release.

See my reply to cover mail.

> PF representor is not considered here, how about moving one bit from vf/sf id?
> 1k SF devices should be fine for me so far.

We could reserve max VF/SF number to denote PF itself.

> The controller ID and PF ID is related to the context device, how device configured
> and bonding state is critical for PMD to interpret the IDs. For example:
> "<BDF>,representor=pf1vf1" is valid for bonding device, invalid for standalone device.

I guess it is mlx5 specific. IMHO, pf1vf1 makes sense even
without bonding.

> "c#" is meaningful for multi-host scenario, invalid for normal NIC. PMD is responsible to
> encode representor ID correctly according to device configuration to make Device+ReprID
> unique, because the ReprID is used in device iterator. So the user app should specify
> representor syntax with necessary parts to cover device configuration, PMD should 
> extract required info according to device state.

See cover mail reply.

Re: [dpdk-dev] [PATCH v5 8/9] ethdev: add capability of sub-function representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 4:06 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>
>Subject: Re: [PATCH v5 8/9] ethdev: add capability of sub-function representor
>
>On 1/19/21 10:15 AM, Xueming Li wrote:
>> Old DPDK version or some drivers didn't support SubFunction representor.
>> For application to adapt different DPDK version automatically, or to
>> be used for different NICs, this patch introduces new eth device
>> capability of supporting SubFunction representor device.
>
>Sorry, it does not sound sufficient motivation to introduce the capability. I
>simply need real life example why application need to know it.

I had same internal discussion on this as well :)
A simple example, for customer running DPDK based app with NICs from different vendors,
app need a flag to know whether the device support SF representor, hotplug SF if the
capability shows "support". This also happens with different model/fw even from same vendor.
PMD report device+driver capability that whether SF supported.

>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>> ---
>>  lib/librte_ethdev/rte_ethdev.h | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/lib/librte_ethdev/rte_ethdev.h
>> b/lib/librte_ethdev/rte_ethdev.h index 2cbce958cf..da65adf1ab 100644
>> --- a/lib/librte_ethdev/rte_ethdev.h
>> +++ b/lib/librte_ethdev/rte_ethdev.h
>> @@ -1433,6 +1433,8 @@ struct rte_eth_conf {  #define
>> RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
>>  /** Device supports Tx queue setup after device started. */  #define
>> RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
>> +/** Device supports SubFunction representor. */
>> +#define RTE_ETH_DEV_CAPA_REPRESENTOR_SF         0x00000004
>
>Will we have a PMD which supports it in the release cycle?

Please see reply above, it also reflect device hw capability.
>
>>  /**@}*/
>>
>>  /*

Re: [dpdk-dev] [PATCH v5 8/9] ethdev: add capability of sub-function representor

[Andrew Rybchenko]

On 1/19/21 2:19 PM, Xueming(Steven) Li wrote:
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Tuesday, January 19, 2021 4:06 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
>> <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
>> <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>
>> Subject: Re: [PATCH v5 8/9] ethdev: add capability of sub-function representor
>>
>> On 1/19/21 10:15 AM, Xueming Li wrote:
>>> Old DPDK version or some drivers didn't support SubFunction representor.
>>> For application to adapt different DPDK version automatically, or to
>>> be used for different NICs, this patch introduces new eth device
>>> capability of supporting SubFunction representor device.
>>
>> Sorry, it does not sound sufficient motivation to introduce the capability. I
>> simply need real life example why application need to know it.
> 
> I had same internal discussion on this as well :)
> A simple example, for customer running DPDK based app with NICs from different vendors,
> app need a flag to know whether the device support SF representor, hotplug SF if the
> capability shows "support". This also happens with different model/fw even from same vendor.
> PMD report device+driver capability that whether SF supported.

Single feature bit is insufficient. Application needs to know
how many SF may be used on which PF.

>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>

[snip]

Re: [dpdk-dev] [PATCH v5 5/9] ethdev: support PF index in representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 5:36 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>
>Subject: Re: [PATCH v5 5/9] ethdev: support PF index in representor
>
>On 1/19/21 12:30 PM, Xueming(Steven) Li wrote:
>> Hi Andrew,
>>
>>> -----Original Message-----
>>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>> Sent: Tuesday, January 19, 2021 4:01 PM
>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>> Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
>>> <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>
>>> Subject: Re: [PATCH v5 5/9] ethdev: support PF index in representor
>>>
>>> On 1/19/21 10:14 AM, Xueming Li wrote:
>>>> With Kernel bonding, multiple underlying PFs are bonded, VFs come
>>>> from different PF, need to identify representor of VFs unambiguously
>>>> by adding PF index.
>>>>
>>>> This patch introduces optional 'pf' section to representor devargs
>>>> syntax, examples:
>>>>  representor=pf0vf0             - single VF representor
>>>>  representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs
>>>
>>>
>>> Don't we need
>>> representor=pf3
>>> i.e. without VF or sub-function?
>>
>> Standalone PF not used by Mellnaox PMD, but should be supported.
>> Will update.
>>>
>>>>
>>>>
>>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>>> ---
>>>>  doc/guides/prog_guide/poll_mode_drv.rst |  2 ++
>>>>  lib/librte_ethdev/ethdev_private.c      | 13 +++++++++++--
>>>>  2 files changed, 13 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst
>>>> b/doc/guides/prog_guide/poll_mode_drv.rst
>>>> index 86e5867f1b..b2147aad30 100644
>>>> --- a/doc/guides/prog_guide/poll_mode_drv.rst
>>>> +++ b/doc/guides/prog_guide/poll_mode_drv.rst
>>>> @@ -382,6 +382,8 @@ parameters to those ports.
>>>>     -a DBDF,representor=sf[1,3,5]
>>>>     -a DBDF,representor=sf[0-1023]
>>>>     -a DBDF,representor=sf[0,2-4,7,9-11]
>>>> +   -a DBDF,representor=pf1vf0
>>>> +   -a DBDF,representor=pf[0-1]sf[0-127]
>>>>
>>>>  Note: PMDs are not required to support the standard device
>>>> arguments and users  should consult the relevant PMD documentation
>>>> to see support
>>> devargs.
>>>> diff --git a/lib/librte_ethdev/ethdev_private.c
>>>> b/lib/librte_ethdev/ethdev_private.c
>>>> index d513f035d0..b9fdbd0f72 100644
>>>> --- a/lib/librte_ethdev/ethdev_private.c
>>>> +++ b/lib/librte_ethdev/ethdev_private.c
>>>> @@ -120,8 +120,8 @@ rte_eth_devargs_process_list(char *str, uint16_t
>>> *list, uint16_t *len_list,
>>>>   *
>>>>   * Representor format:
>>>>   *   #: range or single number of VF representor - legacy
>>>> - *   vf#: VF port representor/s
>>>> - *   sf#: SF port representor/s
>>>> + *   [pf#]vf#: VF port representor/s
>>>> + *   [pf#]sf#: SF port representor/s
>>>>   *
>>>>   * Examples of #:
>>>>   *  2               - single
>>>> @@ -133,6 +133,14 @@ rte_eth_devargs_parse_representor_ports(char
>>>> *str, void *data)  {
>>>>  	struct rte_eth_devargs *eth_da = data;
>>>>
>>>> +	if (str[0] == 'p' && str[1] == 'f') {
>>>> +		eth_da->type = RTE_ETH_REPRESENTOR_PF;
>>>> +		str += 2;
>>>> +		str = rte_eth_devargs_process_list(str, eth_da->ports,
>>>> +				&eth_da->nb_ports, RTE_MAX_ETHPORTS);
>>>
>>> May be RTE_MAX_ETHPORTS -> RTE_DIM(eth_da->ports) ?
>>
>> Same here, the dim could be different than MAX value.
>
>Hold on, just for my understanding. The maximum says how many entries
>could be added to the array. So, why?

Right, will change to RTE_DIM(), thanks!
>
>>>
>>>> +		if (str == NULL)
>>>> +			goto err;
>>>
>>> Below we should not allow legacy VF syntax without "vf" prefix.
>>
>> For backward compatibility, default numbers to "vf", otherwise some
>> existing app like OVS that working with VF will break.
>
>I mean if new syntax is used (i.e. we have pfX prefix), we must deny legacy
>syntax for VFs below.

Correct, will add check, thanks!
>
>>>
>>>> +	}
>>>>  	if (str[0] == 'v' && str[1] == 'f') {
>>>>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>>>>  		str += 2;
>>>> @@ -144,6 +152,7 @@ rte_eth_devargs_parse_representor_ports(char
>>>> *str,
>>> void *data)
>>>>  	}
>>>>  	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>>>>  		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
>>>> +err:
>>>>  	if (str == NULL)
>>>>  		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
>>>>  	return str == NULL ? -1 : 0;
>>>>
>>

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, January 19, 2021 4:41 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction
>representor
>
>On 1/19/21 10:13 AM, Xueming Li wrote:
>> dedicated queues(txq, rxq). A SF netdev supports E-Switch
>> representation offload similar to existing PF and VF representors. A
>> SF shares PCI level resources with other SFs and/or with its parent PCI
>function.
>>
>>>From SmartNIC perspective, when PCI device is shared for multi-host,
>> representors for host controller and host PF is required.
>>
>> This patch set introduces new representor types in addtion to existing
>> VF representor. Syntax:
>>
>> [[c#]pf#]vf#: VF port representor/s from controller/pf
>> [[c#]pf#]sf#: SF port representor/s from controller/pf
>> #: VF representor - for backwards compatibility
>>
>> "#" is number instance, list or range, valid examples:
>>   1, [1,3,5], [0-3], [0,2-4,6]
>>
>> For backward compatibility, this patch also introduces new netdev
>> capability to indicate the capability of supportting SF representor.
>
>The patch series looks really nice. Thanks.
>
>As before, my biggest concern is making representor ID a bitmap. See my
>comments to a specific patch.
>Plus absence of defined semantics of caller function.
>Basically it is looks like it is assumed that controller #0 and PF #0 is the caller.
>However, it could be wrong.

From devargs syntax perspective, a VF representor on PF1 could be referenced either way:
  1:  <PF0_BDF>,representor=pf1vfX
  2:  <PF1_BDF>,representor=vfX
  3:  <PF1_BDF>,representor=pf0vfX // works but not suggested, use PF0 BDF as caller
If probe a device with PF0 BDF and locate it with PF1 BDF, the device iterator will fail,
devargs is parsed by api, EAL layer simply compare PCI BDF and then representor ID, 
i.e. "c#pf#vf#". The caller device should be consistent, better to use the first device.

Devrg "<PF1_BDF>,representor=vfX" will work, representor controller ID and pf ID
default to #0, relative to caller context PF1 BDF.

Is it good to add such suggestion/behavior on rte_eth_devargs comments?

>
>The next biggest concert is the absence of capability reporting API. How many
>controller are available?
>How many PFs on each controller are available?
>How many VFs on each controller/PF are available?
>How many SFs?
>
>From the first sight it sounds not that important right now and an extra
>feature which could be added in the follow up patches, but IMHO addition of
>the API would allow to avoid making representor ID a bitmap.
>Basically capabilities API can provide an array of available functions with
>representor ID assigned to each entry. Also it could make the entire patch
>series optional since it would allow to interpret numbers in representor=[....]
>as representor IDs which are mapped to controller/PF/VF/SF by the
>capabilities reporting API.
>
>I realize that sometimes it could be more convenient to use syntax suggested
>here. Mainly for human.

Agree, mostly for human. Regarding to capability reporting API, how about remove
current one and enhance it later with a complete patch set?

Re: [dpdk-dev] [PATCH v4 7/9] devarg: change representor ID to bitmap

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ajit Khaparde <ajit.khaparde@broadcom.com>
>Sent: Tuesday, January 19, 2021 3:02 AM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
><ferruh.yigit@intel.com>; Andrew Rybchenko
><andrew.rybchenko@oktetlabs.ru>; Olivier Matz <olivier.matz@6wind.com>;
>dpdk-dev <dev@dpdk.org>; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>Penso <asafp@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v4 7/9] devarg: change representor ID to
>bitmap
>
>On Mon, Jan 18, 2021 at 3:18 AM Xueming Li <xuemingl@nvidia.com> wrote:
>>
>> The NIC can have multiple PCIe links and can be attached to multiple
>> hosts, for example the same single NIC can be shared for multiple server
>> units in the rack. On each PCIe link NIC can provide multiple PFs and
>> VFs/SFs based on these ones. The full representor identifier consists of
>> three indices - controller index, PF index, and VF or SF index (if any).
>>
>> SR-IOV and SubFunction are created on top of PF. PF index is introduced
>> because there might be multiple PFs in the bonding configuration and
>> only bonding device is probed.
>>
>> In eth representor comparator callback, ethdev was compared with devarg.
>> Since ethdev representor port didn't contain controller index and PF
>> index information, callback returned representor from other PF or
>> controller.
>>
>> This patch changes representor ID to bitmap so that the ethdev
>> representor comparer callback returns correct ethdev by comparing full
>> representor information including: controller index, PF index,
>> representor type, SF or VF index.
>>
>> Representor ID bitmap definition:
>>  xxxx xxxx xxxx xxxx
>>  |||| |LLL LLLL LLLL vf/sf id
>>  |||| L 1:sf, 0:vf
>>  ||LL pf id
>>  LL controller(host) id
>>
>> This approach keeps binary compatibility with all drivers, VF
>> representor id matches with simple id for non-bonding and non-multi-host
>> configurations.
>>
>> In the future, the representor ID field and each section should extend
>> to bigger width to support more devices.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>Not just in this patch, there is a lot of info in the commit logs.
>Unless I missed it, I don't see an update to
>doc/guides/prog_guide/switch_representation.rst
>Can you please update that.

Thanks, updated in next version.
>
>> ---
>>  drivers/net/bnxt/bnxt_reps.c             |  3 +-
>>  drivers/net/enic/enic_vf_representor.c   |  3 +-
>>  drivers/net/i40e/i40e_vf_representor.c   |  3 +-
>>  drivers/net/ixgbe/ixgbe_vf_representor.c |  3 +-
>>  drivers/net/mlx5/linux/mlx5_os.c         |  4 +-
>>  lib/librte_ethdev/ethdev_private.c       |  5 ++-
>>  lib/librte_ethdev/rte_class_eth.c        | 38 +++++++++++++----
>>  lib/librte_ethdev/rte_ethdev.c           | 26 ++++++++++++
>>  lib/librte_ethdev/rte_ethdev_driver.h    | 53 ++++++++++++++++++++++++
>>  lib/librte_ethdev/version.map            |  2 +
>>  10 files changed, 126 insertions(+), 14 deletions(-)
>>
>> diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
>> index f7bbf77d3f..34febc2d1e 100644
>> --- a/drivers/net/bnxt/bnxt_reps.c
>> +++ b/drivers/net/bnxt/bnxt_reps.c
>> @@ -186,7 +186,8 @@ int bnxt_representor_init(struct rte_eth_dev
>*eth_dev, void *params)
>>
>>         eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
>>                                         RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
>> -       eth_dev->data->representor_id = rep_params->vf_id;
>> +       eth_dev->data->representor_id = rte_eth_representor_id_encode(
>> +               0, 0, RTE_ETH_REPRESENTOR_VF, rep_params->vf_id);
>>
>>         rte_eth_random_addr(vf_rep_bp->dflt_mac_addr);
>>         memcpy(vf_rep_bp->mac_addr, vf_rep_bp->dflt_mac_addr,
>> diff --git a/drivers/net/enic/enic_vf_representor.c
>b/drivers/net/enic/enic_vf_representor.c
>> index c2c03c0281..632317af15 100644
>> --- a/drivers/net/enic/enic_vf_representor.c
>> +++ b/drivers/net/enic/enic_vf_representor.c
>> @@ -674,7 +674,8 @@ int enic_vf_representor_init(struct rte_eth_dev
>*eth_dev, void *init_params)
>>         eth_dev->dev_ops = &enic_vf_representor_dev_ops;
>>         eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
>>                                         RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
>> -       eth_dev->data->representor_id = vf->vf_id;
>> +       eth_dev->data->representor_id = rte_eth_representor_id_encode(
>> +               0, 0, RTE_ETH_REPRESENTOR_VF, vf->vf_id);
>>         eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
>>                 sizeof(struct rte_ether_addr) *
>>                 ENIC_UNICAST_PERFECT_FILTERS, 0);
>> diff --git a/drivers/net/i40e/i40e_vf_representor.c
>b/drivers/net/i40e/i40e_vf_representor.c
>> index 9e40406a3d..d90d0fdb9d 100644
>> --- a/drivers/net/i40e/i40e_vf_representor.c
>> +++ b/drivers/net/i40e/i40e_vf_representor.c
>> @@ -510,7 +510,8 @@ i40e_vf_representor_init(struct rte_eth_dev *ethdev,
>void *init_params)
>>
>>         ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
>>                                         RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
>> -       ethdev->data->representor_id = representor->vf_id;
>> +       ethdev->data->representor_id = rte_eth_representor_id_encode(
>> +                       0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
>>
>>         /* Setting the number queues allocated to the VF */
>>         ethdev->data->nb_rx_queues = vf->vsi->nb_qps;
>> diff --git a/drivers/net/ixgbe/ixgbe_vf_representor.c
>b/drivers/net/ixgbe/ixgbe_vf_representor.c
>> index 8185f0d3bb..e15b794761 100644
>> --- a/drivers/net/ixgbe/ixgbe_vf_representor.c
>> +++ b/drivers/net/ixgbe/ixgbe_vf_representor.c
>> @@ -196,7 +196,8 @@ ixgbe_vf_representor_init(struct rte_eth_dev
>*ethdev, void *init_params)
>>                 return -ENODEV;
>>
>>         ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
>> -       ethdev->data->representor_id = representor->vf_id;
>> +       ethdev->data->representor_id = rte_eth_representor_id_encode(
>> +                       0, 0, RTE_ETH_REPRESENTOR_VF, representor->vf_id);
>>
>>         /* Set representor device ops */
>>         ethdev->dev_ops = &ixgbe_vf_representor_dev_ops;
>> diff --git a/drivers/net/mlx5/linux/mlx5_os.c
>b/drivers/net/mlx5/linux/mlx5_os.c
>> index caead107b0..4d7940bcca 100644
>> --- a/drivers/net/mlx5/linux/mlx5_os.c
>> +++ b/drivers/net/mlx5/linux/mlx5_os.c
>> @@ -1025,7 +1025,9 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
>>  #endif
>>         /* representor_id field keeps the unmodified VF index. */
>>         priv->representor_id = switch_info->representor ?
>> -                              switch_info->port_name : -1;
>> +               rte_eth_representor_id_encode(0, 0, RTE_ETH_REPRESENTOR_VF,
>> +                                             switch_info->port_name) :
>> +               -1;
>>         /*
>>          * Look for sibling devices in order to reuse their switch domain
>>          * if any, otherwise allocate one.
>> diff --git a/lib/librte_ethdev/ethdev_private.c
>b/lib/librte_ethdev/ethdev_private.c
>> index 57473b5a39..0b3b4aa871 100644
>> --- a/lib/librte_ethdev/ethdev_private.c
>> +++ b/lib/librte_ethdev/ethdev_private.c
>> @@ -106,10 +106,13 @@ rte_eth_devargs_process_list(char *str, uint16_t
>*list, uint16_t *len_list,
>>  }
>>
>>  /*
>> - * representor format:
>> + * Parse representor ports, expand and update representor port ID.
>> + * Representor format:
>>   *   #: range or single number of VF representor - legacy
>>   *   [[c#]pf#]vf#: VF port representor/s
>>   *   [[c#]pf#]sf#: SF port representor/s
>> + *
>> + * See RTE_ETH_REPR() for representor ID format.
>>   */
>>  int
>>  rte_eth_devargs_parse_representor_ports(char *str, void *data)
>> diff --git a/lib/librte_ethdev/rte_class_eth.c
>b/lib/librte_ethdev/rte_class_eth.c
>> index efe6149df5..994db96960 100644
>> --- a/lib/librte_ethdev/rte_class_eth.c
>> +++ b/lib/librte_ethdev/rte_class_eth.c
>> @@ -66,8 +66,8 @@ eth_representor_cmp(const char *key __rte_unused,
>>         int ret;
>>         char *values;
>>         const struct rte_eth_dev_data *data = opaque;
>> -       struct rte_eth_devargs representors;
>> -       uint16_t index;
>> +       struct rte_eth_devargs eth_da;
>> +       uint16_t index, c, p, f;
>>
>>         if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
>>                 return -1; /* not a representor port */
>> @@ -76,17 +76,39 @@ eth_representor_cmp(const char *key __rte_unused,
>>         values = strdup(value);
>>         if (values == NULL)
>>                 return -1;
>> -       memset(&representors, 0, sizeof(representors));
>> -       ret = rte_eth_devargs_parse_representor_ports(values, &representors);
>> +       memset(&eth_da, 0, sizeof(eth_da));
>> +       ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
>>         free(values);
>>         if (ret != 0)
>>                 return -1; /* invalid devargs value */
>>
>> +       /* Set default values. */
>> +       if (eth_da.nb_mh_controllers == 0) {
>> +               eth_da.nb_mh_controllers = 1;
>> +               eth_da.mh_controllers[0] = 0;
>> +       }
>> +       if (eth_da.nb_ports == 0) {
>> +               eth_da.nb_ports = 1;
>> +               eth_da.ports[0] = 0;
>> +       }
>> +       if (eth_da.nb_representor_ports == 0) {
>> +               eth_da.nb_representor_ports = 1;
>> +               eth_da.representor_ports[0] = 0;
>> +       }
>>         /* Return 0 if representor id is matching one of the values. */
>> -       for (index = 0; index < representors.nb_representor_ports; index++)
>> -               if (data->representor_id ==
>> -                               representors.representor_ports[index])
>> -                       return 0;
>> +       for (c = 0; c < eth_da.nb_mh_controllers; ++c) {
>> +               for (p = 0; p < eth_da.nb_ports; ++p) {
>> +                       for (f = 0; f < eth_da.nb_representor_ports; ++f) {
>> +                               index = rte_eth_representor_id_encode(
>> +                                       eth_da.mh_controllers[c],
>> +                                       eth_da.ports[p],
>> +                                       eth_da.type,
>> +                                       eth_da.representor_ports[f]);
>> +                               if (data->representor_id == index)
>> +                                       return 0;
>> +                       }
>> +               }
>> +       }
>>         return -1; /* no match */
>>  }
>>
>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>> index 2ac51ac149..276f42e54b 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -5556,6 +5556,32 @@ rte_eth_devargs_parse(const char *dargs, struct
>rte_eth_devargs *eth_da)
>>         return result;
>>  }
>>
>> +uint16_t
>> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
>> +                             enum rte_eth_representor_type type,
>> +                             uint16_t representor_port)
>> +{
>> +       return (((controller & 3) << 14) |
>> +               ((pf & 3) << 12) |
>> +               (!!(type == RTE_ETH_REPRESENTOR_SF) << 11) |
>> +               (representor_port & 0x7ff));
>> +}
>> +
>> +uint16_t
>> +rte_eth_representor_id_parse(const uint16_t representor_id,
>> +                            uint16_t *controller, uint16_t *pf,
>> +                            enum rte_eth_representor_type *type)
>> +{
>> +       if (controller)
>> +               *controller = (representor_id >> 14) & 3;
>> +       if (pf)
>> +               *pf = (representor_id >> 12) & 3;
>> +       if (type)
>> +               *type = ((representor_id >> 11) & 1) ?
>> +                       RTE_ETH_REPRESENTOR_SF : RTE_ETH_REPRESENTOR_VF;
>> +       return representor_id & 0x7ff;
>> +}
>> +
>>  static int
>>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>>                 const char *params __rte_unused,
>> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
>b/lib/librte_ethdev/rte_ethdev_driver.h
>> index 8e04634660..0d8893693e 100644
>> --- a/lib/librte_ethdev/rte_ethdev_driver.h
>> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
>> @@ -1218,6 +1218,59 @@ struct rte_eth_devargs {
>>         enum rte_eth_representor_type type; /* type of representor */
>>  };
>>
>> +#define RTE_NO_REPRESENTOR_ID UINT16_MAX /**< No representor ID.
>*/
>> +
>> +/**
>> + * PMD helper function to encode representor ID
>> + *
>> + * The compact format is used for device iterator that comparing
>> + * ethdev representor ID with target devargs.
>> + *
>> + * xxxx xxxx xxxx xxxx
>> + * |||| |LLL LLLL LLLL vf/sf id
>> + * |||| L 1:sf, 0:vf
>> + * ||LL pf id
>> + * LL controller(host) id
>> + *
>> + * @param controller
>> + *  Controller ID.
>> + * @param pf
>> + *  PF port ID.
>> + * @param type
>> + *  Representor type.
>> + * @param representor_port
>> + *  Representor port ID.
>> + *
>> + * @return
>> + *   Encoded representor ID.
>> + */
>> +__rte_internal
>> +uint16_t
>> +rte_eth_representor_id_encode(uint16_t controller, uint16_t pf,
>> +                             enum rte_eth_representor_type type,
>> +                             uint16_t representor_port);
>> +
>> +/**
>> + * PMD helper function to parse representor ID
>> + *
>> + * @param representor_id
>> + *  Representor ID.
>> + * @param controller
>> + *  Parsed controller ID.
>> + * @param pf
>> + *  Parsed PF port ID.
>> + * @param type
>> + *  Parsed representor type.
>> + *
>> + * @return
>> + *   Parsed representor port ID.
>> + */
>> +__rte_internal
>> +uint16_t
>> +rte_eth_representor_id_parse(const uint16_t representor_id,
>> +                            uint16_t *controller, uint16_t *pf,
>> +                            enum rte_eth_representor_type *type);
>> +
>>  /**
>>   * PMD helper function to parse ethdev arguments
>>   *
>> diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
>> index d3f5410806..44edaed507 100644
>> --- a/lib/librte_ethdev/version.map
>> +++ b/lib/librte_ethdev/version.map
>> @@ -257,6 +257,8 @@ INTERNAL {
>>         rte_eth_dev_release_port;
>>         rte_eth_dev_internal_reset;
>>         rte_eth_devargs_parse;
>> +       rte_eth_representor_id_encode;
>> +       rte_eth_representor_id_parse;
>>         rte_eth_dma_zone_free;
>>         rte_eth_dma_zone_reserve;
>>         rte_eth_hairpin_queue_peer_bind;
>> --
>> 2.25.1
>>

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Tu, Lijuan]

> dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
> offload similar to existing PF and VF representors. A SF shares PCI level
> resources with other SFs and/or with its parent PCI function.
> 
> From SmartNIC perspective, when PCI device is shared for multi-host,
> representors for host controller and host PF is required.
> 
> This patch set introduces new representor types in addtion to existing VF
> representor. Syntax:
> 
> [[c#]pf#]vf#: VF port representor/s from controller/pf
> [[c#]pf#]sf#: SF port representor/s from controller/pf
> #: VF representor - for backwards compatibility
> 
> "#" is number instance, list or range, valid examples:
>   1, [1,3,5], [0-3], [0,2-4,6]
> 
> For backward compatibility, this patch also introduces new netdev capability
> to indicate the capability of supportting SF representor.
> 
> Version history:
>  RFC:
>  	initial version [2]
>  V2:
>     - separate patch for represnetor infrastructure, controller, pf and
>       sf.
>     - replace representor ID macro with functions:
>       rte_eth_representor_id_encode()
>       rte_eth_representor_id_parse()
>     - new patch to allow devargs with same PCI BDF but different
>       representors.
>     - other minor code updates according to comments, thanks Andrew!
>     - update document
>  V3:
>     - improve probing of allowed devargs with same name.
>     - parse single word of kvargs as key.
>     - update kvargs test cases.
>  V4:
>     - split first representor refactor patch into
>       1: add representor type
>       2: refector representor list parsing
>     - push the patch supporting multi-devargs for same device.
>  V5:
>     - add comments for parsing functions
>     - update switch_representation.rst - Thanks Ajit

Hi,

The series can't recognize i40e device, so testpmd start failed.

	./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 4-8 -n 6 -a 17:00.0 -- -i --txq=2 --rxq=2
	<...>
	EAL: Probe PCI driver: net_i40e (8086:158b) device: 0000:17:00.0 (socket 0)
	EAL: No legacy callbacks, legacy socket not created
	testpmd: No probed ethernet devices
	Interactive-mode selected
	Fail: input txq (2) can't be greater than max_tx_queues (0) of port 0
	EAL: Error - exiting with code: 1
	  Cause: txq 2 invalid - must be >= 0 && <= 0

thanks

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Andrew Rybchenko]

On 1/19/21 5:24 PM, Xueming(Steven) Li wrote:
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Tuesday, January 19, 2021 4:41 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
>> <asafp@nvidia.com>
>> Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction
>> representor
>>
>> On 1/19/21 10:13 AM, Xueming Li wrote:
>>> dedicated queues(txq, rxq). A SF netdev supports E-Switch
>>> representation offload similar to existing PF and VF representors. A
>>> SF shares PCI level resources with other SFs and/or with its parent PCI
>> function.
>>>
>>> >From SmartNIC perspective, when PCI device is shared for multi-host,
>>> representors for host controller and host PF is required.
>>>
>>> This patch set introduces new representor types in addtion to existing
>>> VF representor. Syntax:
>>>
>>> [[c#]pf#]vf#: VF port representor/s from controller/pf
>>> [[c#]pf#]sf#: SF port representor/s from controller/pf
>>> #: VF representor - for backwards compatibility
>>>
>>> "#" is number instance, list or range, valid examples:
>>>   1, [1,3,5], [0-3], [0,2-4,6]
>>>
>>> For backward compatibility, this patch also introduces new netdev
>>> capability to indicate the capability of supportting SF representor.
>>
>> The patch series looks really nice. Thanks.
>>
>> As before, my biggest concern is making representor ID a bitmap. See my
>> comments to a specific patch.
>> Plus absence of defined semantics of caller function.
>> Basically it is looks like it is assumed that controller #0 and PF #0 is the caller.
>> However, it could be wrong.
> 
> From devargs syntax perspective, a VF representor on PF1 could be referenced either way:
>   1:  <PF0_BDF>,representor=pf1vfX
>   2:  <PF1_BDF>,representor=vfX
>   3:  <PF1_BDF>,representor=pf0vfX // works but not suggested, use PF0 BDF as caller
> If probe a device with PF0 BDF and locate it with PF1 BDF, the device iterator will fail,
> devargs is parsed by api, EAL layer simply compare PCI BDF and then representor ID, 
> i.e. "c#pf#vf#". The caller device should be consistent, better to use the first device.
> 
> Devrg "<PF1_BDF>,representor=vfX" will work, representor controller ID and pf ID
> default to #0, relative to caller context PF1 BDF.
> 
> Is it good to add such suggestion/behavior on rte_eth_devargs comments?

Sorry, but I disagree. Solution with vf1 and pf0vf1 making the
same representor ID sounds not acceptable to me. It is
definitely different things if I pass it to PF1.


>>
>> The next biggest concert is the absence of capability reporting API. How many
>> controller are available?
>> How many PFs on each controller are available?
>> How many VFs on each controller/PF are available?
>> How many SFs?
>>
>>From the first sight it sounds not that important right now and an extra
>> feature which could be added in the follow up patches, but IMHO addition of
>> the API would allow to avoid making representor ID a bitmap.
>> Basically capabilities API can provide an array of available functions with
>> representor ID assigned to each entry. Also it could make the entire patch
>> series optional since it would allow to interpret numbers in representor=[....]
>> as representor IDs which are mapped to controller/PF/VF/SF by the
>> capabilities reporting API.
>>
>> I realize that sometimes it could be more convenient to use syntax suggested
>> here. Mainly for human.
> 
> Agree, mostly for human. Regarding to capability reporting API, how about remove
> current one and enhance it later with a complete patch set?
> 

Current one? Sorry I don't understand.

Andrew.

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Xueming(Steven) Li]

Hi Andrew,

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Friday, January 22, 2021 4:21 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
><asafp@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction
>representor
>
>On 1/19/21 5:24 PM, Xueming(Steven) Li wrote:
>>
>>> -----Original Message-----
>>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>> Sent: Tuesday, January 19, 2021 4:41 PM
>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>> Penso <asafp@nvidia.com>
>>> Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction
>>> representor
>>>
>>> On 1/19/21 10:13 AM, Xueming Li wrote:
>>>> dedicated queues(txq, rxq). A SF netdev supports E-Switch
>>>> representation offload similar to existing PF and VF representors. A
>>>> SF shares PCI level resources with other SFs and/or with its parent
>>>> PCI
>>> function.
>>>>
>>>> >From SmartNIC perspective, when PCI device is shared for
>>>> >multi-host,
>>>> representors for host controller and host PF is required.
>>>>
>>>> This patch set introduces new representor types in addtion to
>>>> existing VF representor. Syntax:
>>>>
>>>> [[c#]pf#]vf#: VF port representor/s from controller/pf
>>>> [[c#]pf#]sf#: SF port representor/s from controller/pf
>>>> #: VF representor - for backwards compatibility
>>>>
>>>> "#" is number instance, list or range, valid examples:
>>>>   1, [1,3,5], [0-3], [0,2-4,6]
>>>>
>>>> For backward compatibility, this patch also introduces new netdev
>>>> capability to indicate the capability of supportting SF representor.
>>>
>>> The patch series looks really nice. Thanks.
>>>
>>> As before, my biggest concern is making representor ID a bitmap. See
>>> my comments to a specific patch.
>>> Plus absence of defined semantics of caller function.
>>> Basically it is looks like it is assumed that controller #0 and PF #0 is the
>caller.
>>> However, it could be wrong.
>>
>> From devargs syntax perspective, a VF representor on PF1 could be
>referenced either way:
>>   1:  <PF0_BDF>,representor=pf1vfX
>>   2:  <PF1_BDF>,representor=vfX
>>   3:  <PF1_BDF>,representor=pf0vfX // works but not suggested, use PF0
>> BDF as caller If probe a device with PF0 BDF and locate it with PF1
>> BDF, the device iterator will fail, devargs is parsed by api, EAL
>> layer simply compare PCI BDF and then representor ID, i.e. "c#pf#vf#". The
>caller device should be consistent, better to use the first device.
>>
>> Devrg "<PF1_BDF>,representor=vfX" will work, representor controller ID
>> and pf ID default to #0, relative to caller context PF1 BDF.
>>
>> Is it good to add such suggestion/behavior on rte_eth_devargs comments?
>
>Sorry, but I disagree. Solution with vf1 and pf0vf1 making the same
>representor ID sounds not acceptable to me. It is definitely different things if I
>pass it to PF1.

My understanding is that representor is an "offset" to caller(context) PF, so the
caller PF has to be the lowest, i.e. PF0. The usage for mlx5 PMD is to probe VF 
representor on a kernel bonding:
1. PF0 is the only device for PMD to probe the bonding device, PMD detects underlay PFs.
2. PF0_BDF,reprensetor=pf1vf0, to probe the representor for first VF on PF1.
PF1_BDF,reprenstor=pf0vf0 works for the same representor, but not encouraged because 
it confuses EAL device iterator which can't tell the difference, as you said: "mainly for human"

Controller ID is not used by mlx5 PMD currently, just a place holder for now, but I think 
same policy applies: lowest as caller(context).

Your suggestion of reserving an c# and pf# ID for caller device which c# or pf# not specified looks
good, it makes the usage of the representor ID bitmap flexible, but considering some NIC with 4 PFs,
it's hard to choose, as you know, only 3 controller and 3 PFs left.

BTW, I guess your assumption is that  representor as "absolute" to caller PF, can't default to 0.
Is there a scenario for PF1 as caller?

>
>
>>>
>>> The next biggest concert is the absence of capability reporting API.
>>> How many controller are available?
>>> How many PFs on each controller are available?
>>> How many VFs on each controller/PF are available?
>>> How many SFs?
>>>
>>>From the first sight it sounds not that important right now and an
>>>extra  feature which could be added in the follow up patches, but IMHO
>>>addition of  the API would allow to avoid making representor ID a bitmap.
>>> Basically capabilities API can provide an array of available
>>>functions with  representor ID assigned to each entry. Also it could
>>>make the entire patch  series optional since it would allow to
>>>interpret numbers in representor=[....]  as representor IDs which are
>>>mapped to controller/PF/VF/SF by the  capabilities reporting API.
>>>
>>> I realize that sometimes it could be more convenient to use syntax
>>> suggested here. Mainly for human.
>>
>> Agree, mostly for human. Regarding to capability reporting API, how
>> about remove current one and enhance it later with a complete patch set?
>>
>
>Current one? Sorry I don't understand.

The patch of device SF capability, but seems I misunderstood your suggestion.
Let me explain process to create a SF:
1. SF can be created on the fly with scripts, unlike VF which is statically pre-created.
2. SF is created on a PF with a SF number. SF number is named per PF, different PF may have same SF number.
3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no need to use pf#sf# here.
4. For bonding netdev, hot plug to DPDK using "PF0_BDF,representor=pf#sf#"
If using new api to return all representor IDs, need some way locate the new created SF by PF and SF number,
that's why "pf#sf#" is used in this patch set.

In the future, I think representor could be processed by PMD, so PMD could have enough flexibility
to support more device expressions and types. But that will introduce a fundamental change of devargs and 
device management, need a full plan.

>
>Andrew.

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Andrew Rybchenko]

On 1/27/21 6:04 AM, Xueming(Steven) Li wrote:
> Hi Andrew,
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Friday, January 22, 2021 4:21 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
>> <asafp@nvidia.com>
>> Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction
>> representor
>>
>> On 1/19/21 5:24 PM, Xueming(Steven) Li wrote:
>>>
>>>> -----Original Message-----
>>>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>>> Sent: Tuesday, January 19, 2021 4:41 PM
>>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>>> Penso <asafp@nvidia.com>
>>>> Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction
>>>> representor
>>>>
>>>> On 1/19/21 10:13 AM, Xueming Li wrote:
>>>>> dedicated queues(txq, rxq). A SF netdev supports E-Switch
>>>>> representation offload similar to existing PF and VF representors. A
>>>>> SF shares PCI level resources with other SFs and/or with its parent
>>>>> PCI
>>>> function.
>>>>>
>>>>> >From SmartNIC perspective, when PCI device is shared for
>>>>>> multi-host,
>>>>> representors for host controller and host PF is required.
>>>>>
>>>>> This patch set introduces new representor types in addtion to
>>>>> existing VF representor. Syntax:
>>>>>
>>>>> [[c#]pf#]vf#: VF port representor/s from controller/pf
>>>>> [[c#]pf#]sf#: SF port representor/s from controller/pf
>>>>> #: VF representor - for backwards compatibility
>>>>>
>>>>> "#" is number instance, list or range, valid examples:
>>>>>   1, [1,3,5], [0-3], [0,2-4,6]
>>>>>
>>>>> For backward compatibility, this patch also introduces new netdev
>>>>> capability to indicate the capability of supportting SF representor.
>>>>
>>>> The patch series looks really nice. Thanks.
>>>>
>>>> As before, my biggest concern is making representor ID a bitmap. See
>>>> my comments to a specific patch.
>>>> Plus absence of defined semantics of caller function.
>>>> Basically it is looks like it is assumed that controller #0 and PF #0 is the
>> caller.
>>>> However, it could be wrong.
>>>
>>> From devargs syntax perspective, a VF representor on PF1 could be
>> referenced either way:
>>>   1:  <PF0_BDF>,representor=pf1vfX
>>>   2:  <PF1_BDF>,representor=vfX
>>>   3:  <PF1_BDF>,representor=pf0vfX // works but not suggested, use PF0
>>> BDF as caller If probe a device with PF0 BDF and locate it with PF1
>>> BDF, the device iterator will fail, devargs is parsed by api, EAL
>>> layer simply compare PCI BDF and then representor ID, i.e. "c#pf#vf#". The
>> caller device should be consistent, better to use the first device.
>>>
>>> Devrg "<PF1_BDF>,representor=vfX" will work, representor controller ID
>>> and pf ID default to #0, relative to caller context PF1 BDF.
>>>
>>> Is it good to add such suggestion/behavior on rte_eth_devargs comments?
>>
>> Sorry, but I disagree. Solution with vf1 and pf0vf1 making the same
>> representor ID sounds not acceptable to me. It is definitely different things if I
>> pass it to PF1.
> 
> My understanding is that representor is an "offset" to caller(context) PF, so the
> caller PF has to be the lowest, i.e. PF0.

Sorry, how to address PF0 from PF1?

> The usage for mlx5 PMD is to probe VF 
> representor on a kernel bonding:
> 1. PF0 is the only device for PMD to probe the bonding device, PMD detects underlay PFs.
> 2. PF0_BDF,reprensetor=pf1vf0, to probe the representor for first VF on PF1.
> PF1_BDF,reprenstor=pf0vf0 works for the same representor, but not encouraged because 
> it confuses EAL device iterator which can't tell the difference, as you said: "mainly for human"
> 
> Controller ID is not used by mlx5 PMD currently, just a place holder for now, but I think 
> same policy applies: lowest as caller(context).
> 
> Your suggestion of reserving an c# and pf# ID for caller device which c# or pf# not specified looks
> good, it makes the usage of the representor ID bitmap flexible, but considering some NIC with 4 PFs,
> it's hard to choose, as you know, only 3 controller and 3 PFs left.

I'd prefer the option if we make a choice between these two.
At least it a bit more consistent.
But IMHO anyway bitmap is a road to wrong direction.

> BTW, I guess your assumption is that  representor as "absolute" to caller PF, can't default to 0.
> Is there a scenario for PF1 as caller?

Yes.

>>>>
>>>> The next biggest concert is the absence of capability reporting API.
>>>> How many controller are available?
>>>> How many PFs on each controller are available?
>>>> How many VFs on each controller/PF are available?
>>>> How many SFs?
>>>>
>>> >From the first sight it sounds not that important right now and an
>>>> extra  feature which could be added in the follow up patches, but IMHO
>>>> addition of  the API would allow to avoid making representor ID a bitmap.
>>>> Basically capabilities API can provide an array of available
>>>> functions with  representor ID assigned to each entry. Also it could
>>>> make the entire patch  series optional since it would allow to
>>>> interpret numbers in representor=[....]  as representor IDs which are
>>>> mapped to controller/PF/VF/SF by the  capabilities reporting API.
>>>>
>>>> I realize that sometimes it could be more convenient to use syntax
>>>> suggested here. Mainly for human.
>>>
>>> Agree, mostly for human. Regarding to capability reporting API, how
>>> about remove current one and enhance it later with a complete patch set?
>>>
>>
>> Current one? Sorry I don't understand.
> 
> The patch of device SF capability, but seems I misunderstood your suggestion.
> Let me explain process to create a SF:
> 1. SF can be created on the fly with scripts, unlike VF which is statically pre-created.

Is there a maximum index and maximum total number of SF's
created? How to find it?

> 2. SF is created on a PF with a SF number. SF number is named per PF, different PF may have same SF number.
> 3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no need to use pf#sf# here.
> 4. For bonding netdev, hot plug to DPDK using "PF0_BDF,representor=pf#sf#"
> If using new api to return all representor IDs, need some way locate the new created SF by PF and SF number,
> that's why "pf#sf#" is used in this patch set.

I think the API should simply reserve/report space for maximum
number of SFs. So, IDs are stable across restart/reboot in
assumption that NIC is not reconfigured (changed maximum number
of VF or maximum number of SFs of any PF).

> 
> In the future, I think representor could be processed by PMD, so PMD could have enough flexibility
> to support more device expressions and types. But that will introduce a fundamental change of devargs and 
> device management, need a full plan.
> 
>>
>> Andrew.
> 

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Ajit Khaparde]

[-- Attachment #1: Type: text/plain, Size: 1150 bytes --]

On Tue, Jan 26, 2021 at 7:05 PM Xueming(Steven) Li <xuemingl@nvidia.com> wrote:
>
::::[snip]::::
> The patch of device SF capability, but seems I misunderstood your suggestion.
> Let me explain process to create a SF:
> 1. SF can be created on the fly with scripts, unlike VF which is statically pre-created.
> 2. SF is created on a PF with a SF number. SF number is named per PF, different PF may have same SF number.
> 3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no need to use pf#sf# here.
> 4. For bonding netdev, hot plug to DPDK using "PF0_BDF,representor=pf#sf#"
> If using new api to return all representor IDs, need some way locate the new created SF by PF and SF number,
> that's why "pf#sf#" is used in this patch set.
>
> In the future, I think representor could be processed by PMD, so PMD could have enough flexibility
> to support more device expressions and types. But that will introduce a fundamental change of devargs and
> device management, need a full plan.
Do you mean all changes will be contained within the PMD? The
fundamental changes will be in the PMD?
More types of what?

>
> >
> >Andrew.
>

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Xueming(Steven) Li]

Hi Ajit,

>-----Original Message-----
>From: Ajit Khaparde <ajit.khaparde@broadcom.com>
>Sent: Thursday, January 28, 2021 1:43 AM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>; dev@dpdk.org;
>Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction
>representor
>
>On Tue, Jan 26, 2021 at 7:05 PM Xueming(Steven) Li <xuemingl@nvidia.com>
>wrote:
>>
>::::[snip]::::
>> The patch of device SF capability, but seems I misunderstood your
>suggestion.
>> Let me explain process to create a SF:
>> 1. SF can be created on the fly with scripts, unlike VF which is statically pre-
>created.
>> 2. SF is created on a PF with a SF number. SF number is named per PF,
>different PF may have same SF number.
>> 3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no
>need to use pf#sf# here.
>> 4. For bonding netdev, hot plug to DPDK using
>"PF0_BDF,representor=pf#sf#"
>> If using new api to return all representor IDs, need some way locate the new
>created SF by PF and SF number,
>> that's why "pf#sf#" is used in this patch set.
>>
>> In the future, I think representor could be processed by PMD, so PMD could
>have enough flexibility
>> to support more device expressions and types. But that will introduce a
>fundamental change of devargs and
>> device management, need a full plan.
>Do you mean all changes will be contained within the PMD? The
>fundamental changes will be in the PMD?

Yes, PMD detect device type and IDs from devargs and figure out a unique name for device.

>More types of what?

SF, representor of SF, representor of VF...

>
>>
>> >
>> >Andrew.
>>

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Xueming(Steven) Li]

<snip>
>> The patch of device SF capability, but seems I misunderstood your suggestion.
>> Let me explain process to create a SF:
>> 1. SF can be created on the fly with scripts, unlike VF which is statically pre-created.
>
>Is there a maximum index and maximum total number of SF's created? How to find it?

The maximum index is defined by firmware configuration, all SF's information could be found
from sysfs. To create a SF, both PCI and sfnum have to be specified.

>
>> 2. SF is created on a PF with a SF number. SF number is named per PF, different PF may have same SF number.
>> 3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no need to use pf#sf# here.
>> 4. For bonding netdev, hot plug to DPDK using "PF0_BDF,representor=pf#sf#"
>> If using new api to return all representor IDs, need some way locate
>> the new created SF by PF and SF number, that's why "pf#sf#" is used in this patch set.
>
>I think the API should simply reserve/report space for maximum number of SFs. So, IDs are stable across restart/reboot in assumption
>that NIC is not reconfigured (changed maximum number of VF or maximum number of SFs of any PF).

Yes, IDs should be stable as long as no  NIC firmware configuration change.

Just clarify, this api should be common enough to report all devices that a bus device supports:
1. name, might contains controller and pf info, example: "eth:representor:c0pf1vf"
2. ID range, example: 0-127
The api describes ID ranges for each sub device type, users have to query the api and choose representor ID to probe.

Prototype:
struct rte_bus_device_range {
	char name[64];
	uint32_t base;
	uint32_t number;
}
/* return number of ranges filled, or number of ranges if list is NULL. */
int rte_bus_ dev_range_get(struct rte_bus_device_range *list, int n);


>
<snip>

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Andrew Rybchenko]

On 1/28/21 5:31 PM, Xueming(Steven) Li wrote:
> <snip>
>>> The patch of device SF capability, but seems I misunderstood your suggestion.
>>> Let me explain process to create a SF:
>>> 1. SF can be created on the fly with scripts, unlike VF which is statically pre-created.
>>
>> Is there a maximum index and maximum total number of SF's created? How to find it?
> 
> The maximum index is defined by firmware configuration, all SF's information could be found
> from sysfs. To create a SF, both PCI and sfnum have to be specified.

sysfs is obviously Linux specific. I think the information
should be available via DPDK API.

>>
>>> 2. SF is created on a PF with a SF number. SF number is named per PF, different PF may have same SF number.
>>> 3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no need to use pf#sf# here.
>>> 4. For bonding netdev, hot plug to DPDK using "PF0_BDF,representor=pf#sf#"
>>> If using new api to return all representor IDs, need some way locate
>>> the new created SF by PF and SF number, that's why "pf#sf#" is used in this patch set.
>>
>> I think the API should simply reserve/report space for maximum number of SFs. So, IDs are stable across restart/reboot in assumption
>> that NIC is not reconfigured (changed maximum number of VF or maximum number of SFs of any PF).
> 
> Yes, IDs should be stable as long as no  NIC firmware configuration change.
> 
> Just clarify, this api should be common enough to report all devices that a bus device supports:
> 1. name, might contains controller and pf info, example: "eth:representor:c0pf1vf"
> 2. ID range, example: 0-127
> The api describes ID ranges for each sub device type, users have to query the api and choose representor ID to probe.
> 
> Prototype:
> struct rte_bus_device_range {
> 	char name[64];
> 	uint32_t base;
> 	uint32_t number;
> }
> /* return number of ranges filled, or number of ranges if list is NULL. */
> int rte_bus_ dev_range_get(struct rte_bus_device_range *list, int n);

Hm, I thought about more port representor specific API.
For me it is hard to tell if such generic naming is good or
bad. I think it should be proven that such generic API
makes sense. Any other potential users / use cases?

I've considered ethdev API which returns (in similar way as
above) list of possible port representors which could be
controlled by the device. Also I think it would be useful
to include type information (enum with PF, VF, SF),
controller ID.

There is one more bit which is not in the picture yet -
switch_info.port_id. Should it be equal to representor
ID? Or different and provided in the info structure?

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Monday, February 1, 2021 4:39 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor
>
>On 1/28/21 5:31 PM, Xueming(Steven) Li wrote:
>> <snip>
>>>> The patch of device SF capability, but seems I misunderstood your suggestion.
>>>> Let me explain process to create a SF:
>>>> 1. SF can be created on the fly with scripts, unlike VF which is statically pre-created.
>>>
>>> Is there a maximum index and maximum total number of SF's created? How to find it?
>>
>> The maximum index is defined by firmware configuration, all SF's
>> information could be found from sysfs. To create a SF, both PCI and sfnum have to be specified.
>
>sysfs is obviously Linux specific. I think the information should be available via DPDK API.

Yes, the new api discussed below should resolve this issue.

>
>>>
>>>> 2. SF is created on a PF with a SF number. SF number is named per PF, different PF may have same SF number.
>>>> 3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no need to use pf#sf# here.
>>>> 4. For bonding netdev, hot plug to DPDK using "PF0_BDF,representor=pf#sf#"
>>>> If using new api to return all representor IDs, need some way locate
>>>> the new created SF by PF and SF number, that's why "pf#sf#" is used in this patch set.
>>>
>>> I think the API should simply reserve/report space for maximum number
>>> of SFs. So, IDs are stable across restart/reboot in assumption that NIC is not reconfigured (changed maximum number of VF or
>maximum number of SFs of any PF).
>>
>> Yes, IDs should be stable as long as no  NIC firmware configuration change.
>>
>> Just clarify, this api should be common enough to report all devices that a bus device supports:
>> 1. name, might contains controller and pf info, example: "eth:representor:c0pf1vf"
>> 2. ID range, example: 0-127
>> The api describes ID ranges for each sub device type, users have to query the api and choose representor ID to probe.
>>
>> Prototype:
>> struct rte_bus_device_range {
>> 	char name[64];
>> 	uint32_t base;
>> 	uint32_t number;
>> }
>> /* return number of ranges filled, or number of ranges if list is
>> NULL. */ int rte_bus_ dev_range_get(struct rte_bus_device_range *list,
>> int n);
>
>Hm, I thought about more port representor specific API.
>For me it is hard to tell if such generic naming is good or bad. I think it should be proven that such generic API makes sense. Any other
>potential users / use cases?

I was thinking about SF, but SF is PCI specific, not suitable for this api. So I'm fine to make it as ethdev api.
To append new api into eth_dev_ops, is there ABI concern?

>
>I've considered ethdev API which returns (in similar way as
>above) list of possible port representors which could be controlled by the device. Also I think it would be useful to include type
>information (enum with PF, VF, SF), controller ID.

Agree. 

There is a new concern from orchestration side, currently, no interface in openstack and OVS to retrieve representor ID range info,
It will take time to adapt this solution. To probe a representor, orchestration need to know how to calculate representor ID, 
and the ID might vary on different max SF number, i.e. VF4 on PP1 might got different ID. Representor ID change before that will
break the product.

Considering both orchestration and testpmd users, how about keeping both solution together? This will bring max flexibility IMHO.

In struct rte_eth_dev_data, reserved bits could be used to define controller and port, this will avoid bitmap. How do you think?

>
>There is one more bit which is not in the picture yet - switch_info.port_id. Should it be equal to representor ID? Or different and
>provided in the info structure?

Not exactly same AFAIK, the id used in e-switch.

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Andrew Rybchenko]

On 2/4/21 5:15 PM, Xueming(Steven) Li wrote:
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Monday, February 1, 2021 4:39 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
>> Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor
>>
>> On 1/28/21 5:31 PM, Xueming(Steven) Li wrote:
>>> <snip>
>>>>> The patch of device SF capability, but seems I misunderstood your suggestion.
>>>>> Let me explain process to create a SF:
>>>>> 1. SF can be created on the fly with scripts, unlike VF which is statically pre-created.
>>>>
>>>> Is there a maximum index and maximum total number of SF's created? How to find it?
>>>
>>> The maximum index is defined by firmware configuration, all SF's
>>> information could be found from sysfs. To create a SF, both PCI and sfnum have to be specified.
>>
>> sysfs is obviously Linux specific. I think the information should be available via DPDK API.
> 
> Yes, the new api discussed below should resolve this issue.
> 
>>
>>>>
>>>>> 2. SF is created on a PF with a SF number. SF number is named per PF, different PF may have same SF number.
>>>>> 3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no need to use pf#sf# here.
>>>>> 4. For bonding netdev, hot plug to DPDK using "PF0_BDF,representor=pf#sf#"
>>>>> If using new api to return all representor IDs, need some way locate
>>>>> the new created SF by PF and SF number, that's why "pf#sf#" is used in this patch set.
>>>>
>>>> I think the API should simply reserve/report space for maximum number
>>>> of SFs. So, IDs are stable across restart/reboot in assumption that NIC is not reconfigured (changed maximum number of VF or
>> maximum number of SFs of any PF).
>>>
>>> Yes, IDs should be stable as long as no  NIC firmware configuration change.
>>>
>>> Just clarify, this api should be common enough to report all devices that a bus device supports:
>>> 1. name, might contains controller and pf info, example: "eth:representor:c0pf1vf"
>>> 2. ID range, example: 0-127
>>> The api describes ID ranges for each sub device type, users have to query the api and choose representor ID to probe.
>>>
>>> Prototype:
>>> struct rte_bus_device_range {
>>> 	char name[64];
>>> 	uint32_t base;
>>> 	uint32_t number;
>>> }
>>> /* return number of ranges filled, or number of ranges if list is
>>> NULL. */ int rte_bus_ dev_range_get(struct rte_bus_device_range *list,
>>> int n);
>>
>> Hm, I thought about more port representor specific API.
>> For me it is hard to tell if such generic naming is good or bad. I think it should be proven that such generic API makes sense. Any other
>> potential users / use cases?
> 
> I was thinking about SF, but SF is PCI specific, not suitable for this api. So I'm fine to make it as ethdev api.
> To append new api into eth_dev_ops, is there ABI concern?

No, eth_dev_ops are internal

>> I've considered ethdev API which returns (in similar way as
>> above) list of possible port representors which could be controlled by the device. Also I think it would be useful to include type
>> information (enum with PF, VF, SF), controller ID.
> 
> Agree. 
> 
> There is a new concern from orchestration side, currently, no interface in openstack and OVS to retrieve representor ID range info,
> It will take time to adapt this solution. To probe a representor, orchestration need to know how to calculate representor ID, 
> and the ID might vary on different max SF number, i.e. VF4 on PP1 might got different ID. Representor ID change before that will
> break the product.

I see.

> Considering both orchestration and testpmd users, how about keeping both solution together? This will bring max flexibility IMHO.

As I said before I don't mind and I really think it is a good
idea to add suggested interface to specify representor
(i.e. cXpfYvfZ), but the problem is making bitmap from
representor ID.

ethdev API should use new representor info API to
make a representor ID from controller/PF/{VF,SF}.
Or do you see any problems with such approach?

> In struct rte_eth_dev_data, reserved bits could be used to define controller and port, this will avoid bitmap. How do you think?

Could you add a bit more on it? Just a bit more details to
the idea since I don't understand what exactly you mean and
how it could help.

>>
>> There is one more bit which is not in the picture yet - switch_info.port_id. Should it be equal to representor ID? Or different and
>> provided in the info structure?
> 
> Not exactly same AFAIK, the id used in e-switch.
> 
> 

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Friday, February 5, 2021 3:35 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; Thomas Monjalon
><tmonjalon@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor
>
>On 2/4/21 5:15 PM, Xueming(Steven) Li wrote:
>>
>>> -----Original Message-----
>>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>> Sent: Monday, February 1, 2021 4:39 PM
>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>> Penso <asafp@nvidia.com>
>>> Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction
>>> representor
>>>
>>> On 1/28/21 5:31 PM, Xueming(Steven) Li wrote:
>>>> <snip>
>>>>>> The patch of device SF capability, but seems I misunderstood your suggestion.
>>>>>> Let me explain process to create a SF:
>>>>>> 1. SF can be created on the fly with scripts, unlike VF which is statically pre-created.
>>>>>
>>>>> Is there a maximum index and maximum total number of SF's created? How to find it?
>>>>
>>>> The maximum index is defined by firmware configuration, all SF's
>>>> information could be found from sysfs. To create a SF, both PCI and sfnum have to be specified.
>>>
>>> sysfs is obviously Linux specific. I think the information should be available via DPDK API.
>>
>> Yes, the new api discussed below should resolve this issue.
>>
>>>
>>>>>
>>>>>> 2. SF is created on a PF with a SF number. SF number is named per PF, different PF may have same SF number.
>>>>>> 3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no need to use pf#sf# here.
>>>>>> 4. For bonding netdev, hot plug to DPDK using "PF0_BDF,representor=pf#sf#"
>>>>>> If using new api to return all representor IDs, need some way
>>>>>> locate the new created SF by PF and SF number, that's why "pf#sf#" is used in this patch set.
>>>>>
>>>>> I think the API should simply reserve/report space for maximum
>>>>> number of SFs. So, IDs are stable across restart/reboot in
>>>>> assumption that NIC is not reconfigured (changed maximum number of
>>>>> VF or
>>> maximum number of SFs of any PF).
>>>>
>>>> Yes, IDs should be stable as long as no  NIC firmware configuration change.
>>>>
>>>> Just clarify, this api should be common enough to report all devices that a bus device supports:
>>>> 1. name, might contains controller and pf info, example: "eth:representor:c0pf1vf"
>>>> 2. ID range, example: 0-127
>>>> The api describes ID ranges for each sub device type, users have to query the api and choose representor ID to probe.
>>>>
>>>> Prototype:
>>>> struct rte_bus_device_range {
>>>> 	char name[64];
>>>> 	uint32_t base;
>>>> 	uint32_t number;
>>>> }
>>>> /* return number of ranges filled, or number of ranges if list is
>>>> NULL. */ int rte_bus_ dev_range_get(struct rte_bus_device_range
>>>> *list, int n);
>>>
>>> Hm, I thought about more port representor specific API.
>>> For me it is hard to tell if such generic naming is good or bad. I
>>> think it should be proven that such generic API makes sense. Any other potential users / use cases?
>>
>> I was thinking about SF, but SF is PCI specific, not suitable for this api. So I'm fine to make it as ethdev api.
>> To append new api into eth_dev_ops, is there ABI concern?
>
>No, eth_dev_ops are internal
>
>>> I've considered ethdev API which returns (in similar way as
>>> above) list of possible port representors which could be controlled
>>> by the device. Also I think it would be useful to include type information (enum with PF, VF, SF), controller ID.
>>
>> Agree.
>>
>> There is a new concern from orchestration side, currently, no
>> interface in openstack and OVS to retrieve representor ID range info,
>> It will take time to adapt this solution. To probe a representor,
>> orchestration need to know how to calculate representor ID, and the ID might vary on different max SF number, i.e. VF4 on PP1
>might got different ID. Representor ID change before that will break the product.
>
>I see.
>
>> Considering both orchestration and testpmd users, how about keeping both solution together? This will bring max flexibility IMHO.
>
>As I said before I don't mind and I really think it is a good idea to add suggested interface to specify representor (i.e. cXpfYvfZ), but the
>problem is making bitmap from representor ID.
>
>ethdev API should use new representor info API to make a representor ID from controller/PF/{VF,SF}.
>Or do you see any problems with such approach?

Sorry I thought the user to figure out representor ID from api.
This combination look good, thanks for clarification :)

So the new api looks like this:
struct rte_eth_representor_info {
  Enum representor_type;
  Uint16_t controller; // -1 for any
  Uint16_t port; // -1 for any
  Uint16_t representor_id;
  Uint16_t count;
  char name[N];

int rte_eth_representor_info_get(struct rte_eth_representor_info *infos);
- Return number of entries.
- NULL infos just return number of entries supported.
Sample outputs:
 VF, -1, 0, 0, 		128, 	"pf0vf"
 SF, -1, 0, 128, 		2048, 	"pf0sf"
 PF, -1, 0, 32767, 	1, 	"pf"
 VF, -1, 1, 32768, 	128, 	"pf1vf"
 SF, -1, 0, (32768+128), 	2048, 	"pf1sf"
 PF, -1, 0, 65535, 	1,	 "pf"

>
>> In struct rte_eth_dev_data, reserved bits could be used to define controller and port, this will avoid bitmap. How do you think?
>
>Could you add a bit more on it? Just a bit more details to the idea since I don't understand what exactly you mean and how it could
>help.

The idea is replacing reserved_64s and adding more device location info in rte_eth-dev_data like this:
  Uint16_t representor_id;
  Uint16_t port_id;
  Uint16_t controller_id;
  Enum representor_type;
Compare them all when matching a device, this will also avoid bitmap encoding. 
Reserved_64s[] was added to mitigate ABI conflicts, IIRC.
But seems no need if making representor info API to make ID.

>
>>>
>>> There is one more bit which is not in the picture yet -
>>> switch_info.port_id. Should it be equal to representor ID? Or different and provided in the info structure?
>>
>> Not exactly same AFAIK, the id used in e-switch.
>>
>>

Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor

[Andrew Rybchenko]

On 2/5/21 12:13 PM, Xueming(Steven) Li wrote:
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Friday, February 5, 2021 3:35 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; Thomas Monjalon
>> <tmonjalon@nvidia.com>
>> Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction representor
>>
>> On 2/4/21 5:15 PM, Xueming(Steven) Li wrote:
>>>
>>>> -----Original Message-----
>>>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>>> Sent: Monday, February 1, 2021 4:39 PM
>>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>>> Penso <asafp@nvidia.com>
>>>> Subject: Re: [dpdk-dev] [PATCH v5 0/9] ethdev: support SubFunction
>>>> representor
>>>>
>>>> On 1/28/21 5:31 PM, Xueming(Steven) Li wrote:
>>>>> <snip>
>>>>>>> The patch of device SF capability, but seems I misunderstood your suggestion.
>>>>>>> Let me explain process to create a SF:
>>>>>>> 1. SF can be created on the fly with scripts, unlike VF which is statically pre-created.
>>>>>>
>>>>>> Is there a maximum index and maximum total number of SF's created? How to find it?
>>>>>
>>>>> The maximum index is defined by firmware configuration, all SF's
>>>>> information could be found from sysfs. To create a SF, both PCI and sfnum have to be specified.
>>>>
>>>> sysfs is obviously Linux specific. I think the information should be available via DPDK API.
>>>
>>> Yes, the new api discussed below should resolve this issue.
>>>
>>>>
>>>>>>
>>>>>>> 2. SF is created on a PF with a SF number. SF number is named per PF, different PF may have same SF number.
>>>>>>> 3. For standalone PF, hot plug to DPDK using "PF#_BDF,representor=sf#", no need to use pf#sf# here.
>>>>>>> 4. For bonding netdev, hot plug to DPDK using "PF0_BDF,representor=pf#sf#"
>>>>>>> If using new api to return all representor IDs, need some way
>>>>>>> locate the new created SF by PF and SF number, that's why "pf#sf#" is used in this patch set.
>>>>>>
>>>>>> I think the API should simply reserve/report space for maximum
>>>>>> number of SFs. So, IDs are stable across restart/reboot in
>>>>>> assumption that NIC is not reconfigured (changed maximum number of
>>>>>> VF or
>>>> maximum number of SFs of any PF).
>>>>>
>>>>> Yes, IDs should be stable as long as no  NIC firmware configuration change.
>>>>>
>>>>> Just clarify, this api should be common enough to report all devices that a bus device supports:
>>>>> 1. name, might contains controller and pf info, example: "eth:representor:c0pf1vf"
>>>>> 2. ID range, example: 0-127
>>>>> The api describes ID ranges for each sub device type, users have to query the api and choose representor ID to probe.
>>>>>
>>>>> Prototype:
>>>>> struct rte_bus_device_range {
>>>>> 	char name[64];
>>>>> 	uint32_t base;
>>>>> 	uint32_t number;
>>>>> }
>>>>> /* return number of ranges filled, or number of ranges if list is
>>>>> NULL. */ int rte_bus_ dev_range_get(struct rte_bus_device_range
>>>>> *list, int n);
>>>>
>>>> Hm, I thought about more port representor specific API.
>>>> For me it is hard to tell if such generic naming is good or bad. I
>>>> think it should be proven that such generic API makes sense. Any other potential users / use cases?
>>>
>>> I was thinking about SF, but SF is PCI specific, not suitable for this api. So I'm fine to make it as ethdev api.
>>> To append new api into eth_dev_ops, is there ABI concern?
>>
>> No, eth_dev_ops are internal
>>
>>>> I've considered ethdev API which returns (in similar way as
>>>> above) list of possible port representors which could be controlled
>>>> by the device. Also I think it would be useful to include type information (enum with PF, VF, SF), controller ID.
>>>
>>> Agree.
>>>
>>> There is a new concern from orchestration side, currently, no
>>> interface in openstack and OVS to retrieve representor ID range info,
>>> It will take time to adapt this solution. To probe a representor,
>>> orchestration need to know how to calculate representor ID, and the ID might vary on different max SF number, i.e. VF4 on PP1
>> might got different ID. Representor ID change before that will break the product.
>>
>> I see.
>>
>>> Considering both orchestration and testpmd users, how about keeping both solution together? This will bring max flexibility IMHO.
>>
>> As I said before I don't mind and I really think it is a good idea to add suggested interface to specify representor (i.e. cXpfYvfZ), but the
>> problem is making bitmap from representor ID.
>>
>> ethdev API should use new representor info API to make a representor ID from controller/PF/{VF,SF}.
>> Or do you see any problems with such approach?
> 
> Sorry I thought the user to figure out representor ID from api.
> This combination look good, thanks for clarification :)
> 
> So the new api looks like this:

Roughly speaking - yes

> struct rte_eth_representor_info {
>   Enum representor_type;
>   Uint16_t controller; // -1 for any

I'm not sure that I understand what does "any" mean in this
case. I think it should be the zero in examples below.
I think that API should return caller controller ID and
PF ID. It would allow to interpret "vf5" correctly when
caller is not controller #0 and/or PF #0.

>   Uint16_t port; // -1 for any

port sounds like physical port, but it should be PF
 (pf, phys_fn or something like this). It could be many
PFs per physical network port.

>   Uint16_t representor_id;

May be base_id? Or rep_base_id?

The question is what to do if range for VF or SF is
not contiguous. Should we have one more index after phys_fn
to  represent it? E.g.
union {
    uint16_t vf;
    uint16_t sf;
};

>   Uint16_t count;

May be id_range which should be 1 to show one function.
It could be convenient to treat 0 this way as well,
but I doubt that it is a good idea.

>   char name[N];
> 
> int rte_eth_representor_info_get(struct rte_eth_representor_info *infos);
> - Return number of entries.
> - NULL infos just return number of entries supported.
> Sample outputs:
>  VF, -1, 0, 0, 		128, 	"pf0vf"
>  SF, -1, 0, 128, 		2048, 	"pf0sf"
>  PF, -1, 0, 32767, 	1, 	"pf"
>  VF, -1, 1, 32768, 	128, 	"pf1vf"
>  SF, -1, 0, (32768+128), 	2048, 	"pf1sf"
>  PF, -1, 0, 65535, 	1,	 "pf"
> 
>>
>>> In struct rte_eth_dev_data, reserved bits could be used to define controller and port, this will avoid bitmap. How do you think?
>>
>> Could you add a bit more on it? Just a bit more details to the idea since I don't understand what exactly you mean and how it could
>> help.
> 
> The idea is replacing reserved_64s and adding more device location info in rte_eth-dev_data like this:
>   Uint16_t representor_id;
>   Uint16_t port_id;
>   Uint16_t controller_id;
>   Enum representor_type;
> Compare them all when matching a device, this will also avoid bitmap encoding. 
> Reserved_64s[] was added to mitigate ABI conflicts, IIRC.
> But seems no need if making representor info API to make ID.
> 
>>
>>>>
>>>> There is one more bit which is not in the picture yet -
>>>> switch_info.port_id. Should it be equal to representor ID? Or different and provided in the info structure?
>>>
>>> Not exactly same AFAIK, the id used in e-switch.
>>>
>>>
> 

[dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor

[Xueming Li]

SubFunction [1] is a portion of the PCI device, a SF netdev has its own
dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to existing
VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For backward compatibility, this patch also introduces new netdev
capability to indicate the capability of supportting SF representor.

Version history:
 RFC:
 	initial version [2]
 V2:
    - separate patch for represnetor infrastructure, controller, pf and
      sf.
    - replace representor ID macro with functions:
      rte_eth_representor_id_encode()
      rte_eth_representor_id_parse()
    - new patch to allow devargs with same PCI BDF but different
      representors.
    - other minor code updates according to comments, thanks Andrew!
    - update document
 V3:
    - improve probing of allowed devargs with same name.
    - parse single word of kvargs as key.
    - update kvargs test cases.
 V4:
    - split first representor refactor patch into
      1: add representor type
      2: refector representor list parsing
    - push the patch supporting multi-devargs for same device.
 V5:
    - add comments for parsing functions
    - update switch_representation.rst - Thanks Ajit
 V6:
    - split representor types into different patches, move to
      rte_ethdev.h
    - improvements of rte_eth_devargs_process_list() according to
      Andrew's suggestion
    - fixed PF probe failure for Intel i40e
    - replace ethdev SF capability with rte_eth_representor_info_get()
    - add new ethdev ops api to get representor info from PMD
    - replace representor ID encode/decode with conversion from
      representor info
    - change ethdev representor iterator to use new ID encoding


Xueming Li (9):
  ethdev: introduce representor type
  ethdev: support representor port list
  ethdev: support new VF representor syntax
  ethdev: support sub function representor
  ethdev: support PF index in representor
  ethdev: support multi-host in representor
  ethdev: new API to get representor info
  ethdev: representor iterator compare complete info
  kvargs: update parser to support lists

 app/test/test_kvargs.c                        |  46 ++++-
 config/rte_config.h                           |   1 +
 doc/guides/prog_guide/poll_mode_drv.rst       |  13 +-
 .../prog_guide/switch_representation.rst      |  35 +++-
 drivers/net/bnxt/bnxt_ethdev.c                |   7 +
 drivers/net/enic/enic_ethdev.c                |   6 +
 drivers/net/i40e/i40e_ethdev.c                |   7 +
 drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
 drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
 lib/librte_ethdev/ethdev_driver.h             |  49 ++++-
 lib/librte_ethdev/ethdev_private.c            | 173 ++++++++++++------
 lib/librte_ethdev/ethdev_private.h            |   3 -
 lib/librte_ethdev/rte_class_eth.c             |  40 ++--
 lib/librte_ethdev/rte_ethdev.c                | 102 ++++++++++-
 lib/librte_ethdev/rte_ethdev.h                |  53 ++++++
 lib/librte_ethdev/version.map                 |   4 +
 lib/librte_kvargs/rte_kvargs.c                | 101 +++++++---
 17 files changed, 535 insertions(+), 123 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH v6 1/9] ethdev: introduce representor type

[Xueming Li]

To support more representor type, this patch introduces representor type
enum. The enum is subject to be extended to support new representor in
patches upcoming.

For each devarg structure, only one type supported.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 drivers/net/bnxt/bnxt_ethdev.c     |  7 +++++++
 drivers/net/enic/enic_ethdev.c     |  6 ++++++
 drivers/net/i40e/i40e_ethdev.c     |  7 +++++++
 drivers/net/ixgbe/ixgbe_ethdev.c   |  7 +++++++
 drivers/net/mlx5/linux/mlx5_os.c   | 11 +++++++++++
 lib/librte_ethdev/ethdev_driver.h  |  7 ++++++-
 lib/librte_ethdev/ethdev_private.c |  5 +++++
 lib/librte_ethdev/rte_ethdev.h     |  8 ++++++++
 8 files changed, 57 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 22c880c5c3..1a276c9b9e 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5618,6 +5618,13 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
 	int i, ret = 0;
 	struct rte_kvargs *kvlist = NULL;
 
+	if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
+			    eth_da->type);
+		return -ENOTSUP;
+	}
 	num_rep = eth_da->nb_representor_ports;
 	if (num_rep > BNXT_MAX_VF_REPS) {
 		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index b3f441c8f7..def669f411 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1312,6 +1312,12 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (retval)
 			return retval;
 	}
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct enic),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index d7cd049891..9b86bcdc69 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -639,6 +639,13 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 			return retval;
 	}
 
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct i40e_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 761a0f26bb..8a9a21e7c2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1718,6 +1718,13 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	} else
 		memset(&eth_da, 0, sizeof(eth_da));
 
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct ixgbe_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 2dc079779d..b041e1752d 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -704,6 +704,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				strerror(rte_errno));
 			return NULL;
 		}
+		if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) {
+			/* Representor not specified. */
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+			rte_errno = ENOTSUP;
+			DRV_LOG(ERR, "unsupported representor type: %s",
+				dpdk_dev->devargs->args);
+			return NULL;
+		}
 		for (i = 0; i < eth_da.nb_representor_ports; ++i)
 			if (eth_da.representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 57fdedaa1a..d68b8401e7 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1216,7 +1216,11 @@ __rte_internal
 int
 rte_eth_switch_domain_free(uint16_t domain_id);
 
-/** Generic Ethernet device arguments  */
+/**
+ * Generic Ethernet device arguments
+ *
+ * One type of representor each structure.
+ */
 struct rte_eth_devargs {
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
@@ -1226,6 +1230,7 @@ struct rte_eth_devargs {
 	/** representor port/s identifier to enable on device */
 	uint16_t nb_representor_ports;
 	/** number of ports in representor port field */
+	enum rte_eth_representor_type type; /* type of representor */
 };
 
 /**
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 5960c75cc5..2d51e3857a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -111,11 +111,16 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	return 0;
 }
 
+/*
+ * representor format:
+ *   #: range or single number of VF representor
+ */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 }
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 059a061072..1f378958ca 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1506,6 +1506,14 @@ struct rte_eth_rxseg_capa {
  * Ethernet device information
  */
 
+/**
+ * Ethernet device representor port type.
+ */
+enum rte_eth_representor_type {
+	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
+	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
+};
+
 /**
  * A structure used to retrieve the contextual information of
  * an Ethernet device, such as the controlling driver of the
-- 
2.25.1

[dpdk-dev] [PATCH v6 2/9] ethdev: support representor port list

[Xueming Li]

To support extended representor syntax, this patch extends the
representor list parsing to support for representor port range in
devargs, examples:
   representor=[1,2,3]         - single list
   representor=[1,3-5,7,9-11]  - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 lib/librte_ethdev/ethdev_private.c | 129 ++++++++++++++++-------------
 lib/librte_ethdev/ethdev_private.h |   3 -
 lib/librte_ethdev/rte_class_eth.c  |   4 +-
 lib/librte_ethdev/rte_ethdev.c     |   5 +-
 4 files changed, 74 insertions(+), 67 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 2d51e3857a..d5ef05ea6f 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -38,82 +38,91 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
 	return NULL;
 }
 
-int
-rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data)
+/* Put new value into list. */
+static int
+rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
+		       const uint16_t max_list, uint16_t val)
 {
-	char *str_start;
-	int state;
-	int result;
-
-	if (*str != '[')
-		/* Single element, not a list */
-		return callback(str, data);
-
-	/* Sanity check, then strip the brackets */
-	str_start = &str[strlen(str) - 1];
-	if (*str_start != ']') {
-		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
-		return -EINVAL;
-	}
-	str++;
-	*str_start = '\0';
+	uint16_t i;
 
-	/* Process list elements */
-	state = 0;
-	while (1) {
-		if (state == 0) {
-			if (*str == '\0')
-				break;
-			if (*str != ',') {
-				str_start = str;
-				state = 1;
-			}
-		} else if (state == 1) {
-			if (*str == ',' || *str == '\0') {
-				if (str > str_start) {
-					/* Non-empty string fragment */
-					*str = '\0';
-					result = callback(str_start, data);
-					if (result < 0)
-						return result;
-				}
-				state = 0;
-			}
-		}
-		str++;
+	for (i = 0; i < *len_list; i++) {
+		if (list[i] == val)
+			return 0;
 	}
+	if (*len_list >= max_list)
+		return -1;
+	list[(*len_list)++] = val;
 	return 0;
 }
 
-static int
+/* Parse and enlist a range expression of "min-max" or a single value. */
+static char *
 rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	const uint16_t max_list)
 {
 	uint16_t lo, hi, val;
-	int result;
+	int result, n = 0;
+	char *pos = str;
 
-	result = sscanf(str, "%hu-%hu", &lo, &hi);
+	result = sscanf(str, "%hu%n-%hu%n", &lo, &n, &hi, &n);
 	if (result == 1) {
-		if (*len_list >= max_list)
-			return -ENOMEM;
-		list[(*len_list)++] = lo;
+		if (rte_eth_devargs_enlist(list, len_list, max_list, lo) != 0)
+			return NULL;
 	} else if (result == 2) {
-		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
-			return -EINVAL;
+		if (lo > hi)
+			return NULL;
 		for (val = lo; val <= hi; val++) {
-			if (*len_list >= max_list)
-				return -ENOMEM;
-			list[(*len_list)++] = val;
+			if (rte_eth_devargs_enlist(list, len_list, max_list,
+						   val) != 0)
+				return NULL;
 		}
 	} else
-		return -EINVAL;
-	return 0;
+		return NULL;
+	return pos + n;
+}
+
+/*
+ * Parse list of values separated by ",".
+ * Each value could be a range [min-max] or single number.
+ * Examples:
+ *  2               - single
+ *  [1,2,3]         - single list
+ *  [1,3-5,7,9-11]  - list with singles and ranges
+ */
+static char *
+rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
+	const uint16_t max_list)
+{
+	char *pos = str;
+
+	if (*pos == '[')
+		pos++;
+	while (1) {
+		pos = rte_eth_devargs_process_range(pos, list, len_list,
+						    max_list);
+		if (pos == NULL)
+			return NULL;
+		if (*pos != ',') /* end of list */
+			break;
+		pos++;
+	}
+	if (*str == '[' && *pos != ']')
+		return NULL;
+	if (*pos == ']')
+		pos++;
+	return pos;
 }
 
 /*
- * representor format:
+ * Parse representor ports from a single value or lists.
+ *
+ * Representor format:
  *   #: range or single number of VF representor
+ *
+ * Examples of #:
+ *  2               - single
+ *  [1,2,3]         - single list
+ *  [1,3-5,7,9-11]  - list with singles and ranges
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -121,6 +130,10 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
-		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
+		&eth_da->nb_representor_ports,
+		RTE_DIM(eth_da->representor_ports));
+	if (str == NULL)
+		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
+	return str == NULL ? -1 : 0;
 }
diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
index 905a45c337..220ddd4408 100644
--- a/lib/librte_ethdev/ethdev_private.h
+++ b/lib/librte_ethdev/ethdev_private.h
@@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
 		const void *data);
 
 /* Parse devargs value for representor parameter. */
-typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
-int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data);
 int rte_eth_devargs_parse_representor_ports(char *str, void *data);
 
 #ifdef __cplusplus
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index ca2ce87f7a..051c892b40 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
 	if (values == NULL)
 		return -1;
 	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_list(values,
-			rte_eth_devargs_parse_representor_ports,
-			&representors);
+	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 6f514c388b..fe9466a03e 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5589,9 +5589,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
-			result = rte_eth_devargs_parse_list(pair->value,
-				rte_eth_devargs_parse_representor_ports,
-				eth_da);
+			result = rte_eth_devargs_parse_representor_ports(
+					pair->value, eth_da);
 			if (result < 0)
 				goto parse_cleanup;
 		}
-- 
2.25.1

[dpdk-dev] [PATCH v6 3/9] ethdev: support new VF representor syntax

[Xueming Li]

Current VF representor syntax:
 representor=2          - single representor
 representor=[0-3]      - single range

To prepare for more representor types, this patch adds compatible VF
representor devargs syntax:

vf#:
 representor=vf2          - single representor
 representor=vf[1,3,5]    - single list
 representor=vf[0-3]      - single range
 representor=vf[0,1,4-7]  - list with singles and range

For backwards compatibility, representor "#" is interpreted as "vf#".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 7 ++++---
 lib/librte_ethdev/ethdev_private.c      | 5 ++++-
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 239ec820ea..0117c2af07 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -374,9 +374,10 @@ parameters to those ports.
   this argument allows user to specify which switch ports to enable port
   representors for.::
 
-   -a DBDF,representor=0
-   -a DBDF,representor=[0,4,6,9]
-   -a DBDF,representor=[0-31]
+   -a DBDF,representor=vf0
+   -a DBDF,representor=vf[0,4,6,9]
+   -a DBDF,representor=vf[0-31]
+   -a DBDF,representor=vf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index d5ef05ea6f..4bb3879859 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -117,7 +117,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Parse representor ports from a single value or lists.
  *
  * Representor format:
- *   #: range or single number of VF representor
+ *   #: range or single number of VF representor - legacy
+ *   vf#: VF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -130,6 +131,8 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	if (str[0] == 'v' && str[1] == 'f')
+		str += 2;
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
-- 
2.25.1

[dpdk-dev] [PATCH v6 4/9] ethdev: support sub function representor

[Xueming Li]

SubFunction is a portion of the PCI device, created on demand, a SF
netdev has its own dedicated queues(txq, rxq). A SF netdev supports
eswitch representation offload similar to existing PF and VF
representors.

To support SF representor, this patch introduces new devargs syntax,
examples:
 representor=sf0               - single SubFunction representor
 representor=sf[1,3,5]         - single list
 representor=sf[0-3],          - single range
 representor=sf[0,2-6,8,10-12] - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst       |  4 +++
 .../prog_guide/switch_representation.rst      | 35 +++++++++++++------
 lib/librte_ethdev/ethdev_private.c            | 11 ++++--
 lib/librte_ethdev/rte_ethdev.h                |  1 +
 4 files changed, 39 insertions(+), 12 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 0117c2af07..86e5867f1b 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -378,6 +378,10 @@ parameters to those ports.
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
    -a DBDF,representor=vf[0,2-4,7,9-11]
+   -a DBDF,representor=sf0
+   -a DBDF,representor=sf[1,3,5]
+   -a DBDF,representor=sf[0-1023]
+   -a DBDF,representor=sf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/doc/guides/prog_guide/switch_representation.rst b/doc/guides/prog_guide/switch_representation.rst
index 07ba12bea6..ff6aa91c80 100644
--- a/doc/guides/prog_guide/switch_representation.rst
+++ b/doc/guides/prog_guide/switch_representation.rst
@@ -13,7 +13,7 @@ Introduction
 
 Network adapters with multiple physical ports and/or SR-IOV capabilities
 usually support the offload of traffic steering rules between their virtual
-functions (VFs), physical functions (PFs) and ports.
+functions (VFs), sub functions (SFs), physical functions (PFs) and ports.
 
 Like for standard Ethernet switches, this involves a combination of
 automatic MAC learning and manual configuration. For most purposes it is
@@ -24,7 +24,7 @@ layer 2 (L2) traffic (such as OVS) need to steer traffic themselves
 according on their own criteria.
 
 Without a standard software interface to manage traffic steering rules
-between VFs, PFs and the various physical ports of a given device,
+between VFs, SFs, PFs and the various physical ports of a given device,
 applications cannot take advantage of these offloads; software processing is
 mandatory even for traffic which ends up re-injected into the device it
 originates from.
@@ -34,6 +34,17 @@ the DPDK flow API (**rte_flow**), with emphasis on the SR-IOV use case
 (PF/VF steering) using a single physical port for clarity, however the same
 logic applies to any number of ports without necessarily involving SR-IOV.
 
+Sub Function
+------------
+Besides SR-IOV, Sub function is a portion of the PCI device, a SF netdev
+has its own dedicated queues(txq, rxq). A SF netdev supports E-Switch
+representation offload similar to existing PF and VF representors.
+A SF shares PCI level resources with other SFs and/or with its parent PCI
+function.
+
+Sub function is created on-demand, coexists with VFs. Number of SFs is
+limited by hardware resources.
+
 Port Representors
 -----------------
 
@@ -42,15 +53,16 @@ applications usually have to process a bit of traffic in software before
 thinking about offloading specific flows to hardware.
 
 Applications therefore need the ability to receive and inject traffic to
-various device endpoints (other VFs, PFs or physical ports) before
+various device endpoints (other VFs, SFs, PFs or physical ports) before
 connecting them together. Device drivers must provide means to hook the
 "other end" of these endpoints and to refer them when configuring flow
 rules.
 
 This role is left to so-called "port representors" (also known as "VF
-representors" in the specific context of VFs), which are to DPDK what the
-Ethernet switch device driver model (**switchdev**) [1]_ is to Linux, and
-which can be thought as a software "patch panel" front-end for applications.
+representors" in the specific context of VFs, "SF representors" in the
+specific context of SFs), which are to DPDK what the Ethernet switch
+device driver model (**switchdev**) [1]_ is to Linux, and which can be
+thought as a software "patch panel" front-end for applications.
 
 - DPDK port representors are implemented as additional virtual Ethernet
   device (**ethdev**) instances, spawned on an as needed basis through
@@ -59,9 +71,12 @@ which can be thought as a software "patch panel" front-end for applications.
 
 ::
 
-   -a pci:dbdf,representor=0
-   -a pci:dbdf,representor=[0-3]
-   -a pci:dbdf,representor=[0,5-11]
+   -a pci:dbdf,representor=vf0
+   -a pci:dbdf,representor=vf[0-3]
+   -a pci:dbdf,representor=vf[0,5-11]
+   -a pci:dbdf,representor=sf1
+   -a pci:dbdf,representor=sf[0-1023]
+   -a pci:dbdf,representor=sf[0,2-1023]
 
 - As virtual devices, they may be more limited than their physical
   counterparts, for instance by exposing only a subset of device
@@ -360,7 +375,7 @@ Compared to creating a brand new dedicated interface, **rte_flow** was
 deemed flexible enough to manage representor traffic only with minor
 extensions:
 
-- Using physical ports, PF, VF or port representors as targets.
+- Using physical ports, PF, SF, VF or port representors as targets.
 
 - Affecting traffic that is not necessarily addressed to the DPDK port ID a
   flow rule is associated with (e.g. forcing VF traffic redirection to PF).
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 4bb3879859..13c191192e 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -119,6 +119,7 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Representor format:
  *   #: range or single number of VF representor - legacy
  *   vf#: VF port representor/s
+ *   sf#: SF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -130,9 +131,15 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
-	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	if (str[0] == 'v' && str[1] == 'f')
+	if (str[0] == 'v' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
+	} else if (str[0] == 's' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_SF;
+		str += 2;
+	} else {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 1f378958ca..26b5e109c3 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1512,6 +1512,7 @@ struct rte_eth_rxseg_capa {
 enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
 	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
+	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v6 5/9] ethdev: support PF index in representor

[Xueming Li]

With Kernel bonding, multiple underlying PFs are bonded, VFs come
from different PF, need to identify representor of VFs unambiguously by
adding PF index.

This patch introduces optional 'pf' section to representor devargs
syntax, examples:
 representor=pf0vf0             - single VF representor
 representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs

PF type representor is supported by using standalone 'pf' section:
 representor=pf1                - PF representor


Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  2 ++
 lib/librte_ethdev/ethdev_private.c      | 18 ++++++++++++++++--
 lib/librte_ethdev/rte_ethdev.h          |  1 +
 3 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 86e5867f1b..b2147aad30 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -382,6 +382,8 @@ parameters to those ports.
    -a DBDF,representor=sf[1,3,5]
    -a DBDF,representor=sf[0-1023]
    -a DBDF,representor=sf[0,2-4,7,9-11]
+   -a DBDF,representor=pf1vf0
+   -a DBDF,representor=pf[0-1]sf[0-127]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 13c191192e..eea0686020 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -118,8 +118,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   vf#: VF port representor/s
- *   sf#: SF port representor/s
+ *   [pf#]vf#: VF port representor/s
+ *   [pf#]sf#: SF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -131,6 +131,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'p' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_PF;
+		str += 2;
+		str = rte_eth_devargs_process_list(str, eth_da->ports,
+				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
+		if (str == NULL)
+			goto err;
+	}
 	if (str[0] == 'v' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
@@ -138,11 +146,17 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 		eth_da->type = RTE_ETH_REPRESENTOR_SF;
 		str += 2;
 	} else {
+		/* Don't mix legacy syntax with 'pf' section. */
+		if (eth_da->type == RTE_ETH_REPRESENTOR_PF) {
+			str = NULL;
+			goto err;
+		}
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
+err:
 	if (str == NULL)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
 	return str == NULL ? -1 : 0;
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 26b5e109c3..9cd519bf59 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1513,6 +1513,7 @@ enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
 	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
 	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
+	RTE_ETH_REPRESENTOR_PF,   /**< representor of Physical Function. */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v6 6/9] ethdev: support multi-host in representor

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to the multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. To provide the unambiguous identification
of the PCIe function the controller index is added. The full representor
identifier consists of three indices - controller index, PF index, and
VF or SF index (if any).

This patch introduces controller index to ethdev representor syntax,
examples:

[[c#]pf#]vf#: VF port representor/s, example: pf0vf1
[[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]

c# is controller(host) ID/range in case of multi-host, optional.

For user application (e.g. OVS), PMD is responsible to interpret and
locate representor device based on controller ID, PF ID and VF/SF ID in
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 config/rte_config.h                |  1 +
 lib/librte_ethdev/ethdev_driver.h  |  4 ++++
 lib/librte_ethdev/ethdev_private.c | 25 +++++++++++++++++++++----
 3 files changed, 26 insertions(+), 4 deletions(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index 55a2fc50ed..904a40b3c1 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -57,6 +57,7 @@
 #define RTE_MAX_QUEUES_PER_PORT 1024
 #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
 #define RTE_ETHDEV_RXTX_CALLBACKS 1
+#define RTE_MAX_MULTI_HOST_CTRLS 4
 
 /* cryptodev defines */
 #define RTE_CRYPTO_MAX_DEVS 64
diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index d68b8401e7..06ff35266f 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1222,6 +1222,10 @@ rte_eth_switch_domain_free(uint16_t domain_id);
  * One type of representor each structure.
  */
 struct rte_eth_devargs {
+	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
+	/** controller/s number in case of multi-host */
+	uint16_t nb_mh_controllers;
+	/** number of controllers in multi-host controllers field */
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
 	uint16_t nb_ports;
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index eea0686020..5cd62de6f8 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -118,8 +118,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   [pf#]vf#: VF port representor/s
- *   [pf#]sf#: SF port representor/s
+ *   [[c#]pf#]vf#: VF port representor/s
+ *   [[c#]pf#]sf#: SF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -131,6 +131,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'c') {
+		str += 1;
+		str = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
+				&eth_da->nb_mh_controllers,
+				RTE_DIM(eth_da->mh_controllers));
+		if (str == NULL)
+			goto err;
+	}
 	if (str[0] == 'p' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_PF;
 		str += 2;
@@ -146,8 +154,9 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 		eth_da->type = RTE_ETH_REPRESENTOR_SF;
 		str += 2;
 	} else {
-		/* Don't mix legacy syntax with 'pf' section. */
-		if (eth_da->type == RTE_ETH_REPRESENTOR_PF) {
+		/* Don't mix legacy syntax with 'pf' and 'c' section. */
+		if (eth_da->type == RTE_ETH_REPRESENTOR_PF ||
+		    eth_da->nb_mh_controllers > 0) {
 			str = NULL;
 			goto err;
 		}
@@ -156,6 +165,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
+	if (str == NULL)
+		goto err;
+	/* "pf" must sit in middle of "c" and "vf"/"sf". */
+	if (eth_da->nb_representor_ports > 0 && eth_da->nb_mh_controllers > 0 &&
+	    eth_da->nb_ports == 0) {
+		RTE_LOG(ERR, EAL, "'pf' section missing: %s\n", str);
+		return -1;
+	}
 err:
 	if (str == NULL)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
-- 
2.25.1

[dpdk-dev] [PATCH v6 7/9] ethdev: new API to get representor info

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

This patch introduces a new API rte_eth_representor_info_get() to
retrieve representor corresponding info mapping:
 - caller controller index and pf index.
 - supported representor ID ranges.
 - type, controller, pf and start vf/sf ID of each range.
The API is useful to convert representor from devargs to representor ID.

New ethdev callback representor_info_get() is added to retrieve info
from PMD driver, optional for PMD that doesn't support new devargs
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_ethdev/ethdev_driver.h |  6 +++++
 lib/librte_ethdev/rte_ethdev.c    | 14 ++++++++++
 lib/librte_ethdev/rte_ethdev.h    | 43 +++++++++++++++++++++++++++++++
 lib/librte_ethdev/version.map     |  3 +++
 4 files changed, 66 insertions(+)

diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 06ff35266f..abcbc3112d 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -289,6 +289,10 @@ typedef int (*eth_fw_version_get_t)(struct rte_eth_dev *dev,
 				     char *fw_version, size_t fw_size);
 /**< @internal Get firmware information of an Ethernet device. */
 
+typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
+	struct rte_eth_representor_info *info);
+/**< @internal Get representor type and ID range. */
+
 typedef int (*eth_tx_done_cleanup_t)(void *txq, uint32_t free_cnt);
 /**< @internal Force mbufs to be from TX ring. */
 
@@ -823,6 +827,8 @@ struct eth_dev_ops {
 	eth_burst_mode_get_t       rx_burst_mode_get; /**< Get RX burst mode */
 	eth_burst_mode_get_t       tx_burst_mode_get; /**< Get TX burst mode */
 	eth_fw_version_get_t       fw_version_get; /**< Get firmware version. */
+	eth_representor_info_get_t representor_info_get;
+	/**< Get representor info. */
 	eth_dev_supported_ptypes_get_t dev_supported_ptypes_get;
 	/**< Get packet types supported and identified by device. */
 	eth_dev_ptypes_set_t dev_ptypes_set;
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index fe9466a03e..07c6debb58 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -3265,6 +3265,20 @@ rte_eth_dev_fw_version_get(uint16_t port_id, char *fw_version, size_t fw_size)
 							fw_version, fw_size));
 }
 
+int
+rte_eth_representor_info_get(uint16_t port_id,
+			     struct rte_eth_representor_info *info)
+{
+	struct rte_eth_dev *dev;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->fw_version_get, -ENOTSUP);
+	return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev,
+								      info));
+}
+
 int
 rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info)
 {
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 9cd519bf59..35eb0a5721 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1581,6 +1581,30 @@ struct rte_eth_dev_info {
 	void *reserved_ptrs[2];   /**< Reserved for future fields */
 };
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice.
+ *
+ * Ethernet device representor information
+ */
+struct rte_eth_representor_info {
+	uint16_t controller; /**< Controller ID of caller device. */
+	uint16_t pf; /**< Physical function ID of caller device. */
+	struct {
+		enum rte_eth_representor_type type; /**< Representor type */
+		int controller; /**< Controller ID, -1 to ignore */
+		int pf; /**< Physical function ID, -1 to ignore */
+		__extension__
+		union {
+			int vf; /**< VF start index */
+			int sf; /**< SF start index */
+		};
+		uint16_t id_base; /**< Representor ID start index */
+		uint16_t id_end;  /**< Representor ID end index */
+		char name[RTE_DEV_NAME_MAX_LEN];	/**< Representor name */
+	} ranges[]; /**< Representor ID range by type */
+};
+
 /**
  * Ethernet device RX queue information structure.
  * Used to retrieve information about configured queue.
@@ -3038,6 +3062,25 @@ int rte_eth_macaddr_get(uint16_t port_id, struct rte_ether_addr *mac_addr);
  */
 int rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info);
 
+/**
+ * Retrieve the representor info of the device.
+ *
+ * @param port_id
+ *   The port identifier of the device.
+ * @param info
+ *   A pointer to a representor info structure.
+ *   NULL to return number of range entries and allocate memory
+ *   for next call to store detail.
+ * @return
+ *   - (-ENOTSUP) if operation is not supported.
+ *   - (-ENODEV) if *port_id* invalid.
+ *   - (-EIO) if device is removed.
+ *   - (>=0) number of representor range entries supported by device.
+ */
+__rte_experimental
+int rte_eth_representor_info_get(uint16_t port_id,
+				 struct rte_eth_representor_info *info);
+
 /**
  * Retrieve the firmware version of a device.
  *
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index a124e1e370..bb6f7436c2 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -243,6 +243,9 @@ EXPERIMENTAL {
 
 	# added in 21.02
 	rte_eth_get_monitor_addr;
+
+	# added in 21.05
+	rte_eth_representor_info_get;
 };
 
 INTERNAL {
-- 
2.25.1

[dpdk-dev] [PATCH v6 8/9] ethdev: representor iterator compare complete info

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

SR-IOV and SubFunction are created on top of PF. PF index is introduced
because there might be multiple PFs in the bonding configuration and
only bonding device is probed.

In eth representor comparator callback, ethdev representor ID was
compared with devarg. Since controller index and PF index not compared,
callback returned representor from other PF or controller.

This patch adds new API to convert representor controller, pf and vf/sf
index to representor ID. Representor comparer callback convert
representor info into ID and compare with device representor ID.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_ethdev/ethdev_driver.h | 32 ++++++++++++
 lib/librte_ethdev/rte_class_eth.c | 38 ++++++++++----
 lib/librte_ethdev/rte_ethdev.c    | 83 +++++++++++++++++++++++++++++++
 lib/librte_ethdev/version.map     |  1 +
 4 files changed, 145 insertions(+), 9 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index abcbc3112d..23342f1be2 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1243,6 +1243,38 @@ struct rte_eth_devargs {
 	enum rte_eth_representor_type type; /* type of representor */
 };
 
+/**
+ * PMD helper function to convert representor ID from location detail
+ *
+ * Convert representor ID from controller, pf and (sf or vf).
+ * The mapping is retrieved from rte_eth_representor_info_get().
+ *
+ * If PMD doesn't return representor range info, simply ignore controller
+ * and pf to keep backward compatibility.
+ *
+ * @param ethdev
+ *  Handle of ethdev port.
+ * @param id
+ *  Pointer to converted representor ID.
+ * @param type
+ *  Representor type.
+ * @param controller
+ *  Controller ID, -1 if unspecified.
+ * @param pf
+ *  PF port ID, -1 if unspecified.
+ * @param representor_port
+ *  Representor port ID, -1 if unspecified.
+ *
+ * @return
+ *  Negative errno value on error, 0 on success.
+ */
+__rte_internal
+int
+rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
+			       uint16_t *id,
+			       enum rte_eth_representor_type type,
+			       int controller, int pf, int representor_port);
+
 /**
  * PMD helper function to parse ethdev arguments
  *
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index 051c892b40..f7b7e659e7 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -65,9 +65,10 @@ eth_representor_cmp(const char *key __rte_unused,
 {
 	int ret;
 	char *values;
-	const struct rte_eth_dev_data *data = opaque;
-	struct rte_eth_devargs representors;
-	uint16_t index;
+	const struct rte_eth_dev *edev = opaque;
+	const struct rte_eth_dev_data *data = edev->data;
+	struct rte_eth_devargs eth_da;
+	uint16_t id, nc, np, nf, i, c, p, f;
 
 	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
 		return -1; /* not a representor port */
@@ -76,17 +77,36 @@ eth_representor_cmp(const char *key __rte_unused,
 	values = strdup(value);
 	if (values == NULL)
 		return -1;
-	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
+	memset(&eth_da, 0, sizeof(eth_da));
+	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
 
+	if (eth_da.nb_mh_controllers == 0 && eth_da.nb_ports == 0 &&
+	    eth_da.nb_representor_ports == 0)
+		return -1;
+	nc = eth_da.nb_mh_controllers > 0 ? eth_da.nb_mh_controllers : 1;
+	np = eth_da.nb_ports > 0 ? eth_da.nb_ports : 1;
+	nf = eth_da.nb_representor_ports > 0 ? eth_da.nb_representor_ports : 1;
+
 	/* Return 0 if representor id is matching one of the values. */
-	for (index = 0; index < representors.nb_representor_ports; index++)
-		if (data->representor_id ==
-				representors.representor_ports[index])
+	for (i = 0; i < nc * np * nf; ++i) {
+		c = i / (np * nf);
+		p = (i / nf) % np;
+		f = i % nf;
+		if (rte_eth_representor_id_convert(edev,
+			&id,
+			eth_da.type,
+			eth_da.nb_mh_controllers == 0 ? -1 :
+					eth_da.mh_controllers[c],
+			eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
+			eth_da.nb_representor_ports == 0 ? -1 :
+					eth_da.representor_ports[f]) < 0)
+			continue;
+		if (data->representor_id == id)
 			return 0;
+	}
 	return -1; /* no match */
 }
 
@@ -112,7 +132,7 @@ eth_dev_match(const struct rte_eth_dev *edev,
 
 	ret = rte_kvargs_process(kvlist,
 			eth_params_keys[RTE_ETH_PARAM_REPRESENTOR],
-			eth_representor_cmp, edev->data);
+			eth_representor_cmp, (void *)(uintptr_t)edev);
 	if (ret != 0)
 		return -1;
 	/* search for representor key */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 07c6debb58..da0cf1a920 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5617,6 +5617,89 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	return result;
 }
 
+int
+rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
+			       uint16_t *id,
+			       enum rte_eth_representor_type type,
+			       int controller, int pf, int representor_port)
+{
+	int ret, n, i, count;
+	struct rte_eth_representor_info *info = NULL;
+	size_t size;
+
+	if (type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (id == NULL)
+		return -EINVAL;
+
+	/* Get PMD representor range info. */
+	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
+	if (ret < 0) {
+		/* Fallback to direct mapping for compatibility. */
+		*id = representor_port;
+	}
+	n = ret;
+	size = sizeof(*info) + n * sizeof(info->ranges[0]);
+	info = calloc(1, size);
+	if (info == NULL)
+		return -ENOMEM;
+	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
+	if (ret < 0)
+		goto out;
+
+	/* Default controller and pf to caller. */
+	if (controller == -1)
+		controller = info->controller;
+	if (pf == -1)
+		pf = info->pf;
+
+	/* Locate representor ID. */
+	for (i = 0; i < n; ++i) {
+		if (info->ranges[i].type != type)
+			continue;
+		/* PMD hit: ignore controller if -1. */
+		if (info->ranges[i].controller != -1 &&
+		    info->ranges[i].controller != (uint16_t)controller)
+			continue;
+		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;
+		if (info->ranges[i].type == RTE_ETH_REPRESENTOR_PF) {
+			/* PF. */
+			if (pf >= info->ranges[i].pf + count)
+				continue;
+			*id = info->ranges[i].id_base +
+			      (pf - info->ranges[i].pf);
+			goto out;
+		}
+		/* VF or SF. */
+		/* PMD hit: ignore pf if -1. */
+		if (info->ranges[i].pf != -1 &&
+		    info->ranges[i].pf != (uint16_t)pf)
+			continue;
+		if (info->ranges[i].type == RTE_ETH_REPRESENTOR_VF) {
+			/* VF. */
+			if (representor_port >= info->ranges[i].vf + count)
+				continue;
+			*id = info->ranges[i].id_base +
+			      (representor_port - info->ranges[i].vf);
+			goto out;
+		} else if (info->ranges[i].type == RTE_ETH_REPRESENTOR_SF) {
+			/* SF. */
+			if (representor_port >= info->ranges[i].sf + count)
+				continue;
+			*id = info->ranges[i].id_base +
+			      (representor_port - info->ranges[i].sf);
+			goto out;
+		}
+	}
+	/* Not matching representor ID range. */
+	ret = -ENOENT;
+
+out:
+	if (info != NULL)
+		free(info);
+	return ret;
+}
+
 static int
 eth_dev_handle_port_list(const char *cmd __rte_unused,
 		const char *params __rte_unused,
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index bb6f7436c2..2891f5734e 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -268,6 +268,7 @@ INTERNAL {
 	rte_eth_hairpin_queue_peer_bind;
 	rte_eth_hairpin_queue_peer_unbind;
 	rte_eth_hairpin_queue_peer_update;
+	rte_eth_representor_id_convert;
 	rte_eth_switch_domain_alloc;
 	rte_eth_switch_domain_free;
 };
-- 
2.25.1

[dpdk-dev] [PATCH v6 9/9] kvargs: update parser to support lists

[Xueming Li]

This patch updates kvargs parser to support lists on top of range, allow
multiple lists or range:
  k1=a[1,2]b[3-5]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test/test_kvargs.c         |  46 +++++++++++++--
 lib/librte_kvargs/rte_kvargs.c | 101 +++++++++++++++++++++++----------
 2 files changed, 112 insertions(+), 35 deletions(-)

diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index 2a2dae43a0..a91ea8dc47 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -35,6 +35,25 @@ static int check_handler(const char *key, const char *value,
 	return 0;
 }
 
+/* test parsing. */
+static int test_kvargs_parsing(const char *args, unsigned int n)
+{
+	struct rte_kvargs *kvlist;
+
+	kvlist = rte_kvargs_parse(args, NULL);
+	if (kvlist == NULL) {
+		printf("rte_kvargs_parse() error: %s\n", args);
+		return -1;
+	}
+	if (kvlist->count != n) {
+		printf("invalid count value %d: %s\n", kvlist->count, args);
+		rte_kvargs_free(kvlist);
+		return -1;
+	}
+	rte_kvargs_free(kvlist);
+	return 0;
+}
+
 /* test a valid case */
 static int test_valid_kvargs(void)
 {
@@ -42,6 +61,19 @@ static int test_valid_kvargs(void)
 	const char *args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
 	const char **valid_keys;
+	static const struct {
+		unsigned int expected;
+		const char *input;
+	} valid_inputs[] = {
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo" },
+		{ 2, "foo=1,=2" },
+		{ 1, "foo=[1,2" },
+		{ 1, ",=" },
+		{ 1, "foo=[" },
+	};
+	unsigned int i;
 
 	/* empty args is valid */
 	args = "";
@@ -191,6 +223,14 @@ static int test_valid_kvargs(void)
 	}
 	rte_kvargs_free(kvlist);
 
+	valid_keys = NULL;
+
+	for (i = 0; i < RTE_DIM(valid_inputs); ++i) {
+		args = valid_inputs[i].input;
+		if (test_kvargs_parsing(args, valid_inputs[i].expected))
+			goto fail;
+	}
+
 	return 0;
 
  fail:
@@ -212,12 +252,6 @@ static int test_invalid_kvargs(void)
 	/* list of argument that should fail */
 	const char *args_list[] = {
 		"wrong-key=x",     /* key not in valid_keys_list */
-		"foo=1,foo=",      /* empty value */
-		"foo=1,foo",       /* no value */
-		"foo=1,=2",        /* no key */
-		"foo=[1,2",        /* no closing bracket in value */
-		",=",              /* also test with a smiley */
-		"foo=[",           /* no value in list and no closing bracket */
 		NULL };
 	const char **args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
index 285081c86c..ffae8914cf 100644
--- a/lib/librte_kvargs/rte_kvargs.c
+++ b/lib/librte_kvargs/rte_kvargs.c
@@ -5,6 +5,7 @@
 
 #include <string.h>
 #include <stdlib.h>
+#include <stdbool.h>
 
 #include <rte_string_fns.h>
 
@@ -13,15 +14,19 @@
 /*
  * Receive a string with a list of arguments following the pattern
  * key=value,key=value,... and insert them into the list.
- * strtok() is used so the params string will be copied to be modified.
+ * Params string will be copied to be modified.
+ * list "[]" and list element splitter ",", "-" is treated as value.
+ * Supported examples:
+ *   k1=v1,k2=v2
+ *   k1
+ *   k1=x[0-1]y[1,3-5,9]z
  */
 static int
 rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 {
-	unsigned i;
-	char *str;
-	char *ctx1 = NULL;
-	char *ctx2 = NULL;
+	char *str, *start;
+	bool in_list = false, end_key = false, end_value = false;
+	bool save = false, end_pair = false;
 
 	/* Copy the const char *params to a modifiable string
 	 * to pass to rte_strsplit
@@ -32,36 +37,74 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 
 	/* browse each key/value pair and add it in kvlist */
 	str = kvlist->str;
-	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
+	start = str; /* start of current key or value */
+	while (1) {
+		switch (*str) {
+		case '=': /* End of key. */
+			end_key = true;
+			save = true;
+			break;
+		case ',':
+			/* End of value, skip comma in middle of range */
+			if (!in_list) {
+				if (end_key)
+					end_value = true;
+				else
+					end_key = true;
+				save = true;
+				end_pair = true;
+			}
+			break;
+		case '[': /* Start of list. */
+			in_list = true;
+			break;
+		case ']': /* End of list.  */
+			if (in_list)
+				in_list = false;
+			break;
+		case '\0': /* End of string */
+			if (end_key)
+				end_value = true;
+			else
+				end_key = true;
+			save = true;
+			end_pair = true;
+			break;
+		default:
+			break;
+		}
 
-		i = kvlist->count;
-		if (i >= RTE_KVARGS_MAX)
-			return -1;
+		if (!save) {
+			/* Continue if not end of key or value. */
+			str++;
+			continue;
+		}
 
-		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
-		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
-		if (kvlist->pairs[i].key == NULL ||
-		    kvlist->pairs[i].value == NULL)
+		if (kvlist->count >= RTE_KVARGS_MAX)
 			return -1;
 
-		/* Detect list [a,b] to skip comma delimiter in list. */
-		str = kvlist->pairs[i].value;
-		if (str[0] == '[') {
-			/* Find the end of the list. */
-			while (str[strlen(str) - 1] != ']') {
-				/* Restore the comma erased by strtok_r(). */
-				if (ctx1 == NULL || ctx1[0] == '\0')
-					return -1; /* no closing bracket */
-				str[strlen(str)] = ',';
-				/* Parse until next comma. */
-				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
-				if (str == NULL)
-					return -1; /* no closing bracket */
-			}
+		if (end_value)
+			/* Value parsed */
+			kvlist->pairs[kvlist->count].value = start;
+		else if (end_key)
+			/* Key parsed. */
+			kvlist->pairs[kvlist->count].key = start;
+
+		if (end_pair) {
+			if (end_value || str != start)
+				/* Ignore empty pair. */
+				kvlist->count++;
+			end_key = false;
+			end_value = false;
+			end_pair = false;
 		}
 
-		kvlist->count++;
-		str = NULL;
+		if (*str == '\0') /* End of string. */
+			break;
+		*str = '\0';
+		str++;
+		start = str;
+		save = false;
 	}
 
 	return 0;
-- 
2.25.1

Re: [dpdk-dev] [PATCH v6 1/9] ethdev: introduce representor type

[Hyong Youb Kim (hyonkim)]

> -----Original Message-----
> From: Xueming Li <xuemingl@nvidia.com>
> Sent: Sunday, February 14, 2021 12:22 PM
> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>;
> xuemingl@nvidia.com; Asaf Penso <asafp@nvidia.com>; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Somnath Kotur
> <somnath.kotur@broadcom.com>; John Daley (johndale)
> <johndale@cisco.com>; Hyong Youb Kim (hyonkim) <hyonkim@cisco.com>;
> Beilei Xing <beilei.xing@intel.com>; Jeff Guo <jia.guo@intel.com>; Haiyue
> Wang <haiyue.wang@intel.com>; Matan Azrad <matan@nvidia.com>;
> Shahaf Shuler <shahafs@nvidia.com>; Thomas Monjalon
> <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Andrew
> Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Subject: [PATCH v6 1/9] ethdev: introduce representor type
> 
> To support more representor type, this patch introduces representor type
> enum. The enum is subject to be extended to support new representor in
> patches upcoming.
> 
> For each devarg structure, only one type supported.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
[...]
> diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
> index b3f441c8f7..def669f411 100644
> --- a/drivers/net/enic/enic_ethdev.c
> +++ b/drivers/net/enic/enic_ethdev.c
> @@ -1312,6 +1312,12 @@ static int eth_enic_pci_probe(struct rte_pci_driver
> *pci_drv __rte_unused,
>  		if (retval)
>  			return retval;
>  	}
> +	if (eth_da.nb_representor_ports > 0 &&
> +	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
> +		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
> +			    pci_dev->device.devargs->args);
> +		return -ENOTSUP;
> +	}
>  	retval = rte_eth_dev_create(&pci_dev->device, pci_dev-
> >device.name,
>  		sizeof(struct enic),
>  		eth_dev_pci_specific_init, pci_dev,

For enic,

Acked-by: Hyong Youb Kim <hyonkim@cisco.com>

Thanks.
-Hyong

Re: [dpdk-dev] [PATCH v6 2/9] ethdev: support representor port list

[Andrew Rybchenko]

On 2/14/21 6:21 AM, Xueming Li wrote:
> To support extended representor syntax, this patch extends the
> representor list parsing to support for representor port range in
> devargs, examples:
>    representor=[1,2,3]         - single list
>    representor=[1,3-5,7,9-11]  - list with singles and ranges
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>

Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

Re: [dpdk-dev] [PATCH v6 4/9] ethdev: support sub function representor

[Andrew Rybchenko]

Hi,

On 2/14/21 6:21 AM, Xueming Li wrote:
> SubFunction is a portion of the PCI device, created on demand, a SF
> netdev has its own dedicated queues(txq, rxq). A SF netdev supports
> eswitch representation offload similar to existing PF and VF
> representors.
> 
> To support SF representor, this patch introduces new devargs syntax,
> examples:
>  representor=sf0               - single SubFunction representor
>  representor=sf[1,3,5]         - single list
>  representor=sf[0-3],          - single range
>  representor=sf[0,2-6,8,10-12] - list with singles and ranges
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

What happens if I try to mix different types of representors:
A: -a DBDF,representor=sf0,representor=vf1
B: -a DBDF,representor=sf0 -a DBDF,representor=vf1
(DBDF is the same in args B in both cases).

I'm not trying to say that it must work, since most likely
hotplug will be used to add representors. But behaviour
must be consistent (error?).

The question is raised here, since it is the first patch
where the second type of representor appears.

Andrew.

Re: [dpdk-dev] [PATCH v6 5/9] ethdev: support PF index in representor

[Andrew Rybchenko]

On 2/14/21 6:21 AM, Xueming Li wrote:
> With Kernel bonding, multiple underlying PFs are bonded, VFs come
> from different PF, need to identify representor of VFs unambiguously by
> adding PF index.
> 
> This patch introduces optional 'pf' section to representor devargs
> syntax, examples:
>  representor=pf0vf0             - single VF representor
>  representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs
> 
> PF type representor is supported by using standalone 'pf' section:
>  representor=pf1                - PF representor
> 
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>

Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

Re: [dpdk-dev] [PATCH v6 5/9] ethdev: support PF index in representor

[Andrew Rybchenko]

On 2/14/21 6:21 AM, Xueming Li wrote:
> With Kernel bonding, multiple underlying PFs are bonded, VFs come
> from different PF, need to identify representor of VFs unambiguously by
> adding PF index.
> 
> This patch introduces optional 'pf' section to representor devargs
> syntax, examples:
>  representor=pf0vf0             - single VF representor
>  representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs
> 
> PF type representor is supported by using standalone 'pf' section:
>  representor=pf1                - PF representor
> 
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  doc/guides/prog_guide/poll_mode_drv.rst |  2 ++
>  lib/librte_ethdev/ethdev_private.c      | 18 ++++++++++++++++--
>  lib/librte_ethdev/rte_ethdev.h          |  1 +
>  3 files changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
> index 86e5867f1b..b2147aad30 100644
> --- a/doc/guides/prog_guide/poll_mode_drv.rst
> +++ b/doc/guides/prog_guide/poll_mode_drv.rst
> @@ -382,6 +382,8 @@ parameters to those ports.
>     -a DBDF,representor=sf[1,3,5]
>     -a DBDF,representor=sf[0-1023]
>     -a DBDF,representor=sf[0,2-4,7,9-11]
> +   -a DBDF,representor=pf1vf0
> +   -a DBDF,representor=pf[0-1]sf[0-127]
>  
>  Note: PMDs are not required to support the standard device arguments and users
>  should consult the relevant PMD documentation to see support devargs.
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index 13c191192e..eea0686020 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -118,8 +118,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>   *
>   * Representor format:
>   *   #: range or single number of VF representor - legacy
> - *   vf#: VF port representor/s
> - *   sf#: SF port representor/s
> + *   [pf#]vf#: VF port representor/s
> + *   [pf#]sf#: SF port representor/s
>   *
>   * Examples of #:
>   *  2               - single
> @@ -131,6 +131,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  {
>  	struct rte_eth_devargs *eth_da = data;
>  
> +	if (str[0] == 'p' && str[1] == 'f') {
> +		eth_da->type = RTE_ETH_REPRESENTOR_PF;

Is it actually supported? Above documentation does not say so,
i.e. -a DBDF,representor=pf1 is missing.

> +		str += 2;
> +		str = rte_eth_devargs_process_list(str, eth_da->ports,
> +				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
> +		if (str == NULL)
> +			goto err;
> +	}
>  	if (str[0] == 'v' && str[1] == 'f') {
>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>  		str += 2;
> @@ -138,11 +146,17 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  		eth_da->type = RTE_ETH_REPRESENTOR_SF;
>  		str += 2;
>  	} else {

If PF representors are actually supported, shouldn't it be
if (str[0] != '\0')
plus rte_eth_devargs_process_list() after the body should
be inside in fact?

> +		/* Don't mix legacy syntax with 'pf' section. */
> +		if (eth_da->type == RTE_ETH_REPRESENTOR_PF) {
> +			str = NULL;
> +			goto err;
> +		}
>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>  	}
>  	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>  		&eth_da->nb_representor_ports,
>  		RTE_DIM(eth_da->representor_ports));
> +err:
>  	if (str == NULL)
>  		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
>  	return str == NULL ? -1 : 0;
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 26b5e109c3..9cd519bf59 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -1513,6 +1513,7 @@ enum rte_eth_representor_type {
>  	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
>  	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
>  	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
> +	RTE_ETH_REPRESENTOR_PF,   /**< representor of Physical Function. */
>  };
>  
>  /**
> 

Re: [dpdk-dev] [PATCH v6 6/9] ethdev: support multi-host in representor

[Andrew Rybchenko]

On 2/14/21 6:21 AM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to the multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. To provide the unambiguous identification
> of the PCIe function the controller index is added. The full representor
> identifier consists of three indices - controller index, PF index, and
> VF or SF index (if any).
> 
> This patch introduces controller index to ethdev representor syntax,
> examples:
> 
> [[c#]pf#]vf#: VF port representor/s, example: pf0vf1
> [[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]
> 
> c# is controller(host) ID/range in case of multi-host, optional.
> 
> For user application (e.g. OVS), PMD is responsible to interpret and
> locate representor device based on controller ID, PF ID and VF/SF ID in
> representor syntax.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>

Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

Re: [dpdk-dev] [PATCH v6 7/9] ethdev: new API to get representor info

[Andrew Rybchenko]

On 2/14/21 6:21 AM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> This patch introduces a new API rte_eth_representor_info_get() to
> retrieve representor corresponding info mapping:
>  - caller controller index and pf index.
>  - supported representor ID ranges.
>  - type, controller, pf and start vf/sf ID of each range.
> The API is useful to convert representor from devargs to representor ID.
> 
> New ethdev callback representor_info_get() is added to retrieve info
> from PMD driver, optional for PMD that doesn't support new devargs
> representor syntax.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>

LGTM, except minor notes below.

> ---
>  lib/librte_ethdev/ethdev_driver.h |  6 +++++
>  lib/librte_ethdev/rte_ethdev.c    | 14 ++++++++++
>  lib/librte_ethdev/rte_ethdev.h    | 43 +++++++++++++++++++++++++++++++
>  lib/librte_ethdev/version.map     |  3 +++
>  4 files changed, 66 insertions(+)
> 
> diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
> index 06ff35266f..abcbc3112d 100644
> --- a/lib/librte_ethdev/ethdev_driver.h
> +++ b/lib/librte_ethdev/ethdev_driver.h
> @@ -289,6 +289,10 @@ typedef int (*eth_fw_version_get_t)(struct rte_eth_dev *dev,
>  				     char *fw_version, size_t fw_size);
>  /**< @internal Get firmware information of an Ethernet device. */
>  
> +typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
> +	struct rte_eth_representor_info *info);
> +/**< @internal Get representor type and ID range. */
> +
>  typedef int (*eth_tx_done_cleanup_t)(void *txq, uint32_t free_cnt);
>  /**< @internal Force mbufs to be from TX ring. */
>  
> @@ -823,6 +827,8 @@ struct eth_dev_ops {
>  	eth_burst_mode_get_t       rx_burst_mode_get; /**< Get RX burst mode */
>  	eth_burst_mode_get_t       tx_burst_mode_get; /**< Get TX burst mode */
>  	eth_fw_version_get_t       fw_version_get; /**< Get firmware version. */
> +	eth_representor_info_get_t representor_info_get;
> +	/**< Get representor info. */

Why is it added in the middle of the ops structure?

>  	eth_dev_supported_ptypes_get_t dev_supported_ptypes_get;
>  	/**< Get packet types supported and identified by device. */
>  	eth_dev_ptypes_set_t dev_ptypes_set;
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index fe9466a03e..07c6debb58 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -3265,6 +3265,20 @@ rte_eth_dev_fw_version_get(uint16_t port_id, char *fw_version, size_t fw_size)
>  							fw_version, fw_size));
>  }
>  
> +int
> +rte_eth_representor_info_get(uint16_t port_id,
> +			     struct rte_eth_representor_info *info)
> +{
> +	struct rte_eth_dev *dev;
> +
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> +	dev = &rte_eth_devices[port_id];
> +
> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->fw_version_get, -ENOTSUP);

I guess you mean to check representor_info_get here.

> +	return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev,
> +								      info));
> +}
> +
>  int
>  rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info)
>  {
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 9cd519bf59..35eb0a5721 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -1581,6 +1581,30 @@ struct rte_eth_dev_info {
>  	void *reserved_ptrs[2];   /**< Reserved for future fields */
>  };
>  
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this structure may change without prior notice.
> + *
> + * Ethernet device representor information
> + */
> +struct rte_eth_representor_info {
> +	uint16_t controller; /**< Controller ID of caller device. */
> +	uint16_t pf; /**< Physical function ID of caller device. */
> +	struct {
> +		enum rte_eth_representor_type type; /**< Representor type */
> +		int controller; /**< Controller ID, -1 to ignore */
> +		int pf; /**< Physical function ID, -1 to ignore */
> +		__extension__
> +		union {
> +			int vf; /**< VF start index */
> +			int sf; /**< SF start index */
> +		};
> +		uint16_t id_base; /**< Representor ID start index */
> +		uint16_t id_end;  /**< Representor ID end index */
> +		char name[RTE_DEV_NAME_MAX_LEN];	/**< Representor name */
> +	} ranges[]; /**< Representor ID range by type */

I'm pretty sure that you need separate type for the structure
when you add support, since you need to allocate memory and
calculate required size.

> +};
> +
>  /**
>   * Ethernet device RX queue information structure.
>   * Used to retrieve information about configured queue.
> @@ -3038,6 +3062,25 @@ int rte_eth_macaddr_get(uint16_t port_id, struct rte_ether_addr *mac_addr);
>   */
>  int rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info);
>  
> +/**
> + * Retrieve the representor info of the device.
> + *
> + * @param port_id
> + *   The port identifier of the device.
> + * @param info
> + *   A pointer to a representor info structure.
> + *   NULL to return number of range entries and allocate memory
> + *   for next call to store detail.
> + * @return
> + *   - (-ENOTSUP) if operation is not supported.
> + *   - (-ENODEV) if *port_id* invalid.
> + *   - (-EIO) if device is removed.
> + *   - (>=0) number of representor range entries supported by device.
> + */
> +__rte_experimental
> +int rte_eth_representor_info_get(uint16_t port_id,
> +				 struct rte_eth_representor_info *info);
> +
>  /**
>   * Retrieve the firmware version of a device.
>   *

Re: [dpdk-dev] [PATCH v6 8/9] ethdev: representor iterator compare complete info

[Andrew Rybchenko]

On 2/14/21 6:21 AM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> SR-IOV and SubFunction are created on top of PF. PF index is introduced
> because there might be multiple PFs in the bonding configuration and
> only bonding device is probed.
> 
> In eth representor comparator callback, ethdev representor ID was
> compared with devarg. Since controller index and PF index not compared,
> callback returned representor from other PF or controller.
> 
> This patch adds new API to convert representor controller, pf and vf/sf
> index to representor ID. Representor comparer callback convert
> representor info into ID and compare with device representor ID.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> ---
>  lib/librte_ethdev/ethdev_driver.h | 32 ++++++++++++
>  lib/librte_ethdev/rte_class_eth.c | 38 ++++++++++----
>  lib/librte_ethdev/rte_ethdev.c    | 83 +++++++++++++++++++++++++++++++
>  lib/librte_ethdev/version.map     |  1 +
>  4 files changed, 145 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
> index abcbc3112d..23342f1be2 100644
> --- a/lib/librte_ethdev/ethdev_driver.h
> +++ b/lib/librte_ethdev/ethdev_driver.h
> @@ -1243,6 +1243,38 @@ struct rte_eth_devargs {
>  	enum rte_eth_representor_type type; /* type of representor */
>  };
>  
> +/**
> + * PMD helper function to convert representor ID from location detail
> + *
> + * Convert representor ID from controller, pf and (sf or vf).
> + * The mapping is retrieved from rte_eth_representor_info_get().
> + *
> + * If PMD doesn't return representor range info, simply ignore controller
> + * and pf to keep backward compatibility.

It does not sound right. If controller and/or pf is specified,
it must not be ignored.

> + *
> + * @param ethdev
> + *  Handle of ethdev port.
> + * @param id

May I suggest to name it 'repr_id' to make it less ambguous.

> + *  Pointer to converted representor ID.

I'd prefer do not mix in and out paramters. I suggest to make
it the last parameter.

> + * @param type
> + *  Representor type.
> + * @param controller
> + *  Controller ID, -1 if unspecified.
> + * @param pf
> + *  PF port ID, -1 if unspecified.
> + * @param representor_port
> + *  Representor port ID, -1 if unspecified.

Not sure that I understand what is it? Is it vf or sf number?

> + *
> + * @return
> + *  Negative errno value on error, 0 on success.
> + */
> +__rte_internal
> +int
> +rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
> +			       uint16_t *id,
> +			       enum rte_eth_representor_type type,
> +			       int controller, int pf, int representor_port);
> +
>  /**
>   * PMD helper function to parse ethdev arguments
>   *
> diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
> index 051c892b40..f7b7e659e7 100644
> --- a/lib/librte_ethdev/rte_class_eth.c
> +++ b/lib/librte_ethdev/rte_class_eth.c
> @@ -65,9 +65,10 @@ eth_representor_cmp(const char *key __rte_unused,
>  {
>  	int ret;
>  	char *values;
> -	const struct rte_eth_dev_data *data = opaque;
> -	struct rte_eth_devargs representors;
> -	uint16_t index;
> +	const struct rte_eth_dev *edev = opaque;
> +	const struct rte_eth_dev_data *data = edev->data;
> +	struct rte_eth_devargs eth_da;
> +	uint16_t id, nc, np, nf, i, c, p, f;
>  
>  	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
>  		return -1; /* not a representor port */
> @@ -76,17 +77,36 @@ eth_representor_cmp(const char *key __rte_unused,
>  	values = strdup(value);
>  	if (values == NULL)
>  		return -1;
> -	memset(&representors, 0, sizeof(representors));
> -	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
> +	memset(&eth_da, 0, sizeof(eth_da));
> +	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
>  	free(values);
>  	if (ret != 0)
>  		return -1; /* invalid devargs value */
>  
> +	if (eth_da.nb_mh_controllers == 0 && eth_da.nb_ports == 0 &&
> +	    eth_da.nb_representor_ports == 0)
> +		return -1;
> +	nc = eth_da.nb_mh_controllers > 0 ? eth_da.nb_mh_controllers : 1;
> +	np = eth_da.nb_ports > 0 ? eth_da.nb_ports : 1;
> +	nf = eth_da.nb_representor_ports > 0 ? eth_da.nb_representor_ports : 1;
> +
>  	/* Return 0 if representor id is matching one of the values. */
> -	for (index = 0; index < representors.nb_representor_ports; index++)
> -		if (data->representor_id ==
> -				representors.representor_ports[index])
> +	for (i = 0; i < nc * np * nf; ++i) {
> +		c = i / (np * nf);
> +		p = (i / nf) % np;
> +		f = i % nf;
> +		if (rte_eth_representor_id_convert(edev,
> +			&id,
> +			eth_da.type,
> +			eth_da.nb_mh_controllers == 0 ? -1 :
> +					eth_da.mh_controllers[c],
> +			eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
> +			eth_da.nb_representor_ports == 0 ? -1 :
> +					eth_da.representor_ports[f]) < 0)
> +			continue;
> +		if (data->representor_id == id)
>  			return 0;
> +	}
>  	return -1; /* no match */
>  }
>  
> @@ -112,7 +132,7 @@ eth_dev_match(const struct rte_eth_dev *edev,
>  
>  	ret = rte_kvargs_process(kvlist,
>  			eth_params_keys[RTE_ETH_PARAM_REPRESENTOR],
> -			eth_representor_cmp, edev->data);
> +			eth_representor_cmp, (void *)(uintptr_t)edev);
>  	if (ret != 0)
>  		return -1;
>  	/* search for representor key */
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 07c6debb58..da0cf1a920 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -5617,6 +5617,89 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>  	return result;
>  }
>  
> +int
> +rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
> +			       uint16_t *id,
> +			       enum rte_eth_representor_type type,
> +			       int controller, int pf, int representor_port)
> +{
> +	int ret, n, i, count;
> +	struct rte_eth_representor_info *info = NULL;
> +	size_t size;
> +
> +	if (type == RTE_ETH_REPRESENTOR_NONE)
> +		return 0;
> +	if (id == NULL)
> +		return -EINVAL;
> +
> +	/* Get PMD representor range info. */
> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
> +	if (ret < 0) {
> +		/* Fallback to direct mapping for compatibility. */
> +		*id = representor_port;

I think it is a bad behaviour as I stated above. It is only if
and only if type is VF, controller and PF are unspecified and
representor_port is specified.

> +	}
> +	n = ret;
> +	size = sizeof(*info) + n * sizeof(info->ranges[0]);
> +	info = calloc(1, size);
> +	if (info == NULL)
> +		return -ENOMEM;
> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
> +	if (ret < 0)
> +		goto out;
> +
> +	/* Default controller and pf to caller. */
> +	if (controller == -1)
> +		controller = info->controller;
> +	if (pf == -1)
> +		pf = info->pf;
> +
> +	/* Locate representor ID. */
> +	for (i = 0; i < n; ++i) {
> +		if (info->ranges[i].type != type)
> +			continue;
> +		/* PMD hit: ignore controller if -1. */
> +		if (info->ranges[i].controller != -1 &&
> +		    info->ranges[i].controller != (uint16_t)controller)
> +			continue;

I think it is incorrect to ignore controller in range
if controller is specified in request. It must match.

> +		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;
> +		if (info->ranges[i].type == RTE_ETH_REPRESENTOR_PF) {
> +			/* PF. */
> +			if (pf >= info->ranges[i].pf + count)
> +				continue;
> +			*id = info->ranges[i].id_base +
> +			      (pf - info->ranges[i].pf);
> +			goto out;
> +		}
> +		/* VF or SF. */
> +		/* PMD hit: ignore pf if -1. */
> +		if (info->ranges[i].pf != -1 &&
> +		    info->ranges[i].pf != (uint16_t)pf)
> +			continue;

Same for PF.

> +		if (info->ranges[i].type == RTE_ETH_REPRESENTOR_VF) {

Typically switch/case looks a bit a bitter for such code.

> +			/* VF. */

The comment is useless

> +			if (representor_port >= info->ranges[i].vf + count)
> +				continue;
> +			*id = info->ranges[i].id_base +
> +			      (representor_port - info->ranges[i].vf);
> +			goto out;
> +		} else if (info->ranges[i].type == RTE_ETH_REPRESENTOR_SF) {
> +			/* SF. */

The comment is useless

> +			if (representor_port >= info->ranges[i].sf + count)
> +				continue;
> +			*id = info->ranges[i].id_base +
> +			      (representor_port - info->ranges[i].sf);
> +			goto out;
> +		}
> +	}
> +	/* Not matching representor ID range. */
> +	ret = -ENOENT;
> +
> +out:
> +	if (info != NULL)
> +		free(info);

There is no necessity to check against NULL above, free() does
it in any case.

> +	return ret;
> +}
> +
>  static int
>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>  		const char *params __rte_unused,
> diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
> index bb6f7436c2..2891f5734e 100644
> --- a/lib/librte_ethdev/version.map
> +++ b/lib/librte_ethdev/version.map
> @@ -268,6 +268,7 @@ INTERNAL {
>  	rte_eth_hairpin_queue_peer_bind;
>  	rte_eth_hairpin_queue_peer_unbind;
>  	rte_eth_hairpin_queue_peer_update;
> +	rte_eth_representor_id_convert;
>  	rte_eth_switch_domain_alloc;
>  	rte_eth_switch_domain_free;
>  };
> 

Re: [dpdk-dev] [PATCH v6 4/9] ethdev: support sub function representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Monday, February 15, 2021 4:26 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>
>Subject: Re: [PATCH v6 4/9] ethdev: support sub function representor
>
>Hi,
>
>On 2/14/21 6:21 AM, Xueming Li wrote:
>> SubFunction is a portion of the PCI device, created on demand, a SF
>> netdev has its own dedicated queues(txq, rxq). A SF netdev supports
>> eswitch representation offload similar to existing PF and VF
>> representors.
>>
>> To support SF representor, this patch introduces new devargs syntax,
>> examples:
>>  representor=sf0               - single SubFunction representor
>>  representor=sf[1,3,5]         - single list
>>  representor=sf[0-3],          - single range
>>  representor=sf[0,2-6,8,10-12] - list with singles and ranges
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>
>What happens if I try to mix different types of representors:
>A: -a DBDF,representor=sf0,representor=vf1

Currently, the next representor will overwrite type of first, but append to representor ports.

>B: -a DBDF,representor=sf0 -a DBDF,representor=vf1 (DBDF is the same in args B in both cases).

This is a behavior of EAL, if DBDF are same, only the first get probed, even if the second is "sf1"

>
>I'm not trying to say that it must work, since most likely hotplug will be used to add representors. But behaviour must be consistent
>(error?).

Good catch, agree with error, will add check.

>
>The question is raised here, since it is the first patch where the second type of representor appears.
>
>Andrew.

Re: [dpdk-dev] [PATCH v6 5/9] ethdev: support PF index in representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Monday, February 15, 2021 4:35 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>
>Subject: Re: [PATCH v6 5/9] ethdev: support PF index in representor
>
>On 2/14/21 6:21 AM, Xueming Li wrote:
>> With Kernel bonding, multiple underlying PFs are bonded, VFs come from
>> different PF, need to identify representor of VFs unambiguously by
>> adding PF index.
>>
>> This patch introduces optional 'pf' section to representor devargs
>> syntax, examples:
>>  representor=pf0vf0             - single VF representor
>>  representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs
>>
>> PF type representor is supported by using standalone 'pf' section:
>>  representor=pf1                - PF representor
>>
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>> ---
>>  doc/guides/prog_guide/poll_mode_drv.rst |  2 ++
>>  lib/librte_ethdev/ethdev_private.c      | 18 ++++++++++++++++--
>>  lib/librte_ethdev/rte_ethdev.h          |  1 +
>>  3 files changed, 19 insertions(+), 2 deletions(-)
>>
>> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst
>> b/doc/guides/prog_guide/poll_mode_drv.rst
>> index 86e5867f1b..b2147aad30 100644
>> --- a/doc/guides/prog_guide/poll_mode_drv.rst
>> +++ b/doc/guides/prog_guide/poll_mode_drv.rst
>> @@ -382,6 +382,8 @@ parameters to those ports.
>>     -a DBDF,representor=sf[1,3,5]
>>     -a DBDF,representor=sf[0-1023]
>>     -a DBDF,representor=sf[0,2-4,7,9-11]
>> +   -a DBDF,representor=pf1vf0
>> +   -a DBDF,representor=pf[0-1]sf[0-127]
>>
>>  Note: PMDs are not required to support the standard device arguments
>> and users  should consult the relevant PMD documentation to see support devargs.
>> diff --git a/lib/librte_ethdev/ethdev_private.c
>> b/lib/librte_ethdev/ethdev_private.c
>> index 13c191192e..eea0686020 100644
>> --- a/lib/librte_ethdev/ethdev_private.c
>> +++ b/lib/librte_ethdev/ethdev_private.c
>> @@ -118,8 +118,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>>   *
>>   * Representor format:
>>   *   #: range or single number of VF representor - legacy
>> - *   vf#: VF port representor/s
>> - *   sf#: SF port representor/s
>> + *   [pf#]vf#: VF port representor/s
>> + *   [pf#]sf#: SF port representor/s
>>   *
>>   * Examples of #:
>>   *  2               - single
>> @@ -131,6 +131,14 @@ rte_eth_devargs_parse_representor_ports(char
>> *str, void *data)  {
>>  	struct rte_eth_devargs *eth_da = data;
>>
>> +	if (str[0] == 'p' && str[1] == 'f') {
>> +		eth_da->type = RTE_ETH_REPRESENTOR_PF;
>
>Is it actually supported? Above documentation does not say so, i.e. -a DBDF,representor=pf1 is missing.

Yes, will update.

>
>> +		str += 2;
>> +		str = rte_eth_devargs_process_list(str, eth_da->ports,
>> +				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
>> +		if (str == NULL)
>> +			goto err;
>> +	}
>>  	if (str[0] == 'v' && str[1] == 'f') {
>>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>>  		str += 2;
>> @@ -138,11 +146,17 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>>  		eth_da->type = RTE_ETH_REPRESENTOR_SF;
>>  		str += 2;
>>  	} else {
>
>If PF representors are actually supported, shouldn't it be if (str[0] != '\0') plus rte_eth_devargs_process_list() after the body should be
>inside in fact?

Yes, need this check, I'll add it after 'pf' parsing.
Also, 'c' should be followed by 'pf', 'pf' should followed by 'vf' or 'sf', I'll update check.

>
>> +		/* Don't mix legacy syntax with 'pf' section. */
>> +		if (eth_da->type == RTE_ETH_REPRESENTOR_PF) {
>> +			str = NULL;
>> +			goto err;
>> +		}
>>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>>  	}
>>  	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>>  		&eth_da->nb_representor_ports,
>>  		RTE_DIM(eth_da->representor_ports));
>> +err:
>>  	if (str == NULL)
>>  		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
>>  	return str == NULL ? -1 : 0;
>> diff --git a/lib/librte_ethdev/rte_ethdev.h
>> b/lib/librte_ethdev/rte_ethdev.h index 26b5e109c3..9cd519bf59 100644
>> --- a/lib/librte_ethdev/rte_ethdev.h
>> +++ b/lib/librte_ethdev/rte_ethdev.h
>> @@ -1513,6 +1513,7 @@ enum rte_eth_representor_type {
>>  	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
>>  	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
>>  	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
>> +	RTE_ETH_REPRESENTOR_PF,   /**< representor of Physical Function. */
>>  };
>>
>>  /**
>>

Re: [dpdk-dev] [PATCH v6 7/9] ethdev: new API to get representor info

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Monday, February 15, 2021 4:50 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman
><nhorman@tuxdriver.com>
>Subject: Re: [PATCH v6 7/9] ethdev: new API to get representor info
>
>On 2/14/21 6:21 AM, Xueming Li wrote:
>> The NIC can have multiple PCIe links and can be attached to multiple
>> hosts, for example the same single NIC can be shared for multiple
>> server units in the rack. On each PCIe link NIC can provide multiple
>> PFs and VFs/SFs based on these ones. The full representor identifier
>> consists of three indices - controller index, PF index, and VF or SF index (if any).
>>
>> This patch introduces a new API rte_eth_representor_info_get() to
>> retrieve representor corresponding info mapping:
>>  - caller controller index and pf index.
>>  - supported representor ID ranges.
>>  - type, controller, pf and start vf/sf ID of each range.
>> The API is useful to convert representor from devargs to representor ID.
>>
>> New ethdev callback representor_info_get() is added to retrieve info
>> from PMD driver, optional for PMD that doesn't support new devargs
>> representor syntax.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>
>LGTM, except minor notes below.
>
>> ---
>>  lib/librte_ethdev/ethdev_driver.h |  6 +++++
>>  lib/librte_ethdev/rte_ethdev.c    | 14 ++++++++++
>>  lib/librte_ethdev/rte_ethdev.h    | 43 +++++++++++++++++++++++++++++++
>>  lib/librte_ethdev/version.map     |  3 +++
>>  4 files changed, 66 insertions(+)
>>
>> diff --git a/lib/librte_ethdev/ethdev_driver.h
>> b/lib/librte_ethdev/ethdev_driver.h
>> index 06ff35266f..abcbc3112d 100644
>> --- a/lib/librte_ethdev/ethdev_driver.h
>> +++ b/lib/librte_ethdev/ethdev_driver.h
>> @@ -289,6 +289,10 @@ typedef int (*eth_fw_version_get_t)(struct rte_eth_dev *dev,
>>  				     char *fw_version, size_t fw_size);  /**< @internal Get
>> firmware information of an Ethernet device. */
>>
>> +typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
>> +	struct rte_eth_representor_info *info); /**< @internal Get
>> +representor type and ID range. */
>> +
>>  typedef int (*eth_tx_done_cleanup_t)(void *txq, uint32_t free_cnt);
>> /**< @internal Force mbufs to be from TX ring. */
>>
>> @@ -823,6 +827,8 @@ struct eth_dev_ops {
>>  	eth_burst_mode_get_t       rx_burst_mode_get; /**< Get RX burst mode */
>>  	eth_burst_mode_get_t       tx_burst_mode_get; /**< Get TX burst mode */
>>  	eth_fw_version_get_t       fw_version_get; /**< Get firmware version. */
>> +	eth_representor_info_get_t representor_info_get;
>> +	/**< Get representor info. */
>
>Why is it added in the middle of the ops structure?

Looks like here group of function that getting information. Will move to end.

>
>>  	eth_dev_supported_ptypes_get_t dev_supported_ptypes_get;
>>  	/**< Get packet types supported and identified by device. */
>>  	eth_dev_ptypes_set_t dev_ptypes_set; diff --git
>> a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>> index fe9466a03e..07c6debb58 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -3265,6 +3265,20 @@ rte_eth_dev_fw_version_get(uint16_t port_id, char *fw_version, size_t fw_size)
>>  							fw_version, fw_size));
>>  }
>>
>> +int
>> +rte_eth_representor_info_get(uint16_t port_id,
>> +			     struct rte_eth_representor_info *info) {
>> +	struct rte_eth_dev *dev;
>> +
>> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
>> +	dev = &rte_eth_devices[port_id];
>> +
>> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->fw_version_get, -ENOTSUP);
>
>I guess you mean to check representor_info_get here.

Thanks, my bad.

>
>> +	return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev,
>> +								      info));
>> +}
>> +
>>  int
>>  rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info
>> *dev_info)  { diff --git a/lib/librte_ethdev/rte_ethdev.h
>> b/lib/librte_ethdev/rte_ethdev.h index 9cd519bf59..35eb0a5721 100644
>> --- a/lib/librte_ethdev/rte_ethdev.h
>> +++ b/lib/librte_ethdev/rte_ethdev.h
>> @@ -1581,6 +1581,30 @@ struct rte_eth_dev_info {
>>  	void *reserved_ptrs[2];   /**< Reserved for future fields */
>>  };
>>
>> +/**
>> + * @warning
>> + * @b EXPERIMENTAL: this structure may change without prior notice.
>> + *
>> + * Ethernet device representor information  */ struct
>> +rte_eth_representor_info {
>> +	uint16_t controller; /**< Controller ID of caller device. */
>> +	uint16_t pf; /**< Physical function ID of caller device. */
>> +	struct {
>> +		enum rte_eth_representor_type type; /**< Representor type */
>> +		int controller; /**< Controller ID, -1 to ignore */
>> +		int pf; /**< Physical function ID, -1 to ignore */
>> +		__extension__
>> +		union {
>> +			int vf; /**< VF start index */
>> +			int sf; /**< SF start index */
>> +		};
>> +		uint16_t id_base; /**< Representor ID start index */
>> +		uint16_t id_end;  /**< Representor ID end index */
>> +		char name[RTE_DEV_NAME_MAX_LEN];	/**< Representor name */
>> +	} ranges[]; /**< Representor ID range by type */
>
>I'm pretty sure that you need separate type for the structure when you add support, since you need to allocate memory and calculate
>required size.

Sizeof(info.ranges[0]) works, but splitting looks a good idea :)

>
>> +};
>> +
>>  /**
>>   * Ethernet device RX queue information structure.
>>   * Used to retrieve information about configured queue.
>> @@ -3038,6 +3062,25 @@ int rte_eth_macaddr_get(uint16_t port_id, struct rte_ether_addr *mac_addr);
>>   */
>>  int rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info
>> *dev_info);
>>
>> +/**
>> + * Retrieve the representor info of the device.
>> + *
>> + * @param port_id
>> + *   The port identifier of the device.
>> + * @param info
>> + *   A pointer to a representor info structure.
>> + *   NULL to return number of range entries and allocate memory
>> + *   for next call to store detail.
>> + * @return
>> + *   - (-ENOTSUP) if operation is not supported.
>> + *   - (-ENODEV) if *port_id* invalid.
>> + *   - (-EIO) if device is removed.
>> + *   - (>=0) number of representor range entries supported by device.
>> + */
>> +__rte_experimental
>> +int rte_eth_representor_info_get(uint16_t port_id,
>> +				 struct rte_eth_representor_info *info);
>> +
>>  /**
>>   * Retrieve the firmware version of a device.
>>   *

Re: [dpdk-dev] [PATCH v6 8/9] ethdev: representor iterator compare complete info

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Monday, February 15, 2021 5:32 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman
><nhorman@tuxdriver.com>
>Subject: Re: [PATCH v6 8/9] ethdev: representor iterator compare complete info
>
>On 2/14/21 6:21 AM, Xueming Li wrote:
>> The NIC can have multiple PCIe links and can be attached to multiple
>> hosts, for example the same single NIC can be shared for multiple
>> server units in the rack. On each PCIe link NIC can provide multiple
>> PFs and VFs/SFs based on these ones. The full representor identifier
>> consists of three indices - controller index, PF index, and VF or SF index (if any).
>>
>> SR-IOV and SubFunction are created on top of PF. PF index is
>> introduced because there might be multiple PFs in the bonding
>> configuration and only bonding device is probed.
>>
>> In eth representor comparator callback, ethdev representor ID was
>> compared with devarg. Since controller index and PF index not
>> compared, callback returned representor from other PF or controller.
>>
>> This patch adds new API to convert representor controller, pf and
>> vf/sf index to representor ID. Representor comparer callback convert
>> representor info into ID and compare with device representor ID.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> ---
>>  lib/librte_ethdev/ethdev_driver.h | 32 ++++++++++++
>> lib/librte_ethdev/rte_class_eth.c | 38 ++++++++++----
>>  lib/librte_ethdev/rte_ethdev.c    | 83 +++++++++++++++++++++++++++++++
>>  lib/librte_ethdev/version.map     |  1 +
>>  4 files changed, 145 insertions(+), 9 deletions(-)
>>
>> diff --git a/lib/librte_ethdev/ethdev_driver.h
>> b/lib/librte_ethdev/ethdev_driver.h
>> index abcbc3112d..23342f1be2 100644
>> --- a/lib/librte_ethdev/ethdev_driver.h
>> +++ b/lib/librte_ethdev/ethdev_driver.h
>> @@ -1243,6 +1243,38 @@ struct rte_eth_devargs {
>>  	enum rte_eth_representor_type type; /* type of representor */  };
>>
>> +/**
>> + * PMD helper function to convert representor ID from location detail
>> + *
>> + * Convert representor ID from controller, pf and (sf or vf).
>> + * The mapping is retrieved from rte_eth_representor_info_get().
>> + *
>> + * If PMD doesn't return representor range info, simply ignore
>> +controller
>> + * and pf to keep backward compatibility.
>
>It does not sound right. If controller and/or pf is specified, it must not be ignored.

Okay, return error if info not found.

>
>> + *
>> + * @param ethdev
>> + *  Handle of ethdev port.
>> + * @param id
>
>May I suggest to name it 'repr_id' to make it less ambguous.

Accept.

>
>> + *  Pointer to converted representor ID.
>
>I'd prefer do not mix in and out paramters. I suggest to make it the last parameter.
>
>> + * @param type
>> + *  Representor type.
>> + * @param controller
>> + *  Controller ID, -1 if unspecified.
>> + * @param pf
>> + *  PF port ID, -1 if unspecified.
>> + * @param representor_port
>> + *  Representor port ID, -1 if unspecified.
>
>Not sure that I understand what is it? Is it vf or sf number?

Yes, will make it clear.

>
>> + *
>> + * @return
>> + *  Negative errno value on error, 0 on success.
>> + */
>> +__rte_internal
>> +int
>> +rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
>> +			       uint16_t *id,
>> +			       enum rte_eth_representor_type type,
>> +			       int controller, int pf, int representor_port);
>> +
>>  /**
>>   * PMD helper function to parse ethdev arguments
>>   *
>> diff --git a/lib/librte_ethdev/rte_class_eth.c
>> b/lib/librte_ethdev/rte_class_eth.c
>> index 051c892b40..f7b7e659e7 100644
>> --- a/lib/librte_ethdev/rte_class_eth.c
>> +++ b/lib/librte_ethdev/rte_class_eth.c
>> @@ -65,9 +65,10 @@ eth_representor_cmp(const char *key __rte_unused,
>> {
>>  	int ret;
>>  	char *values;
>> -	const struct rte_eth_dev_data *data = opaque;
>> -	struct rte_eth_devargs representors;
>> -	uint16_t index;
>> +	const struct rte_eth_dev *edev = opaque;
>> +	const struct rte_eth_dev_data *data = edev->data;
>> +	struct rte_eth_devargs eth_da;
>> +	uint16_t id, nc, np, nf, i, c, p, f;
>>
>>  	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
>>  		return -1; /* not a representor port */ @@ -76,17 +77,36 @@
>> eth_representor_cmp(const char *key __rte_unused,
>>  	values = strdup(value);
>>  	if (values == NULL)
>>  		return -1;
>> -	memset(&representors, 0, sizeof(representors));
>> -	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
>> +	memset(&eth_da, 0, sizeof(eth_da));
>> +	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
>>  	free(values);
>>  	if (ret != 0)
>>  		return -1; /* invalid devargs value */
>>
>> +	if (eth_da.nb_mh_controllers == 0 && eth_da.nb_ports == 0 &&
>> +	    eth_da.nb_representor_ports == 0)
>> +		return -1;
>> +	nc = eth_da.nb_mh_controllers > 0 ? eth_da.nb_mh_controllers : 1;
>> +	np = eth_da.nb_ports > 0 ? eth_da.nb_ports : 1;
>> +	nf = eth_da.nb_representor_ports > 0 ? eth_da.nb_representor_ports :
>> +1;
>> +
>>  	/* Return 0 if representor id is matching one of the values. */
>> -	for (index = 0; index < representors.nb_representor_ports; index++)
>> -		if (data->representor_id ==
>> -				representors.representor_ports[index])
>> +	for (i = 0; i < nc * np * nf; ++i) {
>> +		c = i / (np * nf);
>> +		p = (i / nf) % np;
>> +		f = i % nf;
>> +		if (rte_eth_representor_id_convert(edev,
>> +			&id,
>> +			eth_da.type,
>> +			eth_da.nb_mh_controllers == 0 ? -1 :
>> +					eth_da.mh_controllers[c],
>> +			eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
>> +			eth_da.nb_representor_ports == 0 ? -1 :
>> +					eth_da.representor_ports[f]) < 0)
>> +			continue;
>> +		if (data->representor_id == id)
>>  			return 0;
>> +	}
>>  	return -1; /* no match */
>>  }
>>
>> @@ -112,7 +132,7 @@ eth_dev_match(const struct rte_eth_dev *edev,
>>
>>  	ret = rte_kvargs_process(kvlist,
>>  			eth_params_keys[RTE_ETH_PARAM_REPRESENTOR],
>> -			eth_representor_cmp, edev->data);
>> +			eth_representor_cmp, (void *)(uintptr_t)edev);
>>  	if (ret != 0)
>>  		return -1;
>>  	/* search for representor key */
>> diff --git a/lib/librte_ethdev/rte_ethdev.c
>> b/lib/librte_ethdev/rte_ethdev.c index 07c6debb58..da0cf1a920 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -5617,6 +5617,89 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>>  	return result;
>>  }
>>
>> +int
>> +rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
>> +			       uint16_t *id,
>> +			       enum rte_eth_representor_type type,
>> +			       int controller, int pf, int representor_port) {
>> +	int ret, n, i, count;
>> +	struct rte_eth_representor_info *info = NULL;
>> +	size_t size;
>> +
>> +	if (type == RTE_ETH_REPRESENTOR_NONE)
>> +		return 0;
>> +	if (id == NULL)
>> +		return -EINVAL;
>> +
>> +	/* Get PMD representor range info. */
>> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
>> +	if (ret < 0) {
>> +		/* Fallback to direct mapping for compatibility. */
>> +		*id = representor_port;
>
>I think it is a bad behaviour as I stated above. It is only if and only if type is VF, controller and PF are unspecified and representor_port
>is specified.

Agree, will make it only valid for legacy VF representor.

>
>> +	}
>> +	n = ret;
>> +	size = sizeof(*info) + n * sizeof(info->ranges[0]);
>> +	info = calloc(1, size);
>> +	if (info == NULL)
>> +		return -ENOMEM;
>> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
>> +	if (ret < 0)
>> +		goto out;
>> +
>> +	/* Default controller and pf to caller. */
>> +	if (controller == -1)
>> +		controller = info->controller;
>> +	if (pf == -1)
>> +		pf = info->pf;
>> +
>> +	/* Locate representor ID. */
>> +	for (i = 0; i < n; ++i) {
>> +		if (info->ranges[i].type != type)
>> +			continue;
>> +		/* PMD hit: ignore controller if -1. */
>> +		if (info->ranges[i].controller != -1 &&
>> +		    info->ranges[i].controller != (uint16_t)controller)
>> +			continue;
>
>I think it is incorrect to ignore controller in range if controller is specified in request. It must match.

This is a real requirement I'm facing from orchestration, before orchestration having knowledge on port
status, it always send "pf#vf#" to OVS->DPDK, although "pf#" is meaningless in standard mode. It will take
time for orchestration and OVS to evolve... PMD uses this option to decide ignore them or not.

>
>> +		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;
>> +		if (info->ranges[i].type == RTE_ETH_REPRESENTOR_PF) {
>> +			/* PF. */
>> +			if (pf >= info->ranges[i].pf + count)
>> +				continue;
>> +			*id = info->ranges[i].id_base +
>> +			      (pf - info->ranges[i].pf);
>> +			goto out;
>> +		}
>> +		/* VF or SF. */
>> +		/* PMD hit: ignore pf if -1. */
>> +		if (info->ranges[i].pf != -1 &&
>> +		    info->ranges[i].pf != (uint16_t)pf)
>> +			continue;
>
>Same for PF.
>
>> +		if (info->ranges[i].type == RTE_ETH_REPRESENTOR_VF) {
>
>Typically switch/case looks a bit a bitter for such code.

'bitter'? :) will update.

>
>> +			/* VF. */
>
>The comment is useless

Accept this one and bellows.

>
>> +			if (representor_port >= info->ranges[i].vf + count)
>> +				continue;
>> +			*id = info->ranges[i].id_base +
>> +			      (representor_port - info->ranges[i].vf);
>> +			goto out;
>> +		} else if (info->ranges[i].type == RTE_ETH_REPRESENTOR_SF) {
>> +			/* SF. */
>
>The comment is useless
>
>> +			if (representor_port >= info->ranges[i].sf + count)
>> +				continue;
>> +			*id = info->ranges[i].id_base +
>> +			      (representor_port - info->ranges[i].sf);
>> +			goto out;
>> +		}
>> +	}
>> +	/* Not matching representor ID range. */
>> +	ret = -ENOENT;
>> +
>> +out:
>> +	if (info != NULL)
>> +		free(info);
>
>There is no necessity to check against NULL above, free() does it in any case.
>
>> +	return ret;
>> +}
>> +
>>  static int
>>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>>  		const char *params __rte_unused,
>> diff --git a/lib/librte_ethdev/version.map
>> b/lib/librte_ethdev/version.map index bb6f7436c2..2891f5734e 100644
>> --- a/lib/librte_ethdev/version.map
>> +++ b/lib/librte_ethdev/version.map
>> @@ -268,6 +268,7 @@ INTERNAL {
>>  	rte_eth_hairpin_queue_peer_bind;
>>  	rte_eth_hairpin_queue_peer_unbind;
>>  	rte_eth_hairpin_queue_peer_update;
>> +	rte_eth_representor_id_convert;
>>  	rte_eth_switch_domain_alloc;
>>  	rte_eth_switch_domain_free;
>>  };
>>

Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor

[Stephen Hemminger]

On Sun, 14 Feb 2021 03:21:30 +0000
Xueming Li <xuemingl@nvidia.com> wrote:

> SubFunction [1] is a portion of the PCI device, a SF netdev has its own
> dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
> offload similar to existing PF and VF representors. A SF shares PCI
> level resources with other SFs and/or with its parent PCI function.
> 
> From SmartNIC perspective, when PCI device is shared for multi-host,
> representors for host controller and host PF is required.
> 
> This patch set introduces new representor types in addtion to existing
> VF representor. Syntax:
> 
> [[c#]pf#]vf#: VF port representor/s from controller/pf
> [[c#]pf#]sf#: SF port representor/s from controller/pf
> #: VF representor - for backwards compatibility
> 
> "#" is number instance, list or range, valid examples:
>   1, [1,3,5], [0-3], [0,2-4,6]
> 
> For backward compatibility, this patch also introduces new netdev
> capability to indicate the capability of supportting SF representor.
> 
> Version history:
>  RFC:
>  	initial version [2]
>  V2:
>     - separate patch for represnetor infrastructure, controller, pf and
>       sf.
>     - replace representor ID macro with functions:
>       rte_eth_representor_id_encode()
>       rte_eth_representor_id_parse()
>     - new patch to allow devargs with same PCI BDF but different
>       representors.
>     - other minor code updates according to comments, thanks Andrew!
>     - update document
>  V3:
>     - improve probing of allowed devargs with same name.
>     - parse single word of kvargs as key.
>     - update kvargs test cases.
>  V4:
>     - split first representor refactor patch into
>       1: add representor type
>       2: refector representor list parsing
>     - push the patch supporting multi-devargs for same device.
>  V5:
>     - add comments for parsing functions
>     - update switch_representation.rst - Thanks Ajit
>  V6:
>     - split representor types into different patches, move to
>       rte_ethdev.h
>     - improvements of rte_eth_devargs_process_list() according to
>       Andrew's suggestion
>     - fixed PF probe failure for Intel i40e
>     - replace ethdev SF capability with rte_eth_representor_info_get()
>     - add new ethdev ops api to get representor info from PMD
>     - replace representor ID encode/decode with conversion from
>       representor info
>     - change ethdev representor iterator to use new ID encoding
> 
> 
> Xueming Li (9):
>   ethdev: introduce representor type
>   ethdev: support representor port list
>   ethdev: support new VF representor syntax
>   ethdev: support sub function representor
>   ethdev: support PF index in representor
>   ethdev: support multi-host in representor
>   ethdev: new API to get representor info
>   ethdev: representor iterator compare complete info
>   kvargs: update parser to support lists
> 
>  app/test/test_kvargs.c                        |  46 ++++-
>  config/rte_config.h                           |   1 +
>  doc/guides/prog_guide/poll_mode_drv.rst       |  13 +-
>  .../prog_guide/switch_representation.rst      |  35 +++-
>  drivers/net/bnxt/bnxt_ethdev.c                |   7 +
>  drivers/net/enic/enic_ethdev.c                |   6 +
>  drivers/net/i40e/i40e_ethdev.c                |   7 +
>  drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
>  drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
>  lib/librte_ethdev/ethdev_driver.h             |  49 ++++-
>  lib/librte_ethdev/ethdev_private.c            | 173 ++++++++++++------
>  lib/librte_ethdev/ethdev_private.h            |   3 -
>  lib/librte_ethdev/rte_class_eth.c             |  40 ++--
>  lib/librte_ethdev/rte_ethdev.c                | 102 ++++++++++-
>  lib/librte_ethdev/rte_ethdev.h                |  53 ++++++
>  lib/librte_ethdev/version.map                 |   4 +
>  lib/librte_kvargs/rte_kvargs.c                | 101 +++++++---
>  17 files changed, 535 insertions(+), 123 deletions(-)
> 

A couple of higher level design questions:
1. How does Linux and other OS handle this in their API?
2. This solution seems quite tied into a specific implementation on your hardware.
   Is there a PCI standard for this?
3. Maybe a more general solution where these were represented as buses would
   allow for other connection methods in the future?

Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor

[Wang, Haiyue]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger
> Sent: Tuesday, February 23, 2021 09:55
> To: Xueming Li <xuemingl@nvidia.com>
> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
> Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor
> 
> On Sun, 14 Feb 2021 03:21:30 +0000
> Xueming Li <xuemingl@nvidia.com> wrote:
> 
> > SubFunction [1] is a portion of the PCI device, a SF netdev has its own
> > dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
> > offload similar to existing PF and VF representors. A SF shares PCI
> > level resources with other SFs and/or with its parent PCI function.
> >
> > From SmartNIC perspective, when PCI device is shared for multi-host,
> > representors for host controller and host PF is required.
> >
> > This patch set introduces new representor types in addtion to existing
> > VF representor. Syntax:
> >
> > [[c#]pf#]vf#: VF port representor/s from controller/pf
> > [[c#]pf#]sf#: SF port representor/s from controller/pf
> > #: VF representor - for backwards compatibility
> >
> > "#" is number instance, list or range, valid examples:
> >   1, [1,3,5], [0-3], [0,2-4,6]
> >
> > For backward compatibility, this patch also introduces new netdev
> > capability to indicate the capability of supportting SF representor.
> >
> > Version history:
> >  RFC:
> >  	initial version [2]
> >  V2:
> >     - separate patch for represnetor infrastructure, controller, pf and
> >       sf.
> >     - replace representor ID macro with functions:
> >       rte_eth_representor_id_encode()
> >       rte_eth_representor_id_parse()
> >     - new patch to allow devargs with same PCI BDF but different
> >       representors.
> >     - other minor code updates according to comments, thanks Andrew!
> >     - update document
> >  V3:
> >     - improve probing of allowed devargs with same name.
> >     - parse single word of kvargs as key.
> >     - update kvargs test cases.
> >  V4:
> >     - split first representor refactor patch into
> >       1: add representor type
> >       2: refector representor list parsing
> >     - push the patch supporting multi-devargs for same device.
> >  V5:
> >     - add comments for parsing functions
> >     - update switch_representation.rst - Thanks Ajit
> >  V6:
> >     - split representor types into different patches, move to
> >       rte_ethdev.h
> >     - improvements of rte_eth_devargs_process_list() according to
> >       Andrew's suggestion
> >     - fixed PF probe failure for Intel i40e
> >     - replace ethdev SF capability with rte_eth_representor_info_get()
> >     - add new ethdev ops api to get representor info from PMD
> >     - replace representor ID encode/decode with conversion from
> >       representor info
> >     - change ethdev representor iterator to use new ID encoding
> >
> >
> > Xueming Li (9):
> >   ethdev: introduce representor type
> >   ethdev: support representor port list
> >   ethdev: support new VF representor syntax
> >   ethdev: support sub function representor
> >   ethdev: support PF index in representor
> >   ethdev: support multi-host in representor
> >   ethdev: new API to get representor info
> >   ethdev: representor iterator compare complete info
> >   kvargs: update parser to support lists
> >
> >  app/test/test_kvargs.c                        |  46 ++++-
> >  config/rte_config.h                           |   1 +
> >  doc/guides/prog_guide/poll_mode_drv.rst       |  13 +-
> >  .../prog_guide/switch_representation.rst      |  35 +++-
> >  drivers/net/bnxt/bnxt_ethdev.c                |   7 +
> >  drivers/net/enic/enic_ethdev.c                |   6 +
> >  drivers/net/i40e/i40e_ethdev.c                |   7 +
> >  drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
> >  drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
> >  lib/librte_ethdev/ethdev_driver.h             |  49 ++++-
> >  lib/librte_ethdev/ethdev_private.c            | 173 ++++++++++++------
> >  lib/librte_ethdev/ethdev_private.h            |   3 -
> >  lib/librte_ethdev/rte_class_eth.c             |  40 ++--
> >  lib/librte_ethdev/rte_ethdev.c                | 102 ++++++++++-
> >  lib/librte_ethdev/rte_ethdev.h                |  53 ++++++
> >  lib/librte_ethdev/version.map                 |   4 +
> >  lib/librte_kvargs/rte_kvargs.c                | 101 +++++++---
> >  17 files changed, 535 insertions(+), 123 deletions(-)
> >
> 
> A couple of higher level design questions:
> 1. How does Linux and other OS handle this in their API?
> 2. This solution seems quite tied into a specific implementation on your hardware.
>    Is there a PCI standard for this?

> 3. Maybe a more general solution where these were represented as buses would
>    allow for other connection methods in the future?

It should be "auxiliary bus", I think. ;-)
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c

mlx5 subfunction support
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9becc8799241716672f6

Re: [dpdk-dev] [PATCH v6 8/9] ethdev: representor iterator compare complete info

[Andrew Rybchenko]

On 2/16/21 7:35 PM, Xueming(Steven) Li wrote:
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Monday, February 15, 2021 5:32 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
>> <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman
>> <nhorman@tuxdriver.com>
>> Subject: Re: [PATCH v6 8/9] ethdev: representor iterator compare complete info
>>
>> On 2/14/21 6:21 AM, Xueming Li wrote:
>>> The NIC can have multiple PCIe links and can be attached to multiple
>>> hosts, for example the same single NIC can be shared for multiple
>>> server units in the rack. On each PCIe link NIC can provide multiple
>>> PFs and VFs/SFs based on these ones. The full representor identifier
>>> consists of three indices - controller index, PF index, and VF or SF index (if any).
>>>
>>> SR-IOV and SubFunction are created on top of PF. PF index is
>>> introduced because there might be multiple PFs in the bonding
>>> configuration and only bonding device is probed.
>>>
>>> In eth representor comparator callback, ethdev representor ID was
>>> compared with devarg. Since controller index and PF index not
>>> compared, callback returned representor from other PF or controller.
>>>
>>> This patch adds new API to convert representor controller, pf and
>>> vf/sf index to representor ID. Representor comparer callback convert
>>> representor info into ID and compare with device representor ID.
>>>
>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>

[snip]

>>> +	}
>>> +	n = ret;
>>> +	size = sizeof(*info) + n * sizeof(info->ranges[0]);
>>> +	info = calloc(1, size);
>>> +	if (info == NULL)
>>> +		return -ENOMEM;
>>> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
>>> +	if (ret < 0)
>>> +		goto out;
>>> +
>>> +	/* Default controller and pf to caller. */
>>> +	if (controller == -1)
>>> +		controller = info->controller;
>>> +	if (pf == -1)
>>> +		pf = info->pf;
>>> +
>>> +	/* Locate representor ID. */
>>> +	for (i = 0; i < n; ++i) {
>>> +		if (info->ranges[i].type != type)
>>> +			continue;
>>> +		/* PMD hit: ignore controller if -1. */
>>> +		if (info->ranges[i].controller != -1 &&
>>> +		    info->ranges[i].controller != (uint16_t)controller)
>>> +			continue;
>>
>> I think it is incorrect to ignore controller in range if controller is specified in request. It must match.
> 
> This is a real requirement I'm facing from orchestration, before orchestration having knowledge on port
> status, it always send "pf#vf#" to OVS->DPDK, although "pf#" is meaningless in standard mode. It will take
> time for orchestration and OVS to evolve... PMD uses this option to decide ignore them or not.

I'll take a look at it on the next version, but it still sounds
wrong when something is specified, but ignored. May be I simply
misunderstand.

Thanks for working on the patch series.

Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Wang, Haiyue <haiyue.wang@intel.com>
>Sent: Tuesday, February 23, 2021 7:46 PM
>To: Stephen Hemminger <stephen@networkplumber.org>; Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
>Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor
>
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger
>> Sent: Tuesday, February 23, 2021 09:55
>> To: Xueming Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Asaf
>> Penso <asafp@nvidia.com>
>> Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction
>> representor
>>
>> On Sun, 14 Feb 2021 03:21:30 +0000
>> Xueming Li <xuemingl@nvidia.com> wrote:
>>
>> > SubFunction [1] is a portion of the PCI device, a SF netdev has its
>> > own dedicated queues(txq, rxq). A SF netdev supports E-Switch
>> > representation offload similar to existing PF and VF representors. A
>> > SF shares PCI level resources with other SFs and/or with its parent PCI function.
>> >
>> > From SmartNIC perspective, when PCI device is shared for multi-host,
>> > representors for host controller and host PF is required.
>> >
>> > This patch set introduces new representor types in addtion to
>> > existing VF representor. Syntax:
>> >
>> > [[c#]pf#]vf#: VF port representor/s from controller/pf
>> > [[c#]pf#]sf#: SF port representor/s from controller/pf
>> > #: VF representor - for backwards compatibility
>> >
>> > "#" is number instance, list or range, valid examples:
>> >   1, [1,3,5], [0-3], [0,2-4,6]
>> >
>> > For backward compatibility, this patch also introduces new netdev
>> > capability to indicate the capability of supportting SF representor.
>> >
>> > Version history:
>> >  RFC:
>> >  	initial version [2]
>> >  V2:
>> >     - separate patch for represnetor infrastructure, controller, pf and
>> >       sf.
>> >     - replace representor ID macro with functions:
>> >       rte_eth_representor_id_encode()
>> >       rte_eth_representor_id_parse()
>> >     - new patch to allow devargs with same PCI BDF but different
>> >       representors.
>> >     - other minor code updates according to comments, thanks Andrew!
>> >     - update document
>> >  V3:
>> >     - improve probing of allowed devargs with same name.
>> >     - parse single word of kvargs as key.
>> >     - update kvargs test cases.
>> >  V4:
>> >     - split first representor refactor patch into
>> >       1: add representor type
>> >       2: refector representor list parsing
>> >     - push the patch supporting multi-devargs for same device.
>> >  V5:
>> >     - add comments for parsing functions
>> >     - update switch_representation.rst - Thanks Ajit
>> >  V6:
>> >     - split representor types into different patches, move to
>> >       rte_ethdev.h
>> >     - improvements of rte_eth_devargs_process_list() according to
>> >       Andrew's suggestion
>> >     - fixed PF probe failure for Intel i40e
>> >     - replace ethdev SF capability with rte_eth_representor_info_get()
>> >     - add new ethdev ops api to get representor info from PMD
>> >     - replace representor ID encode/decode with conversion from
>> >       representor info
>> >     - change ethdev representor iterator to use new ID encoding
>> >
>> >
>> > Xueming Li (9):
>> >   ethdev: introduce representor type
>> >   ethdev: support representor port list
>> >   ethdev: support new VF representor syntax
>> >   ethdev: support sub function representor
>> >   ethdev: support PF index in representor
>> >   ethdev: support multi-host in representor
>> >   ethdev: new API to get representor info
>> >   ethdev: representor iterator compare complete info
>> >   kvargs: update parser to support lists
>> >
>> >  app/test/test_kvargs.c                        |  46 ++++-
>> >  config/rte_config.h                           |   1 +
>> >  doc/guides/prog_guide/poll_mode_drv.rst       |  13 +-
>> >  .../prog_guide/switch_representation.rst      |  35 +++-
>> >  drivers/net/bnxt/bnxt_ethdev.c                |   7 +
>> >  drivers/net/enic/enic_ethdev.c                |   6 +
>> >  drivers/net/i40e/i40e_ethdev.c                |   7 +
>> >  drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
>> >  drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
>> >  lib/librte_ethdev/ethdev_driver.h             |  49 ++++-
>> >  lib/librte_ethdev/ethdev_private.c            | 173 ++++++++++++------
>> >  lib/librte_ethdev/ethdev_private.h            |   3 -
>> >  lib/librte_ethdev/rte_class_eth.c             |  40 ++--
>> >  lib/librte_ethdev/rte_ethdev.c                | 102 ++++++++++-
>> >  lib/librte_ethdev/rte_ethdev.h                |  53 ++++++
>> >  lib/librte_ethdev/version.map                 |   4 +
>> >  lib/librte_kvargs/rte_kvargs.c                | 101 +++++++---
>> >  17 files changed, 535 insertions(+), 123 deletions(-)
>> >
>>
>> A couple of higher level design questions:
>> 1. How does Linux and other OS handle this in their API?

Hi Stephen, thanks for looking this!
Representor was a type of ethdev, OS independent.
SF representor is a new type besides existing VF representor type.

>> 2. This solution seems quite tied into a specific implementation on your hardware.
>>    Is there a PCI standard for this?

SF has been carved in kernel, supported by devlink. The devlink document in the link Haiyue posted should be helpful to understand the overall picture:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9becc8799241716672f6

>
>> 3. Maybe a more general solution where these were represented as buses would
>>    allow for other connection methods in the future?
>
>It should be "auxiliary bus", I think. ;-) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c

Good link, Mellanox has exactly implemented SF based auxiliary bus, I'm going to post another RFC to support auxiliary bus.
But this patchset still needed to support representor of SF.

>
>mlx5 subfunction support
>https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9becc8799241716672f6

[dpdk-dev] [PATCH v7 0/9] ethdev: support SubFunction representor

[Xueming Li]

SubFunction [1] is a portion of the PCI device, a SF netdev has its own
dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to existing
VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For backward compatibility, this patch also introduces new netdev
capability to indicate the capability of supportting SF representor.

Version history:
 RFC:
 	initial version [2]
 V2:
    - separate patch for represnetor infrastructure, controller, pf and
      sf.
    - replace representor ID macro with functions:
      rte_eth_representor_id_encode()
      rte_eth_representor_id_parse()
    - new patch to allow devargs with same PCI BDF but different
      representors.
    - other minor code updates according to comments, thanks Andrew!
    - update document
 V3:
    - improve probing of allowed devargs with same name.
    - parse single word of kvargs as key.
    - update kvargs test cases.
 V4:
    - split first representor refactor patch into
      1: add representor type
      2: refector representor list parsing
    - push the patch supporting multi-devargs for same device.
 V5:
    - add comments for parsing functions
    - update switch_representation.rst - Thanks Ajit
 V6:
    - split representor types into different patches, move to
      rte_ethdev.h
    - improvements of rte_eth_devargs_process_list() according to
      Andrew's suggestion
    - fixed PF probe failure for Intel i40e
    - replace ethdev SF capability with rte_eth_representor_info_get()
    - add new ethdev ops api to get representor info from PMD
    - replace representor ID encode/decode with conversion from
      representor info
    - change ethdev representor iterator to use new ID encoding
 V7:
    - error on duplicated representor argument:
      DBDF,representor=a,representor=b
    - fix pf representor type parsing
    - other small fixes suggested by Andrew. 
    


[1] SubFunction in kernel:
https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/

[2] RFC:
http://patchwork.dpdk.org/project/dpdk/list/?series=14376

[3] V2:
http://patchwork.dpdk.org/project/dpdk/list/?series=14559

[4] V3:
1 http://patchwork.dpdk.org/patch/86460/
2 http://patchwork.dpdk.org/patch/86461/
3 http://patchwork.dpdk.org/patch/86462/
4 http://patchwork.dpdk.org/patch/86463/
5 http://patchwork.dpdk.org/patch/86464/
6 http://patchwork.dpdk.org/patch/86465/
7 http://patchwork.dpdk.org/patch/86467/
8 http://patchwork.dpdk.org/patch/86466/
9 http://patchwork.dpdk.org/patch/86468/

[5] V4:
http://patchwork.dpdk.org/project/dpdk/list/?series=14809

[6] V5:
http://patchwork.dpdk.org/project/dpdk/list/?series=14834

[7] V6:
http://patchwork.dpdk.org/project/dpdk/list/?series=15258

Xueming Li (9):
  ethdev: introduce representor type
  ethdev: support representor port list
  ethdev: support new VF representor syntax
  ethdev: support sub function representor
  ethdev: support PF index in representor
  ethdev: support multi-host in representor
  ethdev: new API to get representor info
  ethdev: representor iterator compare complete info
  kvargs: update parser to support lists

 app/test/test_kvargs.c                        |  46 ++++-
 config/rte_config.h                           |   1 +
 doc/guides/prog_guide/poll_mode_drv.rst       |  18 +-
 .../prog_guide/switch_representation.rst      |  35 ++--
 drivers/net/bnxt/bnxt_ethdev.c                |   7 +
 drivers/net/enic/enic_ethdev.c                |   6 +
 drivers/net/i40e/i40e_ethdev.c                |   7 +
 drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
 drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
 lib/librte_ethdev/ethdev_driver.h             |  50 +++++-
 lib/librte_ethdev/ethdev_private.c            | 169 ++++++++++++------
 lib/librte_ethdev/ethdev_private.h            |   3 -
 lib/librte_ethdev/rte_class_eth.c             |  40 +++--
 lib/librte_ethdev/rte_ethdev.c                | 116 +++++++++++-
 lib/librte_ethdev/rte_ethdev.h                |  61 +++++++
 lib/librte_ethdev/version.map                 |   4 +
 lib/librte_kvargs/rte_kvargs.c                | 101 ++++++++---
 17 files changed, 557 insertions(+), 125 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH v7 1/9] ethdev: introduce representor type

[Xueming Li]

To support more representor type, this patch introduces representor type
enum. The enum is subject to be extended to support new representor in
patches upcoming.

For each devarg structure, only one type supported.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Hyong Youb Kim <hyonkim@cisco.com>
---
 drivers/net/bnxt/bnxt_ethdev.c     |  7 +++++++
 drivers/net/enic/enic_ethdev.c     |  6 ++++++
 drivers/net/i40e/i40e_ethdev.c     |  7 +++++++
 drivers/net/ixgbe/ixgbe_ethdev.c   |  7 +++++++
 drivers/net/mlx5/linux/mlx5_os.c   | 11 +++++++++++
 lib/librte_ethdev/ethdev_driver.h  |  7 ++++++-
 lib/librte_ethdev/ethdev_private.c |  5 +++++
 lib/librte_ethdev/rte_ethdev.h     |  8 ++++++++
 8 files changed, 57 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 22c880c5c3..1a276c9b9e 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5618,6 +5618,13 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
 	int i, ret = 0;
 	struct rte_kvargs *kvlist = NULL;
 
+	if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
+			    eth_da->type);
+		return -ENOTSUP;
+	}
 	num_rep = eth_da->nb_representor_ports;
 	if (num_rep > BNXT_MAX_VF_REPS) {
 		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index b3f441c8f7..def669f411 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1312,6 +1312,12 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (retval)
 			return retval;
 	}
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct enic),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index d7cd049891..9b86bcdc69 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -639,6 +639,13 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 			return retval;
 	}
 
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct i40e_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 761a0f26bb..8a9a21e7c2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1718,6 +1718,13 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	} else
 		memset(&eth_da, 0, sizeof(eth_da));
 
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct ixgbe_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 2dc079779d..b041e1752d 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -704,6 +704,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				strerror(rte_errno));
 			return NULL;
 		}
+		if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) {
+			/* Representor not specified. */
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+			rte_errno = ENOTSUP;
+			DRV_LOG(ERR, "unsupported representor type: %s",
+				dpdk_dev->devargs->args);
+			return NULL;
+		}
 		for (i = 0; i < eth_da.nb_representor_ports; ++i)
 			if (eth_da.representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 57fdedaa1a..d68b8401e7 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1216,7 +1216,11 @@ __rte_internal
 int
 rte_eth_switch_domain_free(uint16_t domain_id);
 
-/** Generic Ethernet device arguments  */
+/**
+ * Generic Ethernet device arguments
+ *
+ * One type of representor each structure.
+ */
 struct rte_eth_devargs {
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
@@ -1226,6 +1230,7 @@ struct rte_eth_devargs {
 	/** representor port/s identifier to enable on device */
 	uint16_t nb_representor_ports;
 	/** number of ports in representor port field */
+	enum rte_eth_representor_type type; /* type of representor */
 };
 
 /**
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 5960c75cc5..2d51e3857a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -111,11 +111,16 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	return 0;
 }
 
+/*
+ * representor format:
+ *   #: range or single number of VF representor
+ */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 }
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 059a061072..1f378958ca 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1506,6 +1506,14 @@ struct rte_eth_rxseg_capa {
  * Ethernet device information
  */
 
+/**
+ * Ethernet device representor port type.
+ */
+enum rte_eth_representor_type {
+	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
+	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
+};
+
 /**
  * A structure used to retrieve the contextual information of
  * an Ethernet device, such as the controlling driver of the
-- 
2.25.1

[dpdk-dev] [PATCH v7 3/9] ethdev: support new VF representor syntax

[Xueming Li]

Current VF representor syntax:
 representor=2          - single representor
 representor=[0-3]      - single range

To prepare for more representor types, this patch adds compatible VF
representor devargs syntax:

vf#:
 representor=vf2          - single representor
 representor=vf[1,3,5]    - single list
 representor=vf[0-3]      - single range
 representor=vf[0,1,4-7]  - list with singles and range

For backwards compatibility, representor "#" is interpreted as "vf#".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 7 ++++---
 lib/librte_ethdev/ethdev_private.c      | 5 ++++-
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 239ec820ea..0117c2af07 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -374,9 +374,10 @@ parameters to those ports.
   this argument allows user to specify which switch ports to enable port
   representors for.::
 
-   -a DBDF,representor=0
-   -a DBDF,representor=[0,4,6,9]
-   -a DBDF,representor=[0-31]
+   -a DBDF,representor=vf0
+   -a DBDF,representor=vf[0,4,6,9]
+   -a DBDF,representor=vf[0-31]
+   -a DBDF,representor=vf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index d5ef05ea6f..4bb3879859 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -117,7 +117,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Parse representor ports from a single value or lists.
  *
  * Representor format:
- *   #: range or single number of VF representor
+ *   #: range or single number of VF representor - legacy
+ *   vf#: VF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -130,6 +131,8 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	if (str[0] == 'v' && str[1] == 'f')
+		str += 2;
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
-- 
2.25.1

[dpdk-dev] [PATCH v7 4/9] ethdev: support sub function representor

[Xueming Li]

SubFunction is a portion of the PCI device, created on demand, a SF
netdev has its own dedicated queues(txq, rxq). A SF netdev supports
eswitch representation offload similar to existing PF and VF
representors.

To support SF representor, this patch introduces new devargs syntax,
examples:
 representor=sf0               - single SubFunction representor
 representor=sf[1,3,5]         - single list
 representor=sf[0-3],          - single range
 representor=sf[0,2-6,8,10-12] - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst       |  6 +++-
 .../prog_guide/switch_representation.rst      | 35 +++++++++++++------
 lib/librte_ethdev/ethdev_private.c            | 11 ++++--
 lib/librte_ethdev/rte_ethdev.c                |  6 ++++
 lib/librte_ethdev/rte_ethdev.h                |  1 +
 5 files changed, 46 insertions(+), 13 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 0117c2af07..063a4680df 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -372,12 +372,16 @@ parameters to those ports.
 
 * ``representor`` for a device which supports the creation of representor ports
   this argument allows user to specify which switch ports to enable port
-  representors for.::
+  representors for. Multiple representors in one device argument is invalid::
 
    -a DBDF,representor=vf0
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
    -a DBDF,representor=vf[0,2-4,7,9-11]
+   -a DBDF,representor=sf0
+   -a DBDF,representor=sf[1,3,5]
+   -a DBDF,representor=sf[0-1023]
+   -a DBDF,representor=sf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/doc/guides/prog_guide/switch_representation.rst b/doc/guides/prog_guide/switch_representation.rst
index 07ba12bea6..ff6aa91c80 100644
--- a/doc/guides/prog_guide/switch_representation.rst
+++ b/doc/guides/prog_guide/switch_representation.rst
@@ -13,7 +13,7 @@ Introduction
 
 Network adapters with multiple physical ports and/or SR-IOV capabilities
 usually support the offload of traffic steering rules between their virtual
-functions (VFs), physical functions (PFs) and ports.
+functions (VFs), sub functions (SFs), physical functions (PFs) and ports.
 
 Like for standard Ethernet switches, this involves a combination of
 automatic MAC learning and manual configuration. For most purposes it is
@@ -24,7 +24,7 @@ layer 2 (L2) traffic (such as OVS) need to steer traffic themselves
 according on their own criteria.
 
 Without a standard software interface to manage traffic steering rules
-between VFs, PFs and the various physical ports of a given device,
+between VFs, SFs, PFs and the various physical ports of a given device,
 applications cannot take advantage of these offloads; software processing is
 mandatory even for traffic which ends up re-injected into the device it
 originates from.
@@ -34,6 +34,17 @@ the DPDK flow API (**rte_flow**), with emphasis on the SR-IOV use case
 (PF/VF steering) using a single physical port for clarity, however the same
 logic applies to any number of ports without necessarily involving SR-IOV.
 
+Sub Function
+------------
+Besides SR-IOV, Sub function is a portion of the PCI device, a SF netdev
+has its own dedicated queues(txq, rxq). A SF netdev supports E-Switch
+representation offload similar to existing PF and VF representors.
+A SF shares PCI level resources with other SFs and/or with its parent PCI
+function.
+
+Sub function is created on-demand, coexists with VFs. Number of SFs is
+limited by hardware resources.
+
 Port Representors
 -----------------
 
@@ -42,15 +53,16 @@ applications usually have to process a bit of traffic in software before
 thinking about offloading specific flows to hardware.
 
 Applications therefore need the ability to receive and inject traffic to
-various device endpoints (other VFs, PFs or physical ports) before
+various device endpoints (other VFs, SFs, PFs or physical ports) before
 connecting them together. Device drivers must provide means to hook the
 "other end" of these endpoints and to refer them when configuring flow
 rules.
 
 This role is left to so-called "port representors" (also known as "VF
-representors" in the specific context of VFs), which are to DPDK what the
-Ethernet switch device driver model (**switchdev**) [1]_ is to Linux, and
-which can be thought as a software "patch panel" front-end for applications.
+representors" in the specific context of VFs, "SF representors" in the
+specific context of SFs), which are to DPDK what the Ethernet switch
+device driver model (**switchdev**) [1]_ is to Linux, and which can be
+thought as a software "patch panel" front-end for applications.
 
 - DPDK port representors are implemented as additional virtual Ethernet
   device (**ethdev**) instances, spawned on an as needed basis through
@@ -59,9 +71,12 @@ which can be thought as a software "patch panel" front-end for applications.
 
 ::
 
-   -a pci:dbdf,representor=0
-   -a pci:dbdf,representor=[0-3]
-   -a pci:dbdf,representor=[0,5-11]
+   -a pci:dbdf,representor=vf0
+   -a pci:dbdf,representor=vf[0-3]
+   -a pci:dbdf,representor=vf[0,5-11]
+   -a pci:dbdf,representor=sf1
+   -a pci:dbdf,representor=sf[0-1023]
+   -a pci:dbdf,representor=sf[0,2-1023]
 
 - As virtual devices, they may be more limited than their physical
   counterparts, for instance by exposing only a subset of device
@@ -360,7 +375,7 @@ Compared to creating a brand new dedicated interface, **rte_flow** was
 deemed flexible enough to manage representor traffic only with minor
 extensions:
 
-- Using physical ports, PF, VF or port representors as targets.
+- Using physical ports, PF, SF, VF or port representors as targets.
 
 - Affecting traffic that is not necessarily addressed to the DPDK port ID a
   flow rule is associated with (e.g. forcing VF traffic redirection to PF).
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 4bb3879859..13c191192e 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -119,6 +119,7 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Representor format:
  *   #: range or single number of VF representor - legacy
  *   vf#: VF port representor/s
+ *   sf#: SF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -130,9 +131,15 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
-	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	if (str[0] == 'v' && str[1] == 'f')
+	if (str[0] == 'v' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
+	} else if (str[0] == 's' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_SF;
+		str += 2;
+	} else {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index fe9466a03e..60889a8663 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5589,6 +5589,12 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
+			if (eth_da->type != RTE_ETH_REPRESENTOR_NONE) {
+				RTE_LOG(ERR, EAL, "duplicated representor key: %s\n",
+					dargs);
+				result = -1;
+				goto parse_cleanup;
+			}
 			result = rte_eth_devargs_parse_representor_ports(
 					pair->value, eth_da);
 			if (result < 0)
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 1f378958ca..26b5e109c3 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1512,6 +1512,7 @@ struct rte_eth_rxseg_capa {
 enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
 	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
+	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v7 5/9] ethdev: support PF index in representor

[Xueming Li]

With Kernel bonding, multiple underlying PFs are bonded, VFs come
from different PF, need to identify representor of VFs unambiguously by
adding PF index.

This patch introduces optional 'pf' section to representor devargs
syntax, examples:
 representor=pf0vf0             - single VF representor
 representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs

PF type representor is supported by using standalone 'pf' section:
 representor=pf1                - PF representor

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  3 +++
 lib/librte_ethdev/ethdev_private.c      | 19 +++++++++++++++++--
 lib/librte_ethdev/rte_ethdev.h          |  1 +
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 063a4680df..0d4ac77a7c 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -382,6 +382,9 @@ parameters to those ports.
    -a DBDF,representor=sf[1,3,5]
    -a DBDF,representor=sf[0-1023]
    -a DBDF,representor=sf[0,2-4,7,9-11]
+   -a DBDF,representor=pf1vf0
+   -a DBDF,representor=pf[0-1]sf[0-127]
+   -a DBDF,representor=pf1
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 13c191192e..cd87add37f 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -118,8 +118,9 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   vf#: VF port representor/s
- *   sf#: SF port representor/s
+ *   [pf#]vf#: VF port representor/s
+ *   [pf#]sf#: SF port representor/s
+ *   pf#:      PF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -131,6 +132,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'p' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_PF;
+		str += 2;
+		str = rte_eth_devargs_process_list(str, eth_da->ports,
+				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
+		if (str == NULL || str[0] == '\0')
+			goto err;
+	}
 	if (str[0] == 'v' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
@@ -138,11 +147,17 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 		eth_da->type = RTE_ETH_REPRESENTOR_SF;
 		str += 2;
 	} else {
+		/* 'pf' must followed by 'vf' or 'sf'. */
+		if (eth_da->type == RTE_ETH_REPRESENTOR_PF) {
+			str = NULL;
+			goto err;
+		}
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
+err:
 	if (str == NULL)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
 	return str == NULL ? -1 : 0;
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 26b5e109c3..9cd519bf59 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1513,6 +1513,7 @@ enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
 	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
 	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
+	RTE_ETH_REPRESENTOR_PF,   /**< representor of Physical Function. */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v7 6/9] ethdev: support multi-host in representor

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to the multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. To provide the unambiguous identification
of the PCIe function the controller index is added. The full representor
identifier consists of three indices - controller index, PF index, and
VF or SF index (if any).

This patch introduces controller index to ethdev representor syntax,
examples:

[[c#]pf#]vf#: VF port representor/s, example: pf0vf1
[[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]

c# is controller(host) ID/range in case of multi-host, optional.

For user application (e.g. OVS), PMD is responsible to interpret and
locate representor device based on controller ID, PF ID and VF/SF ID in
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 config/rte_config.h                |  1 +
 lib/librte_ethdev/ethdev_driver.h  |  4 ++++
 lib/librte_ethdev/ethdev_private.c | 18 +++++++++++++++---
 3 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index 55a2fc50ed..904a40b3c1 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -57,6 +57,7 @@
 #define RTE_MAX_QUEUES_PER_PORT 1024
 #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
 #define RTE_ETHDEV_RXTX_CALLBACKS 1
+#define RTE_MAX_MULTI_HOST_CTRLS 4
 
 /* cryptodev defines */
 #define RTE_CRYPTO_MAX_DEVS 64
diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index d68b8401e7..06ff35266f 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1222,6 +1222,10 @@ rte_eth_switch_domain_free(uint16_t domain_id);
  * One type of representor each structure.
  */
 struct rte_eth_devargs {
+	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
+	/** controller/s number in case of multi-host */
+	uint16_t nb_mh_controllers;
+	/** number of controllers in multi-host controllers field */
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
 	uint16_t nb_ports;
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index cd87add37f..3c3ae5501c 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -118,9 +118,9 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   [pf#]vf#: VF port representor/s
- *   [pf#]sf#: SF port representor/s
- *   pf#:      PF port representor/s
+ *   [[c#]pf#]vf#: VF port representor/s
+ *   [[c#]pf#]sf#: SF port representor/s
+ *   [c#]pf#:      PF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -132,6 +132,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'c') {
+		str += 1;
+		str = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
+				&eth_da->nb_mh_controllers,
+				RTE_DIM(eth_da->mh_controllers));
+		if (str == NULL)
+			goto err;
+	}
 	if (str[0] == 'p' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_PF;
 		str += 2;
@@ -139,6 +147,10 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
 		if (str == NULL || str[0] == '\0')
 			goto err;
+	} else if (eth_da->nb_mh_controllers > 0) {
+		/* 'c' must followed by 'pf'. */
+		str = NULL;
+		goto err;
 	}
 	if (str[0] == 'v' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
-- 
2.25.1

[dpdk-dev] [PATCH v7 7/9] ethdev: new API to get representor info

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

This patch introduces a new API rte_eth_representor_info_get() to
retrieve representor corresponding info mapping:
 - caller controller index and pf index.
 - supported representor ID ranges.
 - type, controller, pf and start vf/sf ID of each range.
The API is useful to convert representor from devargs to representor ID.

New ethdev callback representor_info_get() is added to retrieve info
from PMD driver, optional for PMD that doesn't support new devargs
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_ethdev/ethdev_driver.h |  7 +++++
 lib/librte_ethdev/rte_ethdev.c    | 14 +++++++++
 lib/librte_ethdev/rte_ethdev.h    | 51 +++++++++++++++++++++++++++++++
 lib/librte_ethdev/version.map     |  3 ++
 4 files changed, 75 insertions(+)

diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 06ff35266f..7b0f392e34 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -289,6 +289,10 @@ typedef int (*eth_fw_version_get_t)(struct rte_eth_dev *dev,
 				     char *fw_version, size_t fw_size);
 /**< @internal Get firmware information of an Ethernet device. */
 
+typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
+	struct rte_eth_representor_info *info);
+/**< @internal Get representor type and ID range. */
+
 typedef int (*eth_tx_done_cleanup_t)(void *txq, uint32_t free_cnt);
 /**< @internal Force mbufs to be from TX ring. */
 
@@ -940,6 +944,9 @@ struct eth_dev_ops {
 
 	eth_get_monitor_addr_t get_monitor_addr;
 	/**< Get power monitoring condition for Rx queue. */
+
+	eth_representor_info_get_t representor_info_get;
+	/**< Get representor info. */
 };
 
 /**
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 60889a8663..c88e345e7d 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -3265,6 +3265,20 @@ rte_eth_dev_fw_version_get(uint16_t port_id, char *fw_version, size_t fw_size)
 							fw_version, fw_size));
 }
 
+int
+rte_eth_representor_info_get(uint16_t port_id,
+			     struct rte_eth_representor_info *info)
+{
+	struct rte_eth_dev *dev;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->representor_info_get, -ENOTSUP);
+	return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev,
+								      info));
+}
+
 int
 rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info)
 {
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 9cd519bf59..a72843c65c 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1581,6 +1581,38 @@ struct rte_eth_dev_info {
 	void *reserved_ptrs[2];   /**< Reserved for future fields */
 };
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice.
+ *
+ * ethernet device representor ID range entry
+ */
+struct rte_eth_representor_range {
+	enum rte_eth_representor_type type; /**< Representor type */
+	int controller; /**< Controller ID, -1 to ignore */
+	int pf; /**< Physical function ID, -1 to ignore */
+	__extension__
+	union {
+		int vf; /**< VF start index */
+		int sf; /**< SF start index */
+	};
+	uint16_t id_base; /**< Representor ID start index */
+	uint16_t id_end;  /**< Representor ID end index */
+	char name[RTE_DEV_NAME_MAX_LEN]; /**< Representor name */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice.
+ *
+ * Ethernet device representor information
+ */
+struct rte_eth_representor_info {
+	uint16_t controller; /**< Controller ID of caller device. */
+	uint16_t pf; /**< Physical function ID of caller device. */
+	struct rte_eth_representor_range ranges[];/**< Representor ID range. */
+};
+
 /**
  * Ethernet device RX queue information structure.
  * Used to retrieve information about configured queue.
@@ -3038,6 +3070,25 @@ int rte_eth_macaddr_get(uint16_t port_id, struct rte_ether_addr *mac_addr);
  */
 int rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info);
 
+/**
+ * Retrieve the representor info of the device.
+ *
+ * @param port_id
+ *   The port identifier of the device.
+ * @param info
+ *   A pointer to a representor info structure.
+ *   NULL to return number of range entries and allocate memory
+ *   for next call to store detail.
+ * @return
+ *   - (-ENOTSUP) if operation is not supported.
+ *   - (-ENODEV) if *port_id* invalid.
+ *   - (-EIO) if device is removed.
+ *   - (>=0) number of representor range entries supported by device.
+ */
+__rte_experimental
+int rte_eth_representor_info_get(uint16_t port_id,
+				 struct rte_eth_representor_info *info);
+
 /**
  * Retrieve the firmware version of a device.
  *
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index a124e1e370..bb6f7436c2 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -243,6 +243,9 @@ EXPERIMENTAL {
 
 	# added in 21.02
 	rte_eth_get_monitor_addr;
+
+	# added in 21.05
+	rte_eth_representor_info_get;
 };
 
 INTERNAL {
-- 
2.25.1

[dpdk-dev] [PATCH v7 8/9] ethdev: representor iterator compare complete info

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

SR-IOV and SubFunction are created on top of PF. PF index is introduced
because there might be multiple PFs in the bonding configuration and
only bonding device is probed.

In eth representor comparator callback, ethdev representor ID was
compared with devarg. Since controller index and PF index not compared,
callback returned representor from other PF or controller.

This patch adds new API to convert representor controller, pf and vf/sf
index to representor ID. Representor comparer callback convert
representor info into ID and compare with device representor ID.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_ethdev/ethdev_driver.h | 32 +++++++++++
 lib/librte_ethdev/rte_class_eth.c | 38 ++++++++++---
 lib/librte_ethdev/rte_ethdev.c    | 91 +++++++++++++++++++++++++++++++
 lib/librte_ethdev/version.map     |  1 +
 4 files changed, 153 insertions(+), 9 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 7b0f392e34..3ff1b90b2f 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1244,6 +1244,38 @@ struct rte_eth_devargs {
 	enum rte_eth_representor_type type; /* type of representor */
 };
 
+/**
+ * PMD helper function to convert representor ID from location detail
+ *
+ * Convert representor ID from controller, pf and (sf or vf).
+ * The mapping is retrieved from rte_eth_representor_info_get().
+ *
+ * For backward compatibility, if no representor info, direct
+ * map legacy VF (no controller and pf).
+ *
+ * @param ethdev
+ *  Handle of ethdev port.
+ * @param repr_id
+ *  Pointer to converted representor ID.
+ * @param type
+ *  Representor type.
+ * @param controller
+ *  Controller ID, -1 if unspecified.
+ * @param pf
+ *  PF port ID, -1 if unspecified.
+ * @param representor_port
+ *  VF or SF representor port number, -1 if unspecified.
+ *
+ * @return
+ *  Negative errno value on error, 0 on success.
+ */
+__rte_internal
+int
+rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
+			       uint16_t *repr_id,
+			       enum rte_eth_representor_type type,
+			       int controller, int pf, int representor_port);
+
 /**
  * PMD helper function to parse ethdev arguments
  *
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index 051c892b40..f7b7e659e7 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -65,9 +65,10 @@ eth_representor_cmp(const char *key __rte_unused,
 {
 	int ret;
 	char *values;
-	const struct rte_eth_dev_data *data = opaque;
-	struct rte_eth_devargs representors;
-	uint16_t index;
+	const struct rte_eth_dev *edev = opaque;
+	const struct rte_eth_dev_data *data = edev->data;
+	struct rte_eth_devargs eth_da;
+	uint16_t id, nc, np, nf, i, c, p, f;
 
 	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
 		return -1; /* not a representor port */
@@ -76,17 +77,36 @@ eth_representor_cmp(const char *key __rte_unused,
 	values = strdup(value);
 	if (values == NULL)
 		return -1;
-	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
+	memset(&eth_da, 0, sizeof(eth_da));
+	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
 
+	if (eth_da.nb_mh_controllers == 0 && eth_da.nb_ports == 0 &&
+	    eth_da.nb_representor_ports == 0)
+		return -1;
+	nc = eth_da.nb_mh_controllers > 0 ? eth_da.nb_mh_controllers : 1;
+	np = eth_da.nb_ports > 0 ? eth_da.nb_ports : 1;
+	nf = eth_da.nb_representor_ports > 0 ? eth_da.nb_representor_ports : 1;
+
 	/* Return 0 if representor id is matching one of the values. */
-	for (index = 0; index < representors.nb_representor_ports; index++)
-		if (data->representor_id ==
-				representors.representor_ports[index])
+	for (i = 0; i < nc * np * nf; ++i) {
+		c = i / (np * nf);
+		p = (i / nf) % np;
+		f = i % nf;
+		if (rte_eth_representor_id_convert(edev,
+			&id,
+			eth_da.type,
+			eth_da.nb_mh_controllers == 0 ? -1 :
+					eth_da.mh_controllers[c],
+			eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
+			eth_da.nb_representor_ports == 0 ? -1 :
+					eth_da.representor_ports[f]) < 0)
+			continue;
+		if (data->representor_id == id)
 			return 0;
+	}
 	return -1; /* no match */
 }
 
@@ -112,7 +132,7 @@ eth_dev_match(const struct rte_eth_dev *edev,
 
 	ret = rte_kvargs_process(kvlist,
 			eth_params_keys[RTE_ETH_PARAM_REPRESENTOR],
-			eth_representor_cmp, edev->data);
+			eth_representor_cmp, (void *)(uintptr_t)edev);
 	if (ret != 0)
 		return -1;
 	/* search for representor key */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index c88e345e7d..78cdef11be 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5623,6 +5623,97 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	return result;
 }
 
+int
+rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
+			       uint16_t *repr_id,
+			       enum rte_eth_representor_type type,
+			       int controller, int pf, int representor_port)
+{
+	int ret, n, i, count;
+	struct rte_eth_representor_info *info = NULL;
+	size_t size;
+
+	if (type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (repr_id == NULL)
+		return -EINVAL;
+
+	/* Get PMD representor range info. */
+	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
+	if (ret < 0) {
+		if (type == RTE_ETH_REPRESENTOR_VF && controller == -1 &&
+		    pf == -1) {
+			/* Direct mapping for legacy VF representor. */
+			*repr_id = representor_port;
+			return 0;
+		} else {
+			return ret;
+		}
+	}
+	n = ret;
+	size = sizeof(*info) + n * sizeof(info->ranges[0]);
+	info = calloc(1, size);
+	if (info == NULL)
+		return -ENOMEM;
+	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
+	if (ret < 0)
+		goto out;
+
+	/* Default controller and pf to caller. */
+	if (controller == -1)
+		controller = info->controller;
+	if (pf == -1)
+		pf = info->pf;
+
+	/* Locate representor ID. */
+	ret = -ENOENT;
+	for (i = 0; i < n; ++i) {
+		if (info->ranges[i].type != type)
+			continue;
+		/* PMD hit: ignore controller if -1. */
+		if (info->ranges[i].controller != -1 &&
+		    info->ranges[i].controller != (uint16_t)controller)
+			continue;
+		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;
+		switch (info->ranges[i].type) {
+		case RTE_ETH_REPRESENTOR_PF:
+			if (pf >= info->ranges[i].pf + count)
+				continue;
+			*repr_id = info->ranges[i].id_base +
+				   (pf - info->ranges[i].pf);
+			ret = 0;
+			goto out;
+		case RTE_ETH_REPRESENTOR_VF:
+			/* PMD hit: ignore pf if -1. */
+			if (info->ranges[i].pf != -1 &&
+			    info->ranges[i].pf != (uint16_t)pf)
+				continue;
+			if (representor_port >= info->ranges[i].vf + count)
+				continue;
+			*repr_id = info->ranges[i].id_base +
+				   (representor_port - info->ranges[i].vf);
+			ret = 0;
+			goto out;
+		case RTE_ETH_REPRESENTOR_SF:
+			/* PMD hit: ignore pf if -1. */
+			if (info->ranges[i].pf != -1 &&
+			    info->ranges[i].pf != (uint16_t)pf)
+				continue;
+			if (representor_port >= info->ranges[i].sf + count)
+				continue;
+			*repr_id = info->ranges[i].id_base +
+			      (representor_port - info->ranges[i].sf);
+			ret = 0;
+			goto out;
+		default:
+			break;
+		}
+	}
+out:
+	free(info);
+	return ret;
+}
+
 static int
 eth_dev_handle_port_list(const char *cmd __rte_unused,
 		const char *params __rte_unused,
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index bb6f7436c2..2891f5734e 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -268,6 +268,7 @@ INTERNAL {
 	rte_eth_hairpin_queue_peer_bind;
 	rte_eth_hairpin_queue_peer_unbind;
 	rte_eth_hairpin_queue_peer_update;
+	rte_eth_representor_id_convert;
 	rte_eth_switch_domain_alloc;
 	rte_eth_switch_domain_free;
 };
-- 
2.25.1

[dpdk-dev] [PATCH v7 9/9] kvargs: update parser to support lists

[Xueming Li]

This patch updates kvargs parser to support lists on top of range, allow
multiple lists or range:
  k1=a[1,2]b[3-5]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test/test_kvargs.c         |  46 +++++++++++++--
 lib/librte_kvargs/rte_kvargs.c | 101 +++++++++++++++++++++++----------
 2 files changed, 112 insertions(+), 35 deletions(-)

diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index 2a2dae43a0..a91ea8dc47 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -35,6 +35,25 @@ static int check_handler(const char *key, const char *value,
 	return 0;
 }
 
+/* test parsing. */
+static int test_kvargs_parsing(const char *args, unsigned int n)
+{
+	struct rte_kvargs *kvlist;
+
+	kvlist = rte_kvargs_parse(args, NULL);
+	if (kvlist == NULL) {
+		printf("rte_kvargs_parse() error: %s\n", args);
+		return -1;
+	}
+	if (kvlist->count != n) {
+		printf("invalid count value %d: %s\n", kvlist->count, args);
+		rte_kvargs_free(kvlist);
+		return -1;
+	}
+	rte_kvargs_free(kvlist);
+	return 0;
+}
+
 /* test a valid case */
 static int test_valid_kvargs(void)
 {
@@ -42,6 +61,19 @@ static int test_valid_kvargs(void)
 	const char *args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
 	const char **valid_keys;
+	static const struct {
+		unsigned int expected;
+		const char *input;
+	} valid_inputs[] = {
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo" },
+		{ 2, "foo=1,=2" },
+		{ 1, "foo=[1,2" },
+		{ 1, ",=" },
+		{ 1, "foo=[" },
+	};
+	unsigned int i;
 
 	/* empty args is valid */
 	args = "";
@@ -191,6 +223,14 @@ static int test_valid_kvargs(void)
 	}
 	rte_kvargs_free(kvlist);
 
+	valid_keys = NULL;
+
+	for (i = 0; i < RTE_DIM(valid_inputs); ++i) {
+		args = valid_inputs[i].input;
+		if (test_kvargs_parsing(args, valid_inputs[i].expected))
+			goto fail;
+	}
+
 	return 0;
 
  fail:
@@ -212,12 +252,6 @@ static int test_invalid_kvargs(void)
 	/* list of argument that should fail */
 	const char *args_list[] = {
 		"wrong-key=x",     /* key not in valid_keys_list */
-		"foo=1,foo=",      /* empty value */
-		"foo=1,foo",       /* no value */
-		"foo=1,=2",        /* no key */
-		"foo=[1,2",        /* no closing bracket in value */
-		",=",              /* also test with a smiley */
-		"foo=[",           /* no value in list and no closing bracket */
 		NULL };
 	const char **args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
index 285081c86c..ffae8914cf 100644
--- a/lib/librte_kvargs/rte_kvargs.c
+++ b/lib/librte_kvargs/rte_kvargs.c
@@ -5,6 +5,7 @@
 
 #include <string.h>
 #include <stdlib.h>
+#include <stdbool.h>
 
 #include <rte_string_fns.h>
 
@@ -13,15 +14,19 @@
 /*
  * Receive a string with a list of arguments following the pattern
  * key=value,key=value,... and insert them into the list.
- * strtok() is used so the params string will be copied to be modified.
+ * Params string will be copied to be modified.
+ * list "[]" and list element splitter ",", "-" is treated as value.
+ * Supported examples:
+ *   k1=v1,k2=v2
+ *   k1
+ *   k1=x[0-1]y[1,3-5,9]z
  */
 static int
 rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 {
-	unsigned i;
-	char *str;
-	char *ctx1 = NULL;
-	char *ctx2 = NULL;
+	char *str, *start;
+	bool in_list = false, end_key = false, end_value = false;
+	bool save = false, end_pair = false;
 
 	/* Copy the const char *params to a modifiable string
 	 * to pass to rte_strsplit
@@ -32,36 +37,74 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 
 	/* browse each key/value pair and add it in kvlist */
 	str = kvlist->str;
-	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
+	start = str; /* start of current key or value */
+	while (1) {
+		switch (*str) {
+		case '=': /* End of key. */
+			end_key = true;
+			save = true;
+			break;
+		case ',':
+			/* End of value, skip comma in middle of range */
+			if (!in_list) {
+				if (end_key)
+					end_value = true;
+				else
+					end_key = true;
+				save = true;
+				end_pair = true;
+			}
+			break;
+		case '[': /* Start of list. */
+			in_list = true;
+			break;
+		case ']': /* End of list.  */
+			if (in_list)
+				in_list = false;
+			break;
+		case '\0': /* End of string */
+			if (end_key)
+				end_value = true;
+			else
+				end_key = true;
+			save = true;
+			end_pair = true;
+			break;
+		default:
+			break;
+		}
 
-		i = kvlist->count;
-		if (i >= RTE_KVARGS_MAX)
-			return -1;
+		if (!save) {
+			/* Continue if not end of key or value. */
+			str++;
+			continue;
+		}
 
-		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
-		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
-		if (kvlist->pairs[i].key == NULL ||
-		    kvlist->pairs[i].value == NULL)
+		if (kvlist->count >= RTE_KVARGS_MAX)
 			return -1;
 
-		/* Detect list [a,b] to skip comma delimiter in list. */
-		str = kvlist->pairs[i].value;
-		if (str[0] == '[') {
-			/* Find the end of the list. */
-			while (str[strlen(str) - 1] != ']') {
-				/* Restore the comma erased by strtok_r(). */
-				if (ctx1 == NULL || ctx1[0] == '\0')
-					return -1; /* no closing bracket */
-				str[strlen(str)] = ',';
-				/* Parse until next comma. */
-				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
-				if (str == NULL)
-					return -1; /* no closing bracket */
-			}
+		if (end_value)
+			/* Value parsed */
+			kvlist->pairs[kvlist->count].value = start;
+		else if (end_key)
+			/* Key parsed. */
+			kvlist->pairs[kvlist->count].key = start;
+
+		if (end_pair) {
+			if (end_value || str != start)
+				/* Ignore empty pair. */
+				kvlist->count++;
+			end_key = false;
+			end_value = false;
+			end_pair = false;
 		}
 
-		kvlist->count++;
-		str = NULL;
+		if (*str == '\0') /* End of string. */
+			break;
+		*str = '\0';
+		str++;
+		start = str;
+		save = false;
 	}
 
 	return 0;
-- 
2.25.1

Re: [dpdk-dev] [PATCH v7 5/9] ethdev: support PF index in representor

[Andrew Rybchenko]

On 3/2/21 2:42 PM, Xueming Li wrote:
> With Kernel bonding, multiple underlying PFs are bonded, VFs come
> from different PF, need to identify representor of VFs unambiguously by
> adding PF index.
> 
> This patch introduces optional 'pf' section to representor devargs
> syntax, examples:
>  representor=pf0vf0             - single VF representor
>  representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs
> 
> PF type representor is supported by using standalone 'pf' section:
>  representor=pf1                - PF representor
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>

A nit below.

Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

> ---
>  doc/guides/prog_guide/poll_mode_drv.rst |  3 +++
>  lib/librte_ethdev/ethdev_private.c      | 19 +++++++++++++++++--
>  lib/librte_ethdev/rte_ethdev.h          |  1 +
>  3 files changed, 21 insertions(+), 2 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
> index 063a4680df..0d4ac77a7c 100644
> --- a/doc/guides/prog_guide/poll_mode_drv.rst
> +++ b/doc/guides/prog_guide/poll_mode_drv.rst
> @@ -382,6 +382,9 @@ parameters to those ports.
>     -a DBDF,representor=sf[1,3,5]
>     -a DBDF,representor=sf[0-1023]
>     -a DBDF,representor=sf[0,2-4,7,9-11]
> +   -a DBDF,representor=pf1vf0
> +   -a DBDF,representor=pf[0-1]sf[0-127]
> +   -a DBDF,representor=pf1
>  
>  Note: PMDs are not required to support the standard device arguments and users
>  should consult the relevant PMD documentation to see support devargs.
> diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
> index 13c191192e..cd87add37f 100644
> --- a/lib/librte_ethdev/ethdev_private.c
> +++ b/lib/librte_ethdev/ethdev_private.c
> @@ -118,8 +118,9 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
>   *
>   * Representor format:
>   *   #: range or single number of VF representor - legacy
> - *   vf#: VF port representor/s
> - *   sf#: SF port representor/s
> + *   [pf#]vf#: VF port representor/s
> + *   [pf#]sf#: SF port representor/s
> + *   pf#:      PF port representor/s
>   *
>   * Examples of #:
>   *  2               - single
> @@ -131,6 +132,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  {
>  	struct rte_eth_devargs *eth_da = data;
>  
> +	if (str[0] == 'p' && str[1] == 'f') {
> +		eth_da->type = RTE_ETH_REPRESENTOR_PF;
> +		str += 2;
> +		str = rte_eth_devargs_process_list(str, eth_da->ports,
> +				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
> +		if (str == NULL || str[0] == '\0')
> +			goto err;

'err' label name is misleading here, since it is not always
error as far as I can see. It is a normal path in the case
of 'pfX'. May be rename it as 'done' or something like this.

> +	}
>  	if (str[0] == 'v' && str[1] == 'f') {
>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>  		str += 2;
> @@ -138,11 +147,17 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
>  		eth_da->type = RTE_ETH_REPRESENTOR_SF;
>  		str += 2;
>  	} else {
> +		/* 'pf' must followed by 'vf' or 'sf'. */
> +		if (eth_da->type == RTE_ETH_REPRESENTOR_PF) {
> +			str = NULL;
> +			goto err;
> +		}
>  		eth_da->type = RTE_ETH_REPRESENTOR_VF;
>  	}
>  	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
>  		&eth_da->nb_representor_ports,
>  		RTE_DIM(eth_da->representor_ports));
> +err:
>  	if (str == NULL)
>  		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
>  	return str == NULL ? -1 : 0;
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 26b5e109c3..9cd519bf59 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -1513,6 +1513,7 @@ enum rte_eth_representor_type {
>  	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
>  	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
>  	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
> +	RTE_ETH_REPRESENTOR_PF,   /**< representor of Physical Function. */
>  };
>  
>  /**
> 

Re: [dpdk-dev] [PATCH v7 7/9] ethdev: new API to get representor info

[Andrew Rybchenko]

On 3/2/21 2:43 PM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> This patch introduces a new API rte_eth_representor_info_get() to
> retrieve representor corresponding info mapping:
>  - caller controller index and pf index.
>  - supported representor ID ranges.
>  - type, controller, pf and start vf/sf ID of each range.
> The API is useful to convert representor from devargs to representor ID.
> 
> New ethdev callback representor_info_get() is added to retrieve info
> from PMD driver, optional for PMD that doesn't support new devargs
> representor syntax.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>

LGTM, except style nitpicking, see below. Just be consistent.

Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

> ---
>  lib/librte_ethdev/ethdev_driver.h |  7 +++++
>  lib/librte_ethdev/rte_ethdev.c    | 14 +++++++++
>  lib/librte_ethdev/rte_ethdev.h    | 51 +++++++++++++++++++++++++++++++
>  lib/librte_ethdev/version.map     |  3 ++
>  4 files changed, 75 insertions(+)
> 
> diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
> index 06ff35266f..7b0f392e34 100644
> --- a/lib/librte_ethdev/ethdev_driver.h
> +++ b/lib/librte_ethdev/ethdev_driver.h
> @@ -289,6 +289,10 @@ typedef int (*eth_fw_version_get_t)(struct rte_eth_dev *dev,
>  				     char *fw_version, size_t fw_size);
>  /**< @internal Get firmware information of an Ethernet device. */
>  
> +typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
> +	struct rte_eth_representor_info *info);
> +/**< @internal Get representor type and ID range. */
> +

Here the snippet is added after FW version get.

>  typedef int (*eth_tx_done_cleanup_t)(void *txq, uint32_t free_cnt);
>  /**< @internal Force mbufs to be from TX ring. */
>  
> @@ -940,6 +944,9 @@ struct eth_dev_ops {
>  
>  	eth_get_monitor_addr_t get_monitor_addr;
>  	/**< Get power monitoring condition for Rx queue. */
> +
> +	eth_representor_info_get_t representor_info_get;
> +	/**< Get representor info. */

Here at the end.

>  };
>  
>  /**
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 60889a8663..c88e345e7d 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -3265,6 +3265,20 @@ rte_eth_dev_fw_version_get(uint16_t port_id, char *fw_version, size_t fw_size)
>  							fw_version, fw_size));
>  }
>  
> +int
> +rte_eth_representor_info_get(uint16_t port_id,
> +			     struct rte_eth_representor_info *info)
> +{
> +	struct rte_eth_dev *dev;
> +
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> +	dev = &rte_eth_devices[port_id];
> +
> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->representor_info_get, -ENOTSUP);
> +	return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev,
> +								      info));
> +}
> +

Here after FW version get again.

>  int
>  rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info)
>  {
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 9cd519bf59..a72843c65c 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -1581,6 +1581,38 @@ struct rte_eth_dev_info {
>  	void *reserved_ptrs[2];   /**< Reserved for future fields */
>  };
>  
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this structure may change without prior notice.
> + *
> + * ethernet device representor ID range entry
> + */
> +struct rte_eth_representor_range {
> +	enum rte_eth_representor_type type; /**< Representor type */
> +	int controller; /**< Controller ID, -1 to ignore */
> +	int pf; /**< Physical function ID, -1 to ignore */
> +	__extension__
> +	union {
> +		int vf; /**< VF start index */
> +		int sf; /**< SF start index */
> +	};
> +	uint16_t id_base; /**< Representor ID start index */
> +	uint16_t id_end;  /**< Representor ID end index */
> +	char name[RTE_DEV_NAME_MAX_LEN]; /**< Representor name */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this structure may change without prior notice.
> + *
> + * Ethernet device representor information
> + */
> +struct rte_eth_representor_info {
> +	uint16_t controller; /**< Controller ID of caller device. */
> +	uint16_t pf; /**< Physical function ID of caller device. */
> +	struct rte_eth_representor_range ranges[];/**< Representor ID range. */
> +};
> +

I'd put it just before rte_eth_representor_info_get().

>  /**
>   * Ethernet device RX queue information structure.
>   * Used to retrieve information about configured queue.
> @@ -3038,6 +3070,25 @@ int rte_eth_macaddr_get(uint16_t port_id, struct rte_ether_addr *mac_addr);
>   */
>  int rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info);
>  
> +/**
> + * Retrieve the representor info of the device.
> + *
> + * @param port_id
> + *   The port identifier of the device.
> + * @param info
> + *   A pointer to a representor info structure.
> + *   NULL to return number of range entries and allocate memory
> + *   for next call to store detail.
> + * @return
> + *   - (-ENOTSUP) if operation is not supported.
> + *   - (-ENODEV) if *port_id* invalid.
> + *   - (-EIO) if device is removed.
> + *   - (>=0) number of representor range entries supported by device.
> + */
> +__rte_experimental
> +int rte_eth_representor_info_get(uint16_t port_id,
> +				 struct rte_eth_representor_info *info);
> +

Here you add it *before* FW version get.

>  /**
>   * Retrieve the firmware version of a device.
>   *
> diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
> index a124e1e370..bb6f7436c2 100644
> --- a/lib/librte_ethdev/version.map
> +++ b/lib/librte_ethdev/version.map
> @@ -243,6 +243,9 @@ EXPERIMENTAL {
>  
>  	# added in 21.02
>  	rte_eth_get_monitor_addr;
> +
> +	# added in 21.05
> +	rte_eth_representor_info_get;
>  };
>  
>  INTERNAL {
> 

Re: [dpdk-dev] [PATCH v7 8/9] ethdev: representor iterator compare complete info

[Andrew Rybchenko]

On 3/2/21 2:43 PM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> SR-IOV and SubFunction are created on top of PF. PF index is introduced
> because there might be multiple PFs in the bonding configuration and
> only bonding device is probed.
> 
> In eth representor comparator callback, ethdev representor ID was
> compared with devarg. Since controller index and PF index not compared,
> callback returned representor from other PF or controller.
> 
> This patch adds new API to convert representor controller, pf and vf/sf
> index to representor ID. Representor comparer callback convert
> representor info into ID and compare with device representor ID.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> ---
>  lib/librte_ethdev/ethdev_driver.h | 32 +++++++++++
>  lib/librte_ethdev/rte_class_eth.c | 38 ++++++++++---
>  lib/librte_ethdev/rte_ethdev.c    | 91 +++++++++++++++++++++++++++++++
>  lib/librte_ethdev/version.map     |  1 +
>  4 files changed, 153 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
> index 7b0f392e34..3ff1b90b2f 100644
> --- a/lib/librte_ethdev/ethdev_driver.h
> +++ b/lib/librte_ethdev/ethdev_driver.h
> @@ -1244,6 +1244,38 @@ struct rte_eth_devargs {
>  	enum rte_eth_representor_type type; /* type of representor */
>  };
>  
> +/**
> + * PMD helper function to convert representor ID from location detail

Full stop is missing above.

Also I'm not sure in term "convert" here. It sounds a bit
misleading to me. May be just "get". I.e
PMD helper function to get representor ID by location detail.
and
rte_eth_representor_id_get()

What do you think?

> + *
> + * Convert representor ID from controller, pf and (sf or vf).
> + * The mapping is retrieved from rte_eth_representor_info_get().
> + *
> + * For backward compatibility, if no representor info, direct
> + * map legacy VF (no controller and pf).
> + *
> + * @param ethdev
> + *  Handle of ethdev port.
> + * @param repr_id
> + *  Pointer to converted representor ID.
> + * @param type
> + *  Representor type.
> + * @param controller
> + *  Controller ID, -1 if unspecified.
> + * @param pf
> + *  PF port ID, -1 if unspecified.
> + * @param representor_port
> + *  VF or SF representor port number, -1 if unspecified.

Mixing input and output parameters looks bad to me.
May I suggest to put repr_id the last?
I.e. to have all input parameters first.
(May be I've already mentioned it, if I missed your
reply, please, repeat it once again).

> + *
> + * @return
> + *  Negative errno value on error, 0 on success.
> + */
> +__rte_internal
> +int
> +rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
> +			       uint16_t *repr_id,
> +			       enum rte_eth_representor_type type,
> +			       int controller, int pf, int representor_port);
> +
>  /**
>   * PMD helper function to parse ethdev arguments
>   *
> diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
> index 051c892b40..f7b7e659e7 100644
> --- a/lib/librte_ethdev/rte_class_eth.c
> +++ b/lib/librte_ethdev/rte_class_eth.c
> @@ -65,9 +65,10 @@ eth_representor_cmp(const char *key __rte_unused,
>  {
>  	int ret;
>  	char *values;
> -	const struct rte_eth_dev_data *data = opaque;
> -	struct rte_eth_devargs representors;
> -	uint16_t index;
> +	const struct rte_eth_dev *edev = opaque;
> +	const struct rte_eth_dev_data *data = edev->data;
> +	struct rte_eth_devargs eth_da;
> +	uint16_t id, nc, np, nf, i, c, p, f;
>  
>  	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
>  		return -1; /* not a representor port */
> @@ -76,17 +77,36 @@ eth_representor_cmp(const char *key __rte_unused,
>  	values = strdup(value);
>  	if (values == NULL)
>  		return -1;
> -	memset(&representors, 0, sizeof(representors));
> -	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
> +	memset(&eth_da, 0, sizeof(eth_da));
> +	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
>  	free(values);
>  	if (ret != 0)
>  		return -1; /* invalid devargs value */
>  
> +	if (eth_da.nb_mh_controllers == 0 && eth_da.nb_ports == 0 &&
> +	    eth_da.nb_representor_ports == 0)
> +		return -1;
> +	nc = eth_da.nb_mh_controllers > 0 ? eth_da.nb_mh_controllers : 1;
> +	np = eth_da.nb_ports > 0 ? eth_da.nb_ports : 1;
> +	nf = eth_da.nb_representor_ports > 0 ? eth_da.nb_representor_ports : 1;
> +
>  	/* Return 0 if representor id is matching one of the values. */
> -	for (index = 0; index < representors.nb_representor_ports; index++)
> -		if (data->representor_id ==
> -				representors.representor_ports[index])
> +	for (i = 0; i < nc * np * nf; ++i) {
> +		c = i / (np * nf);
> +		p = (i / nf) % np;
> +		f = i % nf;
> +		if (rte_eth_representor_id_convert(edev,
> +			&id,
> +			eth_da.type,
> +			eth_da.nb_mh_controllers == 0 ? -1 :
> +					eth_da.mh_controllers[c],
> +			eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
> +			eth_da.nb_representor_ports == 0 ? -1 :
> +					eth_da.representor_ports[f]) < 0)
> +			continue;
> +		if (data->representor_id == id)
>  			return 0;
> +	}
>  	return -1; /* no match */
>  }
>  
> @@ -112,7 +132,7 @@ eth_dev_match(const struct rte_eth_dev *edev,
>  
>  	ret = rte_kvargs_process(kvlist,
>  			eth_params_keys[RTE_ETH_PARAM_REPRESENTOR],
> -			eth_representor_cmp, edev->data);
> +			eth_representor_cmp, (void *)(uintptr_t)edev);
>  	if (ret != 0)
>  		return -1;
>  	/* search for representor key */
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index c88e345e7d..78cdef11be 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -5623,6 +5623,97 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>  	return result;
>  }
>  
> +int
> +rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
> +			       uint16_t *repr_id,
> +			       enum rte_eth_representor_type type,
> +			       int controller, int pf, int representor_port)
> +{
> +	int ret, n, i, count;
> +	struct rte_eth_representor_info *info = NULL;
> +	size_t size;
> +
> +	if (type == RTE_ETH_REPRESENTOR_NONE)
> +		return 0;
> +	if (repr_id == NULL)
> +		return -EINVAL;
> +
> +	/* Get PMD representor range info. */
> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
> +	if (ret < 0) {
> +		if (type == RTE_ETH_REPRESENTOR_VF && controller == -1 &&
> +		    pf == -1) {
> +			/* Direct mapping for legacy VF representor. */
> +			*repr_id = representor_port;
> +			return 0;
> +		} else {
> +			return ret;
> +		}
> +	}
> +	n = ret;
> +	size = sizeof(*info) + n * sizeof(info->ranges[0]);
> +	info = calloc(1, size);
> +	if (info == NULL)
> +		return -ENOMEM;
> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
> +	if (ret < 0)
> +		goto out;
> +
> +	/* Default controller and pf to caller. */
> +	if (controller == -1)
> +		controller = info->controller;
> +	if (pf == -1)
> +		pf = info->pf;
> +
> +	/* Locate representor ID. */
> +	ret = -ENOENT;
> +	for (i = 0; i < n; ++i) {
> +		if (info->ranges[i].type != type)
> +			continue;
> +		/* PMD hit: ignore controller if -1. */
> +		if (info->ranges[i].controller != -1 &&
> +		    info->ranges[i].controller != (uint16_t)controller)

First of all I don't understand why 'controller' is cast to
uint16_t here. Both 'controller' and range->controller have
'int' type.

Second, I'm sorry, but I still don't understand the ignore
logic. Why information retrieval may return -1 for controller
and/or PF? What does it mean?

Above fallback to the device controller and pf if unspecified
by the caller look good and make sense.

> +			continue;
> +		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;
> +		switch (info->ranges[i].type) {
> +		case RTE_ETH_REPRESENTOR_PF:
> +			if (pf >= info->ranges[i].pf + count)
> +				continue;

Condition must be stricter. We must ensure that requested
port within both boundaries of the range.
I.e. representor_port should not be smaller than
info->ranges[i].pf.
It is required for below subtraction.

> +			*repr_id = info->ranges[i].id_base +
> +				   (pf - info->ranges[i].pf);
> +			ret = 0;
> +			goto out;
> +		case RTE_ETH_REPRESENTOR_VF:
> +			/* PMD hit: ignore pf if -1. */
> +			if (info->ranges[i].pf != -1 &&
> +			    info->ranges[i].pf != (uint16_t)pf)

Same as above. Cast seems to be not required.

> +				continue;
> +			if (representor_port >= info->ranges[i].vf + count)

Same as above.

> +				continue;
> +			*repr_id = info->ranges[i].id_base +
> +				   (representor_port - info->ranges[i].vf);
> +			ret = 0;
> +			goto out;
> +		case RTE_ETH_REPRESENTOR_SF:
> +			/* PMD hit: ignore pf if -1. */
> +			if (info->ranges[i].pf != -1 &&
> +			    info->ranges[i].pf != (uint16_t)pf)

Same as above. Cast seems to be not required.

> +				continue;
> +			if (representor_port >= info->ranges[i].sf + count)
> +				continue;

Same as above.

> +			*repr_id = info->ranges[i].id_base +
> +			      (representor_port - info->ranges[i].sf);
> +			ret = 0;
> +			goto out;
> +		default:
> +			break;
> +		}
> +	}
> +out:
> +	free(info);
> +	return ret;
> +}
> +
>  static int
>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>  		const char *params __rte_unused,
> diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
> index bb6f7436c2..2891f5734e 100644
> --- a/lib/librte_ethdev/version.map
> +++ b/lib/librte_ethdev/version.map
> @@ -268,6 +268,7 @@ INTERNAL {
>  	rte_eth_hairpin_queue_peer_bind;
>  	rte_eth_hairpin_queue_peer_unbind;
>  	rte_eth_hairpin_queue_peer_update;
> +	rte_eth_representor_id_convert;
>  	rte_eth_switch_domain_alloc;
>  	rte_eth_switch_domain_free;
>  };
> 

Re: [dpdk-dev] [PATCH v7 8/9] ethdev: representor iterator compare complete info

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, March 2, 2021 10:10 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman
><nhorman@tuxdriver.com>
>Subject: Re: [PATCH v7 8/9] ethdev: representor iterator compare complete info
>
>On 3/2/21 2:43 PM, Xueming Li wrote:
>> The NIC can have multiple PCIe links and can be attached to multiple
>> hosts, for example the same single NIC can be shared for multiple
>> server units in the rack. On each PCIe link NIC can provide multiple
>> PFs and VFs/SFs based on these ones. The full representor identifier
>> consists of three indices - controller index, PF index, and VF or SF index (if any).
>>
>> SR-IOV and SubFunction are created on top of PF. PF index is
>> introduced because there might be multiple PFs in the bonding
>> configuration and only bonding device is probed.
>>
>> In eth representor comparator callback, ethdev representor ID was
>> compared with devarg. Since controller index and PF index not
>> compared, callback returned representor from other PF or controller.
>>
>> This patch adds new API to convert representor controller, pf and
>> vf/sf index to representor ID. Representor comparer callback convert
>> representor info into ID and compare with device representor ID.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> ---
>>  lib/librte_ethdev/ethdev_driver.h | 32 +++++++++++
>> lib/librte_ethdev/rte_class_eth.c | 38 ++++++++++---
>>  lib/librte_ethdev/rte_ethdev.c    | 91 +++++++++++++++++++++++++++++++
>>  lib/librte_ethdev/version.map     |  1 +
>>  4 files changed, 153 insertions(+), 9 deletions(-)
>>
>> diff --git a/lib/librte_ethdev/ethdev_driver.h
>> b/lib/librte_ethdev/ethdev_driver.h
>> index 7b0f392e34..3ff1b90b2f 100644
>> --- a/lib/librte_ethdev/ethdev_driver.h
>> +++ b/lib/librte_ethdev/ethdev_driver.h
>> @@ -1244,6 +1244,38 @@ struct rte_eth_devargs {
>>  	enum rte_eth_representor_type type; /* type of representor */  };
>>
>> +/**
>> + * PMD helper function to convert representor ID from location detail
>
>Full stop is missing above.
>
>Also I'm not sure in term "convert" here. It sounds a bit misleading to me. May be just "get". I.e PMD helper function to get
>representor ID by location detail.
>and
>rte_eth_representor_id_get()
>
>What do you think?

Looks good, I was struggling on the naming, 'get' looks straightforward, no conversion but lookup, thanks!

>
>> + *
>> + * Convert representor ID from controller, pf and (sf or vf).
>> + * The mapping is retrieved from rte_eth_representor_info_get().
>> + *
>> + * For backward compatibility, if no representor info, direct
>> + * map legacy VF (no controller and pf).
>> + *
>> + * @param ethdev
>> + *  Handle of ethdev port.
>> + * @param repr_id
>> + *  Pointer to converted representor ID.
>> + * @param type
>> + *  Representor type.
>> + * @param controller
>> + *  Controller ID, -1 if unspecified.
>> + * @param pf
>> + *  PF port ID, -1 if unspecified.
>> + * @param representor_port
>> + *  VF or SF representor port number, -1 if unspecified.
>
>Mixing input and output parameters looks bad to me.
>May I suggest to put repr_id the last?
>I.e. to have all input parameters first.
>(May be I've already mentioned it, if I missed your reply, please, repeat it once again).

Make sense

>
>> + *
>> + * @return
>> + *  Negative errno value on error, 0 on success.
>> + */
>> +__rte_internal
>> +int
>> +rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
>> +			       uint16_t *repr_id,
>> +			       enum rte_eth_representor_type type,
>> +			       int controller, int pf, int representor_port);
>> +
>>  /**
>>   * PMD helper function to parse ethdev arguments
>>   *
>> diff --git a/lib/librte_ethdev/rte_class_eth.c
>> b/lib/librte_ethdev/rte_class_eth.c
>> index 051c892b40..f7b7e659e7 100644
>> --- a/lib/librte_ethdev/rte_class_eth.c
>> +++ b/lib/librte_ethdev/rte_class_eth.c
>> @@ -65,9 +65,10 @@ eth_representor_cmp(const char *key __rte_unused,
>> {
>>  	int ret;
>>  	char *values;
>> -	const struct rte_eth_dev_data *data = opaque;
>> -	struct rte_eth_devargs representors;
>> -	uint16_t index;
>> +	const struct rte_eth_dev *edev = opaque;
>> +	const struct rte_eth_dev_data *data = edev->data;
>> +	struct rte_eth_devargs eth_da;
>> +	uint16_t id, nc, np, nf, i, c, p, f;
>>
>>  	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
>>  		return -1; /* not a representor port */ @@ -76,17 +77,36 @@
>> eth_representor_cmp(const char *key __rte_unused,
>>  	values = strdup(value);
>>  	if (values == NULL)
>>  		return -1;
>> -	memset(&representors, 0, sizeof(representors));
>> -	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
>> +	memset(&eth_da, 0, sizeof(eth_da));
>> +	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
>>  	free(values);
>>  	if (ret != 0)
>>  		return -1; /* invalid devargs value */
>>
>> +	if (eth_da.nb_mh_controllers == 0 && eth_da.nb_ports == 0 &&
>> +	    eth_da.nb_representor_ports == 0)
>> +		return -1;
>> +	nc = eth_da.nb_mh_controllers > 0 ? eth_da.nb_mh_controllers : 1;
>> +	np = eth_da.nb_ports > 0 ? eth_da.nb_ports : 1;
>> +	nf = eth_da.nb_representor_ports > 0 ? eth_da.nb_representor_ports :
>> +1;
>> +
>>  	/* Return 0 if representor id is matching one of the values. */
>> -	for (index = 0; index < representors.nb_representor_ports; index++)
>> -		if (data->representor_id ==
>> -				representors.representor_ports[index])
>> +	for (i = 0; i < nc * np * nf; ++i) {
>> +		c = i / (np * nf);
>> +		p = (i / nf) % np;
>> +		f = i % nf;
>> +		if (rte_eth_representor_id_convert(edev,
>> +			&id,
>> +			eth_da.type,
>> +			eth_da.nb_mh_controllers == 0 ? -1 :
>> +					eth_da.mh_controllers[c],
>> +			eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
>> +			eth_da.nb_representor_ports == 0 ? -1 :
>> +					eth_da.representor_ports[f]) < 0)
>> +			continue;
>> +		if (data->representor_id == id)
>>  			return 0;
>> +	}
>>  	return -1; /* no match */
>>  }
>>
>> @@ -112,7 +132,7 @@ eth_dev_match(const struct rte_eth_dev *edev,
>>
>>  	ret = rte_kvargs_process(kvlist,
>>  			eth_params_keys[RTE_ETH_PARAM_REPRESENTOR],
>> -			eth_representor_cmp, edev->data);
>> +			eth_representor_cmp, (void *)(uintptr_t)edev);
>>  	if (ret != 0)
>>  		return -1;
>>  	/* search for representor key */
>> diff --git a/lib/librte_ethdev/rte_ethdev.c
>> b/lib/librte_ethdev/rte_ethdev.c index c88e345e7d..78cdef11be 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -5623,6 +5623,97 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>>  	return result;
>>  }
>>
>> +int
>> +rte_eth_representor_id_convert(const struct rte_eth_dev *ethdev,
>> +			       uint16_t *repr_id,
>> +			       enum rte_eth_representor_type type,
>> +			       int controller, int pf, int representor_port) {
>> +	int ret, n, i, count;
>> +	struct rte_eth_representor_info *info = NULL;
>> +	size_t size;
>> +
>> +	if (type == RTE_ETH_REPRESENTOR_NONE)
>> +		return 0;
>> +	if (repr_id == NULL)
>> +		return -EINVAL;
>> +
>> +	/* Get PMD representor range info. */
>> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
>> +	if (ret < 0) {
>> +		if (type == RTE_ETH_REPRESENTOR_VF && controller == -1 &&
>> +		    pf == -1) {
>> +			/* Direct mapping for legacy VF representor. */
>> +			*repr_id = representor_port;
>> +			return 0;
>> +		} else {
>> +			return ret;
>> +		}
>> +	}
>> +	n = ret;
>> +	size = sizeof(*info) + n * sizeof(info->ranges[0]);
>> +	info = calloc(1, size);
>> +	if (info == NULL)
>> +		return -ENOMEM;
>> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
>> +	if (ret < 0)
>> +		goto out;
>> +
>> +	/* Default controller and pf to caller. */
>> +	if (controller == -1)
>> +		controller = info->controller;
>> +	if (pf == -1)
>> +		pf = info->pf;
>> +
>> +	/* Locate representor ID. */
>> +	ret = -ENOENT;
>> +	for (i = 0; i < n; ++i) {
>> +		if (info->ranges[i].type != type)
>> +			continue;
>> +		/* PMD hit: ignore controller if -1. */
>> +		if (info->ranges[i].controller != -1 &&
>> +		    info->ranges[i].controller != (uint16_t)controller)
>
>First of all I don't understand why 'controller' is cast to uint16_t here. Both 'controller' and range->controller have 'int' type.
>

Yes, I should have removed the cast.

>Second, I'm sorry, but I still don't understand the ignore logic. Why information retrieval may return -1 for controller and/or PF? What
>does it mean?

In some circumstances, PMD don't need the controller and PF, i.e. no bonding and multi-host, it locate device from PCI address.
Since openstack can't tell different case, these info are sent without difference, that's why PMD want to ignore them.

But I have a workaround in PMD now, will remove it in next version. But keeping flexibility might be a good choice IMHO.

>
>Above fallback to the device controller and pf if unspecified by the caller look good and make sense.
>
>> +			continue;
>> +		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;
>> +		switch (info->ranges[i].type) {
>> +		case RTE_ETH_REPRESENTOR_PF:
>> +			if (pf >= info->ranges[i].pf + count)
>> +				continue;
>
>Condition must be stricter. We must ensure that requested port within both boundaries of the range.
>I.e. representor_port should not be smaller than
>info->ranges[i].pf.
>It is required for below subtraction.

Good catch!

>
>> +			*repr_id = info->ranges[i].id_base +
>> +				   (pf - info->ranges[i].pf);
>> +			ret = 0;
>> +			goto out;
>> +		case RTE_ETH_REPRESENTOR_VF:
>> +			/* PMD hit: ignore pf if -1. */
>> +			if (info->ranges[i].pf != -1 &&
>> +			    info->ranges[i].pf != (uint16_t)pf)
>
>Same as above. Cast seems to be not required.
>
>> +				continue;
>> +			if (representor_port >= info->ranges[i].vf + count)
>
>Same as above.
>
>> +				continue;
>> +			*repr_id = info->ranges[i].id_base +
>> +				   (representor_port - info->ranges[i].vf);
>> +			ret = 0;
>> +			goto out;
>> +		case RTE_ETH_REPRESENTOR_SF:
>> +			/* PMD hit: ignore pf if -1. */
>> +			if (info->ranges[i].pf != -1 &&
>> +			    info->ranges[i].pf != (uint16_t)pf)
>
>Same as above. Cast seems to be not required.
>
>> +				continue;
>> +			if (representor_port >= info->ranges[i].sf + count)
>> +				continue;
>
>Same as above.
>
>> +			*repr_id = info->ranges[i].id_base +
>> +			      (representor_port - info->ranges[i].sf);
>> +			ret = 0;
>> +			goto out;
>> +		default:
>> +			break;
>> +		}
>> +	}
>> +out:
>> +	free(info);
>> +	return ret;
>> +}
>> +
>>  static int
>>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>>  		const char *params __rte_unused,
>> diff --git a/lib/librte_ethdev/version.map
>> b/lib/librte_ethdev/version.map index bb6f7436c2..2891f5734e 100644
>> --- a/lib/librte_ethdev/version.map
>> +++ b/lib/librte_ethdev/version.map
>> @@ -268,6 +268,7 @@ INTERNAL {
>>  	rte_eth_hairpin_queue_peer_bind;
>>  	rte_eth_hairpin_queue_peer_unbind;
>>  	rte_eth_hairpin_queue_peer_update;
>> +	rte_eth_representor_id_convert;
>>  	rte_eth_switch_domain_alloc;
>>  	rte_eth_switch_domain_free;
>>  };
>>

[dpdk-dev] [PATCH v8 0/9] ethdev: support SubFunction representor

[Xueming Li]

SubFunction [1] is a portion of the PCI device, a SF netdev has its own
dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to existing
VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For backward compatibility, this patch also introduces new netdev
capability to indicate the capability of supportting SF representor.

Version history:
 RFC:
 	initial version [2]
 V2:
    - separate patch for represnetor infrastructure, controller, pf and
      sf.
    - replace representor ID macro with functions:
      rte_eth_representor_id_encode()
      rte_eth_representor_id_parse()
    - new patch to allow devargs with same PCI BDF but different
      representors.
    - other minor code updates according to comments, thanks Andrew!
    - update document
 V3:
    - improve probing of allowed devargs with same name.
    - parse single word of kvargs as key.
    - update kvargs test cases.
 V4:
    - split first representor refactor patch into
      1: add representor type
      2: refector representor list parsing
    - push the patch supporting multi-devargs for same device.
 V5:
    - add comments for parsing functions
    - update switch_representation.rst - Thanks Ajit
 V6:
    - split representor types into different patches, move to
      rte_ethdev.h
    - improvements of rte_eth_devargs_process_list() according to
      Andrew's suggestion
    - fixed PF probe failure for Intel i40e
    - replace ethdev SF capability with rte_eth_representor_info_get()
    - add new ethdev ops api to get representor info from PMD
    - replace representor ID encode/decode with conversion from
      representor info
    - change ethdev representor iterator to use new ID encoding
 V7:
    - error on duplicated representor argument:
      DBDF,representor=a,representor=b
    - fix pf representor type parsing
    - other small fixes suggested by Andrew.
 V8:
 	- remove the "ignore" hint from pmd representor info
 	- changed representor_id type in representor info to u32
 	- rename rte_eth_representor_id_convert() to ..._get()
 	- other small fixes suggested by Andrew
   


[1] SubFunction in kernel:
https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/

[2] RFC:
http://patchwork.dpdk.org/project/dpdk/list/?series=14376

[3] V2:
http://patchwork.dpdk.org/project/dpdk/list/?series=14559

[4] V3:
1 http://patchwork.dpdk.org/patch/86460/
2 http://patchwork.dpdk.org/patch/86461/
3 http://patchwork.dpdk.org/patch/86462/
4 http://patchwork.dpdk.org/patch/86463/
5 http://patchwork.dpdk.org/patch/86464/
6 http://patchwork.dpdk.org/patch/86465/
7 http://patchwork.dpdk.org/patch/86467/
8 http://patchwork.dpdk.org/patch/86466/
9 http://patchwork.dpdk.org/patch/86468/

[5] V4:
http://patchwork.dpdk.org/project/dpdk/list/?series=14809

[6] V5:
http://patchwork.dpdk.org/project/dpdk/list/?series=14834

[7] V6:
http://patchwork.dpdk.org/project/dpdk/list/?series=15258

[8] v7:
http://patchwork.dpdk.org/project/dpdk/list/?series=15441

Xueming Li (9):
  ethdev: introduce representor type
  ethdev: support representor port list
  ethdev: support new VF representor syntax
  ethdev: support sub function representor
  ethdev: support PF index in representor
  ethdev: support multi-host in representor
  ethdev: new API to get representor info
  ethdev: representor iterator compare complete info
  kvargs: update parser to support lists

 app/test/test_kvargs.c                        |  46 ++++-
 config/rte_config.h                           |   1 +
 doc/guides/prog_guide/poll_mode_drv.rst       |  18 +-
 .../prog_guide/switch_representation.rst      |  35 ++--
 drivers/net/bnxt/bnxt_ethdev.c                |   7 +
 drivers/net/enic/enic_ethdev.c                |   6 +
 drivers/net/i40e/i40e_ethdev.c                |   7 +
 drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
 drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
 lib/librte_ethdev/ethdev_driver.h             |  50 +++++-
 lib/librte_ethdev/ethdev_private.c            | 169 ++++++++++++------
 lib/librte_ethdev/ethdev_private.h            |   3 -
 lib/librte_ethdev/rte_class_eth.c             |  40 +++--
 lib/librte_ethdev/rte_ethdev.c                | 113 +++++++++++-
 lib/librte_ethdev/rte_ethdev.h                |  61 +++++++
 lib/librte_ethdev/version.map                 |   4 +
 lib/librte_kvargs/rte_kvargs.c                | 101 ++++++++---
 17 files changed, 554 insertions(+), 125 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH v8 1/9] ethdev: introduce representor type

[Xueming Li]

To support more representor type, this patch introduces representor type
enum. The enum is subject to be extended to support new representor in
patches upcoming.

For each devarg structure, only one type supported.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Hyong Youb Kim <hyonkim@cisco.com>
---
 drivers/net/bnxt/bnxt_ethdev.c     |  7 +++++++
 drivers/net/enic/enic_ethdev.c     |  6 ++++++
 drivers/net/i40e/i40e_ethdev.c     |  7 +++++++
 drivers/net/ixgbe/ixgbe_ethdev.c   |  7 +++++++
 drivers/net/mlx5/linux/mlx5_os.c   | 11 +++++++++++
 lib/librte_ethdev/ethdev_driver.h  |  7 ++++++-
 lib/librte_ethdev/ethdev_private.c |  5 +++++
 lib/librte_ethdev/rte_ethdev.h     |  8 ++++++++
 8 files changed, 57 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 9824cdb6d8..55110ee984 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5646,6 +5646,13 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
 	int i, ret = 0;
 	struct rte_kvargs *kvlist = NULL;
 
+	if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
+			    eth_da->type);
+		return -ENOTSUP;
+	}
 	num_rep = eth_da->nb_representor_ports;
 	if (num_rep > BNXT_MAX_VF_REPS) {
 		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index b3f441c8f7..def669f411 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1312,6 +1312,12 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (retval)
 			return retval;
 	}
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct enic),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index d7cd049891..9b86bcdc69 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -639,6 +639,13 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 			return retval;
 	}
 
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct i40e_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 761a0f26bb..8a9a21e7c2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1718,6 +1718,13 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	} else
 		memset(&eth_da, 0, sizeof(eth_da));
 
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct ixgbe_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 81eb2e4b05..92fa178b46 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -751,6 +751,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				strerror(rte_errno));
 			return NULL;
 		}
+		if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) {
+			/* Representor not specified. */
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+			rte_errno = ENOTSUP;
+			DRV_LOG(ERR, "unsupported representor type: %s",
+				dpdk_dev->devargs->args);
+			return NULL;
+		}
 		for (i = 0; i < eth_da.nb_representor_ports; ++i)
 			if (eth_da.representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 57fdedaa1a..d68b8401e7 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1216,7 +1216,11 @@ __rte_internal
 int
 rte_eth_switch_domain_free(uint16_t domain_id);
 
-/** Generic Ethernet device arguments  */
+/**
+ * Generic Ethernet device arguments
+ *
+ * One type of representor each structure.
+ */
 struct rte_eth_devargs {
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
@@ -1226,6 +1230,7 @@ struct rte_eth_devargs {
 	/** representor port/s identifier to enable on device */
 	uint16_t nb_representor_ports;
 	/** number of ports in representor port field */
+	enum rte_eth_representor_type type; /* type of representor */
 };
 
 /**
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 5960c75cc5..2d51e3857a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -111,11 +111,16 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	return 0;
 }
 
+/*
+ * representor format:
+ *   #: range or single number of VF representor
+ */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 }
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 059a061072..1f378958ca 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1506,6 +1506,14 @@ struct rte_eth_rxseg_capa {
  * Ethernet device information
  */
 
+/**
+ * Ethernet device representor port type.
+ */
+enum rte_eth_representor_type {
+	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
+	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
+};
+
 /**
  * A structure used to retrieve the contextual information of
  * an Ethernet device, such as the controlling driver of the
-- 
2.25.1

[dpdk-dev] [PATCH v8 2/9] ethdev: support representor port list

[Xueming Li]

To support extended representor syntax, this patch extends the
representor list parsing to support for representor port range in
devargs, examples:
   representor=[1,2,3]         - single list
   representor=[1,3-5,7,9-11]  - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 lib/librte_ethdev/ethdev_private.c | 129 ++++++++++++++++-------------
 lib/librte_ethdev/ethdev_private.h |   3 -
 lib/librte_ethdev/rte_class_eth.c  |   4 +-
 lib/librte_ethdev/rte_ethdev.c     |   5 +-
 4 files changed, 74 insertions(+), 67 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 2d51e3857a..d5ef05ea6f 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -38,82 +38,91 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
 	return NULL;
 }
 
-int
-rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data)
+/* Put new value into list. */
+static int
+rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
+		       const uint16_t max_list, uint16_t val)
 {
-	char *str_start;
-	int state;
-	int result;
-
-	if (*str != '[')
-		/* Single element, not a list */
-		return callback(str, data);
-
-	/* Sanity check, then strip the brackets */
-	str_start = &str[strlen(str) - 1];
-	if (*str_start != ']') {
-		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
-		return -EINVAL;
-	}
-	str++;
-	*str_start = '\0';
+	uint16_t i;
 
-	/* Process list elements */
-	state = 0;
-	while (1) {
-		if (state == 0) {
-			if (*str == '\0')
-				break;
-			if (*str != ',') {
-				str_start = str;
-				state = 1;
-			}
-		} else if (state == 1) {
-			if (*str == ',' || *str == '\0') {
-				if (str > str_start) {
-					/* Non-empty string fragment */
-					*str = '\0';
-					result = callback(str_start, data);
-					if (result < 0)
-						return result;
-				}
-				state = 0;
-			}
-		}
-		str++;
+	for (i = 0; i < *len_list; i++) {
+		if (list[i] == val)
+			return 0;
 	}
+	if (*len_list >= max_list)
+		return -1;
+	list[(*len_list)++] = val;
 	return 0;
 }
 
-static int
+/* Parse and enlist a range expression of "min-max" or a single value. */
+static char *
 rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	const uint16_t max_list)
 {
 	uint16_t lo, hi, val;
-	int result;
+	int result, n = 0;
+	char *pos = str;
 
-	result = sscanf(str, "%hu-%hu", &lo, &hi);
+	result = sscanf(str, "%hu%n-%hu%n", &lo, &n, &hi, &n);
 	if (result == 1) {
-		if (*len_list >= max_list)
-			return -ENOMEM;
-		list[(*len_list)++] = lo;
+		if (rte_eth_devargs_enlist(list, len_list, max_list, lo) != 0)
+			return NULL;
 	} else if (result == 2) {
-		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
-			return -EINVAL;
+		if (lo > hi)
+			return NULL;
 		for (val = lo; val <= hi; val++) {
-			if (*len_list >= max_list)
-				return -ENOMEM;
-			list[(*len_list)++] = val;
+			if (rte_eth_devargs_enlist(list, len_list, max_list,
+						   val) != 0)
+				return NULL;
 		}
 	} else
-		return -EINVAL;
-	return 0;
+		return NULL;
+	return pos + n;
+}
+
+/*
+ * Parse list of values separated by ",".
+ * Each value could be a range [min-max] or single number.
+ * Examples:
+ *  2               - single
+ *  [1,2,3]         - single list
+ *  [1,3-5,7,9-11]  - list with singles and ranges
+ */
+static char *
+rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
+	const uint16_t max_list)
+{
+	char *pos = str;
+
+	if (*pos == '[')
+		pos++;
+	while (1) {
+		pos = rte_eth_devargs_process_range(pos, list, len_list,
+						    max_list);
+		if (pos == NULL)
+			return NULL;
+		if (*pos != ',') /* end of list */
+			break;
+		pos++;
+	}
+	if (*str == '[' && *pos != ']')
+		return NULL;
+	if (*pos == ']')
+		pos++;
+	return pos;
 }
 
 /*
- * representor format:
+ * Parse representor ports from a single value or lists.
+ *
+ * Representor format:
  *   #: range or single number of VF representor
+ *
+ * Examples of #:
+ *  2               - single
+ *  [1,2,3]         - single list
+ *  [1,3-5,7,9-11]  - list with singles and ranges
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -121,6 +130,10 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
-		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
+		&eth_da->nb_representor_ports,
+		RTE_DIM(eth_da->representor_ports));
+	if (str == NULL)
+		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
+	return str == NULL ? -1 : 0;
 }
diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
index 905a45c337..220ddd4408 100644
--- a/lib/librte_ethdev/ethdev_private.h
+++ b/lib/librte_ethdev/ethdev_private.h
@@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
 		const void *data);
 
 /* Parse devargs value for representor parameter. */
-typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
-int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data);
 int rte_eth_devargs_parse_representor_ports(char *str, void *data);
 
 #ifdef __cplusplus
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index ca2ce87f7a..051c892b40 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
 	if (values == NULL)
 		return -1;
 	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_list(values,
-			rte_eth_devargs_parse_representor_ports,
-			&representors);
+	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 6f514c388b..fe9466a03e 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5589,9 +5589,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
-			result = rte_eth_devargs_parse_list(pair->value,
-				rte_eth_devargs_parse_representor_ports,
-				eth_da);
+			result = rte_eth_devargs_parse_representor_ports(
+					pair->value, eth_da);
 			if (result < 0)
 				goto parse_cleanup;
 		}
-- 
2.25.1

[dpdk-dev] [PATCH v8 3/9] ethdev: support new VF representor syntax

[Xueming Li]

Current VF representor syntax:
 representor=2          - single representor
 representor=[0-3]      - single range

To prepare for more representor types, this patch adds compatible VF
representor devargs syntax:

vf#:
 representor=vf2          - single representor
 representor=vf[1,3,5]    - single list
 representor=vf[0-3]      - single range
 representor=vf[0,1,4-7]  - list with singles and range

For backwards compatibility, representor "#" is interpreted as "vf#".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 7 ++++---
 lib/librte_ethdev/ethdev_private.c      | 5 ++++-
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 239ec820ea..0117c2af07 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -374,9 +374,10 @@ parameters to those ports.
   this argument allows user to specify which switch ports to enable port
   representors for.::
 
-   -a DBDF,representor=0
-   -a DBDF,representor=[0,4,6,9]
-   -a DBDF,representor=[0-31]
+   -a DBDF,representor=vf0
+   -a DBDF,representor=vf[0,4,6,9]
+   -a DBDF,representor=vf[0-31]
+   -a DBDF,representor=vf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index d5ef05ea6f..4bb3879859 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -117,7 +117,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Parse representor ports from a single value or lists.
  *
  * Representor format:
- *   #: range or single number of VF representor
+ *   #: range or single number of VF representor - legacy
+ *   vf#: VF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -130,6 +131,8 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	if (str[0] == 'v' && str[1] == 'f')
+		str += 2;
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
-- 
2.25.1

[dpdk-dev] [PATCH v8 4/9] ethdev: support sub function representor

[Xueming Li]

SubFunction is a portion of the PCI device, created on demand, a SF
netdev has its own dedicated queues(txq, rxq). A SF netdev supports
eswitch representation offload similar to existing PF and VF
representors.

To support SF representor, this patch introduces new devargs syntax,
examples:
 representor=sf0               - single SubFunction representor
 representor=sf[1,3,5]         - single list
 representor=sf[0-3],          - single range
 representor=sf[0,2-6,8,10-12] - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst       |  6 +++-
 .../prog_guide/switch_representation.rst      | 35 +++++++++++++------
 lib/librte_ethdev/ethdev_private.c            | 11 ++++--
 lib/librte_ethdev/rte_ethdev.c                |  6 ++++
 lib/librte_ethdev/rte_ethdev.h                |  1 +
 5 files changed, 46 insertions(+), 13 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 0117c2af07..063a4680df 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -372,12 +372,16 @@ parameters to those ports.
 
 * ``representor`` for a device which supports the creation of representor ports
   this argument allows user to specify which switch ports to enable port
-  representors for.::
+  representors for. Multiple representors in one device argument is invalid::
 
    -a DBDF,representor=vf0
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
    -a DBDF,representor=vf[0,2-4,7,9-11]
+   -a DBDF,representor=sf0
+   -a DBDF,representor=sf[1,3,5]
+   -a DBDF,representor=sf[0-1023]
+   -a DBDF,representor=sf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/doc/guides/prog_guide/switch_representation.rst b/doc/guides/prog_guide/switch_representation.rst
index 07ba12bea6..ff6aa91c80 100644
--- a/doc/guides/prog_guide/switch_representation.rst
+++ b/doc/guides/prog_guide/switch_representation.rst
@@ -13,7 +13,7 @@ Introduction
 
 Network adapters with multiple physical ports and/or SR-IOV capabilities
 usually support the offload of traffic steering rules between their virtual
-functions (VFs), physical functions (PFs) and ports.
+functions (VFs), sub functions (SFs), physical functions (PFs) and ports.
 
 Like for standard Ethernet switches, this involves a combination of
 automatic MAC learning and manual configuration. For most purposes it is
@@ -24,7 +24,7 @@ layer 2 (L2) traffic (such as OVS) need to steer traffic themselves
 according on their own criteria.
 
 Without a standard software interface to manage traffic steering rules
-between VFs, PFs and the various physical ports of a given device,
+between VFs, SFs, PFs and the various physical ports of a given device,
 applications cannot take advantage of these offloads; software processing is
 mandatory even for traffic which ends up re-injected into the device it
 originates from.
@@ -34,6 +34,17 @@ the DPDK flow API (**rte_flow**), with emphasis on the SR-IOV use case
 (PF/VF steering) using a single physical port for clarity, however the same
 logic applies to any number of ports without necessarily involving SR-IOV.
 
+Sub Function
+------------
+Besides SR-IOV, Sub function is a portion of the PCI device, a SF netdev
+has its own dedicated queues(txq, rxq). A SF netdev supports E-Switch
+representation offload similar to existing PF and VF representors.
+A SF shares PCI level resources with other SFs and/or with its parent PCI
+function.
+
+Sub function is created on-demand, coexists with VFs. Number of SFs is
+limited by hardware resources.
+
 Port Representors
 -----------------
 
@@ -42,15 +53,16 @@ applications usually have to process a bit of traffic in software before
 thinking about offloading specific flows to hardware.
 
 Applications therefore need the ability to receive and inject traffic to
-various device endpoints (other VFs, PFs or physical ports) before
+various device endpoints (other VFs, SFs, PFs or physical ports) before
 connecting them together. Device drivers must provide means to hook the
 "other end" of these endpoints and to refer them when configuring flow
 rules.
 
 This role is left to so-called "port representors" (also known as "VF
-representors" in the specific context of VFs), which are to DPDK what the
-Ethernet switch device driver model (**switchdev**) [1]_ is to Linux, and
-which can be thought as a software "patch panel" front-end for applications.
+representors" in the specific context of VFs, "SF representors" in the
+specific context of SFs), which are to DPDK what the Ethernet switch
+device driver model (**switchdev**) [1]_ is to Linux, and which can be
+thought as a software "patch panel" front-end for applications.
 
 - DPDK port representors are implemented as additional virtual Ethernet
   device (**ethdev**) instances, spawned on an as needed basis through
@@ -59,9 +71,12 @@ which can be thought as a software "patch panel" front-end for applications.
 
 ::
 
-   -a pci:dbdf,representor=0
-   -a pci:dbdf,representor=[0-3]
-   -a pci:dbdf,representor=[0,5-11]
+   -a pci:dbdf,representor=vf0
+   -a pci:dbdf,representor=vf[0-3]
+   -a pci:dbdf,representor=vf[0,5-11]
+   -a pci:dbdf,representor=sf1
+   -a pci:dbdf,representor=sf[0-1023]
+   -a pci:dbdf,representor=sf[0,2-1023]
 
 - As virtual devices, they may be more limited than their physical
   counterparts, for instance by exposing only a subset of device
@@ -360,7 +375,7 @@ Compared to creating a brand new dedicated interface, **rte_flow** was
 deemed flexible enough to manage representor traffic only with minor
 extensions:
 
-- Using physical ports, PF, VF or port representors as targets.
+- Using physical ports, PF, SF, VF or port representors as targets.
 
 - Affecting traffic that is not necessarily addressed to the DPDK port ID a
   flow rule is associated with (e.g. forcing VF traffic redirection to PF).
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 4bb3879859..13c191192e 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -119,6 +119,7 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Representor format:
  *   #: range or single number of VF representor - legacy
  *   vf#: VF port representor/s
+ *   sf#: SF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -130,9 +131,15 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
-	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	if (str[0] == 'v' && str[1] == 'f')
+	if (str[0] == 'v' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
+	} else if (str[0] == 's' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_SF;
+		str += 2;
+	} else {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index fe9466a03e..60889a8663 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5589,6 +5589,12 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
+			if (eth_da->type != RTE_ETH_REPRESENTOR_NONE) {
+				RTE_LOG(ERR, EAL, "duplicated representor key: %s\n",
+					dargs);
+				result = -1;
+				goto parse_cleanup;
+			}
 			result = rte_eth_devargs_parse_representor_ports(
 					pair->value, eth_da);
 			if (result < 0)
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 1f378958ca..26b5e109c3 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1512,6 +1512,7 @@ struct rte_eth_rxseg_capa {
 enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
 	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
+	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v8 5/9] ethdev: support PF index in representor

[Xueming Li]

With Kernel bonding, multiple underlying PFs are bonded, VFs come
from different PF, need to identify representor of VFs unambiguously by
adding PF index.

This patch introduces optional 'pf' section to representor devargs
syntax, examples:
 representor=pf0vf0             - single VF representor
 representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs

PF type representor is supported by using standalone 'pf' section:
 representor=pf1                - PF representor

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  3 +++
 lib/librte_ethdev/ethdev_private.c      | 19 +++++++++++++++++--
 lib/librte_ethdev/rte_ethdev.h          |  1 +
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 063a4680df..0d4ac77a7c 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -382,6 +382,9 @@ parameters to those ports.
    -a DBDF,representor=sf[1,3,5]
    -a DBDF,representor=sf[0-1023]
    -a DBDF,representor=sf[0,2-4,7,9-11]
+   -a DBDF,representor=pf1vf0
+   -a DBDF,representor=pf[0-1]sf[0-127]
+   -a DBDF,representor=pf1
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 13c191192e..35a57c56ec 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -118,8 +118,9 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   vf#: VF port representor/s
- *   sf#: SF port representor/s
+ *   [pf#]vf#: VF port representor/s
+ *   [pf#]sf#: SF port representor/s
+ *   pf#:      PF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -131,6 +132,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'p' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_PF;
+		str += 2;
+		str = rte_eth_devargs_process_list(str, eth_da->ports,
+				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
+		if (str == NULL || str[0] == '\0')
+			goto done;
+	}
 	if (str[0] == 'v' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
@@ -138,11 +147,17 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 		eth_da->type = RTE_ETH_REPRESENTOR_SF;
 		str += 2;
 	} else {
+		/* 'pf' must followed by 'vf' or 'sf'. */
+		if (eth_da->type == RTE_ETH_REPRESENTOR_PF) {
+			str = NULL;
+			goto done;
+		}
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
+done:
 	if (str == NULL)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
 	return str == NULL ? -1 : 0;
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 26b5e109c3..9cd519bf59 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1513,6 +1513,7 @@ enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
 	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
 	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
+	RTE_ETH_REPRESENTOR_PF,   /**< representor of Physical Function. */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v8 6/9] ethdev: support multi-host in representor

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to the multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. To provide the unambiguous identification
of the PCIe function the controller index is added. The full representor
identifier consists of three indices - controller index, PF index, and
VF or SF index (if any).

This patch introduces controller index to ethdev representor syntax,
examples:

[[c#]pf#]vf#: VF port representor/s, example: pf0vf1
[[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]

c# is controller(host) ID/range in case of multi-host, optional.

For user application (e.g. OVS), PMD is responsible to interpret and
locate representor device based on controller ID, PF ID and VF/SF ID in
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 config/rte_config.h                |  1 +
 lib/librte_ethdev/ethdev_driver.h  |  4 ++++
 lib/librte_ethdev/ethdev_private.c | 18 +++++++++++++++---
 3 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index 55a2fc50ed..904a40b3c1 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -57,6 +57,7 @@
 #define RTE_MAX_QUEUES_PER_PORT 1024
 #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
 #define RTE_ETHDEV_RXTX_CALLBACKS 1
+#define RTE_MAX_MULTI_HOST_CTRLS 4
 
 /* cryptodev defines */
 #define RTE_CRYPTO_MAX_DEVS 64
diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index d68b8401e7..06ff35266f 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1222,6 +1222,10 @@ rte_eth_switch_domain_free(uint16_t domain_id);
  * One type of representor each structure.
  */
 struct rte_eth_devargs {
+	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
+	/** controller/s number in case of multi-host */
+	uint16_t nb_mh_controllers;
+	/** number of controllers in multi-host controllers field */
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
 	uint16_t nb_ports;
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 35a57c56ec..012cf73ca2 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -118,9 +118,9 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   [pf#]vf#: VF port representor/s
- *   [pf#]sf#: SF port representor/s
- *   pf#:      PF port representor/s
+ *   [[c#]pf#]vf#: VF port representor/s
+ *   [[c#]pf#]sf#: SF port representor/s
+ *   [c#]pf#:      PF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -132,6 +132,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'c') {
+		str += 1;
+		str = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
+				&eth_da->nb_mh_controllers,
+				RTE_DIM(eth_da->mh_controllers));
+		if (str == NULL)
+			goto done;
+	}
 	if (str[0] == 'p' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_PF;
 		str += 2;
@@ -139,6 +147,10 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
 		if (str == NULL || str[0] == '\0')
 			goto done;
+	} else if (eth_da->nb_mh_controllers > 0) {
+		/* 'c' must followed by 'pf'. */
+		str = NULL;
+		goto done;
 	}
 	if (str[0] == 'v' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
-- 
2.25.1

[dpdk-dev] [PATCH v8 7/9] ethdev: new API to get representor info

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

This patch introduces a new API rte_eth_representor_info_get() to
retrieve representor corresponding info mapping:
 - caller controller index and pf index.
 - supported representor ID ranges.
 - type, controller, pf and start vf/sf ID of each range.
The API is useful to convert representor from devargs to representor ID.

New ethdev callback representor_info_get() is added to retrieve info
from PMD driver, optional for PMD that doesn't support new devargs
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 lib/librte_ethdev/ethdev_driver.h |  7 +++++
 lib/librte_ethdev/rte_ethdev.c    | 14 +++++++++
 lib/librte_ethdev/rte_ethdev.h    | 51 +++++++++++++++++++++++++++++++
 lib/librte_ethdev/version.map     |  3 ++
 4 files changed, 75 insertions(+)

diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 06ff35266f..36e866aa03 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -783,6 +783,10 @@ typedef int (*eth_hairpin_queue_peer_unbind_t)
 typedef int (*eth_get_monitor_addr_t)(void *rxq,
 		struct rte_power_monitor_cond *pmc);
 
+typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
+	struct rte_eth_representor_info *info);
+/**< @internal Get representor type and ID range. */
+
 /**
  * @internal A structure containing the functions exported by an Ethernet driver.
  */
@@ -940,6 +944,9 @@ struct eth_dev_ops {
 
 	eth_get_monitor_addr_t get_monitor_addr;
 	/**< Get power monitoring condition for Rx queue. */
+
+	eth_representor_info_get_t representor_info_get;
+	/**< Get representor info. */
 };
 
 /**
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 60889a8663..3d4ec8ad5c 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5816,6 +5816,20 @@ rte_eth_hairpin_queue_peer_unbind(uint16_t cur_port, uint16_t cur_queue,
 							  direction);
 }
 
+int
+rte_eth_representor_info_get(uint16_t port_id,
+			     struct rte_eth_representor_info *info)
+{
+	struct rte_eth_dev *dev;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->representor_info_get, -ENOTSUP);
+	return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev,
+								      info));
+}
+
 RTE_LOG_REGISTER(rte_eth_dev_logtype, lib.ethdev, INFO);
 
 RTE_INIT(ethdev_init_telemetry)
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 9cd519bf59..9c7a80c9b5 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -4791,6 +4791,57 @@ __rte_experimental
 int rte_eth_dev_hairpin_capability_get(uint16_t port_id,
 				       struct rte_eth_hairpin_cap *cap);
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice.
+ *
+ * ethernet device representor ID range entry
+ */
+struct rte_eth_representor_range {
+	enum rte_eth_representor_type type; /**< Representor type */
+	int controller; /**< Controller index */
+	int pf; /**< Physical function index */
+	__extension__
+	union {
+		int vf; /**< VF start index */
+		int sf; /**< SF start index */
+	};
+	uint32_t id_base; /**< Representor ID start index */
+	uint32_t id_end;  /**< Representor ID end index */
+	char name[RTE_DEV_NAME_MAX_LEN]; /**< Representor name */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice.
+ *
+ * Ethernet device representor information
+ */
+struct rte_eth_representor_info {
+	uint16_t controller; /**< Controller ID of caller device. */
+	uint16_t pf; /**< Physical function ID of caller device. */
+	struct rte_eth_representor_range ranges[];/**< Representor ID range. */
+};
+
+/**
+ * Retrieve the representor info of the device.
+ *
+ * @param port_id
+ *   The port identifier of the device.
+ * @param info
+ *   A pointer to a representor info structure.
+ *   NULL to return number of range entries and allocate memory
+ *   for next call to store detail.
+ * @return
+ *   - (-ENOTSUP) if operation is not supported.
+ *   - (-ENODEV) if *port_id* invalid.
+ *   - (-EIO) if device is removed.
+ *   - (>=0) number of representor range entries supported by device.
+ */
+__rte_experimental
+int rte_eth_representor_info_get(uint16_t port_id,
+				 struct rte_eth_representor_info *info);
+
 #include <rte_ethdev_core.h>
 
 /**
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index a124e1e370..bb6f7436c2 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -243,6 +243,9 @@ EXPERIMENTAL {
 
 	# added in 21.02
 	rte_eth_get_monitor_addr;
+
+	# added in 21.05
+	rte_eth_representor_info_get;
 };
 
 INTERNAL {
-- 
2.25.1

[dpdk-dev] [PATCH v8 8/9] ethdev: representor iterator compare complete info

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

SR-IOV and SubFunction are created on top of PF. PF index is introduced
because there might be multiple PFs in the bonding configuration and
only bonding device is probed.

In eth representor comparator callback, ethdev representor ID was
compared with devarg. Since controller index and PF index not compared,
callback returned representor from other PF or controller.

This patch adds new API to get representor ID from controller, pf and
vf/sf index. Representor comparer callback get representor ID then
compare with device representor ID.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 lib/librte_ethdev/ethdev_driver.h | 32 +++++++++++
 lib/librte_ethdev/rte_class_eth.c | 38 +++++++++----
 lib/librte_ethdev/rte_ethdev.c    | 88 +++++++++++++++++++++++++++++++
 lib/librte_ethdev/version.map     |  1 +
 4 files changed, 150 insertions(+), 9 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 36e866aa03..abe2738348 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1244,6 +1244,38 @@ struct rte_eth_devargs {
 	enum rte_eth_representor_type type; /* type of representor */
 };
 
+/**
+ * PMD helper function to get representor ID from location detail.
+ *
+ * Get representor ID from controller, pf and (sf or vf).
+ * The mapping is retrieved from rte_eth_representor_info_get().
+ *
+ * For backward compatibility, if no representor info, direct
+ * map legacy VF (no controller and pf).
+ *
+ * @param ethdev
+ *  Handle of ethdev port.
+ * @param type
+ *  Representor type.
+ * @param controller
+ *  Controller ID, -1 if unspecified.
+ * @param pf
+ *  PF port ID, -1 if unspecified.
+ * @param representor_port
+ *  VF or SF representor port number, -1 if unspecified.
+ * @param repr_id
+ *  Pointer to output representor ID.
+ *
+ * @return
+ *  Negative errno value on error, 0 on success.
+ */
+__rte_internal
+int
+rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
+			   enum rte_eth_representor_type type,
+			   int controller, int pf, int representor_port,
+			   uint16_t *repr_id);
+
 /**
  * PMD helper function to parse ethdev arguments
  *
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index 051c892b40..1fe5fa1f36 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -65,9 +65,10 @@ eth_representor_cmp(const char *key __rte_unused,
 {
 	int ret;
 	char *values;
-	const struct rte_eth_dev_data *data = opaque;
-	struct rte_eth_devargs representors;
-	uint16_t index;
+	const struct rte_eth_dev *edev = opaque;
+	const struct rte_eth_dev_data *data = edev->data;
+	struct rte_eth_devargs eth_da;
+	uint16_t id, nc, np, nf, i, c, p, f;
 
 	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
 		return -1; /* not a representor port */
@@ -76,17 +77,36 @@ eth_representor_cmp(const char *key __rte_unused,
 	values = strdup(value);
 	if (values == NULL)
 		return -1;
-	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
+	memset(&eth_da, 0, sizeof(eth_da));
+	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
 
+	if (eth_da.nb_mh_controllers == 0 && eth_da.nb_ports == 0 &&
+	    eth_da.nb_representor_ports == 0)
+		return -1;
+	nc = eth_da.nb_mh_controllers > 0 ? eth_da.nb_mh_controllers : 1;
+	np = eth_da.nb_ports > 0 ? eth_da.nb_ports : 1;
+	nf = eth_da.nb_representor_ports > 0 ? eth_da.nb_representor_ports : 1;
+
 	/* Return 0 if representor id is matching one of the values. */
-	for (index = 0; index < representors.nb_representor_ports; index++)
-		if (data->representor_id ==
-				representors.representor_ports[index])
+	for (i = 0; i < nc * np * nf; ++i) {
+		c = i / (np * nf);
+		p = (i / nf) % np;
+		f = i % nf;
+		if (rte_eth_representor_id_get(edev,
+			eth_da.type,
+			eth_da.nb_mh_controllers == 0 ? -1 :
+					eth_da.mh_controllers[c],
+			eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
+			eth_da.nb_representor_ports == 0 ? -1 :
+					eth_da.representor_ports[f],
+			&id) < 0)
+			continue;
+		if (data->representor_id == id)
 			return 0;
+	}
 	return -1; /* no match */
 }
 
@@ -112,7 +132,7 @@ eth_dev_match(const struct rte_eth_dev *edev,
 
 	ret = rte_kvargs_process(kvlist,
 			eth_params_keys[RTE_ETH_PARAM_REPRESENTOR],
-			eth_representor_cmp, edev->data);
+			eth_representor_cmp, (void *)(uintptr_t)edev);
 	if (ret != 0)
 		return -1;
 	/* search for representor key */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 3d4ec8ad5c..2dc464a7ad 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5609,6 +5609,94 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	return result;
 }
 
+int
+rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
+			   enum rte_eth_representor_type type,
+			   int controller, int pf, int representor_port,
+			   uint16_t *repr_id)
+{
+	int ret, n, i, count;
+	struct rte_eth_representor_info *info = NULL;
+	size_t size;
+
+	if (type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (repr_id == NULL)
+		return -EINVAL;
+
+	/* Get PMD representor range info. */
+	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
+	if (ret < 0) {
+		if (type == RTE_ETH_REPRESENTOR_VF && controller == -1 &&
+		    pf == -1) {
+			/* Direct mapping for legacy VF representor. */
+			*repr_id = representor_port;
+			return 0;
+		} else {
+			return ret;
+		}
+	}
+	n = ret;
+	size = sizeof(*info) + n * sizeof(info->ranges[0]);
+	info = calloc(1, size);
+	if (info == NULL)
+		return -ENOMEM;
+	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
+	if (ret < 0)
+		goto out;
+
+	/* Default controller and pf to caller. */
+	if (controller == -1)
+		controller = info->controller;
+	if (pf == -1)
+		pf = info->pf;
+
+	/* Locate representor ID. */
+	ret = -ENOENT;
+	for (i = 0; i < n; ++i) {
+		if (info->ranges[i].type != type)
+			continue;
+		if (info->ranges[i].controller != controller)
+			continue;
+		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;
+		switch (info->ranges[i].type) {
+		case RTE_ETH_REPRESENTOR_PF:
+			if (pf < info->ranges[i].pf ||
+			    pf >= info->ranges[i].pf + count)
+				continue;
+			*repr_id = info->ranges[i].id_base +
+				   (pf - info->ranges[i].pf);
+			ret = 0;
+			goto out;
+		case RTE_ETH_REPRESENTOR_VF:
+			if (info->ranges[i].pf != pf)
+				continue;
+			if (representor_port < info->ranges[i].vf ||
+			    representor_port >= info->ranges[i].vf + count)
+				continue;
+			*repr_id = info->ranges[i].id_base +
+				   (representor_port - info->ranges[i].vf);
+			ret = 0;
+			goto out;
+		case RTE_ETH_REPRESENTOR_SF:
+			if (info->ranges[i].pf != pf)
+				continue;
+			if (representor_port < info->ranges[i].sf ||
+			    representor_port >= info->ranges[i].sf + count)
+				continue;
+			*repr_id = info->ranges[i].id_base +
+			      (representor_port - info->ranges[i].sf);
+			ret = 0;
+			goto out;
+		default:
+			break;
+		}
+	}
+out:
+	free(info);
+	return ret;
+}
+
 static int
 eth_dev_handle_port_list(const char *cmd __rte_unused,
 		const char *params __rte_unused,
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index bb6f7436c2..93ad388e96 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -268,6 +268,7 @@ INTERNAL {
 	rte_eth_hairpin_queue_peer_bind;
 	rte_eth_hairpin_queue_peer_unbind;
 	rte_eth_hairpin_queue_peer_update;
+	rte_eth_representor_id_get;
 	rte_eth_switch_domain_alloc;
 	rte_eth_switch_domain_free;
 };
-- 
2.25.1

[dpdk-dev] [PATCH v8 9/9] kvargs: update parser to support lists

[Xueming Li]

This patch updates kvargs parser to support lists on top of range, allow
multiple lists or range:
  k1=a[1,2]b[3-5]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test/test_kvargs.c         |  46 +++++++++++++--
 lib/librte_kvargs/rte_kvargs.c | 101 +++++++++++++++++++++++----------
 2 files changed, 112 insertions(+), 35 deletions(-)

diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index 2a2dae43a0..a91ea8dc47 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -35,6 +35,25 @@ static int check_handler(const char *key, const char *value,
 	return 0;
 }
 
+/* test parsing. */
+static int test_kvargs_parsing(const char *args, unsigned int n)
+{
+	struct rte_kvargs *kvlist;
+
+	kvlist = rte_kvargs_parse(args, NULL);
+	if (kvlist == NULL) {
+		printf("rte_kvargs_parse() error: %s\n", args);
+		return -1;
+	}
+	if (kvlist->count != n) {
+		printf("invalid count value %d: %s\n", kvlist->count, args);
+		rte_kvargs_free(kvlist);
+		return -1;
+	}
+	rte_kvargs_free(kvlist);
+	return 0;
+}
+
 /* test a valid case */
 static int test_valid_kvargs(void)
 {
@@ -42,6 +61,19 @@ static int test_valid_kvargs(void)
 	const char *args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
 	const char **valid_keys;
+	static const struct {
+		unsigned int expected;
+		const char *input;
+	} valid_inputs[] = {
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo" },
+		{ 2, "foo=1,=2" },
+		{ 1, "foo=[1,2" },
+		{ 1, ",=" },
+		{ 1, "foo=[" },
+	};
+	unsigned int i;
 
 	/* empty args is valid */
 	args = "";
@@ -191,6 +223,14 @@ static int test_valid_kvargs(void)
 	}
 	rte_kvargs_free(kvlist);
 
+	valid_keys = NULL;
+
+	for (i = 0; i < RTE_DIM(valid_inputs); ++i) {
+		args = valid_inputs[i].input;
+		if (test_kvargs_parsing(args, valid_inputs[i].expected))
+			goto fail;
+	}
+
 	return 0;
 
  fail:
@@ -212,12 +252,6 @@ static int test_invalid_kvargs(void)
 	/* list of argument that should fail */
 	const char *args_list[] = {
 		"wrong-key=x",     /* key not in valid_keys_list */
-		"foo=1,foo=",      /* empty value */
-		"foo=1,foo",       /* no value */
-		"foo=1,=2",        /* no key */
-		"foo=[1,2",        /* no closing bracket in value */
-		",=",              /* also test with a smiley */
-		"foo=[",           /* no value in list and no closing bracket */
 		NULL };
 	const char **args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
index 285081c86c..ffae8914cf 100644
--- a/lib/librte_kvargs/rte_kvargs.c
+++ b/lib/librte_kvargs/rte_kvargs.c
@@ -5,6 +5,7 @@
 
 #include <string.h>
 #include <stdlib.h>
+#include <stdbool.h>
 
 #include <rte_string_fns.h>
 
@@ -13,15 +14,19 @@
 /*
  * Receive a string with a list of arguments following the pattern
  * key=value,key=value,... and insert them into the list.
- * strtok() is used so the params string will be copied to be modified.
+ * Params string will be copied to be modified.
+ * list "[]" and list element splitter ",", "-" is treated as value.
+ * Supported examples:
+ *   k1=v1,k2=v2
+ *   k1
+ *   k1=x[0-1]y[1,3-5,9]z
  */
 static int
 rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 {
-	unsigned i;
-	char *str;
-	char *ctx1 = NULL;
-	char *ctx2 = NULL;
+	char *str, *start;
+	bool in_list = false, end_key = false, end_value = false;
+	bool save = false, end_pair = false;
 
 	/* Copy the const char *params to a modifiable string
 	 * to pass to rte_strsplit
@@ -32,36 +37,74 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 
 	/* browse each key/value pair and add it in kvlist */
 	str = kvlist->str;
-	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
+	start = str; /* start of current key or value */
+	while (1) {
+		switch (*str) {
+		case '=': /* End of key. */
+			end_key = true;
+			save = true;
+			break;
+		case ',':
+			/* End of value, skip comma in middle of range */
+			if (!in_list) {
+				if (end_key)
+					end_value = true;
+				else
+					end_key = true;
+				save = true;
+				end_pair = true;
+			}
+			break;
+		case '[': /* Start of list. */
+			in_list = true;
+			break;
+		case ']': /* End of list.  */
+			if (in_list)
+				in_list = false;
+			break;
+		case '\0': /* End of string */
+			if (end_key)
+				end_value = true;
+			else
+				end_key = true;
+			save = true;
+			end_pair = true;
+			break;
+		default:
+			break;
+		}
 
-		i = kvlist->count;
-		if (i >= RTE_KVARGS_MAX)
-			return -1;
+		if (!save) {
+			/* Continue if not end of key or value. */
+			str++;
+			continue;
+		}
 
-		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
-		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
-		if (kvlist->pairs[i].key == NULL ||
-		    kvlist->pairs[i].value == NULL)
+		if (kvlist->count >= RTE_KVARGS_MAX)
 			return -1;
 
-		/* Detect list [a,b] to skip comma delimiter in list. */
-		str = kvlist->pairs[i].value;
-		if (str[0] == '[') {
-			/* Find the end of the list. */
-			while (str[strlen(str) - 1] != ']') {
-				/* Restore the comma erased by strtok_r(). */
-				if (ctx1 == NULL || ctx1[0] == '\0')
-					return -1; /* no closing bracket */
-				str[strlen(str)] = ',';
-				/* Parse until next comma. */
-				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
-				if (str == NULL)
-					return -1; /* no closing bracket */
-			}
+		if (end_value)
+			/* Value parsed */
+			kvlist->pairs[kvlist->count].value = start;
+		else if (end_key)
+			/* Key parsed. */
+			kvlist->pairs[kvlist->count].key = start;
+
+		if (end_pair) {
+			if (end_value || str != start)
+				/* Ignore empty pair. */
+				kvlist->count++;
+			end_key = false;
+			end_value = false;
+			end_pair = false;
 		}
 
-		kvlist->count++;
-		str = NULL;
+		if (*str == '\0') /* End of string. */
+			break;
+		*str = '\0';
+		str++;
+		start = str;
+		save = false;
 	}
 
 	return 0;
-- 
2.25.1

Re: [dpdk-dev] [PATCH v8 1/9] ethdev: introduce representor type

[Ferruh Yigit]

On 3/4/2021 2:30 PM, Xueming Li wrote:
> To support more representor type, this patch introduces representor type
> enum. The enum is subject to be extended to support new representor in
> patches upcoming.
> 
> For each devarg structure, only one type supported.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Acked-by: Hyong Youb Kim <hyonkim@cisco.com>
> ---
>   drivers/net/bnxt/bnxt_ethdev.c     |  7 +++++++
>   drivers/net/enic/enic_ethdev.c     |  6 ++++++
>   drivers/net/i40e/i40e_ethdev.c     |  7 +++++++
>   drivers/net/ixgbe/ixgbe_ethdev.c   |  7 +++++++
>   drivers/net/mlx5/linux/mlx5_os.c   | 11 +++++++++++
>   lib/librte_ethdev/ethdev_driver.h  |  7 ++++++-
>   lib/librte_ethdev/ethdev_private.c |  5 +++++
>   lib/librte_ethdev/rte_ethdev.h     |  8 ++++++++
>   8 files changed, 57 insertions(+), 1 deletion(-)

Hi Xueming,

There are a few instance of 'rte_eth_devargs_parse()' usages in the drivers, 
they may be added after you started this work.
Can you please rebase on latest head for it?

Re: [dpdk-dev] [PATCH v8 2/9] ethdev: support representor port list

[Ferruh Yigit]

On 3/4/2021 2:30 PM, Xueming Li wrote:
> To support extended representor syntax, this patch extends the
> representor list parsing to support for representor port range in
> devargs, examples:
>     representor=[1,2,3]         - single list
>     representor=[1,3-5,7,9-11]  - list with singles and ranges
> 

Isn't above syntax already supported, can you please document what is added?

 From documentation, what already supported is:
-w pci:dbdf,representor=0
-w pci:dbdf,representor=[0-3]
-w pci:dbdf,representor=[0,5-11]


> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>


<...>

Re: [dpdk-dev] [PATCH v8 7/9] ethdev: new API to get representor info

[Ferruh Yigit]

On 3/4/2021 2:30 PM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> This patch introduces a new API rte_eth_representor_info_get() to
> retrieve representor corresponding info mapping:
>   - caller controller index and pf index.
>   - supported representor ID ranges.
>   - type, controller, pf and start vf/sf ID of each range.
> The API is useful to convert representor from devargs to representor ID.
> 
> New ethdev callback representor_info_get() is added to retrieve info
> from PMD driver, optional for PMD that doesn't support new devargs
> representor syntax.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

This is middle layer implementation, and there is not problem with it but 
without PMD and application implementations it is harder to get why/how this API 
will be used.

As far as I can see this API is not directly needed for this set, what do you 
think making this another set with PMD and application implementations on top of 
current set?

Re: [dpdk-dev] [PATCH v8 9/9] kvargs: update parser to support lists

[Ferruh Yigit]

On 3/4/2021 2:30 PM, Xueming Li wrote:
> This patch updates kvargs parser to support lists on top of range, allow
> multiple lists or range:
>    k1=a[1,2]b[3-5]
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>

Is this update used at all in this set?

Re: [dpdk-dev] [PATCH v8 9/9] kvargs: update parser to support lists

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Monday, March 8, 2021 10:45 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; Olivier Matz
><olivier.matz@6wind.com>
>Subject: Re: [dpdk-dev] [PATCH v8 9/9] kvargs: update parser to support lists
>
>On 3/4/2021 2:30 PM, Xueming Li wrote:
>> This patch updates kvargs parser to support lists on top of range,
>> allow multiple lists or range:
>>    k1=a[1,2]b[3-5]
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>
>Is this update used at all in this set?

Yes, for example to support 'pf[0,1]sf[1,3,5]', this patch is a must.

Re: [dpdk-dev] [PATCH v8 7/9] ethdev: new API to get representor info

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Monday, March 8, 2021 10:44 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
>Subject: Re: [PATCH v8 7/9] ethdev: new API to get representor info
>
>On 3/4/2021 2:30 PM, Xueming Li wrote:
>> The NIC can have multiple PCIe links and can be attached to multiple
>> hosts, for example the same single NIC can be shared for multiple
>> server units in the rack. On each PCIe link NIC can provide multiple
>> PFs and VFs/SFs based on these ones. The full representor identifier
>> consists of three indices - controller index, PF index, and VF or SF index (if any).
>>
>> This patch introduces a new API rte_eth_representor_info_get() to
>> retrieve representor corresponding info mapping:
>>   - caller controller index and pf index.
>>   - supported representor ID ranges.
>>   - type, controller, pf and start vf/sf ID of each range.
>> The API is useful to convert representor from devargs to representor ID.
>>
>> New ethdev callback representor_info_get() is added to retrieve info
>> from PMD driver, optional for PMD that doesn't support new devargs
>> representor syntax.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>
>This is middle layer implementation, and there is not problem with it but without PMD and application implementations it is harder to
>get why/how this API will be used.
>
>As far as I can see this API is not directly needed for this set, what do you think making this another set with PMD and application
>implementations on top of current set?

Hi Ferruh,

Thanks for checking this! The patch next, 8/9 which update device iterator for SF representor needs this API to get representor ID then compare.

Re: [dpdk-dev] [PATCH v8 9/9] kvargs: update parser to support lists

[Ferruh Yigit]

On 3/8/2021 2:59 PM, Xueming(Steven) Li wrote:
> 
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Monday, March 8, 2021 10:45 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; Olivier Matz
>> <olivier.matz@6wind.com>
>> Subject: Re: [dpdk-dev] [PATCH v8 9/9] kvargs: update parser to support lists
>>
>> On 3/4/2021 2:30 PM, Xueming Li wrote:
>>> This patch updates kvargs parser to support lists on top of range,
>>> allow multiple lists or range:
>>>     k1=a[1,2]b[3-5]
>>>
>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>
>> Is this update used at all in this set?
> 
> Yes, for example to support 'pf[0,1]sf[1,3,5]', this patch is a must.
> 

So should this patch be before where 'pf[0,1]sf[1,3,5]' syntax is required/used 
in this set?

Re: [dpdk-dev] [PATCH v8 2/9] ethdev: support representor port list

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Monday, March 8, 2021 10:38 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; declan.doherty@intel.com
>Subject: Re: [PATCH v8 2/9] ethdev: support representor port list
>
>On 3/4/2021 2:30 PM, Xueming Li wrote:
>> To support extended representor syntax, this patch extends the
>> representor list parsing to support for representor port range in
>> devargs, examples:
>>     representor=[1,2,3]         - single list
>>     representor=[1,3-5,7,9-11]  - list with singles and ranges
>>
>
>Isn't above syntax already supported, can you please document what is added?
>
> From documentation, what already supported is:
>-w pci:dbdf,representor=0
>-w pci:dbdf,representor=[0-3]
>-w pci:dbdf,representor=[0,5-11]
>

Just a refactor to prepare for the extended representor syntax which need to reuse the value parsing function for controller and pf section.

>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>
>
><...>
>

Re: [dpdk-dev] [PATCH v8 9/9] kvargs: update parser to support lists

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Monday, March 8, 2021 11:55 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; Olivier Matz
><olivier.matz@6wind.com>
>Subject: Re: [dpdk-dev] [PATCH v8 9/9] kvargs: update parser to support lists
>
>On 3/8/2021 2:59 PM, Xueming(Steven) Li wrote:
>>
>>
>>> -----Original Message-----
>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>> Sent: Monday, March 8, 2021 10:45 PM
>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko
>>> <andrew.rybchenko@oktetlabs.ru>
>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>> Penso <asafp@nvidia.com>; Olivier Matz <olivier.matz@6wind.com>
>>> Subject: Re: [dpdk-dev] [PATCH v8 9/9] kvargs: update parser to
>>> support lists
>>>
>>> On 3/4/2021 2:30 PM, Xueming Li wrote:
>>>> This patch updates kvargs parser to support lists on top of range,
>>>> allow multiple lists or range:
>>>>     k1=a[1,2]b[3-5]
>>>>
>>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>>
>>> Is this update used at all in this set?
>>
>> Yes, for example to support 'pf[0,1]sf[1,3,5]', this patch is a must.
>>
>
>So should this patch be before where 'pf[0,1]sf[1,3,5]' syntax is required/used in this set?

Good suggestion, I'll update in next version, thanks!

Re: [dpdk-dev] [PATCH v8 7/9] ethdev: new API to get representor info

[Ferruh Yigit]

On 3/8/2021 3:31 PM, Xueming(Steven) Li wrote:
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Monday, March 8, 2021 10:44 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
>> <thomas@monjalon.net>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
>> Subject: Re: [PATCH v8 7/9] ethdev: new API to get representor info
>>
>> On 3/4/2021 2:30 PM, Xueming Li wrote:
>>> The NIC can have multiple PCIe links and can be attached to multiple
>>> hosts, for example the same single NIC can be shared for multiple
>>> server units in the rack. On each PCIe link NIC can provide multiple
>>> PFs and VFs/SFs based on these ones. The full representor identifier
>>> consists of three indices - controller index, PF index, and VF or SF index (if any).
>>>
>>> This patch introduces a new API rte_eth_representor_info_get() to
>>> retrieve representor corresponding info mapping:
>>>    - caller controller index and pf index.
>>>    - supported representor ID ranges.
>>>    - type, controller, pf and start vf/sf ID of each range.
>>> The API is useful to convert representor from devargs to representor ID.
>>>
>>> New ethdev callback representor_info_get() is added to retrieve info
>>> from PMD driver, optional for PMD that doesn't support new devargs
>>> representor syntax.
>>>
>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>
>> This is middle layer implementation, and there is not problem with it but without PMD and application implementations it is harder to
>> get why/how this API will be used.
>>
>> As far as I can see this API is not directly needed for this set, what do you think making this another set with PMD and application
>> implementations on top of current set?
> 
> Hi Ferruh,
> 
> Thanks for checking this! The patch next, 8/9 which update device iterator for SF representor needs this API to get representor ID then compare.
> 

Got it thanks.
Intention of the new API seems to get info to be able to calculate the unique 
"representor ID" and the helper function 'rte_eth_representor_id_get()' 
implements a logic to calculate this unique ID but that logic is not clear, can 
you please document it more to help the PMD developers to implement 
'representor_info_get()'?

Re: [dpdk-dev] [PATCH v8 8/9] ethdev: representor iterator compare complete info

[Ferruh Yigit]

On 3/4/2021 2:30 PM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> SR-IOV and SubFunction are created on top of PF. PF index is introduced
> because there might be multiple PFs in the bonding configuration and
> only bonding device is probed.
> 
> In eth representor comparator callback, ethdev representor ID was
> compared with devarg. Since controller index and PF index not compared,
> callback returned representor from other PF or controller.
> 
> This patch adds new API to get representor ID from controller, pf and
> vf/sf index. Representor comparer callback get representor ID then
> compare with device representor ID.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>

<...>

> +int
> +rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
> +			   enum rte_eth_representor_type type,
> +			   int controller, int pf, int representor_port,
> +			   uint16_t *repr_id)
> +{
> +	int ret, n, i, count;
> +	struct rte_eth_representor_info *info = NULL;
> +	size_t size;
> +
> +	if (type == RTE_ETH_REPRESENTOR_NONE)
> +		return 0;
> +	if (repr_id == NULL)
> +		return -EINVAL;
> +
> +	/* Get PMD representor range info. */
> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
> +	if (ret < 0) {

This seems to support the legacy format, should the return value checked 
explicitly against the '-ENOTSUP', instead of any error?

> +		if (type == RTE_ETH_REPRESENTOR_VF && controller == -1 &&
> +		    pf == -1) {
> +			/* Direct mapping for legacy VF representor. */
> +			*repr_id = representor_port;
> +			return 0;
> +		} else {
> +			return ret;
> +		}
> +	}


<...>

Re: [dpdk-dev] [PATCH v8 2/9] ethdev: support representor port list

[Ferruh Yigit]

On 3/8/2021 3:58 PM, Xueming(Steven) Li wrote:
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Monday, March 8, 2021 10:38 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
>> <thomas@monjalon.net>; declan.doherty@intel.com
>> Subject: Re: [PATCH v8 2/9] ethdev: support representor port list
>>
>> On 3/4/2021 2:30 PM, Xueming Li wrote:
>>> To support extended representor syntax, this patch extends the
>>> representor list parsing to support for representor port range in
>>> devargs, examples:
>>>      representor=[1,2,3]         - single list
>>>      representor=[1,3-5,7,9-11]  - list with singles and ranges
>>>
>>
>> Isn't above syntax already supported, can you please document what is added?
>>
>>  From documentation, what already supported is:
>> -w pci:dbdf,representor=0
>> -w pci:dbdf,representor=[0-3]
>> -w pci:dbdf,representor=[0,5-11]
>>
> 
> Just a refactor to prepare for the extended representor syntax which need to reuse the value parsing function for controller and pf section.
> 

OK, can you please highlight this is a refactoring, from patch title and commit 
log it feels like new syntax support is added.

>>
>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>
>>
>> <...>
>>
> 

Re: [dpdk-dev] [PATCH v8 0/9] ethdev: support SubFunction representor

[Ferruh Yigit]

On 3/4/2021 2:30 PM, Xueming Li wrote:
> SubFunction [1] is a portion of the PCI device, a SF netdev has its own
> dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
> offload similar to existing PF and VF representors. A SF shares PCI
> level resources with other SFs and/or with its parent PCI function.
> 
>  From SmartNIC perspective, when PCI device is shared for multi-host,
> representors for host controller and host PF is required.
> 
> This patch set introduces new representor types in addtion to existing
> VF representor. Syntax:
> 
> [[c#]pf#]vf#: VF port representor/s from controller/pf
> [[c#]pf#]sf#: SF port representor/s from controller/pf
> #: VF representor - for backwards compatibility
> 
> "#" is number instance, list or range, valid examples:
>    1, [1,3,5], [0-3], [0,2-4,6]
> 
> For backward compatibility, this patch also introduces new netdev
> capability to indicate the capability of supportting SF representor.
> 
> Version history:
>   RFC:
>   	initial version [2]
>   V2:
>      - separate patch for represnetor infrastructure, controller, pf and
>        sf.
>      - replace representor ID macro with functions:
>        rte_eth_representor_id_encode()
>        rte_eth_representor_id_parse()
>      - new patch to allow devargs with same PCI BDF but different
>        representors.
>      - other minor code updates according to comments, thanks Andrew!
>      - update document
>   V3:
>      - improve probing of allowed devargs with same name.
>      - parse single word of kvargs as key.
>      - update kvargs test cases.
>   V4:
>      - split first representor refactor patch into
>        1: add representor type
>        2: refector representor list parsing
>      - push the patch supporting multi-devargs for same device.
>   V5:
>      - add comments for parsing functions
>      - update switch_representation.rst - Thanks Ajit
>   V6:
>      - split representor types into different patches, move to
>        rte_ethdev.h
>      - improvements of rte_eth_devargs_process_list() according to
>        Andrew's suggestion
>      - fixed PF probe failure for Intel i40e
>      - replace ethdev SF capability with rte_eth_representor_info_get()
>      - add new ethdev ops api to get representor info from PMD
>      - replace representor ID encode/decode with conversion from
>        representor info
>      - change ethdev representor iterator to use new ID encoding
>   V7:
>      - error on duplicated representor argument:
>        DBDF,representor=a,representor=b
>      - fix pf representor type parsing
>      - other small fixes suggested by Andrew.
>   V8:
>   	- remove the "ignore" hint from pmd representor info
>   	- changed representor_id type in representor info to u32
>   	- rename rte_eth_representor_id_convert() to ..._get()
>   	- other small fixes suggested by Andrew
>     
> 
> 
> [1] SubFunction in kernel:
> https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/
> 
> [2] RFC:
> http://patchwork.dpdk.org/project/dpdk/list/?series=14376
> 
> [3] V2:
> http://patchwork.dpdk.org/project/dpdk/list/?series=14559
> 
> [4] V3:
> 1 http://patchwork.dpdk.org/patch/86460/
> 2 http://patchwork.dpdk.org/patch/86461/
> 3 http://patchwork.dpdk.org/patch/86462/
> 4 http://patchwork.dpdk.org/patch/86463/
> 5 http://patchwork.dpdk.org/patch/86464/
> 6 http://patchwork.dpdk.org/patch/86465/
> 7 http://patchwork.dpdk.org/patch/86467/
> 8 http://patchwork.dpdk.org/patch/86466/
> 9 http://patchwork.dpdk.org/patch/86468/
> 
> [5] V4:
> http://patchwork.dpdk.org/project/dpdk/list/?series=14809
> 
> [6] V5:
> http://patchwork.dpdk.org/project/dpdk/list/?series=14834
> 
> [7] V6:
> http://patchwork.dpdk.org/project/dpdk/list/?series=15258
> 
> [8] v7:
> http://patchwork.dpdk.org/project/dpdk/list/?series=15441
> 
> Xueming Li (9):
>    ethdev: introduce representor type
>    ethdev: support representor port list
>    ethdev: support new VF representor syntax
>    ethdev: support sub function representor
>    ethdev: support PF index in representor
>    ethdev: support multi-host in representor
>    ethdev: new API to get representor info
>    ethdev: representor iterator compare complete info
>    kvargs: update parser to support lists
> 
>   app/test/test_kvargs.c                        |  46 ++++-
>   config/rte_config.h                           |   1 +
>   doc/guides/prog_guide/poll_mode_drv.rst       |  18 +-
>   .../prog_guide/switch_representation.rst      |  35 ++--
>   drivers/net/bnxt/bnxt_ethdev.c                |   7 +
>   drivers/net/enic/enic_ethdev.c                |   6 +
>   drivers/net/i40e/i40e_ethdev.c                |   7 +
>   drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
>   drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
>   lib/librte_ethdev/ethdev_driver.h             |  50 +++++-
>   lib/librte_ethdev/ethdev_private.c            | 169 ++++++++++++------
>   lib/librte_ethdev/ethdev_private.h            |   3 -
>   lib/librte_ethdev/rte_class_eth.c             |  40 +++--
>   lib/librte_ethdev/rte_ethdev.c                | 113 +++++++++++-
>   lib/librte_ethdev/rte_ethdev.h                |  61 +++++++
>   lib/librte_ethdev/version.map                 |   4 +
>   lib/librte_kvargs/rte_kvargs.c                | 101 ++++++++---
>   17 files changed, 554 insertions(+), 125 deletions(-)
> 

Btw, can you also update the release notes for the feature, if there will be 
more versions it is OK to hold the update to prevent the release notes rebases 
on each version, but please include it as the patchset matures.

Re: [dpdk-dev] [PATCH v8 7/9] ethdev: new API to get representor info

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Tuesday, March 9, 2021 12:12 AM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
>Subject: Re: [PATCH v8 7/9] ethdev: new API to get representor info
>
>On 3/8/2021 3:31 PM, Xueming(Steven) Li wrote:
>>
>>> -----Original Message-----
>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>> Sent: Monday, March 8, 2021 10:44 PM
>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko
>>> <andrew.rybchenko@oktetlabs.ru>
>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>> Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
>>> <thomas@monjalon.net>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman
>>> <nhorman@tuxdriver.com>
>>> Subject: Re: [PATCH v8 7/9] ethdev: new API to get representor info
>>>
>>> On 3/4/2021 2:30 PM, Xueming Li wrote:
>>>> The NIC can have multiple PCIe links and can be attached to multiple
>>>> hosts, for example the same single NIC can be shared for multiple
>>>> server units in the rack. On each PCIe link NIC can provide multiple
>>>> PFs and VFs/SFs based on these ones. The full representor identifier
>>>> consists of three indices - controller index, PF index, and VF or SF index (if any).
>>>>
>>>> This patch introduces a new API rte_eth_representor_info_get() to
>>>> retrieve representor corresponding info mapping:
>>>>    - caller controller index and pf index.
>>>>    - supported representor ID ranges.
>>>>    - type, controller, pf and start vf/sf ID of each range.
>>>> The API is useful to convert representor from devargs to representor ID.
>>>>
>>>> New ethdev callback representor_info_get() is added to retrieve info
>>>> from PMD driver, optional for PMD that doesn't support new devargs
>>>> representor syntax.
>>>>
>>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>>> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>>
>>> This is middle layer implementation, and there is not problem with it
>>> but without PMD and application implementations it is harder to get why/how this API will be used.
>>>
>>> As far as I can see this API is not directly needed for this set,
>>> what do you think making this another set with PMD and application implementations on top of current set?
>>
>> Hi Ferruh,
>>
>> Thanks for checking this! The patch next, 8/9 which update device iterator for SF representor needs this API to get representor ID
>then compare.
>>
>
>Got it thanks.
>Intention of the new API seems to get info to be able to calculate the unique "representor ID" and the helper function
>'rte_eth_representor_id_get()'
>implements a logic to calculate this unique ID but that logic is not clear, can you please document it more to help the PMD developers
>to implement 'representor_info_get()'?

Sure, will update.

Re: [dpdk-dev] [PATCH v8 8/9] ethdev: representor iterator compare complete info

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Tuesday, March 9, 2021 12:19 AM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
>Subject: Re: [PATCH v8 8/9] ethdev: representor iterator compare complete info
>
>On 3/4/2021 2:30 PM, Xueming Li wrote:
>> The NIC can have multiple PCIe links and can be attached to multiple
>> hosts, for example the same single NIC can be shared for multiple
>> server units in the rack. On each PCIe link NIC can provide multiple
>> PFs and VFs/SFs based on these ones. The full representor identifier
>> consists of three indices - controller index, PF index, and VF or SF index (if any).
>>
>> SR-IOV and SubFunction are created on top of PF. PF index is
>> introduced because there might be multiple PFs in the bonding
>> configuration and only bonding device is probed.
>>
>> In eth representor comparator callback, ethdev representor ID was
>> compared with devarg. Since controller index and PF index not
>> compared, callback returned representor from other PF or controller.
>>
>> This patch adds new API to get representor ID from controller, pf and
>> vf/sf index. Representor comparer callback get representor ID then
>> compare with device representor ID.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>
><...>
>
>> +int
>> +rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
>> +			   enum rte_eth_representor_type type,
>> +			   int controller, int pf, int representor_port,
>> +			   uint16_t *repr_id)
>> +{
>> +	int ret, n, i, count;
>> +	struct rte_eth_representor_info *info = NULL;
>> +	size_t size;
>> +
>> +	if (type == RTE_ETH_REPRESENTOR_NONE)
>> +		return 0;
>> +	if (repr_id == NULL)
>> +		return -EINVAL;
>> +
>> +	/* Get PMD representor range info. */
>> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
>> +	if (ret < 0) {
>
>This seems to support the legacy format, should the return value checked explicitly against the '-ENOTSUP', instead of any error?

Good suggestion, thanks

>
>> +		if (type == RTE_ETH_REPRESENTOR_VF && controller == -1 &&
>> +		    pf == -1) {
>> +			/* Direct mapping for legacy VF representor. */
>> +			*repr_id = representor_port;
>> +			return 0;
>> +		} else {
>> +			return ret;
>> +		}
>> +	}
>
>
><...>

Re: [dpdk-dev] [PATCH v8 2/9] ethdev: support representor port list

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Tuesday, March 9, 2021 12:22 AM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; declan.doherty@intel.com
>Subject: Re: [PATCH v8 2/9] ethdev: support representor port list
>
>On 3/8/2021 3:58 PM, Xueming(Steven) Li wrote:
>>
>>> -----Original Message-----
>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>> Sent: Monday, March 8, 2021 10:38 PM
>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko
>>> <andrew.rybchenko@oktetlabs.ru>
>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf
>>> Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
>>> <thomas@monjalon.net>; declan.doherty@intel.com
>>> Subject: Re: [PATCH v8 2/9] ethdev: support representor port list
>>>
>>> On 3/4/2021 2:30 PM, Xueming Li wrote:
>>>> To support extended representor syntax, this patch extends the
>>>> representor list parsing to support for representor port range in
>>>> devargs, examples:
>>>>      representor=[1,2,3]         - single list
>>>>      representor=[1,3-5,7,9-11]  - list with singles and ranges
>>>>
>>>
>>> Isn't above syntax already supported, can you please document what is added?
>>>
>>>  From documentation, what already supported is:
>>> -w pci:dbdf,representor=0
>>> -w pci:dbdf,representor=[0-3]
>>> -w pci:dbdf,representor=[0,5-11]
>>>
>>
>> Just a refactor to prepare for the extended representor syntax which need to reuse the value parsing function for controller and pf
>section.
>>
>
>OK, can you please highlight this is a refactoring, from patch title and commit log it feels like new syntax support is added.

Sure.

>
>>>
>>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>>> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>>> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>>>
>>>
>>> <...>
>>>
>>

Re: [dpdk-dev] [PATCH v8 0/9] ethdev: support SubFunction representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Tuesday, March 9, 2021 12:44 AM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
>Subject: Re: [dpdk-dev] [PATCH v8 0/9] ethdev: support SubFunction representor
>
>On 3/4/2021 2:30 PM, Xueming Li wrote:
>> SubFunction [1] is a portion of the PCI device, a SF netdev has its
>> own dedicated queues(txq, rxq). A SF netdev supports E-Switch
>> representation offload similar to existing PF and VF representors. A
>> SF shares PCI level resources with other SFs and/or with its parent PCI function.
>>
>>  From SmartNIC perspective, when PCI device is shared for multi-host,
>> representors for host controller and host PF is required.
>>
>> This patch set introduces new representor types in addtion to existing
>> VF representor. Syntax:
>>
>> [[c#]pf#]vf#: VF port representor/s from controller/pf
>> [[c#]pf#]sf#: SF port representor/s from controller/pf
>> #: VF representor - for backwards compatibility
>>
>> "#" is number instance, list or range, valid examples:
>>    1, [1,3,5], [0-3], [0,2-4,6]
>>
>> For backward compatibility, this patch also introduces new netdev
>> capability to indicate the capability of supportting SF representor.
>>
>> Version history:
>>   RFC:
>>   	initial version [2]
>>   V2:
>>      - separate patch for represnetor infrastructure, controller, pf and
>>        sf.
>>      - replace representor ID macro with functions:
>>        rte_eth_representor_id_encode()
>>        rte_eth_representor_id_parse()
>>      - new patch to allow devargs with same PCI BDF but different
>>        representors.
>>      - other minor code updates according to comments, thanks Andrew!
>>      - update document
>>   V3:
>>      - improve probing of allowed devargs with same name.
>>      - parse single word of kvargs as key.
>>      - update kvargs test cases.
>>   V4:
>>      - split first representor refactor patch into
>>        1: add representor type
>>        2: refector representor list parsing
>>      - push the patch supporting multi-devargs for same device.
>>   V5:
>>      - add comments for parsing functions
>>      - update switch_representation.rst - Thanks Ajit
>>   V6:
>>      - split representor types into different patches, move to
>>        rte_ethdev.h
>>      - improvements of rte_eth_devargs_process_list() according to
>>        Andrew's suggestion
>>      - fixed PF probe failure for Intel i40e
>>      - replace ethdev SF capability with rte_eth_representor_info_get()
>>      - add new ethdev ops api to get representor info from PMD
>>      - replace representor ID encode/decode with conversion from
>>        representor info
>>      - change ethdev representor iterator to use new ID encoding
>>   V7:
>>      - error on duplicated representor argument:
>>        DBDF,representor=a,representor=b
>>      - fix pf representor type parsing
>>      - other small fixes suggested by Andrew.
>>   V8:
>>   	- remove the "ignore" hint from pmd representor info
>>   	- changed representor_id type in representor info to u32
>>   	- rename rte_eth_representor_id_convert() to ..._get()
>>   	- other small fixes suggested by Andrew
>>
>>
>>
>> [1] SubFunction in kernel:
>> https://lore.kernel.org/netdev/20201112192424.2742-1-parav@nvidia.com/
>>
>> [2] RFC:
>> http://patchwork.dpdk.org/project/dpdk/list/?series=14376
>>
>> [3] V2:
>> http://patchwork.dpdk.org/project/dpdk/list/?series=14559
>>
>> [4] V3:
>> 1 http://patchwork.dpdk.org/patch/86460/
>> 2 http://patchwork.dpdk.org/patch/86461/
>> 3 http://patchwork.dpdk.org/patch/86462/
>> 4 http://patchwork.dpdk.org/patch/86463/
>> 5 http://patchwork.dpdk.org/patch/86464/
>> 6 http://patchwork.dpdk.org/patch/86465/
>> 7 http://patchwork.dpdk.org/patch/86467/
>> 8 http://patchwork.dpdk.org/patch/86466/
>> 9 http://patchwork.dpdk.org/patch/86468/
>>
>> [5] V4:
>> http://patchwork.dpdk.org/project/dpdk/list/?series=14809
>>
>> [6] V5:
>> http://patchwork.dpdk.org/project/dpdk/list/?series=14834
>>
>> [7] V6:
>> http://patchwork.dpdk.org/project/dpdk/list/?series=15258
>>
>> [8] v7:
>> http://patchwork.dpdk.org/project/dpdk/list/?series=15441
>>
>> Xueming Li (9):
>>    ethdev: introduce representor type
>>    ethdev: support representor port list
>>    ethdev: support new VF representor syntax
>>    ethdev: support sub function representor
>>    ethdev: support PF index in representor
>>    ethdev: support multi-host in representor
>>    ethdev: new API to get representor info
>>    ethdev: representor iterator compare complete info
>>    kvargs: update parser to support lists
>>
>>   app/test/test_kvargs.c                        |  46 ++++-
>>   config/rte_config.h                           |   1 +
>>   doc/guides/prog_guide/poll_mode_drv.rst       |  18 +-
>>   .../prog_guide/switch_representation.rst      |  35 ++--
>>   drivers/net/bnxt/bnxt_ethdev.c                |   7 +
>>   drivers/net/enic/enic_ethdev.c                |   6 +
>>   drivers/net/i40e/i40e_ethdev.c                |   7 +
>>   drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
>>   drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
>>   lib/librte_ethdev/ethdev_driver.h             |  50 +++++-
>>   lib/librte_ethdev/ethdev_private.c            | 169 ++++++++++++------
>>   lib/librte_ethdev/ethdev_private.h            |   3 -
>>   lib/librte_ethdev/rte_class_eth.c             |  40 +++--
>>   lib/librte_ethdev/rte_ethdev.c                | 113 +++++++++++-
>>   lib/librte_ethdev/rte_ethdev.h                |  61 +++++++
>>   lib/librte_ethdev/version.map                 |   4 +
>>   lib/librte_kvargs/rte_kvargs.c                | 101 ++++++++---
>>   17 files changed, 554 insertions(+), 125 deletions(-)
>>
>
>Btw, can you also update the release notes for the feature, if there will be more versions it is OK to hold the update to prevent the
>release notes rebases on each version, but please include it as the patchset matures.

Got it, thanks!

Re: [dpdk-dev] [PATCH v8 8/9] ethdev: representor iterator compare complete info

[Andrew Rybchenko]

On 3/4/21 5:30 PM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> SR-IOV and SubFunction are created on top of PF. PF index is introduced
> because there might be multiple PFs in the bonding configuration and
> only bonding device is probed.
> 
> In eth representor comparator callback, ethdev representor ID was
> compared with devarg. Since controller index and PF index not compared,
> callback returned representor from other PF or controller.
> 
> This patch adds new API to get representor ID from controller, pf and
> vf/sf index. Representor comparer callback get representor ID then
> compare with device representor ID.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>

Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

One minor suggestions below to make it a bit more robust
against errors in PMDs.

Make thanks for careful review notes processing.

[snip]

> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 3d4ec8ad5c..2dc464a7ad 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -5609,6 +5609,94 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>  	return result;
>  }
>  
> +int
> +rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
> +			   enum rte_eth_representor_type type,
> +			   int controller, int pf, int representor_port,
> +			   uint16_t *repr_id)
> +{
> +	int ret, n, i, count;
> +	struct rte_eth_representor_info *info = NULL;
> +	size_t size;
> +
> +	if (type == RTE_ETH_REPRESENTOR_NONE)
> +		return 0;
> +	if (repr_id == NULL)
> +		return -EINVAL;
> +
> +	/* Get PMD representor range info. */
> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
> +	if (ret < 0) {
> +		if (type == RTE_ETH_REPRESENTOR_VF && controller == -1 &&
> +		    pf == -1) {
> +			/* Direct mapping for legacy VF representor. */
> +			*repr_id = representor_port;
> +			return 0;
> +		} else {
> +			return ret;
> +		}
> +	}
> +	n = ret;
> +	size = sizeof(*info) + n * sizeof(info->ranges[0]);
> +	info = calloc(1, size);
> +	if (info == NULL)
> +		return -ENOMEM;
> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
> +	if (ret < 0)
> +		goto out;
> +
> +	/* Default controller and pf to caller. */
> +	if (controller == -1)
> +		controller = info->controller;
> +	if (pf == -1)
> +		pf = info->pf;
> +
> +	/* Locate representor ID. */
> +	ret = -ENOENT;
> +	for (i = 0; i < n; ++i) {
> +		if (info->ranges[i].type != type)
> +			continue;
> +		if (info->ranges[i].controller != controller)
> +			continue;
> +		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;

I think it would be useful to sanity test these values to
ensure that we have no overflow here.
Let's check above, log error and skip the range if id_end is
smaller than id_base.

> +		switch (info->ranges[i].type) {
> +		case RTE_ETH_REPRESENTOR_PF:
> +			if (pf < info->ranges[i].pf ||
> +			    pf >= info->ranges[i].pf + count)
> +				continue;
> +			*repr_id = info->ranges[i].id_base +
> +				   (pf - info->ranges[i].pf);
> +			ret = 0;
> +			goto out;
> +		case RTE_ETH_REPRESENTOR_VF:
> +			if (info->ranges[i].pf != pf)
> +				continue;
> +			if (representor_port < info->ranges[i].vf ||
> +			    representor_port >= info->ranges[i].vf + count)
> +				continue;
> +			*repr_id = info->ranges[i].id_base +
> +				   (representor_port - info->ranges[i].vf);
> +			ret = 0;
> +			goto out;
> +		case RTE_ETH_REPRESENTOR_SF:
> +			if (info->ranges[i].pf != pf)
> +				continue;
> +			if (representor_port < info->ranges[i].sf ||
> +			    representor_port >= info->ranges[i].sf + count)
> +				continue;
> +			*repr_id = info->ranges[i].id_base +
> +			      (representor_port - info->ranges[i].sf);
> +			ret = 0;
> +			goto out;
> +		default:
> +			break;
> +		}
> +	}
> +out:
> +	free(info);
> +	return ret;
> +}
> +
>  static int
>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>  		const char *params __rte_unused,

[snip]

[dpdk-dev] [PATCH v9 00/10] ethdev: support SubFunction representor

[Xueming Li]

SubFunction [1] is a portion of the PCI device, a SF netdev has its own
dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
offload similar to existing PF and VF representors. A SF shares PCI
level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to existing
VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For backward compatibility, this patch also introduces new netdev
capability to indicate the capability of supportting SF representor.

Version history:
 RFC:
 	initial version [2]
 V2:
    - separate patch for represnetor infrastructure, controller, pf and
      sf.
    - replace representor ID macro with functions:
      rte_eth_representor_id_encode()
      rte_eth_representor_id_parse()
    - new patch to allow devargs with same PCI BDF but different
      representors.
    - other minor code updates according to comments, thanks Andrew!
    - update document
 V3:
    - improve probing of allowed devargs with same name.
    - parse single word of kvargs as key.
    - update kvargs test cases.
 V4:
    - split first representor refactor patch into
      1: add representor type
      2: refector representor list parsing
    - push the patch supporting multi-devargs for same device.
 V5:
    - add comments for parsing functions
    - update switch_representation.rst - Thanks Ajit
 V6:
    - split representor types into different patches, move to
      rte_ethdev.h
    - improvements of rte_eth_devargs_process_list() according to
      Andrew's suggestion
    - fixed PF probe failure for Intel i40e
    - replace ethdev SF capability with rte_eth_representor_info_get()
    - add new ethdev ops api to get representor info from PMD
    - replace representor ID encode/decode with conversion from
      representor info
    - change ethdev representor iterator to use new ID encoding
 V7:
    - error on duplicated representor argument:
      DBDF,representor=a,representor=b
    - fix pf representor type parsing
    - other small fixes suggested by Andrew.
 V8:
 	- remove the "ignore" hint from pmd representor info
 	- changed representor_id type in representor info to u32
 	- rename rte_eth_representor_id_convert() to ..._get()
 	- other small fixes suggested by Andrew
 V9:
	- more document for PMD to implement representor_info_get
	  callback
	- updated ice and txgbe to adapt new representor type
	- updated 21.05 release notes
	- other small fixes suggested by Ferruh and Andrew, thanks.


Xueming Li (10):
  ethdev: introduce representor type
  ethdev: refactor representor port list parsing
  ethdev: support new VF representor syntax
  ethdev: support sub function representor
  kvargs: update parser to support multiple lists
  ethdev: support PF index in representor
  ethdev: support multi-host in representor
  ethdev: new API to get representor info
  ethdev: representor iterator compare complete info
  doc/release: add representor enhancements

 app/test/test_kvargs.c                        |  46 ++++-
 config/rte_config.h                           |   1 +
 doc/guides/prog_guide/poll_mode_drv.rst       |  18 +-
 .../prog_guide/switch_representation.rst      |  35 ++--
 doc/guides/rel_notes/release_21_05.rst        |  18 ++
 drivers/net/bnxt/bnxt_ethdev.c                |   7 +
 drivers/net/enic/enic_ethdev.c                |   6 +
 drivers/net/i40e/i40e_ethdev.c                |   7 +
 drivers/net/ice/ice_dcf_ethdev.c              |   2 +
 drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
 drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
 drivers/net/txgbe/txgbe_ethdev.c              |   2 +
 lib/librte_ethdev/ethdev_driver.h             |  70 +++++++-
 lib/librte_ethdev/ethdev_private.c            | 169 ++++++++++++------
 lib/librte_ethdev/ethdev_private.h            |   3 -
 lib/librte_ethdev/rte_class_eth.c             |  40 +++--
 lib/librte_ethdev/rte_ethdev.c                | 118 +++++++++++-
 lib/librte_ethdev/rte_ethdev.h                |  64 +++++++
 lib/librte_ethdev/version.map                 |   4 +
 lib/librte_kvargs/rte_kvargs.c                | 101 ++++++++---
 20 files changed, 604 insertions(+), 125 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH v9 01/10] ethdev: introduce representor type

[Xueming Li]

To support more representor type, this patch introduces representor type
enum. The enum is subject to be extended to support new representor in
patches upcoming.

For each devarg structure, only one type supported.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Hyong Youb Kim <hyonkim@cisco.com>
---
 drivers/net/bnxt/bnxt_ethdev.c     |  7 +++++++
 drivers/net/enic/enic_ethdev.c     |  6 ++++++
 drivers/net/i40e/i40e_ethdev.c     |  7 +++++++
 drivers/net/ice/ice_dcf_ethdev.c   |  2 ++
 drivers/net/ixgbe/ixgbe_ethdev.c   |  7 +++++++
 drivers/net/mlx5/linux/mlx5_os.c   | 11 +++++++++++
 drivers/net/txgbe/txgbe_ethdev.c   |  2 ++
 lib/librte_ethdev/ethdev_driver.h  |  7 ++++++-
 lib/librte_ethdev/ethdev_private.c |  5 +++++
 lib/librte_ethdev/rte_ethdev.h     |  8 ++++++++
 10 files changed, 61 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 9824cdb6d8..55110ee984 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5646,6 +5646,13 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
 	int i, ret = 0;
 	struct rte_kvargs *kvlist = NULL;
 
+	if (eth_da->type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (eth_da->type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type %d\n",
+			    eth_da->type);
+		return -ENOTSUP;
+	}
 	num_rep = eth_da->nb_representor_ports;
 	if (num_rep > BNXT_MAX_VF_REPS) {
 		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index b3f441c8f7..def669f411 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1312,6 +1312,12 @@ static int eth_enic_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (retval)
 			return retval;
 	}
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		ENICPMD_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct enic),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index d7cd049891..9b86bcdc69 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -639,6 +639,13 @@ eth_i40e_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 			return retval;
 	}
 
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct i40e_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index e0772295e9..194f57d7b7 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -1089,6 +1089,8 @@ eth_ice_dcf_pci_probe(__rte_unused struct rte_pci_driver *pci_drv,
 					    ice_dcf_dev_init);
 	if (ret || !eth_da.nb_representor_ports)
 		return ret;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF)
+		return -ENOTSUP;
 
 	dcf_ethdev = rte_eth_dev_allocated(pci_dev->device.name);
 	if (dcf_ethdev == NULL)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 761a0f26bb..8a9a21e7c2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1718,6 +1718,13 @@ eth_ixgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	} else
 		memset(&eth_da, 0, sizeof(eth_da));
 
+	if (eth_da.nb_representor_ports > 0 &&
+	    eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+		PMD_DRV_LOG(ERR, "unsupported representor type: %s\n",
+			    pci_dev->device.devargs->args);
+		return -ENOTSUP;
+	}
+
 	retval = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
 		sizeof(struct ixgbe_adapter),
 		eth_dev_pci_specific_init, pci_dev,
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 81eb2e4b05..92fa178b46 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -751,6 +751,17 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 				strerror(rte_errno));
 			return NULL;
 		}
+		if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) {
+			/* Representor not specified. */
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		if (eth_da.type != RTE_ETH_REPRESENTOR_VF) {
+			rte_errno = ENOTSUP;
+			DRV_LOG(ERR, "unsupported representor type: %s",
+				dpdk_dev->devargs->args);
+			return NULL;
+		}
 		for (i = 0; i < eth_da.nb_representor_ports; ++i)
 			if (eth_da.representor_ports[i] ==
 			    (uint16_t)switch_info->port_name)
diff --git a/drivers/net/txgbe/txgbe_ethdev.c b/drivers/net/txgbe/txgbe_ethdev.c
index 90137d0ceb..a869b06339 100644
--- a/drivers/net/txgbe/txgbe_ethdev.c
+++ b/drivers/net/txgbe/txgbe_ethdev.c
@@ -882,6 +882,8 @@ eth_txgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 
 	if (retval || eth_da.nb_representor_ports < 1)
 		return retval;
+	if (eth_da.type != RTE_ETH_REPRESENTOR_VF)
+		return -ENOTSUP;
 
 	pf_ethdev = rte_eth_dev_allocated(pci_dev->device.name);
 	if (pf_ethdev == NULL)
diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 57fdedaa1a..d68b8401e7 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1216,7 +1216,11 @@ __rte_internal
 int
 rte_eth_switch_domain_free(uint16_t domain_id);
 
-/** Generic Ethernet device arguments  */
+/**
+ * Generic Ethernet device arguments
+ *
+ * One type of representor each structure.
+ */
 struct rte_eth_devargs {
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
@@ -1226,6 +1230,7 @@ struct rte_eth_devargs {
 	/** representor port/s identifier to enable on device */
 	uint16_t nb_representor_ports;
 	/** number of ports in representor port field */
+	enum rte_eth_representor_type type; /* type of representor */
 };
 
 /**
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 5960c75cc5..2d51e3857a 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -111,11 +111,16 @@ rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	return 0;
 }
 
+/*
+ * representor format:
+ *   #: range or single number of VF representor
+ */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
 }
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 059a061072..1f378958ca 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1506,6 +1506,14 @@ struct rte_eth_rxseg_capa {
  * Ethernet device information
  */
 
+/**
+ * Ethernet device representor port type.
+ */
+enum rte_eth_representor_type {
+	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
+	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
+};
+
 /**
  * A structure used to retrieve the contextual information of
  * an Ethernet device, such as the controlling driver of the
-- 
2.25.1

[dpdk-dev] [PATCH v9 02/10] ethdev: refactor representor port list parsing

[Xueming Li]

To the extended representor syntax which need to reuse the value parsing
function for controller and pf section, this patch refactors the port
list parsing.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 lib/librte_ethdev/ethdev_private.c | 129 ++++++++++++++++-------------
 lib/librte_ethdev/ethdev_private.h |   3 -
 lib/librte_ethdev/rte_class_eth.c  |   4 +-
 lib/librte_ethdev/rte_ethdev.c     |   5 +-
 4 files changed, 74 insertions(+), 67 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 2d51e3857a..d5ef05ea6f 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -38,82 +38,91 @@ eth_find_device(const struct rte_eth_dev *start, rte_eth_cmp_t cmp,
 	return NULL;
 }
 
-int
-rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data)
+/* Put new value into list. */
+static int
+rte_eth_devargs_enlist(uint16_t *list, uint16_t *len_list,
+		       const uint16_t max_list, uint16_t val)
 {
-	char *str_start;
-	int state;
-	int result;
-
-	if (*str != '[')
-		/* Single element, not a list */
-		return callback(str, data);
-
-	/* Sanity check, then strip the brackets */
-	str_start = &str[strlen(str) - 1];
-	if (*str_start != ']') {
-		RTE_LOG(ERR, EAL, "(%s): List does not end with ']'\n", str);
-		return -EINVAL;
-	}
-	str++;
-	*str_start = '\0';
+	uint16_t i;
 
-	/* Process list elements */
-	state = 0;
-	while (1) {
-		if (state == 0) {
-			if (*str == '\0')
-				break;
-			if (*str != ',') {
-				str_start = str;
-				state = 1;
-			}
-		} else if (state == 1) {
-			if (*str == ',' || *str == '\0') {
-				if (str > str_start) {
-					/* Non-empty string fragment */
-					*str = '\0';
-					result = callback(str_start, data);
-					if (result < 0)
-						return result;
-				}
-				state = 0;
-			}
-		}
-		str++;
+	for (i = 0; i < *len_list; i++) {
+		if (list[i] == val)
+			return 0;
 	}
+	if (*len_list >= max_list)
+		return -1;
+	list[(*len_list)++] = val;
 	return 0;
 }
 
-static int
+/* Parse and enlist a range expression of "min-max" or a single value. */
+static char *
 rte_eth_devargs_process_range(char *str, uint16_t *list, uint16_t *len_list,
 	const uint16_t max_list)
 {
 	uint16_t lo, hi, val;
-	int result;
+	int result, n = 0;
+	char *pos = str;
 
-	result = sscanf(str, "%hu-%hu", &lo, &hi);
+	result = sscanf(str, "%hu%n-%hu%n", &lo, &n, &hi, &n);
 	if (result == 1) {
-		if (*len_list >= max_list)
-			return -ENOMEM;
-		list[(*len_list)++] = lo;
+		if (rte_eth_devargs_enlist(list, len_list, max_list, lo) != 0)
+			return NULL;
 	} else if (result == 2) {
-		if (lo >= hi || lo > RTE_MAX_ETHPORTS || hi > RTE_MAX_ETHPORTS)
-			return -EINVAL;
+		if (lo > hi)
+			return NULL;
 		for (val = lo; val <= hi; val++) {
-			if (*len_list >= max_list)
-				return -ENOMEM;
-			list[(*len_list)++] = val;
+			if (rte_eth_devargs_enlist(list, len_list, max_list,
+						   val) != 0)
+				return NULL;
 		}
 	} else
-		return -EINVAL;
-	return 0;
+		return NULL;
+	return pos + n;
+}
+
+/*
+ * Parse list of values separated by ",".
+ * Each value could be a range [min-max] or single number.
+ * Examples:
+ *  2               - single
+ *  [1,2,3]         - single list
+ *  [1,3-5,7,9-11]  - list with singles and ranges
+ */
+static char *
+rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
+	const uint16_t max_list)
+{
+	char *pos = str;
+
+	if (*pos == '[')
+		pos++;
+	while (1) {
+		pos = rte_eth_devargs_process_range(pos, list, len_list,
+						    max_list);
+		if (pos == NULL)
+			return NULL;
+		if (*pos != ',') /* end of list */
+			break;
+		pos++;
+	}
+	if (*str == '[' && *pos != ']')
+		return NULL;
+	if (*pos == ']')
+		pos++;
+	return pos;
 }
 
 /*
- * representor format:
+ * Parse representor ports from a single value or lists.
+ *
+ * Representor format:
  *   #: range or single number of VF representor
+ *
+ * Examples of #:
+ *  2               - single
+ *  [1,2,3]         - single list
+ *  [1,3-5,7,9-11]  - list with singles and ranges
  */
 int
 rte_eth_devargs_parse_representor_ports(char *str, void *data)
@@ -121,6 +130,10 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	return rte_eth_devargs_process_range(str, eth_da->representor_ports,
-		&eth_da->nb_representor_ports, RTE_MAX_ETHPORTS);
+	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
+		&eth_da->nb_representor_ports,
+		RTE_DIM(eth_da->representor_ports));
+	if (str == NULL)
+		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
+	return str == NULL ? -1 : 0;
 }
diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h
index 905a45c337..220ddd4408 100644
--- a/lib/librte_ethdev/ethdev_private.h
+++ b/lib/librte_ethdev/ethdev_private.h
@@ -26,9 +26,6 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
 		const void *data);
 
 /* Parse devargs value for representor parameter. */
-typedef int (*rte_eth_devargs_callback_t)(char *str, void *data);
-int rte_eth_devargs_parse_list(char *str, rte_eth_devargs_callback_t callback,
-	void *data);
 int rte_eth_devargs_parse_representor_ports(char *str, void *data);
 
 #ifdef __cplusplus
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index ca2ce87f7a..051c892b40 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -77,9 +77,7 @@ eth_representor_cmp(const char *key __rte_unused,
 	if (values == NULL)
 		return -1;
 	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_list(values,
-			rte_eth_devargs_parse_representor_ports,
-			&representors);
+	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 6f514c388b..fe9466a03e 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5589,9 +5589,8 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
-			result = rte_eth_devargs_parse_list(pair->value,
-				rte_eth_devargs_parse_representor_ports,
-				eth_da);
+			result = rte_eth_devargs_parse_representor_ports(
+					pair->value, eth_da);
 			if (result < 0)
 				goto parse_cleanup;
 		}
-- 
2.25.1

[dpdk-dev] [PATCH v9 03/10] ethdev: support new VF representor syntax

[Xueming Li]

Current VF representor syntax:
 representor=2          - single representor
 representor=[0-3]      - single range

To prepare for more representor types, this patch adds compatible VF
representor devargs syntax:

vf#:
 representor=vf2          - single representor
 representor=vf[1,3,5]    - single list
 representor=vf[0-3]      - single range
 representor=vf[0,1,4-7]  - list with singles and range

For backwards compatibility, representor "#" is interpreted as "vf#".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 7 ++++---
 lib/librte_ethdev/ethdev_private.c      | 5 ++++-
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 239ec820ea..0117c2af07 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -374,9 +374,10 @@ parameters to those ports.
   this argument allows user to specify which switch ports to enable port
   representors for.::
 
-   -a DBDF,representor=0
-   -a DBDF,representor=[0,4,6,9]
-   -a DBDF,representor=[0-31]
+   -a DBDF,representor=vf0
+   -a DBDF,representor=vf[0,4,6,9]
+   -a DBDF,representor=vf[0-31]
+   -a DBDF,representor=vf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index d5ef05ea6f..4bb3879859 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -117,7 +117,8 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Parse representor ports from a single value or lists.
  *
  * Representor format:
- *   #: range or single number of VF representor
+ *   #: range or single number of VF representor - legacy
+ *   vf#: VF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -130,6 +131,8 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 	struct rte_eth_devargs *eth_da = data;
 
 	eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	if (str[0] == 'v' && str[1] == 'f')
+		str += 2;
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
-- 
2.25.1

[dpdk-dev] [PATCH v9 04/10] ethdev: support sub function representor

[Xueming Li]

SubFunction is a portion of the PCI device, created on demand, a SF
netdev has its own dedicated queues(txq, rxq). A SF netdev supports
eswitch representation offload similar to existing PF and VF
representors.

To support SF representor, this patch introduces new devargs syntax,
examples:
 representor=sf0               - single SubFunction representor
 representor=sf[1,3,5]         - single list
 representor=sf[0-3],          - single range
 representor=sf[0,2-6,8,10-12] - list with singles and ranges

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst       |  6 +++-
 .../prog_guide/switch_representation.rst      | 35 +++++++++++++------
 lib/librte_ethdev/ethdev_private.c            | 11 ++++--
 lib/librte_ethdev/rte_ethdev.c                |  6 ++++
 lib/librte_ethdev/rte_ethdev.h                |  1 +
 5 files changed, 46 insertions(+), 13 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 0117c2af07..063a4680df 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -372,12 +372,16 @@ parameters to those ports.
 
 * ``representor`` for a device which supports the creation of representor ports
   this argument allows user to specify which switch ports to enable port
-  representors for.::
+  representors for. Multiple representors in one device argument is invalid::
 
    -a DBDF,representor=vf0
    -a DBDF,representor=vf[0,4,6,9]
    -a DBDF,representor=vf[0-31]
    -a DBDF,representor=vf[0,2-4,7,9-11]
+   -a DBDF,representor=sf0
+   -a DBDF,representor=sf[1,3,5]
+   -a DBDF,representor=sf[0-1023]
+   -a DBDF,representor=sf[0,2-4,7,9-11]
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/doc/guides/prog_guide/switch_representation.rst b/doc/guides/prog_guide/switch_representation.rst
index 07ba12bea6..ff6aa91c80 100644
--- a/doc/guides/prog_guide/switch_representation.rst
+++ b/doc/guides/prog_guide/switch_representation.rst
@@ -13,7 +13,7 @@ Introduction
 
 Network adapters with multiple physical ports and/or SR-IOV capabilities
 usually support the offload of traffic steering rules between their virtual
-functions (VFs), physical functions (PFs) and ports.
+functions (VFs), sub functions (SFs), physical functions (PFs) and ports.
 
 Like for standard Ethernet switches, this involves a combination of
 automatic MAC learning and manual configuration. For most purposes it is
@@ -24,7 +24,7 @@ layer 2 (L2) traffic (such as OVS) need to steer traffic themselves
 according on their own criteria.
 
 Without a standard software interface to manage traffic steering rules
-between VFs, PFs and the various physical ports of a given device,
+between VFs, SFs, PFs and the various physical ports of a given device,
 applications cannot take advantage of these offloads; software processing is
 mandatory even for traffic which ends up re-injected into the device it
 originates from.
@@ -34,6 +34,17 @@ the DPDK flow API (**rte_flow**), with emphasis on the SR-IOV use case
 (PF/VF steering) using a single physical port for clarity, however the same
 logic applies to any number of ports without necessarily involving SR-IOV.
 
+Sub Function
+------------
+Besides SR-IOV, Sub function is a portion of the PCI device, a SF netdev
+has its own dedicated queues(txq, rxq). A SF netdev supports E-Switch
+representation offload similar to existing PF and VF representors.
+A SF shares PCI level resources with other SFs and/or with its parent PCI
+function.
+
+Sub function is created on-demand, coexists with VFs. Number of SFs is
+limited by hardware resources.
+
 Port Representors
 -----------------
 
@@ -42,15 +53,16 @@ applications usually have to process a bit of traffic in software before
 thinking about offloading specific flows to hardware.
 
 Applications therefore need the ability to receive and inject traffic to
-various device endpoints (other VFs, PFs or physical ports) before
+various device endpoints (other VFs, SFs, PFs or physical ports) before
 connecting them together. Device drivers must provide means to hook the
 "other end" of these endpoints and to refer them when configuring flow
 rules.
 
 This role is left to so-called "port representors" (also known as "VF
-representors" in the specific context of VFs), which are to DPDK what the
-Ethernet switch device driver model (**switchdev**) [1]_ is to Linux, and
-which can be thought as a software "patch panel" front-end for applications.
+representors" in the specific context of VFs, "SF representors" in the
+specific context of SFs), which are to DPDK what the Ethernet switch
+device driver model (**switchdev**) [1]_ is to Linux, and which can be
+thought as a software "patch panel" front-end for applications.
 
 - DPDK port representors are implemented as additional virtual Ethernet
   device (**ethdev**) instances, spawned on an as needed basis through
@@ -59,9 +71,12 @@ which can be thought as a software "patch panel" front-end for applications.
 
 ::
 
-   -a pci:dbdf,representor=0
-   -a pci:dbdf,representor=[0-3]
-   -a pci:dbdf,representor=[0,5-11]
+   -a pci:dbdf,representor=vf0
+   -a pci:dbdf,representor=vf[0-3]
+   -a pci:dbdf,representor=vf[0,5-11]
+   -a pci:dbdf,representor=sf1
+   -a pci:dbdf,representor=sf[0-1023]
+   -a pci:dbdf,representor=sf[0,2-1023]
 
 - As virtual devices, they may be more limited than their physical
   counterparts, for instance by exposing only a subset of device
@@ -360,7 +375,7 @@ Compared to creating a brand new dedicated interface, **rte_flow** was
 deemed flexible enough to manage representor traffic only with minor
 extensions:
 
-- Using physical ports, PF, VF or port representors as targets.
+- Using physical ports, PF, SF, VF or port representors as targets.
 
 - Affecting traffic that is not necessarily addressed to the DPDK port ID a
   flow rule is associated with (e.g. forcing VF traffic redirection to PF).
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 4bb3879859..13c191192e 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -119,6 +119,7 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  * Representor format:
  *   #: range or single number of VF representor - legacy
  *   vf#: VF port representor/s
+ *   sf#: SF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -130,9 +131,15 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
-	eth_da->type = RTE_ETH_REPRESENTOR_VF;
-	if (str[0] == 'v' && str[1] == 'f')
+	if (str[0] == 'v' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
+	} else if (str[0] == 's' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_SF;
+		str += 2;
+	} else {
+		eth_da->type = RTE_ETH_REPRESENTOR_VF;
+	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index fe9466a03e..60889a8663 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5589,6 +5589,12 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	for (i = 0; i < args.count; i++) {
 		pair = &args.pairs[i];
 		if (strcmp("representor", pair->key) == 0) {
+			if (eth_da->type != RTE_ETH_REPRESENTOR_NONE) {
+				RTE_LOG(ERR, EAL, "duplicated representor key: %s\n",
+					dargs);
+				result = -1;
+				goto parse_cleanup;
+			}
 			result = rte_eth_devargs_parse_representor_ports(
 					pair->value, eth_da);
 			if (result < 0)
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 1f378958ca..26b5e109c3 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1512,6 +1512,7 @@ struct rte_eth_rxseg_capa {
 enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
 	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
+	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v9 05/10] kvargs: update parser to support multiple lists

[Xueming Li]

This patch updates kvargs parser to support value of multiple lists or
ranges:
  k1=v[1,2]v[3-5]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test/test_kvargs.c         |  46 +++++++++++++--
 lib/librte_kvargs/rte_kvargs.c | 101 +++++++++++++++++++++++----------
 2 files changed, 112 insertions(+), 35 deletions(-)

diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index 2a2dae43a0..a91ea8dc47 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -35,6 +35,25 @@ static int check_handler(const char *key, const char *value,
 	return 0;
 }
 
+/* test parsing. */
+static int test_kvargs_parsing(const char *args, unsigned int n)
+{
+	struct rte_kvargs *kvlist;
+
+	kvlist = rte_kvargs_parse(args, NULL);
+	if (kvlist == NULL) {
+		printf("rte_kvargs_parse() error: %s\n", args);
+		return -1;
+	}
+	if (kvlist->count != n) {
+		printf("invalid count value %d: %s\n", kvlist->count, args);
+		rte_kvargs_free(kvlist);
+		return -1;
+	}
+	rte_kvargs_free(kvlist);
+	return 0;
+}
+
 /* test a valid case */
 static int test_valid_kvargs(void)
 {
@@ -42,6 +61,19 @@ static int test_valid_kvargs(void)
 	const char *args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
 	const char **valid_keys;
+	static const struct {
+		unsigned int expected;
+		const char *input;
+	} valid_inputs[] = {
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo=" },
+		{ 2, "foo=1,foo" },
+		{ 2, "foo=1,=2" },
+		{ 1, "foo=[1,2" },
+		{ 1, ",=" },
+		{ 1, "foo=[" },
+	};
+	unsigned int i;
 
 	/* empty args is valid */
 	args = "";
@@ -191,6 +223,14 @@ static int test_valid_kvargs(void)
 	}
 	rte_kvargs_free(kvlist);
 
+	valid_keys = NULL;
+
+	for (i = 0; i < RTE_DIM(valid_inputs); ++i) {
+		args = valid_inputs[i].input;
+		if (test_kvargs_parsing(args, valid_inputs[i].expected))
+			goto fail;
+	}
+
 	return 0;
 
  fail:
@@ -212,12 +252,6 @@ static int test_invalid_kvargs(void)
 	/* list of argument that should fail */
 	const char *args_list[] = {
 		"wrong-key=x",     /* key not in valid_keys_list */
-		"foo=1,foo=",      /* empty value */
-		"foo=1,foo",       /* no value */
-		"foo=1,=2",        /* no key */
-		"foo=[1,2",        /* no closing bracket in value */
-		",=",              /* also test with a smiley */
-		"foo=[",           /* no value in list and no closing bracket */
 		NULL };
 	const char **args;
 	const char *valid_keys_list[] = { "foo", "check", NULL };
diff --git a/lib/librte_kvargs/rte_kvargs.c b/lib/librte_kvargs/rte_kvargs.c
index 285081c86c..ffae8914cf 100644
--- a/lib/librte_kvargs/rte_kvargs.c
+++ b/lib/librte_kvargs/rte_kvargs.c
@@ -5,6 +5,7 @@
 
 #include <string.h>
 #include <stdlib.h>
+#include <stdbool.h>
 
 #include <rte_string_fns.h>
 
@@ -13,15 +14,19 @@
 /*
  * Receive a string with a list of arguments following the pattern
  * key=value,key=value,... and insert them into the list.
- * strtok() is used so the params string will be copied to be modified.
+ * Params string will be copied to be modified.
+ * list "[]" and list element splitter ",", "-" is treated as value.
+ * Supported examples:
+ *   k1=v1,k2=v2
+ *   k1
+ *   k1=x[0-1]y[1,3-5,9]z
  */
 static int
 rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 {
-	unsigned i;
-	char *str;
-	char *ctx1 = NULL;
-	char *ctx2 = NULL;
+	char *str, *start;
+	bool in_list = false, end_key = false, end_value = false;
+	bool save = false, end_pair = false;
 
 	/* Copy the const char *params to a modifiable string
 	 * to pass to rte_strsplit
@@ -32,36 +37,74 @@ rte_kvargs_tokenize(struct rte_kvargs *kvlist, const char *params)
 
 	/* browse each key/value pair and add it in kvlist */
 	str = kvlist->str;
-	while ((str = strtok_r(str, RTE_KVARGS_PAIRS_DELIM, &ctx1)) != NULL) {
+	start = str; /* start of current key or value */
+	while (1) {
+		switch (*str) {
+		case '=': /* End of key. */
+			end_key = true;
+			save = true;
+			break;
+		case ',':
+			/* End of value, skip comma in middle of range */
+			if (!in_list) {
+				if (end_key)
+					end_value = true;
+				else
+					end_key = true;
+				save = true;
+				end_pair = true;
+			}
+			break;
+		case '[': /* Start of list. */
+			in_list = true;
+			break;
+		case ']': /* End of list.  */
+			if (in_list)
+				in_list = false;
+			break;
+		case '\0': /* End of string */
+			if (end_key)
+				end_value = true;
+			else
+				end_key = true;
+			save = true;
+			end_pair = true;
+			break;
+		default:
+			break;
+		}
 
-		i = kvlist->count;
-		if (i >= RTE_KVARGS_MAX)
-			return -1;
+		if (!save) {
+			/* Continue if not end of key or value. */
+			str++;
+			continue;
+		}
 
-		kvlist->pairs[i].key = strtok_r(str, RTE_KVARGS_KV_DELIM, &ctx2);
-		kvlist->pairs[i].value = strtok_r(NULL, RTE_KVARGS_KV_DELIM, &ctx2);
-		if (kvlist->pairs[i].key == NULL ||
-		    kvlist->pairs[i].value == NULL)
+		if (kvlist->count >= RTE_KVARGS_MAX)
 			return -1;
 
-		/* Detect list [a,b] to skip comma delimiter in list. */
-		str = kvlist->pairs[i].value;
-		if (str[0] == '[') {
-			/* Find the end of the list. */
-			while (str[strlen(str) - 1] != ']') {
-				/* Restore the comma erased by strtok_r(). */
-				if (ctx1 == NULL || ctx1[0] == '\0')
-					return -1; /* no closing bracket */
-				str[strlen(str)] = ',';
-				/* Parse until next comma. */
-				str = strtok_r(NULL, RTE_KVARGS_PAIRS_DELIM, &ctx1);
-				if (str == NULL)
-					return -1; /* no closing bracket */
-			}
+		if (end_value)
+			/* Value parsed */
+			kvlist->pairs[kvlist->count].value = start;
+		else if (end_key)
+			/* Key parsed. */
+			kvlist->pairs[kvlist->count].key = start;
+
+		if (end_pair) {
+			if (end_value || str != start)
+				/* Ignore empty pair. */
+				kvlist->count++;
+			end_key = false;
+			end_value = false;
+			end_pair = false;
 		}
 
-		kvlist->count++;
-		str = NULL;
+		if (*str == '\0') /* End of string. */
+			break;
+		*str = '\0';
+		str++;
+		start = str;
+		save = false;
 	}
 
 	return 0;
-- 
2.25.1

[dpdk-dev] [PATCH v9 06/10] ethdev: support PF index in representor

[Xueming Li]

With Kernel bonding, multiple underlying PFs are bonded, VFs come
from different PF, need to identify representor of VFs unambiguously by
adding PF index.

This patch introduces optional 'pf' section to representor devargs
syntax, examples:
 representor=pf0vf0             - single VF representor
 representor=pf[0-1]sf[0-1023]  - SF representors from 2 PFs

PF type representor is supported by using standalone 'pf' section:
 representor=pf1                - PF representor

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 doc/guides/prog_guide/poll_mode_drv.rst |  3 +++
 lib/librte_ethdev/ethdev_private.c      | 19 +++++++++++++++++--
 lib/librte_ethdev/rte_ethdev.h          |  1 +
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 063a4680df..0d4ac77a7c 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -382,6 +382,9 @@ parameters to those ports.
    -a DBDF,representor=sf[1,3,5]
    -a DBDF,representor=sf[0-1023]
    -a DBDF,representor=sf[0,2-4,7,9-11]
+   -a DBDF,representor=pf1vf0
+   -a DBDF,representor=pf[0-1]sf[0-127]
+   -a DBDF,representor=pf1
 
 Note: PMDs are not required to support the standard device arguments and users
 should consult the relevant PMD documentation to see support devargs.
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 13c191192e..35a57c56ec 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -118,8 +118,9 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   vf#: VF port representor/s
- *   sf#: SF port representor/s
+ *   [pf#]vf#: VF port representor/s
+ *   [pf#]sf#: SF port representor/s
+ *   pf#:      PF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -131,6 +132,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'p' && str[1] == 'f') {
+		eth_da->type = RTE_ETH_REPRESENTOR_PF;
+		str += 2;
+		str = rte_eth_devargs_process_list(str, eth_da->ports,
+				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
+		if (str == NULL || str[0] == '\0')
+			goto done;
+	}
 	if (str[0] == 'v' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 		str += 2;
@@ -138,11 +147,17 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 		eth_da->type = RTE_ETH_REPRESENTOR_SF;
 		str += 2;
 	} else {
+		/* 'pf' must followed by 'vf' or 'sf'. */
+		if (eth_da->type == RTE_ETH_REPRESENTOR_PF) {
+			str = NULL;
+			goto done;
+		}
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
 	}
 	str = rte_eth_devargs_process_list(str, eth_da->representor_ports,
 		&eth_da->nb_representor_ports,
 		RTE_DIM(eth_da->representor_ports));
+done:
 	if (str == NULL)
 		RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
 	return str == NULL ? -1 : 0;
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 26b5e109c3..9cd519bf59 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1513,6 +1513,7 @@ enum rte_eth_representor_type {
 	RTE_ETH_REPRESENTOR_NONE, /**< not a representor. */
 	RTE_ETH_REPRESENTOR_VF,   /**< representor of Virtual Function. */
 	RTE_ETH_REPRESENTOR_SF,   /**< representor of Sub Function. */
+	RTE_ETH_REPRESENTOR_PF,   /**< representor of Physical Function. */
 };
 
 /**
-- 
2.25.1

[dpdk-dev] [PATCH v9 07/10] ethdev: support multi-host in representor

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to the multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. To provide the unambiguous identification
of the PCIe function the controller index is added. The full representor
identifier consists of three indices - controller index, PF index, and
VF or SF index (if any).

This patch introduces controller index to ethdev representor syntax,
examples:

[[c#]pf#]vf#: VF port representor/s, example: pf0vf1
[[c#]pf#]sf#: SF port representor/s, example: c1pf1sf[0-3]

c# is controller(host) ID/range in case of multi-host, optional.

For user application (e.g. OVS), PMD is responsible to interpret and
locate representor device based on controller ID, PF ID and VF/SF ID in
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 config/rte_config.h                |  1 +
 lib/librte_ethdev/ethdev_driver.h  |  4 ++++
 lib/librte_ethdev/ethdev_private.c | 18 +++++++++++++++---
 3 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index 55a2fc50ed..904a40b3c1 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -57,6 +57,7 @@
 #define RTE_MAX_QUEUES_PER_PORT 1024
 #define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */
 #define RTE_ETHDEV_RXTX_CALLBACKS 1
+#define RTE_MAX_MULTI_HOST_CTRLS 4
 
 /* cryptodev defines */
 #define RTE_CRYPTO_MAX_DEVS 64
diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index d68b8401e7..06ff35266f 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1222,6 +1222,10 @@ rte_eth_switch_domain_free(uint16_t domain_id);
  * One type of representor each structure.
  */
 struct rte_eth_devargs {
+	uint16_t mh_controllers[RTE_MAX_MULTI_HOST_CTRLS];
+	/** controller/s number in case of multi-host */
+	uint16_t nb_mh_controllers;
+	/** number of controllers in multi-host controllers field */
 	uint16_t ports[RTE_MAX_ETHPORTS];
 	/** port/s number to enable on a multi-port single function */
 	uint16_t nb_ports;
diff --git a/lib/librte_ethdev/ethdev_private.c b/lib/librte_ethdev/ethdev_private.c
index 35a57c56ec..012cf73ca2 100644
--- a/lib/librte_ethdev/ethdev_private.c
+++ b/lib/librte_ethdev/ethdev_private.c
@@ -118,9 +118,9 @@ rte_eth_devargs_process_list(char *str, uint16_t *list, uint16_t *len_list,
  *
  * Representor format:
  *   #: range or single number of VF representor - legacy
- *   [pf#]vf#: VF port representor/s
- *   [pf#]sf#: SF port representor/s
- *   pf#:      PF port representor/s
+ *   [[c#]pf#]vf#: VF port representor/s
+ *   [[c#]pf#]sf#: SF port representor/s
+ *   [c#]pf#:      PF port representor/s
  *
  * Examples of #:
  *  2               - single
@@ -132,6 +132,14 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 {
 	struct rte_eth_devargs *eth_da = data;
 
+	if (str[0] == 'c') {
+		str += 1;
+		str = rte_eth_devargs_process_list(str, eth_da->mh_controllers,
+				&eth_da->nb_mh_controllers,
+				RTE_DIM(eth_da->mh_controllers));
+		if (str == NULL)
+			goto done;
+	}
 	if (str[0] == 'p' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_PF;
 		str += 2;
@@ -139,6 +147,10 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
 				&eth_da->nb_ports, RTE_DIM(eth_da->ports));
 		if (str == NULL || str[0] == '\0')
 			goto done;
+	} else if (eth_da->nb_mh_controllers > 0) {
+		/* 'c' must followed by 'pf'. */
+		str = NULL;
+		goto done;
 	}
 	if (str[0] == 'v' && str[1] == 'f') {
 		eth_da->type = RTE_ETH_REPRESENTOR_VF;
-- 
2.25.1

[dpdk-dev] [PATCH v9 08/10] ethdev: new API to get representor info

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

This patch introduces a new API rte_eth_representor_info_get() to
retrieve representor corresponding info mapping:
 - caller controller index and pf index.
 - supported representor ID ranges.
 - type, controller, pf and start vf/sf ID of each range.
The API is useful to calculate representor from devargs to representor
ID.

New ethdev callback representor_info_get() is added to retrieve info
from PMD driver, optional for PMD that doesn't support new devargs
representor syntax.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 lib/librte_ethdev/ethdev_driver.h | 27 ++++++++++++++++
 lib/librte_ethdev/rte_ethdev.c    | 14 ++++++++
 lib/librte_ethdev/rte_ethdev.h    | 54 +++++++++++++++++++++++++++++++
 lib/librte_ethdev/version.map     |  3 ++
 4 files changed, 98 insertions(+)

diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 06ff35266f..067be38f33 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -783,6 +783,30 @@ typedef int (*eth_hairpin_queue_peer_unbind_t)
 typedef int (*eth_get_monitor_addr_t)(void *rxq,
 		struct rte_power_monitor_cond *pmc);
 
+/**
+ * @internal
+ * Get representor info to be able to calculate the unique representor ID.
+ *
+ * Caller should pass NULL as pointer of info to get number of entries,
+ * allocate info buffer according to returned entry number, then call
+ * again with buffer to get real info.
+ *
+ * To calculate the representor ID, caller should iterate each entry,
+ * match controller index, pf index, vf or sf start index and range,
+ * then calculate representor ID from offset to vf/sf start index.
+ * @see rte_eth_representor_id_get.
+ *
+ * @param dev
+ *   Ethdev handle of port.
+ * @param [out] info
+ *   Pointer to memory to save device representor info.
+ * @return
+ *   Negative errno value on error, number of info entries otherwise.
+ */
+
+typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
+	struct rte_eth_representor_info *info);
+
 /**
  * @internal A structure containing the functions exported by an Ethernet driver.
  */
@@ -940,6 +964,9 @@ struct eth_dev_ops {
 
 	eth_get_monitor_addr_t get_monitor_addr;
 	/**< Get power monitoring condition for Rx queue. */
+
+	eth_representor_info_get_t representor_info_get;
+	/**< Get representor info. */
 };
 
 /**
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 60889a8663..3d4ec8ad5c 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5816,6 +5816,20 @@ rte_eth_hairpin_queue_peer_unbind(uint16_t cur_port, uint16_t cur_queue,
 							  direction);
 }
 
+int
+rte_eth_representor_info_get(uint16_t port_id,
+			     struct rte_eth_representor_info *info)
+{
+	struct rte_eth_dev *dev;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->representor_info_get, -ENOTSUP);
+	return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev,
+								      info));
+}
+
 RTE_LOG_REGISTER(rte_eth_dev_logtype, lib.ethdev, INFO);
 
 RTE_INIT(ethdev_init_telemetry)
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 9cd519bf59..efda3137f7 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -4791,6 +4791,60 @@ __rte_experimental
 int rte_eth_dev_hairpin_capability_get(uint16_t port_id,
 				       struct rte_eth_hairpin_cap *cap);
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice.
+ *
+ * ethernet device representor ID range entry
+ */
+struct rte_eth_representor_range {
+	enum rte_eth_representor_type type; /**< Representor type */
+	int controller; /**< Controller index */
+	int pf; /**< Physical function index */
+	__extension__
+	union {
+		int vf; /**< VF start index */
+		int sf; /**< SF start index */
+	};
+	uint32_t id_base; /**< Representor ID start index */
+	uint32_t id_end;  /**< Representor ID end index */
+	char name[RTE_DEV_NAME_MAX_LEN]; /**< Representor name */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice.
+ *
+ * Ethernet device representor information
+ */
+struct rte_eth_representor_info {
+	uint16_t controller; /**< Controller ID of caller device. */
+	uint16_t pf; /**< Physical function ID of caller device. */
+	struct rte_eth_representor_range ranges[];/**< Representor ID range. */
+};
+
+/**
+ * Retrieve the representor info of the device.
+ *
+ * Get device representor info to be able to calculate a unique
+ * representor ID. @see rte_eth_representor_id_get helper.
+ *
+ * @param port_id
+ *   The port identifier of the device.
+ * @param info
+ *   A pointer to a representor info structure.
+ *   NULL to return number of range entries and allocate memory
+ *   for next call to store detail.
+ * @return
+ *   - (-ENOTSUP) if operation is not supported.
+ *   - (-ENODEV) if *port_id* invalid.
+ *   - (-EIO) if device is removed.
+ *   - (>=0) number of representor range entries supported by device.
+ */
+__rte_experimental
+int rte_eth_representor_info_get(uint16_t port_id,
+				 struct rte_eth_representor_info *info);
+
 #include <rte_ethdev_core.h>
 
 /**
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index a124e1e370..bb6f7436c2 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -243,6 +243,9 @@ EXPERIMENTAL {
 
 	# added in 21.02
 	rte_eth_get_monitor_addr;
+
+	# added in 21.05
+	rte_eth_representor_info_get;
 };
 
 INTERNAL {
-- 
2.25.1

[dpdk-dev] [PATCH v9 09/10] ethdev: representor iterator compare complete info

[Xueming Li]

The NIC can have multiple PCIe links and can be attached to multiple
hosts, for example the same single NIC can be shared for multiple server
units in the rack. On each PCIe link NIC can provide multiple PFs and
VFs/SFs based on these ones. The full representor identifier consists of
three indices - controller index, PF index, and VF or SF index (if any).

SR-IOV and SubFunction are created on top of PF. PF index is introduced
because there might be multiple PFs in the bonding configuration and
only bonding device is probed.

In eth representor comparator callback, ethdev representor ID was
compared with devarg. Since controller index and PF index not compared,
callback returned representor from other PF or controller.

This patch adds new API to get representor ID from controller, pf and
vf/sf index. Representor comparer callback get representor ID then
compare with device representor ID.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
 lib/librte_ethdev/ethdev_driver.h | 32 +++++++++++
 lib/librte_ethdev/rte_class_eth.c | 38 ++++++++++---
 lib/librte_ethdev/rte_ethdev.c    | 93 +++++++++++++++++++++++++++++++
 lib/librte_ethdev/version.map     |  1 +
 4 files changed, 155 insertions(+), 9 deletions(-)

diff --git a/lib/librte_ethdev/ethdev_driver.h b/lib/librte_ethdev/ethdev_driver.h
index 067be38f33..cdd4b4376c 100644
--- a/lib/librte_ethdev/ethdev_driver.h
+++ b/lib/librte_ethdev/ethdev_driver.h
@@ -1264,6 +1264,38 @@ struct rte_eth_devargs {
 	enum rte_eth_representor_type type; /* type of representor */
 };
 
+/**
+ * PMD helper function to get representor ID from location detail.
+ *
+ * Get representor ID from controller, pf and (sf or vf).
+ * The mapping is retrieved from rte_eth_representor_info_get().
+ *
+ * For backward compatibility, if no representor info, direct
+ * map legacy VF (no controller and pf).
+ *
+ * @param ethdev
+ *  Handle of ethdev port.
+ * @param type
+ *  Representor type.
+ * @param controller
+ *  Controller ID, -1 if unspecified.
+ * @param pf
+ *  PF port ID, -1 if unspecified.
+ * @param representor_port
+ *  VF or SF representor port number, -1 if unspecified.
+ * @param repr_id
+ *  Pointer to output representor ID.
+ *
+ * @return
+ *  Negative errno value on error, 0 on success.
+ */
+__rte_internal
+int
+rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
+			   enum rte_eth_representor_type type,
+			   int controller, int pf, int representor_port,
+			   uint16_t *repr_id);
+
 /**
  * PMD helper function to parse ethdev arguments
  *
diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
index 051c892b40..1fe5fa1f36 100644
--- a/lib/librte_ethdev/rte_class_eth.c
+++ b/lib/librte_ethdev/rte_class_eth.c
@@ -65,9 +65,10 @@ eth_representor_cmp(const char *key __rte_unused,
 {
 	int ret;
 	char *values;
-	const struct rte_eth_dev_data *data = opaque;
-	struct rte_eth_devargs representors;
-	uint16_t index;
+	const struct rte_eth_dev *edev = opaque;
+	const struct rte_eth_dev_data *data = edev->data;
+	struct rte_eth_devargs eth_da;
+	uint16_t id, nc, np, nf, i, c, p, f;
 
 	if ((data->dev_flags & RTE_ETH_DEV_REPRESENTOR) == 0)
 		return -1; /* not a representor port */
@@ -76,17 +77,36 @@ eth_representor_cmp(const char *key __rte_unused,
 	values = strdup(value);
 	if (values == NULL)
 		return -1;
-	memset(&representors, 0, sizeof(representors));
-	ret = rte_eth_devargs_parse_representor_ports(values, &representors);
+	memset(&eth_da, 0, sizeof(eth_da));
+	ret = rte_eth_devargs_parse_representor_ports(values, &eth_da);
 	free(values);
 	if (ret != 0)
 		return -1; /* invalid devargs value */
 
+	if (eth_da.nb_mh_controllers == 0 && eth_da.nb_ports == 0 &&
+	    eth_da.nb_representor_ports == 0)
+		return -1;
+	nc = eth_da.nb_mh_controllers > 0 ? eth_da.nb_mh_controllers : 1;
+	np = eth_da.nb_ports > 0 ? eth_da.nb_ports : 1;
+	nf = eth_da.nb_representor_ports > 0 ? eth_da.nb_representor_ports : 1;
+
 	/* Return 0 if representor id is matching one of the values. */
-	for (index = 0; index < representors.nb_representor_ports; index++)
-		if (data->representor_id ==
-				representors.representor_ports[index])
+	for (i = 0; i < nc * np * nf; ++i) {
+		c = i / (np * nf);
+		p = (i / nf) % np;
+		f = i % nf;
+		if (rte_eth_representor_id_get(edev,
+			eth_da.type,
+			eth_da.nb_mh_controllers == 0 ? -1 :
+					eth_da.mh_controllers[c],
+			eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
+			eth_da.nb_representor_ports == 0 ? -1 :
+					eth_da.representor_ports[f],
+			&id) < 0)
+			continue;
+		if (data->representor_id == id)
 			return 0;
+	}
 	return -1; /* no match */
 }
 
@@ -112,7 +132,7 @@ eth_dev_match(const struct rte_eth_dev *edev,
 
 	ret = rte_kvargs_process(kvlist,
 			eth_params_keys[RTE_ETH_PARAM_REPRESENTOR],
-			eth_representor_cmp, edev->data);
+			eth_representor_cmp, (void *)(uintptr_t)edev);
 	if (ret != 0)
 		return -1;
 	/* search for representor key */
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 3d4ec8ad5c..3059aa55b3 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5609,6 +5609,99 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
 	return result;
 }
 
+int
+rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
+			   enum rte_eth_representor_type type,
+			   int controller, int pf, int representor_port,
+			   uint16_t *repr_id)
+{
+	int ret, n, i, count;
+	struct rte_eth_representor_info *info = NULL;
+	size_t size;
+
+	if (type == RTE_ETH_REPRESENTOR_NONE)
+		return 0;
+	if (repr_id == NULL)
+		return -EINVAL;
+
+	/* Get PMD representor range info. */
+	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
+	if (ret == -ENOTSUP && type == RTE_ETH_REPRESENTOR_VF &&
+	    controller == -1 && pf == -1) {
+		/* Direct mapping for legacy VF representor. */
+		*repr_id = representor_port;
+		return 0;
+	} else if (ret < 0) {
+		return ret;
+	}
+	n = ret;
+	size = sizeof(*info) + n * sizeof(info->ranges[0]);
+	info = calloc(1, size);
+	if (info == NULL)
+		return -ENOMEM;
+	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
+	if (ret < 0)
+		goto out;
+
+	/* Default controller and pf to caller. */
+	if (controller == -1)
+		controller = info->controller;
+	if (pf == -1)
+		pf = info->pf;
+
+	/* Locate representor ID. */
+	ret = -ENOENT;
+	for (i = 0; i < n; ++i) {
+		if (info->ranges[i].type != type)
+			continue;
+		if (info->ranges[i].controller != controller)
+			continue;
+		if (info->ranges[i].id_end < info->ranges[i].id_base) {
+			RTE_LOG(WARNING, EAL, "Port %hu invalid representor ID Range %u - %u, entry %d\n",
+				ethdev->data->port_id, info->ranges[i].id_base,
+				info->ranges[i].id_end, i);
+			continue;
+
+		}
+		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;
+		switch (info->ranges[i].type) {
+		case RTE_ETH_REPRESENTOR_PF:
+			if (pf < info->ranges[i].pf ||
+			    pf >= info->ranges[i].pf + count)
+				continue;
+			*repr_id = info->ranges[i].id_base +
+				   (pf - info->ranges[i].pf);
+			ret = 0;
+			goto out;
+		case RTE_ETH_REPRESENTOR_VF:
+			if (info->ranges[i].pf != pf)
+				continue;
+			if (representor_port < info->ranges[i].vf ||
+			    representor_port >= info->ranges[i].vf + count)
+				continue;
+			*repr_id = info->ranges[i].id_base +
+				   (representor_port - info->ranges[i].vf);
+			ret = 0;
+			goto out;
+		case RTE_ETH_REPRESENTOR_SF:
+			if (info->ranges[i].pf != pf)
+				continue;
+			if (representor_port < info->ranges[i].sf ||
+			    representor_port >= info->ranges[i].sf + count)
+				continue;
+			*repr_id = info->ranges[i].id_base +
+			      (representor_port - info->ranges[i].sf);
+			ret = 0;
+			goto out;
+		default:
+			break;
+		}
+	}
+out:
+	free(info);
+	return ret;
+}
+
 static int
 eth_dev_handle_port_list(const char *cmd __rte_unused,
 		const char *params __rte_unused,
diff --git a/lib/librte_ethdev/version.map b/lib/librte_ethdev/version.map
index bb6f7436c2..93ad388e96 100644
--- a/lib/librte_ethdev/version.map
+++ b/lib/librte_ethdev/version.map
@@ -268,6 +268,7 @@ INTERNAL {
 	rte_eth_hairpin_queue_peer_bind;
 	rte_eth_hairpin_queue_peer_unbind;
 	rte_eth_hairpin_queue_peer_update;
+	rte_eth_representor_id_get;
 	rte_eth_switch_domain_alloc;
 	rte_eth_switch_domain_free;
 };
-- 
2.25.1

[dpdk-dev] [PATCH v9 10/10] doc/release: add representor enhancements

[Xueming Li]

Updates 21.05 release notes to include representor enhancements.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 doc/guides/rel_notes/release_21_05.rst | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/doc/guides/rel_notes/release_21_05.rst b/doc/guides/rel_notes/release_21_05.rst
index 23f7f0bff9..d9e23756d1 100644
--- a/doc/guides/rel_notes/release_21_05.rst
+++ b/doc/guides/rel_notes/release_21_05.rst
@@ -70,6 +70,24 @@ New Features
   * Added command to display Rx queue used descriptor count.
     ``show port (port_id) rxq (queue_id) desc used count``
 
+* **Ethdev representor enhancement**
+
+    Enhanced ethdev representor to support SubFunction and multi-host::
+
+      representor=#            [0,2-4]      /* Legacy VF compatible.         */
+      representor=[[c#]pf#]vf# c1pf2vf3     /* VF 3 on PF 2 of controller 1. */
+      representor=[[c#]pf#]sf# sf[0,2-1023] /* 1023 SFs.                     */
+      representor=[c#]pf#      c2pf[0,1]    /* 2 PFs on controller 2.        */
+
+  * Introduced representor type of VF, SF and PF.
+  * Enhanced representor syntax parsing.
+  * Supported SF and PF type representor.
+  * Supported multi-host in representor.
+  * Added new PMD callback ``representor_info_get`` to get representor info.
+  * Added new helper function ``rte_eth_representor_id_get`` to calculate
+    unique representor ID from info.
+  * Updated representor iterator to match new enhanced syntax.
+
 
 Removed Items
 -------------
-- 
2.25.1

Re: [dpdk-dev] [PATCH v8 8/9] ethdev: representor iterator compare complete info

[Xueming(Steven) Li]

>-----Original Message-----
>From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Sent: Tuesday, March 9, 2021 4:20 PM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman
><nhorman@tuxdriver.com>
>Subject: Re: [PATCH v8 8/9] ethdev: representor iterator compare complete info
>
>On 3/4/21 5:30 PM, Xueming Li wrote:
>> The NIC can have multiple PCIe links and can be attached to multiple
>> hosts, for example the same single NIC can be shared for multiple
>> server units in the rack. On each PCIe link NIC can provide multiple
>> PFs and VFs/SFs based on these ones. The full representor identifier
>> consists of three indices - controller index, PF index, and VF or SF index (if any).
>>
>> SR-IOV and SubFunction are created on top of PF. PF index is
>> introduced because there might be multiple PFs in the bonding
>> configuration and only bonding device is probed.
>>
>> In eth representor comparator callback, ethdev representor ID was
>> compared with devarg. Since controller index and PF index not
>> compared, callback returned representor from other PF or controller.
>>
>> This patch adds new API to get representor ID from controller, pf and
>> vf/sf index. Representor comparer callback get representor ID then
>> compare with device representor ID.
>>
>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>
>Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>
>One minor suggestions below to make it a bit more robust against errors in PMDs.
>
>Make thanks for careful review notes processing.

Many thanks to your generous and careful review all the time, with patience :)
This is my first long public api patchset, can't do it without your help :)

>
>[snip]
>
>> diff --git a/lib/librte_ethdev/rte_ethdev.c
>> b/lib/librte_ethdev/rte_ethdev.c index 3d4ec8ad5c..2dc464a7ad 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -5609,6 +5609,94 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da)
>>  	return result;
>>  }
>>
>> +int
>> +rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
>> +			   enum rte_eth_representor_type type,
>> +			   int controller, int pf, int representor_port,
>> +			   uint16_t *repr_id)
>> +{
>> +	int ret, n, i, count;
>> +	struct rte_eth_representor_info *info = NULL;
>> +	size_t size;
>> +
>> +	if (type == RTE_ETH_REPRESENTOR_NONE)
>> +		return 0;
>> +	if (repr_id == NULL)
>> +		return -EINVAL;
>> +
>> +	/* Get PMD representor range info. */
>> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
>> +	if (ret < 0) {
>> +		if (type == RTE_ETH_REPRESENTOR_VF && controller == -1 &&
>> +		    pf == -1) {
>> +			/* Direct mapping for legacy VF representor. */
>> +			*repr_id = representor_port;
>> +			return 0;
>> +		} else {
>> +			return ret;
>> +		}
>> +	}
>> +	n = ret;
>> +	size = sizeof(*info) + n * sizeof(info->ranges[0]);
>> +	info = calloc(1, size);
>> +	if (info == NULL)
>> +		return -ENOMEM;
>> +	ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
>> +	if (ret < 0)
>> +		goto out;
>> +
>> +	/* Default controller and pf to caller. */
>> +	if (controller == -1)
>> +		controller = info->controller;
>> +	if (pf == -1)
>> +		pf = info->pf;
>> +
>> +	/* Locate representor ID. */
>> +	ret = -ENOENT;
>> +	for (i = 0; i < n; ++i) {
>> +		if (info->ranges[i].type != type)
>> +			continue;
>> +		if (info->ranges[i].controller != controller)
>> +			continue;
>> +		count = info->ranges[i].id_end - info->ranges[i].id_base + 1;
>
>I think it would be useful to sanity test these values to ensure that we have no overflow here.
>Let's check above, log error and skip the range if id_end is smaller than id_base.
>
>> +		switch (info->ranges[i].type) {
>> +		case RTE_ETH_REPRESENTOR_PF:
>> +			if (pf < info->ranges[i].pf ||
>> +			    pf >= info->ranges[i].pf + count)
>> +				continue;
>> +			*repr_id = info->ranges[i].id_base +
>> +				   (pf - info->ranges[i].pf);
>> +			ret = 0;
>> +			goto out;
>> +		case RTE_ETH_REPRESENTOR_VF:
>> +			if (info->ranges[i].pf != pf)
>> +				continue;
>> +			if (representor_port < info->ranges[i].vf ||
>> +			    representor_port >= info->ranges[i].vf + count)
>> +				continue;
>> +			*repr_id = info->ranges[i].id_base +
>> +				   (representor_port - info->ranges[i].vf);
>> +			ret = 0;
>> +			goto out;
>> +		case RTE_ETH_REPRESENTOR_SF:
>> +			if (info->ranges[i].pf != pf)
>> +				continue;
>> +			if (representor_port < info->ranges[i].sf ||
>> +			    representor_port >= info->ranges[i].sf + count)
>> +				continue;
>> +			*repr_id = info->ranges[i].id_base +
>> +			      (representor_port - info->ranges[i].sf);
>> +			ret = 0;
>> +			goto out;
>> +		default:
>> +			break;
>> +		}
>> +	}
>> +out:
>> +	free(info);
>> +	return ret;
>> +}
>> +
>>  static int
>>  eth_dev_handle_port_list(const char *cmd __rte_unused,
>>  		const char *params __rte_unused,
>
>[snip]

Re: [dpdk-dev] [PATCH v9 08/10] ethdev: new API to get representor info

[Ferruh Yigit]

On 3/11/2021 1:13 PM, Xueming Li wrote:
> The NIC can have multiple PCIe links and can be attached to multiple
> hosts, for example the same single NIC can be shared for multiple server
> units in the rack. On each PCIe link NIC can provide multiple PFs and
> VFs/SFs based on these ones. The full representor identifier consists of
> three indices - controller index, PF index, and VF or SF index (if any).
> 
> This patch introduces a new API rte_eth_representor_info_get() to
> retrieve representor corresponding info mapping:
>   - caller controller index and pf index.
>   - supported representor ID ranges.
>   - type, controller, pf and start vf/sf ID of each range.
> The API is useful to calculate representor from devargs to representor
> ID.
> 
> New ethdev callback representor_info_get() is added to retrieve info
> from PMD driver, optional for PMD that doesn't support new devargs
> representor syntax.
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

<...>

> @@ -243,6 +243,9 @@ EXPERIMENTAL {
>   
>   	# added in 21.02
>   	rte_eth_get_monitor_addr;
> +
> +	# added in 21.05
> +	rte_eth_representor_info_get;
>   };

New API documented in release notes, "API Changes" section, while merging.

Re: [dpdk-dev] [PATCH v9 00/10] ethdev: support SubFunction representor

[Ferruh Yigit]

On 3/11/2021 1:13 PM, Xueming Li wrote:
> SubFunction [1] is a portion of the PCI device, a SF netdev has its own
> dedicated queues(txq, rxq). A SF netdev supports E-Switch representation
> offload similar to existing PF and VF representors. A SF shares PCI
> level resources with other SFs and/or with its parent PCI function.
> 
>  From SmartNIC perspective, when PCI device is shared for multi-host,
> representors for host controller and host PF is required.
> 
> This patch set introduces new representor types in addtion to existing
> VF representor. Syntax:
> 
> [[c#]pf#]vf#: VF port representor/s from controller/pf
> [[c#]pf#]sf#: SF port representor/s from controller/pf
> #: VF representor - for backwards compatibility
> 
> "#" is number instance, list or range, valid examples:
>    1, [1,3,5], [0-3], [0,2-4,6]
> 
> For backward compatibility, this patch also introduces new netdev
> capability to indicate the capability of supportting SF representor.
> 
> Version history:
>   RFC:
>   	initial version [2]
>   V2:
>      - separate patch for represnetor infrastructure, controller, pf and
>        sf.
>      - replace representor ID macro with functions:
>        rte_eth_representor_id_encode()
>        rte_eth_representor_id_parse()
>      - new patch to allow devargs with same PCI BDF but different
>        representors.
>      - other minor code updates according to comments, thanks Andrew!
>      - update document
>   V3:
>      - improve probing of allowed devargs with same name.
>      - parse single word of kvargs as key.
>      - update kvargs test cases.
>   V4:
>      - split first representor refactor patch into
>        1: add representor type
>        2: refector representor list parsing
>      - push the patch supporting multi-devargs for same device.
>   V5:
>      - add comments for parsing functions
>      - update switch_representation.rst - Thanks Ajit
>   V6:
>      - split representor types into different patches, move to
>        rte_ethdev.h
>      - improvements of rte_eth_devargs_process_list() according to
>        Andrew's suggestion
>      - fixed PF probe failure for Intel i40e
>      - replace ethdev SF capability with rte_eth_representor_info_get()
>      - add new ethdev ops api to get representor info from PMD
>      - replace representor ID encode/decode with conversion from
>        representor info
>      - change ethdev representor iterator to use new ID encoding
>   V7:
>      - error on duplicated representor argument:
>        DBDF,representor=a,representor=b
>      - fix pf representor type parsing
>      - other small fixes suggested by Andrew.
>   V8:
>   	- remove the "ignore" hint from pmd representor info
>   	- changed representor_id type in representor info to u32
>   	- rename rte_eth_representor_id_convert() to ..._get()
>   	- other small fixes suggested by Andrew
>   V9:
> 	- more document for PMD to implement representor_info_get
> 	  callback
> 	- updated ice and txgbe to adapt new representor type
> 	- updated 21.05 release notes
> 	- other small fixes suggested by Ferruh and Andrew, thanks.
> 
> 
> Xueming Li (10):
>    ethdev: introduce representor type
>    ethdev: refactor representor port list parsing
>    ethdev: support new VF representor syntax
>    ethdev: support sub function representor
>    kvargs: update parser to support multiple lists
>    ethdev: support PF index in representor
>    ethdev: support multi-host in representor
>    ethdev: new API to get representor info
>    ethdev: representor iterator compare complete info
>    doc/release: add representor enhancements
> 

Series applied to dpdk-next-net/main, thanks.

Re: [dpdk-dev] [PATCH v9 08/10] ethdev: new API to get representor info

[Thomas Monjalon]

16/03/2021 20:18, Ferruh Yigit:
> On 3/11/2021 1:13 PM, Xueming Li wrote:
> > +	# added in 21.05
> > +	rte_eth_representor_info_get;
> 
> New API documented in release notes, "API Changes" section, while merging.

In general we don't mention new API in release notes.
What is the intent Ferruh?

Re: [dpdk-dev] [PATCH v9 08/10] ethdev: new API to get representor info

[Ferruh Yigit]

On 3/16/2021 9:19 PM, Thomas Monjalon wrote:
> 16/03/2021 20:18, Ferruh Yigit:
>> On 3/11/2021 1:13 PM, Xueming Li wrote:
>>> +	# added in 21.05
>>> +	rte_eth_representor_info_get;
>>
>> New API documented in release notes, "API Changes" section, while merging.
> 
> In general we don't mention new API in release notes.
> What is the intent Ferruh?
> 

Intent is just document new APIs.
I recognized that new APIs are not mentioned previously, is there a specific 
reason to it? Shouldn't be good to document them?

Re: [dpdk-dev] [PATCH v9 08/10] ethdev: new API to get representor info

[Thomas Monjalon]

17/03/2021 00:34, Ferruh Yigit:
> On 3/16/2021 9:19 PM, Thomas Monjalon wrote:
> > 16/03/2021 20:18, Ferruh Yigit:
> >> On 3/11/2021 1:13 PM, Xueming Li wrote:
> >>> +	# added in 21.05
> >>> +	rte_eth_representor_info_get;
> >>
> >> New API documented in release notes, "API Changes" section, while merging.
> > 
> > In general we don't mention new API in release notes.
> > What is the intent Ferruh?
> > 
> 
> Intent is just document new APIs.
> I recognized that new APIs are not mentioned previously, is there a specific 
> reason to it? Shouldn't be good to document them?

In general, a new API comes with a new feature description.

The API changes section is for documenting what must be adapted
in the applications.
If we add more info in this section, I'm afraid it will be less
straightforward for those who just need to maintain apps.

Re: [dpdk-dev] [PATCH v9 00/10] ethdev: support SubFunction representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Wednesday, March 17, 2021 3:19 AM
>To: Xueming(Steven) Li <xuemingl@nvidia.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
>Subject: Re: [PATCH v9 00/10] ethdev: support SubFunction representor
>
>On 3/11/2021 1:13 PM, Xueming Li wrote:
>> SubFunction [1] is a portion of the PCI device, a SF netdev has its
>> own dedicated queues(txq, rxq). A SF netdev supports E-Switch
>> representation offload similar to existing PF and VF representors. A
>> SF shares PCI level resources with other SFs and/or with its parent PCI function.
>>
>>  From SmartNIC perspective, when PCI device is shared for multi-host,
>> representors for host controller and host PF is required.
>>
>> This patch set introduces new representor types in addtion to existing
>> VF representor. Syntax:
>>
>> [[c#]pf#]vf#: VF port representor/s from controller/pf
>> [[c#]pf#]sf#: SF port representor/s from controller/pf
>> #: VF representor - for backwards compatibility
>>
>> "#" is number instance, list or range, valid examples:
>>    1, [1,3,5], [0-3], [0,2-4,6]
>>
>> For backward compatibility, this patch also introduces new netdev
>> capability to indicate the capability of supportting SF representor.
>>
>> Version history:
>>   RFC:
>>   	initial version [2]
>>   V2:
>>      - separate patch for represnetor infrastructure, controller, pf and
>>        sf.
>>      - replace representor ID macro with functions:
>>        rte_eth_representor_id_encode()
>>        rte_eth_representor_id_parse()
>>      - new patch to allow devargs with same PCI BDF but different
>>        representors.
>>      - other minor code updates according to comments, thanks Andrew!
>>      - update document
>>   V3:
>>      - improve probing of allowed devargs with same name.
>>      - parse single word of kvargs as key.
>>      - update kvargs test cases.
>>   V4:
>>      - split first representor refactor patch into
>>        1: add representor type
>>        2: refector representor list parsing
>>      - push the patch supporting multi-devargs for same device.
>>   V5:
>>      - add comments for parsing functions
>>      - update switch_representation.rst - Thanks Ajit
>>   V6:
>>      - split representor types into different patches, move to
>>        rte_ethdev.h
>>      - improvements of rte_eth_devargs_process_list() according to
>>        Andrew's suggestion
>>      - fixed PF probe failure for Intel i40e
>>      - replace ethdev SF capability with rte_eth_representor_info_get()
>>      - add new ethdev ops api to get representor info from PMD
>>      - replace representor ID encode/decode with conversion from
>>        representor info
>>      - change ethdev representor iterator to use new ID encoding
>>   V7:
>>      - error on duplicated representor argument:
>>        DBDF,representor=a,representor=b
>>      - fix pf representor type parsing
>>      - other small fixes suggested by Andrew.
>>   V8:
>>   	- remove the "ignore" hint from pmd representor info
>>   	- changed representor_id type in representor info to u32
>>   	- rename rte_eth_representor_id_convert() to ..._get()
>>   	- other small fixes suggested by Andrew
>>   V9:
>> 	- more document for PMD to implement representor_info_get
>> 	  callback
>> 	- updated ice and txgbe to adapt new representor type
>> 	- updated 21.05 release notes
>> 	- other small fixes suggested by Ferruh and Andrew, thanks.
>>
>>
>> Xueming Li (10):
>>    ethdev: introduce representor type
>>    ethdev: refactor representor port list parsing
>>    ethdev: support new VF representor syntax
>>    ethdev: support sub function representor
>>    kvargs: update parser to support multiple lists
>>    ethdev: support PF index in representor
>>    ethdev: support multi-host in representor
>>    ethdev: new API to get representor info
>>    ethdev: representor iterator compare complete info
>>    doc/release: add representor enhancements
>>
>
>Series applied to dpdk-next-net/main, thanks.

Thanks!

Re: [dpdk-dev] [PATCH v9 08/10] ethdev: new API to get representor info

[Ferruh Yigit]

On 3/17/2021 6:57 AM, Thomas Monjalon wrote:
> 17/03/2021 00:34, Ferruh Yigit:
>> On 3/16/2021 9:19 PM, Thomas Monjalon wrote:
>>> 16/03/2021 20:18, Ferruh Yigit:
>>>> On 3/11/2021 1:13 PM, Xueming Li wrote:
>>>>> +	# added in 21.05
>>>>> +	rte_eth_representor_info_get;
>>>>
>>>> New API documented in release notes, "API Changes" section, while merging.
>>>
>>> In general we don't mention new API in release notes.
>>> What is the intent Ferruh?
>>>
>>
>> Intent is just document new APIs.
>> I recognized that new APIs are not mentioned previously, is there a specific
>> reason to it? Shouldn't be good to document them?
> 
> In general, a new API comes with a new feature description.
> 
> The API changes section is for documenting what must be adapted
> in the applications.
> If we add more info in this section, I'm afraid it will be less
> straightforward for those who just need to maintain apps.
> 
> 

OK, I will remove it back.

Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor

[Xueming(Steven) Li]

>-----Original Message-----
>From: Wang, Haiyue <haiyue.wang@intel.com>
>Sent: Tuesday, February 23, 2021 7:46 PM
>To: Stephen Hemminger <stephen@networkplumber.org>; Xueming(Steven) Li <xuemingl@nvidia.com>
>Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
>Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor
>
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger
>> Sent: Tuesday, February 23, 2021 09:55
>> To: Xueming Li <xuemingl@nvidia.com>
>> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Asaf
>> Penso <asafp@nvidia.com>
>> Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction
>> representor
>>
>> On Sun, 14 Feb 2021 03:21:30 +0000
>> Xueming Li <xuemingl@nvidia.com> wrote:
>>
>> > SubFunction [1] is a portion of the PCI device, a SF netdev has its
>> > own dedicated queues(txq, rxq). A SF netdev supports E-Switch
>> > representation offload similar to existing PF and VF representors. A
>> > SF shares PCI level resources with other SFs and/or with its parent PCI function.
>> >
>> > From SmartNIC perspective, when PCI device is shared for multi-host,
>> > representors for host controller and host PF is required.
>> >
>> > This patch set introduces new representor types in addtion to
>> > existing VF representor. Syntax:
>> >
>> > [[c#]pf#]vf#: VF port representor/s from controller/pf
>> > [[c#]pf#]sf#: SF port representor/s from controller/pf
>> > #: VF representor - for backwards compatibility
>> >
>> > "#" is number instance, list or range, valid examples:
>> >   1, [1,3,5], [0-3], [0,2-4,6]
>> >
>> > For backward compatibility, this patch also introduces new netdev
>> > capability to indicate the capability of supportting SF representor.
>> >
>> > Version history:
>> >  RFC:
>> >  	initial version [2]
>> >  V2:
>> >     - separate patch for represnetor infrastructure, controller, pf and
>> >       sf.
>> >     - replace representor ID macro with functions:
>> >       rte_eth_representor_id_encode()
>> >       rte_eth_representor_id_parse()
>> >     - new patch to allow devargs with same PCI BDF but different
>> >       representors.
>> >     - other minor code updates according to comments, thanks Andrew!
>> >     - update document
>> >  V3:
>> >     - improve probing of allowed devargs with same name.
>> >     - parse single word of kvargs as key.
>> >     - update kvargs test cases.
>> >  V4:
>> >     - split first representor refactor patch into
>> >       1: add representor type
>> >       2: refector representor list parsing
>> >     - push the patch supporting multi-devargs for same device.
>> >  V5:
>> >     - add comments for parsing functions
>> >     - update switch_representation.rst - Thanks Ajit
>> >  V6:
>> >     - split representor types into different patches, move to
>> >       rte_ethdev.h
>> >     - improvements of rte_eth_devargs_process_list() according to
>> >       Andrew's suggestion
>> >     - fixed PF probe failure for Intel i40e
>> >     - replace ethdev SF capability with rte_eth_representor_info_get()
>> >     - add new ethdev ops api to get representor info from PMD
>> >     - replace representor ID encode/decode with conversion from
>> >       representor info
>> >     - change ethdev representor iterator to use new ID encoding
>> >
>> >
>> > Xueming Li (9):
>> >   ethdev: introduce representor type
>> >   ethdev: support representor port list
>> >   ethdev: support new VF representor syntax
>> >   ethdev: support sub function representor
>> >   ethdev: support PF index in representor
>> >   ethdev: support multi-host in representor
>> >   ethdev: new API to get representor info
>> >   ethdev: representor iterator compare complete info
>> >   kvargs: update parser to support lists
>> >
>> >  app/test/test_kvargs.c                        |  46 ++++-
>> >  config/rte_config.h                           |   1 +
>> >  doc/guides/prog_guide/poll_mode_drv.rst       |  13 +-
>> >  .../prog_guide/switch_representation.rst      |  35 +++-
>> >  drivers/net/bnxt/bnxt_ethdev.c                |   7 +
>> >  drivers/net/enic/enic_ethdev.c                |   6 +
>> >  drivers/net/i40e/i40e_ethdev.c                |   7 +
>> >  drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
>> >  drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
>> >  lib/librte_ethdev/ethdev_driver.h             |  49 ++++-
>> >  lib/librte_ethdev/ethdev_private.c            | 173 ++++++++++++------
>> >  lib/librte_ethdev/ethdev_private.h            |   3 -
>> >  lib/librte_ethdev/rte_class_eth.c             |  40 ++--
>> >  lib/librte_ethdev/rte_ethdev.c                | 102 ++++++++++-
>> >  lib/librte_ethdev/rte_ethdev.h                |  53 ++++++
>> >  lib/librte_ethdev/version.map                 |   4 +
>> >  lib/librte_kvargs/rte_kvargs.c                | 101 +++++++---
>> >  17 files changed, 535 insertions(+), 123 deletions(-)
>> >
>>
>> A couple of higher level design questions:
>> 1. How does Linux and other OS handle this in their API?
>> 2. This solution seems quite tied into a specific implementation on your hardware.
>>    Is there a PCI standard for this?
>
>> 3. Maybe a more general solution where these were represented as buses would
>>    allow for other connection methods in the future?
>
>It should be "auxiliary bus", I think. ;-) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c
>
>mlx5 subfunction support
>https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9becc8799241716672f6

Hi,

The Auxiliary driver for DPDK posted, welcome to have a look :)
http://patchwork.dpdk.org/project/dpdk/patch/20210311130113.31382-1-xuemingl@nvidia.com/

Thanks

Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor

[Wang, Haiyue]

> -----Original Message-----
> From: Xueming(Steven) Li <xuemingl@nvidia.com>
> Sent: Wednesday, March 31, 2021 13:49
> To: Wang, Haiyue <haiyue.wang@intel.com>; Stephen Hemminger <stephen@networkplumber.org>
> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
> Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor
> 
> 
> 
> >-----Original Message-----
> >From: Wang, Haiyue <haiyue.wang@intel.com>
> >Sent: Tuesday, February 23, 2021 7:46 PM
> >To: Stephen Hemminger <stephen@networkplumber.org>; Xueming(Steven) Li <xuemingl@nvidia.com>
> >Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com>
> >Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor
> >
> >> -----Original Message-----
> >> From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger
> >> Sent: Tuesday, February 23, 2021 09:55
> >> To: Xueming Li <xuemingl@nvidia.com>
> >> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Asaf
> >> Penso <asafp@nvidia.com>
> >> Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction
> >> representor
> >>
> >> On Sun, 14 Feb 2021 03:21:30 +0000
> >> Xueming Li <xuemingl@nvidia.com> wrote:
> >>
> >> > SubFunction [1] is a portion of the PCI device, a SF netdev has its
> >> > own dedicated queues(txq, rxq). A SF netdev supports E-Switch
> >> > representation offload similar to existing PF and VF representors. A
> >> > SF shares PCI level resources with other SFs and/or with its parent PCI function.
> >> >
> >> > From SmartNIC perspective, when PCI device is shared for multi-host,
> >> > representors for host controller and host PF is required.
> >> >
> >> > This patch set introduces new representor types in addtion to
> >> > existing VF representor. Syntax:
> >> >
> >> > [[c#]pf#]vf#: VF port representor/s from controller/pf
> >> > [[c#]pf#]sf#: SF port representor/s from controller/pf
> >> > #: VF representor - for backwards compatibility
> >> >
> >> > "#" is number instance, list or range, valid examples:
> >> >   1, [1,3,5], [0-3], [0,2-4,6]
> >> >
> >> > For backward compatibility, this patch also introduces new netdev
> >> > capability to indicate the capability of supportting SF representor.
> >> >
> >> > Version history:
> >> >  RFC:
> >> >  	initial version [2]
> >> >  V2:
> >> >     - separate patch for represnetor infrastructure, controller, pf and
> >> >       sf.
> >> >     - replace representor ID macro with functions:
> >> >       rte_eth_representor_id_encode()
> >> >       rte_eth_representor_id_parse()
> >> >     - new patch to allow devargs with same PCI BDF but different
> >> >       representors.
> >> >     - other minor code updates according to comments, thanks Andrew!
> >> >     - update document
> >> >  V3:
> >> >     - improve probing of allowed devargs with same name.
> >> >     - parse single word of kvargs as key.
> >> >     - update kvargs test cases.
> >> >  V4:
> >> >     - split first representor refactor patch into
> >> >       1: add representor type
> >> >       2: refector representor list parsing
> >> >     - push the patch supporting multi-devargs for same device.
> >> >  V5:
> >> >     - add comments for parsing functions
> >> >     - update switch_representation.rst - Thanks Ajit
> >> >  V6:
> >> >     - split representor types into different patches, move to
> >> >       rte_ethdev.h
> >> >     - improvements of rte_eth_devargs_process_list() according to
> >> >       Andrew's suggestion
> >> >     - fixed PF probe failure for Intel i40e
> >> >     - replace ethdev SF capability with rte_eth_representor_info_get()
> >> >     - add new ethdev ops api to get representor info from PMD
> >> >     - replace representor ID encode/decode with conversion from
> >> >       representor info
> >> >     - change ethdev representor iterator to use new ID encoding
> >> >
> >> >
> >> > Xueming Li (9):
> >> >   ethdev: introduce representor type
> >> >   ethdev: support representor port list
> >> >   ethdev: support new VF representor syntax
> >> >   ethdev: support sub function representor
> >> >   ethdev: support PF index in representor
> >> >   ethdev: support multi-host in representor
> >> >   ethdev: new API to get representor info
> >> >   ethdev: representor iterator compare complete info
> >> >   kvargs: update parser to support lists
> >> >
> >> >  app/test/test_kvargs.c                        |  46 ++++-
> >> >  config/rte_config.h                           |   1 +
> >> >  doc/guides/prog_guide/poll_mode_drv.rst       |  13 +-
> >> >  .../prog_guide/switch_representation.rst      |  35 +++-
> >> >  drivers/net/bnxt/bnxt_ethdev.c                |   7 +
> >> >  drivers/net/enic/enic_ethdev.c                |   6 +
> >> >  drivers/net/i40e/i40e_ethdev.c                |   7 +
> >> >  drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
> >> >  drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
> >> >  lib/librte_ethdev/ethdev_driver.h             |  49 ++++-
> >> >  lib/librte_ethdev/ethdev_private.c            | 173 ++++++++++++------
> >> >  lib/librte_ethdev/ethdev_private.h            |   3 -
> >> >  lib/librte_ethdev/rte_class_eth.c             |  40 ++--
> >> >  lib/librte_ethdev/rte_ethdev.c                | 102 ++++++++++-
> >> >  lib/librte_ethdev/rte_ethdev.h                |  53 ++++++
> >> >  lib/librte_ethdev/version.map                 |   4 +
> >> >  lib/librte_kvargs/rte_kvargs.c                | 101 +++++++---
> >> >  17 files changed, 535 insertions(+), 123 deletions(-)
> >> >
> >>
> >> A couple of higher level design questions:
> >> 1. How does Linux and other OS handle this in their API?
> >> 2. This solution seems quite tied into a specific implementation on your hardware.
> >>    Is there a PCI standard for this?
> >
> >> 3. Maybe a more general solution where these were represented as buses would
> >>    allow for other connection methods in the future?
> >
> >It should be "auxiliary bus", I think. ;-)
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c
> >
> >mlx5 subfunction support
> >https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9be
> cc8799241716672f6
> 
> Hi,
> 
> The Auxiliary driver for DPDK posted, welcome to have a look :)
> http://patchwork.dpdk.org/project/dpdk/patch/20210311130113.31382-1-xuemingl@nvidia.com/

Cool. ;-)

> 
> Thanks
> 

Re: [dpdk-dev] [PATCH v9 05/10] kvargs: update parser to support multiple lists

[Kinsella, Ray]

On 11/03/2021 13:13, Xueming Li wrote:
> This patch updates kvargs parser to support value of multiple lists or
> ranges:
>   k1=v[1,2]v[3-5]
> 
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  app/test/test_kvargs.c         |  46 +++++++++++++--
>  lib/librte_kvargs/rte_kvargs.c | 101 +++++++++++++++++++++++----------
>  2 files changed, 112 insertions(+), 35 deletions(-)
> 

Hi folks,

This is essentially an FYI.
This change introduced a weak functional ABI regression into DPDK.

When I test librte_kvargs using the v20.11 unit test binary I get the following.

EAL: Detected 80 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: 4090 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size
APP: HPET is not enabled, using TSC as default timer
RTE>>kvargs_autotest
== test valid case ==
== test invalid case ==
rte_kvargs_parse() returned 0 (but should not)
while processing <foo=1,foo=> using valid_keys=<foo,check>
Test Failed
RTE>>

The reason this is failing in v20.11 and passing at the HEAD, 
is that "no value" test case (foo=1,foo=) were removed from test_invalid_kvargs, and was added to test_valid_kvargs.

So tokens that were definitely invalid in v20.11, are now is valid in v21.11.
My 2c is that as long as the valid values in v20.11 are still OK, there is no regression.

Thanks,

Ray K