Soft Patch Panel
 help / color / mirror / Atom feed
* [spp] [PATCH 00/17] Adding Hardware offload capability
@ 2020-02-18  6:37 x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 01/17] shared: add support of multi-queue x-fn-spp-ml
                   ` (34 more replies)
  0 siblings, 35 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch set provides hardware offload capability
for spp.
Note that related documents will be provided soon.

Hideyuki Yamashita (17):
  shared: add support of multi-queue
  spp_vf: add support of multi-queue
  spp_mirror: add support of multi-queue
  spp_pcap: add support of multi-queue
  spp_primary: add support of multi-queue
  spp_primary: add support of rte_flow
  spp_primary: add common function of rte_flow
  spp_primary: add attribute of rte_flow
  spp_primary: add patterns of rte_flow
  spp_primary: add actions of rte_flow
  bin: add parameter for hardrare offload
  cli: add support of hardware offload
  cli: add support of rte_flow in vf
  cli: add support of rte_flow in mirror
  cli: add support of rte_flow in nfv
  spp-ctl: add APIs for flow rules
  spp_nfv: add support of multi-queue

 bin/sample/config.sh                          |   15 +
 bin/spp_pri.sh                                |   26 +-
 bin/start.sh                                  |    2 +
 src/cli/commands/mirror.py                    |  175 ++-
 src/cli/commands/nfv.py                       |  245 ++--
 src/cli/commands/pri.py                       |   52 +-
 src/cli/commands/pri_flow.py                  |  936 +++++++++++++++
 src/cli/commands/pri_flow_compl_action.py     |  115 ++
 src/cli/commands/pri_flow_compl_pattern.py    |   84 ++
 src/cli/commands/vf.py                        |  460 +++++---
 src/mirror/mir_cmd_runner.c                   |    3 +-
 src/mirror/spp_mirror.c                       |   12 +-
 src/nfv/commands.h                            |   71 +-
 src/nfv/main.c                                |   18 +-
 src/nfv/nfv_status.c                          |  315 +++--
 src/nfv/nfv_status.h                          |   12 +-
 src/pcap/cmd_runner.c                         |    3 +-
 src/pcap/cmd_utils.c                          |   73 +-
 src/pcap/cmd_utils.h                          |   19 +-
 src/pcap/spp_pcap.c                           |  102 +-
 src/primary/Makefile                          |   10 +
 src/primary/args.c                            |  138 ++-
 src/primary/args.h                            |    3 +
 src/primary/flow/action/jump.c                |   42 +
 src/primary/flow/action/jump.h                |   12 +
 src/primary/flow/action/of_push_vlan.c        |   44 +
 src/primary/flow/action/of_push_vlan.h        |   13 +
 src/primary/flow/action/of_set_vlan_pcp.c     |   44 +
 src/primary/flow/action/of_set_vlan_pcp.h     |   13 +
 src/primary/flow/action/of_set_vlan_vid.c     |   44 +
 src/primary/flow/action/of_set_vlan_vid.h     |   13 +
 src/primary/flow/action/queue.c               |   42 +
 src/primary/flow/action/queue.h               |   13 +
 src/primary/flow/attr.c                       |  105 ++
 src/primary/flow/attr.h                       |   13 +
 src/primary/flow/common.c                     |  646 ++++++++++
 src/primary/flow/common.h                     |   53 +
 src/primary/flow/flow.c                       | 1048 +++++++++++++++++
 src/primary/flow/flow.h                       |   94 ++
 src/primary/flow/pattern/eth.c                |   63 +
 src/primary/flow/pattern/eth.h                |   13 +
 src/primary/flow/pattern/vlan.c               |   71 ++
 src/primary/flow/pattern/vlan.h               |   13 +
 src/primary/init.c                            |    8 +-
 src/primary/init.h                            |    3 +-
 src/primary/main.c                            |   88 +-
 src/shared/basic_forwarder.c                  |   62 +-
 src/shared/basic_forwarder.h                  |    2 +-
 src/shared/common.h                           |   30 +-
 src/shared/port_manager.c                     |  180 ++-
 src/shared/port_manager.h                     |   13 +-
 .../secondary/spp_worker_th/cmd_parser.c      |  137 ++-
 .../spp_worker_th/cmd_res_formatter.c         |   63 +-
 .../secondary/spp_worker_th/cmd_utils.c       |  111 +-
 .../secondary/spp_worker_th/cmd_utils.h       |   19 +-
 .../secondary/spp_worker_th/data_types.h      |   18 +-
 .../secondary/spp_worker_th/port_capability.c |    8 +-
 .../secondary/spp_worker_th/port_capability.h |    4 +-
 src/shared/secondary/spp_worker_th/vf_deps.h  |   15 +-
 src/shared/secondary/utils.c                  |   28 +-
 src/shared/secondary/utils.h                  |    3 +-
 src/spp-ctl/spp_proc.py                       |    4 +
 src/spp-ctl/spp_webapi.py                     |  200 +++-
 src/vf/classifier.c                           |   40 +-
 src/vf/forwarder.c                            |   12 +-
 src/vf/vf_cmd_runner.c                        |   32 +-
 66 files changed, 5624 insertions(+), 744 deletions(-)
 create mode 100644 src/cli/commands/pri_flow.py
 create mode 100644 src/cli/commands/pri_flow_compl_action.py
 create mode 100644 src/cli/commands/pri_flow_compl_pattern.py
 create mode 100644 src/primary/flow/action/jump.c
 create mode 100644 src/primary/flow/action/jump.h
 create mode 100644 src/primary/flow/action/of_push_vlan.c
 create mode 100644 src/primary/flow/action/of_push_vlan.h
 create mode 100644 src/primary/flow/action/of_set_vlan_pcp.c
 create mode 100644 src/primary/flow/action/of_set_vlan_pcp.h
 create mode 100644 src/primary/flow/action/of_set_vlan_vid.c
 create mode 100644 src/primary/flow/action/of_set_vlan_vid.h
 create mode 100644 src/primary/flow/action/queue.c
 create mode 100644 src/primary/flow/action/queue.h
 create mode 100644 src/primary/flow/attr.c
 create mode 100644 src/primary/flow/attr.h
 create mode 100644 src/primary/flow/common.c
 create mode 100644 src/primary/flow/common.h
 create mode 100644 src/primary/flow/flow.c
 create mode 100644 src/primary/flow/flow.h
 create mode 100644 src/primary/flow/pattern/eth.c
 create mode 100644 src/primary/flow/pattern/eth.h
 create mode 100644 src/primary/flow/pattern/vlan.c
 create mode 100644 src/primary/flow/pattern/vlan.h

-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 01/17] shared: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 02/17] spp_vf: " x-fn-spp-ml
                   ` (33 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support multi-queue in SPP,changes like following should be
introduced:
- add queue number parameter in port command
- parse newly added parameter
- add queue number parameter in status response

Under shared directory, common logic among secondary process exist
and those should be affected.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/shared/basic_forwarder.c                  |  62 +++---
 src/shared/basic_forwarder.h                  |   2 +-
 src/shared/common.h                           |  30 ++-
 src/shared/port_manager.c                     | 180 ++++++++++++++----
 src/shared/port_manager.h                     |  13 +-
 .../secondary/spp_worker_th/cmd_parser.c      | 137 ++++++++++---
 .../spp_worker_th/cmd_res_formatter.c         |  63 ++++--
 .../secondary/spp_worker_th/cmd_utils.c       | 111 ++++++++---
 .../secondary/spp_worker_th/cmd_utils.h       |  19 +-
 .../secondary/spp_worker_th/data_types.h      |  18 +-
 .../secondary/spp_worker_th/port_capability.c |   8 +-
 .../secondary/spp_worker_th/port_capability.h |   4 +-
 src/shared/secondary/spp_worker_th/vf_deps.h  |  15 +-
 src/shared/secondary/utils.c                  |  28 ++-
 src/shared/secondary/utils.h                  |   3 +-
 15 files changed, 535 insertions(+), 158 deletions(-)

diff --git a/src/shared/basic_forwarder.c b/src/shared/basic_forwarder.c
index 7aefaaa..66f8643 100644
--- a/src/shared/basic_forwarder.c
+++ b/src/shared/basic_forwarder.c
@@ -5,6 +5,7 @@
 #include <stdint.h>
 #include "shared/common.h"
 #include "shared/basic_forwarder.h"
+#include "shared/port_manager.h"
 
 void
 forward(void)
@@ -14,42 +15,53 @@ forward(void)
 	int in_port;
 	int out_port;
 	uint16_t buf;
-	int i;
+	int i, j;
+	uint16_t max_queue, in_queue, out_queue;
 
 	/* Go through every possible port numbers*/
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
-		struct rte_mbuf *bufs[MAX_PKT_BURST];
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
-			continue;
+		max_queue = get_port_max_queues(i);
 
-		if (ports_fwd_array[i].out_port_id == PORT_RESET)
-			continue;
+		/* Checks the number of rxq declared for the port */
+		for (j = 0; j < max_queue; j++) {
 
-		/* if status active, i count is in port*/
-		in_port = i;
-		out_port = ports_fwd_array[i].out_port_id;
+			struct rte_mbuf *bufs[MAX_PKT_BURST];
 
-		/* Get burst of RX packets, from first port of pair. */
-		/*first port rx, second port tx*/
-		nb_rx = ports_fwd_array[in_port].rx_func(in_port, 0, bufs,
-			MAX_PKT_BURST);
-		if (unlikely(nb_rx == 0))
-			continue;
+			if (ports_fwd_array[i][j].in_port_id == PORT_RESET)
+				continue;
 
-		port_map[in_port].stats->rx += nb_rx;
+			if (ports_fwd_array[i][j].out_port_id == PORT_RESET)
+				continue;
 
-		/* Send burst of TX packets, to second port of pair. */
-		nb_tx = ports_fwd_array[out_port].tx_func(out_port, 0, bufs,
-			nb_rx);
+			/* if status active, i count is in port*/
+			in_port = i;
+			in_queue = j;
+			out_port = ports_fwd_array[i][j].out_port_id;
+			out_queue = ports_fwd_array[i][j].out_queue_id;
 
-		port_map[out_port].stats->tx += nb_tx;
+			/* Get burst of RX packets, from first port of pair. */
+			/*first port rx, second port tx*/
+			nb_rx = ports_fwd_array[in_port][in_queue].rx_func(
+				in_port, in_queue, bufs, MAX_PKT_BURST);
+			if (unlikely(nb_rx == 0))
+				continue;
 
-		/* Free any unsent packets. */
-		if (unlikely(nb_tx < nb_rx)) {
-			port_map[out_port].stats->tx_drop += nb_rx - nb_tx;
-			for (buf = nb_tx; buf < nb_rx; buf++)
-				rte_pktmbuf_free(bufs[buf]);
+			port_map[in_port].stats->rx += nb_rx;
+
+			/* Send burst of TX packets, to second port of pair. */
+			nb_tx = ports_fwd_array[out_port][out_queue].tx_func(
+				out_port, out_queue, bufs, nb_rx);
+
+			port_map[out_port].stats->tx += nb_tx;
+
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_rx)) {
+				port_map[out_port].stats->tx_drop += (nb_rx -
+					nb_tx);
+				for (buf = nb_tx; buf < nb_rx; buf++)
+					rte_pktmbuf_free(bufs[buf]);
+			}
 		}
 	}
 }
diff --git a/src/shared/basic_forwarder.h b/src/shared/basic_forwarder.h
index 2e8225d..915b5cc 100644
--- a/src/shared/basic_forwarder.h
+++ b/src/shared/basic_forwarder.h
@@ -8,7 +8,7 @@
 #include "shared/common.h"
 
 struct port_map port_map[RTE_MAX_ETHPORTS];
-struct port ports_fwd_array[RTE_MAX_ETHPORTS];
+struct port ports_fwd_array[RTE_MAX_ETHPORTS][RTE_MAX_QUEUES_PER_PORT];
 
 void forward(void);
 
diff --git a/src/shared/common.h b/src/shared/common.h
index b4af73c..d311e82 100644
--- a/src/shared/common.h
+++ b/src/shared/common.h
@@ -10,7 +10,23 @@
 #include <unistd.h>
 #include <rte_ethdev_driver.h>
 
-#define MSG_SIZE 2048  /* socket buffer len */
+/*
+ * TODO(tx_h-yamashita): Remove this definition because it was used from
+ * spp_primary and spp_nfv and its value was 2048, but the value should
+ * be changed to 32KiB for containing status of spp_primary.
+ * For spp_nfv, 2048 is still enough. This buff len should be defined in
+ * each of spp_primary and spp_nfv as appropriate size.
+ */
+/*
+ * NOTE: The size of 32768(32Kbyte) is enough to set maximum size of JSON
+ * response string. This maximum size comes from primary`s `status` command
+ * response which includes `LCORE`, `PHY` and `RING` information.
+ * For supporting rte_flow API, `PHY` information may contain a lot of
+ * flow data which defined by user. Our design choice is that `PHY`
+ * information size would be 30Kbyte and `LCORE` size plus `PHY` size
+ * would be 2Kbyte.
+ */
+#define MSG_SIZE 32768  /* socket buffer max len */
 
 #define SOCK_RESET  -1
 #define PORT_RESET  UINT16_MAX
@@ -70,11 +86,19 @@ struct stats {
 	uint64_t tx_drop;
 } __rte_cache_aligned;
 
+/* rx_queue and tx_queue set to port. */
+struct port_queue {
+	uint16_t rxq;
+	uint16_t txq;
+};
+
 struct port_info {
 	uint16_t num_ports;
 	uint16_t id[RTE_MAX_ETHPORTS];
 	struct stats port_stats[RTE_MAX_ETHPORTS];
 	struct stats client_stats[MAX_CLIENT];
+	/* num of queues per port */
+	struct port_queue queue_info[RTE_MAX_ETHPORTS];
 };
 
 enum port_type {
@@ -93,11 +117,15 @@ struct port_map {
 	enum port_type port_type;
 	struct stats *stats;
 	struct stats default_stats;
+	/* num of queues per port */
+	struct port_queue *queue_info;
 };
 
 struct port {
 	uint16_t in_port_id;
+	uint16_t in_queue_id;
 	uint16_t out_port_id;
+	uint16_t out_queue_id;
 	uint16_t (*rx_func)(uint16_t, uint16_t, struct rte_mbuf **, uint16_t);
 	uint16_t (*tx_func)(uint16_t, uint16_t, struct rte_mbuf **, uint16_t);
 };
diff --git a/src/shared/port_manager.c b/src/shared/port_manager.c
index 534af99..6beeca7 100644
--- a/src/shared/port_manager.c
+++ b/src/shared/port_manager.c
@@ -16,35 +16,46 @@ struct porttype_map portmap[] = {
 };
 
 void
-forward_array_init_one(unsigned int i)
+forward_array_init_one(unsigned int i, unsigned int j)
 {
-	ports_fwd_array[i].in_port_id = PORT_RESET;
-	ports_fwd_array[i].out_port_id = PORT_RESET;
+	ports_fwd_array[i][j].in_port_id = PORT_RESET;
+	ports_fwd_array[i][j].in_queue_id = 0;
+	ports_fwd_array[i][j].out_port_id = PORT_RESET;
+	ports_fwd_array[i][j].out_queue_id = 0;
 }
 
 /* initialize forward array with default value */
 void
 forward_array_init(void)
 {
-	unsigned int i;
+	unsigned int i, j;
 
 	/* initialize port forward array*/
-	for (i = 0; i < RTE_MAX_ETHPORTS; i++)
-		forward_array_init_one(i);
+	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		for (j = 0; j < RTE_MAX_QUEUES_PER_PORT; j++)
+			forward_array_init_one(i, j);
+	}
 }
 
 void
 forward_array_reset(void)
 {
-	unsigned int i;
+	unsigned int i, j;
+	uint16_t max_queue;
 
 	/* initialize port forward array*/
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
-		if (ports_fwd_array[i].in_port_id != PORT_RESET) {
-			ports_fwd_array[i].out_port_id = PORT_RESET;
-			RTE_LOG(INFO, SHARED, "Port ID %d\n", i);
-			RTE_LOG(INFO, SHARED, "out_port_id %d\n",
-				ports_fwd_array[i].out_port_id);
+
+		max_queue = get_port_max_queues(i);
+
+		for (j = 0; j < max_queue; j++) {
+			if (ports_fwd_array[i][j].in_port_id != PORT_RESET) {
+				ports_fwd_array[i][j].out_port_id = PORT_RESET;
+				RTE_LOG(INFO, SHARED, "Port ID %d\n", i);
+				RTE_LOG(INFO, SHARED, "Queue ID %d\n", j);
+				RTE_LOG(INFO, SHARED, "out_port_id %d\n",
+					ports_fwd_array[i][j].out_port_id);
+			}
 		}
 	}
 }
@@ -55,6 +66,7 @@ port_map_init_one(unsigned int i)
 	port_map[i].id = PORT_RESET;
 	port_map[i].port_type = UNDEF;
 	port_map[i].stats = &port_map[i].default_stats;
+	port_map[i].queue_info = NULL;
 }
 
 void
@@ -68,28 +80,46 @@ port_map_init(void)
 
 /* Return -1 as an error if given patch is invalid */
 int
-add_patch(uint16_t in_port, uint16_t out_port)
+add_patch(uint16_t in_port, uint16_t in_queue,
+		uint16_t out_port, uint16_t out_queue)
 {
-	if (!is_valid_port(in_port) || !is_valid_port(out_port))
+	if (!is_valid_port(in_port, in_queue) ||
+		!is_valid_port(out_port, out_queue))
 		return -1;
 
+	if (!is_valid_port_rxq(in_port, in_queue) ||
+		!is_valid_port_txq(out_port, out_queue))
+		return 1;
+
 	/* Populate in port data */
-	ports_fwd_array[in_port].in_port_id = in_port;
-	ports_fwd_array[in_port].rx_func = &rte_eth_rx_burst;
-	ports_fwd_array[in_port].tx_func = &rte_eth_tx_burst;
-	ports_fwd_array[in_port].out_port_id = out_port;
+	ports_fwd_array[in_port][in_queue].in_port_id = in_port;
+	ports_fwd_array[in_port][in_queue].in_queue_id = in_queue;
+	ports_fwd_array[in_port][in_queue].rx_func = &rte_eth_rx_burst;
+	ports_fwd_array[in_port][in_queue].tx_func = &rte_eth_tx_burst;
+	ports_fwd_array[in_port][in_queue].out_port_id = out_port;
+	ports_fwd_array[in_port][in_queue].out_queue_id = out_queue;
 
 	/* Populate out port data */
-	ports_fwd_array[out_port].in_port_id = out_port;
-	ports_fwd_array[out_port].rx_func = &rte_eth_rx_burst;
-	ports_fwd_array[out_port].tx_func = &rte_eth_tx_burst;
+	ports_fwd_array[out_port][out_queue].in_port_id = out_port;
+	ports_fwd_array[out_port][out_queue].in_queue_id = out_queue;
+	ports_fwd_array[out_port][out_queue].rx_func = &rte_eth_rx_burst;
+	ports_fwd_array[out_port][out_queue].tx_func = &rte_eth_tx_burst;
 
-	RTE_LOG(DEBUG, SHARED, "STATUS: in port %d in_port_id %d\n", in_port,
-		ports_fwd_array[in_port].in_port_id);
-	RTE_LOG(DEBUG, SHARED, "STATUS: in port %d patch out port id %d\n",
-		in_port, ports_fwd_array[in_port].out_port_id);
-	RTE_LOG(DEBUG, SHARED, "STATUS: outport %d in_port_id %d\n", out_port,
-		ports_fwd_array[out_port].in_port_id);
+	RTE_LOG(DEBUG, SHARED, "STATUS: in port %d in queue %d"
+		" in_port_id %d in_queue_id %d\n",
+		in_port, in_queue,
+		ports_fwd_array[in_port][in_queue].in_port_id,
+		ports_fwd_array[in_port][in_queue].in_queue_id);
+	RTE_LOG(DEBUG, SHARED, "STATUS: in port %d in queue %d"
+		" patch out_port_id %d out_queue_id %d\n",
+		in_port, in_queue,
+		ports_fwd_array[in_port][in_queue].out_port_id,
+		ports_fwd_array[in_port][in_queue].out_queue_id);
+	RTE_LOG(DEBUG, SHARED, "STATUS: out port %d out queue %d"
+		" in_port_id %d in_queue_id %d\n",
+		out_port, out_queue,
+		ports_fwd_array[out_port][out_queue].in_port_id,
+		ports_fwd_array[out_port][out_queue].in_queue_id);
 
 	return 0;
 }
@@ -119,30 +149,92 @@ find_port_id(int id, enum port_type type)
 
 /* Return 0 if invalid */
 int
-is_valid_port(uint16_t port_id)
+is_valid_port(uint16_t port_id, uint16_t queue_id)
 {
+	uint16_t max_queue;
+
 	if (port_id > RTE_MAX_ETHPORTS)
 		return 0;
 
-	return port_map[port_id].id != PORT_RESET;
+	if (port_map[port_id].id == PORT_RESET)
+		return 0;
+
+	max_queue = get_port_max_queues(port_id);
+	if (queue_id >= max_queue)
+		return 0;
+
+	return 1;
+}
+
+/*
+ * Check if rxq exceeds the number of queues defined for the port.
+ * Return 0 if invalid.
+ */
+int
+is_valid_port_rxq(uint16_t port_id, uint16_t rxq)
+{
+	uint16_t nof_queues;
+
+	if (port_map[port_id].queue_info != NULL) {
+		nof_queues = port_map[port_id].queue_info->rxq;
+	} else {
+		/* default number of queues is 1 */
+		nof_queues = 1;
+	}
+	if (rxq >= nof_queues)
+		return 0;
+
+	return 1;
+}
+
+/*
+ * Check if txq exceeds the number of queues defined for the port.
+ * Return 0 if invalid.
+ */
+int
+is_valid_port_txq(uint16_t port_id, uint16_t txq)
+{
+	uint16_t nof_queues;
+
+	if (port_map[port_id].queue_info != NULL) {
+		nof_queues = port_map[port_id].queue_info->txq;
+	} else {
+		/* default number of queues is 1 */
+		nof_queues = 1;
+	}
+	if (txq >= nof_queues)
+		return 0;
+
+	return 1;
 }
 
 void
-forward_array_remove(int port_id)
+forward_array_remove(int port_id, uint16_t queue_id)
 {
-	unsigned int i;
+	unsigned int i, j;
+	uint16_t max_queue;
+	int remove_flg = 0;
 
 	/* Update ports_fwd_array */
-	forward_array_init_one(port_id);
+	forward_array_init_one(port_id, queue_id);
 
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
-			continue;
 
-		if (ports_fwd_array[i].out_port_id == port_id) {
-			ports_fwd_array[i].out_port_id = PORT_RESET;
+		max_queue = get_port_max_queues(i);
+
+		for (j = 0; j < max_queue; j++) {
+			if (ports_fwd_array[i][j].in_port_id == PORT_RESET &&
+				(ports_fwd_array[i][j].out_port_id != port_id ||
+				ports_fwd_array[i][j].out_queue_id != queue_id))
+				continue;
+
+			ports_fwd_array[i][j].out_port_id = PORT_RESET;
+			remove_flg = 1;
 			break;
 		}
+
+		if (remove_flg)
+			break;
 	}
 }
 
@@ -157,3 +249,19 @@ enum port_type get_port_type(char *portname)
 	}
 	return UNDEF;
 }
+
+/* Returns a larger number of queues of RX or TX port as the maximum number */
+uint16_t
+get_port_max_queues(uint16_t port_id)
+{
+	uint16_t max_queue = 1; /* default max_queue is 1 */
+
+	if (port_map[port_id].queue_info != NULL) {
+		if (port_map[port_id].queue_info->rxq >=
+			port_map[port_id].queue_info->txq)
+			max_queue = port_map[port_id].queue_info->rxq;
+		else
+			max_queue = port_map[port_id].queue_info->txq;
+	}
+	return max_queue;
+}
diff --git a/src/shared/port_manager.h b/src/shared/port_manager.h
index c529998..8317687 100644
--- a/src/shared/port_manager.h
+++ b/src/shared/port_manager.h
@@ -17,20 +17,25 @@ struct porttype_map {
 };
 
 /* initialize forward array with default value */
-void forward_array_init_one(unsigned int i);
+void forward_array_init_one(unsigned int i, unsigned int j);
 void forward_array_init(void);
 void forward_array_reset(void);
-void forward_array_remove(int port_id);
+void forward_array_remove(int port_id, uint16_t queue_id);
 
 void port_map_init_one(unsigned int i);
 void port_map_init(void);
 
 enum port_type get_port_type(char *portname);
 
-int add_patch(uint16_t in_port, uint16_t out_port);
+int add_patch(uint16_t in_port, uint16_t in_queue,
+	uint16_t out_port, uint16_t out_queue);
 
 uint16_t find_port_id(int id, enum port_type type);
 
-int is_valid_port(uint16_t port_id);
+int is_valid_port(uint16_t port_id, uint16_t queue_id);
+int is_valid_port_rxq(uint16_t port_id, uint16_t rxq);
+int is_valid_port_txq(uint16_t port_id, uint16_t txq);
+
+uint16_t get_port_max_queues(uint16_t port_id);
 
 #endif  // __SHARED_PORT_MANAGER_H__
diff --git a/src/shared/secondary/spp_worker_th/cmd_parser.c b/src/shared/secondary/spp_worker_th/cmd_parser.c
index 3f947f5..6e66f35 100644
--- a/src/shared/secondary/spp_worker_th/cmd_parser.c
+++ b/src/shared/secondary/spp_worker_th/cmd_parser.c
@@ -120,10 +120,10 @@ const char *PORT_ABILITY_LIST[] = {
 static int
 is_used_with_addr(
 		int vid, uint64_t mac_addr,
-		enum port_type iface_type, int iface_no)
+		enum port_type iface_type, int iface_no, int queue_no)
 {
 	struct sppwk_port_info *wk_port = get_sppwk_port(
-			iface_type, iface_no);
+			iface_type, iface_no, queue_no);
 
 	return ((mac_addr == wk_port->cls_attrs.mac_addr) &&
 		(vid == wk_port->cls_attrs.vlantag.vid));
@@ -131,25 +131,32 @@ is_used_with_addr(
 
 /* Return 1 as true if given port is already used. */
 static int
-is_added_port(enum port_type iface_type, int iface_no)
+is_added_port(enum port_type iface_type, int iface_no, int queue_no)
 {
-	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no);
+	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no,
+			queue_no);
 	return port->iface_type != UNDEF;
 }
 
 /**
- * Separate resource UID of combination of iface type and number and assign to
- * given argument, iface_type and iface_no. For instance, 'ring:0' is separated
- * to 'ring' and '0'. The supported types are `phy`, `vhost` and `ring`.
+ * Extract `iface_type`, `iface_no` and `queue_no` from `res_uid`.
+ * Only phy port has `queue_no`, such as 'phy:0nq1', if using multi-queue.
+ * For other port types other than `phy`,
+ * it returnes `DEFAULT_QUEUE_ID` as `queue_no`.
  */
 static int
 parse_resource_uid(const char *res_uid,
-		    enum port_type *iface_type,
-		    int *iface_no)
+			enum port_type *iface_type,
+			int *iface_no,
+			int *queue_no)
 {
 	enum port_type ptype = UNDEF;
+	const char *iface_no_and_queue_no_str = NULL;
+	const char *queue_no_with_separator_str = NULL;
 	const char *iface_no_str = NULL;
-	char *endptr = NULL;
+	char *queue_no_endptr = NULL;
+	char *iface_no_endptr = NULL;
+	int queue_id, multi_queue_flg;
 
 	/**
 	 * TODO(yasufum) consider this checking of zero value is recommended
@@ -158,33 +165,84 @@ parse_resource_uid(const char *res_uid,
 	if (strncmp(res_uid, SPPWK_PHY_STR ":",
 			strlen(SPPWK_PHY_STR)+1) == 0) {
 		ptype = PHY;
-		iface_no_str = &res_uid[strlen(SPPWK_PHY_STR)+1];
+		iface_no_and_queue_no_str = &res_uid[strlen(SPPWK_PHY_STR)+1];
 	} else if (strncmp(res_uid, SPPWK_VHOST_STR ":",
 			strlen(SPPWK_VHOST_STR)+1) == 0) {
 		ptype = VHOST;
-		iface_no_str = &res_uid[strlen(SPPWK_VHOST_STR)+1];
+		iface_no_and_queue_no_str = &res_uid[strlen(SPPWK_VHOST_STR)+1];
 	} else if (strncmp(res_uid, SPPWK_RING_STR ":",
 			strlen(SPPWK_RING_STR)+1) == 0) {
 		ptype = RING;
-		iface_no_str = &res_uid[strlen(SPPWK_RING_STR)+1];
+		iface_no_and_queue_no_str = &res_uid[strlen(SPPWK_RING_STR)+1];
+	} else {
+		RTE_LOG(ERR, WK_CMD_PARSER, "Unexpected port type in '%s'.\n",
+				res_uid);
+		return SPPWK_RET_NG;
+	}
+
+	/* Parse queue number. */
+	queue_no_with_separator_str = strstr(iface_no_and_queue_no_str,
+			DELIM_PHY_MQ);
+	if (queue_no_with_separator_str == NULL) {
+		iface_no_str = iface_no_and_queue_no_str;
+		queue_id = DEFAULT_QUEUE_ID;
+		multi_queue_flg = 0;
+	} else if (ptype == PHY) {
+		const char *queue_no_str =
+				&queue_no_with_separator_str[
+				strlen(DELIM_PHY_MQ)];
+		queue_id = strtol(queue_no_str, &queue_no_endptr, 0);
+		if (unlikely(queue_no_str == queue_no_endptr) ||
+				unlikely(*queue_no_endptr != '\0')) {
+			RTE_LOG(ERR, WK_CMD_PARSER, "No queue number in '%s'.\n",
+					res_uid);
+			return SPPWK_RET_NG;
+		}
+		iface_no_str = iface_no_and_queue_no_str;
+		multi_queue_flg = 1;
 	} else {
 		RTE_LOG(ERR, WK_CMD_PARSER, "Unexpected port type in '%s'.\n",
 				res_uid);
 		return SPPWK_RET_NG;
 	}
 
-	int port_id = strtol(iface_no_str, &endptr, 0);
-	if (unlikely(iface_no_str == endptr) || unlikely(*endptr != '\0')) {
+	/* Parse interface number. */
+	int port_id = strtol(iface_no_str, &iface_no_endptr, 0);
+	if (unlikely(iface_no_str == iface_no_endptr) ||
+		(unlikely(*iface_no_endptr != '\0') &&
+		(unlikely(strstr(iface_no_endptr,
+			DELIM_PHY_MQ) != iface_no_endptr)))) {
 		RTE_LOG(ERR, WK_CMD_PARSER, "No interface number in '%s'.\n",
 				res_uid);
 		return SPPWK_RET_NG;
 	}
+	if (unlikely(port_id > RTE_MAX_ETHPORTS) || unlikely(port_id < 0)) {
+		RTE_LOG(ERR, WK_CMD_PARSER, "Unexpected interface number in '%s'.\n",
+				res_uid);
+		return SPPWK_RET_NG;
+	}
+
+	/**
+	 * Check whether the queue number is specified according
+	 * to the port format.
+	 */
+	if (ptype == PHY) {
+		int port_multi_queue_flg =
+			(get_port_max_queues(ptype, port_id) > 1);
+		if (unlikely(multi_queue_flg != port_multi_queue_flg)) {
+			RTE_LOG(ERR, WK_CMD_PARSER,
+					"Unexpected queue number format in"
+					" '%s'.\n", res_uid);
+			return SPPWK_RET_NG;
+		}
+	}
 
 	*iface_type = ptype;
 	*iface_no = port_id;
+	*queue_no = queue_id;
 
-	RTE_LOG(DEBUG, WK_CMD_PARSER, "Parsed '%s' to '%d' and '%d'.\n",
-			res_uid, *iface_type, *iface_no);
+	RTE_LOG(DEBUG, WK_CMD_PARSER, "Parsed '%s' to '%d' and '%d' and '%d'.\n",
+			res_uid, *iface_type, *iface_no, *queue_no);
 	return SPPWK_RET_OK;
 }
 
@@ -302,7 +360,8 @@ parse_port_uid(void *output, const char *arg_val)
 {
 	int ret;
 	struct sppwk_port_idx *port = output;
-	ret = parse_resource_uid(arg_val, &port->iface_type, &port->iface_no);
+	ret = parse_resource_uid(arg_val, &port->iface_type, &port->iface_no,
+			&port->queue_no);
 	if (unlikely(ret != 0)) {
 		RTE_LOG(ERR, WK_CMD_PARSER,
 				"Invalid resource UID '%s'.\n", arg_val);
@@ -465,9 +524,11 @@ parse_port(void *output, const char *arg_val, int allow_override)
 		if ((port->wk_action == SPPWK_ACT_ADD) &&
 				(sppwk_check_used_port(tmp_port.iface_type,
 						tmp_port.iface_no,
+						tmp_port.queue_no,
 						SPPWK_PORT_DIR_RX) >= 0) &&
 				(sppwk_check_used_port(tmp_port.iface_type,
 						tmp_port.iface_no,
+						tmp_port.queue_no,
 						SPPWK_PORT_DIR_TX) >= 0)) {
 			RTE_LOG(ERR, WK_CMD_PARSER,
 				"Port `%s` is already used.\n",
@@ -478,6 +539,7 @@ parse_port(void *output, const char *arg_val, int allow_override)
 
 	port->port.iface_type = tmp_port.iface_type;
 	port->port.iface_no   = tmp_port.iface_no;
+	port->port.queue_no   = tmp_port.queue_no;
 	return SPPWK_RET_OK;
 }
 
@@ -495,11 +557,36 @@ parse_port_direction(void *output, const char *arg_val, int allow_override)
 		return SPPWK_RET_NG;
 	}
 
-	/* add vlantag command check */
+	/* Add queue number command check. */
+	if ((port->wk_action == SPPWK_ACT_ADD) &&
+			(port->port.iface_type == PHY)) {
+		const struct port_info *ports = NULL;
+		const struct rte_memzone *mz;
+		mz = rte_memzone_lookup(MZ_PORT_INFO);
+		ports = mz->addr;
+		int max_port_queue = -1;
+		if (ret == SPPWK_PORT_DIR_RX)
+			max_port_queue = ports->queue_info
+					[port->port.iface_no].rxq - 1;
+		else if (ret == SPPWK_PORT_DIR_TX)
+			max_port_queue = ports->queue_info
+					[port->port.iface_no].txq - 1;
+		if (unlikely(port->port.queue_no > max_port_queue)) {
+			RTE_LOG(ERR, WK_CMD_PARSER,
+					"The queue number exceeds the %s max value."
+					" queue=%d, max=%d.\n",
+					PORT_DIR_LIST[ret], port->port.queue_no,
+					max_port_queue);
+			return SPPWK_RET_NG;
+		}
+	}
+
+	/* Add vlantag command check. */
 	if (allow_override == 0) {
 		if ((port->wk_action == SPPWK_ACT_ADD) &&
 				(sppwk_check_used_port(port->port.iface_type,
-					port->port.iface_no, ret) >= 0)) {
+					port->port.iface_no,
+					port->port.queue_no, ret) >= 0)) {
 			RTE_LOG(ERR, WK_CMD_PARSER,
 				"Port in used. (port command) val=%s\n",
 				arg_val);
@@ -726,7 +813,8 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val,
 	if (ret < SPPWK_RET_OK)
 		return SPPWK_RET_NG;
 
-	if (is_added_port(tmp_port.iface_type, tmp_port.iface_no) == 0) {
+	if (is_added_port(tmp_port.iface_type, tmp_port.iface_no,
+			tmp_port.queue_no) == 0) {
 		RTE_LOG(ERR, WK_CMD_PARSER, "Port not added. val=%s\n",
 				arg_val);
 		return SPPWK_RET_NG;
@@ -737,7 +825,8 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val,
 
 	if (unlikely(cls_attrs->wk_action == SPPWK_ACT_ADD)) {
 		if (!is_used_with_addr(ETH_VLAN_ID_MAX, 0,
-				tmp_port.iface_type, tmp_port.iface_no)) {
+				tmp_port.iface_type, tmp_port.iface_no,
+				tmp_port.queue_no)) {
 			RTE_LOG(ERR, WK_CMD_PARSER, "Port in used. "
 					"(classifier_table command) val=%s\n",
 					arg_val);
@@ -750,7 +839,8 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val,
 
 		if (!is_used_with_addr(cls_attrs->vid,
 				(uint64_t)mac_addr,
-				tmp_port.iface_type, tmp_port.iface_no)) {
+				tmp_port.iface_type, tmp_port.iface_no,
+				tmp_port.queue_no)) {
 			RTE_LOG(ERR, WK_CMD_PARSER, "Port in used. "
 					"(classifier_table command) val=%s\n",
 					arg_val);
@@ -760,6 +850,7 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val,
 
 	cls_attrs->port.iface_type = tmp_port.iface_type;
 	cls_attrs->port.iface_no   = tmp_port.iface_no;
+	cls_attrs->port.queue_no   = tmp_port.queue_no;
 	return SPPWK_RET_OK;
 }
 
diff --git a/src/shared/secondary/spp_worker_th/cmd_res_formatter.c b/src/shared/secondary/spp_worker_th/cmd_res_formatter.c
index 85b48af..a04bc91 100644
--- a/src/shared/secondary/spp_worker_th/cmd_res_formatter.c
+++ b/src/shared/secondary/spp_worker_th/cmd_res_formatter.c
@@ -79,9 +79,10 @@ append_error_details_value(const char *name, char **output, void *tmp)
 
 /* Check if port is already flushed. */
 static int
-is_port_flushed(enum port_type iface_type, int iface_no)
+is_port_flushed(enum port_type iface_type, int iface_no, int queue_no)
 {
-	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no);
+	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no,
+			queue_no);
 	return port->ethdev_port_id >= 0;
 }
 
@@ -89,23 +90,43 @@ is_port_flushed(enum port_type iface_type, int iface_no)
 int
 append_interface_array(char **output, const enum port_type type)
 {
-	int i, port_cnt = 0;
+	int port_cnt, str_cnt = 0;
+	uint16_t queue_cnt;
 	char tmp_str[CMD_TAG_APPEND_SIZE];
 
-	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
-		if (!is_port_flushed(type, i))
-			continue;
+	for (port_cnt = 0; port_cnt < RTE_MAX_ETHPORTS; port_cnt++) {
+		int max_queue_port = get_port_max_queues(type, port_cnt);
+		if (unlikely(max_queue_port < 0))
+			return SPPWK_RET_NG;
 
-		sprintf(tmp_str, "%s%d", JSON_APPEND_COMMA(port_cnt), i);
+		for (queue_cnt = 0; queue_cnt < max_queue_port;
+			queue_cnt++) {
+			if (!is_port_flushed(type, port_cnt, queue_cnt))
+				continue;
+
+			if (max_queue_port <= 1) {
+				sprintf(tmp_str, "%s\"%d\"",
+						JSON_APPEND_COMMA(str_cnt),
+						port_cnt);
+			} else {
+				sprintf(tmp_str, "%s\"%d nq %d\"",
+						JSON_APPEND_COMMA(str_cnt),
+						port_cnt, queue_cnt);
+			}
 
-		*output = spp_strbuf_append(*output, tmp_str, strlen(tmp_str));
-		if (unlikely(*output == NULL)) {
-			RTE_LOG(ERR, WK_CMD_RES_FMT,
-				/* TODO(yasufum) replace %d to string. */
-				"Failed to add index for type `%d`.\n", type);
-			return SPPWK_RET_NG;
+			*output = spp_strbuf_append(*output,
+					tmp_str, strlen(tmp_str));
+			if (unlikely(*output == NULL)) {
+				RTE_LOG(ERR, WK_CMD_RES_FMT,
+					/**
+					 * TODO(yasufum) replace %d to string.
+					 */
+					"Failed to add index for type `%d`.\n",
+							type);
+				return SPPWK_RET_NG;
+			}
+			str_cnt++;
 		}
-		port_cnt++;
 	}
 	return SPPWK_RET_OK;
 }
@@ -195,14 +216,14 @@ append_vlan_block(const char *name, char **output,
  * It returns a port ID, or error code if it's failed to.
  */
 static int
-get_ethdev_port_id(enum port_type iface_type, int iface_no)
+get_ethdev_port_id(enum port_type iface_type, int iface_no, int queue_no)
 {
 	struct iface_info *iface_info = NULL;
 
 	sppwk_get_mng_data(&iface_info, NULL, NULL, NULL, NULL, NULL);
 	switch (iface_type) {
 	case PHY:
-		return iface_info->phy[iface_no].ethdev_port_id;
+		return iface_info->phy[iface_no][queue_no].ethdev_port_id;
 	case RING:
 		return iface_info->ring[iface_no].ethdev_port_id;
 	case VHOST:
@@ -227,15 +248,17 @@ append_port_block(char **output, const struct sppwk_port_idx *port,
 		return SPPWK_RET_NG;
 	}
 
-	sppwk_port_uid(port_str, port->iface_type, port->iface_no);
+	sppwk_port_uid(port_str, port->iface_type, port->iface_no,
+			port->queue_no);
 	ret = append_json_str_value(&tmp_buff, "port", port_str);
 	if (unlikely(ret < SPPWK_RET_OK))
 		return SPPWK_RET_NG;
 
 	ret = append_vlan_block("vlan", &tmp_buff,
 			get_ethdev_port_id(
-				port->iface_type, port->iface_no),
-			dir);
+				port->iface_type, port->iface_no,
+				port->queue_no),
+				dir);
 	if (unlikely(ret < SPPWK_RET_OK))
 		return SPPWK_RET_NG;
 
@@ -519,7 +542,7 @@ add_client_id(const char *name, char **output,
 	return append_json_int_value(output, name, get_client_id());
 }
 
-/* Add entry of port to a response in JSON such as "phy:0". */
+/* Add entry of port to a response in JSON such as "phy:0nq0". */
 int
 add_interface(const char *name, char **output,
 		void *tmp __attribute__ ((unused)))
diff --git a/src/shared/secondary/spp_worker_th/cmd_utils.c b/src/shared/secondary/spp_worker_th/cmd_utils.c
index 69d7222..3dea02c 100644
--- a/src/shared/secondary/spp_worker_th/cmd_utils.c
+++ b/src/shared/secondary/spp_worker_th/cmd_utils.c
@@ -154,13 +154,13 @@ stop_process(int signal)
  * if given type is invalid.
  */
 struct sppwk_port_info *
-get_sppwk_port(enum port_type iface_type, int iface_no)
+get_sppwk_port(enum port_type iface_type, int iface_no, int queue_no)
 {
 	struct iface_info *iface_info = g_mng_data.p_iface_info;
 
 	switch (iface_type) {
 	case PHY:
-		return &iface_info->phy[iface_no];
+		return &iface_info->phy[iface_no][queue_no];
 	case VHOST:
 		return &iface_info->vhost[iface_no];
 	case RING:
@@ -229,19 +229,24 @@ log_interface_info(const struct iface_info *iface_info)
 {
 	const struct sppwk_port_info *port = NULL;
 	int cnt = 0;
+	int queue_cnt;
 	for (cnt = 0; cnt < RTE_MAX_ETHPORTS; cnt++) {
-		port = &iface_info->phy[cnt];
-		if (port->iface_type == UNDEF)
-			continue;
-
-		RTE_LOG(DEBUG, WK_CMD_UTILS,
-				"phy  [%d] type=%d, no=%d, port=%d, "
-				"vid = %u, mac=%08lx(%s)\n",
-				cnt, port->iface_type, port->iface_no,
-				port->ethdev_port_id,
-				port->cls_attrs.vlantag.vid,
-				port->cls_attrs.mac_addr,
-				port->cls_attrs.mac_addr_str);
+		int max_queue_port = get_port_max_queues(PHY, cnt);
+		for (queue_cnt = 0; queue_cnt < max_queue_port;
+				queue_cnt++) {
+			port = &iface_info->phy[cnt][queue_cnt];
+			if (port->iface_type == UNDEF)
+				continue;
+
+			RTE_LOG(DEBUG, WK_CMD_UTILS,
+					"phy [%d] type=%d, no=%d, port=%d, "
+					"queue=%d, vid = %u, mac=%08lx(%s)\n",
+					cnt, port->iface_type, port->iface_no,
+					port->ethdev_port_id, port->queue_no,
+					port->cls_attrs.vlantag.vid,
+					port->cls_attrs.mac_addr,
+					port->cls_attrs.mac_addr_str);
+		}
 	}
 	for (cnt = 0; cnt < RTE_MAX_ETHPORTS; cnt++) {
 		port = &iface_info->vhost[cnt];
@@ -356,21 +361,33 @@ static void
 init_iface_info(void)
 {
 	int port_cnt;  /* increment ether ports */
+	int queue_cnt;  /* increment ether queue per ports */
 	struct iface_info *p_iface_info = g_mng_data.p_iface_info;
 	memset(p_iface_info, 0x00, sizeof(struct iface_info));
 	for (port_cnt = 0; port_cnt < RTE_MAX_ETHPORTS; port_cnt++) {
-		p_iface_info->phy[port_cnt].iface_type = UNDEF;
-		p_iface_info->phy[port_cnt].iface_no = port_cnt;
-		p_iface_info->phy[port_cnt].ethdev_port_id = -1;
-		p_iface_info->phy[port_cnt].cls_attrs.vlantag.vid =
-			ETH_VLAN_ID_MAX;
+		for (queue_cnt = 0; queue_cnt < RTE_MAX_QUEUES_PER_PORT;
+				queue_cnt++) {
+			p_iface_info->phy[port_cnt][queue_cnt].iface_type =
+					UNDEF;
+			p_iface_info->phy[port_cnt][queue_cnt].iface_no =
+					port_cnt;
+			p_iface_info->phy[port_cnt][queue_cnt].queue_no =
+					queue_cnt;
+			p_iface_info->phy[port_cnt][queue_cnt]
+					.ethdev_port_id = -1;
+			p_iface_info->phy[port_cnt][queue_cnt]
+					.cls_attrs.vlantag.vid =
+					ETH_VLAN_ID_MAX;
+		}
 		p_iface_info->vhost[port_cnt].iface_type = UNDEF;
 		p_iface_info->vhost[port_cnt].iface_no = port_cnt;
+		p_iface_info->vhost[port_cnt].queue_no = DEFAULT_QUEUE_ID;
 		p_iface_info->vhost[port_cnt].ethdev_port_id = -1;
 		p_iface_info->vhost[port_cnt].cls_attrs.vlantag.vid =
 			ETH_VLAN_ID_MAX;
 		p_iface_info->ring[port_cnt].iface_type = UNDEF;
 		p_iface_info->ring[port_cnt].iface_no = port_cnt;
+		p_iface_info->ring[port_cnt].queue_no = DEFAULT_QUEUE_ID;
 		p_iface_info->ring[port_cnt].ethdev_port_id = -1;
 		p_iface_info->ring[port_cnt].cls_attrs.vlantag.vid =
 			ETH_VLAN_ID_MAX;
@@ -411,7 +428,7 @@ static int
 init_host_port_info(void)
 {
 	int port_type, port_id;
-	int i, ret;
+	int i, ret, queue_id;
 	int nof_phys = 0;
 	char dev_name[RTE_DEV_NAME_MAX_LEN] = { 0 };
 	struct iface_info *p_iface_info = g_mng_data.p_iface_info;
@@ -433,8 +450,13 @@ init_host_port_info(void)
 
 		switch (port_type) {
 		case PHY:
-			p_iface_info->phy[port_id].iface_type = port_type;
-			p_iface_info->phy[port_id].ethdev_port_id = port_id;
+			for (queue_id = 0; queue_id < RTE_MAX_QUEUES_PER_PORT;
+					queue_id++) {
+				p_iface_info->phy[port_id][queue_id]
+						.iface_type = port_type;
+				p_iface_info->phy[port_id][queue_id]
+						.ethdev_port_id = port_id;
+			}
 			break;
 		case VHOST:
 			/* NOTE: a vhost can be used by one process.
@@ -526,12 +548,14 @@ int
 sppwk_check_used_port(
 		enum port_type iface_type,
 		int iface_no,
+		int queue_no,
 		enum sppwk_port_dir dir)
 {
 	int cnt, port_cnt, max = 0;
 	struct sppwk_comp_info *component = NULL;
 	struct sppwk_port_info **port_array = NULL;
-	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no);
+	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no,
+			queue_no);
 	struct sppwk_comp_info *component_info =
 					g_mng_data.p_component_info;
 
@@ -567,14 +591,14 @@ set_component_change_port(struct sppwk_port_info *port,
 	int ret = 0;
 	if ((dir == SPPWK_PORT_DIR_RX) || (dir == SPPWK_PORT_DIR_BOTH)) {
 		ret = sppwk_check_used_port(port->iface_type, port->iface_no,
-				SPPWK_PORT_DIR_RX);
+				port->queue_no, SPPWK_PORT_DIR_RX);
 		if (ret >= 0)
 			*(g_mng_data.p_change_component + ret) = 1;
 	}
 
 	if ((dir == SPPWK_PORT_DIR_TX) || (dir == SPPWK_PORT_DIR_BOTH)) {
 		ret = sppwk_check_used_port(port->iface_type, port->iface_no,
-				SPPWK_PORT_DIR_TX);
+				port->queue_no, SPPWK_PORT_DIR_TX);
 		if (ret >= 0)
 			*(g_mng_data.p_change_component + ret) = 1;
 	}
@@ -628,6 +652,28 @@ get_idx_port_info(struct sppwk_port_info *p_info, int nof_ports,
 	return ret;
 }
 
+/* Returns a larger number of queues of RX or TX port as the maximum number */
+int
+get_port_max_queues(const enum port_type iface_type, int iface_no)
+{
+	if (unlikely(iface_no > RTE_MAX_ETHPORTS) || unlikely(iface_no < 0))
+		return SPPWK_RET_NG;
+
+	if (iface_type != PHY)
+		return 1;
+
+	const struct port_info *ports = NULL;
+	const struct rte_memzone *mz;
+	mz = rte_memzone_lookup(MZ_PORT_INFO);
+	ports = mz->addr;
+	int max_q_rx = ports->queue_info[iface_no].rxq;
+	int max_q_tx = ports->queue_info[iface_no].txq;
+	if (max_q_rx > max_q_tx)
+		return max_q_rx;
+	else
+		return max_q_tx;
+}
+
 /* Delete given port info from the port info array. */
 int
 delete_port_info(struct sppwk_port_info *p_info, int nof_ports,
@@ -716,8 +762,9 @@ update_lcore_info(void)
 	}
 }
 
-/* Return port uid such as `phy:0`, `ring:1` or so. */
-int sppwk_port_uid(char *port_uid, enum port_type p_type, int iface_no)
+/* Return port uid such as `phy:0nq0`, `ring:1` or so. */
+int sppwk_port_uid(char *port_uid, enum port_type p_type, int iface_no,
+			int queue_no)
 {
 	const char *p_type_str;
 
@@ -735,7 +782,15 @@ int sppwk_port_uid(char *port_uid, enum port_type p_type, int iface_no)
 		return SPPWK_RET_NG;
 	}
 
-	sprintf(port_uid, "%s:%d", p_type_str, iface_no);
+	int max_queue_port = get_port_max_queues(p_type, iface_no);
+	if (unlikely(max_queue_port == SPPWK_RET_NG))
+		return SPPWK_RET_NG;
+
+	if (max_queue_port <= 1)
+		sprintf(port_uid, "%s:%d", p_type_str, iface_no);
+	else
+		sprintf(port_uid, "%s:%d nq %d",
+			p_type_str, iface_no, queue_no);
 
 	return SPPWK_RET_OK;
 }
diff --git a/src/shared/secondary/spp_worker_th/cmd_utils.h b/src/shared/secondary/spp_worker_th/cmd_utils.h
index df1b0dc..eda55c6 100644
--- a/src/shared/secondary/spp_worker_th/cmd_utils.h
+++ b/src/shared/secondary/spp_worker_th/cmd_utils.h
@@ -174,7 +174,7 @@ void stop_process(int signal);
 
 /* Return sppwk_port_info of given type and num of interface. */
 struct sppwk_port_info *
-get_sppwk_port(enum port_type iface_type, int iface_no);
+get_sppwk_port(enum port_type iface_type, int iface_no, int queue_no);
 
 /* Output log message for core information */
 void log_core_info(const struct core_mng_info *core_info);
@@ -221,6 +221,7 @@ int sppwk_is_lcore_updated(unsigned int lcore_id);
  *
  * @param iface_type Interface type to be validated.
  * @param iface_no Interface number to be validated.
+ * @param queue_no Queue number of interface to be validated.
  * @param rxtx Value of spp_port_rxtx to be validated.
  * @retval 0~127      If match component ID
  * @retval SPPWK_RET_NG If failed.
@@ -228,6 +229,7 @@ int sppwk_is_lcore_updated(unsigned int lcore_id);
 int sppwk_check_used_port(
 		enum port_type iface_type,
 		int iface_no,
+		int queue_no,
 		enum sppwk_port_dir dir);
 
 /**
@@ -269,6 +271,15 @@ int sppwk_get_lcore_id(const char *comp_name);
 int get_idx_port_info(struct sppwk_port_info *p_info, int nof_ports,
 		struct sppwk_port_info *p_info_ary[]);
 
+/**
+ * Returns max queue number of the target port.
+ *
+ * @param[in] iface_type Interface type such as PHY or so.
+ * @param[in] iface_no Interface number.
+ * @return Max queue number if succeeded, or SPPWK_RET_NG if failed.
+ */
+int get_port_max_queues(const enum port_type iface_type, int iface_no);
+
 /**
  *  search matched port_info from array and delete it.
  *
@@ -294,15 +305,17 @@ int update_port_info(void);
 void update_lcore_info(void);
 
 /**
- * Return port uid such as `phy:0`, `ring:1` or so.
+ * Return port uid such as `phy:0nq0`, `ring:1` or so.
  *
  * @param[in,out] port_uid String of port type to be converted.
  * @param[in] iface_type Interface type such as PHY or so.
  * @param[in] iface_no Interface number.
+ * @param[in] queue_no Queue number of interface.
  * @return SPPWK_RET_OK If succeeded, or SPPWK_RET_NG if failed.
  */
 int
-sppwk_port_uid(char *port_uid, enum port_type iface_type, int iface_no);
+sppwk_port_uid(char *port_uid, enum port_type iface_type, int iface_no,
+		int queue_no);
 
 /**
  * Change string of MAC address to int64.
diff --git a/src/shared/secondary/spp_worker_th/data_types.h b/src/shared/secondary/spp_worker_th/data_types.h
index 57bf182..a05eb75 100644
--- a/src/shared/secondary/spp_worker_th/data_types.h
+++ b/src/shared/secondary/spp_worker_th/data_types.h
@@ -15,9 +15,17 @@
 #define SPPWK_VHOST_STR "vhost"
 #define SPPWK_RING_STR "ring"
 
+/*
+ * Delimiter for phy port supporting multi queue.
+ * Example of phy port supporting multi-queue: `phy: 0nq1`.
+ */
+#define DELIM_PHY_MQ "nq"
+
 /* TODO(yasufum) confirm usage of this value and why it is 4. */
 #define PORT_CAPABL_MAX 4  /* Max num of port abilities. */
 
+#define DEFAULT_QUEUE_ID 0  /* Queue ID is counted up from 0. */
+
 /* Status of a component on lcore. */
 enum sppwk_lcore_status {
 	SPPWK_LCORE_UNUSED,
@@ -89,6 +97,7 @@ struct sppwk_cls_attrs {
 struct sppwk_port_idx {
 	enum port_type iface_type;  /**< phy, vhost or ring. */
 	int iface_no;
+	int queue_no;
 };
 
 /* Define detailed port params in addition to `sppwk_port_idx`. */
@@ -96,6 +105,7 @@ struct sppwk_port_info {
 	enum port_type iface_type;  /**< phy, vhost or ring */
 	int iface_no;
 	int ethdev_port_id;  /**< Consistent ID of ethdev */
+	int queue_no;
 	struct sppwk_cls_attrs cls_attrs;
 	struct sppwk_port_attrs port_attrs[PORT_CAPABL_MAX];
 };
@@ -108,8 +118,10 @@ struct sppwk_comp_info {
 	int comp_id;  /**< Component ID */
 	int nof_rx;  /**< The number of rx ports */
 	int nof_tx;  /**< The number of tx ports */
-	struct sppwk_port_info *rx_ports[RTE_MAX_ETHPORTS]; /**< rx ports */
-	struct sppwk_port_info *tx_ports[RTE_MAX_ETHPORTS]; /**< tx ports */
+	/**< rx ports */
+	struct sppwk_port_info *rx_ports[RTE_MAX_QUEUES_PER_PORT];
+	/**< tx ports */
+	struct sppwk_port_info *tx_ports[RTE_MAX_QUEUES_PER_PORT];
 };
 
 /* Manage number of interfaces  and port information as global variable. */
@@ -120,7 +132,7 @@ struct sppwk_comp_info {
  * or not.
  */
 struct iface_info {
-	struct sppwk_port_info phy[RTE_MAX_ETHPORTS];
+	struct sppwk_port_info phy[RTE_MAX_ETHPORTS][RTE_MAX_QUEUES_PER_PORT];
 	struct sppwk_port_info vhost[RTE_MAX_ETHPORTS];
 	struct sppwk_port_info ring[RTE_MAX_ETHPORTS];
 };
diff --git a/src/shared/secondary/spp_worker_th/port_capability.c b/src/shared/secondary/spp_worker_th/port_capability.c
index 5435db0..5bd759f 100644
--- a/src/shared/secondary/spp_worker_th/port_capability.c
+++ b/src/shared/secondary/spp_worker_th/port_capability.c
@@ -378,11 +378,11 @@ vlan_operation(uint16_t port_id, struct rte_mbuf **pkts, const uint16_t nb_pkts,
 /* Wrapper function for rte_eth_rx_burst() with VLAN feature. */
 uint16_t
 sppwk_eth_vlan_rx_burst(uint16_t port_id,
-		uint16_t queue_id __attribute__ ((unused)),
+		uint16_t queue_id,
 		struct rte_mbuf **rx_pkts, const uint16_t nb_pkts)
 {
 	uint16_t nb_rx;
-	nb_rx = rte_eth_rx_burst(port_id, 0, rx_pkts, nb_pkts);
+	nb_rx = rte_eth_rx_burst(port_id, queue_id, rx_pkts, nb_pkts);
 	if (unlikely(nb_rx == 0))
 		return SPPWK_RET_OK;
 
@@ -393,7 +393,7 @@ sppwk_eth_vlan_rx_burst(uint16_t port_id,
 /* Wrapper function for rte_eth_tx_burst() with VLAN feature. */
 uint16_t
 sppwk_eth_vlan_tx_burst(uint16_t port_id,
-		uint16_t queue_id __attribute__ ((unused)),
+		uint16_t queue_id,
 		struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 {
 	uint16_t nb_tx;
@@ -404,7 +404,7 @@ sppwk_eth_vlan_tx_burst(uint16_t port_id,
 	if (unlikely(nb_tx == 0))
 		return SPPWK_RET_OK;
 
-	return rte_eth_tx_burst(port_id, 0, tx_pkts, nb_tx);
+	return rte_eth_tx_burst(port_id, queue_id, tx_pkts, nb_tx);
 }
 
 #ifdef SPP_RINGLATENCYSTATS_ENABLE
diff --git a/src/shared/secondary/spp_worker_th/port_capability.h b/src/shared/secondary/spp_worker_th/port_capability.h
index 284ba30..17d3a16 100644
--- a/src/shared/secondary/spp_worker_th/port_capability.h
+++ b/src/shared/secondary/spp_worker_th/port_capability.h
@@ -62,7 +62,7 @@ void sppwk_update_port_dir(const struct sppwk_comp_info *comp);
  * Wrapper function for rte_eth_rx_burst() with VLAN feature.
  *
  * @param[in] port_id Etherdev ID.
- * @param[in] queue_id RX queue ID, but fixed value 0 in SPP.
+ * @param[in] queue_id RX queue ID.
  * @param[in] rx_pkts Pointers to mbuf should be enough to store nb_pkts.
  * @param nb_pkts Maximum number of RX packets.
  * @return Number of RX packets as number of pointers to mbuf.
@@ -74,7 +74,7 @@ uint16_t sppwk_eth_vlan_rx_burst(uint16_t port_id, uint16_t queue_id,
  * Wrapper function for rte_eth_tx_burst() with VLAN feature.
  *
  * @param port_id Etherdev ID.
- * @param[in] queue_id TX queue ID, but fixed value 0 in SPP.
+ * @param[in] queue_id TX queue ID.
  * @param[in] tx_pkts Pointers to mbuf should be enough to store nb_pkts.
  * @param nb_pkts Maximum number of TX packets.
  * @return Number of TX packets as number of pointers to mbuf.
diff --git a/src/shared/secondary/spp_worker_th/vf_deps.h b/src/shared/secondary/spp_worker_th/vf_deps.h
index f2ea62a..dadcf7a 100644
--- a/src/shared/secondary/spp_worker_th/vf_deps.h
+++ b/src/shared/secondary/spp_worker_th/vf_deps.h
@@ -24,7 +24,12 @@
 struct mac_classifier {
 	struct rte_hash *cls_tbl;  /* Hash table for MAC classification. */
 	int nof_cls_ports;  /* Num of ports classified validly. */
-	int cls_ports[RTE_MAX_ETHPORTS];  /* Ports for classification. */
+	/**
+	 * Ports for classification.
+	 * For multi-queue case, size of ports should be
+	 * RTE_MAX_QUEUES_PER_PORT. RTE_MAX_ETHPORTS is not enough.
+	 */
+	int cls_ports[RTE_MAX_QUEUES_PER_PORT];
 	int default_cls_idx;  /* Default index for classification. */
 };
 
@@ -33,6 +38,7 @@ struct mac_classifier {
 struct cls_port_info {
 	enum port_type iface_type;
 	int iface_no;   /* Index of ports handled by classifier. */
+	int queue_no;   /* Index of queue per port handled by classifier. */
 	int iface_no_global;  /* ID for interface generated by spp_vf */
 	uint16_t ethdev_port_id;  /* Ethdev port ID. */
 	uint16_t nof_pkts;  /* Number of packets in pkts[]. */
@@ -47,7 +53,12 @@ struct cls_comp_info {
 	int nof_tx_ports;  /* Number of TX ports info entries. */
 	/* Classifier has one RX port and several TX ports. */
 	struct cls_port_info rx_port_i;  /* RX port info classified. */
-	struct cls_port_info tx_ports_i[RTE_MAX_ETHPORTS];  /* TX info. */
+	/**
+	 * TX info.
+	 * For multi-queue case, size of ports should be
+	 * RTE_MAX_QUEUES_PER_PORT. RTE_MAX_ETHPORTS is not enough.
+	 */
+	struct cls_port_info tx_ports_i[RTE_MAX_QUEUES_PER_PORT];
 };
 
 int add_core(const char *name, char **output,
diff --git a/src/shared/secondary/utils.c b/src/shared/secondary/utils.c
index c80a478..7cca4a4 100644
--- a/src/shared/secondary/utils.c
+++ b/src/shared/secondary/utils.c
@@ -5,6 +5,7 @@
 #include <stdint.h>
 #include "shared/common.h"
 #include "shared/secondary/utils.h"
+#include "shared/secondary/spp_worker_th/data_types.h"
 
 #define RTE_LOGTYPE_SHARED RTE_LOGTYPE_USER1
 
@@ -66,15 +67,18 @@ parse_client_id(int *cli_id, const char *cli_id_str)
 }
 
 /**
- * Retieve port type and ID from resource UID. For example, resource UID
- * 'ring:0' is  parsed to retrieve port tyep 'ring' and ID '0'.
+ * Retieve port type, ID and queue id from resource UID. For example,
+ * resource UID 'ring:0' is  parsed to retrieve port type 'ring' and ID '0'.
  */
 int
-parse_resource_uid(char *str, char **port_type, int *port_id)
+parse_resource_uid(char *str, char **port_type, int *port_id,
+		uint16_t *queue_id)
 {
 	char *token;
 	char delim[] = ":";
 	char *endp;
+	uint16_t default_queue = DEFAULT_QUEUE_ID;
+	char *queue_id_str = NULL;
 
 	RTE_LOG(DEBUG, SHARED, "Parsing resource UID: '%s'\n", str);
 	if (strstr(str, delim) == NULL) {
@@ -86,13 +90,27 @@ parse_resource_uid(char *str, char **port_type, int *port_id)
 	*port_type = strtok(str, delim);
 
 	token = strtok(NULL, delim);
+	queue_id_str = strstr(token, DELIM_PHY_MQ);
 	*port_id = strtol(token, &endp, 10);
-
-	if (*endp) {
+	if ((queue_id_str == NULL && *endp) ||
+		(queue_id_str != NULL && endp != queue_id_str)) {
 		RTE_LOG(ERR, SHARED, "Bad integer value: %s\n", str);
 		return -1;
 	}
 
+	if (queue_id_str == NULL) {
+		*queue_id = default_queue;
+	} else {
+		*queue_id = strtol(&queue_id_str[strlen(DELIM_PHY_MQ)],
+			&endp, 10);
+
+		if (*endp) {
+			RTE_LOG(ERR, SHARED,
+				"queue_id is bad integer value: %s\n", str);
+			return -1;
+		}
+	}
+
 	return 0;
 }
 
diff --git a/src/shared/secondary/utils.h b/src/shared/secondary/utils.h
index eba221f..4ae0a8e 100644
--- a/src/shared/secondary/utils.h
+++ b/src/shared/secondary/utils.h
@@ -5,7 +5,8 @@
 #ifndef _SHARED_SECONDARY_UTILS_H_
 #define _SHARED_SECONDARY_UTILS_H_
 
-int parse_resource_uid(char *str, char **port_type, int *port_id);
+int parse_resource_uid(char *str, char **port_type, int *port_id,
+	uint16_t *queue_id);
 
 int spp_atoi(const char *str, int *val);
 
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 02/17] spp_vf: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 01/17] shared: add support of multi-queue x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 03/17] spp_mirror: " x-fn-spp-ml
                   ` (32 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To achieve hardware offload in secondary process, multi-queue should be
supported.
Multi-queue support is divided into several changes like following:
- add queue number parameter in port command
- parse newly added parameter
- add queue number parameter in status response

For spp_vf, code for forwarder, classifier and merger should be changed.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/vf/classifier.c    | 40 +++++++++++++++++++++++++++-------------
 src/vf/forwarder.c     | 12 +++++++-----
 src/vf/vf_cmd_runner.c | 32 ++++++++++++++++++++------------
 3 files changed, 54 insertions(+), 30 deletions(-)

diff --git a/src/vf/classifier.c b/src/vf/classifier.c
index 8a4c8fc..2d95289 100644
--- a/src/vf/classifier.c
+++ b/src/vf/classifier.c
@@ -199,7 +199,8 @@ log_classification(long clsd_idx, struct rte_mbuf *pkt,
 	else
 		sppwk_port_uid(iface_str,
 				clsd_data[clsd_idx].iface_type,
-				clsd_data[clsd_idx].iface_no_global);
+				clsd_data[clsd_idx].iface_no_global,
+				clsd_data[clsd_idx].queue_no);
 
 	RTE_LOG_DP(DEBUG, VF_CLS,
 			"[%s]Classification(%s:%d). d_addr=%s, "
@@ -227,7 +228,8 @@ log_entry(long clsd_idx, uint16_t vid, const char *mac_addr_str,
 	else
 		sppwk_port_uid(iface_str,
 				clsd_data[clsd_idx].iface_type,
-				clsd_data[clsd_idx].iface_no_global);
+				clsd_data[clsd_idx].iface_no_global,
+				clsd_data[clsd_idx].queue_no);
 
 	RTE_LOG_DP(DEBUG, VF_CLS,
 			"[%s]Entry(%s:%d). vid=%hu, mac_addr=%s, iface=%s\n",
@@ -320,6 +322,7 @@ init_component_info(struct cls_comp_info *cmp_info,
 	if (wk_comp_info->nof_rx == 0) {
 		cls_rx_port_info->iface_type = UNDEF;
 		cls_rx_port_info->iface_no = 0;
+		cls_rx_port_info->queue_no = DEFAULT_QUEUE_ID;
 		cls_rx_port_info->iface_no_global = 0;
 		cls_rx_port_info->ethdev_port_id = 0;
 		cls_rx_port_info->nof_pkts = 0;
@@ -327,6 +330,8 @@ init_component_info(struct cls_comp_info *cmp_info,
 		cls_rx_port_info->iface_type =
 			wk_comp_info->rx_ports[0]->iface_type;
 		cls_rx_port_info->iface_no = 0;
+		cls_rx_port_info->queue_no =
+			wk_comp_info->rx_ports[0]->queue_no;
 		cls_rx_port_info->iface_no_global =
 			wk_comp_info->rx_ports[0]->iface_no;
 		cls_rx_port_info->ethdev_port_id =
@@ -344,6 +349,7 @@ init_component_info(struct cls_comp_info *cmp_info,
 		/* store ports information */
 		cls_tx_ports_info[i].iface_type = tx_port->iface_type;
 		cls_tx_ports_info[i].iface_no = i;
+		cls_tx_ports_info[i].queue_no = tx_port->queue_no;
 		cls_tx_ports_info[i].iface_no_global = tx_port->iface_no;
 		cls_tx_ports_info[i].ethdev_port_id = tx_port->ethdev_port_id;
 		cls_tx_ports_info[i].nof_pkts = 0;
@@ -374,10 +380,10 @@ init_component_info(struct cls_comp_info *cmp_info,
 			mac_cls->default_cls_idx = i;
 			RTE_LOG(INFO, VF_CLS,
 					"default classified. vid=%hu, "
-					"iface_type=%d, iface_no=%d, "
+					"iface_type=%d, iface_no=%d, queue_no=%d, "
 					"ethdev_port_id=%d\n",
 					vid, tx_port->iface_type,
-					tx_port->iface_no,
+					tx_port->iface_no, tx_port->queue_no,
 					tx_port->ethdev_port_id);
 			continue;
 		}
@@ -401,9 +407,10 @@ init_component_info(struct cls_comp_info *cmp_info,
 		RTE_LOG(INFO, VF_CLS,
 				"Add entry to classifier table. "
 				"vid=%hu, mac_addr=%s, iface_type=%d, "
-				"iface_no=%d, ethdev_port_id=%d\n",
+				"iface_no=%d, queue_no=%d, ethdev_port_id=%d\n",
 				vid, mac_addr_str, tx_port->iface_type,
-				tx_port->iface_no, tx_port->ethdev_port_id);
+				tx_port->iface_no, tx_port->queue_no,
+				tx_port->ethdev_port_id);
 	}
 
 	return SPPWK_RET_OK;
@@ -422,8 +429,9 @@ transmit_packets(struct cls_port_info *clsd_data)
 			clsd_data->iface_type, clsd_data->iface_no,
 			0, clsd_data->pkts, clsd_data->nof_pkts);
 #else
-	n_tx = sppwk_eth_vlan_tx_burst(clsd_data->ethdev_port_id, 0,
-			clsd_data->pkts, clsd_data->nof_pkts);
+	n_tx = sppwk_eth_vlan_tx_burst(clsd_data->ethdev_port_id,
+			clsd_data->queue_no, clsd_data->pkts,
+			clsd_data->nof_pkts);
 #endif
 
 	/* free cannot transmit packets */
@@ -467,11 +475,12 @@ push_packet(struct rte_mbuf *pkt, struct cls_port_info *clsd_data)
 	if (unlikely(clsd_data->nof_pkts == MAX_PKT_BURST)) {
 		RTE_LOG(DEBUG, VF_CLS,
 				"transmit packets (buffer is filled). "
-				"iface_type=%d, iface_no={%d,%d}, "
+				"iface_type=%d, iface_no={%d,%d}, queue_no=%d, "
 				"tx_port=%hu, nof_pkts=%hu\n",
 				clsd_data->iface_type,
 				clsd_data->iface_no_global,
 				clsd_data->iface_no,
+				clsd_data->queue_no,
 				clsd_data->ethdev_port_id,
 				clsd_data->nof_pkts);
 		transmit_packets(clsd_data);
@@ -754,8 +763,8 @@ classify_packets(int comp_id)
 			clsd_data_rx->iface_type, clsd_data_rx->iface_no,
 			0, rx_pkts, MAX_PKT_BURST);
 #else
-	n_rx = sppwk_eth_vlan_rx_burst(clsd_data_rx->ethdev_port_id, 0,
-			rx_pkts, MAX_PKT_BURST);
+	n_rx = sppwk_eth_vlan_rx_burst(clsd_data_rx->ethdev_port_id,
+			clsd_data_rx->queue_no, rx_pkts, MAX_PKT_BURST);
 #endif
 	if (unlikely(n_rx == 0))
 		return SPPWK_RET_OK;
@@ -776,8 +785,8 @@ get_classifier_status(unsigned int lcore_id, int id,
 	struct cls_mng_info *mng_info;
 	struct cls_comp_info *cmp_info;
 	struct cls_port_info *port_info;
-	struct sppwk_port_idx rx_ports[RTE_MAX_ETHPORTS];
-	struct sppwk_port_idx tx_ports[RTE_MAX_ETHPORTS];
+	struct sppwk_port_idx rx_ports[RTE_MAX_QUEUES_PER_PORT];
+	struct sppwk_port_idx tx_ports[RTE_MAX_QUEUES_PER_PORT];
 
 	mng_info = cls_mng_info_list + id;
 	if (!is_used_mng_info(mng_info)) {
@@ -796,6 +805,7 @@ get_classifier_status(unsigned int lcore_id, int id,
 		nof_rx = 1;
 		rx_ports[0].iface_type = cmp_info->rx_port_i.iface_type;
 		rx_ports[0].iface_no = cmp_info->rx_port_i.iface_no_global;
+		rx_ports[0].queue_no = cmp_info->rx_port_i.queue_no;
 	}
 
 	memset(tx_ports, 0x00, sizeof(tx_ports));
@@ -803,6 +813,7 @@ get_classifier_status(unsigned int lcore_id, int id,
 	for (i = 0; i < nof_tx; i++) {
 		tx_ports[i].iface_type = port_info[i].iface_type;
 		tx_ports[i].iface_no = port_info[i].iface_no_global;
+		tx_ports[i].queue_no = port_info[i].queue_no;
 	}
 
 	/* Set the information with the function specified by the command. */
@@ -840,6 +851,8 @@ add_mac_entry(struct classifier_table_params *params,
 				mac_cls->default_cls_idx)->iface_type;
 		port.iface_no = (port_info +
 				mac_cls->default_cls_idx)->iface_no_global;
+		port.queue_no = (port_info +
+				mac_cls->default_cls_idx)->queue_no;
 
 		LOG_ENT((long)mac_cls->default_cls_idx, vid,
 				SPPWK_TERM_DEFAULT, cmp_info, port_info);
@@ -863,6 +876,7 @@ add_mac_entry(struct classifier_table_params *params,
 
 		port.iface_type = (port_info + (long)data)->iface_type;
 		port.iface_no = (port_info + (long)data)->iface_no_global;
+		port.queue_no = (port_info + (long)data)->queue_no;
 
 		LOG_ENT((long)data, vid, mac_addr_str, cmp_info, port_info);
 
diff --git a/src/vf/forwarder.c b/src/vf/forwarder.c
index 04bf5c3..66e1682 100644
--- a/src/vf/forwarder.c
+++ b/src/vf/forwarder.c
@@ -62,8 +62,8 @@ get_forwarder_status(unsigned int lcore_id, int id,
 	const char *component_type = NULL;
 	struct forward_info *fwd_info = &g_forward_info[id];
 	struct forward_path *fwd_path = &fwd_info->path[fwd_info->ref_index];
-	struct sppwk_port_idx rx_ports[RTE_MAX_ETHPORTS];
-	struct sppwk_port_idx tx_ports[RTE_MAX_ETHPORTS];
+	struct sppwk_port_idx rx_ports[RTE_MAX_QUEUES_PER_PORT];
+	struct sppwk_port_idx tx_ports[RTE_MAX_QUEUES_PER_PORT];
 
 	if (unlikely(fwd_path->wk_type == SPPWK_TYPE_NONE)) {
 		RTE_LOG(ERR, FORWARD,
@@ -82,12 +82,14 @@ get_forwarder_status(unsigned int lcore_id, int id,
 	for (cnt = 0; cnt < fwd_path->nof_rx; cnt++) {
 		rx_ports[cnt].iface_type = fwd_path->ports[cnt].rx.iface_type;
 		rx_ports[cnt].iface_no = fwd_path->ports[cnt].rx.iface_no;
+		rx_ports[cnt].queue_no = fwd_path->ports[cnt].rx.queue_no;
 	}
 
 	memset(tx_ports, 0x00, sizeof(tx_ports));
 	for (cnt = 0; cnt < fwd_path->nof_tx; cnt++) {
 		tx_ports[cnt].iface_type = fwd_path->ports[cnt].tx.iface_type;
 		tx_ports[cnt].iface_no = fwd_path->ports[cnt].tx.iface_no;
+		tx_ports[cnt].queue_no = fwd_path->ports[cnt].tx.queue_no;
 	}
 
 	/* Set the information with the function specified by the command. */
@@ -217,8 +219,8 @@ forward_packets(int id)
 				rx->iface_type, rx->iface_no, 0,
 				bufs, MAX_PKT_BURST);
 #else
-		nb_rx = sppwk_eth_vlan_rx_burst(rx->ethdev_port_id, 0,
-				bufs, MAX_PKT_BURST);
+		nb_rx = sppwk_eth_vlan_rx_burst(rx->ethdev_port_id,
+				rx->queue_no, bufs, MAX_PKT_BURST);
 #endif
 		if (unlikely(nb_rx == 0))
 			continue;
@@ -231,7 +233,7 @@ forward_packets(int id)
 					tx->iface_no, 0, bufs, nb_rx);
 #else
 			nb_tx = sppwk_eth_vlan_tx_burst(tx->ethdev_port_id,
-					0, bufs, nb_rx);
+					tx->queue_no, bufs, nb_rx);
 #endif
 
 		/* Discard remained packets to release mbuf */
diff --git a/src/vf/vf_cmd_runner.c b/src/vf/vf_cmd_runner.c
index 305c4a2..c5e8278 100644
--- a/src/vf/vf_cmd_runner.c
+++ b/src/vf/vf_cmd_runner.c
@@ -43,8 +43,9 @@ update_cls_table(enum sppwk_action wk_action,
 	struct sppwk_port_info *port_info;
 
 	RTE_LOG(DEBUG, VF_CMD_RUNNER, "Called __func__ with "
-			"type `mac`, mac_addr `%s`, and port `%d:%d`.\n",
-			mac_str, port->iface_type, port->iface_no);
+			"type `mac`, mac_addr `%s`, and port `%d:%d nq %d`.\n",
+			mac_str, port->iface_type, port->iface_no,
+			port->queue_no);
 
 	mac_int64 = sppwk_convert_mac_str_to_int64(mac_str);
 	if (unlikely(mac_int64 == -1)) {
@@ -54,15 +55,18 @@ update_cls_table(enum sppwk_action wk_action,
 	}
 	mac_uint64 = (uint64_t)mac_int64;
 
-	port_info = get_sppwk_port(port->iface_type, port->iface_no);
+	port_info = get_sppwk_port(port->iface_type, port->iface_no,
+			port->queue_no);
 	if (unlikely(port_info == NULL)) {
-		RTE_LOG(ERR, VF_CMD_RUNNER, "Failed to get port %d:%d.\n",
-				port->iface_type, port->iface_no);
+		RTE_LOG(ERR, VF_CMD_RUNNER, "Failed to get port %d:%d nq %d.\n",
+				port->iface_type, port->iface_no,
+				port->queue_no);
 		return SPPWK_RET_NG;
 	}
 	if (unlikely(port_info->iface_type == UNDEF)) {
-		RTE_LOG(ERR, VF_CMD_RUNNER, "Port %d:%d doesn't exist.\n",
-				port->iface_type, port->iface_no);
+		RTE_LOG(ERR, VF_CMD_RUNNER, "Port %d:%d nq %d doesn't exist.\n",
+				port->iface_type, port->iface_no,
+				port->queue_no);
 		return SPPWK_RET_NG;
 	}
 
@@ -88,15 +92,17 @@ update_cls_table(enum sppwk_action wk_action,
 		if (unlikely(port_info->cls_attrs.vlantag.vid !=
 				ETH_VLAN_ID_MAX)) {
 			/* TODO(yasufum) why two vids are required in msg ? */
-			RTE_LOG(ERR, VF_CMD_RUNNER, "Used port %d:%d, vid %d != %d.\n",
+			RTE_LOG(ERR, VF_CMD_RUNNER, "Used port %d:%d nq %d, vid %d != %d.\n",
 					port->iface_type, port->iface_no,
+					port->queue_no,
 					port_info->cls_attrs.vlantag.vid, vid);
 			return SPPWK_RET_NG;
 		}
 		if (unlikely(port_info->cls_attrs.mac_addr != 0)) {
 			/* TODO(yasufum) why two macs are required in msg ? */
-			RTE_LOG(ERR, VF_CMD_RUNNER, "Used port %d:%d, mac %s != %s.\n",
+			RTE_LOG(ERR, VF_CMD_RUNNER, "Used port %d:%d nq %d, mac %s != %s.\n",
 					port->iface_type, port->iface_no,
+					port->queue_no,
 					port_info->cls_attrs.mac_addr_str,
 					mac_str);
 			return SPPWK_RET_NG;
@@ -277,7 +283,8 @@ update_port(enum sppwk_action wk_action,
 	sppwk_get_mng_data(NULL, &comp_info_base, NULL, NULL,
 			&change_component, NULL);
 	comp_info = (comp_info_base + comp_lcore_id);
-	port_info = get_sppwk_port(port->iface_type, port->iface_no);
+	port_info = get_sppwk_port(port->iface_type, port->iface_no,
+			port->queue_no);
 	if (dir == SPPWK_PORT_DIR_RX) {
 		nof_ports = &comp_info->nof_rx;
 		ports = comp_info->rx_ports;
@@ -317,7 +324,7 @@ update_port(enum sppwk_action wk_action,
 			return SPPWK_RET_OK;
 		}
 
-		if (*nof_ports >= RTE_MAX_ETHPORTS) {
+		if (*nof_ports >= RTE_MAX_QUEUES_PER_PORT) {
 			RTE_LOG(ERR, VF_CMD_RUNNER, "Cannot assign port over the "
 				"maximum number.\n");
 			return SPPWK_RET_NG;
@@ -565,7 +572,8 @@ append_classifier_element_value(
 		return ret;
 	}
 
-	sppwk_port_uid(port_str, port->iface_type, port->iface_no);
+	sppwk_port_uid(port_str, port->iface_type, port->iface_no,
+			port->queue_no);
 
 	ret = append_json_str_value(&tmp_buff, "type",
 			CLS_TYPE_A_LIST[cls_type]);
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 03/17] spp_mirror: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 01/17] shared: add support of multi-queue x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 02/17] spp_vf: " x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 04/17] spp_pcap: " x-fn-spp-ml
                   ` (31 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

By changing common code under shared directory to achieve multi-queue,
existing code of spp_mirror should be changed also for successful
compile.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/mirror/mir_cmd_runner.c |  3 ++-
 src/mirror/spp_mirror.c     | 12 ++++++++----
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/src/mirror/mir_cmd_runner.c b/src/mirror/mir_cmd_runner.c
index fb1695f..5153b65 100644
--- a/src/mirror/mir_cmd_runner.c
+++ b/src/mirror/mir_cmd_runner.c
@@ -156,7 +156,8 @@ update_port(enum sppwk_action wk_action,
 	sppwk_get_mng_data(NULL, &comp_info_base, NULL, NULL,
 			&change_component, NULL);
 	comp_info = (comp_info_base + comp_lcore_id);
-	port_info = get_sppwk_port(port->iface_type, port->iface_no);
+	port_info = get_sppwk_port(port->iface_type, port->iface_no,
+			port->queue_no);
 	if (dir == SPPWK_PORT_DIR_RX) {
 		nof_ports = &comp_info->nof_rx;
 		ports = comp_info->rx_ports;
diff --git a/src/mirror/spp_mirror.c b/src/mirror/spp_mirror.c
index e5852aa..2bfbf66 100644
--- a/src/mirror/spp_mirror.c
+++ b/src/mirror/spp_mirror.c
@@ -340,7 +340,8 @@ mirror_proc(int id)
 	nb_rx = sppwk_eth_ring_stats_rx_burst(rx->ethdev_port_id,
 			rx->iface_type, rx->iface_no, 0, bufs, MAX_PKT_BURST);
 #else
-	nb_rx = rte_eth_rx_burst(rx->ethdev_port_id, 0, bufs, MAX_PKT_BURST);
+	nb_rx = rte_eth_rx_burst(rx->ethdev_port_id, rx->queue_no, bufs,
+			MAX_PKT_BURST);
 #endif
 
 	if (unlikely(nb_rx == 0))
@@ -403,8 +404,8 @@ mirror_proc(int id)
 					tx->ethdev_port_id, tx->iface_type,
 					tx->iface_no, 0, copybufs, cnt);
 #else
-			nb_tx2 = rte_eth_tx_burst(tx->ethdev_port_id, 0,
-					copybufs, cnt);
+			nb_tx2 = rte_eth_tx_burst(tx->ethdev_port_id,
+					tx->queue_no, copybufs, cnt);
 #endif
 	}
 
@@ -415,7 +416,8 @@ mirror_proc(int id)
 		nb_tx1 = sppwk_eth_ring_stats_tx_burst(tx->ethdev_port_id,
 				tx->iface_type, tx->iface_no, 0, bufs, nb_rx);
 #else
-		nb_tx1 = rte_eth_tx_burst(tx->ethdev_port_id, 0, bufs, nb_rx);
+		nb_tx1 = rte_eth_tx_burst(tx->ethdev_port_id, tx->queue_no,
+				bufs, nb_rx);
 #endif
 	nb_tx = nb_tx1;
 
@@ -685,12 +687,14 @@ get_mirror_status(unsigned int lcore_id, int id,
 	for (cnt = 0; cnt < path->nof_rx; cnt++) {
 		rx_ports[cnt].iface_type = path->ports[cnt].rx.iface_type;
 		rx_ports[cnt].iface_no   = path->ports[cnt].rx.iface_no;
+		rx_ports[cnt].queue_no   = path->ports[cnt].rx.queue_no;
 	}
 
 	memset(tx_ports, 0x00, sizeof(tx_ports));
 	for (cnt = 0; cnt < path->nof_tx; cnt++) {
 		tx_ports[cnt].iface_type = path->ports[cnt].tx.iface_type;
 		tx_ports[cnt].iface_no   = path->ports[cnt].tx.iface_no;
+		tx_ports[cnt].queue_no   = path->ports[cnt].tx.queue_no;
 	}
 
 	/* Set the information with the function specified by the command. */
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 04/17] spp_pcap: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (2 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 03/17] spp_mirror: " x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 05/17] spp_primary: " x-fn-spp-ml
                   ` (30 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

By changing common code under shared directory to achieve multi-queue,
existing code of spp_pcap should be changed also for successful compile.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/pcap/cmd_runner.c |   3 +-
 src/pcap/cmd_utils.c  |  73 +++++++++++++++++++++++++-----
 src/pcap/cmd_utils.h  |  19 +++++++-
 src/pcap/spp_pcap.c   | 102 +++++++++++++++++++++++++++++++++++++-----
 4 files changed, 171 insertions(+), 26 deletions(-)

diff --git a/src/pcap/cmd_runner.c b/src/pcap/cmd_runner.c
index 85c02cd..ea6ec4d 100644
--- a/src/pcap/cmd_runner.c
+++ b/src/pcap/cmd_runner.c
@@ -398,7 +398,8 @@ append_port_entry(char **output, const struct sppwk_port_idx *port,
 		return SPPWK_RET_NG;
 	}
 
-	sppwk_port_uid(port_str, port->iface_type, port->iface_no);
+	sppwk_port_uid(port_str, port->iface_type, port->iface_no,
+			port->queue_no);
 	ret = append_json_str_value("port", &tmp_buff, port_str);
 	if (unlikely(ret < SPPWK_RET_OK))
 		return SPPWK_RET_NG;
diff --git a/src/pcap/cmd_utils.c b/src/pcap/cmd_utils.c
index 5aa131b..383f783 100644
--- a/src/pcap/cmd_utils.c
+++ b/src/pcap/cmd_utils.c
@@ -10,6 +10,7 @@
 
 #include "cmd_utils.h"
 #include "shared/secondary/return_codes.h"
+#include "shared/common.h"
 
 #define RTE_LOGTYPE_PCAP_UTILS RTE_LOGTYPE_USER2
 
@@ -149,13 +150,13 @@ stop_process(int signal)
  * It returns NULL value if given type is invalid.
  */
 struct sppwk_port_info *
-get_iface_info(enum port_type iface_type, int iface_no)
+get_iface_info(enum port_type iface_type, int iface_no, int queue_no)
 {
 	struct iface_info *iface_info = g_mng_data_addr.p_iface_info;
 
 	switch (iface_type) {
 	case PHY:
-		return &iface_info->phy[iface_no];
+		return &iface_info->phy[iface_no][queue_no];
 	case RING:
 		return &iface_info->ring[iface_no];
 	default:
@@ -172,16 +173,27 @@ static void
 init_iface_info(void)
 {
 	int port_cnt;  /* increment ether ports */
+	int queue_cnt;  /* increment queues per port */
 	struct iface_info *p_iface_info = g_mng_data_addr.p_iface_info;
 	memset(p_iface_info, 0x00, sizeof(struct iface_info));
 	for (port_cnt = 0; port_cnt < RTE_MAX_ETHPORTS; port_cnt++) {
-		p_iface_info->phy[port_cnt].iface_type = UNDEF;
-		p_iface_info->phy[port_cnt].iface_no   = port_cnt;
-		p_iface_info->phy[port_cnt].ethdev_port_id  = -1;
-		p_iface_info->phy[port_cnt].cls_attrs.vlantag.vid =
-				ETH_VLAN_ID_MAX;
+		for (queue_cnt = 0; queue_cnt < RTE_MAX_QUEUES_PER_PORT;
+				queue_cnt++) {
+			p_iface_info->phy[port_cnt][queue_cnt].iface_type =
+					UNDEF;
+			p_iface_info->phy[port_cnt][queue_cnt].iface_no =
+					port_cnt;
+			p_iface_info->phy[port_cnt][queue_cnt].queue_no =
+					queue_cnt;
+			p_iface_info->phy[port_cnt][queue_cnt]
+					.ethdev_port_id = -1;
+			p_iface_info->phy[port_cnt][queue_cnt]
+					.cls_attrs.vlantag.vid =
+					ETH_VLAN_ID_MAX;
+		}
 		p_iface_info->ring[port_cnt].iface_type = UNDEF;
 		p_iface_info->ring[port_cnt].iface_no   = port_cnt;
+		p_iface_info->ring[port_cnt].queue_no   = DEFAULT_QUEUE_ID;
 		p_iface_info->ring[port_cnt].ethdev_port_id  = -1;
 		p_iface_info->ring[port_cnt].cls_attrs.vlantag.vid =
 				ETH_VLAN_ID_MAX;
@@ -206,7 +218,7 @@ static int
 init_host_port_info(void)
 {
 	int port_type, port_id;
-	int i, ret;
+	int i, ret, queue_id;
 	int nof_phys = 0;
 	char dev_name[RTE_DEV_NAME_MAX_LEN] = { 0 };
 	struct iface_info *p_iface_info = g_mng_data_addr.p_iface_info;
@@ -227,8 +239,13 @@ init_host_port_info(void)
 
 		switch (port_type) {
 		case PHY:
-			p_iface_info->phy[port_id].iface_type = port_type;
-			p_iface_info->phy[port_id].ethdev_port_id = port_id;
+			for (queue_id = 0; queue_id < RTE_MAX_QUEUES_PER_PORT;
+					queue_id++) {
+				p_iface_info->phy[port_id][queue_id]
+						.iface_type = port_type;
+				p_iface_info->phy[port_id][queue_id]
+						.ethdev_port_id = port_id;
+			}
 			break;
 		case RING:
 			p_iface_info->ring[port_id].iface_type = port_type;
@@ -263,9 +280,11 @@ init_mng_data(void)
  * Generate a formatted string of combination from interface type and
  * number and assign to given 'port'
  */
-int sppwk_port_uid(char *port, enum port_type iface_type, int iface_no)
+int sppwk_port_uid(char *port, enum port_type iface_type, int iface_no,
+		int queue_no)
 {
 	const char *iface_type_str;
+	int max_queue;
 
 	switch (iface_type) {
 	case PHY:
@@ -278,7 +297,15 @@ int sppwk_port_uid(char *port, enum port_type iface_type, int iface_no)
 		return SPPWK_RET_NG;
 	}
 
-	sprintf(port, "%s:%d", iface_type_str, iface_no);
+	max_queue = get_port_max_queues(iface_type, iface_no);
+	if (unlikely(max_queue == SPPWK_RET_NG))
+		return SPPWK_RET_NG;
+
+	if (max_queue <= 1)
+		sprintf(port, "%s:%d", iface_type_str, iface_no);
+	else
+		sprintf(port, "%s:%d nq %d",
+			iface_type_str, iface_no, queue_no);
 
 	return SPPWK_RET_OK;
 }
@@ -319,3 +346,25 @@ void spp_get_mng_data_addr(struct iface_info **iface_p,
 		*capture_status_p = g_mng_data_addr.p_capture_status;
 
 }
+
+/* Returns a larger number of queues of RX or TX port as the maximum number */
+int
+get_port_max_queues(const enum port_type iface_type, int iface_no)
+{
+	if (unlikely(iface_no > RTE_MAX_ETHPORTS) || unlikely(iface_no < 0))
+		return SPPWK_RET_NG;
+
+	if (iface_type != PHY)
+		return 1;
+
+	const struct port_info *ports = NULL;
+	const struct rte_memzone *mz;
+	mz = rte_memzone_lookup(MZ_PORT_INFO);
+	ports = mz->addr;
+	int max_q_rx = ports->queue_info[iface_no].rxq;
+	int max_q_tx = ports->queue_info[iface_no].txq;
+	if (max_q_rx > max_q_tx)
+		return max_q_rx;
+	else
+		return max_q_tx;
+}
diff --git a/src/pcap/cmd_utils.h b/src/pcap/cmd_utils.h
index 95b8a8a..94e7087 100644
--- a/src/pcap/cmd_utils.h
+++ b/src/pcap/cmd_utils.h
@@ -87,12 +87,14 @@ void stop_process(int signal);
  *  Interface type to be validated.
  * @param iface_no
  *  Interface number to be validated.
+ * @param queue_no
+ *  Interface queue number to be validated.
  *
  * @retval !NULL  sppwk_port_info.
  * @retval NULL   failed.
  */
 struct sppwk_port_info *
-get_iface_info(enum port_type iface_type, int iface_no);
+get_iface_info(enum port_type iface_type, int iface_no, int queue_no);
 
 /**
  * Setup management info for spp_vf
@@ -105,15 +107,19 @@ struct core_info *get_core_info(unsigned int lcore_id);
 /**
  * Port type to string
  *
+ * TODO(smurakami) Change spp_pcap to use queue_no for supporting multi-queue
+ *
  * @param port String of port type to be converted.
  * @param iface_type Interface type.
  * @param iface_no Interface number.
+ * @param queue_no Queue number of interface.
  * @retval SPPWK_RET_OK If succeeded.
  * @retval SPPWK_RET_NG If failed.
  */
 /* TODO(yasufum) consider to merge to shared. */
 int
-sppwk_port_uid(char *port, enum port_type iface_type, int iface_no);
+sppwk_port_uid(char *port, enum port_type iface_type, int iface_no,
+		int queue_no);
 
 /**
  * Set mange data address
@@ -143,4 +149,13 @@ void spp_get_mng_data_addr(struct iface_info **iface_p,
 			   int **capture_request_p,
 			   int **capture_status_p);
 
+/**
+ * Returns max queue number of the target port.
+ *
+ * @param[in] iface_type Interface type such as PHY or so.
+ * @param[in] iface_no Interface number.
+ * @return Max queue number if succeeded, or SPPWK_RET_NG if failed.
+ */
+int get_port_max_queues(const enum port_type iface_type, int iface_no);
+
 #endif
diff --git a/src/pcap/spp_pcap.c b/src/pcap/spp_pcap.c
index ab4d796..1f20e1e 100644
--- a/src/pcap/spp_pcap.c
+++ b/src/pcap/spp_pcap.c
@@ -185,7 +185,7 @@ usage(const char *progname)
 		" [--fsize MAX_FILE_SIZE]\n"
 		" --client-id CLIENT_ID: My client ID\n"
 		" -s IPADDR:PORT: IP addr and sec port for spp-ctl\n"
-		" -c: Captured port (e.g. 'phy:0' or 'ring:1')\n"
+		" -c: Captured port (e.g. 'phy:0', 'phy:0 nq 1' or 'ring:1')\n"
 		" --out-dir: Output dir (Default is /tmp)\n"
 		" --fsize: Maximum captured file size (Default is 1GiB)\n"
 		, progname);
@@ -209,12 +209,15 @@ parse_fsize(const char *fsize_str, uint64_t *fsize)
 
 /* Parse `-c` option for captured port and get the port type and ID */
 static int
-parse_captured_port(const char *port_str, enum port_type *iface_type,
-			int *iface_no)
+parse_captured_port(const char *port_str, int option_index,
+			const int argcopt, char *argvopt[],
+			enum port_type *iface_type, int *iface_no,
+			int *queue_no)
 {
 	enum port_type type = UNDEF;
 	const char *no_str = NULL;
 	char *endptr = NULL;
+	int q_no = 0; /* Init value is default queue_no */
 
 	/* Find out which type of interface from resource UID */
 	if (strncmp(port_str, SPPWK_PHY_STR ":",
@@ -243,11 +246,45 @@ parse_captured_port(const char *port_str, enum port_type *iface_type,
 		return SPPWK_RET_NG;
 	}
 
+	/* Convert to numeric if queue_no */
+	if (type == PHY && option_index + 1 <= argcopt &&
+		!strcmp(argvopt[option_index], DELIM_PHY_MQ)) {
+
+		no_str = argvopt[option_index + 1];
+
+		q_no = strtol(no_str, &endptr, 0);
+		if (unlikely(no_str == endptr) || unlikely(*endptr != '\0')) {
+			/* No Queue number */
+			RTE_LOG(ERR, SPP_PCAP,
+				"No queue number. (port = %s)\n", port_str);
+			return SPPWK_RET_NG;
+		}
+
+		/* Check max num of queue_no */
+		if (q_no >= get_port_max_queues(type, ret_no)) {
+			RTE_LOG(ERR, SPP_PCAP,
+				"queue_no exceeds the definition of primary.\n"
+				);
+			return SPPWK_RET_NG;
+		}
+
+	} else {
+		if (get_port_max_queues(type, ret_no) > 1) {
+			RTE_LOG(ERR, SPP_PCAP,
+				"Queue_no is required for the specified "
+				"physical port because of multi-queue.\n"
+				);
+			return SPPWK_RET_NG;
+		}
+	}
+
 	*iface_type = type;
 	*iface_no = ret_no;
+	*queue_no = q_no;
 
-	RTE_LOG(DEBUG, SPP_PCAP, "Port = %s => Type = %d No = %d\n",
-					port_str, *iface_type, *iface_no);
+	RTE_LOG(DEBUG, SPP_PCAP,
+		"Port = %s => Type = %d No = %d\n Queue = %d",
+		port_str, *iface_type, *iface_no, *queue_no);
 	return SPPWK_RET_OK;
 }
 
@@ -322,13 +359,23 @@ parse_app_args(int argc, char *argv[])
 			break;
 		case 'c':  /* captured port */
 			strcpy(cap_port_str, optarg);
-			if (parse_captured_port(optarg,
+			if (parse_captured_port(optarg, optind,
+					argcopt, argvopt,
 					&g_pcap_option.port_cap.iface_type,
-					&g_pcap_option.port_cap.iface_no) !=
+					&g_pcap_option.port_cap.iface_no,
+					&g_pcap_option.port_cap.queue_no) !=
 					SPPWK_RET_OK) {
 				usage(progname);
 				return SPPWK_RET_NG;
 			}
+			if (get_port_max_queues(
+				g_pcap_option.port_cap.iface_type,
+				g_pcap_option.port_cap.iface_no) > 1) {
+				snprintf(cap_port_str, PORT_STR_SIZE,
+					"%s nq %d",
+					optarg,
+					g_pcap_option.port_cap.queue_no);
+			}
 			port_flg = 1;
 			break;
 		case 's':  /* server addr */
@@ -384,6 +431,7 @@ spp_pcap_get_core_status(
 		memset(rx_ports, 0x00, sizeof(rx_ports));
 		rx_ports[0].iface_type = g_pcap_option.port_cap.iface_type;
 		rx_ports[0].iface_no   = g_pcap_option.port_cap.iface_no;
+		rx_ports[0].queue_no   = g_pcap_option.port_cap.queue_no;
 		rx_num = 1;
 		strcpy(role_type, "receive");
 	}
@@ -472,7 +520,22 @@ static int file_compression_operation(struct pcap_mng_info *info,
 			iface_type_str = SPPWK_PHY_STR;
 		else
 			iface_type_str = SPPWK_RING_STR;
-		snprintf(info->compress_file_name,
+
+		if (get_port_max_queues(
+			g_pcap_option.port_cap.iface_type,
+			g_pcap_option.port_cap.iface_no) > 1)
+			/* If multi-queue, add queue_no */
+			snprintf(info->compress_file_name,
+					PCAP_FNAME_STRLEN - 1,
+					"spp_pcap.%s.%s%dnq%d.%u.%u.pcap.lz4",
+					g_pcap_option.compress_file_date,
+					iface_type_str,
+					g_pcap_option.port_cap.iface_no,
+					g_pcap_option.port_cap.queue_no,
+					info->thread_no,
+					info->file_no);
+		else
+			snprintf(info->compress_file_name,
 					PCAP_FNAME_STRLEN - 1,
 					"spp_pcap.%s.%s%d.%u.%u.pcap.lz4",
 					g_pcap_option.compress_file_date,
@@ -527,7 +590,22 @@ static int file_compression_operation(struct pcap_mng_info *info,
 			iface_type_str = SPPWK_PHY_STR;
 		else
 			iface_type_str = SPPWK_RING_STR;
-		snprintf(info->compress_file_name,
+
+		if (get_port_max_queues(
+			g_pcap_option.port_cap.iface_type,
+			g_pcap_option.port_cap.iface_no) > 1)
+			/* If multi-queue, add queue_no */
+			snprintf(info->compress_file_name,
+					PCAP_FNAME_STRLEN - 1,
+					"spp_pcap.%s.%s%dnq%d.%u.%u.pcap.lz4",
+					g_pcap_option.compress_file_date,
+					iface_type_str,
+					g_pcap_option.port_cap.iface_no,
+					g_pcap_option.port_cap.queue_no,
+					info->thread_no,
+					info->file_no);
+		else
+			snprintf(info->compress_file_name,
 					PCAP_FNAME_STRLEN - 1,
 					"spp_pcap.%s.%s%d.%u.%u.pcap.lz4",
 					g_pcap_option.compress_file_date,
@@ -773,7 +851,8 @@ static int pcap_proc_receive(int lcore_id)
 	nb_rx = sppwk_eth_ring_stats_rx_burst(rx->ethdev_port_id,
 			rx->iface_type, rx->iface_no, 0, bufs, MAX_PCAP_BURST);
 #else
-	nb_rx = rte_eth_rx_burst(rx->ethdev_port_id, 0, bufs, MAX_PCAP_BURST);
+	nb_rx = rte_eth_rx_burst(rx->ethdev_port_id, rx->queue_no, bufs,
+			MAX_PCAP_BURST);
 #endif
 	if (unlikely(nb_rx == 0))
 		return SPPWK_RET_OK;
@@ -992,7 +1071,8 @@ main(int argc, char *argv[])
 		struct sppwk_port_info *port_cap = &g_pcap_option.port_cap;
 		struct sppwk_port_info *port_info = get_iface_info(
 						port_cap->iface_type,
-						port_cap->iface_no);
+						port_cap->iface_no,
+						port_cap->queue_no);
 		if (port_info == NULL) {
 			RTE_LOG(ERR, SPP_PCAP, "caputre port undefined.\n");
 			break;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 05/17] spp_primary: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (3 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 04/17] spp_pcap: " x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 06/17] spp_primary: add support of rte_flow x-fn-spp-ml
                   ` (29 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

Multi-queue support is provided for both tx and rx of physical NIC.
To initialize physical NIC with multi-queue mode, startup parameters of
the primary process should be newly introduced.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/Makefile |  10 ++++
 src/primary/args.c   | 138 ++++++++++++++++++++++++++++++++++++++++++-
 src/primary/args.h   |   3 +
 src/primary/init.c   |   8 ++-
 src/primary/init.h   |   3 +-
 src/primary/main.c   |  40 ++++++++++---
 6 files changed, 189 insertions(+), 13 deletions(-)

diff --git a/src/primary/Makefile b/src/primary/Makefile
index ace6228..35ae788 100644
--- a/src/primary/Makefile
+++ b/src/primary/Makefile
@@ -20,12 +20,22 @@ APP = spp_primary
 
 # TODO: revise to not use functions in secondary's.
 SPP_SEC_DIR = ../shared/secondary
+SPP_FLOW_DIR = ./flow
+SPP_FLOW_SRC = flow.c attr.c common.c
+SPP_FLOW_PTN_DIR = $(SPP_FLOW_DIR)/pattern
+SPP_FLOW_PTN_SRC = eth.c vlan.c
+SPP_FLOW_ACT_DIR = $(SPP_FLOW_DIR)/action
+SPP_FLOW_ACT_SRC = jump.c queue.c of_push_vlan.c of_set_vlan_vid.c
+SPP_FLOW_ACT_SRC += of_set_vlan_pcp.c
 
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 SRCS-y += ../shared/common.c ../shared/basic_forwarder.c ../shared/port_manager.c
 SRCS-y += $(SPP_SEC_DIR)/add_port.c
 SRCS-y += $(SPP_SEC_DIR)/utils.c
+SRCS-y += $(addprefix $(SPP_FLOW_DIR)/,$(SPP_FLOW_SRC))
+SRCS-y += $(addprefix $(SPP_FLOW_PTN_DIR)/,$(SPP_FLOW_PTN_SRC))
+SRCS-y += $(addprefix $(SPP_FLOW_ACT_DIR)/,$(SPP_FLOW_ACT_SRC))
 
 INC := $(wildcard *.h)
 
diff --git a/src/primary/args.c b/src/primary/args.c
index 346bcbf..e9d1c58 100644
--- a/src/primary/args.c
+++ b/src/primary/args.c
@@ -29,10 +29,12 @@ int do_forwarding;
 enum {
 	CMD_LINE_OPT_MIN_NUM = 256,
 	CMD_OPT_DISP_STATS,
+	CMD_OPT_PORT_NUM, /* For `--port-num` */
 };
 
 struct option lgopts[] = {
 	{"disp-stats", no_argument, NULL, CMD_OPT_DISP_STATS},
+	{"port-num", required_argument, NULL, CMD_OPT_PORT_NUM},
 	{0}
 };
 
@@ -45,9 +47,14 @@ static void
 usage(void)
 {
 	RTE_LOG(INFO, PRIMARY,
-	    "%s [EAL options] -- -p PORTMASK -n NUM_CLIENTS [-s NUM_SOCKETS]\n"
+	    "%s [EAL options] -- -p PORTMASK -n NUM_CLIENTS [-s NUM_SOCKETS]"
+		" [--port-num NUM_PORT"
+		" rxq NUM_RX_QUEUE txq NUM_TX_QUEUE]...\n"
 	    " -p PORTMASK: hexadecimal bitmask of ports to use\n"
 	    " -n NUM_RINGS: number of ring ports used from secondaries\n"
+		" --port-num NUM_PORT: number of ports for multi-queue setting\n"
+		" rxq NUM_RX_QUEUE: number of receive queues\n"
+		" txq NUM_TX_QUEUE number of transmit queues\n"
 	    , progname);
 }
 
@@ -131,6 +138,120 @@ parse_nof_rings(uint16_t *num_clients, const char *clients)
 	return 0;
 }
 
+/* Extract the number of queues from startup option. */
+static int
+parse_nof_queues(struct port_queue *arg_queues, const char *str_port_num,
+		int option_index, uint16_t max_ports, int argc, char *argv[])
+{
+	char *end = NULL;
+	unsigned long temp;
+	uint16_t port_num, rxq, txq;
+
+
+	if (str_port_num == NULL || *str_port_num == '\0') {
+		RTE_LOG(ERR, PRIMARY,
+			"PORT_NUM is not specified(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* Parameter check of port_num */
+	temp = strtoul(str_port_num, &end, 10);
+	if (end == NULL || *end != '\0') {
+		RTE_LOG(ERR, PRIMARY,
+			"PORT_NUM is not a number(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+	port_num = (uint16_t)temp;
+
+	if (port_num > max_ports) {
+		RTE_LOG(ERR, PRIMARY,
+			"PORT_NUM exceeds the number of available ports"
+			"(%s:%d)\n",
+			__func__, __LINE__);
+		return 1;
+	}
+
+	/* Check if both 'rxq' and 'txq' are inclued in parameter string. */
+	if (option_index + 3 > argc) {
+		RTE_LOG(ERR, PRIMARY,
+			"rxq NUM_RX_QUEUE txq NUM_TX_QUEUE is not specified"
+			"(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	if (strcmp(argv[option_index], "rxq")) {
+		RTE_LOG(ERR, PRIMARY,
+			"rxq is not specified in the --port_num option"
+			"(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* Parameter check of rxq */
+	temp = strtoul(argv[option_index + 1], &end, 10);
+	if (end == NULL || *end != '\0' || temp == 0) {
+		RTE_LOG(ERR, PRIMARY,
+			"NUM_RX_QUEUE is not a number(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+	rxq = (uint16_t)temp;
+
+	if (strcmp(argv[option_index + 2], "txq")) {
+		RTE_LOG(ERR, PRIMARY,
+			"txq is not specified in the --port_num option"
+			"(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* Parameter check of txq */
+	temp = strtoul(argv[option_index + 3], &end, 10);
+	if (end == NULL || *end != '\0' || temp == 0) {
+		RTE_LOG(ERR, PRIMARY,
+			"NUM_TX_QUEUE is not a number(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+	txq = (uint16_t)temp;
+
+	arg_queues[port_num].rxq = rxq;
+	arg_queues[port_num].txq = txq;
+
+	return 0;
+}
+
+/**
+ * Set the number of queues for port_id.
+ * If not specified number of queue is set as 1.
+ */
+static int
+set_nof_queues(struct port_info *ports, struct port_queue *arg_queues)
+{
+	int index;
+	uint16_t port_id, rxq, txq;
+
+	for (index = 0; index < ports->num_ports; index++) {
+		port_id = ports->id[index];
+
+		if (arg_queues[port_id].rxq == 0 ||
+			arg_queues[port_id].txq == 0) {
+			rxq = 1;
+			txq = 1;
+		} else {
+			rxq = arg_queues[port_id].rxq;
+			txq = arg_queues[port_id].txq;
+		}
+
+		ports->queue_info[index].rxq = rxq;
+		ports->queue_info[index].txq = txq;
+	}
+	return 0;
+}
+
 /**
  * The application specific arguments follow the DPDK-specific
  * arguments which are stripped by the DPDK init. This function
@@ -143,6 +264,7 @@ parse_app_args(uint16_t max_ports, int argc, char *argv[])
 	int option_index, opt;
 	char **argvopt = argv;
 	int ret;
+	struct port_queue arg_queues[RTE_MAX_ETHPORTS] = { 0 };
 
 	progname = argv[0];
 
@@ -171,6 +293,14 @@ parse_app_args(uint16_t max_ports, int argc, char *argv[])
 				return -1;
 			}
 			break;
+		case CMD_OPT_PORT_NUM:
+			ret = parse_nof_queues(arg_queues, optarg, optind,
+					max_ports, argc, argv);
+			if (ret != 0) {
+				usage();
+				return -1;
+			}
+			break;
 		default:
 			RTE_LOG(ERR,
 				PRIMARY, "ERROR: Unknown option '%c'\n", opt);
@@ -184,5 +314,11 @@ parse_app_args(uint16_t max_ports, int argc, char *argv[])
 		return -1;
 	}
 
+	ret = set_nof_queues(ports, arg_queues);
+	if (ret != 0) {
+		usage();
+		return -1;
+	}
+
 	return 0;
 }
diff --git a/src/primary/args.h b/src/primary/args.h
index 644cd56..f73772c 100644
--- a/src/primary/args.h
+++ b/src/primary/args.h
@@ -13,6 +13,9 @@ extern uint16_t num_rings;
 extern char *server_ip;
 extern int server_port;
 
+/* Return value definition for getopt_long(). Only for long option. */
+#define SPP_LONGOPT_RETVAL_PORT_NUM 1 /* For `--port-num` */
+
 /**
  * Set flg from given argument.
  *
diff --git a/src/primary/init.c b/src/primary/init.c
index 9e47fcf..e9c12f1 100644
--- a/src/primary/init.c
+++ b/src/primary/init.c
@@ -160,7 +160,9 @@ init(int argc, char *argv[])
 	/* now initialise the ports we will use */
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		for (count = 0; count < ports->num_ports; count++) {
-			retval = init_port(ports->id[count], pktmbuf_pool);
+			retval = init_port(ports->id[count], pktmbuf_pool,
+				ports->queue_info[count].rxq,
+				ports->queue_info[count].txq);
 			if (retval != 0)
 				rte_exit(EXIT_FAILURE,
 					"Cannot initialise port %d\n", count);
@@ -254,7 +256,8 @@ check_all_ports_link_status(struct port_info *ports, uint16_t port_num,
  * - start the port and report its status to stdout
  */
 int
-init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool)
+init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool,
+	uint16_t rx_rings, uint16_t tx_rings)
 {
 	/* for port configuration all features are off by default */
 	const struct rte_eth_conf port_conf = {
@@ -262,7 +265,6 @@ init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool)
 			.mq_mode = ETH_MQ_RX_RSS,
 		},
 	};
-	const uint16_t rx_rings = 1, tx_rings = 1;
 	const uint16_t rx_ring_size = RTE_MP_RX_DESC_DEFAULT;
 	const uint16_t tx_ring_size = RTE_MP_TX_DESC_DEFAULT;
 	uint16_t q;
diff --git a/src/primary/init.h b/src/primary/init.h
index 6344377..5384154 100644
--- a/src/primary/init.h
+++ b/src/primary/init.h
@@ -50,6 +50,7 @@ int init(int argc, char *argv[]);
 void check_all_ports_link_status(struct port_info *ports, uint16_t port_num,
 		uint32_t port_mask);
 
-int init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool);
+int init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool,
+		uint16_t rx_rings, uint16_t tx_rings);
 
 #endif /* ifndef _PRIMARY_INIT_H_ */
diff --git a/src/primary/main.c b/src/primary/main.c
index d3828e8..ca81636 100644
--- a/src/primary/main.c
+++ b/src/primary/main.c
@@ -16,6 +16,7 @@
 #include "args.h"
 #include "init.h"
 #include "primary.h"
+#include "primary/flow/flow.h"
 
 #include "shared/port_manager.h"
 #include "shared/secondary/add_port.h"
@@ -23,10 +24,10 @@
 
 /*
  * Buffer sizes of status message of primary. Total number of size
- * must be equal to MSG_SIZE 2048 defined in `shared/common.h`.
+ * must be equal to MSG_SIZE 32768 defined in `shared/common.h`.
  */
 #define PRI_BUF_SIZE_LCORE 128
-#define PRI_BUF_SIZE_PHY 512
+#define PRI_BUF_SIZE_PHY 30720
 #define PRI_BUF_SIZE_RING (MSG_SIZE - PRI_BUF_SIZE_LCORE - PRI_BUF_SIZE_PHY)
 
 #define SPP_PATH_LEN 1024  /* seems enough for path of spp procs */
@@ -639,9 +640,10 @@ forwarder_status_json(char *str)
 static int
 phy_port_stats_json(char *str)
 {
-	int i;
-	int buf_size = 256;  /* size of temp buffer */
-	char phy_port[buf_size];
+	int i, ret;
+	int buf_size = PRI_BUF_SIZE_PHY - 512;  /* size of temp buffer */
+	char phy_port[PRI_BUF_SIZE_PHY];
+	char flow[buf_size];
 	char buf_phy_ports[PRI_BUF_SIZE_PHY];
 	memset(phy_port, '\0', sizeof(phy_port));
 	memset(buf_phy_ports, '\0', sizeof(buf_phy_ports));
@@ -651,16 +653,29 @@ phy_port_stats_json(char *str)
 		RTE_LOG(DEBUG, PRIMARY, "Size of buf_phy_ports str: %d\n",
 				(int)strlen(buf_phy_ports));
 
-		memset(phy_port, '\0', buf_size);
+		memset(phy_port, '\0', PRI_BUF_SIZE_PHY);
+		memset(flow, '\0', buf_size);
+
+		ret = append_flow_json(i, buf_size, flow);
+		if (ret != 0) {
+			sprintf(buf_phy_ports + strlen(buf_phy_ports) - 1,
+					"%s", "");
+			break;
+		}
 
 		sprintf(phy_port, "{\"id\":%u,\"eth\":\"%s\","
 				"\"rx\":%"PRIu64",\"tx\":%"PRIu64","
-				"\"tx_drop\":%"PRIu64"}",
+				"\"tx_drop\":%"PRIu64","
+				"\"nof_queues\":{\"rx\":%d,\"tx\":%d},"
+				"\"flow\":%s}",
 				ports->id[i],
 				get_printable_mac_addr(ports->id[i]),
 				ports->port_stats[i].rx,
 				ports->port_stats[i].tx,
-				ports->port_stats[i].tx_drop);
+				ports->port_stats[i].tx_drop,
+				ports->queue_info[i].rxq,
+				ports->queue_info[i].txq,
+				flow);
 
 		int cur_buf_size = (int)strlen(buf_phy_ports) +
 			(int)strlen(phy_port);
@@ -948,8 +963,10 @@ parse_command(char *str)
 	char patch_set[64] = { 0 };  /* "{\"src\":\"%s:%d\",\"dst\":...}" */
 	char *p_type;
 	int p_id;
+	char tmp_response[MSG_SIZE];
 
 	memset(sec_name, '\0', 16);
+	memset(tmp_response, '\0', MSG_SIZE);
 
 	/* tokenize the user commands from controller */
 	token_list[max_token] = strtok(str, " ");
@@ -1150,6 +1167,13 @@ parse_command(char *str)
 		sprintf(str, "{%s:%s,%s:%s}",
 				"\"result\"", "\"succeeded\"",
 				"\"command\"", "\"clear\"");
+
+	} else if (!strcmp(token_list[0], "flow")) {
+		RTE_LOG(DEBUG, PRIMARY, "'%s' command received.\n",
+				token_list[0]);
+		ret = parse_flow(token_list, tmp_response);
+		memset(str, '\0', MSG_SIZE);
+		strncpy(str, tmp_response, MSG_SIZE-1);
 	}
 
 	return ret;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 06/17] spp_primary: add support of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (4 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 05/17] spp_primary: " x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-19  2:24   ` Yasufumi Ogawa
  2020-02-18  6:37 ` [spp] [PATCH 07/17] spp_primary: add common function " x-fn-spp-ml
                   ` (28 subsequent siblings)
  34 siblings, 1 reply; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch provides main functions which
defines validate, create, delete and flush flow

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/flow.c | 1048 +++++++++++++++++++++++++++++++++++++++
 src/primary/flow/flow.h |   94 ++++
 2 files changed, 1142 insertions(+)
 create mode 100644 src/primary/flow/flow.c
 create mode 100644 src/primary/flow/flow.h

diff --git a/src/primary/flow/flow.c b/src/primary/flow/flow.c
new file mode 100644
index 0000000..52a3e59
--- /dev/null
+++ b/src/primary/flow/flow.c
@@ -0,0 +1,1048 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+#include <rte_common.h>
+#include <rte_ether.h>
+#include <rte_byteorder.h>
+
+#include "shared/common.h"
+#include "shared/secondary/utils.h"
+#include "shared/secondary/spp_worker_th/data_types.h"
+#include "primary/primary.h"
+#include "flow.h"
+#include "attr.h"
+#include "common.h"
+
+#include "primary/flow/pattern/eth.h"
+#include "primary/flow/pattern/vlan.h"
+
+#include "primary/flow/action/jump.h"
+#include "primary/flow/action/queue.h"
+#include "primary/flow/action/of_push_vlan.h"
+#include "primary/flow/action/of_set_vlan_vid.h"
+#include "primary/flow/action/of_set_vlan_pcp.h"
+
+
+/* Flow list for each port */
+static struct port_flow port_list[RTE_MAX_ETHPORTS] = { 0 };
+
+/* Define item operations */
+static struct flow_item_ops flow_item_ops_list[] = {
+	{
+		.str_type = "end",
+		.type = RTE_FLOW_ITEM_TYPE_END,
+		.parse = NULL,
+		.detail_list = NULL
+	},
+	{
+		.str_type = "eth",
+		.type = RTE_FLOW_ITEM_TYPE_ETH,
+		.size = sizeof(struct rte_flow_item_eth),
+		.parse = parse_item_common,
+		.detail_list = eth_ops_list,
+		.status = append_item_eth_json,
+	},
+	{
+		.str_type = "vlan",
+		.type = RTE_FLOW_ITEM_TYPE_VLAN,
+		.size = sizeof(struct rte_flow_item_vlan),
+		.parse = parse_item_common,
+		.detail_list = vlan_ops_list,
+		.status = append_item_vlan_json,
+	},
+};
+
+/* Define action operations */
+static struct flow_action_ops flow_action_ops_list[] = {
+	{
+		.str_type = "end",
+		.type = RTE_FLOW_ACTION_TYPE_END,
+		.size = 0,
+		.parse = NULL,
+		.detail_list = NULL,
+		.status = NULL,
+	},
+	{
+		.str_type = "jump",
+		.type = RTE_FLOW_ACTION_TYPE_JUMP,
+		.size = sizeof(struct rte_flow_action_jump),
+		.parse = parse_action_common,
+		.detail_list = jump_ops_list,
+		.status = append_action_jump_json,
+	},
+	{
+		.str_type = "queue",
+		.type = RTE_FLOW_ACTION_TYPE_QUEUE,
+		.size = sizeof(struct rte_flow_action_queue),
+		.parse = parse_action_common,
+		.detail_list = queue_ops_list,
+		.status = append_action_queue_json,
+	},
+	{
+		.str_type = "of_pop_vlan",
+		.type = RTE_FLOW_ACTION_TYPE_OF_POP_VLAN,
+		.size = 0,
+		.parse = NULL,
+		.detail_list = NULL,
+		.status = append_action_null_json,
+	},
+	{
+		.str_type = "of_push_vlan",
+		.type = RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN,
+		.size = sizeof(struct rte_flow_action_of_push_vlan),
+		.parse = parse_action_common,
+		.detail_list = of_push_vlan_ops_list,
+		.status = append_action_of_push_vlan_json,
+	},
+	{
+		.str_type = "of_set_vlan_vid",
+		.type = RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID,
+		.size = sizeof(struct rte_flow_action_of_set_vlan_vid),
+		.parse = parse_action_common,
+		.detail_list = of_set_vlan_vid_ops_list,
+		.status = append_action_of_set_vlan_vid_json,
+	},
+	{
+		.str_type = "of_set_vlan_pcp",
+		.type = RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP,
+		.size = sizeof(struct rte_flow_action_of_set_vlan_pcp),
+		.parse = parse_action_common,
+		.detail_list = of_set_vlan_pcp_ops_list,
+		.status = append_action_of_set_vlan_pcp_json,
+	},
+};
+
+/* Free memory of "flow_args". */
+static void
+free_flow_args(struct flow_args *input)
+{
+	int i;
+	struct rte_flow_item *pattern;
+	struct rte_flow_action *actions;
+	char **target;
+
+	if ((input->command != VALIDATE) &&
+		(input->command != CREATE))
+		return;
+
+	pattern = input->args.rule.pattern;
+	if (pattern != NULL) {
+		for (i = 0; pattern[i].type != RTE_FLOW_ITEM_TYPE_END; i++) {
+			target = (char **)((char *)(&pattern[i]) +
+				offsetof(struct rte_flow_item, spec));
+			if (*target != NULL)
+				free(*target);
+
+			target = (char **)((char *)(&pattern[i]) +
+				offsetof(struct rte_flow_item, last));
+			if (*target != NULL)
+				free(*target);
+
+			target = (char **)((char *)(&pattern[i]) +
+				offsetof(struct rte_flow_item, mask));
+			if (*target != NULL)
+				free(*target);
+		}
+
+		free(pattern);
+	}
+
+	actions = input->args.rule.actions;
+	if (actions != NULL) {
+		for (i = 0; actions[i].type != RTE_FLOW_ACTION_TYPE_END; i++) {
+			target = (char **)((char *)(&actions[i]) +
+				offsetof(struct rte_flow_action, conf));
+			if (*target != NULL)
+				free(*target);
+		}
+
+		free(actions);
+	}
+}
+
+/*
+ * Create response in JSON format.
+ * `rule_id` must be empty if flow create is failed.
+ */
+static void
+make_response(char *response, const char *result, const char *message,
+	char *rule_id)
+{
+	if (rule_id == NULL)
+		snprintf(response, MSG_SIZE,
+			"{\"result\": \"%s\", \"message\": \"%s\"}",
+			result, message);
+	else
+		snprintf(response, MSG_SIZE,
+			"{\"result\": \"%s\", \"message\": \"%s\", "
+			"\"rule_id\": \"%s\"}",
+			result, message, rule_id);
+}
+
+/* Create error response from rte_flow_error */
+static void
+make_error_response(char *response, const char *message,
+	struct rte_flow_error error, char *rule_id)
+{
+	/* Define description for each error type */
+	static const char *const errstr_list[] = {
+		[RTE_FLOW_ERROR_TYPE_NONE] = "No error",
+		[RTE_FLOW_ERROR_TYPE_UNSPECIFIED] = "Cause unspecified",
+		[RTE_FLOW_ERROR_TYPE_HANDLE] = "Flow rule (handle)",
+		[RTE_FLOW_ERROR_TYPE_ATTR_GROUP] = "Group field",
+		[RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY] = "Priority field",
+		[RTE_FLOW_ERROR_TYPE_ATTR_INGRESS] = "Ingress field",
+		[RTE_FLOW_ERROR_TYPE_ATTR_EGRESS] = "Egress field",
+		[RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER] = "Transfer field",
+		[RTE_FLOW_ERROR_TYPE_ATTR] = "Attributes structure",
+		[RTE_FLOW_ERROR_TYPE_ITEM_NUM] = "Pattern length",
+		[RTE_FLOW_ERROR_TYPE_ITEM_SPEC] = "Item specification",
+		[RTE_FLOW_ERROR_TYPE_ITEM_LAST] = "Item specification range",
+		[RTE_FLOW_ERROR_TYPE_ITEM_MASK] = "Item specification mask",
+		[RTE_FLOW_ERROR_TYPE_ITEM] = "Specific pattern item",
+		[RTE_FLOW_ERROR_TYPE_ACTION_NUM] = "Number of actions",
+		[RTE_FLOW_ERROR_TYPE_ACTION_CONF] = "Action configuration",
+		[RTE_FLOW_ERROR_TYPE_ACTION] = "Specific action",
+	};
+	int err = rte_errno;
+	char msg[512] = "";
+	char cause[32] = "";
+	const char *errstr;
+
+	if ((unsigned int)error.type >= RTE_DIM(errstr_list) ||
+	    !errstr_list[error.type])
+		errstr = "Unknown type";
+	else
+		errstr = errstr_list[error.type];
+
+
+	if (error.cause != NULL)
+		snprintf(cause, sizeof(cause), "cause: %p\\n", error.cause);
+
+	snprintf(msg, sizeof(msg),
+		"%s\\nerror type: %d (%s)\\n"
+		"%serror message: %s\\nrte_errno: %s",
+		message, error.type, errstr, cause,
+		error.message ? error.message : "(no stated reason)",
+		rte_strerror(err));
+	make_response(response, "error", msg, rule_id);
+}
+
+/* Add to array, redeclare memory. */
+static int
+append_object_list(void **list, void *add, size_t obj_size, int num)
+{
+	char *new_list;
+
+	new_list = malloc(obj_size * num);
+	if (new_list == NULL)
+		return -1;
+
+	/* Copy original list*/
+	if (*list != NULL) {
+		memcpy(new_list, *list, obj_size * (num - 1));
+		free(*list);
+	}
+
+	/* Add to list */
+	memcpy(new_list + (obj_size * (num - 1)), add, obj_size);
+
+	*list = (void *)new_list;
+	return 0;
+}
+
+static int
+parse_flow_actions(char *token_list[], int *index,
+	struct rte_flow_action **actions)
+{
+	int ret;
+	int action_count = 0;
+	uint16_t i;
+	char *token;
+	struct flow_action_ops *ops;
+	struct rte_flow_action action;
+
+	if (strcmp(token_list[*index], "actions")) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Invalid parameter is %s(%s:%d)\n",
+			token_list[*index], __func__, __LINE__);
+		return -1;
+	}
+
+	/* Next to word */
+	(*index)++;
+
+	while (token_list[*index] != NULL) {
+		token = token_list[*index];
+
+		for (i = 0; i < RTE_DIM(flow_action_ops_list); i++) {
+			ops = &flow_action_ops_list[i];
+			if (strcmp(token, ops->str_type))
+				continue;
+
+			memset(&action, 0, sizeof(struct rte_flow_action));
+			action.type = ops->type;
+			if (ops->parse != NULL) {
+				ret = ops->parse(token_list, index, &action,
+					ops);
+				if (ret < 0)
+					return -1;
+			} else {
+				(*index)++;
+			}
+			break;
+		}
+
+		/*
+		 * Error occurs if a action string that is not defined in
+		 * str_type of flow_action_ops_list is specified
+		 */
+		if (i == RTE_DIM(flow_action_ops_list)) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Invalid parameter "
+				"is %s action(%s:%d)\n",
+				token, __func__, __LINE__);
+			return -1;
+		}
+
+		/* Add to "actions" list */
+		action_count++;
+		ret = append_object_list((void **)actions, &action,
+			sizeof(struct rte_flow_action), action_count);
+
+		if (!strcmp(token, "end"))
+			break;
+
+		(*index)++;
+	}
+
+	return 0;
+}
+
+static int
+parse_flow_pattern(char *token_list[], int *index,
+	struct rte_flow_item **pattern)
+{
+	int ret;
+	int item_count = 0;
+	uint32_t i;
+	char *token;
+	struct flow_item_ops *ops;
+	struct rte_flow_item item;
+
+	while (token_list[*index] != NULL) {
+		token = token_list[*index];
+
+		for (i = 0; i < RTE_DIM(flow_item_ops_list); i++) {
+			ops = &flow_item_ops_list[i];
+			if (strcmp(token, ops->str_type))
+				continue;
+
+			memset(&item, 0, sizeof(struct rte_flow_item));
+			item.type = ops->type;
+			if (ops->parse != NULL) {
+				ret = ops->parse(token_list, index, &item,
+					ops);
+				if (ret < 0)
+					return -1;
+			}
+			break;
+		}
+
+		/*
+		 * Error occurs if a pattern string that is not defined in
+		 * str_type of flow_item_ops_list is specified
+		 */
+		if (i == RTE_DIM(flow_item_ops_list)) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Invalid parameter "
+				"is %s pattern(%s:%d)\n",
+				token, __func__, __LINE__);
+			return -1;
+		}
+
+		/* Add to "pattern" list */
+		item_count++;
+		ret = append_object_list((void **)pattern, &item,
+			sizeof(struct rte_flow_item), item_count);
+
+		if (!strcmp(token, "end"))
+			break;
+
+		(*index)++;
+	}
+
+	return 0;
+}
+
+static int
+parse_flow_rule(char *token_list[], struct flow_args *input)
+{
+	int ret = 0;
+	int index;
+
+	ret = parse_phy_port_id(token_list[2], &input->port_id);
+	if (ret < 0)
+		return -1;
+
+	/* The next index of the port */
+	index = 3;
+
+	/* Attribute parse */
+	ret = parse_flow_attr(token_list, &index, &input->args.rule.attr);
+	if (ret < 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Failed to parse Attribute(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* The next index of the pattern */
+	index++;
+
+	/* Pattern parse */
+	ret = parse_flow_pattern(token_list, &index,
+		&input->args.rule.pattern);
+	if (ret < 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Failed to parse Pattern(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* The next index of the actions */
+	index++;
+
+	/* Actions parse */
+	ret = parse_flow_actions(token_list, &index,
+		&input->args.rule.actions);
+	if (ret < 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Failed to parse Actions(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	return 0;
+}
+
+static int
+parse_flow_destroy(char *token_list[], struct flow_args *input)
+{
+	int ret;
+	char *end;
+
+	ret = parse_phy_port_id(token_list[2], &input->port_id);
+	if (ret < 0)
+		return -1;
+
+	if (token_list[3] == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"rule_id is not specified(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	if (!strcmp(token_list[3], "ALL")) {
+		input->command = FLUSH;
+
+	} else {
+		input->command = DESTROY;
+		input->args.destroy.rule_id = strtoul(token_list[3],
+			&end, 10);
+	}
+
+	return 0;
+}
+
+/** Generate a flow_rule entry from attributes/pattern/actions. */
+static struct flow_rule *
+create_flow_rule(struct rte_flow_attr *attr,
+	struct rte_flow_item *pattern,
+	struct rte_flow_action *actions,
+	struct rte_flow_error *error)
+{
+	const struct rte_flow_conv_rule conv_rule = {
+		.attr_ro = attr,
+		.pattern_ro = pattern,
+		.actions_ro = actions,
+	};
+	struct flow_rule *rule;
+	int ret;
+
+	ret = rte_flow_conv(RTE_FLOW_CONV_OP_RULE, NULL, 0, &conv_rule,
+		error);
+	if (ret < 0)
+		return NULL;
+
+	rule = calloc(1, offsetof(struct flow_rule, rule) + ret);
+	if (!rule) {
+		rte_flow_error_set
+			(error, errno, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+			 "calloc() failed");
+		return NULL;
+	}
+
+	ret = rte_flow_conv(RTE_FLOW_CONV_OP_RULE, &rule->rule, ret, &conv_rule,
+			  error);
+	if (ret >= 0)
+		return rule;
+
+	free(rule);
+	return NULL;
+}
+
+/* Execute rte_flow_validate().*/
+static void
+exec_flow_validate(int port_id,
+	struct rte_flow_attr *attr,
+	struct rte_flow_item *pattern,
+	struct rte_flow_action *actions,
+	char *response)
+{
+	int ret;
+	struct rte_flow_error error;
+
+	memset(&error, 0, sizeof(error));
+
+	ret = rte_flow_validate(port_id, attr, pattern, actions, &error);
+	if (ret != 0)
+		make_error_response(response, "Flow validate error", error,
+			NULL);
+	else
+		make_response(response, "success", "Flow rule validated",
+			NULL);
+}
+
+/* Execute rte_flow_create(). Save flow rules globally */
+static void
+exec_flow_create(int port_id,
+	struct rte_flow_attr *attr,
+	struct rte_flow_item *pattern,
+	struct rte_flow_action *actions,
+	char *response)
+{
+	uint32_t rule_id;
+	char mes[32];
+	char rule_id_str[11] = {0};
+	struct rte_flow_error error;
+	struct rte_flow *flow;
+	struct flow_rule *rule;
+	struct port_flow *port;
+
+	memset(&error, 0, sizeof(error));
+
+	flow = rte_flow_create(port_id, attr, pattern, actions, &error);
+	if (flow == NULL) {
+		make_error_response(response, "Flow create error", error,
+			rule_id_str);
+		return;
+	}
+
+	port = &port_list[port_id];
+	if (port->flow_list != NULL) {
+		if (port->flow_list->rule_id >= UINT32_MAX) {
+			make_response(response, "error",
+				"Rule ID must be less than %"PRIu32,
+				rule_id_str);
+			rte_flow_destroy(port_id, flow, NULL);
+			return;
+		}
+		rule_id = port->flow_list->rule_id + 1;
+	} else {
+		rule_id = 0;
+	}
+
+	rule = create_flow_rule(attr, pattern, actions, &error);
+	if (rule == NULL) {
+		rte_flow_destroy(port_id, flow, NULL);
+		make_error_response(response, "Flow create error", error,
+			rule_id_str);
+		return;
+	}
+
+	/* Keep it globally as a list */
+	rule->rule_id = rule_id;
+	rule->flow_handle = flow;
+
+	if (port->flow_list == NULL)
+		rule->prev = NULL;
+	else
+		rule->prev = port->flow_list;
+
+	port->flow_list = rule;
+
+	sprintf(mes, "Flow rule #%d created", rule_id);
+	sprintf(rule_id_str, "%d", rule_id);
+	make_response(response, "success", mes, rule_id_str);
+	return;
+}
+
+/* Execute rte_flow_destroy(). Destroying a globally saved flow rule */
+static void
+exec_flow_destroy(int port_id, uint32_t rule_id, char *response)
+{
+	int ret;
+	int found_flg = 0;
+	char mes[64];
+	struct flow_rule *rule, **next_ptr;
+	struct rte_flow_error error;
+
+	memset(&error, 0, sizeof(error));
+
+	ret = is_portid_used(port_id);
+	if (ret != 0) {
+		sprintf(mes, "Invalid port %d", port_id);
+		make_response(response, "error", mes, NULL);
+		return;
+	}
+
+	next_ptr = &(port_list[port_id].flow_list);
+	rule = port_list[port_id].flow_list;
+
+	while (rule != NULL) {
+		if (rule->rule_id != rule_id) {
+			next_ptr = &(rule->prev);
+			rule = rule->prev;
+			continue;
+		}
+
+		ret = rte_flow_destroy(port_id, rule->flow_handle, &error);
+		if (ret != 0) {
+			make_error_response(response, "Flow destroy error",
+				error, NULL);
+			return;
+		}
+
+		/* Remove flow from global list */
+		*next_ptr = rule->prev;
+		free(rule);
+		found_flg = 1;
+
+		sprintf(mes, "Flow rule #%d destroyed", rule_id);
+		make_response(response, "success", mes, NULL);
+		break;
+	}
+
+	/* Rule_id not found */
+	if (found_flg == 0) {
+		sprintf(mes, "Flow rule #%d not found", rule_id);
+		make_response(response, "error", mes, NULL);
+	}
+}
+
+/* Delete all globally saved flow rules */
+static void
+exec_flow_flush(int port_id, char *response)
+{
+	int ret;
+	char mes[64];
+	struct flow_rule *rule;
+	struct rte_flow_error error;
+
+	memset(&error, 0, sizeof(error));
+
+	ret = is_portid_used(port_id);
+	if (ret != 0) {
+		sprintf(mes, "Invalid port %d", port_id);
+		make_response(response, "error", mes, NULL);
+		return;
+	}
+
+	ret = rte_flow_flush(port_id, &error);
+	if (ret != 0)
+		make_error_response(response, "Flow destroy error",
+			error, NULL);
+	else
+		make_response(response, "success", "Flow rule all destroyed",
+			NULL);
+
+	/*
+	 * Even if a failure occurs, flow handle is invalidated,
+	 * so delete flow_list.
+	 */
+
+	while (port_list[port_id].flow_list != NULL) {
+		rule = port_list[port_id].flow_list->prev;
+		free(port_list[port_id].flow_list);
+		port_list[port_id].flow_list = rule;
+	}
+}
+
+static void
+exec_flow(struct flow_args *input, char *response)
+{
+	switch(input->command) {
+		case VALIDATE:
+			exec_flow_validate(input->port_id,
+				&input->args.rule.attr,
+				input->args.rule.pattern,
+				input->args.rule.actions,
+				response);
+			break;
+		case CREATE:
+			exec_flow_create(input->port_id,
+				&input->args.rule.attr,
+				input->args.rule.pattern,
+				input->args.rule.actions,
+				response);
+			break;
+		case DESTROY:
+			exec_flow_destroy(input->port_id,
+				input->args.destroy.rule_id,
+				response);
+			break;
+		case FLUSH:
+			exec_flow_flush(input->port_id, response);
+			break;
+	}
+
+	/* Argument data is no longer needed and freed */
+	free_flow_args(input);
+
+	return;
+}
+
+int
+parse_flow(char *token_list[], char *response)
+{
+	int ret = 0;
+	struct flow_args input = { 0 };
+
+	if (token_list[1] == NULL) {
+		ret = -1;
+	} else if (!strcmp(token_list[1], "validate")) {
+		input.command = VALIDATE;
+		ret = parse_flow_rule(token_list, &input);
+
+	} else if (!strcmp(token_list[1], "create")) {
+		input.command = CREATE;
+		ret = parse_flow_rule(token_list, &input);
+
+	} else if (!strcmp(token_list[1], "destroy")) {
+		ret = parse_flow_destroy(token_list, &input);
+
+	} else {
+		ret = -1;
+	}
+
+	if (ret != 0) {
+		free_flow_args(&input);
+		make_response(response, "error",
+			"Flow command invalid argument", NULL);
+		return 0;
+	}
+
+	exec_flow(&input, response);
+
+	return 0;
+}
+
+static int
+append_flow_pattern_json(const struct rte_flow_item *pattern, int buf_size,
+	char *pattern_str)
+{
+	uint32_t i, j;
+	uint32_t nof_elems = 3;
+	int ret = 0;
+	char *tmp_str;
+	const char element_str[][5] = { "spec", "last", "mask" };
+	const struct rte_flow_item *ptn = pattern;
+	struct flow_item_ops *ops;
+	const void *tmp_ptr[nof_elems];
+
+	tmp_str = malloc(buf_size);
+	if (tmp_str == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	while (ptn->type != RTE_FLOW_ITEM_TYPE_END) {
+		memset(tmp_str, 0, buf_size);
+
+		tmp_ptr[0] = ptn->spec;
+		tmp_ptr[1] = ptn->last;
+		tmp_ptr[2] = ptn->mask;
+
+		for (i = 0; i < RTE_DIM(flow_item_ops_list); i++) {
+			ops = &flow_item_ops_list[i];
+			if (ptn->type != ops->type)
+				continue;
+
+			snprintf(tmp_str, buf_size,
+				"{\"type\":\"%s\",",
+				ops->str_type);
+
+			for (j = 0; j < nof_elems; j++) {
+				snprintf(tmp_str + strlen(tmp_str), buf_size,
+					"\"%s\":",
+					element_str[j]);
+
+				if (tmp_ptr[j] != NULL)
+					ret = ops->status(tmp_ptr[j],
+						buf_size - (int)strlen(tmp_str),
+						tmp_str + strlen(tmp_str));
+				else
+					snprintf(tmp_str + strlen(tmp_str),
+						buf_size,
+						"null");
+
+				if (ret != 0)
+					break;
+
+				if (j < nof_elems - 1)
+					tmp_str[strlen(tmp_str)] = ',';
+			}
+
+			tmp_str[strlen(tmp_str)] = '}';
+
+			break;
+		}
+
+		if (ret != 0)
+			break;
+
+		if ((int)strlen(pattern_str) + (int)strlen(tmp_str)
+			> buf_size - 1) {
+			ret = -1;
+			break;
+		}
+		strncat(pattern_str, tmp_str, strlen(tmp_str));
+
+		/*
+		 * If there is the following pattern, add ',' to
+		 * pattern_str
+		 */
+		ptn++;
+		if (ptn->type != RTE_FLOW_ITEM_TYPE_END) {
+			if ((int)strlen(pattern_str) + 1 > buf_size - 1) {
+				ret = -1;
+				break;
+			}
+			pattern_str[strlen(pattern_str)] = ',';
+		}
+	}
+
+	if (tmp_str != NULL)
+		free(tmp_str);
+
+	return ret;
+}
+
+static int
+append_flow_action_json(const struct rte_flow_action *actions, int buf_size,
+	char *actions_str)
+{
+	uint32_t i;
+	int ret = 0;
+	char *tmp_str;
+	const struct rte_flow_action *act = actions;
+	struct flow_action_ops *ops;
+
+	tmp_str = malloc(buf_size);
+	if (tmp_str == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	while (act->type != RTE_FLOW_ACTION_TYPE_END) {
+		memset(tmp_str, 0, buf_size);
+
+		for (i = 0; i < RTE_DIM(flow_action_ops_list); i++) {
+			ops = &flow_action_ops_list[i];
+			if (act->type != ops->type)
+				continue;
+
+			snprintf(tmp_str, buf_size,
+				"{\"type\":\"%s\",\"conf\":",
+				ops->str_type);
+
+			ret = ops->status(act->conf,
+				buf_size - (int)strlen(tmp_str),
+				tmp_str + strlen(tmp_str));
+			tmp_str[strlen(tmp_str)] = '}';
+			break;
+		}
+
+		if (ret != 0)
+			break;
+
+		if ((int)strlen(actions_str) + (int)strlen(tmp_str)
+			> buf_size - 1) {
+			ret = -1;
+			break;
+		}
+		strncat(actions_str, tmp_str, strlen(tmp_str));
+
+		/*
+		 * If there is the following pattern, add ',' to
+		 * actions_str
+		 */
+		act++;
+		if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+			if ((int)strlen(actions_str) + 1 > buf_size - 1) {
+				ret = -1;
+				break;
+			}
+			actions_str[strlen(actions_str)] = ',';
+		}
+	}
+
+	if (tmp_str != NULL)
+		free(tmp_str);
+
+	return ret;
+}
+
+static int
+append_flow_rule_json(struct flow_rule *flow, int buf_size, char *flow_str)
+{
+	int ret = 0;
+	struct rte_flow_conv_rule rule;
+	char *tmp_str, *attr_str, *pattern_str, *actions_str;
+
+	while (1) {
+		tmp_str = malloc(buf_size);
+		attr_str = malloc(buf_size);
+		pattern_str = malloc(buf_size);
+		actions_str = malloc(buf_size);
+		if (tmp_str == NULL || attr_str == NULL
+			|| pattern_str == NULL || actions_str == NULL) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Memory allocation failure(%s:%d)\n",
+				__func__, __LINE__);
+			ret = -1;
+			break;
+		}
+		memset(tmp_str, 0, buf_size);
+		memset(attr_str, 0, buf_size);
+		memset(pattern_str, 0, buf_size);
+		memset(actions_str, 0, buf_size);
+
+		rule = flow->rule;
+
+		ret = append_flow_attr_json(rule.attr_ro, buf_size, attr_str);
+		if (ret != 0)
+			break;
+
+		ret = append_flow_pattern_json(rule.pattern_ro, buf_size,
+			pattern_str);
+		if (ret != 0)
+			break;
+
+		ret = append_flow_action_json(rule.actions_ro, buf_size,
+			actions_str);
+		if (ret != 0)
+			break;
+
+		snprintf(tmp_str, buf_size,
+			"{\"rule_id\":%d,"
+			"\"attr\":%s,"
+			"\"patterns\":[%s],"
+			"\"actions\":[%s]}",
+			flow->rule_id, attr_str, pattern_str, actions_str);
+
+		if ((int)strlen(tmp_str) > buf_size - 1) {
+			ret = -1;
+			break;
+		}
+
+		snprintf(flow_str, buf_size, "%s", tmp_str);
+		break;
+	}
+
+	if (tmp_str != NULL)
+		free(tmp_str);
+	if (attr_str != NULL)
+		free(attr_str);
+	if (pattern_str != NULL)
+		free(pattern_str);
+	if (actions_str != NULL)
+		free(actions_str);
+
+	return ret;
+}
+
+int
+append_flow_json(int port_id, int buf_size, char *output)
+{
+	int ret = 0;
+	int str_size = 0;
+	char *flow_str, *tmp_str;
+	struct flow_rule *flow;
+
+	flow_str = malloc(buf_size);
+	tmp_str = malloc(buf_size);
+	if (flow_str == NULL || tmp_str == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	flow = port_list[port_id].flow_list;
+
+	while (flow != NULL) {
+		memset(flow_str, 0, buf_size);
+
+		ret = append_flow_rule_json(flow, buf_size, flow_str);
+		if (ret != 0)
+			break;
+
+		if (str_size == 0) {
+			snprintf(output, buf_size, "%s", flow_str);
+			str_size += (int)strlen(flow_str);
+
+		} else {
+			str_size += ((int)strlen(flow_str) + 1);
+			if (str_size > buf_size - 1) {
+				ret = -1;
+				break;
+			}
+
+			/*
+			 * Since flow_list is in descending order,
+			 * concatenate the strings in front.
+			 */
+			memset(tmp_str, 0, buf_size);
+			strncpy(tmp_str, output, buf_size);
+			memset(output, 0, buf_size);
+
+			snprintf(output, buf_size, "%s,%s",
+				flow_str, tmp_str);
+		}
+
+		flow = flow->prev;
+	}
+
+	if (ret == 0) {
+		if ((int)strlen("[]") + (int)strlen(flow_str)
+			> buf_size - 1)
+			ret = -1;
+		else {
+			memset(tmp_str, 0, buf_size);
+			strncpy(tmp_str, output, buf_size);
+			memset(output, 0, buf_size);
+
+			snprintf(output, buf_size, "[%s]", tmp_str);
+		}
+	}
+
+	if (ret != 0)
+		RTE_LOG(ERR, SPP_FLOW,
+			"Cannot send all of flow stats(%s:%d)\n",
+			__func__, __LINE__);
+
+	if (flow_str != NULL)
+		free(flow_str);
+	if (tmp_str != NULL)
+		free(tmp_str);
+
+	return ret;
+}
diff --git a/src/primary/flow/flow.h b/src/primary/flow/flow.h
new file mode 100644
index 0000000..ecd4eb3
--- /dev/null
+++ b/src/primary/flow/flow.h
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_H_
+#define _PRIMARY_FLOW_H_
+
+#include <rte_log.h>
+
+#define RTE_LOGTYPE_SPP_FLOW RTE_LOGTYPE_USER1
+
+enum flow_command {
+	VALIDATE = 0,
+	CREATE,
+	DESTROY,
+	FLUSH
+};
+
+/* Parser result of flow command arguments */
+struct flow_args {
+	enum flow_command command;
+	int port_id;
+	union {
+		struct {
+			struct rte_flow_attr attr;
+			struct rte_flow_item *pattern;
+			struct rte_flow_action *actions;
+		} rule; /* validate or create arguments. */
+		struct {
+			uint32_t rule_id;
+		} destroy; /* destroy arguments. */
+	} args;
+};
+
+/* Descriptor for a single flow. */
+struct flow_rule {
+	/* Flow rule ID */
+	uint32_t rule_id;
+
+	/* Previous flow in list. */
+	struct flow_rule *prev;
+
+	/* Opaque flow object returned by PMD. */
+	struct rte_flow *flow_handle;
+
+	/* Saved flow rule description. */
+	struct rte_flow_conv_rule rule;
+};
+
+/* Flow rule list of the port */
+struct port_flow {
+	/* Associated flows */
+	struct flow_rule *flow_list;
+};
+
+/* Detail parse operation for a specific item or action */
+struct flow_detail_ops {
+	const char *token;
+	const size_t offset;
+	const size_t size;
+	int flg_value;
+	int (*parse_detail)(char *str, void *output);
+};
+
+/* Operation for each item type */
+struct flow_item_ops {
+	const char *str_type;
+	enum rte_flow_item_type type;
+	size_t size;
+	int (*parse)(char *token_list[], int *index,
+		struct rte_flow_item *pattern,
+		struct flow_item_ops *ops);
+	struct flow_detail_ops *detail_list;
+	int (*status)(const void *element,
+		int buf_size, char *pattern_str);
+};
+
+/* Operation for each action type */
+struct flow_action_ops {
+	const char *str_type;
+	enum rte_flow_action_type type;
+	size_t size;
+	int (*parse)(char *token_list[], int *index,
+		struct rte_flow_action *action,
+		struct flow_action_ops *ops);
+	struct flow_detail_ops *detail_list;
+	int (*status)(const void *conf,
+		int buf_size, char *action_str);
+};
+
+int parse_flow(char *token_list[], char *response);
+int append_flow_json(int port_id, int buf_size, char *output);
+
+#endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 07/17] spp_primary: add common function of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (5 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 06/17] spp_primary: add support of rte_flow x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 08/17] spp_primary: add attribute " x-fn-spp-ml
                   ` (27 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch provides parse functions those
are commonly used by primary application.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/common.c | 646 ++++++++++++++++++++++++++++++++++++++
 src/primary/flow/common.h |  53 ++++
 src/primary/main.c        |  48 +--
 3 files changed, 725 insertions(+), 22 deletions(-)
 create mode 100644 src/primary/flow/common.c
 create mode 100644 src/primary/flow/common.h

diff --git a/src/primary/flow/common.c b/src/primary/flow/common.c
new file mode 100644
index 0000000..da34619
--- /dev/null
+++ b/src/primary/flow/common.c
@@ -0,0 +1,646 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_ethdev.h>
+
+#include "shared/secondary/spp_worker_th/data_types.h"
+#include "shared/secondary/utils.h"
+#include "flow.h"
+#include "common.h"
+
+/*
+ * Check if port_id is used
+ * Return 0: Port_id used
+ * Return 1: Unused port_id
+ */
+int
+is_portid_used(int port_id)
+{
+	uint16_t pid;
+
+	RTE_ETH_FOREACH_DEV(pid) {
+		if (port_id == pid)
+			return 0;
+	}
+
+	return 1;
+}
+
+/*
+ * Retrieve port ID from source UID of phy port. Return error code if port
+ * type is other than phy.
+ */
+int
+parse_phy_port_id(char *res_uid, int *port_id)
+{
+	int ret;
+	char *port_type;
+	uint16_t queue_id;
+
+	if (res_uid == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"RES UID is NULL(%s:%d)\n", __func__, __LINE__);
+		return -1;
+	}
+
+	ret = parse_resource_uid(res_uid, &port_type, port_id, &queue_id);
+	if (ret < 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Failed to parse RES UID(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	if (strcmp(port_type, SPPWK_PHY_STR) != 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"It's not phy type(%s:%d)\n", __func__, __LINE__);
+		return -1;
+	}
+	return 0;
+}
+
+/*
+ * Convert string to rte_ether_addr.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_rte_ether_addr(char *mac_str, void *output)
+{
+	int i = 0;
+	uint8_t byte;
+	char *end, *token;
+	char tmp_mac_str[32] = { 0 };
+	struct rte_ether_addr *mac_addr = output;
+
+	strncpy(tmp_mac_str, mac_str, 32);
+	token = strtok(tmp_mac_str, ":");
+
+	while (token != NULL) {
+		if (i >= RTE_ETHER_ADDR_LEN)
+			return -1;
+
+		byte = (uint8_t)strtoul(token, &end, 16);
+		if (end == NULL || *end != '\0')
+			return -1;
+
+		mac_addr->addr_bytes[i] = byte;
+		i++;
+		token = strtok(NULL, ":");
+	}
+
+	return 0;
+}
+
+/*
+ * Convert string to tci.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_tci(char *tci_str, void *output)
+{
+	char *end;
+	rte_be16_t *tci = output;
+
+	*tci = (rte_be16_t)strtoul(tci_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	return 0;
+}
+
+/*
+ * Convert string to pcp.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_pcp(char *pcp_str, void *output)
+{
+	char *end;
+	uint8_t *pcp = output;
+
+	*pcp = (uint8_t)strtoul(pcp_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	/* 3bit check */
+	if (*pcp > 0x7)
+		return -1;
+
+	return 0;
+}
+
+/*
+ * Set PCP in TCI. TCI is a 16bits value and consists of 3bits PCP,
+ * 1bit DEI and the rest 12bits VID.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+set_pcp_in_tci(char *pcp_str, void *output)
+{
+	int ret;
+	uint8_t pcp = 0;
+	rte_be16_t *tci = output;
+
+	ret = str_to_pcp(pcp_str, &pcp);
+	if (ret != 0)
+		return -1;
+
+	/* Assign to the first 3 bits */
+	pcp = pcp << 1;
+	((char *)tci)[0] = ((char *)tci)[0] | pcp;
+
+	return 0;
+}
+
+/*
+ * Set DEI in TCI. TCI is a 16bits value and consists of 3bits PCP,
+ * 1bit DEI and the rest 12bits VID.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+set_dei_in_tci(char *dei_str, void *output)
+{
+	char *end;
+	uint8_t dei = 0;
+	rte_be16_t *tci = output;
+
+	dei = (uint8_t)strtoul(dei_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	/* 1bit check */
+	if (dei > 0x1)
+		return -1;
+
+	/* Assign to 4th bit */
+	((char *)tci)[0] = ((char *)tci)[0] | dei;
+
+	return 0;
+}
+
+/*
+ * Set VID in TCI. TCI is a 16bits value and consists of 3bits PCP,
+ * 1bit DEI and the rest 12bits VID.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+set_vid_in_tci(char *vid_str, void *output)
+{
+	char *end;
+	rte_be16_t vid = 0;
+	rte_be16_t *tci = output;
+
+	vid = (rte_be16_t)strtoul(vid_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	/* 12bit check */
+	if (vid > 0x0fff)
+		return -1;
+
+	/* Convert vid to big endian if system is little endian. */
+	int i = 1;
+	if (*(char *)&i) { /* check if little endian */
+		uint8_t b1 = ((char *)&vid)[0];
+		uint8_t b2 = ((char *)&vid)[1];
+		((char *)&vid)[0] = b2;
+		((char *)&vid)[1] = b1;
+	}
+
+	/* Assign to 5-16 bit */
+	*tci = *tci | vid;
+
+	return 0;
+}
+
+/*
+ * Convert string to rte_be16_t.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_rte_be16_t(char *target_str, void *output)
+{
+	char *end;
+	rte_be16_t *value = output;
+
+	*value = (rte_be16_t)strtoul(target_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+	/* Convert vid to big endian if system is little endian. */
+	int i = 1;
+	if (*(char *)&i) { /* check if little endian */
+		uint8_t b1 = ((char *)value)[0];
+		uint8_t b2 = ((char *)value)[1];
+		((char *)value)[0] = b2;
+		((char *)value)[1] = b1;
+	}
+
+	return 0;
+}
+
+/*
+ * Convert string to uint16_t.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_uint16_t(char *target_str, void *output)
+{
+	char *end;
+	uint16_t *value = output;
+
+	*value = (uint16_t)strtoul(target_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	return 0;
+}
+
+/*
+ * Convert string to uint32_t.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_uint32_t(char *target_str, void *output)
+{
+	char *end;
+	uint32_t *value = output;
+
+	*value = (uint32_t)strtoul(target_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	return 0;
+}
+
+int
+parse_rte_flow_item_field(char *token_list[], int *index,
+	struct flow_detail_ops *detail_list, size_t size,
+	void **spec, void **last, void **mask
+	)
+{
+	int ret = 0;
+	uint32_t prefix, i, j, bitmask;
+	char *end, *target;
+
+	if (!strcmp(token_list[*index], "is")) {
+		/* Match value perfectly (with full bit-mask). */
+		(*index)++;
+
+		if (*spec == NULL) {
+			ret = malloc_object(spec, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		ret = detail_list->parse_detail(token_list[*index],
+			(char *)(*spec) + detail_list->offset);
+		if (ret != 0) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"parse_detail error (%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+		/* Set full bit-mask */
+		if (*mask == NULL) {
+			ret = malloc_object(mask, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		memset((char *)(*mask) + detail_list->offset, 0xff,
+			detail_list->size);
+
+	} else if (!strcmp(token_list[*index], "spec")) {
+		/* Match value according to configured bit-mask. */
+		(*index)++;
+
+		if (*spec == NULL) {
+			ret = malloc_object(spec, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		ret = detail_list->parse_detail(token_list[*index],
+			(char *)(*spec) + detail_list->offset);
+		if (ret != 0) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"parse_detail error (%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+	} else if (!strcmp(token_list[*index], "last")) {
+		/* Specify upper bound to establish a range. */
+		(*index)++;
+
+		if (*last == NULL) {
+			ret = malloc_object(last, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		ret = detail_list->parse_detail(token_list[*index],
+			(char *)(*last) + detail_list->offset);
+		if (ret != 0) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"parse_detail error (%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+	} else if (!strcmp(token_list[*index], "mask")) {
+		/* Specify bit-mask with relevant bits set to one. */
+		(*index)++;
+
+		if (*mask == NULL) {
+			ret = malloc_object(mask, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		ret = detail_list->parse_detail(token_list[*index],
+			(char *)(*mask) + detail_list->offset);
+		if (ret != 0) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"parse_detail error (%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+	} else if (!strcmp(token_list[*index], "prefix")) {
+		/*
+		 * generate bit-mask with <prefix-length>
+		 * most-significant bits set to one.
+		 */
+		(*index)++;
+
+		if (*mask == NULL) {
+			ret = malloc_object(mask, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		prefix = strtoul(token_list[*index], &end, 10);
+		if (end == NULL || *end != '\0') {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Prefix is not a number(%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+		/* Compare prefix (bit) and size (byte). */
+		if (prefix > detail_list->size * 8) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Prefix value is too large(%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+		target = (char *)(*mask) + detail_list->offset;
+		memset(target, 0, detail_list->size);
+		for (i = 0; i < detail_list->size; i++) {
+			if (prefix <= 0)
+				break;
+
+			bitmask = 0x80;
+
+			for (j = 0; j < 8; j++) {
+				if (prefix <= 0)
+					break;
+
+				target[i] = target[i] | bitmask;
+				bitmask = bitmask >> 1;
+				prefix--;
+			}
+		}
+
+	} else {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Invalid parameter is %s(%s:%d)\n",
+			token_list[*index], __func__, __LINE__);
+		return -1;
+	}
+
+	return 0;
+}
+
+int
+parse_item_common(char *token_list[], int *index,
+	struct rte_flow_item *item,
+	struct flow_item_ops *ops)
+{
+	int ret = 0;
+	int i = 0;
+	//void *spec, *last, *mask;
+	void *spec = NULL, *last = NULL, *mask = NULL;
+	struct flow_detail_ops *detail_list = ops->detail_list;
+
+	/* Next to pattern word */
+	(*index)++;
+
+	while (token_list[*index] != NULL) {
+
+		/* Exit if "/" */
+		if (!strcmp(token_list[*index], "/"))
+			break;
+
+		/* First is value type */
+		i = 0;
+		while (detail_list[i].token != NULL) {
+			if (!strcmp(token_list[*index],
+				detail_list[i].token)) {
+				break;
+			}
+
+			i++;
+		}
+		if (detail_list[i].token == NULL) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Invalid \"%s\" pattern arguments(%s:%d)\n",
+				ops->str_type, __func__, __LINE__);
+			ret = -1;
+			break;
+		}
+
+		/* Parse token value */
+		if (detail_list[i].flg_value == 1) {
+			(*index)++;
+
+			ret = parse_rte_flow_item_field(token_list, index,
+				&detail_list[i], ops->size,
+				&spec, &last, &mask);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Invalid \"%s\" pattern arguments"
+					"(%s:%d)\n",
+					ops->str_type, __func__, __LINE__);
+				ret = -1;
+				break;
+			}
+		}
+
+		(*index)++;
+	}
+
+	/* Free memory allocated in case of failure. */
+	if (ret != 0) {
+		if (spec != NULL)
+			free(spec);
+
+		if (last != NULL)
+			free(last);
+
+		if (mask != NULL)
+			free(mask);
+	}
+
+	/* Parse result to item. */
+	item->spec = spec;
+	item->last = last;
+	item->mask = mask;
+
+	return ret;
+}
+
+int
+parse_action_common(char *token_list[], int *index,
+	struct rte_flow_action *action,
+	struct flow_action_ops *ops)
+{
+	int ret = 0;
+	int i = 0;
+	struct rte_flow_action_queue  *conf;
+	struct flow_detail_ops *detail_list = ops->detail_list;
+
+	conf = malloc(ops->size);
+	if (conf == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+	memset(conf, 0, ops->size);
+
+	/* Next to word */
+	(*index)++;
+
+	while (token_list[*index] != NULL) {
+
+		/* Exit if "/" */
+		if (!strcmp(token_list[*index], "/"))
+			break;
+
+		/* First is value type */
+		i = 0;
+		while (detail_list[i].token != NULL) {
+			if (!strcmp(token_list[*index],
+				detail_list[i].token)) {
+				break;
+			}
+
+			i++;
+		}
+		if (detail_list[i].token == NULL) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Invalid \"%s\" pattern arguments(%s:%d)\n",
+				ops->str_type, __func__, __LINE__);
+			ret = -1;
+			break;
+		}
+
+		/* Parse token value */
+		if (detail_list[i].flg_value == 1) {
+			(*index)++;
+
+			ret = detail_list[i].parse_detail(
+				token_list[*index],
+				(char *)conf + detail_list[i].offset);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Invalid \"%s\" pattern arguments"
+					"(%s:%d)\n",
+					detail_list[i].token,
+					__func__, __LINE__);
+				ret = -1;
+				break;
+			}
+		}
+
+		(*index)++;
+	}
+
+	/* Free memory allocated in case of failure. */
+	if ((ret != 0) && (conf != NULL))
+		free(conf);
+
+	/* Parse result to action. */
+	action->conf = conf;
+
+	return ret;
+}
+
+/* Append action json, conf field is null */
+int
+append_action_null_json(const void *conf __attribute__ ((unused)),
+	int buf_size, char *action_str)
+{
+	char null_str[] = "null";
+
+	if ((int)strlen(action_str) + (int)strlen(null_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, null_str, strlen(null_str));
+
+	return 0;
+}
+
+int
+malloc_object(void **ptr, size_t size)
+{
+	*ptr = malloc(size);
+	if (*ptr == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	memset(*ptr, 0, size);
+	return 0;
+}
diff --git a/src/primary/flow/common.h b/src/primary/flow/common.h
new file mode 100644
index 0000000..998fd12
--- /dev/null
+++ b/src/primary/flow/common.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_COMMON_H_
+#define _PRIMARY_FLOW_COMMON_H_
+
+int is_portid_used(int port_id);
+
+/* Function for flow command parse */
+int parse_phy_port_id(char *res_uid, int *port_id);
+
+/* Functions for converting string to data */
+int str_to_rte_ether_addr(char *mac_str, void *output);
+int str_to_tci(char *tci_str, void *output);
+int str_to_pcp(char *pcp_str, void *output);
+int str_to_rte_be16_t(char *target_str, void *output);
+int str_to_uint16_t(char *target_str, void *output);
+int str_to_uint32_t(char *target_str, void *output);
+
+/* Functions for setting string to data */
+int set_pcp_in_tci(char *pcp_str, void *output);
+int set_dei_in_tci(char *dei_str, void *output);
+int set_vid_in_tci(char *vid_str, void *output);
+
+/* Parse rte_flow_item for each field */
+int parse_rte_flow_item_field(char *token_list[], int *index,
+	struct flow_detail_ops *detail_list, size_t size,
+	void **spec, void **last, void **mask);
+
+/*
+ * Common parse for item type. Perform detailed parse with
+ * flow_detail_ops according to type.
+ */
+int parse_item_common(char *token_list[], int *index,
+	struct rte_flow_item *item,
+	struct flow_item_ops *ops);
+
+/*
+ * Common parse for action type. Perform detailed parse with
+ * flow_detail_ops according to type.
+ */
+int parse_action_common(char *token_list[], int *index,
+	struct rte_flow_action *action,
+	struct flow_action_ops *ops);
+
+/* Append action json, conf field is null */
+int append_action_null_json(const void *conf, int buf_size, char *action_str);
+
+/* Allocate memory for the size */
+int malloc_object(void **ptr, size_t size);
+
+#endif
diff --git a/src/primary/main.c b/src/primary/main.c
index ca81636..4ef9cb1 100644
--- a/src/primary/main.c
+++ b/src/primary/main.c
@@ -395,9 +395,7 @@ append_lcore_info_json(char *str,
 
 /* TODO(yasufum): change to use shared */
 static int
-append_port_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+append_port_info_json(char *str)
 {
 	unsigned int i;
 	unsigned int has_port = 0;  // for checking having port at last
@@ -405,7 +403,7 @@ append_port_info_json(char *str,
 	sprintf(str + strlen(str), "\"ports\":[");
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
+		if (ports_fwd_array[i][0].in_port_id == PORT_RESET)
 			continue;
 
 		has_port = 1;
@@ -457,9 +455,7 @@ append_port_info_json(char *str,
 
 /* TODO(yasufum): change to use shared */
 static int
-append_patch_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+append_patch_info_json(char *str)
 {
 	unsigned int i;
 	unsigned int has_patch = 0;  // for checking having patch at last
@@ -468,12 +464,12 @@ append_patch_info_json(char *str,
 	sprintf(str + strlen(str), "\"patches\":[");
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
+		if (ports_fwd_array[i][0].in_port_id == PORT_RESET)
 			continue;
 
 		RTE_LOG(INFO, PRIMARY, "Port ID %d\n", i);
 		RTE_LOG(INFO, PRIMARY, "Status %d\n",
-			ports_fwd_array[i].in_port_id);
+			ports_fwd_array[i][0].in_port_id);
 
 		memset(patch_str, '\0', sizeof(patch_str));
 
@@ -533,14 +529,14 @@ append_patch_info_json(char *str,
 		sprintf(patch_str + strlen(patch_str), "\"dst\":");
 
 		RTE_LOG(INFO, PRIMARY, "Out Port ID %d\n",
-				ports_fwd_array[i].out_port_id);
+				ports_fwd_array[i][0].out_port_id);
 
-		if (ports_fwd_array[i].out_port_id == PORT_RESET) {
+		if (ports_fwd_array[i][0].out_port_id == PORT_RESET) {
 			//sprintf(patch_str + strlen(patch_str), "%s", "\"\"");
 			continue;
 		} else {
 			has_patch = 1;
-			unsigned int j = ports_fwd_array[i].out_port_id;
+			unsigned int j = ports_fwd_array[i][0].out_port_id;
 			switch (port_map[j].port_type) {
 			case PHY:
 				RTE_LOG(INFO, PRIMARY, "Type: PHY\n");
@@ -629,8 +625,8 @@ forwarder_status_json(char *str)
 	else
 		sprintf(buf_running + strlen(buf_running), "\"%s\"", "idling");
 
-	append_port_info_json(buf_ports, ports_fwd_array, port_map);
-	append_patch_info_json(buf_patches, ports_fwd_array, port_map);
+	append_port_info_json(buf_ports);
+	append_patch_info_json(buf_patches);
 
 	sprintf(str, "\"forwarder\":{%s,%s,%s}", buf_running, buf_ports,
 			buf_patches);
@@ -871,7 +867,7 @@ add_port(char *p_type, int p_id)
 	 */
 
 	/* Update ports_fwd_array with port id */
-	ports_fwd_array[port_id].in_port_id = port_id;
+	ports_fwd_array[port_id][0].in_port_id = port_id;
 	return 0;
 }
 
@@ -942,7 +938,7 @@ del_port(char *p_type, int p_id)
 	port_id_list[dev_id].port_id = PORT_RESET;
 	port_id_list[dev_id].type = UNDEF;
 
-	forward_array_remove(dev_id);
+	forward_array_remove(dev_id, 0);
 	port_map_init_one(dev_id);
 
 	return 0;
@@ -964,6 +960,7 @@ parse_command(char *str)
 	char *p_type;
 	int p_id;
 	char tmp_response[MSG_SIZE];
+	uint16_t queue_id;
 
 	memset(sec_name, '\0', 16);
 	memset(tmp_response, '\0', MSG_SIZE);
@@ -1033,7 +1030,8 @@ parse_command(char *str)
 		RTE_LOG(DEBUG, PRIMARY, "'%s' command received.\n",
 				token_list[0]);
 
-		ret = parse_resource_uid(token_list[1], &p_type, &p_id);
+		ret = parse_resource_uid(token_list[1], &p_type, &p_id,
+			&queue_id);
 		if (ret < 0) {
 			RTE_LOG(ERR, PRIMARY, "Failed to parse RES UID.\n");
 			return ret;
@@ -1055,7 +1053,8 @@ parse_command(char *str)
 	} else if (!strcmp(token_list[0], "del")) {
 		RTE_LOG(DEBUG, PRIMARY, "Received del command\n");
 
-		ret = parse_resource_uid(token_list[1], &p_type, &p_id);
+		ret = parse_resource_uid(token_list[1], &p_type, &p_id,
+			&queue_id);
 		if (ret < 0) {
 			RTE_LOG(ERR, PRIMARY, "Failed to parse RES UID.\n");
 			return ret;
@@ -1094,13 +1093,15 @@ parse_command(char *str)
 			char *out_p_type;
 			int in_p_id;
 			int out_p_id;
+			uint16_t in_queue_id, out_queue_id;
 
-			parse_resource_uid(token_list[1], &in_p_type, &in_p_id);
+			parse_resource_uid(token_list[1], &in_p_type, &in_p_id,
+				&in_queue_id);
 			in_port = find_port_id(in_p_id,
 					get_port_type(in_p_type));
 
 			parse_resource_uid(token_list[2],
-					&out_p_type, &out_p_id);
+					&out_p_type, &out_p_id, &out_queue_id);
 			out_port = find_port_id(out_p_id,
 					get_port_type(out_p_type));
 
@@ -1128,7 +1129,8 @@ parse_command(char *str)
 				RTE_LOG(ERR, PRIMARY, "%s\n", err_msg);
 			}
 
-			if (add_patch(in_port, out_port) == 0) {
+			if (add_patch(in_port, in_queue_id, out_port,
+				out_queue_id) == 0) {
 				RTE_LOG(INFO, PRIMARY,
 					"Patched '%s:%d' and '%s:%d'\n",
 					in_p_type, in_p_id,
@@ -1323,10 +1325,12 @@ main(int argc, char *argv[])
 			}
 
 			/* Update ports_fwd_array with phy port. */
-			ports_fwd_array[i].in_port_id = i;
+			ports_fwd_array[i][0].in_port_id = i;
+			ports_fwd_array[i][0].in_queue_id = 0;
 			port_map[i].port_type = port_type;
 			port_map[i].id = port_id;
 			port_map[i].stats = &ports->port_stats[i];
+			port_map[i].queue_info = NULL;
 
 			/* TODO(yasufum) convert type of port_type to char */
 			RTE_LOG(DEBUG, PRIMARY, "Add port, type: %d, id: %d\n",
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 08/17] spp_primary: add attribute of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (6 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 07/17] spp_primary: add common function " x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 09/17] spp_primary: add patterns " x-fn-spp-ml
                   ` (26 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch provides support of various
attribute of rte_flow.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/attr.c | 105 ++++++++++++++++++++++++++++++++++++++++
 src/primary/flow/attr.h |  13 +++++
 2 files changed, 118 insertions(+)
 create mode 100644 src/primary/flow/attr.c
 create mode 100644 src/primary/flow/attr.h

diff --git a/src/primary/flow/attr.c b/src/primary/flow/attr.c
new file mode 100644
index 0000000..8be7319
--- /dev/null
+++ b/src/primary/flow/attr.c
@@ -0,0 +1,105 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "attr.h"
+
+int
+parse_flow_attr(char *token_list[], int *index,
+	struct rte_flow_attr *attr)
+{
+	int ret;
+	char *token;
+	char *end;
+	unsigned long temp = 0;
+
+	while (token_list[*index] != NULL) {
+		token = token_list[*index];
+
+		if (!strcmp(token, "group")) {
+			/* "group" requires option argument */
+			if (token_list[*index + 1] == NULL) {
+				ret = -1;
+				break;
+			}
+
+			temp = strtoul(token_list[*index + 1], &end, 10);
+			if (end == NULL || *end != '\0') {
+				ret = -1;
+				break;
+			}
+
+			attr->group = (uint32_t)temp;
+			(*index)++;
+
+		} else if (!strcmp(token, "priority")) {
+			/* "priority" requires option argument */
+			if (token_list[*index + 1] == NULL) {
+				ret = -1;
+				break;
+			}
+
+			temp = strtoul(token_list[*index + 1], &end, 10);
+			if (end == NULL || *end != '\0') {
+				ret = -1;
+				break;
+			}
+
+			attr->priority = (uint32_t)temp;
+			(*index)++;
+
+		} else if (!strcmp(token, "ingress")) {
+			attr->ingress = 1;
+
+		} else if (!strcmp(token, "egress")) {
+			attr->egress = 1;
+
+		} else if (!strcmp(token, "transfer")) {
+			attr->transfer = 1;
+
+		} else if (!strcmp(token, "pattern")) {
+			/* Attribute parameter end */
+			ret = 0;
+			break;
+
+		} else {
+			/* Illegal parameter */
+			ret = -1;
+			break;
+
+		}
+
+		(*index)++;
+	}
+
+	if (token_list[*index] == NULL)
+		ret = -1;
+
+	return ret;
+}
+
+int
+append_flow_attr_json(const struct rte_flow_attr *attr, int buf_size,
+	char *attr_str)
+{
+	char tmp_str[128] = { 0 };
+
+	snprintf(tmp_str, 128,
+		"{\"group\":%d,"
+		"\"priority\":%d,"
+		"\"ingress\":%d,"
+		"\"egress\":%d,"
+		"\"transfer\":%d}",
+		attr->group, attr->priority, attr->ingress,
+		attr->egress, attr->transfer);
+
+	if ((int)strlen(attr_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(attr_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/attr.h b/src/primary/flow/attr.h
new file mode 100644
index 0000000..5fb22e3
--- /dev/null
+++ b/src/primary/flow/attr.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ATTR_H_
+#define _PRIMARY_FLOW_ATTR_H_
+
+int parse_flow_attr(char *token_list[], int *index,
+	struct rte_flow_attr *attr);
+int append_flow_attr_json(const struct rte_flow_attr *attr,
+	int buf_size, char *attr_str);
+
+#endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 09/17] spp_primary: add patterns of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (7 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 08/17] spp_primary: add attribute " x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 10/17] spp_primary: add actions " x-fn-spp-ml
                   ` (25 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch adds the following
patterns of rte_flow.
   - eth
   - vlan
Additional files should be added when new pattern should be
newly supported by SPP.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/pattern/eth.c  | 63 +++++++++++++++++++++++++++++
 src/primary/flow/pattern/eth.h  | 13 ++++++
 src/primary/flow/pattern/vlan.c | 71 +++++++++++++++++++++++++++++++++
 src/primary/flow/pattern/vlan.h | 13 ++++++
 4 files changed, 160 insertions(+)
 create mode 100644 src/primary/flow/pattern/eth.c
 create mode 100644 src/primary/flow/pattern/eth.h
 create mode 100644 src/primary/flow/pattern/vlan.c
 create mode 100644 src/primary/flow/pattern/vlan.h

diff --git a/src/primary/flow/pattern/eth.c b/src/primary/flow/pattern/eth.c
new file mode 100644
index 0000000..d718b77
--- /dev/null
+++ b/src/primary/flow/pattern/eth.c
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "eth.h"
+
+/* Define item "eth" operations */
+struct flow_detail_ops eth_ops_list[] = {
+	{
+		.token = "src",
+		.offset = offsetof(struct rte_flow_item_eth, src),
+		.size = sizeof(struct rte_ether_addr),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_ether_addr,
+	},
+	{
+		.token = "dst",
+		.offset = offsetof(struct rte_flow_item_eth, dst),
+		.size = sizeof(struct rte_ether_addr),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_ether_addr,
+	},
+	{
+		.token = "type",
+		.offset = offsetof(struct rte_flow_item_eth, type),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_be16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_item_eth_json(const void *element, int buf_size, char *pattern_str)
+{
+	const struct rte_flow_item_eth *eth = element;
+	char dst_mac[RTE_ETHER_ADDR_FMT_SIZE] = { 0 };
+	char src_mac[RTE_ETHER_ADDR_FMT_SIZE] = { 0 };
+	char tmp_str[128] = { 0 };
+
+	rte_ether_format_addr(dst_mac, RTE_ETHER_ADDR_FMT_SIZE, &eth->dst);
+	rte_ether_format_addr(src_mac, RTE_ETHER_ADDR_FMT_SIZE, &eth->src);
+
+	snprintf(tmp_str, 128,
+		"{\"dst\":\"%s\","
+		"\"src\":\"%s\","
+		"\"type\":\"0x%04x\"}",
+		dst_mac, src_mac, eth->type);
+
+	if ((int)strlen(pattern_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(pattern_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/pattern/eth.h b/src/primary/flow/pattern/eth.h
new file mode 100644
index 0000000..550813e
--- /dev/null
+++ b/src/primary/flow/pattern/eth.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_PATTERN_ETH_H_
+#define _PRIMARY_FLOW_PATTERN_ETH_H_
+
+extern struct flow_detail_ops eth_ops_list[];
+
+int append_item_eth_json(const void *element, int buf_size,
+	char *pattern_str);
+
+#endif
diff --git a/src/primary/flow/pattern/vlan.c b/src/primary/flow/pattern/vlan.c
new file mode 100644
index 0000000..3ca8ebc
--- /dev/null
+++ b/src/primary/flow/pattern/vlan.c
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "vlan.h"
+
+/* Define item "vlan" operations */
+struct flow_detail_ops vlan_ops_list[] = {
+	{
+		.token = "tci",
+		.offset = offsetof(struct rte_flow_item_vlan, tci),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_tci,
+	},
+	{
+		.token = "pcp",
+		.offset = offsetof(struct rte_flow_item_vlan, tci),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = set_pcp_in_tci,
+	},
+	{
+		.token = "dei",
+		.offset = offsetof(struct rte_flow_item_vlan, tci),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = set_dei_in_tci,
+	},
+	{
+		.token = "vid",
+		.offset = offsetof(struct rte_flow_item_vlan, tci),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = set_vid_in_tci,
+	},
+	{
+		.token = "inner_type",
+		.offset = offsetof(struct rte_flow_item_vlan, inner_type),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_be16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_item_vlan_json(const void *element, int buf_size, char *pattern_str)
+{
+	const struct rte_flow_item_vlan *vlan = element;
+	char tmp_str[128] = { 0 };
+
+	snprintf(tmp_str, 128,
+		"{\"tci\":\"0x%04x\","
+		"\"inner_type\":\"0x%04x\"}",
+		vlan->tci, vlan->inner_type);
+
+	if ((int)strlen(pattern_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(pattern_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/pattern/vlan.h b/src/primary/flow/pattern/vlan.h
new file mode 100644
index 0000000..50aef96
--- /dev/null
+++ b/src/primary/flow/pattern/vlan.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_PATTERN_VLAN_H_
+#define _PRIMARY_FLOW_PATTERN_VLAN_H_
+
+extern struct flow_detail_ops vlan_ops_list[];
+
+int append_item_vlan_json(const void *element, int buf_size,
+	char *pattern_str);
+
+#endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 10/17] spp_primary: add actions of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (8 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 09/17] spp_primary: add patterns " x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 11/17] bin: add parameter for hardrare offload x-fn-spp-ml
                   ` (24 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch adds the following
actions of rte_flow.
   - queue
   - jump
   - pop_vlan
   - push_vlan
   - set_vlan_vid
   - set_vlan_pcp
Additional files should be added when new actions should be
newly supported by SPP.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/action/jump.c            | 42 ++++++++++++++++++++++
 src/primary/flow/action/jump.h            | 12 +++++++
 src/primary/flow/action/of_push_vlan.c    | 44 +++++++++++++++++++++++
 src/primary/flow/action/of_push_vlan.h    | 13 +++++++
 src/primary/flow/action/of_set_vlan_pcp.c | 44 +++++++++++++++++++++++
 src/primary/flow/action/of_set_vlan_pcp.h | 13 +++++++
 src/primary/flow/action/of_set_vlan_vid.c | 44 +++++++++++++++++++++++
 src/primary/flow/action/of_set_vlan_vid.h | 13 +++++++
 src/primary/flow/action/queue.c           | 42 ++++++++++++++++++++++
 src/primary/flow/action/queue.h           | 13 +++++++
 10 files changed, 280 insertions(+)
 create mode 100644 src/primary/flow/action/jump.c
 create mode 100644 src/primary/flow/action/jump.h
 create mode 100644 src/primary/flow/action/of_push_vlan.c
 create mode 100644 src/primary/flow/action/of_push_vlan.h
 create mode 100644 src/primary/flow/action/of_set_vlan_pcp.c
 create mode 100644 src/primary/flow/action/of_set_vlan_pcp.h
 create mode 100644 src/primary/flow/action/of_set_vlan_vid.c
 create mode 100644 src/primary/flow/action/of_set_vlan_vid.h
 create mode 100644 src/primary/flow/action/queue.c
 create mode 100644 src/primary/flow/action/queue.h

diff --git a/src/primary/flow/action/jump.c b/src/primary/flow/action/jump.c
new file mode 100644
index 0000000..b643015
--- /dev/null
+++ b/src/primary/flow/action/jump.c
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "jump.h"
+
+/* Define action "jump" operations */
+struct flow_detail_ops jump_ops_list[] = {
+	{
+		.token = "group",
+		.offset = offsetof(struct rte_flow_action_jump, group),
+		.size = sizeof(uint32_t),
+		.flg_value = 1,
+		.parse_detail = str_to_uint32_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_jump_json(const void *conf, int buf_size, char *action_str)
+{
+	const struct rte_flow_action_jump *jump = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"group\":%d}",
+		jump->group);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/jump.h b/src/primary/flow/action/jump.h
new file mode 100644
index 0000000..f8c6a60
--- /dev/null
+++ b/src/primary/flow/action/jump.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_JUMP_H_
+#define _PRIMARY_FLOW_ACTION_JUMP_H_
+
+extern struct flow_detail_ops jump_ops_list[];
+
+int append_action_jump_json(const void *conf, int buf_size, char *action_str);
+
+#endif
diff --git a/src/primary/flow/action/of_push_vlan.c b/src/primary/flow/action/of_push_vlan.c
new file mode 100644
index 0000000..749490e
--- /dev/null
+++ b/src/primary/flow/action/of_push_vlan.c
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: BSD-3-Claus1
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "of_push_vlan.h"
+
+/* Define action "of_push_vlan" operations */
+struct flow_detail_ops of_push_vlan_ops_list[] = {
+	{
+		.token = "ethertype",
+		.offset = offsetof(struct rte_flow_action_of_push_vlan,
+			ethertype),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_be16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_of_push_vlan_json(const void *conf, int buf_size,
+	char *action_str)
+{
+	const struct rte_flow_action_of_push_vlan *of_push_vlan = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"ethertype\":\"0x%04x\"}",
+		of_push_vlan->ethertype);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/of_push_vlan.h b/src/primary/flow/action/of_push_vlan.h
new file mode 100644
index 0000000..7c37ab1
--- /dev/null
+++ b/src/primary/flow/action/of_push_vlan.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_OF_PUSH_VLAN_H_
+#define _PRIMARY_FLOW_ACTION_OF_PUSH_VLAN_H_
+
+extern struct flow_detail_ops of_push_vlan_ops_list[];
+
+int append_action_of_push_vlan_json(const void *conf, int buf_size,
+	char *action_str);
+
+#endif
diff --git a/src/primary/flow/action/of_set_vlan_pcp.c b/src/primary/flow/action/of_set_vlan_pcp.c
new file mode 100644
index 0000000..33e23d5
--- /dev/null
+++ b/src/primary/flow/action/of_set_vlan_pcp.c
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "of_set_vlan_pcp.h"
+
+/* Define action "of_set_vlan_pcp" operations */
+struct flow_detail_ops of_set_vlan_pcp_ops_list[] = {
+	{
+		.token = "vlan_pcp",
+		.offset = offsetof(struct rte_flow_action_of_set_vlan_pcp,
+			vlan_pcp),
+		.size = sizeof(uint8_t),
+		.flg_value = 1,
+		.parse_detail = str_to_pcp,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_of_set_vlan_pcp_json(const void *conf, int buf_size,
+	char *action_str)
+{
+	const struct rte_flow_action_of_set_vlan_pcp *pcp = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"vlan_pcp\":\"0x%01x\"}",
+		pcp->vlan_pcp);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/of_set_vlan_pcp.h b/src/primary/flow/action/of_set_vlan_pcp.h
new file mode 100644
index 0000000..3f8a4b3
--- /dev/null
+++ b/src/primary/flow/action/of_set_vlan_pcp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_OF_SET_VLAN_PCP_H_
+#define _PRIMARY_FLOW_ACTION_OF_SET_VLAN_PCP_H_
+
+extern struct flow_detail_ops of_set_vlan_pcp_ops_list[];
+
+int append_action_of_set_vlan_pcp_json(const void *conf, int buf_size,
+	char *action_str);
+
+#endif
diff --git a/src/primary/flow/action/of_set_vlan_vid.c b/src/primary/flow/action/of_set_vlan_vid.c
new file mode 100644
index 0000000..930cef5
--- /dev/null
+++ b/src/primary/flow/action/of_set_vlan_vid.c
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "of_set_vlan_vid.h"
+
+/* Define action "of_set_vlan_vid" operations */
+struct flow_detail_ops of_set_vlan_vid_ops_list[] = {
+	{
+		.token = "vlan_vid",
+		.offset = offsetof(struct rte_flow_action_of_set_vlan_vid,
+			vlan_vid),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_be16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_of_set_vlan_vid_json(const void *conf, int buf_size,
+	char *action_str)
+{
+	const struct rte_flow_action_of_set_vlan_vid *vid = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"vlan_vid\":\"0x%04x\"}",
+		vid->vlan_vid);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/of_set_vlan_vid.h b/src/primary/flow/action/of_set_vlan_vid.h
new file mode 100644
index 0000000..a0abe85
--- /dev/null
+++ b/src/primary/flow/action/of_set_vlan_vid.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_OF_SET_VLAN_VID_H_
+#define _PRIMARY_FLOW_ACTION_OF_SET_VLAN_VID_H_
+
+extern struct flow_detail_ops of_set_vlan_vid_ops_list[];
+
+int append_action_of_set_vlan_vid_json(const void *conf, int buf_size,
+	char *action_str);
+
+#endif
diff --git a/src/primary/flow/action/queue.c b/src/primary/flow/action/queue.c
new file mode 100644
index 0000000..f7bcdce
--- /dev/null
+++ b/src/primary/flow/action/queue.c
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "queue.h"
+
+/* Define action "queue" operations */
+struct flow_detail_ops queue_ops_list[] = {
+	{
+		.token = "index",
+		.offset = offsetof(struct rte_flow_action_queue, index),
+		.size = sizeof(uint16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_uint16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_queue_json(const void *conf, int buf_size, char *action_str)
+{
+	const struct rte_flow_action_queue *queue = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"index\":%d}",
+		queue->index);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/queue.h b/src/primary/flow/action/queue.h
new file mode 100644
index 0000000..22816c7
--- /dev/null
+++ b/src/primary/flow/action/queue.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_QUEUE_H_
+#define _PRIMARY_FLOW_ACTION_QUEUE_H_
+
+extern struct flow_detail_ops queue_ops_list[];
+
+int append_action_queue_json(const void *conf, int buf_size,
+	char *action_str);
+
+#endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 11/17] bin: add parameter for hardrare offload
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (9 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 10/17] spp_primary: add actions " x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 12/17] cli: add support of hardware offload x-fn-spp-ml
                   ` (23 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch adds config for the followings to support hardware offload:
  - multi-queue setting
  - whitelist

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 bin/sample/config.sh | 15 +++++++++++++++
 bin/spp_pri.sh       | 26 +++++++++++++++++++++++++-
 bin/start.sh         |  2 ++
 3 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/bin/sample/config.sh b/bin/sample/config.sh
index ec4d4b8..214757e 100644
--- a/bin/sample/config.sh
+++ b/bin/sample/config.sh
@@ -29,3 +29,18 @@ PRI_PORTMASK=0x03  # total num of ports of spp_primary.
 # Log files are created in 'spp/log/'.
 SPP_CTL_LOG=spp_ctl.log
 PRI_LOG=spp_primary.log
+
+# number of ports for multi-queue setting.
+#PRI_PORT_QUEUE=(
+#    "0 rxq 16 txq 16"
+#)
+
+# Add a PCI device in white list.
+# `dv_flow_en=1` is required for HW offload with Mellanox NIC.
+# Set a nonzero value to enables the DV flow steering assuming it is
+# supported by the driver.
+# https://doc.dpdk.org/guides/nics/mlx5.html
+#PRI_WHITE_LIST=(
+#    "0000:04:00.0,dv_flow_en=1"
+#    "0000:05:00.0"
+#)
diff --git a/bin/spp_pri.sh b/bin/spp_pri.sh
index c0b0e1e..6238ee1 100644
--- a/bin/spp_pri.sh
+++ b/bin/spp_pri.sh
@@ -9,6 +9,8 @@ SPP_PRI_RING=""
 SPP_PRI_TAP=""
 SPP_PRI_MEMIF=""
 SPP_PRI_VDEVS=""
+SPP_PRI_PORT_QUEUE=""
+SPP_PRI_WHITE_LIST=""
 
 SOCK_VHOST="/tmp/sock"
 SOCK_MEMIF="/tmp/spp-memif.sock"
@@ -64,6 +66,26 @@ function setup_vdevs() {
     fi
 }
 
+# Add queue number to port
+function setup_queue_number() {
+    if [ ${#PRI_PORT_QUEUE[@]} ]; then
+        for (( i=0; i < ${#PRI_PORT_QUEUE[@]}; i++)); do
+            SPP_PRI_PORT_QUEUE="
+                ${SPP_PRI_PORT_QUEUE} --port-num ${PRI_PORT_QUEUE[${i}]}"
+        done
+    fi
+}
+
+# Add whitelist
+function setup_whitelist() {
+    if [ ${#PRI_WHITE_LIST[@]} ]; then
+        for (( i=0; i < ${#PRI_WHITE_LIST[@]}; i++)); do
+            SPP_PRI_WHITE_LIST="
+                ${SPP_PRI_WHITE_LIST} -w ${PRI_WHITE_LIST[${i}]}"
+        done
+    fi
+}
+
 # Launch spp_primary.
 function spp_pri() {
     SPP_PRI_BIN=${SPP_DIR}/src/primary/${RTE_TARGET}/spp_primary
@@ -86,10 +108,12 @@ function spp_pri() {
         ${SPP_PRI_TAP} \
         ${SPP_PRI_MEMIF} \
         ${SPP_PRI_VDEVS} \
+        ${SPP_PRI_WHITE_LIST} \
         -- \
         -p ${PRI_PORTMASK} \
         -n ${NUM_RINGS} \
-        -s ${SPP_CTL_IP}:5555"
+        -s ${SPP_CTL_IP}:5555 \
+        ${SPP_PRI_PORT_QUEUE}"
 
     if [ ${DRY_RUN} ]; then
         echo ${cmd}
diff --git a/bin/start.sh b/bin/start.sh
index eecb2ed..28dc3d9 100755
--- a/bin/start.sh
+++ b/bin/start.sh
@@ -45,6 +45,8 @@ function start_spp_pri() {
     setup_tap_vdevs  # setup vdevs of net_tap
     setup_memif_vdevs  # setup vdevs of net_memif
     setup_vdevs  # setup any of vdevs
+    setup_queue_number  # setup port number
+    setup_whitelist  # setup whitelist
     spp_pri  # launch spp_primary
 }
 
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 12/17] cli: add support of hardware offload
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (10 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 11/17] bin: add parameter for hardrare offload x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 13/17] cli: add support of rte_flow in vf x-fn-spp-ml
                   ` (22 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch adds support of flow command in primary process
with the following aspects.
1. adds completion of flow command.
2. display status of flow rules
3. support of the following flow command.
   - pri; flow create
   - pri; flow validate
   - pri; flow destroy

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/cli/commands/pri.py                    |  52 +-
 src/cli/commands/pri_flow.py               | 936 +++++++++++++++++++++
 src/cli/commands/pri_flow_compl_action.py  | 115 +++
 src/cli/commands/pri_flow_compl_pattern.py |  84 ++
 4 files changed, 1180 insertions(+), 7 deletions(-)
 create mode 100644 src/cli/commands/pri_flow.py
 create mode 100644 src/cli/commands/pri_flow_compl_action.py
 create mode 100644 src/cli/commands/pri_flow_compl_pattern.py

diff --git a/src/cli/commands/pri.py b/src/cli/commands/pri.py
index a13137d..dbe56d5 100644
--- a/src/cli/commands/pri.py
+++ b/src/cli/commands/pri.py
@@ -4,6 +4,7 @@
 from .. import spp_common
 from ..shell_lib import common
 from ..spp_common import logger
+from .pri_flow import SppPrimaryFlow
 import os
 import time
 
@@ -20,7 +21,7 @@ class SppPrimary(object):
 
     # All of primary commands used for validation and completion.
     PRI_CMDS = ['status', 'add', 'del', 'forward', 'stop', 'patch',
-                'launch', 'clear']
+                'launch', 'clear', 'flow']
 
     ENV_FILE_PREF = 'SPP_FILE_PREFIX'
 
@@ -46,6 +47,8 @@ class SppPrimary(object):
         temp = temp + "{vhost_cli}"
         self.launch_template = temp
 
+        self.flow = SppPrimaryFlow(spp_ctl_cli)
+
     def _do_if_forwarder_exists(self, status, func, params):
         """Execute command of func if forwarder thread is existing.
 
@@ -126,6 +129,9 @@ class SppPrimary(object):
                 else:
                     print('Error: unknown response for clear.')
 
+        elif subcmd == 'flow':
+            self._run_flow(params)
+
         else:
             print('Invalid pri command!')
 
@@ -185,8 +191,8 @@ class SppPrimary(object):
                 - phy:1
             - stats
               - physical ports:
-                  ID          rx          tx     tx_drop  mac_addr
-                   0    78932932    78932931           1  56:48:4f:53:54:00
+                  ID          rx          tx     tx_drop  rxq  txq  mac_addr
+                   0    78932932    78932931           1   16   16  56:48:...
                    ...
               - ring ports:
                   ID          rx          tx     rx_drop     tx_drop
@@ -228,15 +234,19 @@ class SppPrimary(object):
 
             if 'phy_ports' in json_obj:
                 print('  - physical ports:')
-                print('{s6}ID{s10}rx{s10}tx{s4}tx_drop  mac_addr'.format(
-                      s4=sep*4, s6=sep*6, s10=sep*10))
+                print('{s6}ID{s10}rx{s10}tx{s5}tx_drop'
+                      '  rxq  txq mac_addr'.format(
+                          s5=sep*5, s6=sep*6, s10=sep*10))
 
-                temp = '{s6}{portid:2}  {rx:10}  {tx:10}  {tx_d:10}  {eth}'
+                temp = '{s6}{portid:2}  {rx:10}  {tx:10}  {tx_d:10}' \
+                    ' {rxq:4} {txq:4} {eth}'
                 for pports in json_obj['phy_ports']:
                     print(temp.format(s6=sep*6,
                                       portid=pports['id'], rx=pports['rx'],
                                       tx=pports['tx'],
                                       tx_d=pports['tx_drop'],
+                                      rxq=pports["nof_queues"]["rx"],
+                                      txq=pports["nof_queues"]["tx"],
                                       eth=pports['eth']))
 
             if 'ring_ports' in json_obj:
@@ -560,9 +570,20 @@ class SppPrimary(object):
                         candidates = self._compl_del(tokens[1:])
                     elif tokens[1] == 'patch':
                         candidates = self._compl_patch(tokens[1:])
+                    elif tokens[1] == 'flow':
+                        candidates = self._compl_flow(tokens[1:])
 
+            completions = []
             if not text:
-                completions = candidates
+                for candidate in candidates:
+                    if not candidate.startswith(tokens[len(tokens) - 1]):
+                        completions.append(candidate)
+                        continue
+
+                    if ":" in tokens[len(tokens) - 1]:
+                        completions.append(candidate.split(":")[1])
+                    else:
+                        completions.append(candidate)
             else:
                 completions = [p for p in candidates
                                if p.startswith(text)
@@ -717,6 +738,10 @@ class SppPrimary(object):
 
             return res
 
+    def _compl_flow(self, tokens):
+        """Complete `flow` command."""
+        return self.flow.complete_flow(tokens)
+
     def _get_sec_ids(self):
         sec_ids = []
         res = self.spp_ctl_cli.get('processes')
@@ -745,6 +770,7 @@ class SppPrimary(object):
         """
         prekey = None
         opts_dict = {}
+        index = 0
         for opt in opts_list:
             if opt.startswith('-'):
                 opts_dict[opt] = None
@@ -752,7 +778,15 @@ class SppPrimary(object):
             else:
                 if prekey is not None:
                     opts_dict[prekey] = opt
+
+                    if (opt.startswith("phy:")
+                            and len(opts_list) > index + 2
+                            and opts_list[index + 1] == "nq"):
+                        opts_dict[prekey] += " nq {0}".format(
+                            opts_list[index + 2])
+
                     prekey = None
+            index += 1
         return opts_dict
 
     def _run_add(self, params):
@@ -980,6 +1014,10 @@ class SppPrimary(object):
             else:
                 print('Error: unknown response for launch.')
 
+    def _run_flow(self, params):
+        """Run `flow` command."""
+        self.flow.run_flow(params)
+
     @classmethod
     def help(cls):
         msg = """Send a command to primary process.
diff --git a/src/cli/commands/pri_flow.py b/src/cli/commands/pri_flow.py
new file mode 100644
index 0000000..de91a25
--- /dev/null
+++ b/src/cli/commands/pri_flow.py
@@ -0,0 +1,936 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+import re
+import importlib
+import copy
+from . import pri_flow_compl_pattern as flow_compl_ptn
+from . import pri_flow_compl_action as flow_compl_act
+
+
+class SppPrimaryFlow(object):
+    """Exec SPP primary flow command.
+
+    SppPrimaryFlow class is intended to be used in SppPrimary class as a
+    delegator for running 'flow' subcommand.
+
+    """
+
+    # All of flow commands
+    FLOW_CMDS = ["validate", "create", "destroy", "list", "status"]
+
+    # Attribute commands of flow rule
+    ATTR_CMDS = ["group", "priority", "ingress", "egress", "transfer",
+                 "pattern"]
+
+    # Host, port and api version in URL are complemented by
+    # SppCtlClient class and execute API
+    FLOW_API_VALIDATE = "primary/flow_rules/port_id/{port_id}/validate"
+    FLOW_API_CREATE = "primary/flow_rules/port_id/{port_id}"
+    FLOW_API_DESTROY = "primary/flow_rules/{rule_id}/port_id/{port_id}"
+    FLOW_API_ALL_DESTROY = "primary/flow_rules/port_id/{port_id}"
+
+    # Completion class relevant to the pattern item type
+    PTN_COMPL_CLASSES = {
+        "eth": flow_compl_ptn.ComplEth,
+        "vlan": flow_compl_ptn.ComplVlan,
+    }
+
+    # Completion class relevant to the action type
+    ACT_COMPL_CLASSES = {
+        "jump": flow_compl_act.ComplJump,
+        "of_pop_vlan": flow_compl_act.ComplOfPopVlan,
+        "of_push_vlan": flow_compl_act.ComplOfPushVlan,
+        "of_set_vlan_pcp": flow_compl_act.ComplOfSetVlanPCP,
+        "of_set_vlan_vid": flow_compl_act.ComplOfSetVlanVID,
+        "queue": flow_compl_act.ComplQueue,
+    }
+
+    def __init__(self, spp_ctl_cli):
+        self.spp_ctl_cli = spp_ctl_cli
+
+        # Use as index of tokens list when complementing flow_rule.
+        self._flow_rule_token_index = 0
+
+    def run_flow(self, params):
+        """Run `flow` command."""
+
+        if len(params) == 0:
+            print("Flow sub command is NULL")
+            return None
+
+        elif params[0] not in self.FLOW_CMDS:
+            print("Invalid flow sub command: '{0}'".format(params[0]))
+            return None
+
+        if params[0] == "validate":
+            self._run_flow_validate(params[1:])
+
+        elif params[0] == "create":
+            self._run_flow_create(params[1:])
+
+        elif params[0] == "destroy":
+            self._run_flow_destroy(params[1:])
+
+        elif params[0] == "list":
+            self._run_flow_list(params[1:])
+
+        elif params[0] == "status":
+            self._run_flow_status(params[1:])
+
+    def complete_flow(self, tokens):
+        """Completion for flow commands."""
+        candidates = []
+
+        if len(tokens) == 2:
+            for kw in self.FLOW_CMDS:
+                if kw.startswith(tokens[1]):
+                    candidates.append(kw)
+
+        else:
+            if tokens[1] == "validate":
+                candidates = self._compl_flow_rule(tokens[2:])
+
+            elif tokens[1] == "create":
+                candidates = self._compl_flow_rule(tokens[2:])
+
+            elif tokens[1] == "destroy":
+                candidates = self._compl_flow_destroy(tokens[2:])
+
+            elif tokens[1] == "list":
+                candidates = self._compl_flow_list(tokens[2:])
+
+            elif tokens[1] == "status":
+                candidates = self._compl_flow_status(tokens[2:])
+
+        return candidates
+
+    def _compl_flow_rule(self, tokens):
+        """Completion for validate and create commands."""
+        candidates = []
+
+        self._flow_rule_token_index = 0
+
+        if len(tokens) == 1:
+            candidates = self._create_candidacy_phy_ports(tokens[0])
+            return candidates
+
+        # Next index of RES_UID
+        self._flow_rule_token_index += 1
+
+        # Completion of attribute part in flow rule
+        candidates = self._compl_flow_rule_attribute(tokens)
+        if self._flow_rule_token_index == len(tokens):
+            return candidates
+
+        # Completion of pattern part in flow rule
+        candidates = self._compl_flow_rule_pattern(tokens)
+        if self._flow_rule_token_index == len(tokens):
+            return candidates
+
+        # Completion of action part in flow rule
+        candidates = self._compl_flow_rule_action(tokens)
+        if self._flow_rule_token_index == len(tokens):
+            return candidates
+
+        return candidates
+
+    def _compl_flow_rule_attribute(self, tokens):
+        """Completion for flow rule in attribute."""
+        candidates = []
+
+        while self._flow_rule_token_index < len(tokens):
+            # If "group" is specified, "GROUP_ID" is a candidate
+            if tokens[self._flow_rule_token_index - 1] == "group":
+                candidates = ["GROUP_ID"]
+
+            # If "priority" is specified, "LEVEL" is a candidate
+            elif tokens[self._flow_rule_token_index - 1] == "priority":
+                candidates = ["LEVEL"]
+
+            # If "pattern" is specified, exit attribute completion
+            elif tokens[self._flow_rule_token_index - 1] == "pattern":
+                candidates = []
+                break
+
+            else:
+                candidates = self.ATTR_CMDS
+
+            self._flow_rule_token_index += 1
+
+        return candidates
+
+    def _compl_flow_rule_pattern(self, tokens):
+        """Completion for flow rule in pattern."""
+        candidates = []
+
+        while self._flow_rule_token_index < len(tokens):
+            token = tokens[self._flow_rule_token_index - 1]
+
+            if token in self.PTN_COMPL_CLASSES.keys():
+                try:
+                    item_cls = self.PTN_COMPL_CLASSES[token]
+                    item_instance = item_cls()
+
+                    candidates, index = item_instance.compl_item(
+                        tokens, self._flow_rule_token_index)
+
+                    self._flow_rule_token_index = index
+                    if self._flow_rule_token_index == len(tokens):
+                        break
+
+                    if (tokens[self._flow_rule_token_index - 1] == "/"):
+                        # Type candidate and end token
+                        pattern_list = list(self.PTN_COMPL_CLASSES.keys())
+                        candidates = copy.deepcopy(pattern_list)
+                        candidates.append("end")
+
+                except Exception as _:
+                    candidates = []
+
+            elif (token == "end"):
+                candidates = []
+                break
+
+            else:
+                # Type candidate and end token
+                pattern_list = list(self.PTN_COMPL_CLASSES.keys())
+                candidates = copy.deepcopy(pattern_list)
+                candidates.append("end")
+
+            self._flow_rule_token_index += 1
+
+        return candidates
+
+    def _compl_flow_rule_action(self, tokens):
+        """Completion for flow rule in action."""
+        candidates = []
+
+        if (tokens[self._flow_rule_token_index - 1] == "end"):
+            candidates = ["actions"]
+            self._flow_rule_token_index += 1
+
+        while self._flow_rule_token_index < len(tokens):
+            token = tokens[self._flow_rule_token_index - 1]
+
+            if token in self.ACT_COMPL_CLASSES.keys():
+                try:
+                    action_cls = self.ACT_COMPL_CLASSES[token]
+                    action_instance = action_cls()
+
+                    candidates, index = action_instance.compl_action(
+                        tokens, self._flow_rule_token_index)
+
+                    self._flow_rule_token_index = index
+                    if self._flow_rule_token_index == len(tokens):
+                        break
+
+                    if (tokens[self._flow_rule_token_index - 1] == "/"):
+                        # Type candidate and end token
+                        action_list = list(self.ACT_COMPL_CLASSES.keys())
+                        candidates = copy.deepcopy(action_list)
+                        candidates.append("end")
+
+                except Exception as _:
+                    candidates = []
+
+            elif (tokens[self._flow_rule_token_index - 1] == "end"):
+                candidates = []
+                break
+
+            else:
+                # Type candidate and end token
+                action_list = list(self.ACT_COMPL_CLASSES.keys())
+                candidates = copy.deepcopy(action_list)
+                candidates.append("end")
+
+            self._flow_rule_token_index += 1
+
+        return candidates
+
+    def _compl_flow_destroy(self, tokens):
+        """Completion for destroy command."""
+        candidates = []
+
+        if len(tokens) == 1:
+            candidates = self._create_candidacy_phy_ports(tokens[0])
+
+        elif len(tokens) == 2:
+            candidates.append("ALL")
+
+            rule_ids = self._get_rule_ids(tokens[0])
+            if rule_ids is not None:
+                candidates.extend(rule_ids)
+
+        return candidates
+
+    def _compl_flow_list(self, tokens):
+        """Completion for list command."""
+        candidates = []
+
+        if len(tokens) == 1:
+            candidates = self._create_candidacy_phy_ports(tokens[0])
+
+        return candidates
+
+    def _compl_flow_status(self, tokens):
+        """Completion for status command."""
+        candidates = []
+
+        if len(tokens) == 1:
+            candidates = self._create_candidacy_phy_ports(tokens[0])
+
+        elif len(tokens) == 2:
+            rule_ids = self._get_rule_ids(tokens[0])
+
+            if rule_ids is not None:
+                candidates = rule_ids
+
+        return candidates
+
+    def _run_flow_validate(self, params):
+        """Run `validate` command."""
+        if len(params) == 0:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        if len(params) < 2:
+            print("Flow command invalid argument")
+            return None
+
+        body = self._parse_flow_rule(params[1:])
+        if body is None:
+            print("Flow command invalid argument")
+            return None
+
+        url = self.FLOW_API_VALIDATE.format(port_id=port_id)
+
+        response = self.spp_ctl_cli.post(url, body)
+        if response is None or response.status_code != 200:
+            print("Error: API execution failed for flow validate")
+            return None
+
+        try:
+            res_body = response.json()
+        except Exception as _:
+            print("Error: API response is not json")
+            return None
+
+        message = res_body.get("message")
+        if message is not None:
+            print(message)
+        else:
+            print("Error: result message is None")
+
+    def _run_flow_create(self, params):
+        """Run `create` command."""
+        if len(params) == 0:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        if len(params) < 2:
+            print("Flow command invalid argument")
+            return None
+
+        body = self._parse_flow_rule(params[1:])
+        if body is None:
+            print("Flow command invalid argument")
+            return None
+
+        url = self.FLOW_API_CREATE.format(port_id=port_id)
+
+        response = self.spp_ctl_cli.post(url, body)
+        if response is None or response.status_code != 200:
+            print("Error: API execution failed for flow create")
+            return None
+
+        try:
+            res_body = response.json()
+        except Exception as _:
+            print("Error: API response is not json")
+            return None
+
+        message = res_body.get("message")
+        if message is not None:
+            print(message)
+        else:
+            print("Error: result message is None")
+
+    def _run_flow_destroy(self, params):
+        """Run `destroy` command."""
+        if len(params) == 0:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        if len(params) < 2:
+            print("RULE_ID is NULL")
+            return None
+
+        all_destroy = False
+
+        if params[1] == "ALL":
+            all_destroy = True
+        else:
+            try:
+                rule_id = int(params[1])
+            except Exception as _:
+                print("RULE_ID is invalid")
+                return None
+
+        if all_destroy:
+            url = self.FLOW_API_ALL_DESTROY.format(port_id=port_id)
+        else:
+            url = self.FLOW_API_DESTROY.format(
+                port_id=port_id, rule_id=rule_id)
+
+        response = self.spp_ctl_cli.delete(url)
+        if response is None or response.status_code != 200:
+            print("Error: API execution failed for flow destroy")
+            return None
+
+        try:
+            res_body = response.json()
+        except Exception as _:
+            print("Error: API response is not json")
+            return None
+
+        message = res_body.get("message")
+        if message is not None:
+            print(message)
+        else:
+            print("Error: result message is None")
+
+    def _run_flow_list(self, params):
+        """Run `list` command."""
+        if len(params) != 1:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        status = self._get_pri_status()
+        if (status is None) or ("phy_ports" not in status):
+            print("Failed to get primary status")
+            return None
+
+        target_flow_list = None
+        for phy_port in status.get("phy_ports"):
+            if phy_port.get("id") == port_id:
+                target_flow_list = phy_port.get("flow")
+                break
+
+        if target_flow_list is None:
+            print("'{0}' is invalid".format(params[0]))
+            return None
+
+        self._print_flow_list(target_flow_list)
+
+    def _run_flow_status(self, params):
+        """Run `status` command."""
+        if len(params) == 0:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        if len(params) < 2:
+            print("RULE_ID is NULL")
+            return None
+
+        try:
+            rule_id = int(params[1])
+        except Exception as _:
+            print("RULE_ID is invalid")
+            return None
+
+        status = self._get_pri_status()
+        if (status is None) or ("phy_ports" not in status):
+            print("Failed to get primary status")
+            return None
+
+        target_flow_list = None
+        for phy_port in status.get("phy_ports"):
+            if phy_port.get("id") == port_id:
+                target_flow_list = phy_port.get("flow")
+                break
+
+        if target_flow_list is None:
+            print("'{0}' is invalid".format(params[0]))
+            return None
+
+        target_flow = None
+        for flow in target_flow_list:
+            if flow.get("rule_id") == rule_id:
+                target_flow = flow
+
+        if target_flow is None:
+            print("RULE_ID:{0} does not exist.".format(params[1]))
+            return None
+
+        self._print_flow_status(target_flow)
+
+    def _get_pri_status(self):
+        """Get primary status."""
+        try:
+            res = self.spp_ctl_cli.get('primary/status')
+            if res is None or res.status_code != 200:
+                print("Error: receive error response from primary")
+                return None
+        except Exception as _:
+            print("Error: there is an error sending the HTTP request")
+            return None
+
+        try:
+            body = res.json()
+        except Exception as _:
+            print("Error: response body of primary status is not json.")
+            return None
+
+        return body
+
+    def _get_candidacy_phy_ports(self):
+        """Get physical_ports candidates
+
+        Return port_id list, for example: ["0", "1"]
+        If physical_ports candidates cannot be returned, return None.
+        """
+        pri_status = self._get_pri_status()
+        if pri_status is None:
+            return None
+
+        if "phy_ports" not in pri_status:
+            return None
+
+        candidates = []
+
+        for port in pri_status["phy_ports"]:
+            if "id" not in port:
+                continue
+
+            candidates.append(str(port["id"]))
+
+        return candidates
+
+    def _get_rule_ids(self, res_uid):
+        """Get rule_ids condidates
+
+        Return rule_ids list, for example: ["0", "1"]
+        If rule_ids candidates cannot be returned, return None.
+        """
+        if not res_uid.startswith("phy:"):
+            return None
+
+        try:
+            port_id = int(re.sub(r"\D", "", res_uid))
+        except Exception as _:
+            return None
+
+        pri_status = self._get_pri_status()
+        if pri_status is None:
+            return None
+
+        if "phy_ports" not in pri_status:
+            return None
+
+        flow_list = None
+        for port in pri_status["phy_ports"]:
+            if port["id"] == port_id:
+                flow_list = port["flow"]
+                break
+
+        if flow_list is None:
+            return None
+
+        candidates = []
+        for flow in flow_list:
+            candidates.append(str(flow["rule_id"]))
+
+        return candidates
+
+    def _create_candidacy_phy_ports(self, text):
+        """Create physical_ports candidate list
+
+        Return phy_ports candidate list, for example: ["phy:0", "phy:1"].
+        If phy_ports candidates cannot be returned, return None.
+        """
+        port_ids = self._get_candidacy_phy_ports()
+        if port_ids is None:
+            return None
+        return ["phy:" + p for p in port_ids]
+
+    def _parse_phy_res_uid_to_port_id(self, res_uid):
+        """Extract port_id from phy RES_UID string
+
+        For example, extract 0 from `phy:0` and return.
+        If it is not RES_UID or not phy, return None.
+        """
+        if type(res_uid) is not str:
+            return None
+
+        tokens = res_uid.split(":")
+
+        if (len(tokens) != 2) or (tokens[0] != "phy"):
+            return None
+
+        try:
+            port_id = int(tokens[1])
+        except Exception as _:
+            return None
+
+        return port_id
+
+    def _parse_flow_rule(self, params):
+        """Parse the flow rule and convert to dict type."""
+        index = 0
+        max_index = len(params) - 1
+        flow_rule = {"rule": {}}
+
+        index = self._parse_flow_rule_attribute(params, index,
+                                                flow_rule["rule"])
+        if index is None or index >= max_index:
+            return None
+
+        index = self._parse_flow_rule_patterns(params, index,
+                                               flow_rule["rule"])
+        if index is None or index >= max_index:
+            return None
+
+        index = self._parse_flow_rule_actions(params, index,
+                                              flow_rule["rule"])
+        if index is None:
+            return None
+
+        return flow_rule
+
+    def _parse_flow_rule_attribute(self, params, index, flow_rule):
+        """Parse attribute of flow rule and convert to dict type.
+
+        Returns the index where "pattern" is specified
+        and the attribute part is completed.
+        It is abnormal that index exceeds "len(params)"
+        because "pattern" is not specified.
+        """
+        try:
+            while index < len(params):
+                if params[index] == "group":
+                    index += 1
+                    flow_rule["group"] = int(params[index])
+
+                elif params[index] == "priority":
+                    index += 1
+                    flow_rule["priority"] = int(params[index])
+
+                elif params[index] == "ingress" or params[index] == "egress":
+                    flow_rule["direction"] = params[index]
+
+                elif params[index] == "transfer":
+                    flow_rule["transfer"] = True
+
+                elif params[index] == "pattern":
+                    break
+
+                index += 1
+
+        except Exception as _:
+            return None
+
+        return index
+
+    def _parse_flow_rule_patterns(self, params, index, flow_rule):
+        """Parse patterns of flow rule and convert to dict type.
+
+        Returns the index where "end" is specified
+        and the pattern part is completed.
+        It is abnormal that index exceeds "len(params)"
+        because "end" is not specified.
+        """
+        sentensce = ""
+        flow_rule["pattern"] = []
+
+        if params[index] != "pattern":
+            return None
+
+        index += 1
+
+        while index < len(params):
+            if params[index] == "/":
+                flow_rule["pattern"].append(sentensce.rstrip())
+                sentensce = ""
+
+            elif params[index] == "end":
+                index += 1
+                break
+
+            else:
+                sentensce += params[index] + " "
+
+            index += 1
+
+        return index
+
+    def _parse_flow_rule_actions(self, params, index, flow_rule):
+        """Parse actions of flow rule and convert to dict type.
+
+        Returns the index where "end" is specified
+        and the action part is completed.
+        It is abnormal that index exceeds "len(params)"
+        because "end" is not specified.
+        """
+        sentensce = ""
+        flow_rule["actions"] = []
+
+        if params[index] != "actions":
+            return None
+
+        index += 1
+
+        while index < len(params):
+            if params[index] == "/":
+                flow_rule["actions"].append(sentensce.rstrip())
+                sentensce = ""
+
+            elif params[index] == "end":
+                index += 1
+                break
+
+            else:
+                sentensce += params[index] + " "
+
+            index += 1
+
+        return index
+
+    def _print_flow_list(self, flow_list):
+        """Print flow information as a list.
+
+        Print example.
+        -----
+        spp > pri; flow list phy:0
+        ID      Group   Prio    Attr    Rule
+        0       1       0       -e-     ETH => OF_PUSH_VLAN OF_SET_VLAN_VID
+        1       1       0       i--     ETH VLAN => QUEUE OF_POP_VLAN
+        2       0       0       i--     ETH => JUMP
+        -----
+        """
+        print("ID      Group   Prio    Attr    Rule")
+
+        for flow in flow_list:
+            print_data = {}
+
+            try:
+                print_data["id"] = str(flow.get("rule_id")).ljust(7)
+
+                attr = flow.get("attr")
+                if attr is None:
+                    continue
+
+                print_data["group"] = str(attr.get("group")).ljust(7)
+                print_data["prio"] = str(attr.get("priority")).ljust(7)
+
+                ingress = "i" if attr.get("ingress") == 1 else "-"
+                egress = "e" if attr.get("egress") == 1 else "-"
+                transfer = "t" if attr.get("transfer") == 1 else "-"
+                print_data["attr"] = "{0}{1}{2}".format(
+                    ingress, egress, transfer).ljust(7)
+
+                patterns = flow.get("patterns")
+                if patterns is None:
+                    continue
+
+                print_data["rule"] = ""
+                for ptn in patterns:
+                    print_data["rule"] += "{0} ".format(
+                        ptn.get("type").upper())
+                print_data["rule"] += "=> "
+
+                actions = flow.get("actions")
+                if actions is None:
+                    continue
+
+                for act in actions:
+                    print_data["rule"] += "{0} ".format(
+                        act.get("type").upper())
+
+                print("{id} {group} {prio} {attr} {rule}".format(**print_data))
+
+            except Exception as _:
+                continue
+
+    def _print_flow_status(self, flow):
+        """Print details of flow information."""
+        # Attribute print
+        self._print_flow_status_attribute(flow.get("attr"))
+
+        # Patterns print
+        self._print_flow_status_patterns(flow.get("patterns"))
+
+        # Actions print
+        self._print_flow_status_actions(flow.get("actions"))
+
+    def _print_flow_status_attribute(self, attr):
+        """Print attribute in the details of flow information.
+
+        Print example.
+        -----
+        Attribute:
+          Group   Priority Ingress Egress Transfer
+          1       0        true    false  false
+        -----
+        """
+        print_data = {}
+
+        print_data["group"] = str(attr.get("group")).ljust(7)
+        print_data["priority"] = str(attr.get("priority")).ljust(8)
+        print_data["ingress"] = ("true" if attr.get("ingress") == 1
+                                 else "false").ljust(7)
+        print_data["egress"] = ("true" if attr.get("egress") == 1
+                                else "false").ljust(6)
+        print_data["transfer"] = ("true" if attr.get("transfer") == 1
+                                  else "false").ljust(8)
+
+        print("Attribute:")
+        print("  Group   Priority Ingress Egress Transfer")
+        print("  {group} {priority} {ingress} {egress} {transfer}".format(
+            **print_data))
+
+    def _print_flow_status_patterns(self, patterns):
+        """Print patterns in the details of flow information.
+
+        Print example.
+        -----
+        Patterns:
+          - eth:
+            - spec:
+              - dst: 10:22:33:44:55:66
+              - src: 00:00:00:00:00:00
+              - type: 0xffff
+            - last:
+            - mask:
+              - dst: ff:ff:ff:ff:ff:ff
+              - src: 00:00:00:00:00:00
+              - type: 0xffff
+          - vlan:
+            - spec:
+              - tci: 0x0064
+              - inner_type: 0x0000
+            - last:
+            - mask:
+              - tci: 0xffff
+              - inner_type: 0x0000
+        -----
+        """
+        item_type_indent = 2
+        item_fields_indent = 4
+
+        try:
+            print("Patterns:")
+
+            for item in patterns:
+                # Type print
+                self._print_key_value(item.get("type"), None, item_type_indent)
+
+                # Spec print
+                self._print_key_value(
+                    "spec", None, item_fields_indent)
+                spec = item.get("spec")
+                if spec is not None:
+                    self._print_item_fields(spec)
+
+                # Last print
+                self._print_key_value(
+                    "last", None, item_fields_indent)
+                last = item.get("last")
+                if last is not None:
+                    self._print_item_fields(last)
+
+                # Mask print
+                self._print_key_value(
+                    "mask", None, item_fields_indent)
+                mask = item.get("mask")
+                if mask is not None:
+                    self._print_item_fields(mask)
+
+        except Exception as _:
+            print("Error: `patterns` structure of json received "
+                  "from spp-ctl is invalid")
+            return
+
+    def _print_flow_status_actions(self, actions):
+        """Print actions in the details of flow information.
+
+        Print example.
+        -----
+        Actions:
+          - queue:
+            - index: 0
+          - of_pop_vlan:
+        -----
+        """
+        act_type_indent = 2
+        act_fields_indent = 4
+
+        try:
+            print("Actions:")
+
+            for act in actions:
+                # Type print
+                self._print_key_value(act.get("type"), None, act_type_indent)
+
+                # Conf print
+                conf = act.get("conf")
+                if conf is not None:
+                    self._print_action_conf(conf)
+
+        except Exception as _:
+            print("Error: `actions` structure of json received "
+                  "from spp-ctl is invalid")
+            return
+
+    def _print_item_fields(self, fields_dic):
+        """Print each field (spec, last or mask) of flow item."""
+        item_elements_indent = 6
+
+        for key, value in fields_dic.items():
+            self._print_key_value(key, value, item_elements_indent)
+
+    def _print_action_conf(self, conf):
+        """Print conf of flow action."""
+        act_elements_indent = 4
+
+        for key, value in conf.items():
+            self._print_key_value(key, value, act_elements_indent)
+
+    def _print_key_value(self, key, value=None, indent_level=0):
+        """Print key / value combination.
+
+        Print in the format "- key: value".
+        If value is None, print in "- key:" format
+        Insert space for indent.
+        """
+        if value is not None:
+            print_str = "- {key}: {value}".format(
+                key=str(key), value=str(value))
+        else:
+            print_str = "- {key}:".format(key=str(key))
+
+        print((" " * indent_level) + print_str)
diff --git a/src/cli/commands/pri_flow_compl_action.py b/src/cli/commands/pri_flow_compl_action.py
new file mode 100644
index 0000000..748029d
--- /dev/null
+++ b/src/cli/commands/pri_flow_compl_action.py
@@ -0,0 +1,115 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+import copy
+
+
+class BaseComplAction(object):
+    """Base class for completion of actions list.
+
+    Each action inherits from this class.
+    Provides base functionality and command completion.
+    """
+
+    def __init__(self, data=None):
+        if data is not None:
+            self.data = data
+
+    def compl_action(self, tokens, index):
+        """Completion for actions list.
+
+        Complement using the information defined in the inherited class.
+        """
+        candidates = []
+
+        while index < len(tokens):
+            if tokens[index - 1] == "/":
+                # Completion processing end when "/" is specified
+                candidates = []
+                break
+
+            elif tokens[index - 1] in self.DATA_FIELDS:
+                tmp_token = self.DATA_FIELDS_VALUES.get(tokens[index - 1])
+                if tmp_token is not None:
+                    candidates = [tmp_token]
+                else:
+                    candidates = []
+
+            else:
+                # Data fields candidate and end token
+                candidates = copy.deepcopy(self.DATA_FIELDS)
+                candidates.append("/")
+
+            index += 1
+
+        return (candidates, index)
+
+
+class ComplJump(BaseComplAction):
+    """Complete action `jump`."""
+
+    # Jump data fields
+    DATA_FIELDS = ["group"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "group": "UNSIGNED_INT"
+    }
+
+
+class ComplOfPopVlan(BaseComplAction):
+    """Complete action `of_pop_vlan`."""
+
+    # of_pop_vlan data fields
+    DATA_FIELDS = []
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+    }
+
+
+class ComplOfPushVlan(BaseComplAction):
+    """Complete action `of_push_vlan`."""
+
+    # of_push_vlan data fields
+    DATA_FIELDS = ["ethertype"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "ethertype": "UNSIGNED_INT"
+    }
+
+
+class ComplOfSetVlanPCP(BaseComplAction):
+    """Complete action `of_set_vlan_pcp`."""
+
+    # of_set_vlan_pcp data fields
+    DATA_FIELDS = ["vlan_pcp"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "vlan_pcp": "UNSIGNED_INT"
+    }
+
+
+class ComplOfSetVlanVID(BaseComplAction):
+    """Complete action `of_set_vlan_vid`."""
+
+    # of_set_vlan_vid data fields
+    DATA_FIELDS = ["vlan_vid"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "vlan_vid": "UNSIGNED_INT"
+    }
+
+
+class ComplQueue(BaseComplAction):
+    """Complete action `queue`."""
+
+    # Queue data fields
+    DATA_FIELDS = ["index"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "index": "UNSIGNED_INT"
+    }
diff --git a/src/cli/commands/pri_flow_compl_pattern.py b/src/cli/commands/pri_flow_compl_pattern.py
new file mode 100644
index 0000000..8f84a0c
--- /dev/null
+++ b/src/cli/commands/pri_flow_compl_pattern.py
@@ -0,0 +1,84 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+import copy
+
+
+class BaseComplPatternItem(object):
+    """Base class for completion of pattern items.
+
+    Each pattern item inherits from this class.
+    Provides base functionality and command completion.
+    """
+
+    # Token after DATA_FIELDS
+    MATCHING_PATTERN = ["is", "spec", "last", "mask", "prefix"]
+
+    def __init__(self, data=None):
+        if data is not None:
+            self.data = data
+
+    def compl_item(self, tokens, index):
+        """Completion for pattern item commands.
+
+        Complement using the information defined in the inherited class.
+        """
+        candidates = []
+
+        while index < len(tokens):
+            if tokens[index - 1] == "/":
+                # Completion processing end when "/" is specified
+                candidates = []
+                break
+
+            elif tokens[index - 1] in self.DATA_FIELDS:
+                candidates = self.MATCHING_PATTERN
+
+            elif tokens[index - 1] in self.MATCHING_PATTERN:
+                if tokens[index - 1] == "prefix":
+                    candidates = ["Prefix"]
+
+                else:
+                    tmp_token = self.DATA_FIELDS_VALUES.get(tokens[index - 2])
+                    if tmp_token is not None:
+                        candidates = [tmp_token]
+                    else:
+                        candidates = []
+
+            else:
+                # Data fields candidate and end token
+                candidates = copy.deepcopy(self.DATA_FIELDS)
+                candidates.append("/")
+
+            index += 1
+
+        return (candidates, index)
+
+
+class ComplEth(BaseComplPatternItem):
+    """Complete pattern item `eth`."""
+
+    # Eth data fields
+    DATA_FIELDS = ["dst", "src", "type"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "dst": "MAC_ADDRESS",
+        "src": "MAC_ADDRESS",
+        "type": "UNSIGNED_INT"
+    }
+
+
+class ComplVlan(BaseComplPatternItem):
+    """Complete pattern item `vlan`."""
+
+    # Vlan data fields
+    DATA_FIELDS = ["tci", "pcp", "dei", "vid", "inner_type"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "tci": "UNSIGNED_INT",
+        "pcp": "UNSIGNED_INT",
+        "dei": "UNSIGNED_INT",
+        "vid": "UNSIGNED_INT",
+        "inner_type": "UNSIGNED_INT"
+    }
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 13/17] cli: add support of rte_flow in vf
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (11 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 12/17] cli: add support of hardware offload x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 14/17] cli: add support of rte_flow in mirror x-fn-spp-ml
                   ` (21 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch implements support of multi-queue in vf command.
  - vf; status
  - vf; port
  - vf; classifier_table

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/cli/commands/vf.py | 460 +++++++++++++++++++++++++++--------------
 1 file changed, 301 insertions(+), 159 deletions(-)

diff --git a/src/cli/commands/vf.py b/src/cli/commands/vf.py
index 5b361d1..c05412c 100644
--- a/src/cli/commands/vf.py
+++ b/src/cli/commands/vf.py
@@ -206,8 +206,8 @@ class SppVf(object):
 
         To update status of the instance of SppVf, get the status from
         spp-ctl. This method returns the result as a dict. For considering
-        behaviour of spp_vf, it is enough to return worker's name and core
-        IDs as the status, but might need to be update for future updates.
+        behaviour of spp_vf, it is enough to return worker's name, core IDs and
+        ports as the status, but might need to be update for future updates.
 
         # return worker's name and used core IDs, and all of core IDs.
         {
@@ -216,12 +216,13 @@ class SppVf(object):
             {'name': 'mg1', 'core_id': 6},
             ...
           ],
-          'core_ids': [5, 6, 7, ...]
+          'core_ids': [5, 6, 7, ...],
+          'ports': ['phy:0', 'phy:1', ...]
         }
 
         """
 
-        status = {'workers': [], 'core_ids': []}
+        status = {'workers': [], 'core_ids': [], 'ports': []}
         res = self.spp_ctl_cli.get('vfs/%d' % self.sec_id)
         if res is not None:
             if res.status_code == 200:
@@ -236,6 +237,9 @@ class SppVf(object):
                                             'core_id': wk['core']})
                             status['core_ids'].append(wk['core'])
 
+                if 'ports' in json_obj:
+                    status['ports'] = json_obj.get('ports')
+
         return status
 
     def _run_status(self):
@@ -290,82 +294,136 @@ class SppVf(object):
                     print('Error: unknown response.')
 
     def _run_port(self, params):
-        req_params = None
-        if len(params) == 4:
-            if params[0] == 'add':
-                action = 'attach'
-            elif params[0] == 'del':
-                action = 'detach'
-            else:
-                print('Error: Invalid action.')
-                return None
-
-            req_params = {'action': action, 'port': params[1],
-                          'dir': params[2],
-                          'vlan': {'operation': 'none',
-                                   'id': 'none',
-                                   'pcp': 'none'}}
-
-        elif len(params) == 5:  # delete vlan with 'port add' command
-            # TODO(yasufum) Syntax for deleting vlan should be modified
-            #               because deleting with 'port add' is terrible!
-            action = 'attach'
-            req_params = {'action': action, 'port': params[1],
-                          'dir': params[2],
-                          'vlan': {'operation': 'del',
-                                   'id': 'none',
-                                   'pcp': 'none'}}
-
-        elif len(params) == 7:
-            action = 'attach'
-            if params[4] == 'add_vlantag':
-                op = 'add'
-            elif params[4] == 'del_vlantag':
-                op = 'del'
-            req_params = {'action': action, 'port': params[1],
-                          'dir': params[2],
-                          'vlan': {'operation': op, 'id': int(params[5]),
-                                   'pcp': int(params[6])}}
-        else:
-            print('Error: Invalid syntax.')
-
-        if req_params is not None:
-            res = self.spp_ctl_cli.put('vfs/%d/components/%s/ports'
-                                       % (self.sec_id, params[3]), req_params)
-            if res is not None:
-                error_codes = self.spp_ctl_cli.rest_common_error_codes
-                if res.status_code == 204:
-                    print("Succeeded to %s port" % params[0])
-                elif res.status_code in error_codes:
-                    pass
+        params_index = 0
+        req_params = {}
+        vlan_params = {}
+        name = None
+        flg_mq = False
+
+        while params_index < len(params):
+            if params_index == 0:
+                if params[params_index] == 'add':
+                    req_params["action"] = 'attach'
+                elif params[params_index] == 'del':
+                    req_params["action"] = 'detach'
                 else:
-                    print('Error: unknown response.')
+                    print('Error: Invalid action.')
+                    return None
+
+            elif params_index == 1:
+                req_params["port"] = params[params_index]
+
+                # Check Multi queue
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["port"] += "nq" + params[params_index]
+                        flg_mq = True
+                else:
+                    print("Error: Not enough parameters.")
+                    return None
+
+            elif ((params_index == 2 and flg_mq is False) or
+                    (params_index == 4 and flg_mq is True)):
+                req_params["dir"] = params[params_index]
+
+            elif ((params_index == 3 and flg_mq is False) or
+                    (params_index == 5 and flg_mq is True)):
+                name = params[params_index]
+
+            elif ((params_index == 4 and flg_mq is False) or
+                    (params_index == 6 and flg_mq is True)):
+                if params[params_index] == "add_vlantag":
+                    vlan_params["operation"] = "add"
+                elif params[params_index] == "del_vlantag":
+                    vlan_params["operation"] = "del"
+                else:
+                    print('Error: vlantag is Only add_vlantag or del_vlantag.')
+                    return None
+
+            elif ((params_index == 5 and flg_mq is False) or
+                    (params_index == 7 and flg_mq is True)):
+                try:
+                    vlan_params["id"] = int(params[params_index])
+                except Exception as _:
+                    print('Error: vid is not a number.')
+                    return None
+
+            elif ((params_index == 6 and flg_mq is False) or
+                    (params_index == 8 and flg_mq is True)):
+                try:
+                    vlan_params["pcp"] = int(params[params_index])
+                except Exception as _:
+                    print('Error: pcp is not a number.')
+                    return None
+
+            params_index += 1
+
+        req_params["vlan"] = vlan_params
+        res = self.spp_ctl_cli.put('vfs/%d/components/%s/ports'
+                                   % (self.sec_id, name), req_params)
+        if res is not None:
+            error_codes = self.spp_ctl_cli.rest_common_error_codes
+            if res.status_code == 204:
+                print("Succeeded to %s port" % params[0])
+            elif res.status_code in error_codes:
+                pass
+            else:
+                print('Error: unknown response.')
 
     def _run_cls_table(self, params):
-        req_params = None
-        if len(params) == 4:
-            req_params = {'action': params[0], 'type': params[1],
-                          'mac_address': params[2], 'port': params[3]}
-
-        elif len(params) == 5:
-            req_params = {'action': params[0], 'type': params[1],
-                          'vlan': params[2], 'mac_address': params[3],
-                          'port': params[4]}
-        else:
-            print('Error: Invalid syntax.')
+        params_index = 0
+        req_params = {}
+        flg_vlan = False
 
-        if req_params is not None:
-            req = 'vfs/%d/classifier_table' % self.sec_id
-            res = self.spp_ctl_cli.put(req, req_params)
+        # The list elements are:
+        #       action, type, vlan, mac,  port
+        values = [None, None, None, None, None]
+        values_index = 0
 
-            if res is not None:
-                error_codes = self.spp_ctl_cli.rest_common_error_codes
-                if res.status_code == 204:
-                    print("Succeeded to %s" % params[0])
-                elif res.status_code in error_codes:
-                    pass
-                else:
-                    print('Error: unknown response.')
+        while params_index < len(params):
+
+            if params_index == 0:
+                req_params["action"] = params[params_index]
+
+            elif params_index == 1:
+                req_params["type"] = params[params_index]
+                if (req_params["type"] != "vlan" and
+                        req_params["type"] != "mac"):
+                    print("Error: Type is only vlan or mac")
+                    return None
+
+            elif params_index == 2 and req_params["type"] == "vlan":
+                req_params["vlan"] = params[params_index]
+                flg_vlan = True
+
+            elif ((params_index == 2 and flg_vlan is False) or
+                    (params_index == 3 and flg_vlan is True)):
+                req_params["mac_address"] = params[params_index]
+
+            elif ((params_index == 3 and flg_vlan is False) or
+                    (params_index == 4 and flg_vlan is True)):
+                req_params["port"] = params[params_index]
+
+                # Check Multi queue
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["port"] += "nq" + params[params_index]
+
+            params_index += 1
+
+        req = 'vfs/%d/classifier_table' % self.sec_id
+        res = self.spp_ctl_cli.put(req, req_params)
+
+        if res is not None:
+            error_codes = self.spp_ctl_cli.rest_common_error_codes
+            if res.status_code == 204:
+                print("Succeeded to %s" % params[0])
+            elif res.status_code in error_codes:
+                pass
+            else:
+                print('Error: unknown response.')
 
     def _run_exit(self):
         """Run `exit` command."""
@@ -411,96 +469,180 @@ class SppVf(object):
             return res
 
     def _compl_port(self, sub_tokens):
-        if len(sub_tokens) < 9:
-            subsub_cmds = ['add', 'del']
-            res = []
-            if len(sub_tokens) == 2:
-                for kw in subsub_cmds:
-                    if kw.startswith(sub_tokens[1]):
-                        res.append(kw)
-            elif len(sub_tokens) == 3:
-                if sub_tokens[1] in subsub_cmds:
-                    if 'RES_UID'.startswith(sub_tokens[2]):
-                        res.append('RES_UID')
-            elif len(sub_tokens) == 4:
-                if sub_tokens[1] in subsub_cmds:
-                    for direction in ['rx', 'tx']:
-                        if direction.startswith(sub_tokens[3]):
-                            res.append(direction)
-            elif len(sub_tokens) == 5:
-                if sub_tokens[1] in subsub_cmds:
-                    for kw in self.worker_names:
-                        if kw.startswith(sub_tokens[4]):
-                            res.append(kw)
-            elif len(sub_tokens) == 6:
-                if sub_tokens[1] == 'add':
-                    for kw in ['add_vlantag', 'del_vlantag']:
-                        if kw.startswith(sub_tokens[5]):
-                            res.append(kw)
-            elif len(sub_tokens) == 7:
-                if sub_tokens[1] == 'add' and sub_tokens[5] == 'add_vlantag':
-                    if 'VID'.startswith(sub_tokens[6]):
-                        res.append('VID')
-            elif len(sub_tokens) == 8:
-                if sub_tokens[1] == 'add' and sub_tokens[5] == 'add_vlantag':
-                    if 'PCP'.startswith(sub_tokens[7]):
-                        res.append('PCP')
-            return res
+        res = []
+        index = 0
+
+        # compl_phase "add_del"  : candidate is add or del
+        # compl_phase "res_uid"  : candidate is RES_UID
+        # compl_phase "nq"       : candidate is nq
+        # compl_phase "queue_no" : candidate is queue no
+        # compl_phase "dir"      : candidate is DIR
+        # compl_phase "name"     : candidate is NAME
+        # compl_phase "vlan_tag" : candidate is vlan tag
+        # compl_phase "vid"      : candidate is vid
+        # compl_phase "pcp"      : candidate is pcp
+        # compl_phase None       : candidate is None
+        compl_phase = "add_del"
+        add_or_del = None
+
+        while index < len(sub_tokens):
+            if compl_phase == "nq":
+                queue_no_list = self._get_candidate_phy_queue_no(
+                    sub_tokens[index - 1])
+
+                if queue_no_list is None:
+                    compl_phase = "dir"
+
+            if ((compl_phase == "add_del") and
+                    (sub_tokens[index - 1] == "port")):
+                res = ["add", "del"]
+                compl_phase = "res_uid"
+
+            elif ((compl_phase == "res_uid") and
+                    (sub_tokens[index - 1] == "add" or
+                     sub_tokens[index - 1] == "del")):
+                res = ["RES_UID"]
+                compl_phase = "nq"
+                add_or_del = sub_tokens[index - 1]
+
+            elif compl_phase == "nq":
+                res = ["nq"]
+                compl_phase = "queue_no"
+
+            elif compl_phase == "queue_no":
+                res = queue_no_list
+                compl_phase = "dir"
+
+            elif compl_phase == "dir":
+                res = ["rx", "tx"]
+                compl_phase = "name"
+
+            elif compl_phase == "name":
+                res = ["NAME"]
+                if add_or_del == "add":
+                    compl_phase = "vlan_tag"
+                else:
+                    compl_phase = None
+
+            elif compl_phase == "vlan_tag":
+                res = ["add_vlantag", "del_vlantag"]
+                compl_phase = "vid"
+
+            elif (compl_phase == "vid" and
+                  sub_tokens[index - 1] == "add_vlantag"):
+                res = ["VID"]
+                compl_phase = "pcp"
+
+            elif compl_phase == "pcp":
+                res = ["PCP"]
+                compl_phase = None
+
+            else:
+                res = []
+
+            index += 1
+
+        res = [p for p in res
+               if p.startswith(sub_tokens[len(sub_tokens) - 1])]
+
+        return res
+
+    def _get_candidate_phy_queue_no(self, res_uid):
+        """Get phy queue_no candidate.
+        If res_uid is phy and multi-queue, return queue_no in list type.
+        Otherwise returns None.
+        """
+
+        try:
+            port, _ = res_uid.split(":")
+        except Exception as _:
+            return None
+
+        if port != "phy":
+            return None
+
+        status = self._get_status(self.sec_id)
+
+        queue_no_list = []
+        for port in status["ports"]:
+            if not port.startswith(res_uid):
+                continue
+
+            try:
+                _, queue_no = port.split("nq")
+            except Exception as _:
+                continue
+
+            queue_no_list.append(queue_no.strip(" "))
+
+        if len(queue_no_list) == 0:
+            return None
+
+        return queue_no_list
 
     def _compl_cls_table(self, sub_tokens):
-        if len(sub_tokens) < 7:
-            subsub_cmds = ['add', 'del']
-            res = []
+        res = []
+        index = 0
+
+        # compl_phase "add_del"  : candidate is add or del
+        # compl_phase "vlan_mac" : candidate is vlan or mac
+        # compl_phase "vid"      : candidate is VID
+        # compl_phase "mac_addr" : candidate is MAC_ADDR or default
+        # compl_phase "res_uid"  : candidate is RES_UID
+        # compl_phase "nq"       : candidate is nq
+        # compl_phase "queue_no" : candidate is queue_no
+        # compl_phase None : candidate is None
+        compl_phase = "add_del"
+
+        while index < len(sub_tokens):
+
+            if compl_phase == "vid" and sub_tokens[index - 1] == "mac":
+                compl_phase = "mac_addr"
+
+            if compl_phase == "nq":
+                queue_no_list = self._get_candidate_phy_queue_no(
+                    sub_tokens[index - 1])
+                if queue_no_list is None:
+                    compl_phase = None
+
+            if ((compl_phase == "add_del") and
+                    (sub_tokens[index - 1] == "classifier_table")):
+                res = ["add", "del"]
+                compl_phase = "vlan_mac"
+
+            elif compl_phase == "vlan_mac":
+                res = ["vlan", "mac"]
+                compl_phase = "vid"
+
+            elif compl_phase == "vid" and sub_tokens[index - 1] == "vlan":
+                res = ["VID"]
+                compl_phase = "mac_addr"
+
+            elif compl_phase == "mac_addr":
+                res = ["MAC_ADDR", "default"]
+                compl_phase = "res_uid"
+
+            elif compl_phase == "res_uid":
+                res = ["RES_UID"]
+                compl_phase = "nq"
+
+            elif compl_phase == "nq":
+                res = ["nq"]
+                compl_phase = "queue_no"
+
+            elif compl_phase == "queue_no":
+                res = queue_no_list
+                compl_phase = None
 
-            if len(sub_tokens) == 2:
-                for kw in subsub_cmds:
-                    if kw.startswith(sub_tokens[1]):
-                        res.append(kw)
+            else:
+                res = []
 
-            elif len(sub_tokens) == 3:
-                if sub_tokens[1] in subsub_cmds:
-                    for kw in ['mac', 'vlan']:
-                        if kw.startswith(sub_tokens[2]):
-                            res.append(kw)
+            index += 1
 
-            elif len(sub_tokens) == 4:
-                if sub_tokens[1] == 'add':
-                    if sub_tokens[2] == 'mac':
-                        if 'MAC_ADDR'.startswith(sub_tokens[3]):
-                            res.append('MAC_ADDR')
-                    elif sub_tokens[2] == 'vlan':
-                        if 'VID'.startswith('VID'):
-                            res.append('VID')
-                elif sub_tokens[1] == 'del':
-                    if sub_tokens[2] == 'mac':
-                        if 'MAC_ADDR'.startswith(sub_tokens[3]):
-                            res.append('MAC_ADDR')
-                    if sub_tokens[2] == 'vlan':
-                        if 'VID'.startswith(sub_tokens[3]):
-                                res.append('VID')
+        res = [p for p in res
+               if p.startswith(sub_tokens[len(sub_tokens) - 1])]
 
-            elif len(sub_tokens) == 5:
-                if sub_tokens[1] == 'add':
-                    if sub_tokens[2] == 'mac':
-                        if 'RES_UID'.startswith(sub_tokens[4]):
-                            res.append('RES_UID')
-                    elif sub_tokens[2] == 'vlan':
-                        if 'MAC_ADDR'.startswith(sub_tokens[4]):
-                            res.append('MAC_ADDR')
-                if sub_tokens[1] == 'del':
-                    if sub_tokens[2] == 'mac':
-                        if 'RES_UID'.startswith(sub_tokens[4]):
-                            res.append('RES_UID')
-                    elif sub_tokens[2] == 'vlan':
-                        if 'MAC_ADDR'.startswith(sub_tokens[4]):
-                            res.append('MAC_ADDR')
-
-            elif len(sub_tokens) == 6:
-                if sub_tokens[1] in subsub_cmds and \
-                        sub_tokens[2] == 'vlan':
-                            if 'RES_UID'.startswith(sub_tokens[5]):
-                                res.append('RES_UID')
-            return res
+        return res
 
     @classmethod
     def help(cls):
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 14/17] cli: add support of rte_flow in mirror
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (12 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 13/17] cli: add support of rte_flow in vf x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 15/17] cli: add support of rte_flow in nfv x-fn-spp-ml
                   ` (20 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch implements support of multi-queue in mirror command.
  - mirror; status
  - mirror; port

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/cli/commands/mirror.py | 175 +++++++++++++++++++++++++++++--------
 1 file changed, 137 insertions(+), 38 deletions(-)

diff --git a/src/cli/commands/mirror.py b/src/cli/commands/mirror.py
index d7ea4cd..10c29f9 100644
--- a/src/cli/commands/mirror.py
+++ b/src/cli/commands/mirror.py
@@ -56,7 +56,8 @@ class SppMirror(object):
             self.unused_core_ids = core_ids  # used while completion to exclude
 
             self.workers = mirror_status['workers']
-            self.worker_names = [attr['name'] for attr in mirror_status['workers']]
+            self.worker_names = [attr['name']
+                                 for attr in mirror_status['workers']]
 
         cmd = cmdline.split(' ')[0]
         params = cmdline.split(' ')[1:]
@@ -184,8 +185,9 @@ class SppMirror(object):
 
         To update status of the instance of SppMirror, get the status from
         spp-ctl. This method returns the result as a dict. For considering
-        behaviour of spp_mirror, it is enough to return worker's name and core
-        IDs as the status, but might need to be update for future updates.
+        behaviour of spp_mirror, it is enough to return worker's name, core
+        IDs and ports as the status, but might need to be update for future
+        updates.
 
         # return worker's name and used core IDs, and all of core IDs.
         {
@@ -194,12 +196,13 @@ class SppMirror(object):
             {'name': 'mr2', 'core_id': 6},
             ...
           ],
-          'core_ids': [5, 6, 7, ...]
+          'core_ids': [5, 6, 7, ...],
+          'ports': ['phy:0', 'phy:1', ...]
         }
 
         """
 
-        status = {'workers': [], 'core_ids': []}
+        status = {'workers': [], 'core_ids': [], 'ports': []}
         res = self.spp_ctl_cli.get('mirrors/%d' % self.sec_id)
         if res is not None:
             if res.status_code == 200:
@@ -214,8 +217,45 @@ class SppMirror(object):
                                             'core_id': wk['core']})
                             status['core_ids'].append(wk['core'])
 
+                if 'ports' in json_obj:
+                    status['ports'] = json_obj.get('ports')
+
         return status
 
+    def _get_candidate_phy_queue_no(self, res_uid):
+        """Get phy queue_no candidate.
+
+        If res_uid is phy and multi-queue, return queue_no in list type.
+        Otherwise returns None.
+        """
+
+        try:
+            port, _ = res_uid.split(":")
+        except Exception as _:
+            return None
+
+        if port != "phy":
+            return None
+
+        status = self._get_status(self.sec_id)
+
+        queue_no_list = []
+        for port in status["ports"]:
+            if not port.startswith(res_uid):
+                continue
+
+            try:
+                _, queue_no = port.split("nq")
+            except Exception as _:
+                continue
+
+            queue_no_list.append(queue_no.strip(" "))
+
+        if len(queue_no_list) == 0:
+            return None
+
+        return queue_no_list
+
     def _run_status(self):
         res = self.spp_ctl_cli.get('mirrors/%d' % self.sec_id)
         if res is not None:
@@ -267,20 +307,45 @@ class SppMirror(object):
                     print('Error: unknown response.')
 
     def _run_port(self, params):
-        if len(params) == 4:
-            if params[0] == 'add':
-                action = 'attach'
-            elif params[0] == 'del':
-                action = 'detach'
-            else:
-                print('Error: Invalid action.')
-                return None
+        params_index = 0
+        req_params = {}
+        name = None
+        flg_mq = False
+
+        while params_index < len(params):
+            if params_index == 0:
+                if params[params_index] == 'add':
+                    req_params["action"] = 'attach'
+                elif params[params_index] == 'del':
+                    req_params["action"] = 'detach'
+                else:
+                    print('Error: Invalid action.')
+                    return None
+
+            elif params_index == 1:
+                req_params["port"] = params[params_index]
+
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["port"] += "nq" + params[params_index]
+                        flg_mq = True
+                else:
+                    print("Error: Not enough parameters.")
+                    return None
+
+            elif ((params_index == 2 and flg_mq is False) or
+                    (params_index == 4 and flg_mq is True)):
+                req_params["dir"] = params[params_index]
 
-            req_params = {'action': action, 'port': params[1],
-                          'dir': params[2]}
+            elif ((params_index == 3 and flg_mq is False) or
+                    (params_index == 5 and flg_mq is True)):
+                name = params[params_index]
+
+            params_index += 1
 
         res = self.spp_ctl_cli.put('mirrors/%d/components/%s/ports'
-                                   % (self.sec_id, params[3]), req_params)
+                                   % (self.sec_id, name), req_params)
         if res is not None:
             error_codes = self.spp_ctl_cli.rest_common_error_codes
             if res.status_code == 204:
@@ -334,28 +399,62 @@ class SppMirror(object):
             return res
 
     def _compl_port(self, sub_tokens):
-        if len(sub_tokens) < 9:
-            subsub_cmds = ['add', 'del']
-            res = []
-            if len(sub_tokens) == 2:
-                for kw in subsub_cmds:
-                    if kw.startswith(sub_tokens[1]):
-                        res.append(kw)
-            elif len(sub_tokens) == 3:
-                if sub_tokens[1] in subsub_cmds:
-                    if 'RES_UID'.startswith(sub_tokens[2]):
-                        res.append('RES_UID')
-            elif len(sub_tokens) == 4:
-                if sub_tokens[1] in subsub_cmds:
-                    for direction in ['rx', 'tx']:
-                        if direction.startswith(sub_tokens[3]):
-                            res.append(direction)
-            elif len(sub_tokens) == 5:
-                if sub_tokens[1] in subsub_cmds:
-                    for kw in self.worker_names:
-                        if kw.startswith(sub_tokens[4]):
-                            res.append(kw)
-            return res
+        res = []
+        index = 0
+
+        # compl_phase "add_del" : candidate is add or del
+        # compl_phase "res_uid" : candidate is RES_UID
+        # compl_phase "nq"      : candidate is nq
+        # compl_phase "queue_no": candidate is queue no
+        # compl_phase "dir"     : candidate is DIR
+        # compl_phase "name"    : candidate is NAME
+        # compl_phase None      : candidate is None
+        compl_phase = "add_del"
+
+        while index < len(sub_tokens):
+            if compl_phase == "nq":
+                queue_no_list = self._get_candidate_phy_queue_no(
+                    sub_tokens[index - 1])
+
+                if queue_no_list is None:
+                    compl_phase = "dir"
+
+            if ((compl_phase == "add_del") and
+                    (sub_tokens[index - 1] == "port")):
+                res = ["add", "del"]
+                compl_phase = "res_uid"
+
+            elif ((compl_phase == "res_uid") and
+                    (sub_tokens[index - 1] == "add" or
+                     sub_tokens[index - 1] == "del")):
+                res = ["RES_UID"]
+                compl_phase = "nq"
+
+            elif compl_phase == "nq":
+                res = ["nq"]
+                compl_phase = "queue_no"
+
+            elif compl_phase == "queue_no":
+                res = queue_no_list
+                compl_phase = "dir"
+
+            elif compl_phase == "dir":
+                res = ["rx", "tx"]
+                compl_phase = "name"
+
+            elif compl_phase == "name":
+                res = ["NAME"]
+                compl_phase = None
+
+            else:
+                res = []
+
+            index += 1
+
+        res = [p for p in res
+               if p.startswith(sub_tokens[len(sub_tokens) - 1])]
+
+        return res
 
     @classmethod
     def help(cls):
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 15/17] cli: add support of rte_flow in nfv
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (13 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 14/17] cli: add support of rte_flow in mirror x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 16/17] spp-ctl: add APIs for flow rules x-fn-spp-ml
                   ` (19 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch implements support of multi-queue in nfv command.
  - nfv; status
  - nfv; port

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/cli/commands/nfv.py | 245 +++++++++++++++++++++++++++++-----------
 1 file changed, 179 insertions(+), 66 deletions(-)

diff --git a/src/cli/commands/nfv.py b/src/cli/commands/nfv.py
index 6349823..0feb4ac 100644
--- a/src/cli/commands/nfv.py
+++ b/src/cli/commands/nfv.py
@@ -260,49 +260,123 @@ class SppNfv(object):
     def _compl_patch(self, sub_tokens):
         """Complete `patch` command."""
 
-        # Patch command consists of three tokens max, for instance,
-        # `nfv 1; patch phy:0 ring:1`.
-        if len(sub_tokens) < 4:
-            res = []
-
-            if self.use_cache is False:
-                self.ports, self.patches = self._get_ports_and_patches()
+        res = []
+        candidates = []
+        # index 0 is "port", so from 1
+        index = 1
+
+        # compl_phase "src_res_uid" : candidate is src RES_UID or reset
+        # compl_phase "src_nq"      : candidate is nq
+        # compl_phase "src_q_no"    : candidate is queue no
+        # compl_phase "dst_res_uid" : candidate is dst RES_UID
+        # compl_phase "dst_nq"      : candidate is nq
+        # compl_phase "dst_q_no"    : candidate is queue no
+        # compl_phase None          : candidate is None
+        compl_phase = "src_res_uid"
+
+        if self.use_cache is False:
+            self.ports, self.patches = self._get_ports_and_patches()
+
+        # Get patched ports of src and dst to be used for completion.
+        src_ports = []
+        dst_ports = []
+        for pt in self.patches:
+            src_ports.append(pt['src'])
+            dst_ports.append(pt['dst'])
+
+        while index < len(sub_tokens):
+            if compl_phase == "src_nq" or compl_phase == "dst_nq":
+                if sub_tokens[index - 1] == "reset":
+                    candidates = []
+                    compl_phase = None
+                    continue
+
+                queue_no_list = []
+                for port in self.ports:
+                    split_port = port.split()
+                    if len(split_port) != 3:
+                        continue
+                    if sub_tokens[index - 1] != split_port[0]:
+                        continue
+                    queue_no_list.append(split_port[2])
+
+                if len(queue_no_list) == 0:
+                    if compl_phase == "src_nq":
+                        compl_phase = "dst_res_uid"
+                    elif compl_phase == "dst_nq":
+                        compl_phase = None
+
+            if compl_phase == "src_res_uid":
+                candidates = []
+                for port in self.ports:
+                    if port in src_ports:
+                        continue
+                    if port in candidates:
+                        continue
+                    candidates.append(port.split()[0])
+
+                # If some of ports are patched, `reset` should be included
+                if len(self.patches) != 0:
+                    candidates.append("reset")
+
+                compl_phase = "src_nq"
+
+            elif compl_phase == "src_nq":
+                candidates = ["nq"]
+                compl_phase = "src_q_no"
+
+            elif compl_phase == "src_q_no":
+                candidates = []
+                for queue_no in queue_no_list:
+                    res_uid = "{0} nq {1}".format(
+                        sub_tokens[index - 2], queue_no)
+                    if res_uid in src_ports:
+                        continue
+                    candidates.append(queue_no)
+                compl_phase = "dst_res_uid"
+
+            elif compl_phase == "dst_res_uid":
+                candidates = []
+                for port in self.ports:
+                    if port in dst_ports:
+                        continue
+                    if port in candidates:
+                        continue
+                    candidates.append(port.split()[0])
+
+                compl_phase = "dst_nq"
+
+            elif compl_phase == "dst_nq":
+                candidates = ["nq"]
+                compl_phase = "dst_q_no"
+
+            elif compl_phase == "dst_q_no":
+                candidates = []
+                for queue_no in queue_no_list:
+                    res_uid = "{0} nq {1}".format(
+                        sub_tokens[index - 2], queue_no)
+                    if res_uid in dst_ports:
+                        continue
+                    candidates.append(queue_no)
+                compl_phase = None
 
-            # Get patched ports of src and dst to be used for completion.
-            src_ports = []
-            dst_ports = []
-            for pt in self.patches:
-                src_ports.append(pt['src'])
-                dst_ports.append(pt['dst'])
-
-            # Remove patched ports from candidates.
-            target_idx = len(sub_tokens) - 1  # target is src or dst
-            tmp_ports = self.ports[:]  # candidates
-            if target_idx == 1:  # find src port
-                # If some of ports are patched, `reset` should be included.
-                if self.patches != []:
-                    tmp_ports.append('reset')
-                for pt in src_ports:
-                    tmp_ports.remove(pt)  # remove patched ports
-            else:  # find dst port
-                # If `reset` is given, no need to show dst ports.
-                if sub_tokens[target_idx - 1] == 'reset':
-                    tmp_ports = []
+            else:
+                candidates = []
+                compl_phase = None
+
+            index += 1
+
+        last_index = len(sub_tokens) - 1
+        for candidate in candidates:
+            if candidate.startswith(sub_tokens[last_index]):
+                # Completion does not work correctly if `:` is included in
+                # tokens. Required to create keyword only after `:`.
+                if ':' in sub_tokens[last_index]:  # 'ring:' or 'ring:0'
+                    res.append(candidate.split(':')[1])  # add only after `:`
                 else:
-                    for pt in dst_ports:
-                        tmp_ports.remove(pt)
-
-            # Return candidates.
-            for kw in tmp_ports:
-                if kw.startswith(sub_tokens[target_idx]):
-                    # Completion does not work correctly if `:` is included in
-                    # tokens. Required to create keyword only after `:`.
-                    if ':' in sub_tokens[target_idx]:  # 'ring:' or 'ring:0'
-                        res.append(kw.split(':')[1])  # add only after `:`
-                    else:
-                        res.append(kw)
+                    res.append(candidate)
 
-            return res
+        return res
 
     def _run_status(self):
         """Run `status` command."""
@@ -407,34 +481,73 @@ class SppNfv(object):
     def _run_patch(self, params):
         """Run `patch` command."""
 
+        params_index = 0
+        req_params = {}
+        flg_reset = False
+        flg_mq = False
+
         if len(params) == 0:
-            print('Params are required!')
-        elif params[0] == 'reset':
-            res = self.spp_ctl_cli.delete('nfvs/%d/patches' % self.sec_id)
-            if res is not None:
-                error_codes = self.spp_ctl_cli.rest_common_error_codes
-                if res.status_code == 204:
-                    print('Clear all of patches.')
-                elif res.status_code in error_codes:
-                    pass
-                else:
-                    print('Error: unknown response.')
+            print('Error: Params are required!')
+            return
+
+        while params_index < len(params):
+            if params_index == 0 and params[0] == "reset":
+                flg_reset = True
+                break
+
+            elif params_index == 0:
+                req_params["src"] = params[params_index]
+
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["src"] += "nq" + params[params_index]
+                        flg_mq = True
+
+            elif ((params_index == 1 and flg_mq is False) or
+                    (params_index == 3 and flg_mq is True)):
+                req_params["dst"] = params[params_index]
+
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["dst"] += "nq" + params[params_index]
+
+            params_index += 1
+
+        if flg_reset is False:
+            if "src" not in req_params:
+                print("Error: Src port is required!")
+                return
+
+            if "dst" not in req_params:
+                print("Error: Dst port is required!")
+                return
+
+        url = "nfvs/{0}/patches".format(self.sec_id)
+        if flg_reset:
+            res = self.spp_ctl_cli.delete(url)
         else:
-            if len(params) < 2:
-                print('Dst port is required!')
-            else:
-                req_params = {'src': params[0], 'dst': params[1]}
-                res = self.spp_ctl_cli.put(
-                        'nfvs/%d/patches' % self.sec_id, req_params)
-                if res is not None:
-                    error_codes = self.spp_ctl_cli.rest_common_error_codes
-                    if res.status_code == 204:
-                        print('Patch ports (%s -> %s).' % (
-                            params[0], params[1]))
-                    elif res.status_code in error_codes:
-                        pass
-                    else:
-                        print('Error: unknown response.')
+            res = self.spp_ctl_cli.put(url, req_params)
+
+        if res is None:
+            return
+
+        error_codes = self.spp_ctl_cli.rest_common_error_codes
+        if res.status_code in error_codes:
+            pass
+        elif res.status_code != 204:
+            print('Error: unknown response.')
+            return
+
+        if flg_reset:
+            print("Clear all of patches.")
+        else:
+            src = (req_params["src"].replace("nq", " nq ")
+                   if "nq" in req_params["src"] else req_params["src"])
+            dst = (req_params["dst"].replace("nq", " nq ")
+                   if "nq" in req_params["dst"] else req_params["dst"])
+            print("Patch ports ({0} -> {1}).".format(src, dst))
 
     def _run_exit(self):
         """Run `exit` command."""
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 16/17] spp-ctl: add APIs for flow rules
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (14 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 15/17] cli: add support of rte_flow in nfv x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-18  6:37 ` [spp] [PATCH 17/17] spp_nfv: add support of multi-queue x-fn-spp-ml
                   ` (18 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch implements support of flow APIs in spp-ctl.
  - flow validate
  - flow create
  - flow delete
  - flow flush

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/spp-ctl/spp_proc.py   |   4 +
 src/spp-ctl/spp_webapi.py | 200 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 194 insertions(+), 10 deletions(-)

diff --git a/src/spp-ctl/spp_proc.py b/src/spp-ctl/spp_proc.py
index 35919fb..d584f7f 100644
--- a/src/spp-ctl/spp_proc.py
+++ b/src/spp-ctl/spp_proc.py
@@ -317,6 +317,10 @@ class PrimaryProc(SppProc):
     def stop(self):
         return "stop"
 
+    @exec_command
+    def flow(self, command):
+        return command
+
     @exec_command
     def do_launch_sec_proc(self, args):
         proc_name = args['proc_name']
diff --git a/src/spp-ctl/spp_webapi.py b/src/spp-ctl/spp_webapi.py
index 31befe2..d8e6e73 100644
--- a/src/spp-ctl/spp_webapi.py
+++ b/src/spp-ctl/spp_webapi.py
@@ -34,6 +34,20 @@ class KeyInvalid(bottle.HTTPError):
         super(KeyRequired, self).__init__(400, msg)
 
 
+class RequestJSONDecodeHTTPError(bottle.HTTPError):
+
+    def __init__(self):
+        msg = "Not in json format."
+        super().__init__(400, msg)
+
+
+class ResponseJSONDecodeHTTPError(bottle.HTTPError):
+
+    def __init__(self):
+        msg = "Internal Server Error"
+        super().__init__(500, msg)
+
+
 class BaseHandler(bottle.Bottle):
     """Define common methods for each handler."""
 
@@ -55,7 +69,12 @@ class BaseHandler(bottle.Bottle):
             if_type, if_num = port.split(":")
             if if_type not in PORT_TYPES:
                 raise
-            int(if_num)
+            if if_type == "phy" and "nq" in if_num:
+                port_num, queue_num = if_num.split("nq")
+                int(port_num)
+                int(queue_num)
+            else:
+                int(if_num)
         except Exception:
             raise KeyInvalid('port', port)
 
@@ -75,12 +94,15 @@ class BaseHandler(bottle.Bottle):
         def wrapper(*args, **kwargs):
             req = bottle.request
             if req.method in ["POST", "PUT"]:
-                if req.get_header('Content-Type') == "application/json":
-                    body = req.json
-                else:
-                    body = json.loads(req.body.read().decode())
+                try:
+                    if req.get_header('Content-Type') == "application/json":
+                        body = req.json
+                    else:
+                        body = json.loads(req.body.read().decode())
+                    LOG.info("body: %s", body)
+                except Exception:
+                    raise RequestJSONDecodeHTTPError()
                 kwargs['body'] = body
-                LOG.info("body: %s", body)
             return func(*args, **kwargs)
         return wrapper
 
@@ -103,10 +125,15 @@ class BaseHandler(bottle.Bottle):
             ret = func(*args, **kwargs)
             if ret is None:
                 return bottle.HTTPResponse(status=204)
-            else:
-                r = bottle.HTTPResponse(status=200, body=json.dumps(ret))
-                r.content_type = "application/json"
-                return r
+
+            try:
+                body = json.dumps(ret)
+            except Exception:
+                raise ResponseJSONDecodeHTTPError()
+
+            r = bottle.HTTPResponse(status=200, body=body)
+            r.content_type = "application/json"
+            return r
         return wrapper
 
 
@@ -446,12 +473,16 @@ class V1PrimaryHandler(BaseHandler):
 
     def __init__(self, controller):
         super(V1PrimaryHandler, self).__init__(controller)
+        self._initialize()
 
+    def _initialize(self):
         self.set_route()
 
         self.install(self.make_response)
         self.install(self.get_body)
 
+        self.mount("/flow_rules", V1PrimaryFlowHandler(self.ctrl))
+
     def set_route(self):
         self.route('/status', 'GET', callback=self.get_status)
         self.route('/status', 'DELETE', callback=self.clear_status)
@@ -556,6 +587,155 @@ class V1PrimaryHandler(BaseHandler):
         proc.do_exit()
 
 
+class V1PrimaryFlowHandler(V1PrimaryHandler):
+
+    def __init__(self, controller):
+        super().__init__(controller)
+
+    def _initialize(self):
+        self.set_route()
+
+        self.install(self.make_response)
+        self.install(self.get_body)
+
+    def set_route(self):
+        self.route('/port_id/<port_id:int>/validate',
+                   'POST', callback=self.post_flow_validate)
+        self.route('/port_id/<port_id:int>',
+                   'POST', callback=self.post_flow_create)
+        self.route('/port_id/<port_id:int>',
+                   'DELETE', callback=self.delete_flow_all_destroy)
+        self.route('/<rule_id:int>/port_id/<port_id:int>',
+                   'DELETE', callback=self.delete_flow_destroy)
+
+    def post_flow_validate(self, port_id, body):
+        self._check_request_body(body)
+        command = self._create_flow_rule_command(
+            port_id, body.get("rule"), "validate")
+
+        proc = self._get_proc()
+        return proc.flow(command)
+
+    def post_flow_create(self, port_id, body):
+        self._check_request_body(body)
+        command = self._create_flow_rule_command(
+            port_id, body.get("rule"), "create")
+
+        proc = self._get_proc()
+        return proc.flow(command)
+
+    def delete_flow_all_destroy(self, port_id):
+        command = self._gen_flow_destroy(port_id)
+
+        proc = self._get_proc()
+        return proc.flow(command)
+
+    def delete_flow_destroy(self, rule_id, port_id):
+        command = self._gen_flow_destroy(port_id, rule_id)
+
+        proc = self._get_proc()
+        return proc.flow(command)
+
+    def _create_flow_rule_command(self, port_id, rule, sub_command):
+        attr_data = {}
+        data = {}
+
+        # `group`,` priority`, and `transfer` in` attrs` are optional and
+        # may be omitted
+        attr_command = "{group}{priority}{transfer}{direction}"
+
+        attr_data["direction"] = rule.get("direction")
+
+        if "group" in rule:
+            attr_data["group"] = "group {0} ".format(rule.get("group"))
+        else:
+            attr_data["group"] = ""
+
+        if "priority" in rule:
+            attr_data["priority"] = "priority {0} ".format(
+                rule.get("priority"))
+        else:
+            attr_data["priority"] = ""
+
+        if "transfer" in rule:
+            attr_data["transfer"] = "transfer " if rule.get("transfer") else ""
+        else:
+            attr_data["transfer"] = ""
+
+        attrs = attr_command.format(**attr_data)
+
+        command = "flow {sub_command} {res_uid} {attrs} "
+        command += "pattern {pattern} / end "
+        command += "actions {actions} / end"
+
+        data["sub_command"] = sub_command
+        data["res_uid"] = "phy:{0}".format(port_id)
+        data["attrs"] = attrs
+        data["pattern"] = " / ".join(rule.get("pattern"))
+        data["actions"] = " / ".join(rule.get("actions"))
+
+        return command.format(**data)
+
+    def _gen_flow_destroy(self, port_id, rule_id=None):
+        """Delete a flow of given rule ID, or all flows if the ID is None."""
+        if rule_id is not None:
+            target = int(rule_id)
+        else:
+            target = "ALL"
+        return "flow destroy phy:{0} {1}".format(port_id, target)
+
+    def _check_request_body(self, body):
+        self._check_request_body_required_param(body, "rule", dict)
+        rule = body.get("rule")
+
+        self._check_request_body_optional_param(rule, "group", int)
+        self._check_request_body_optional_param(rule, "priority", int)
+        self._check_request_body_required_param(rule, "direction", str)
+        self._check_request_body_optional_param(rule, "transfer", bool)
+        self._check_request_body_required_param(rule, "pattern", list)
+        self._check_request_body_required_param(rule, "actions", list)
+
+        dir = rule.get("direction")
+        if dir != "ingress" and dir != "egress":
+            raise KeyInvalid("direction", dir)
+
+        pattern = rule.get("pattern")
+        for obj in pattern:
+            if obj is None or type(obj) != str:
+                raise KeyInvalid("pattern", pattern)
+
+        actions = rule.get("actions")
+        for obj in actions:
+            if obj is None or type(obj) != str:
+                raise KeyInvalid("actions", actions)
+
+    def _check_request_body_optional_param(self, target, key_name, obj_type):
+        """Check for optional parameter.
+
+        If key_name exists in dict, checking obj_type.
+        If invalid, raise error class. Return True if valid.
+        """
+        if key_name not in target:
+            return True
+        return self._check_request_body_required_param(
+            target, key_name, obj_type)
+
+    def _check_request_body_required_param(self, target, key_name, obj_type):
+        """Check for required parameter.
+
+        key_name must be present and check obj_type.
+        If invalid, raise error class. Return True if valid.
+        """
+        if key_name not in target:
+            raise KeyRequired(key_name)
+
+        obj = target.get(key_name)
+        if obj is None or type(obj) != obj_type:
+            raise KeyInvalid(key_name, obj)
+
+        return True
+
+
 class V1PcapHandler(BaseHandler):
 
     def __init__(self, controller):
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH 17/17] spp_nfv: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (15 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 16/17] spp-ctl: add APIs for flow rules x-fn-spp-ml
@ 2020-02-18  6:37 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (17 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-18  6:37 UTC (permalink / raw)
  To: ferruh.yigit, yasufum.o; +Cc: spp

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To achieve hardware offload in secondary process, multi-queue should be
supported.
Multi-queue support is divided into several changes like following:
- add queue number parameter in patch command
- parse newly added parameter
- add queue number parameter in status response

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/nfv/commands.h   |  71 +++++++---
 src/nfv/main.c       |  18 ++-
 src/nfv/nfv_status.c | 315 +++++++++++++++++++------------------------
 src/nfv/nfv_status.h |  12 +-
 4 files changed, 207 insertions(+), 209 deletions(-)

diff --git a/src/nfv/commands.h b/src/nfv/commands.h
index e5b25be..6b3a935 100644
--- a/src/nfv/commands.h
+++ b/src/nfv/commands.h
@@ -17,7 +17,7 @@
  */
 
 static int
-do_del(char *p_type, int p_id)
+do_del(char *p_type, int p_id, uint16_t queue_id)
 {
 	uint16_t port_id = PORT_RESET;
 
@@ -55,7 +55,7 @@ do_del(char *p_type, int p_id)
 
 	}
 
-	forward_array_remove(port_id);
+	forward_array_remove(port_id, queue_id);
 	port_map_init_one(port_id);
 
 	return 0;
@@ -66,7 +66,7 @@ do_del(char *p_type, int p_id)
  * combination of port type and ID like as 'ring:0'.
  */
 static int
-do_add(char *p_type, int p_id)
+do_add(char *p_type, int p_id, uint16_t queue_id)
 {
 	enum port_type type = UNDEF;
 	uint16_t port_id = PORT_RESET;
@@ -105,9 +105,11 @@ do_add(char *p_type, int p_id)
 	 * other than RING. There is no support to show/clear this stats
 	 * at the moment.
 	 */
+	port_map[port_id].queue_info = NULL;
 
-	/* Update ports_fwd_array with port id */
-	ports_fwd_array[port_id].in_port_id = port_id;
+	/* Update ports_fwd_array with port id and queue id */
+	ports_fwd_array[port_id][queue_id].in_port_id = port_id;
+	ports_fwd_array[port_id][queue_id].in_queue_id = queue_id;
 
 	return 0;
 }
@@ -164,6 +166,7 @@ parse_command(char *str)
 	char port_set[128] = { 0 };
 	char *p_type;
 	int p_id;
+	uint16_t queue_id;
 
 	if (!str)
 		return 0;
@@ -185,12 +188,10 @@ parse_command(char *str)
 		memset(str, '\0', MSG_SIZE);
 		if (cmd == FORWARD)
 			get_sec_stats_json(str, get_client_id(), "running",
-					lcore_id_used,
-					ports_fwd_array, port_map);
+					lcore_id_used);
 		else
 			get_sec_stats_json(str, get_client_id(), "idling",
-					lcore_id_used,
-					ports_fwd_array, port_map);
+					lcore_id_used);
 
 		RTE_ETH_FOREACH_DEV(dev_id) {
 			rte_eth_dev_get_name_by_port(dev_id, dev_name);
@@ -245,11 +246,12 @@ parse_command(char *str)
 	} else if (!strcmp(token_list[0], "add")) {
 		RTE_LOG(DEBUG, SPP_NFV, "Received add command\n");
 
-		ret = parse_resource_uid(token_list[1], &p_type, &p_id);
+		ret = parse_resource_uid(token_list[1], &p_type, &p_id,
+				&queue_id);
 		if (ret < 0)
 			return ret;
 
-		if (do_add(p_type, p_id) < 0) {
+		if (do_add(p_type, p_id, queue_id) < 0) {
 			RTE_LOG(ERR, SPP_NFV, "Failed to do_add()\n");
 			sprintf(result, "%s", "\"failed\"");
 		} else
@@ -282,13 +284,23 @@ parse_command(char *str)
 			char *out_p_type;
 			int in_p_id;
 			int out_p_id;
-
-			parse_resource_uid(token_list[1], &in_p_type, &in_p_id);
+			uint16_t in_queue_id, out_queue_id;
+			int res_uid_str_size = 32;
+			char in_res_uid[res_uid_str_size];
+			char out_res_uid[res_uid_str_size];
+
+			strncpy(in_res_uid, token_list[1],
+				res_uid_str_size - 1);
+			strncpy(out_res_uid, token_list[2],
+				res_uid_str_size - 1);
+
+			parse_resource_uid(token_list[1], &in_p_type, &in_p_id,
+					&in_queue_id);
 			in_port = find_port_id(in_p_id,
 					get_port_type(in_p_type));
 
 			parse_resource_uid(token_list[2],
-					&out_p_type, &out_p_id);
+					&out_p_type, &out_p_id, &out_queue_id);
 			out_port = find_port_id(out_p_id,
 					get_port_type(out_p_type));
 
@@ -314,13 +326,27 @@ parse_command(char *str)
 					"Patch not found, out_port",
 					out_p_type, out_p_id);
 				RTE_LOG(ERR, SPP_NFV, "%s\n", err_msg);
+			} else if (is_valid_port_rxq(in_port, in_queue_id)) {
+				RTE_LOG(ERR, SPP_NFV,
+					"Queue number of in_port"
+					" exceeds definition"
+					" %s:%d nq %d(%s:%d)\n",
+					in_p_type, in_p_id, in_queue_id,
+					__func__, __LINE__);
+			} else if (is_valid_port_txq(out_port, out_queue_id)) {
+				RTE_LOG(ERR, SPP_NFV,
+					"Queue number of out_port"
+					" exceeds definition"
+					" %s:%d nq %d(%s:%d)\n",
+					out_p_type, out_p_id, out_queue_id,
+					__func__, __LINE__);
 			}
 
-			if (add_patch(in_port, out_port) == 0) {
+			if (add_patch(in_port, in_queue_id, out_port,
+				out_queue_id) == 0) {
 				RTE_LOG(INFO, SPP_NFV,
-					"Patched '%s:%d' and '%s:%d'\n",
-					in_p_type, in_p_id,
-					out_p_type, out_p_id);
+					"Patched '%s' and '%s'\n",
+					in_res_uid, out_res_uid);
 				sprintf(result, "%s", "\"succeeded\"");
 			} else {
 				RTE_LOG(ERR, SPP_NFV, "Failed to patch\n");
@@ -328,8 +354,8 @@ parse_command(char *str)
 			}
 
 			sprintf(port_set,
-				"{\"src\":\"%s:%d\",\"dst\":\"%s:%d\"}",
-				in_p_type, in_p_id, out_p_type, out_p_id);
+				"{\"src\":\"%s\",\"dst\":\"%s\"}",
+				in_res_uid, out_res_uid);
 
 			memset(str, '\0', MSG_SIZE);
 			sprintf(str, "{%s:%s,%s:%s,%s:%s}",
@@ -345,11 +371,12 @@ parse_command(char *str)
 
 		cmd = STOP;
 
-		ret = parse_resource_uid(token_list[1], &p_type, &p_id);
+		ret = parse_resource_uid(token_list[1], &p_type, &p_id,
+				&queue_id);
 		if (ret < 0)
 			return ret;
 
-		if (do_del(p_type, p_id) < 0) {
+		if (do_del(p_type, p_id, queue_id) < 0) {
 			RTE_LOG(ERR, SPP_NFV, "Failed to do_del()\n");
 			sprintf(result, "%s", "\"failed\"");
 		} else
diff --git a/src/nfv/main.c b/src/nfv/main.c
index f2c6bfc..32f1074 100644
--- a/src/nfv/main.c
+++ b/src/nfv/main.c
@@ -145,13 +145,14 @@ main(int argc, char *argv[])
 	unsigned int nb_ports;
 	int connected = 0;
 	char str[MSG_SIZE] = { 0 };
-	unsigned int i;
+	unsigned int i, j;
 	int flg_exit;  // used as res of parse_command() to exit if -1
 	int ret;
 	char dev_name[RTE_DEV_NAME_MAX_LEN] = { 0 };
 	int port_type;
 	int nof_phy_port = 0;
 	char log_msg[1024] = { 0 };  /* temporary log message */
+	uint16_t max_queue;
 
 	ret = rte_eal_init(argc, argv);
 	if (ret < 0)
@@ -218,11 +219,22 @@ main(int argc, char *argv[])
 		 * not display to avoid confusion.
 		 */
 
-		/* Update ports_fwd_array with phy port. */
-		ports_fwd_array[i].in_port_id = i;
 		port_map[i].port_type = port_type;
 		port_map[i].id = port_id;
 		port_map[i].stats = &ports->port_stats[i];
+		port_map[i].queue_info = &ports->queue_info[i];
+
+		/* Update ports_fwd_array with phy port. */
+		if (port_map[i].queue_info->rxq >=
+			port_map[i].queue_info->txq)
+			max_queue = port_map[i].queue_info->rxq;
+		else
+			max_queue = port_map[i].queue_info->txq;
+
+		for (j = 0; j < max_queue; j++) {
+			ports_fwd_array[i][j].in_port_id = i;
+			ports_fwd_array[i][j].in_queue_id = j;
+		}
 
 		/* TODO(yasufum) convert from int of port_type to char */
 		RTE_LOG(DEBUG, SPP_NFV, "Add port, type: %d, id: %d\n",
diff --git a/src/nfv/nfv_status.c b/src/nfv/nfv_status.c
index 3947a84..d5d7f44 100644
--- a/src/nfv/nfv_status.c
+++ b/src/nfv/nfv_status.c
@@ -6,6 +6,8 @@
 
 #include <arpa/inet.h>
 #include "shared/common.h"
+#include "shared/basic_forwarder.h"
+#include "shared/port_manager.h"
 #include "nfv_status.h"
 
 /*
@@ -28,9 +30,7 @@
 void
 get_sec_stats_json(char *str, int cli_id,
 		const char *running_stat,
-		uint8_t lcore_id_used[RTE_MAX_LCORE],
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+		uint8_t lcore_id_used[RTE_MAX_LCORE])
 {
 	sprintf(str, "{\"client-id\":%d,", cli_id);
 
@@ -40,10 +40,10 @@ get_sec_stats_json(char *str, int cli_id,
 	append_lcore_info_json(str, lcore_id_used);
 	sprintf(str + strlen(str), ",");
 
-	append_port_info_json(str, ports_fwd_array, port_map);
+	append_port_info_json(str);
 	sprintf(str + strlen(str), ",");
 
-	append_patch_info_json(str, ports_fwd_array, port_map);
+	append_patch_info_json(str);
 	sprintf(str + strlen(str), "}");
 
 	/* Make sure to be terminated with null character. */
@@ -76,53 +76,63 @@ append_lcore_info_json(char *str,
  *     "ports": ["phy:0", "phy:1", "ring:0", "vhost:0"]
  */
 int
-append_port_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+append_port_info_json(char *str)
 {
-	unsigned int i;
+	unsigned int i, j;
 	unsigned int has_port = 0;  // for checking having port at last
+	uint16_t max_queue;
 
 	sprintf(str + strlen(str), "\"ports\":[");
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		max_queue = get_port_max_queues(i);
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
-			continue;
+		for (j = 0; j < max_queue; j++) {
+			if (ports_fwd_array[i][j].in_port_id == PORT_RESET)
+				continue;
 
-		has_port = 1;
-		switch (port_map[i].port_type) {
-		case PHY:
-			sprintf(str + strlen(str), "\"phy:%u\",",
-					port_map[i].id);
-			break;
-		case RING:
-			sprintf(str + strlen(str), "\"ring:%u\",",
-				port_map[i].id);
-			break;
-		case VHOST:
-			sprintf(str + strlen(str), "\"vhost:%u\",",
-				port_map[i].id);
-			break;
-		case PCAP:
-			sprintf(str + strlen(str), "\"pcap:%u\",",
-					port_map[i].id);
-			break;
-		case NULLPMD:
-			sprintf(str + strlen(str), "\"nullpmd:%u\",",
-					port_map[i].id);
-			break;
-		case TAP:
-			sprintf(str + strlen(str), "\"tap:%u\",",
+			has_port = 1;
+			switch (port_map[i].port_type) {
+			case PHY:
+				if (max_queue == 1)
+					sprintf(str + strlen(str),
+						"\"phy:%u\",", port_map[i].id);
+				else
+					sprintf(str + strlen(str),
+						"\"phy:%u nq %u\",",
+						port_map[i].id, j);
+				break;
+			case RING:
+				sprintf(str + strlen(str), "\"ring:%u\",",
 					port_map[i].id);
-			break;
-		case MEMIF:
-			sprintf(str + strlen(str), "\"memif:%u\",",
+				break;
+			case VHOST:
+				sprintf(str + strlen(str), "\"vhost:%u\",",
 					port_map[i].id);
-			break;
-		case UNDEF:
-			/* TODO(yasufum) Need to remove print for undefined ? */
-			sprintf(str + strlen(str), "\"udf\",");
-			break;
+				break;
+			case PCAP:
+				sprintf(str + strlen(str), "\"pcap:%u\",",
+						port_map[i].id);
+				break;
+			case NULLPMD:
+				sprintf(str + strlen(str), "\"nullpmd:%u\",",
+						port_map[i].id);
+				break;
+			case TAP:
+				sprintf(str + strlen(str), "\"tap:%u\",",
+						port_map[i].id);
+				break;
+			case MEMIF:
+				sprintf(str + strlen(str), "\"memif:%u\",",
+						port_map[i].id);
+				break;
+			case UNDEF:
+				/*
+				 * TODO(yasufum) Need to remove print for
+				 * undefined ?
+				 */
+				sprintf(str + strlen(str), "\"udf\",");
+				break;
+			}
 		}
 	}
 
@@ -136,6 +146,57 @@ append_port_info_json(char *str,
 	return 0;
 }
 
+static void
+append_port_string(char *str, enum port_type port_type,
+		uint16_t port_id, uint16_t queue_id, int max_queue)
+{
+	switch (port_type) {
+	case PHY:
+		RTE_LOG(INFO, SHARED, "Type: PHY\n");
+		if (max_queue > 1)
+			sprintf(str, "\"phy:%u nq %u\"", port_id, queue_id);
+		else
+			sprintf(str, "\"phy:%u\"", port_id);
+		break;
+
+	case RING:
+		RTE_LOG(INFO, SHARED, "Type: RING\n");
+		sprintf(str, "\"ring:%u\"", port_id);
+		break;
+
+	case VHOST:
+		RTE_LOG(INFO, SHARED, "Type: VHOST\n");
+		sprintf(str, "\"vhost:%u\"", port_id);
+		break;
+
+	case PCAP:
+		RTE_LOG(INFO, SHARED, "Type: PCAP\n");
+		sprintf(str, "\"pcap:%u\"", port_id);
+		break;
+
+	case NULLPMD:
+		RTE_LOG(INFO, SHARED, "Type: NULLPMD\n");
+		sprintf(str, "\"nullpmd:%u\"", port_id);
+		break;
+
+	case TAP:
+		RTE_LOG(INFO, SHARED, "Type: TAP\n");
+		sprintf(str, "\"tap:%u\"", port_id);
+		break;
+
+	case MEMIF:
+		RTE_LOG(INFO, SHARED, "Type: MEMIF\n");
+		sprintf(str, "\"memif:%u\"", port_id);
+		break;
+
+	case UNDEF:
+		RTE_LOG(INFO, SHARED, "Type: UDF\n");
+		/* TODO(yasufum) Need to remove print for undefined ? */
+		sprintf(str, "\"udf\"");
+		break;
+	}
+}
+
 /*
  * Append patch info to sec status. It is called from get_sec_stats_json()
  * to add a JSON formatted patch info to given 'str'. Here is an example.
@@ -146,152 +207,56 @@ append_port_info_json(char *str,
  *      ]
  */
 int
-append_patch_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+append_patch_info_json(char *str)
 {
-	unsigned int i;
+	unsigned int i, j;
 	unsigned int has_patch = 0;  // for checking having patch at last
+	unsigned int out_port_id;
+	uint16_t out_queue_id;
+	uint16_t in_max_queue, out_max_queue;
 
 	char patch_str[128];
 	sprintf(str + strlen(str), "\"patches\":[");
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		in_max_queue = get_port_max_queues(i);
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
-			continue;
-
-		RTE_LOG(INFO, SHARED, "Port ID %d\n", i);
-		RTE_LOG(INFO, SHARED, "Status %d\n",
-			ports_fwd_array[i].in_port_id);
+		for (j = 0; j < in_max_queue; j++) {
 
-		memset(patch_str, '\0', sizeof(patch_str));
+			if (ports_fwd_array[i][j].in_port_id == PORT_RESET ||
+				ports_fwd_array[i][j].out_port_id == PORT_RESET)
+				continue;
 
-		sprintf(patch_str, "{\"src\":");
-
-		switch (port_map[i].port_type) {
-		case PHY:
-			RTE_LOG(INFO, SHARED, "Type: PHY\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"phy:%u\",",
-					port_map[i].id);
-			break;
-		case RING:
-			RTE_LOG(INFO, SHARED, "Type: RING\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"ring:%u\",",
-					port_map[i].id);
-			break;
-		case VHOST:
-			RTE_LOG(INFO, SHARED, "Type: VHOST\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"vhost:%u\",",
-					port_map[i].id);
-			break;
-		case PCAP:
-			RTE_LOG(INFO, SHARED, "Type: PCAP\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"pcap:%u\",",
-					port_map[i].id);
-			break;
-		case NULLPMD:
-			RTE_LOG(INFO, SHARED, "Type: NULLPMD\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"nullpmd:%u\",",
-					port_map[i].id);
-			break;
-		case TAP:
-			RTE_LOG(INFO, SHARED, "Type: TAP\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"tap:%u\",",
-					port_map[i].id);
-			break;
-		case MEMIF:
-			RTE_LOG(INFO, SHARED, "Type: MEMIF\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"memif:%u\",",
-					port_map[i].id);
-			break;
-		case UNDEF:
-			RTE_LOG(INFO, SHARED, "Type: UDF\n");
-			/* TODO(yasufum) Need to remove print for undefined ? */
-			sprintf(patch_str + strlen(patch_str),
-					"\"udf\",");
-			break;
-		}
-
-		sprintf(patch_str + strlen(patch_str), "\"dst\":");
-
-		RTE_LOG(INFO, SHARED, "Out Port ID %d\n",
-				ports_fwd_array[i].out_port_id);
-
-		if (ports_fwd_array[i].out_port_id == PORT_RESET) {
-			//sprintf(patch_str + strlen(patch_str), "%s", "\"\"");
-			continue;
-		} else {
 			has_patch = 1;
-			unsigned int j = ports_fwd_array[i].out_port_id;
-			switch (port_map[j].port_type) {
-			case PHY:
-				RTE_LOG(INFO, SHARED, "Type: PHY\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"phy:%u\"",
-						port_map[j].id);
-				break;
-			case RING:
-				RTE_LOG(INFO, SHARED, "Type: RING\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"ring:%u\"",
-						port_map[j].id);
-				break;
-			case VHOST:
-				RTE_LOG(INFO, SHARED, "Type: VHOST\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"vhost:%u\"",
-						port_map[j].id);
-				break;
-			case PCAP:
-				RTE_LOG(INFO, SHARED, "Type: PCAP\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"pcap:%u\"",
-						port_map[j].id);
-				break;
-			case NULLPMD:
-				RTE_LOG(INFO, SHARED, "Type: NULLPMD\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"nullpmd:%u\"",
-						port_map[j].id);
-				break;
-			case TAP:
-				RTE_LOG(INFO, SHARED, "Type: TAP\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"tap:%u\"",
-						port_map[j].id);
-				break;
-			case MEMIF:
-				RTE_LOG(INFO, SHARED, "Type: MEMIF\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"memif:%u\"",
-						port_map[j].id);
-				break;
-			case UNDEF:
-				RTE_LOG(INFO, SHARED, "Type: UDF\n");
-				/*
-				 * TODO(yasufum) Need to remove print for
-				 * undefined ?
-				 */
-				sprintf(patch_str + strlen(patch_str),
-						"\"udf\"");
-				break;
-			}
-		}
-
-		sprintf(patch_str + strlen(patch_str), "},");
+			RTE_LOG(INFO, SHARED, "Port ID %d\n", i);
+			RTE_LOG(INFO, SHARED, "Queue ID %d\n", j);
+			RTE_LOG(INFO, SHARED, "Status %d\n",
+				ports_fwd_array[i][j].in_port_id);
+
+			memset(patch_str, '\0', sizeof(patch_str));
+
+			sprintf(patch_str, "{\"src\":");
+			append_port_string(patch_str + strlen(patch_str),
+				port_map[i].port_type, port_map[i].id,
+				j, in_max_queue);
+			sprintf(patch_str + strlen(patch_str), ",\"dst\":");
+
+			out_port_id = ports_fwd_array[i][j].out_port_id;
+			out_queue_id = ports_fwd_array[i][j].out_queue_id;
+			RTE_LOG(INFO, SHARED, "Out Port ID %d\n", out_port_id);
+			RTE_LOG(INFO, SHARED, "Out Queue ID %d\n",
+				out_queue_id);
+
+			out_max_queue = get_port_max_queues(out_port_id);
+			append_port_string(patch_str + strlen(patch_str),
+				port_map[out_port_id].port_type,
+				port_map[out_port_id].id,
+				out_queue_id, out_max_queue);
+			sprintf(patch_str + strlen(patch_str), "},");
 
-		if (has_patch != 0)
 			sprintf(str + strlen(str), "%s", patch_str);
+		}
 	}
 
-
 	/* Check if it has at least one patch to remove ",". */
 	if (has_patch == 0) {
 		sprintf(str + strlen(str), "]");
diff --git a/src/nfv/nfv_status.h b/src/nfv/nfv_status.h
index 14225cc..cf59dde 100644
--- a/src/nfv/nfv_status.h
+++ b/src/nfv/nfv_status.h
@@ -8,21 +8,15 @@
 /* Get status of spp_nfv or spp_vm as JSON format. */
 void get_sec_stats_json(char *str, int client_id,
 		const char *running_stat,
-		uint8_t lcore_id_used[RTE_MAX_LCORE],
-		struct port *ports_fwd_array,
-		struct port_map *port_map);
+		uint8_t lcore_id_used[RTE_MAX_LCORE]);
 
 int append_lcore_info_json(char *str,
 		uint8_t lcore_id_used[RTE_MAX_LCORE]);
 
 /* Append port info to sec status, called from get_sec_stats_json(). */
-int append_port_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map);
+int append_port_info_json(char *str);
 
 /* Append patch info to sec status, called from get_sec_stats_json(). */
-int append_patch_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map);
+int append_patch_info_json(char *str);
 
 #endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [spp] [PATCH 06/17] spp_primary: add support of rte_flow
  2020-02-18  6:37 ` [spp] [PATCH 06/17] spp_primary: add support of rte_flow x-fn-spp-ml
@ 2020-02-19  2:24   ` Yasufumi Ogawa
  2020-02-19 11:57     ` [spp] (x-fn-spp-ml 118) " Hideyuki Yamashita
  0 siblings, 1 reply; 40+ messages in thread
From: Yasufumi Ogawa @ 2020-02-19  2:24 UTC (permalink / raw)
  To: x-fn-spp-ml; +Cc: ferruh.yigit, spp

> From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
> 
> To support rte_flow in SPP, this patch provides main functions which
> defines validate, create, delete and flush flow
This patch has several issues from checkpatches.sh. Please fix bellow.

WARNING:RETURN_VOID: void function return statements are not generally 
useful
#604: FILE: src/primary/flow/flow.c:581:
+	return;
+}

ERROR:SWITCH_CASE_INDENT_LEVEL: switch and case should be at the same indent
#700: FILE: src/primary/flow/flow.c:677:
+	switch(input->command) {
+		case VALIDATE:
[...]
+		case CREATE:
[...]
+		case DESTROY:
[...]
+		case FLUSH:

ERROR:SPACING: space required before the open parenthesis '('
#700: FILE: src/primary/flow/flow.c:677:
+	switch(input->command) {

WARNING:RETURN_VOID: void function return statements are not generally 
useful
#729: FILE: src/primary/flow/flow.c:706:
+	return;
+}

Regards,
Yasufumi
> 
> Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
> Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
> ---
>   src/primary/flow/flow.c | 1048 +++++++++++++++++++++++++++++++++++++++
>   src/primary/flow/flow.h |   94 ++++
>   2 files changed, 1142 insertions(+)
>   create mode 100644 src/primary/flow/flow.c
>   create mode 100644 src/primary/flow/flow.h
> 
> diff --git a/src/primary/flow/flow.c b/src/primary/flow/flow.c
> new file mode 100644
> index 0000000..52a3e59
> --- /dev/null
> +++ b/src/primary/flow/flow.c
> @@ -0,0 +1,1048 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
> + */
> +
> +#include <rte_flow.h>
> +#include <rte_common.h>
> +#include <rte_ether.h>
> +#include <rte_byteorder.h>
> +
> +#include "shared/common.h"
> +#include "shared/secondary/utils.h"
> +#include "shared/secondary/spp_worker_th/data_types.h"
> +#include "primary/primary.h"
> +#include "flow.h"
> +#include "attr.h"
> +#include "common.h"
> +
> +#include "primary/flow/pattern/eth.h"
> +#include "primary/flow/pattern/vlan.h"
> +
> +#include "primary/flow/action/jump.h"
> +#include "primary/flow/action/queue.h"
> +#include "primary/flow/action/of_push_vlan.h"
> +#include "primary/flow/action/of_set_vlan_vid.h"
> +#include "primary/flow/action/of_set_vlan_pcp.h"
> +
> +
> +/* Flow list for each port */
> +static struct port_flow port_list[RTE_MAX_ETHPORTS] = { 0 };
> +
> +/* Define item operations */
> +static struct flow_item_ops flow_item_ops_list[] = {
> +	{
> +		.str_type = "end",
> +		.type = RTE_FLOW_ITEM_TYPE_END,
> +		.parse = NULL,
> +		.detail_list = NULL
> +	},
> +	{
> +		.str_type = "eth",
> +		.type = RTE_FLOW_ITEM_TYPE_ETH,
> +		.size = sizeof(struct rte_flow_item_eth),
> +		.parse = parse_item_common,
> +		.detail_list = eth_ops_list,
> +		.status = append_item_eth_json,
> +	},
> +	{
> +		.str_type = "vlan",
> +		.type = RTE_FLOW_ITEM_TYPE_VLAN,
> +		.size = sizeof(struct rte_flow_item_vlan),
> +		.parse = parse_item_common,
> +		.detail_list = vlan_ops_list,
> +		.status = append_item_vlan_json,
> +	},
> +};
> +
> +/* Define action operations */
> +static struct flow_action_ops flow_action_ops_list[] = {
> +	{
> +		.str_type = "end",
> +		.type = RTE_FLOW_ACTION_TYPE_END,
> +		.size = 0,
> +		.parse = NULL,
> +		.detail_list = NULL,
> +		.status = NULL,
> +	},
> +	{
> +		.str_type = "jump",
> +		.type = RTE_FLOW_ACTION_TYPE_JUMP,
> +		.size = sizeof(struct rte_flow_action_jump),
> +		.parse = parse_action_common,
> +		.detail_list = jump_ops_list,
> +		.status = append_action_jump_json,
> +	},
> +	{
> +		.str_type = "queue",
> +		.type = RTE_FLOW_ACTION_TYPE_QUEUE,
> +		.size = sizeof(struct rte_flow_action_queue),
> +		.parse = parse_action_common,
> +		.detail_list = queue_ops_list,
> +		.status = append_action_queue_json,
> +	},
> +	{
> +		.str_type = "of_pop_vlan",
> +		.type = RTE_FLOW_ACTION_TYPE_OF_POP_VLAN,
> +		.size = 0,
> +		.parse = NULL,
> +		.detail_list = NULL,
> +		.status = append_action_null_json,
> +	},
> +	{
> +		.str_type = "of_push_vlan",
> +		.type = RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN,
> +		.size = sizeof(struct rte_flow_action_of_push_vlan),
> +		.parse = parse_action_common,
> +		.detail_list = of_push_vlan_ops_list,
> +		.status = append_action_of_push_vlan_json,
> +	},
> +	{
> +		.str_type = "of_set_vlan_vid",
> +		.type = RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID,
> +		.size = sizeof(struct rte_flow_action_of_set_vlan_vid),
> +		.parse = parse_action_common,
> +		.detail_list = of_set_vlan_vid_ops_list,
> +		.status = append_action_of_set_vlan_vid_json,
> +	},
> +	{
> +		.str_type = "of_set_vlan_pcp",
> +		.type = RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP,
> +		.size = sizeof(struct rte_flow_action_of_set_vlan_pcp),
> +		.parse = parse_action_common,
> +		.detail_list = of_set_vlan_pcp_ops_list,
> +		.status = append_action_of_set_vlan_pcp_json,
> +	},
> +};
> +
> +/* Free memory of "flow_args". */
> +static void
> +free_flow_args(struct flow_args *input)
> +{
> +	int i;
> +	struct rte_flow_item *pattern;
> +	struct rte_flow_action *actions;
> +	char **target;
> +
> +	if ((input->command != VALIDATE) &&
> +		(input->command != CREATE))
> +		return;
> +
> +	pattern = input->args.rule.pattern;
> +	if (pattern != NULL) {
> +		for (i = 0; pattern[i].type != RTE_FLOW_ITEM_TYPE_END; i++) {
> +			target = (char **)((char *)(&pattern[i]) +
> +				offsetof(struct rte_flow_item, spec));
> +			if (*target != NULL)
> +				free(*target);
> +
> +			target = (char **)((char *)(&pattern[i]) +
> +				offsetof(struct rte_flow_item, last));
> +			if (*target != NULL)
> +				free(*target);
> +
> +			target = (char **)((char *)(&pattern[i]) +
> +				offsetof(struct rte_flow_item, mask));
> +			if (*target != NULL)
> +				free(*target);
> +		}
> +
> +		free(pattern);
> +	}
> +
> +	actions = input->args.rule.actions;
> +	if (actions != NULL) {
> +		for (i = 0; actions[i].type != RTE_FLOW_ACTION_TYPE_END; i++) {
> +			target = (char **)((char *)(&actions[i]) +
> +				offsetof(struct rte_flow_action, conf));
> +			if (*target != NULL)
> +				free(*target);
> +		}
> +
> +		free(actions);
> +	}
> +}
> +
> +/*
> + * Create response in JSON format.
> + * `rule_id` must be empty if flow create is failed.
> + */
> +static void
> +make_response(char *response, const char *result, const char *message,
> +	char *rule_id)
> +{
> +	if (rule_id == NULL)
> +		snprintf(response, MSG_SIZE,
> +			"{\"result\": \"%s\", \"message\": \"%s\"}",
> +			result, message);
> +	else
> +		snprintf(response, MSG_SIZE,
> +			"{\"result\": \"%s\", \"message\": \"%s\", "
> +			"\"rule_id\": \"%s\"}",
> +			result, message, rule_id);
> +}
> +
> +/* Create error response from rte_flow_error */
> +static void
> +make_error_response(char *response, const char *message,
> +	struct rte_flow_error error, char *rule_id)
> +{
> +	/* Define description for each error type */
> +	static const char *const errstr_list[] = {
> +		[RTE_FLOW_ERROR_TYPE_NONE] = "No error",
> +		[RTE_FLOW_ERROR_TYPE_UNSPECIFIED] = "Cause unspecified",
> +		[RTE_FLOW_ERROR_TYPE_HANDLE] = "Flow rule (handle)",
> +		[RTE_FLOW_ERROR_TYPE_ATTR_GROUP] = "Group field",
> +		[RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY] = "Priority field",
> +		[RTE_FLOW_ERROR_TYPE_ATTR_INGRESS] = "Ingress field",
> +		[RTE_FLOW_ERROR_TYPE_ATTR_EGRESS] = "Egress field",
> +		[RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER] = "Transfer field",
> +		[RTE_FLOW_ERROR_TYPE_ATTR] = "Attributes structure",
> +		[RTE_FLOW_ERROR_TYPE_ITEM_NUM] = "Pattern length",
> +		[RTE_FLOW_ERROR_TYPE_ITEM_SPEC] = "Item specification",
> +		[RTE_FLOW_ERROR_TYPE_ITEM_LAST] = "Item specification range",
> +		[RTE_FLOW_ERROR_TYPE_ITEM_MASK] = "Item specification mask",
> +		[RTE_FLOW_ERROR_TYPE_ITEM] = "Specific pattern item",
> +		[RTE_FLOW_ERROR_TYPE_ACTION_NUM] = "Number of actions",
> +		[RTE_FLOW_ERROR_TYPE_ACTION_CONF] = "Action configuration",
> +		[RTE_FLOW_ERROR_TYPE_ACTION] = "Specific action",
> +	};
> +	int err = rte_errno;
> +	char msg[512] = "";
> +	char cause[32] = "";
> +	const char *errstr;
> +
> +	if ((unsigned int)error.type >= RTE_DIM(errstr_list) ||
> +	    !errstr_list[error.type])
> +		errstr = "Unknown type";
> +	else
> +		errstr = errstr_list[error.type];
> +
> +
> +	if (error.cause != NULL)
> +		snprintf(cause, sizeof(cause), "cause: %p\\n", error.cause);
> +
> +	snprintf(msg, sizeof(msg),
> +		"%s\\nerror type: %d (%s)\\n"
> +		"%serror message: %s\\nrte_errno: %s",
> +		message, error.type, errstr, cause,
> +		error.message ? error.message : "(no stated reason)",
> +		rte_strerror(err));
> +	make_response(response, "error", msg, rule_id);
> +}
> +
> +/* Add to array, redeclare memory. */
> +static int
> +append_object_list(void **list, void *add, size_t obj_size, int num)
> +{
> +	char *new_list;
> +
> +	new_list = malloc(obj_size * num);
> +	if (new_list == NULL)
> +		return -1;
> +
> +	/* Copy original list*/
> +	if (*list != NULL) {
> +		memcpy(new_list, *list, obj_size * (num - 1));
> +		free(*list);
> +	}
> +
> +	/* Add to list */
> +	memcpy(new_list + (obj_size * (num - 1)), add, obj_size);
> +
> +	*list = (void *)new_list;
> +	return 0;
> +}
> +
> +static int
> +parse_flow_actions(char *token_list[], int *index,
> +	struct rte_flow_action **actions)
> +{
> +	int ret;
> +	int action_count = 0;
> +	uint16_t i;
> +	char *token;
> +	struct flow_action_ops *ops;
> +	struct rte_flow_action action;
> +
> +	if (strcmp(token_list[*index], "actions")) {
> +		RTE_LOG(ERR, SPP_FLOW,
> +			"Invalid parameter is %s(%s:%d)\n",
> +			token_list[*index], __func__, __LINE__);
> +		return -1;
> +	}
> +
> +	/* Next to word */
> +	(*index)++;
> +
> +	while (token_list[*index] != NULL) {
> +		token = token_list[*index];
> +
> +		for (i = 0; i < RTE_DIM(flow_action_ops_list); i++) {
> +			ops = &flow_action_ops_list[i];
> +			if (strcmp(token, ops->str_type))
> +				continue;
> +
> +			memset(&action, 0, sizeof(struct rte_flow_action));
> +			action.type = ops->type;
> +			if (ops->parse != NULL) {
> +				ret = ops->parse(token_list, index, &action,
> +					ops);
> +				if (ret < 0)
> +					return -1;
> +			} else {
> +				(*index)++;
> +			}
> +			break;
> +		}
> +
> +		/*
> +		 * Error occurs if a action string that is not defined in
> +		 * str_type of flow_action_ops_list is specified
> +		 */
> +		if (i == RTE_DIM(flow_action_ops_list)) {
> +			RTE_LOG(ERR, SPP_FLOW,
> +				"Invalid parameter "
> +				"is %s action(%s:%d)\n",
> +				token, __func__, __LINE__);
> +			return -1;
> +		}
> +
> +		/* Add to "actions" list */
> +		action_count++;
> +		ret = append_object_list((void **)actions, &action,
> +			sizeof(struct rte_flow_action), action_count);
> +
> +		if (!strcmp(token, "end"))
> +			break;
> +
> +		(*index)++;
> +	}
> +
> +	return 0;
> +}
> +
> +static int
> +parse_flow_pattern(char *token_list[], int *index,
> +	struct rte_flow_item **pattern)
> +{
> +	int ret;
> +	int item_count = 0;
> +	uint32_t i;
> +	char *token;
> +	struct flow_item_ops *ops;
> +	struct rte_flow_item item;
> +
> +	while (token_list[*index] != NULL) {
> +		token = token_list[*index];
> +
> +		for (i = 0; i < RTE_DIM(flow_item_ops_list); i++) {
> +			ops = &flow_item_ops_list[i];
> +			if (strcmp(token, ops->str_type))
> +				continue;
> +
> +			memset(&item, 0, sizeof(struct rte_flow_item));
> +			item.type = ops->type;
> +			if (ops->parse != NULL) {
> +				ret = ops->parse(token_list, index, &item,
> +					ops);
> +				if (ret < 0)
> +					return -1;
> +			}
> +			break;
> +		}
> +
> +		/*
> +		 * Error occurs if a pattern string that is not defined in
> +		 * str_type of flow_item_ops_list is specified
> +		 */
> +		if (i == RTE_DIM(flow_item_ops_list)) {
> +			RTE_LOG(ERR, SPP_FLOW,
> +				"Invalid parameter "
> +				"is %s pattern(%s:%d)\n",
> +				token, __func__, __LINE__);
> +			return -1;
> +		}
> +
> +		/* Add to "pattern" list */
> +		item_count++;
> +		ret = append_object_list((void **)pattern, &item,
> +			sizeof(struct rte_flow_item), item_count);
> +
> +		if (!strcmp(token, "end"))
> +			break;
> +
> +		(*index)++;
> +	}
> +
> +	return 0;
> +}
> +
> +static int
> +parse_flow_rule(char *token_list[], struct flow_args *input)
> +{
> +	int ret = 0;
> +	int index;
> +
> +	ret = parse_phy_port_id(token_list[2], &input->port_id);
> +	if (ret < 0)
> +		return -1;
> +
> +	/* The next index of the port */
> +	index = 3;
> +
> +	/* Attribute parse */
> +	ret = parse_flow_attr(token_list, &index, &input->args.rule.attr);
> +	if (ret < 0) {
> +		RTE_LOG(ERR, SPP_FLOW,
> +			"Failed to parse Attribute(%s:%d)\n",
> +			__func__, __LINE__);
> +		return -1;
> +	}
> +
> +	/* The next index of the pattern */
> +	index++;
> +
> +	/* Pattern parse */
> +	ret = parse_flow_pattern(token_list, &index,
> +		&input->args.rule.pattern);
> +	if (ret < 0) {
> +		RTE_LOG(ERR, SPP_FLOW,
> +			"Failed to parse Pattern(%s:%d)\n",
> +			__func__, __LINE__);
> +		return -1;
> +	}
> +
> +	/* The next index of the actions */
> +	index++;
> +
> +	/* Actions parse */
> +	ret = parse_flow_actions(token_list, &index,
> +		&input->args.rule.actions);
> +	if (ret < 0) {
> +		RTE_LOG(ERR, SPP_FLOW,
> +			"Failed to parse Actions(%s:%d)\n",
> +			__func__, __LINE__);
> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
> +static int
> +parse_flow_destroy(char *token_list[], struct flow_args *input)
> +{
> +	int ret;
> +	char *end;
> +
> +	ret = parse_phy_port_id(token_list[2], &input->port_id);
> +	if (ret < 0)
> +		return -1;
> +
> +	if (token_list[3] == NULL) {
> +		RTE_LOG(ERR, SPP_FLOW,
> +			"rule_id is not specified(%s:%d)\n",
> +			__func__, __LINE__);
> +		return -1;
> +	}
> +
> +	if (!strcmp(token_list[3], "ALL")) {
> +		input->command = FLUSH;
> +
> +	} else {
> +		input->command = DESTROY;
> +		input->args.destroy.rule_id = strtoul(token_list[3],
> +			&end, 10);
> +	}
> +
> +	return 0;
> +}
> +
> +/** Generate a flow_rule entry from attributes/pattern/actions. */
> +static struct flow_rule *
> +create_flow_rule(struct rte_flow_attr *attr,
> +	struct rte_flow_item *pattern,
> +	struct rte_flow_action *actions,
> +	struct rte_flow_error *error)
> +{
> +	const struct rte_flow_conv_rule conv_rule = {
> +		.attr_ro = attr,
> +		.pattern_ro = pattern,
> +		.actions_ro = actions,
> +	};
> +	struct flow_rule *rule;
> +	int ret;
> +
> +	ret = rte_flow_conv(RTE_FLOW_CONV_OP_RULE, NULL, 0, &conv_rule,
> +		error);
> +	if (ret < 0)
> +		return NULL;
> +
> +	rule = calloc(1, offsetof(struct flow_rule, rule) + ret);
> +	if (!rule) {
> +		rte_flow_error_set
> +			(error, errno, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> +			 "calloc() failed");
> +		return NULL;
> +	}
> +
> +	ret = rte_flow_conv(RTE_FLOW_CONV_OP_RULE, &rule->rule, ret, &conv_rule,
> +			  error);
> +	if (ret >= 0)
> +		return rule;
> +
> +	free(rule);
> +	return NULL;
> +}
> +
> +/* Execute rte_flow_validate().*/
> +static void
> +exec_flow_validate(int port_id,
> +	struct rte_flow_attr *attr,
> +	struct rte_flow_item *pattern,
> +	struct rte_flow_action *actions,
> +	char *response)
> +{
> +	int ret;
> +	struct rte_flow_error error;
> +
> +	memset(&error, 0, sizeof(error));
> +
> +	ret = rte_flow_validate(port_id, attr, pattern, actions, &error);
> +	if (ret != 0)
> +		make_error_response(response, "Flow validate error", error,
> +			NULL);
> +	else
> +		make_response(response, "success", "Flow rule validated",
> +			NULL);
> +}
> +
> +/* Execute rte_flow_create(). Save flow rules globally */
> +static void
> +exec_flow_create(int port_id,
> +	struct rte_flow_attr *attr,
> +	struct rte_flow_item *pattern,
> +	struct rte_flow_action *actions,
> +	char *response)
> +{
> +	uint32_t rule_id;
> +	char mes[32];
> +	char rule_id_str[11] = {0};
> +	struct rte_flow_error error;
> +	struct rte_flow *flow;
> +	struct flow_rule *rule;
> +	struct port_flow *port;
> +
> +	memset(&error, 0, sizeof(error));
> +
> +	flow = rte_flow_create(port_id, attr, pattern, actions, &error);
> +	if (flow == NULL) {
> +		make_error_response(response, "Flow create error", error,
> +			rule_id_str);
> +		return;
> +	}
> +
> +	port = &port_list[port_id];
> +	if (port->flow_list != NULL) {
> +		if (port->flow_list->rule_id >= UINT32_MAX) {
> +			make_response(response, "error",
> +				"Rule ID must be less than %"PRIu32,
> +				rule_id_str);
> +			rte_flow_destroy(port_id, flow, NULL);
> +			return;
> +		}
> +		rule_id = port->flow_list->rule_id + 1;
> +	} else {
> +		rule_id = 0;
> +	}
> +
> +	rule = create_flow_rule(attr, pattern, actions, &error);
> +	if (rule == NULL) {
> +		rte_flow_destroy(port_id, flow, NULL);
> +		make_error_response(response, "Flow create error", error,
> +			rule_id_str);
> +		return;
> +	}
> +
> +	/* Keep it globally as a list */
> +	rule->rule_id = rule_id;
> +	rule->flow_handle = flow;
> +
> +	if (port->flow_list == NULL)
> +		rule->prev = NULL;
> +	else
> +		rule->prev = port->flow_list;
> +
> +	port->flow_list = rule;
> +
> +	sprintf(mes, "Flow rule #%d created", rule_id);
> +	sprintf(rule_id_str, "%d", rule_id);
> +	make_response(response, "success", mes, rule_id_str);
> +	return;
> +}
> +
> +/* Execute rte_flow_destroy(). Destroying a globally saved flow rule */
> +static void
> +exec_flow_destroy(int port_id, uint32_t rule_id, char *response)
> +{
> +	int ret;
> +	int found_flg = 0;
> +	char mes[64];
> +	struct flow_rule *rule, **next_ptr;
> +	struct rte_flow_error error;
> +
> +	memset(&error, 0, sizeof(error));
> +
> +	ret = is_portid_used(port_id);
> +	if (ret != 0) {
> +		sprintf(mes, "Invalid port %d", port_id);
> +		make_response(response, "error", mes, NULL);
> +		return;
> +	}
> +
> +	next_ptr = &(port_list[port_id].flow_list);
> +	rule = port_list[port_id].flow_list;
> +
> +	while (rule != NULL) {
> +		if (rule->rule_id != rule_id) {
> +			next_ptr = &(rule->prev);
> +			rule = rule->prev;
> +			continue;
> +		}
> +
> +		ret = rte_flow_destroy(port_id, rule->flow_handle, &error);
> +		if (ret != 0) {
> +			make_error_response(response, "Flow destroy error",
> +				error, NULL);
> +			return;
> +		}
> +
> +		/* Remove flow from global list */
> +		*next_ptr = rule->prev;
> +		free(rule);
> +		found_flg = 1;
> +
> +		sprintf(mes, "Flow rule #%d destroyed", rule_id);
> +		make_response(response, "success", mes, NULL);
> +		break;
> +	}
> +
> +	/* Rule_id not found */
> +	if (found_flg == 0) {
> +		sprintf(mes, "Flow rule #%d not found", rule_id);
> +		make_response(response, "error", mes, NULL);
> +	}
> +}
> +
> +/* Delete all globally saved flow rules */
> +static void
> +exec_flow_flush(int port_id, char *response)
> +{
> +	int ret;
> +	char mes[64];
> +	struct flow_rule *rule;
> +	struct rte_flow_error error;
> +
> +	memset(&error, 0, sizeof(error));
> +
> +	ret = is_portid_used(port_id);
> +	if (ret != 0) {
> +		sprintf(mes, "Invalid port %d", port_id);
> +		make_response(response, "error", mes, NULL);
> +		return;
> +	}
> +
> +	ret = rte_flow_flush(port_id, &error);
> +	if (ret != 0)
> +		make_error_response(response, "Flow destroy error",
> +			error, NULL);
> +	else
> +		make_response(response, "success", "Flow rule all destroyed",
> +			NULL);
> +
> +	/*
> +	 * Even if a failure occurs, flow handle is invalidated,
> +	 * so delete flow_list.
> +	 */
> +
> +	while (port_list[port_id].flow_list != NULL) {
> +		rule = port_list[port_id].flow_list->prev;
> +		free(port_list[port_id].flow_list);
> +		port_list[port_id].flow_list = rule;
> +	}
> +}
> +
> +static void
> +exec_flow(struct flow_args *input, char *response)
> +{
> +	switch(input->command) {
> +		case VALIDATE:
> +			exec_flow_validate(input->port_id,
> +				&input->args.rule.attr,
> +				input->args.rule.pattern,
> +				input->args.rule.actions,
> +				response);
> +			break;
> +		case CREATE:
> +			exec_flow_create(input->port_id,
> +				&input->args.rule.attr,
> +				input->args.rule.pattern,
> +				input->args.rule.actions,
> +				response);
> +			break;
> +		case DESTROY:
> +			exec_flow_destroy(input->port_id,
> +				input->args.destroy.rule_id,
> +				response);
> +			break;
> +		case FLUSH:
> +			exec_flow_flush(input->port_id, response);
> +			break;
> +	}
> +
> +	/* Argument data is no longer needed and freed */
> +	free_flow_args(input);
> +
> +	return;
> +}
> +
> +int
> +parse_flow(char *token_list[], char *response)
> +{
> +	int ret = 0;
> +	struct flow_args input = { 0 };
> +
> +	if (token_list[1] == NULL) {
> +		ret = -1;
> +	} else if (!strcmp(token_list[1], "validate")) {
> +		input.command = VALIDATE;
> +		ret = parse_flow_rule(token_list, &input);
> +
> +	} else if (!strcmp(token_list[1], "create")) {
> +		input.command = CREATE;
> +		ret = parse_flow_rule(token_list, &input);
> +
> +	} else if (!strcmp(token_list[1], "destroy")) {
> +		ret = parse_flow_destroy(token_list, &input);
> +
> +	} else {
> +		ret = -1;
> +	}
> +
> +	if (ret != 0) {
> +		free_flow_args(&input);
> +		make_response(response, "error",
> +			"Flow command invalid argument", NULL);
> +		return 0;
> +	}
> +
> +	exec_flow(&input, response);
> +
> +	return 0;
> +}
> +
> +static int
> +append_flow_pattern_json(const struct rte_flow_item *pattern, int buf_size,
> +	char *pattern_str)
> +{
> +	uint32_t i, j;
> +	uint32_t nof_elems = 3;
> +	int ret = 0;
> +	char *tmp_str;
> +	const char element_str[][5] = { "spec", "last", "mask" };
> +	const struct rte_flow_item *ptn = pattern;
> +	struct flow_item_ops *ops;
> +	const void *tmp_ptr[nof_elems];
> +
> +	tmp_str = malloc(buf_size);
> +	if (tmp_str == NULL) {
> +		RTE_LOG(ERR, SPP_FLOW,
> +			"Memory allocation failure(%s:%d)\n",
> +			__func__, __LINE__);
> +		return -1;
> +	}
> +
> +	while (ptn->type != RTE_FLOW_ITEM_TYPE_END) {
> +		memset(tmp_str, 0, buf_size);
> +
> +		tmp_ptr[0] = ptn->spec;
> +		tmp_ptr[1] = ptn->last;
> +		tmp_ptr[2] = ptn->mask;
> +
> +		for (i = 0; i < RTE_DIM(flow_item_ops_list); i++) {
> +			ops = &flow_item_ops_list[i];
> +			if (ptn->type != ops->type)
> +				continue;
> +
> +			snprintf(tmp_str, buf_size,
> +				"{\"type\":\"%s\",",
> +				ops->str_type);
> +
> +			for (j = 0; j < nof_elems; j++) {
> +				snprintf(tmp_str + strlen(tmp_str), buf_size,
> +					"\"%s\":",
> +					element_str[j]);
> +
> +				if (tmp_ptr[j] != NULL)
> +					ret = ops->status(tmp_ptr[j],
> +						buf_size - (int)strlen(tmp_str),
> +						tmp_str + strlen(tmp_str));
> +				else
> +					snprintf(tmp_str + strlen(tmp_str),
> +						buf_size,
> +						"null");
> +
> +				if (ret != 0)
> +					break;
> +
> +				if (j < nof_elems - 1)
> +					tmp_str[strlen(tmp_str)] = ',';
> +			}
> +
> +			tmp_str[strlen(tmp_str)] = '}';
> +
> +			break;
> +		}
> +
> +		if (ret != 0)
> +			break;
> +
> +		if ((int)strlen(pattern_str) + (int)strlen(tmp_str)
> +			> buf_size - 1) {
> +			ret = -1;
> +			break;
> +		}
> +		strncat(pattern_str, tmp_str, strlen(tmp_str));
> +
> +		/*
> +		 * If there is the following pattern, add ',' to
> +		 * pattern_str
> +		 */
> +		ptn++;
> +		if (ptn->type != RTE_FLOW_ITEM_TYPE_END) {
> +			if ((int)strlen(pattern_str) + 1 > buf_size - 1) {
> +				ret = -1;
> +				break;
> +			}
> +			pattern_str[strlen(pattern_str)] = ',';
> +		}
> +	}
> +
> +	if (tmp_str != NULL)
> +		free(tmp_str);
> +
> +	return ret;
> +}
> +
> +static int
> +append_flow_action_json(const struct rte_flow_action *actions, int buf_size,
> +	char *actions_str)
> +{
> +	uint32_t i;
> +	int ret = 0;
> +	char *tmp_str;
> +	const struct rte_flow_action *act = actions;
> +	struct flow_action_ops *ops;
> +
> +	tmp_str = malloc(buf_size);
> +	if (tmp_str == NULL) {
> +		RTE_LOG(ERR, SPP_FLOW,
> +			"Memory allocation failure(%s:%d)\n",
> +			__func__, __LINE__);
> +		return -1;
> +	}
> +
> +	while (act->type != RTE_FLOW_ACTION_TYPE_END) {
> +		memset(tmp_str, 0, buf_size);
> +
> +		for (i = 0; i < RTE_DIM(flow_action_ops_list); i++) {
> +			ops = &flow_action_ops_list[i];
> +			if (act->type != ops->type)
> +				continue;
> +
> +			snprintf(tmp_str, buf_size,
> +				"{\"type\":\"%s\",\"conf\":",
> +				ops->str_type);
> +
> +			ret = ops->status(act->conf,
> +				buf_size - (int)strlen(tmp_str),
> +				tmp_str + strlen(tmp_str));
> +			tmp_str[strlen(tmp_str)] = '}';
> +			break;
> +		}
> +
> +		if (ret != 0)
> +			break;
> +
> +		if ((int)strlen(actions_str) + (int)strlen(tmp_str)
> +			> buf_size - 1) {
> +			ret = -1;
> +			break;
> +		}
> +		strncat(actions_str, tmp_str, strlen(tmp_str));
> +
> +		/*
> +		 * If there is the following pattern, add ',' to
> +		 * actions_str
> +		 */
> +		act++;
> +		if (act->type != RTE_FLOW_ACTION_TYPE_END) {
> +			if ((int)strlen(actions_str) + 1 > buf_size - 1) {
> +				ret = -1;
> +				break;
> +			}
> +			actions_str[strlen(actions_str)] = ',';
> +		}
> +	}
> +
> +	if (tmp_str != NULL)
> +		free(tmp_str);
> +
> +	return ret;
> +}
> +
> +static int
> +append_flow_rule_json(struct flow_rule *flow, int buf_size, char *flow_str)
> +{
> +	int ret = 0;
> +	struct rte_flow_conv_rule rule;
> +	char *tmp_str, *attr_str, *pattern_str, *actions_str;
> +
> +	while (1) {
> +		tmp_str = malloc(buf_size);
> +		attr_str = malloc(buf_size);
> +		pattern_str = malloc(buf_size);
> +		actions_str = malloc(buf_size);
> +		if (tmp_str == NULL || attr_str == NULL
> +			|| pattern_str == NULL || actions_str == NULL) {
> +			RTE_LOG(ERR, SPP_FLOW,
> +				"Memory allocation failure(%s:%d)\n",
> +				__func__, __LINE__);
> +			ret = -1;
> +			break;
> +		}
> +		memset(tmp_str, 0, buf_size);
> +		memset(attr_str, 0, buf_size);
> +		memset(pattern_str, 0, buf_size);
> +		memset(actions_str, 0, buf_size);
> +
> +		rule = flow->rule;
> +
> +		ret = append_flow_attr_json(rule.attr_ro, buf_size, attr_str);
> +		if (ret != 0)
> +			break;
> +
> +		ret = append_flow_pattern_json(rule.pattern_ro, buf_size,
> +			pattern_str);
> +		if (ret != 0)
> +			break;
> +
> +		ret = append_flow_action_json(rule.actions_ro, buf_size,
> +			actions_str);
> +		if (ret != 0)
> +			break;
> +
> +		snprintf(tmp_str, buf_size,
> +			"{\"rule_id\":%d,"
> +			"\"attr\":%s,"
> +			"\"patterns\":[%s],"
> +			"\"actions\":[%s]}",
> +			flow->rule_id, attr_str, pattern_str, actions_str);
> +
> +		if ((int)strlen(tmp_str) > buf_size - 1) {
> +			ret = -1;
> +			break;
> +		}
> +
> +		snprintf(flow_str, buf_size, "%s", tmp_str);
> +		break;
> +	}
> +
> +	if (tmp_str != NULL)
> +		free(tmp_str);
> +	if (attr_str != NULL)
> +		free(attr_str);
> +	if (pattern_str != NULL)
> +		free(pattern_str);
> +	if (actions_str != NULL)
> +		free(actions_str);
> +
> +	return ret;
> +}
> +
> +int
> +append_flow_json(int port_id, int buf_size, char *output)
> +{
> +	int ret = 0;
> +	int str_size = 0;
> +	char *flow_str, *tmp_str;
> +	struct flow_rule *flow;
> +
> +	flow_str = malloc(buf_size);
> +	tmp_str = malloc(buf_size);
> +	if (flow_str == NULL || tmp_str == NULL) {
> +		RTE_LOG(ERR, SPP_FLOW,
> +			"Memory allocation failure(%s:%d)\n",
> +			__func__, __LINE__);
> +		return -1;
> +	}
> +
> +	flow = port_list[port_id].flow_list;
> +
> +	while (flow != NULL) {
> +		memset(flow_str, 0, buf_size);
> +
> +		ret = append_flow_rule_json(flow, buf_size, flow_str);
> +		if (ret != 0)
> +			break;
> +
> +		if (str_size == 0) {
> +			snprintf(output, buf_size, "%s", flow_str);
> +			str_size += (int)strlen(flow_str);
> +
> +		} else {
> +			str_size += ((int)strlen(flow_str) + 1);
> +			if (str_size > buf_size - 1) {
> +				ret = -1;
> +				break;
> +			}
> +
> +			/*
> +			 * Since flow_list is in descending order,
> +			 * concatenate the strings in front.
> +			 */
> +			memset(tmp_str, 0, buf_size);
> +			strncpy(tmp_str, output, buf_size);
> +			memset(output, 0, buf_size);
> +
> +			snprintf(output, buf_size, "%s,%s",
> +				flow_str, tmp_str);
> +		}
> +
> +		flow = flow->prev;
> +	}
> +
> +	if (ret == 0) {
> +		if ((int)strlen("[]") + (int)strlen(flow_str)
> +			> buf_size - 1)
> +			ret = -1;
> +		else {
> +			memset(tmp_str, 0, buf_size);
> +			strncpy(tmp_str, output, buf_size);
> +			memset(output, 0, buf_size);
> +
> +			snprintf(output, buf_size, "[%s]", tmp_str);
> +		}
> +	}
> +
> +	if (ret != 0)
> +		RTE_LOG(ERR, SPP_FLOW,
> +			"Cannot send all of flow stats(%s:%d)\n",
> +			__func__, __LINE__);
> +
> +	if (flow_str != NULL)
> +		free(flow_str);
> +	if (tmp_str != NULL)
> +		free(tmp_str);
> +
> +	return ret;
> +}
> diff --git a/src/primary/flow/flow.h b/src/primary/flow/flow.h
> new file mode 100644
> index 0000000..ecd4eb3
> --- /dev/null
> +++ b/src/primary/flow/flow.h
> @@ -0,0 +1,94 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
> + */
> +
> +#ifndef _PRIMARY_FLOW_H_
> +#define _PRIMARY_FLOW_H_
> +
> +#include <rte_log.h>
> +
> +#define RTE_LOGTYPE_SPP_FLOW RTE_LOGTYPE_USER1
> +
> +enum flow_command {
> +	VALIDATE = 0,
> +	CREATE,
> +	DESTROY,
> +	FLUSH
> +};
> +
> +/* Parser result of flow command arguments */
> +struct flow_args {
> +	enum flow_command command;
> +	int port_id;
> +	union {
> +		struct {
> +			struct rte_flow_attr attr;
> +			struct rte_flow_item *pattern;
> +			struct rte_flow_action *actions;
> +		} rule; /* validate or create arguments. */
> +		struct {
> +			uint32_t rule_id;
> +		} destroy; /* destroy arguments. */
> +	} args;
> +};
> +
> +/* Descriptor for a single flow. */
> +struct flow_rule {
> +	/* Flow rule ID */
> +	uint32_t rule_id;
> +
> +	/* Previous flow in list. */
> +	struct flow_rule *prev;
> +
> +	/* Opaque flow object returned by PMD. */
> +	struct rte_flow *flow_handle;
> +
> +	/* Saved flow rule description. */
> +	struct rte_flow_conv_rule rule;
> +};
> +
> +/* Flow rule list of the port */
> +struct port_flow {
> +	/* Associated flows */
> +	struct flow_rule *flow_list;
> +};
> +
> +/* Detail parse operation for a specific item or action */
> +struct flow_detail_ops {
> +	const char *token;
> +	const size_t offset;
> +	const size_t size;
> +	int flg_value;
> +	int (*parse_detail)(char *str, void *output);
> +};
> +
> +/* Operation for each item type */
> +struct flow_item_ops {
> +	const char *str_type;
> +	enum rte_flow_item_type type;
> +	size_t size;
> +	int (*parse)(char *token_list[], int *index,
> +		struct rte_flow_item *pattern,
> +		struct flow_item_ops *ops);
> +	struct flow_detail_ops *detail_list;
> +	int (*status)(const void *element,
> +		int buf_size, char *pattern_str);
> +};
> +
> +/* Operation for each action type */
> +struct flow_action_ops {
> +	const char *str_type;
> +	enum rte_flow_action_type type;
> +	size_t size;
> +	int (*parse)(char *token_list[], int *index,
> +		struct rte_flow_action *action,
> +		struct flow_action_ops *ops);
> +	struct flow_detail_ops *detail_list;
> +	int (*status)(const void *conf,
> +		int buf_size, char *action_str);
> +};
> +
> +int parse_flow(char *token_list[], char *response);
> +int append_flow_json(int port_id, int buf_size, char *output);
> +
> +#endif
> 

^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 00/17] Adding Hardware offload capability
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (16 preceding siblings ...)
  2020-02-18  6:37 ` [spp] [PATCH 17/17] spp_nfv: add support of multi-queue x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-21  8:17   ` Yasufumi Ogawa
  2020-02-19 11:49 ` [spp] [PATCH v2 01/17] shared: add support of multi-queue x-fn-spp-ml
                   ` (16 subsequent siblings)
  34 siblings, 1 reply; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch set provides hardware offload capability
for spp.
Note that related documents will be provided soon.

Hideyuki Yamashita (17):
  shared: add support of multi-queue
  spp_vf: add support of multi-queue
  spp_mirror: add support of multi-queue
  spp_pcap: add support of multi-queue
  spp_primary: add support of multi-queue
  spp_primary: add support of rte_flow
  spp_primary: add common function of rte_flow
  spp_primary: add attribute of rte_flow
  spp_primary: add patterns of rte_flow
  spp_primary: add actions of rte_flow
  bin: add parameter for hardrare offload
  cli: add support of hardware offload
  cli: add support of rte_flow in vf
  cli: add support of rte_flow in mirror
  cli: add support of rte_flow in nfv
  spp-ctl: add APIs for flow rules
  spp_nfv: add support of multi-queue

 bin/sample/config.sh                          |   15 +
 bin/spp_pri.sh                                |   26 +-
 bin/start.sh                                  |    2 +
 src/cli/commands/mirror.py                    |  175 ++-
 src/cli/commands/nfv.py                       |  245 ++--
 src/cli/commands/pri.py                       |   52 +-
 src/cli/commands/pri_flow.py                  |  936 +++++++++++++++
 src/cli/commands/pri_flow_compl_action.py     |  115 ++
 src/cli/commands/pri_flow_compl_pattern.py    |   84 ++
 src/cli/commands/vf.py                        |  460 +++++---
 src/mirror/mir_cmd_runner.c                   |    3 +-
 src/mirror/spp_mirror.c                       |   12 +-
 src/nfv/commands.h                            |   71 +-
 src/nfv/main.c                                |   18 +-
 src/nfv/nfv_status.c                          |  315 +++--
 src/nfv/nfv_status.h                          |   12 +-
 src/pcap/cmd_runner.c                         |    3 +-
 src/pcap/cmd_utils.c                          |   73 +-
 src/pcap/cmd_utils.h                          |   19 +-
 src/pcap/spp_pcap.c                           |  102 +-
 src/primary/Makefile                          |   10 +
 src/primary/args.c                            |  138 ++-
 src/primary/args.h                            |    3 +
 src/primary/flow/action/jump.c                |   42 +
 src/primary/flow/action/jump.h                |   12 +
 src/primary/flow/action/of_push_vlan.c        |   44 +
 src/primary/flow/action/of_push_vlan.h        |   13 +
 src/primary/flow/action/of_set_vlan_pcp.c     |   44 +
 src/primary/flow/action/of_set_vlan_pcp.h     |   13 +
 src/primary/flow/action/of_set_vlan_vid.c     |   44 +
 src/primary/flow/action/of_set_vlan_vid.h     |   13 +
 src/primary/flow/action/queue.c               |   42 +
 src/primary/flow/action/queue.h               |   13 +
 src/primary/flow/attr.c                       |  105 ++
 src/primary/flow/attr.h                       |   13 +
 src/primary/flow/common.c                     |  646 ++++++++++
 src/primary/flow/common.h                     |   53 +
 src/primary/flow/flow.c                       | 1045 +++++++++++++++++
 src/primary/flow/flow.h                       |   94 ++
 src/primary/flow/pattern/eth.c                |   63 +
 src/primary/flow/pattern/eth.h                |   13 +
 src/primary/flow/pattern/vlan.c               |   71 ++
 src/primary/flow/pattern/vlan.h               |   13 +
 src/primary/init.c                            |    8 +-
 src/primary/init.h                            |    3 +-
 src/primary/main.c                            |   88 +-
 src/shared/basic_forwarder.c                  |   62 +-
 src/shared/basic_forwarder.h                  |    2 +-
 src/shared/common.h                           |   30 +-
 src/shared/port_manager.c                     |  180 ++-
 src/shared/port_manager.h                     |   13 +-
 .../secondary/spp_worker_th/cmd_parser.c      |  137 ++-
 .../spp_worker_th/cmd_res_formatter.c         |   63 +-
 .../secondary/spp_worker_th/cmd_utils.c       |  111 +-
 .../secondary/spp_worker_th/cmd_utils.h       |   19 +-
 .../secondary/spp_worker_th/data_types.h      |   18 +-
 .../secondary/spp_worker_th/port_capability.c |    8 +-
 .../secondary/spp_worker_th/port_capability.h |    4 +-
 src/shared/secondary/spp_worker_th/vf_deps.h  |   15 +-
 src/shared/secondary/utils.c                  |   28 +-
 src/shared/secondary/utils.h                  |    3 +-
 src/spp-ctl/spp_proc.py                       |    4 +
 src/spp-ctl/spp_webapi.py                     |  200 +++-
 src/vf/classifier.c                           |   40 +-
 src/vf/forwarder.c                            |   12 +-
 src/vf/vf_cmd_runner.c                        |   32 +-
 66 files changed, 5621 insertions(+), 744 deletions(-)
 create mode 100644 src/cli/commands/pri_flow.py
 create mode 100644 src/cli/commands/pri_flow_compl_action.py
 create mode 100644 src/cli/commands/pri_flow_compl_pattern.py
 create mode 100644 src/primary/flow/action/jump.c
 create mode 100644 src/primary/flow/action/jump.h
 create mode 100644 src/primary/flow/action/of_push_vlan.c
 create mode 100644 src/primary/flow/action/of_push_vlan.h
 create mode 100644 src/primary/flow/action/of_set_vlan_pcp.c
 create mode 100644 src/primary/flow/action/of_set_vlan_pcp.h
 create mode 100644 src/primary/flow/action/of_set_vlan_vid.c
 create mode 100644 src/primary/flow/action/of_set_vlan_vid.h
 create mode 100644 src/primary/flow/action/queue.c
 create mode 100644 src/primary/flow/action/queue.h
 create mode 100644 src/primary/flow/attr.c
 create mode 100644 src/primary/flow/attr.h
 create mode 100644 src/primary/flow/common.c
 create mode 100644 src/primary/flow/common.h
 create mode 100644 src/primary/flow/flow.c
 create mode 100644 src/primary/flow/flow.h
 create mode 100644 src/primary/flow/pattern/eth.c
 create mode 100644 src/primary/flow/pattern/eth.h
 create mode 100644 src/primary/flow/pattern/vlan.c
 create mode 100644 src/primary/flow/pattern/vlan.h

-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 01/17] shared: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (17 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 00/17] Adding Hardware offload capability x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 02/17] spp_vf: " x-fn-spp-ml
                   ` (15 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support multi-queue in SPP,changes like following should be
introduced:
- add queue number parameter in port command
- parse newly added parameter
- add queue number parameter in status response

Under shared directory, common logic among secondary process exist
and those should be affected.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/shared/basic_forwarder.c                  |  62 +++---
 src/shared/basic_forwarder.h                  |   2 +-
 src/shared/common.h                           |  30 ++-
 src/shared/port_manager.c                     | 180 ++++++++++++++----
 src/shared/port_manager.h                     |  13 +-
 .../secondary/spp_worker_th/cmd_parser.c      | 137 ++++++++++---
 .../spp_worker_th/cmd_res_formatter.c         |  63 ++++--
 .../secondary/spp_worker_th/cmd_utils.c       | 111 ++++++++---
 .../secondary/spp_worker_th/cmd_utils.h       |  19 +-
 .../secondary/spp_worker_th/data_types.h      |  18 +-
 .../secondary/spp_worker_th/port_capability.c |   8 +-
 .../secondary/spp_worker_th/port_capability.h |   4 +-
 src/shared/secondary/spp_worker_th/vf_deps.h  |  15 +-
 src/shared/secondary/utils.c                  |  28 ++-
 src/shared/secondary/utils.h                  |   3 +-
 15 files changed, 535 insertions(+), 158 deletions(-)

diff --git a/src/shared/basic_forwarder.c b/src/shared/basic_forwarder.c
index 7aefaaa..66f8643 100644
--- a/src/shared/basic_forwarder.c
+++ b/src/shared/basic_forwarder.c
@@ -5,6 +5,7 @@
 #include <stdint.h>
 #include "shared/common.h"
 #include "shared/basic_forwarder.h"
+#include "shared/port_manager.h"
 
 void
 forward(void)
@@ -14,42 +15,53 @@ forward(void)
 	int in_port;
 	int out_port;
 	uint16_t buf;
-	int i;
+	int i, j;
+	uint16_t max_queue, in_queue, out_queue;
 
 	/* Go through every possible port numbers*/
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
-		struct rte_mbuf *bufs[MAX_PKT_BURST];
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
-			continue;
+		max_queue = get_port_max_queues(i);
 
-		if (ports_fwd_array[i].out_port_id == PORT_RESET)
-			continue;
+		/* Checks the number of rxq declared for the port */
+		for (j = 0; j < max_queue; j++) {
 
-		/* if status active, i count is in port*/
-		in_port = i;
-		out_port = ports_fwd_array[i].out_port_id;
+			struct rte_mbuf *bufs[MAX_PKT_BURST];
 
-		/* Get burst of RX packets, from first port of pair. */
-		/*first port rx, second port tx*/
-		nb_rx = ports_fwd_array[in_port].rx_func(in_port, 0, bufs,
-			MAX_PKT_BURST);
-		if (unlikely(nb_rx == 0))
-			continue;
+			if (ports_fwd_array[i][j].in_port_id == PORT_RESET)
+				continue;
 
-		port_map[in_port].stats->rx += nb_rx;
+			if (ports_fwd_array[i][j].out_port_id == PORT_RESET)
+				continue;
 
-		/* Send burst of TX packets, to second port of pair. */
-		nb_tx = ports_fwd_array[out_port].tx_func(out_port, 0, bufs,
-			nb_rx);
+			/* if status active, i count is in port*/
+			in_port = i;
+			in_queue = j;
+			out_port = ports_fwd_array[i][j].out_port_id;
+			out_queue = ports_fwd_array[i][j].out_queue_id;
 
-		port_map[out_port].stats->tx += nb_tx;
+			/* Get burst of RX packets, from first port of pair. */
+			/*first port rx, second port tx*/
+			nb_rx = ports_fwd_array[in_port][in_queue].rx_func(
+				in_port, in_queue, bufs, MAX_PKT_BURST);
+			if (unlikely(nb_rx == 0))
+				continue;
 
-		/* Free any unsent packets. */
-		if (unlikely(nb_tx < nb_rx)) {
-			port_map[out_port].stats->tx_drop += nb_rx - nb_tx;
-			for (buf = nb_tx; buf < nb_rx; buf++)
-				rte_pktmbuf_free(bufs[buf]);
+			port_map[in_port].stats->rx += nb_rx;
+
+			/* Send burst of TX packets, to second port of pair. */
+			nb_tx = ports_fwd_array[out_port][out_queue].tx_func(
+				out_port, out_queue, bufs, nb_rx);
+
+			port_map[out_port].stats->tx += nb_tx;
+
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_rx)) {
+				port_map[out_port].stats->tx_drop += (nb_rx -
+					nb_tx);
+				for (buf = nb_tx; buf < nb_rx; buf++)
+					rte_pktmbuf_free(bufs[buf]);
+			}
 		}
 	}
 }
diff --git a/src/shared/basic_forwarder.h b/src/shared/basic_forwarder.h
index 2e8225d..915b5cc 100644
--- a/src/shared/basic_forwarder.h
+++ b/src/shared/basic_forwarder.h
@@ -8,7 +8,7 @@
 #include "shared/common.h"
 
 struct port_map port_map[RTE_MAX_ETHPORTS];
-struct port ports_fwd_array[RTE_MAX_ETHPORTS];
+struct port ports_fwd_array[RTE_MAX_ETHPORTS][RTE_MAX_QUEUES_PER_PORT];
 
 void forward(void);
 
diff --git a/src/shared/common.h b/src/shared/common.h
index b4af73c..d311e82 100644
--- a/src/shared/common.h
+++ b/src/shared/common.h
@@ -10,7 +10,23 @@
 #include <unistd.h>
 #include <rte_ethdev_driver.h>
 
-#define MSG_SIZE 2048  /* socket buffer len */
+/*
+ * TODO(tx_h-yamashita): Remove this definition because it was used from
+ * spp_primary and spp_nfv and its value was 2048, but the value should
+ * be changed to 32KiB for containing status of spp_primary.
+ * For spp_nfv, 2048 is still enough. This buff len should be defined in
+ * each of spp_primary and spp_nfv as appropriate size.
+ */
+/*
+ * NOTE: The size of 32768(32Kbyte) is enough to set maximum size of JSON
+ * response string. This maximum size comes from primary`s `status` command
+ * response which includes `LCORE`, `PHY` and `RING` information.
+ * For supporting rte_flow API, `PHY` information may contain a lot of
+ * flow data which defined by user. Our design choice is that `PHY`
+ * information size would be 30Kbyte and `LCORE` size plus `PHY` size
+ * would be 2Kbyte.
+ */
+#define MSG_SIZE 32768  /* socket buffer max len */
 
 #define SOCK_RESET  -1
 #define PORT_RESET  UINT16_MAX
@@ -70,11 +86,19 @@ struct stats {
 	uint64_t tx_drop;
 } __rte_cache_aligned;
 
+/* rx_queue and tx_queue set to port. */
+struct port_queue {
+	uint16_t rxq;
+	uint16_t txq;
+};
+
 struct port_info {
 	uint16_t num_ports;
 	uint16_t id[RTE_MAX_ETHPORTS];
 	struct stats port_stats[RTE_MAX_ETHPORTS];
 	struct stats client_stats[MAX_CLIENT];
+	/* num of queues per port */
+	struct port_queue queue_info[RTE_MAX_ETHPORTS];
 };
 
 enum port_type {
@@ -93,11 +117,15 @@ struct port_map {
 	enum port_type port_type;
 	struct stats *stats;
 	struct stats default_stats;
+	/* num of queues per port */
+	struct port_queue *queue_info;
 };
 
 struct port {
 	uint16_t in_port_id;
+	uint16_t in_queue_id;
 	uint16_t out_port_id;
+	uint16_t out_queue_id;
 	uint16_t (*rx_func)(uint16_t, uint16_t, struct rte_mbuf **, uint16_t);
 	uint16_t (*tx_func)(uint16_t, uint16_t, struct rte_mbuf **, uint16_t);
 };
diff --git a/src/shared/port_manager.c b/src/shared/port_manager.c
index 534af99..6beeca7 100644
--- a/src/shared/port_manager.c
+++ b/src/shared/port_manager.c
@@ -16,35 +16,46 @@ struct porttype_map portmap[] = {
 };
 
 void
-forward_array_init_one(unsigned int i)
+forward_array_init_one(unsigned int i, unsigned int j)
 {
-	ports_fwd_array[i].in_port_id = PORT_RESET;
-	ports_fwd_array[i].out_port_id = PORT_RESET;
+	ports_fwd_array[i][j].in_port_id = PORT_RESET;
+	ports_fwd_array[i][j].in_queue_id = 0;
+	ports_fwd_array[i][j].out_port_id = PORT_RESET;
+	ports_fwd_array[i][j].out_queue_id = 0;
 }
 
 /* initialize forward array with default value */
 void
 forward_array_init(void)
 {
-	unsigned int i;
+	unsigned int i, j;
 
 	/* initialize port forward array*/
-	for (i = 0; i < RTE_MAX_ETHPORTS; i++)
-		forward_array_init_one(i);
+	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		for (j = 0; j < RTE_MAX_QUEUES_PER_PORT; j++)
+			forward_array_init_one(i, j);
+	}
 }
 
 void
 forward_array_reset(void)
 {
-	unsigned int i;
+	unsigned int i, j;
+	uint16_t max_queue;
 
 	/* initialize port forward array*/
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
-		if (ports_fwd_array[i].in_port_id != PORT_RESET) {
-			ports_fwd_array[i].out_port_id = PORT_RESET;
-			RTE_LOG(INFO, SHARED, "Port ID %d\n", i);
-			RTE_LOG(INFO, SHARED, "out_port_id %d\n",
-				ports_fwd_array[i].out_port_id);
+
+		max_queue = get_port_max_queues(i);
+
+		for (j = 0; j < max_queue; j++) {
+			if (ports_fwd_array[i][j].in_port_id != PORT_RESET) {
+				ports_fwd_array[i][j].out_port_id = PORT_RESET;
+				RTE_LOG(INFO, SHARED, "Port ID %d\n", i);
+				RTE_LOG(INFO, SHARED, "Queue ID %d\n", j);
+				RTE_LOG(INFO, SHARED, "out_port_id %d\n",
+					ports_fwd_array[i][j].out_port_id);
+			}
 		}
 	}
 }
@@ -55,6 +66,7 @@ port_map_init_one(unsigned int i)
 	port_map[i].id = PORT_RESET;
 	port_map[i].port_type = UNDEF;
 	port_map[i].stats = &port_map[i].default_stats;
+	port_map[i].queue_info = NULL;
 }
 
 void
@@ -68,28 +80,46 @@ port_map_init(void)
 
 /* Return -1 as an error if given patch is invalid */
 int
-add_patch(uint16_t in_port, uint16_t out_port)
+add_patch(uint16_t in_port, uint16_t in_queue,
+		uint16_t out_port, uint16_t out_queue)
 {
-	if (!is_valid_port(in_port) || !is_valid_port(out_port))
+	if (!is_valid_port(in_port, in_queue) ||
+		!is_valid_port(out_port, out_queue))
 		return -1;
 
+	if (!is_valid_port_rxq(in_port, in_queue) ||
+		!is_valid_port_txq(out_port, out_queue))
+		return 1;
+
 	/* Populate in port data */
-	ports_fwd_array[in_port].in_port_id = in_port;
-	ports_fwd_array[in_port].rx_func = &rte_eth_rx_burst;
-	ports_fwd_array[in_port].tx_func = &rte_eth_tx_burst;
-	ports_fwd_array[in_port].out_port_id = out_port;
+	ports_fwd_array[in_port][in_queue].in_port_id = in_port;
+	ports_fwd_array[in_port][in_queue].in_queue_id = in_queue;
+	ports_fwd_array[in_port][in_queue].rx_func = &rte_eth_rx_burst;
+	ports_fwd_array[in_port][in_queue].tx_func = &rte_eth_tx_burst;
+	ports_fwd_array[in_port][in_queue].out_port_id = out_port;
+	ports_fwd_array[in_port][in_queue].out_queue_id = out_queue;
 
 	/* Populate out port data */
-	ports_fwd_array[out_port].in_port_id = out_port;
-	ports_fwd_array[out_port].rx_func = &rte_eth_rx_burst;
-	ports_fwd_array[out_port].tx_func = &rte_eth_tx_burst;
+	ports_fwd_array[out_port][out_queue].in_port_id = out_port;
+	ports_fwd_array[out_port][out_queue].in_queue_id = out_queue;
+	ports_fwd_array[out_port][out_queue].rx_func = &rte_eth_rx_burst;
+	ports_fwd_array[out_port][out_queue].tx_func = &rte_eth_tx_burst;
 
-	RTE_LOG(DEBUG, SHARED, "STATUS: in port %d in_port_id %d\n", in_port,
-		ports_fwd_array[in_port].in_port_id);
-	RTE_LOG(DEBUG, SHARED, "STATUS: in port %d patch out port id %d\n",
-		in_port, ports_fwd_array[in_port].out_port_id);
-	RTE_LOG(DEBUG, SHARED, "STATUS: outport %d in_port_id %d\n", out_port,
-		ports_fwd_array[out_port].in_port_id);
+	RTE_LOG(DEBUG, SHARED, "STATUS: in port %d in queue %d"
+		" in_port_id %d in_queue_id %d\n",
+		in_port, in_queue,
+		ports_fwd_array[in_port][in_queue].in_port_id,
+		ports_fwd_array[in_port][in_queue].in_queue_id);
+	RTE_LOG(DEBUG, SHARED, "STATUS: in port %d in queue %d"
+		" patch out_port_id %d out_queue_id %d\n",
+		in_port, in_queue,
+		ports_fwd_array[in_port][in_queue].out_port_id,
+		ports_fwd_array[in_port][in_queue].out_queue_id);
+	RTE_LOG(DEBUG, SHARED, "STATUS: out port %d out queue %d"
+		" in_port_id %d in_queue_id %d\n",
+		out_port, out_queue,
+		ports_fwd_array[out_port][out_queue].in_port_id,
+		ports_fwd_array[out_port][out_queue].in_queue_id);
 
 	return 0;
 }
@@ -119,30 +149,92 @@ find_port_id(int id, enum port_type type)
 
 /* Return 0 if invalid */
 int
-is_valid_port(uint16_t port_id)
+is_valid_port(uint16_t port_id, uint16_t queue_id)
 {
+	uint16_t max_queue;
+
 	if (port_id > RTE_MAX_ETHPORTS)
 		return 0;
 
-	return port_map[port_id].id != PORT_RESET;
+	if (port_map[port_id].id == PORT_RESET)
+		return 0;
+
+	max_queue = get_port_max_queues(port_id);
+	if (queue_id >= max_queue)
+		return 0;
+
+	return 1;
+}
+
+/*
+ * Check if rxq exceeds the number of queues defined for the port.
+ * Return 0 if invalid.
+ */
+int
+is_valid_port_rxq(uint16_t port_id, uint16_t rxq)
+{
+	uint16_t nof_queues;
+
+	if (port_map[port_id].queue_info != NULL) {
+		nof_queues = port_map[port_id].queue_info->rxq;
+	} else {
+		/* default number of queues is 1 */
+		nof_queues = 1;
+	}
+	if (rxq >= nof_queues)
+		return 0;
+
+	return 1;
+}
+
+/*
+ * Check if txq exceeds the number of queues defined for the port.
+ * Return 0 if invalid.
+ */
+int
+is_valid_port_txq(uint16_t port_id, uint16_t txq)
+{
+	uint16_t nof_queues;
+
+	if (port_map[port_id].queue_info != NULL) {
+		nof_queues = port_map[port_id].queue_info->txq;
+	} else {
+		/* default number of queues is 1 */
+		nof_queues = 1;
+	}
+	if (txq >= nof_queues)
+		return 0;
+
+	return 1;
 }
 
 void
-forward_array_remove(int port_id)
+forward_array_remove(int port_id, uint16_t queue_id)
 {
-	unsigned int i;
+	unsigned int i, j;
+	uint16_t max_queue;
+	int remove_flg = 0;
 
 	/* Update ports_fwd_array */
-	forward_array_init_one(port_id);
+	forward_array_init_one(port_id, queue_id);
 
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
-			continue;
 
-		if (ports_fwd_array[i].out_port_id == port_id) {
-			ports_fwd_array[i].out_port_id = PORT_RESET;
+		max_queue = get_port_max_queues(i);
+
+		for (j = 0; j < max_queue; j++) {
+			if (ports_fwd_array[i][j].in_port_id == PORT_RESET &&
+				(ports_fwd_array[i][j].out_port_id != port_id ||
+				ports_fwd_array[i][j].out_queue_id != queue_id))
+				continue;
+
+			ports_fwd_array[i][j].out_port_id = PORT_RESET;
+			remove_flg = 1;
 			break;
 		}
+
+		if (remove_flg)
+			break;
 	}
 }
 
@@ -157,3 +249,19 @@ enum port_type get_port_type(char *portname)
 	}
 	return UNDEF;
 }
+
+/* Returns a larger number of queues of RX or TX port as the maximum number */
+uint16_t
+get_port_max_queues(uint16_t port_id)
+{
+	uint16_t max_queue = 1; /* default max_queue is 1 */
+
+	if (port_map[port_id].queue_info != NULL) {
+		if (port_map[port_id].queue_info->rxq >=
+			port_map[port_id].queue_info->txq)
+			max_queue = port_map[port_id].queue_info->rxq;
+		else
+			max_queue = port_map[port_id].queue_info->txq;
+	}
+	return max_queue;
+}
diff --git a/src/shared/port_manager.h b/src/shared/port_manager.h
index c529998..8317687 100644
--- a/src/shared/port_manager.h
+++ b/src/shared/port_manager.h
@@ -17,20 +17,25 @@ struct porttype_map {
 };
 
 /* initialize forward array with default value */
-void forward_array_init_one(unsigned int i);
+void forward_array_init_one(unsigned int i, unsigned int j);
 void forward_array_init(void);
 void forward_array_reset(void);
-void forward_array_remove(int port_id);
+void forward_array_remove(int port_id, uint16_t queue_id);
 
 void port_map_init_one(unsigned int i);
 void port_map_init(void);
 
 enum port_type get_port_type(char *portname);
 
-int add_patch(uint16_t in_port, uint16_t out_port);
+int add_patch(uint16_t in_port, uint16_t in_queue,
+	uint16_t out_port, uint16_t out_queue);
 
 uint16_t find_port_id(int id, enum port_type type);
 
-int is_valid_port(uint16_t port_id);
+int is_valid_port(uint16_t port_id, uint16_t queue_id);
+int is_valid_port_rxq(uint16_t port_id, uint16_t rxq);
+int is_valid_port_txq(uint16_t port_id, uint16_t txq);
+
+uint16_t get_port_max_queues(uint16_t port_id);
 
 #endif  // __SHARED_PORT_MANAGER_H__
diff --git a/src/shared/secondary/spp_worker_th/cmd_parser.c b/src/shared/secondary/spp_worker_th/cmd_parser.c
index 3f947f5..6e66f35 100644
--- a/src/shared/secondary/spp_worker_th/cmd_parser.c
+++ b/src/shared/secondary/spp_worker_th/cmd_parser.c
@@ -120,10 +120,10 @@ const char *PORT_ABILITY_LIST[] = {
 static int
 is_used_with_addr(
 		int vid, uint64_t mac_addr,
-		enum port_type iface_type, int iface_no)
+		enum port_type iface_type, int iface_no, int queue_no)
 {
 	struct sppwk_port_info *wk_port = get_sppwk_port(
-			iface_type, iface_no);
+			iface_type, iface_no, queue_no);
 
 	return ((mac_addr == wk_port->cls_attrs.mac_addr) &&
 		(vid == wk_port->cls_attrs.vlantag.vid));
@@ -131,25 +131,32 @@ is_used_with_addr(
 
 /* Return 1 as true if given port is already used. */
 static int
-is_added_port(enum port_type iface_type, int iface_no)
+is_added_port(enum port_type iface_type, int iface_no, int queue_no)
 {
-	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no);
+	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no,
+			queue_no);
 	return port->iface_type != UNDEF;
 }
 
 /**
- * Separate resource UID of combination of iface type and number and assign to
- * given argument, iface_type and iface_no. For instance, 'ring:0' is separated
- * to 'ring' and '0'. The supported types are `phy`, `vhost` and `ring`.
+ * Extract `iface_type`, `iface_no` and `queue_no` from `res_uid`.
+ * Only phy port has `queue_no`, such as 'phy:0nq1', if using multi-queue.
+ * For other port types other than `phy`,
+ * it returnes `DEFAULT_QUEUE_ID` as `queue_no`.
  */
 static int
 parse_resource_uid(const char *res_uid,
-		    enum port_type *iface_type,
-		    int *iface_no)
+			enum port_type *iface_type,
+			int *iface_no,
+			int *queue_no)
 {
 	enum port_type ptype = UNDEF;
+	const char *iface_no_and_queue_no_str = NULL;
+	const char *queue_no_with_separator_str = NULL;
 	const char *iface_no_str = NULL;
-	char *endptr = NULL;
+	char *queue_no_endptr = NULL;
+	char *iface_no_endptr = NULL;
+	int queue_id, multi_queue_flg;
 
 	/**
 	 * TODO(yasufum) consider this checking of zero value is recommended
@@ -158,33 +165,84 @@ parse_resource_uid(const char *res_uid,
 	if (strncmp(res_uid, SPPWK_PHY_STR ":",
 			strlen(SPPWK_PHY_STR)+1) == 0) {
 		ptype = PHY;
-		iface_no_str = &res_uid[strlen(SPPWK_PHY_STR)+1];
+		iface_no_and_queue_no_str = &res_uid[strlen(SPPWK_PHY_STR)+1];
 	} else if (strncmp(res_uid, SPPWK_VHOST_STR ":",
 			strlen(SPPWK_VHOST_STR)+1) == 0) {
 		ptype = VHOST;
-		iface_no_str = &res_uid[strlen(SPPWK_VHOST_STR)+1];
+		iface_no_and_queue_no_str = &res_uid[strlen(SPPWK_VHOST_STR)+1];
 	} else if (strncmp(res_uid, SPPWK_RING_STR ":",
 			strlen(SPPWK_RING_STR)+1) == 0) {
 		ptype = RING;
-		iface_no_str = &res_uid[strlen(SPPWK_RING_STR)+1];
+		iface_no_and_queue_no_str = &res_uid[strlen(SPPWK_RING_STR)+1];
+	} else {
+		RTE_LOG(ERR, WK_CMD_PARSER, "Unexpected port type in '%s'.\n",
+				res_uid);
+		return SPPWK_RET_NG;
+	}
+
+	/* Parse queue number. */
+	queue_no_with_separator_str = strstr(iface_no_and_queue_no_str,
+			DELIM_PHY_MQ);
+	if (queue_no_with_separator_str == NULL) {
+		iface_no_str = iface_no_and_queue_no_str;
+		queue_id = DEFAULT_QUEUE_ID;
+		multi_queue_flg = 0;
+	} else if (ptype == PHY) {
+		const char *queue_no_str =
+				&queue_no_with_separator_str[
+				strlen(DELIM_PHY_MQ)];
+		queue_id = strtol(queue_no_str, &queue_no_endptr, 0);
+		if (unlikely(queue_no_str == queue_no_endptr) ||
+				unlikely(*queue_no_endptr != '\0')) {
+			RTE_LOG(ERR, WK_CMD_PARSER, "No queue number in '%s'.\n",
+					res_uid);
+			return SPPWK_RET_NG;
+		}
+		iface_no_str = iface_no_and_queue_no_str;
+		multi_queue_flg = 1;
 	} else {
 		RTE_LOG(ERR, WK_CMD_PARSER, "Unexpected port type in '%s'.\n",
 				res_uid);
 		return SPPWK_RET_NG;
 	}
 
-	int port_id = strtol(iface_no_str, &endptr, 0);
-	if (unlikely(iface_no_str == endptr) || unlikely(*endptr != '\0')) {
+	/* Parse interface number. */
+	int port_id = strtol(iface_no_str, &iface_no_endptr, 0);
+	if (unlikely(iface_no_str == iface_no_endptr) ||
+		(unlikely(*iface_no_endptr != '\0') &&
+		(unlikely(strstr(iface_no_endptr,
+			DELIM_PHY_MQ) != iface_no_endptr)))) {
 		RTE_LOG(ERR, WK_CMD_PARSER, "No interface number in '%s'.\n",
 				res_uid);
 		return SPPWK_RET_NG;
 	}
+	if (unlikely(port_id > RTE_MAX_ETHPORTS) || unlikely(port_id < 0)) {
+		RTE_LOG(ERR, WK_CMD_PARSER, "Unexpected interface number in '%s'.\n",
+				res_uid);
+		return SPPWK_RET_NG;
+	}
+
+	/**
+	 * Check whether the queue number is specified according
+	 * to the port format.
+	 */
+	if (ptype == PHY) {
+		int port_multi_queue_flg =
+			(get_port_max_queues(ptype, port_id) > 1);
+		if (unlikely(multi_queue_flg != port_multi_queue_flg)) {
+			RTE_LOG(ERR, WK_CMD_PARSER,
+					"Unexpected queue number format in"
+					" '%s'.\n", res_uid);
+			return SPPWK_RET_NG;
+		}
+	}
 
 	*iface_type = ptype;
 	*iface_no = port_id;
+	*queue_no = queue_id;
 
-	RTE_LOG(DEBUG, WK_CMD_PARSER, "Parsed '%s' to '%d' and '%d'.\n",
-			res_uid, *iface_type, *iface_no);
+	RTE_LOG(DEBUG, WK_CMD_PARSER, "Parsed '%s' to '%d' and '%d' and '%d'.\n",
+			res_uid, *iface_type, *iface_no, *queue_no);
 	return SPPWK_RET_OK;
 }
 
@@ -302,7 +360,8 @@ parse_port_uid(void *output, const char *arg_val)
 {
 	int ret;
 	struct sppwk_port_idx *port = output;
-	ret = parse_resource_uid(arg_val, &port->iface_type, &port->iface_no);
+	ret = parse_resource_uid(arg_val, &port->iface_type, &port->iface_no,
+			&port->queue_no);
 	if (unlikely(ret != 0)) {
 		RTE_LOG(ERR, WK_CMD_PARSER,
 				"Invalid resource UID '%s'.\n", arg_val);
@@ -465,9 +524,11 @@ parse_port(void *output, const char *arg_val, int allow_override)
 		if ((port->wk_action == SPPWK_ACT_ADD) &&
 				(sppwk_check_used_port(tmp_port.iface_type,
 						tmp_port.iface_no,
+						tmp_port.queue_no,
 						SPPWK_PORT_DIR_RX) >= 0) &&
 				(sppwk_check_used_port(tmp_port.iface_type,
 						tmp_port.iface_no,
+						tmp_port.queue_no,
 						SPPWK_PORT_DIR_TX) >= 0)) {
 			RTE_LOG(ERR, WK_CMD_PARSER,
 				"Port `%s` is already used.\n",
@@ -478,6 +539,7 @@ parse_port(void *output, const char *arg_val, int allow_override)
 
 	port->port.iface_type = tmp_port.iface_type;
 	port->port.iface_no   = tmp_port.iface_no;
+	port->port.queue_no   = tmp_port.queue_no;
 	return SPPWK_RET_OK;
 }
 
@@ -495,11 +557,36 @@ parse_port_direction(void *output, const char *arg_val, int allow_override)
 		return SPPWK_RET_NG;
 	}
 
-	/* add vlantag command check */
+	/* Add queue number command check. */
+	if ((port->wk_action == SPPWK_ACT_ADD) &&
+			(port->port.iface_type == PHY)) {
+		const struct port_info *ports = NULL;
+		const struct rte_memzone *mz;
+		mz = rte_memzone_lookup(MZ_PORT_INFO);
+		ports = mz->addr;
+		int max_port_queue = -1;
+		if (ret == SPPWK_PORT_DIR_RX)
+			max_port_queue = ports->queue_info
+					[port->port.iface_no].rxq - 1;
+		else if (ret == SPPWK_PORT_DIR_TX)
+			max_port_queue = ports->queue_info
+					[port->port.iface_no].txq - 1;
+		if (unlikely(port->port.queue_no > max_port_queue)) {
+			RTE_LOG(ERR, WK_CMD_PARSER,
+					"The queue number exceeds the %s max value."
+					" queue=%d, max=%d.\n",
+					PORT_DIR_LIST[ret], port->port.queue_no,
+					max_port_queue);
+			return SPPWK_RET_NG;
+		}
+	}
+
+	/* Add vlantag command check. */
 	if (allow_override == 0) {
 		if ((port->wk_action == SPPWK_ACT_ADD) &&
 				(sppwk_check_used_port(port->port.iface_type,
-					port->port.iface_no, ret) >= 0)) {
+					port->port.iface_no,
+					port->port.queue_no, ret) >= 0)) {
 			RTE_LOG(ERR, WK_CMD_PARSER,
 				"Port in used. (port command) val=%s\n",
 				arg_val);
@@ -726,7 +813,8 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val,
 	if (ret < SPPWK_RET_OK)
 		return SPPWK_RET_NG;
 
-	if (is_added_port(tmp_port.iface_type, tmp_port.iface_no) == 0) {
+	if (is_added_port(tmp_port.iface_type, tmp_port.iface_no,
+			tmp_port.queue_no) == 0) {
 		RTE_LOG(ERR, WK_CMD_PARSER, "Port not added. val=%s\n",
 				arg_val);
 		return SPPWK_RET_NG;
@@ -737,7 +825,8 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val,
 
 	if (unlikely(cls_attrs->wk_action == SPPWK_ACT_ADD)) {
 		if (!is_used_with_addr(ETH_VLAN_ID_MAX, 0,
-				tmp_port.iface_type, tmp_port.iface_no)) {
+				tmp_port.iface_type, tmp_port.iface_no,
+				tmp_port.queue_no)) {
 			RTE_LOG(ERR, WK_CMD_PARSER, "Port in used. "
 					"(classifier_table command) val=%s\n",
 					arg_val);
@@ -750,7 +839,8 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val,
 
 		if (!is_used_with_addr(cls_attrs->vid,
 				(uint64_t)mac_addr,
-				tmp_port.iface_type, tmp_port.iface_no)) {
+				tmp_port.iface_type, tmp_port.iface_no,
+				tmp_port.queue_no)) {
 			RTE_LOG(ERR, WK_CMD_PARSER, "Port in used. "
 					"(classifier_table command) val=%s\n",
 					arg_val);
@@ -760,6 +850,7 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val,
 
 	cls_attrs->port.iface_type = tmp_port.iface_type;
 	cls_attrs->port.iface_no   = tmp_port.iface_no;
+	cls_attrs->port.queue_no   = tmp_port.queue_no;
 	return SPPWK_RET_OK;
 }
 
diff --git a/src/shared/secondary/spp_worker_th/cmd_res_formatter.c b/src/shared/secondary/spp_worker_th/cmd_res_formatter.c
index 85b48af..a04bc91 100644
--- a/src/shared/secondary/spp_worker_th/cmd_res_formatter.c
+++ b/src/shared/secondary/spp_worker_th/cmd_res_formatter.c
@@ -79,9 +79,10 @@ append_error_details_value(const char *name, char **output, void *tmp)
 
 /* Check if port is already flushed. */
 static int
-is_port_flushed(enum port_type iface_type, int iface_no)
+is_port_flushed(enum port_type iface_type, int iface_no, int queue_no)
 {
-	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no);
+	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no,
+			queue_no);
 	return port->ethdev_port_id >= 0;
 }
 
@@ -89,23 +90,43 @@ is_port_flushed(enum port_type iface_type, int iface_no)
 int
 append_interface_array(char **output, const enum port_type type)
 {
-	int i, port_cnt = 0;
+	int port_cnt, str_cnt = 0;
+	uint16_t queue_cnt;
 	char tmp_str[CMD_TAG_APPEND_SIZE];
 
-	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
-		if (!is_port_flushed(type, i))
-			continue;
+	for (port_cnt = 0; port_cnt < RTE_MAX_ETHPORTS; port_cnt++) {
+		int max_queue_port = get_port_max_queues(type, port_cnt);
+		if (unlikely(max_queue_port < 0))
+			return SPPWK_RET_NG;
 
-		sprintf(tmp_str, "%s%d", JSON_APPEND_COMMA(port_cnt), i);
+		for (queue_cnt = 0; queue_cnt < max_queue_port;
+			queue_cnt++) {
+			if (!is_port_flushed(type, port_cnt, queue_cnt))
+				continue;
+
+			if (max_queue_port <= 1) {
+				sprintf(tmp_str, "%s\"%d\"",
+						JSON_APPEND_COMMA(str_cnt),
+						port_cnt);
+			} else {
+				sprintf(tmp_str, "%s\"%d nq %d\"",
+						JSON_APPEND_COMMA(str_cnt),
+						port_cnt, queue_cnt);
+			}
 
-		*output = spp_strbuf_append(*output, tmp_str, strlen(tmp_str));
-		if (unlikely(*output == NULL)) {
-			RTE_LOG(ERR, WK_CMD_RES_FMT,
-				/* TODO(yasufum) replace %d to string. */
-				"Failed to add index for type `%d`.\n", type);
-			return SPPWK_RET_NG;
+			*output = spp_strbuf_append(*output,
+					tmp_str, strlen(tmp_str));
+			if (unlikely(*output == NULL)) {
+				RTE_LOG(ERR, WK_CMD_RES_FMT,
+					/**
+					 * TODO(yasufum) replace %d to string.
+					 */
+					"Failed to add index for type `%d`.\n",
+							type);
+				return SPPWK_RET_NG;
+			}
+			str_cnt++;
 		}
-		port_cnt++;
 	}
 	return SPPWK_RET_OK;
 }
@@ -195,14 +216,14 @@ append_vlan_block(const char *name, char **output,
  * It returns a port ID, or error code if it's failed to.
  */
 static int
-get_ethdev_port_id(enum port_type iface_type, int iface_no)
+get_ethdev_port_id(enum port_type iface_type, int iface_no, int queue_no)
 {
 	struct iface_info *iface_info = NULL;
 
 	sppwk_get_mng_data(&iface_info, NULL, NULL, NULL, NULL, NULL);
 	switch (iface_type) {
 	case PHY:
-		return iface_info->phy[iface_no].ethdev_port_id;
+		return iface_info->phy[iface_no][queue_no].ethdev_port_id;
 	case RING:
 		return iface_info->ring[iface_no].ethdev_port_id;
 	case VHOST:
@@ -227,15 +248,17 @@ append_port_block(char **output, const struct sppwk_port_idx *port,
 		return SPPWK_RET_NG;
 	}
 
-	sppwk_port_uid(port_str, port->iface_type, port->iface_no);
+	sppwk_port_uid(port_str, port->iface_type, port->iface_no,
+			port->queue_no);
 	ret = append_json_str_value(&tmp_buff, "port", port_str);
 	if (unlikely(ret < SPPWK_RET_OK))
 		return SPPWK_RET_NG;
 
 	ret = append_vlan_block("vlan", &tmp_buff,
 			get_ethdev_port_id(
-				port->iface_type, port->iface_no),
-			dir);
+				port->iface_type, port->iface_no,
+				port->queue_no),
+				dir);
 	if (unlikely(ret < SPPWK_RET_OK))
 		return SPPWK_RET_NG;
 
@@ -519,7 +542,7 @@ add_client_id(const char *name, char **output,
 	return append_json_int_value(output, name, get_client_id());
 }
 
-/* Add entry of port to a response in JSON such as "phy:0". */
+/* Add entry of port to a response in JSON such as "phy:0nq0". */
 int
 add_interface(const char *name, char **output,
 		void *tmp __attribute__ ((unused)))
diff --git a/src/shared/secondary/spp_worker_th/cmd_utils.c b/src/shared/secondary/spp_worker_th/cmd_utils.c
index 69d7222..3dea02c 100644
--- a/src/shared/secondary/spp_worker_th/cmd_utils.c
+++ b/src/shared/secondary/spp_worker_th/cmd_utils.c
@@ -154,13 +154,13 @@ stop_process(int signal)
  * if given type is invalid.
  */
 struct sppwk_port_info *
-get_sppwk_port(enum port_type iface_type, int iface_no)
+get_sppwk_port(enum port_type iface_type, int iface_no, int queue_no)
 {
 	struct iface_info *iface_info = g_mng_data.p_iface_info;
 
 	switch (iface_type) {
 	case PHY:
-		return &iface_info->phy[iface_no];
+		return &iface_info->phy[iface_no][queue_no];
 	case VHOST:
 		return &iface_info->vhost[iface_no];
 	case RING:
@@ -229,19 +229,24 @@ log_interface_info(const struct iface_info *iface_info)
 {
 	const struct sppwk_port_info *port = NULL;
 	int cnt = 0;
+	int queue_cnt;
 	for (cnt = 0; cnt < RTE_MAX_ETHPORTS; cnt++) {
-		port = &iface_info->phy[cnt];
-		if (port->iface_type == UNDEF)
-			continue;
-
-		RTE_LOG(DEBUG, WK_CMD_UTILS,
-				"phy  [%d] type=%d, no=%d, port=%d, "
-				"vid = %u, mac=%08lx(%s)\n",
-				cnt, port->iface_type, port->iface_no,
-				port->ethdev_port_id,
-				port->cls_attrs.vlantag.vid,
-				port->cls_attrs.mac_addr,
-				port->cls_attrs.mac_addr_str);
+		int max_queue_port = get_port_max_queues(PHY, cnt);
+		for (queue_cnt = 0; queue_cnt < max_queue_port;
+				queue_cnt++) {
+			port = &iface_info->phy[cnt][queue_cnt];
+			if (port->iface_type == UNDEF)
+				continue;
+
+			RTE_LOG(DEBUG, WK_CMD_UTILS,
+					"phy [%d] type=%d, no=%d, port=%d, "
+					"queue=%d, vid = %u, mac=%08lx(%s)\n",
+					cnt, port->iface_type, port->iface_no,
+					port->ethdev_port_id, port->queue_no,
+					port->cls_attrs.vlantag.vid,
+					port->cls_attrs.mac_addr,
+					port->cls_attrs.mac_addr_str);
+		}
 	}
 	for (cnt = 0; cnt < RTE_MAX_ETHPORTS; cnt++) {
 		port = &iface_info->vhost[cnt];
@@ -356,21 +361,33 @@ static void
 init_iface_info(void)
 {
 	int port_cnt;  /* increment ether ports */
+	int queue_cnt;  /* increment ether queue per ports */
 	struct iface_info *p_iface_info = g_mng_data.p_iface_info;
 	memset(p_iface_info, 0x00, sizeof(struct iface_info));
 	for (port_cnt = 0; port_cnt < RTE_MAX_ETHPORTS; port_cnt++) {
-		p_iface_info->phy[port_cnt].iface_type = UNDEF;
-		p_iface_info->phy[port_cnt].iface_no = port_cnt;
-		p_iface_info->phy[port_cnt].ethdev_port_id = -1;
-		p_iface_info->phy[port_cnt].cls_attrs.vlantag.vid =
-			ETH_VLAN_ID_MAX;
+		for (queue_cnt = 0; queue_cnt < RTE_MAX_QUEUES_PER_PORT;
+				queue_cnt++) {
+			p_iface_info->phy[port_cnt][queue_cnt].iface_type =
+					UNDEF;
+			p_iface_info->phy[port_cnt][queue_cnt].iface_no =
+					port_cnt;
+			p_iface_info->phy[port_cnt][queue_cnt].queue_no =
+					queue_cnt;
+			p_iface_info->phy[port_cnt][queue_cnt]
+					.ethdev_port_id = -1;
+			p_iface_info->phy[port_cnt][queue_cnt]
+					.cls_attrs.vlantag.vid =
+					ETH_VLAN_ID_MAX;
+		}
 		p_iface_info->vhost[port_cnt].iface_type = UNDEF;
 		p_iface_info->vhost[port_cnt].iface_no = port_cnt;
+		p_iface_info->vhost[port_cnt].queue_no = DEFAULT_QUEUE_ID;
 		p_iface_info->vhost[port_cnt].ethdev_port_id = -1;
 		p_iface_info->vhost[port_cnt].cls_attrs.vlantag.vid =
 			ETH_VLAN_ID_MAX;
 		p_iface_info->ring[port_cnt].iface_type = UNDEF;
 		p_iface_info->ring[port_cnt].iface_no = port_cnt;
+		p_iface_info->ring[port_cnt].queue_no = DEFAULT_QUEUE_ID;
 		p_iface_info->ring[port_cnt].ethdev_port_id = -1;
 		p_iface_info->ring[port_cnt].cls_attrs.vlantag.vid =
 			ETH_VLAN_ID_MAX;
@@ -411,7 +428,7 @@ static int
 init_host_port_info(void)
 {
 	int port_type, port_id;
-	int i, ret;
+	int i, ret, queue_id;
 	int nof_phys = 0;
 	char dev_name[RTE_DEV_NAME_MAX_LEN] = { 0 };
 	struct iface_info *p_iface_info = g_mng_data.p_iface_info;
@@ -433,8 +450,13 @@ init_host_port_info(void)
 
 		switch (port_type) {
 		case PHY:
-			p_iface_info->phy[port_id].iface_type = port_type;
-			p_iface_info->phy[port_id].ethdev_port_id = port_id;
+			for (queue_id = 0; queue_id < RTE_MAX_QUEUES_PER_PORT;
+					queue_id++) {
+				p_iface_info->phy[port_id][queue_id]
+						.iface_type = port_type;
+				p_iface_info->phy[port_id][queue_id]
+						.ethdev_port_id = port_id;
+			}
 			break;
 		case VHOST:
 			/* NOTE: a vhost can be used by one process.
@@ -526,12 +548,14 @@ int
 sppwk_check_used_port(
 		enum port_type iface_type,
 		int iface_no,
+		int queue_no,
 		enum sppwk_port_dir dir)
 {
 	int cnt, port_cnt, max = 0;
 	struct sppwk_comp_info *component = NULL;
 	struct sppwk_port_info **port_array = NULL;
-	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no);
+	struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no,
+			queue_no);
 	struct sppwk_comp_info *component_info =
 					g_mng_data.p_component_info;
 
@@ -567,14 +591,14 @@ set_component_change_port(struct sppwk_port_info *port,
 	int ret = 0;
 	if ((dir == SPPWK_PORT_DIR_RX) || (dir == SPPWK_PORT_DIR_BOTH)) {
 		ret = sppwk_check_used_port(port->iface_type, port->iface_no,
-				SPPWK_PORT_DIR_RX);
+				port->queue_no, SPPWK_PORT_DIR_RX);
 		if (ret >= 0)
 			*(g_mng_data.p_change_component + ret) = 1;
 	}
 
 	if ((dir == SPPWK_PORT_DIR_TX) || (dir == SPPWK_PORT_DIR_BOTH)) {
 		ret = sppwk_check_used_port(port->iface_type, port->iface_no,
-				SPPWK_PORT_DIR_TX);
+				port->queue_no, SPPWK_PORT_DIR_TX);
 		if (ret >= 0)
 			*(g_mng_data.p_change_component + ret) = 1;
 	}
@@ -628,6 +652,28 @@ get_idx_port_info(struct sppwk_port_info *p_info, int nof_ports,
 	return ret;
 }
 
+/* Returns a larger number of queues of RX or TX port as the maximum number */
+int
+get_port_max_queues(const enum port_type iface_type, int iface_no)
+{
+	if (unlikely(iface_no > RTE_MAX_ETHPORTS) || unlikely(iface_no < 0))
+		return SPPWK_RET_NG;
+
+	if (iface_type != PHY)
+		return 1;
+
+	const struct port_info *ports = NULL;
+	const struct rte_memzone *mz;
+	mz = rte_memzone_lookup(MZ_PORT_INFO);
+	ports = mz->addr;
+	int max_q_rx = ports->queue_info[iface_no].rxq;
+	int max_q_tx = ports->queue_info[iface_no].txq;
+	if (max_q_rx > max_q_tx)
+		return max_q_rx;
+	else
+		return max_q_tx;
+}
+
 /* Delete given port info from the port info array. */
 int
 delete_port_info(struct sppwk_port_info *p_info, int nof_ports,
@@ -716,8 +762,9 @@ update_lcore_info(void)
 	}
 }
 
-/* Return port uid such as `phy:0`, `ring:1` or so. */
-int sppwk_port_uid(char *port_uid, enum port_type p_type, int iface_no)
+/* Return port uid such as `phy:0nq0`, `ring:1` or so. */
+int sppwk_port_uid(char *port_uid, enum port_type p_type, int iface_no,
+			int queue_no)
 {
 	const char *p_type_str;
 
@@ -735,7 +782,15 @@ int sppwk_port_uid(char *port_uid, enum port_type p_type, int iface_no)
 		return SPPWK_RET_NG;
 	}
 
-	sprintf(port_uid, "%s:%d", p_type_str, iface_no);
+	int max_queue_port = get_port_max_queues(p_type, iface_no);
+	if (unlikely(max_queue_port == SPPWK_RET_NG))
+		return SPPWK_RET_NG;
+
+	if (max_queue_port <= 1)
+		sprintf(port_uid, "%s:%d", p_type_str, iface_no);
+	else
+		sprintf(port_uid, "%s:%d nq %d",
+			p_type_str, iface_no, queue_no);
 
 	return SPPWK_RET_OK;
 }
diff --git a/src/shared/secondary/spp_worker_th/cmd_utils.h b/src/shared/secondary/spp_worker_th/cmd_utils.h
index df1b0dc..eda55c6 100644
--- a/src/shared/secondary/spp_worker_th/cmd_utils.h
+++ b/src/shared/secondary/spp_worker_th/cmd_utils.h
@@ -174,7 +174,7 @@ void stop_process(int signal);
 
 /* Return sppwk_port_info of given type and num of interface. */
 struct sppwk_port_info *
-get_sppwk_port(enum port_type iface_type, int iface_no);
+get_sppwk_port(enum port_type iface_type, int iface_no, int queue_no);
 
 /* Output log message for core information */
 void log_core_info(const struct core_mng_info *core_info);
@@ -221,6 +221,7 @@ int sppwk_is_lcore_updated(unsigned int lcore_id);
  *
  * @param iface_type Interface type to be validated.
  * @param iface_no Interface number to be validated.
+ * @param queue_no Queue number of interface to be validated.
  * @param rxtx Value of spp_port_rxtx to be validated.
  * @retval 0~127      If match component ID
  * @retval SPPWK_RET_NG If failed.
@@ -228,6 +229,7 @@ int sppwk_is_lcore_updated(unsigned int lcore_id);
 int sppwk_check_used_port(
 		enum port_type iface_type,
 		int iface_no,
+		int queue_no,
 		enum sppwk_port_dir dir);
 
 /**
@@ -269,6 +271,15 @@ int sppwk_get_lcore_id(const char *comp_name);
 int get_idx_port_info(struct sppwk_port_info *p_info, int nof_ports,
 		struct sppwk_port_info *p_info_ary[]);
 
+/**
+ * Returns max queue number of the target port.
+ *
+ * @param[in] iface_type Interface type such as PHY or so.
+ * @param[in] iface_no Interface number.
+ * @return Max queue number if succeeded, or SPPWK_RET_NG if failed.
+ */
+int get_port_max_queues(const enum port_type iface_type, int iface_no);
+
 /**
  *  search matched port_info from array and delete it.
  *
@@ -294,15 +305,17 @@ int update_port_info(void);
 void update_lcore_info(void);
 
 /**
- * Return port uid such as `phy:0`, `ring:1` or so.
+ * Return port uid such as `phy:0nq0`, `ring:1` or so.
  *
  * @param[in,out] port_uid String of port type to be converted.
  * @param[in] iface_type Interface type such as PHY or so.
  * @param[in] iface_no Interface number.
+ * @param[in] queue_no Queue number of interface.
  * @return SPPWK_RET_OK If succeeded, or SPPWK_RET_NG if failed.
  */
 int
-sppwk_port_uid(char *port_uid, enum port_type iface_type, int iface_no);
+sppwk_port_uid(char *port_uid, enum port_type iface_type, int iface_no,
+		int queue_no);
 
 /**
  * Change string of MAC address to int64.
diff --git a/src/shared/secondary/spp_worker_th/data_types.h b/src/shared/secondary/spp_worker_th/data_types.h
index 57bf182..a05eb75 100644
--- a/src/shared/secondary/spp_worker_th/data_types.h
+++ b/src/shared/secondary/spp_worker_th/data_types.h
@@ -15,9 +15,17 @@
 #define SPPWK_VHOST_STR "vhost"
 #define SPPWK_RING_STR "ring"
 
+/*
+ * Delimiter for phy port supporting multi queue.
+ * Example of phy port supporting multi-queue: `phy: 0nq1`.
+ */
+#define DELIM_PHY_MQ "nq"
+
 /* TODO(yasufum) confirm usage of this value and why it is 4. */
 #define PORT_CAPABL_MAX 4  /* Max num of port abilities. */
 
+#define DEFAULT_QUEUE_ID 0  /* Queue ID is counted up from 0. */
+
 /* Status of a component on lcore. */
 enum sppwk_lcore_status {
 	SPPWK_LCORE_UNUSED,
@@ -89,6 +97,7 @@ struct sppwk_cls_attrs {
 struct sppwk_port_idx {
 	enum port_type iface_type;  /**< phy, vhost or ring. */
 	int iface_no;
+	int queue_no;
 };
 
 /* Define detailed port params in addition to `sppwk_port_idx`. */
@@ -96,6 +105,7 @@ struct sppwk_port_info {
 	enum port_type iface_type;  /**< phy, vhost or ring */
 	int iface_no;
 	int ethdev_port_id;  /**< Consistent ID of ethdev */
+	int queue_no;
 	struct sppwk_cls_attrs cls_attrs;
 	struct sppwk_port_attrs port_attrs[PORT_CAPABL_MAX];
 };
@@ -108,8 +118,10 @@ struct sppwk_comp_info {
 	int comp_id;  /**< Component ID */
 	int nof_rx;  /**< The number of rx ports */
 	int nof_tx;  /**< The number of tx ports */
-	struct sppwk_port_info *rx_ports[RTE_MAX_ETHPORTS]; /**< rx ports */
-	struct sppwk_port_info *tx_ports[RTE_MAX_ETHPORTS]; /**< tx ports */
+	/**< rx ports */
+	struct sppwk_port_info *rx_ports[RTE_MAX_QUEUES_PER_PORT];
+	/**< tx ports */
+	struct sppwk_port_info *tx_ports[RTE_MAX_QUEUES_PER_PORT];
 };
 
 /* Manage number of interfaces  and port information as global variable. */
@@ -120,7 +132,7 @@ struct sppwk_comp_info {
  * or not.
  */
 struct iface_info {
-	struct sppwk_port_info phy[RTE_MAX_ETHPORTS];
+	struct sppwk_port_info phy[RTE_MAX_ETHPORTS][RTE_MAX_QUEUES_PER_PORT];
 	struct sppwk_port_info vhost[RTE_MAX_ETHPORTS];
 	struct sppwk_port_info ring[RTE_MAX_ETHPORTS];
 };
diff --git a/src/shared/secondary/spp_worker_th/port_capability.c b/src/shared/secondary/spp_worker_th/port_capability.c
index 5435db0..5bd759f 100644
--- a/src/shared/secondary/spp_worker_th/port_capability.c
+++ b/src/shared/secondary/spp_worker_th/port_capability.c
@@ -378,11 +378,11 @@ vlan_operation(uint16_t port_id, struct rte_mbuf **pkts, const uint16_t nb_pkts,
 /* Wrapper function for rte_eth_rx_burst() with VLAN feature. */
 uint16_t
 sppwk_eth_vlan_rx_burst(uint16_t port_id,
-		uint16_t queue_id __attribute__ ((unused)),
+		uint16_t queue_id,
 		struct rte_mbuf **rx_pkts, const uint16_t nb_pkts)
 {
 	uint16_t nb_rx;
-	nb_rx = rte_eth_rx_burst(port_id, 0, rx_pkts, nb_pkts);
+	nb_rx = rte_eth_rx_burst(port_id, queue_id, rx_pkts, nb_pkts);
 	if (unlikely(nb_rx == 0))
 		return SPPWK_RET_OK;
 
@@ -393,7 +393,7 @@ sppwk_eth_vlan_rx_burst(uint16_t port_id,
 /* Wrapper function for rte_eth_tx_burst() with VLAN feature. */
 uint16_t
 sppwk_eth_vlan_tx_burst(uint16_t port_id,
-		uint16_t queue_id __attribute__ ((unused)),
+		uint16_t queue_id,
 		struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 {
 	uint16_t nb_tx;
@@ -404,7 +404,7 @@ sppwk_eth_vlan_tx_burst(uint16_t port_id,
 	if (unlikely(nb_tx == 0))
 		return SPPWK_RET_OK;
 
-	return rte_eth_tx_burst(port_id, 0, tx_pkts, nb_tx);
+	return rte_eth_tx_burst(port_id, queue_id, tx_pkts, nb_tx);
 }
 
 #ifdef SPP_RINGLATENCYSTATS_ENABLE
diff --git a/src/shared/secondary/spp_worker_th/port_capability.h b/src/shared/secondary/spp_worker_th/port_capability.h
index 284ba30..17d3a16 100644
--- a/src/shared/secondary/spp_worker_th/port_capability.h
+++ b/src/shared/secondary/spp_worker_th/port_capability.h
@@ -62,7 +62,7 @@ void sppwk_update_port_dir(const struct sppwk_comp_info *comp);
  * Wrapper function for rte_eth_rx_burst() with VLAN feature.
  *
  * @param[in] port_id Etherdev ID.
- * @param[in] queue_id RX queue ID, but fixed value 0 in SPP.
+ * @param[in] queue_id RX queue ID.
  * @param[in] rx_pkts Pointers to mbuf should be enough to store nb_pkts.
  * @param nb_pkts Maximum number of RX packets.
  * @return Number of RX packets as number of pointers to mbuf.
@@ -74,7 +74,7 @@ uint16_t sppwk_eth_vlan_rx_burst(uint16_t port_id, uint16_t queue_id,
  * Wrapper function for rte_eth_tx_burst() with VLAN feature.
  *
  * @param port_id Etherdev ID.
- * @param[in] queue_id TX queue ID, but fixed value 0 in SPP.
+ * @param[in] queue_id TX queue ID.
  * @param[in] tx_pkts Pointers to mbuf should be enough to store nb_pkts.
  * @param nb_pkts Maximum number of TX packets.
  * @return Number of TX packets as number of pointers to mbuf.
diff --git a/src/shared/secondary/spp_worker_th/vf_deps.h b/src/shared/secondary/spp_worker_th/vf_deps.h
index f2ea62a..dadcf7a 100644
--- a/src/shared/secondary/spp_worker_th/vf_deps.h
+++ b/src/shared/secondary/spp_worker_th/vf_deps.h
@@ -24,7 +24,12 @@
 struct mac_classifier {
 	struct rte_hash *cls_tbl;  /* Hash table for MAC classification. */
 	int nof_cls_ports;  /* Num of ports classified validly. */
-	int cls_ports[RTE_MAX_ETHPORTS];  /* Ports for classification. */
+	/**
+	 * Ports for classification.
+	 * For multi-queue case, size of ports should be
+	 * RTE_MAX_QUEUES_PER_PORT. RTE_MAX_ETHPORTS is not enough.
+	 */
+	int cls_ports[RTE_MAX_QUEUES_PER_PORT];
 	int default_cls_idx;  /* Default index for classification. */
 };
 
@@ -33,6 +38,7 @@ struct mac_classifier {
 struct cls_port_info {
 	enum port_type iface_type;
 	int iface_no;   /* Index of ports handled by classifier. */
+	int queue_no;   /* Index of queue per port handled by classifier. */
 	int iface_no_global;  /* ID for interface generated by spp_vf */
 	uint16_t ethdev_port_id;  /* Ethdev port ID. */
 	uint16_t nof_pkts;  /* Number of packets in pkts[]. */
@@ -47,7 +53,12 @@ struct cls_comp_info {
 	int nof_tx_ports;  /* Number of TX ports info entries. */
 	/* Classifier has one RX port and several TX ports. */
 	struct cls_port_info rx_port_i;  /* RX port info classified. */
-	struct cls_port_info tx_ports_i[RTE_MAX_ETHPORTS];  /* TX info. */
+	/**
+	 * TX info.
+	 * For multi-queue case, size of ports should be
+	 * RTE_MAX_QUEUES_PER_PORT. RTE_MAX_ETHPORTS is not enough.
+	 */
+	struct cls_port_info tx_ports_i[RTE_MAX_QUEUES_PER_PORT];
 };
 
 int add_core(const char *name, char **output,
diff --git a/src/shared/secondary/utils.c b/src/shared/secondary/utils.c
index c80a478..7cca4a4 100644
--- a/src/shared/secondary/utils.c
+++ b/src/shared/secondary/utils.c
@@ -5,6 +5,7 @@
 #include <stdint.h>
 #include "shared/common.h"
 #include "shared/secondary/utils.h"
+#include "shared/secondary/spp_worker_th/data_types.h"
 
 #define RTE_LOGTYPE_SHARED RTE_LOGTYPE_USER1
 
@@ -66,15 +67,18 @@ parse_client_id(int *cli_id, const char *cli_id_str)
 }
 
 /**
- * Retieve port type and ID from resource UID. For example, resource UID
- * 'ring:0' is  parsed to retrieve port tyep 'ring' and ID '0'.
+ * Retieve port type, ID and queue id from resource UID. For example,
+ * resource UID 'ring:0' is  parsed to retrieve port type 'ring' and ID '0'.
  */
 int
-parse_resource_uid(char *str, char **port_type, int *port_id)
+parse_resource_uid(char *str, char **port_type, int *port_id,
+		uint16_t *queue_id)
 {
 	char *token;
 	char delim[] = ":";
 	char *endp;
+	uint16_t default_queue = DEFAULT_QUEUE_ID;
+	char *queue_id_str = NULL;
 
 	RTE_LOG(DEBUG, SHARED, "Parsing resource UID: '%s'\n", str);
 	if (strstr(str, delim) == NULL) {
@@ -86,13 +90,27 @@ parse_resource_uid(char *str, char **port_type, int *port_id)
 	*port_type = strtok(str, delim);
 
 	token = strtok(NULL, delim);
+	queue_id_str = strstr(token, DELIM_PHY_MQ);
 	*port_id = strtol(token, &endp, 10);
-
-	if (*endp) {
+	if ((queue_id_str == NULL && *endp) ||
+		(queue_id_str != NULL && endp != queue_id_str)) {
 		RTE_LOG(ERR, SHARED, "Bad integer value: %s\n", str);
 		return -1;
 	}
 
+	if (queue_id_str == NULL) {
+		*queue_id = default_queue;
+	} else {
+		*queue_id = strtol(&queue_id_str[strlen(DELIM_PHY_MQ)],
+			&endp, 10);
+
+		if (*endp) {
+			RTE_LOG(ERR, SHARED,
+				"queue_id is bad integer value: %s\n", str);
+			return -1;
+		}
+	}
+
 	return 0;
 }
 
diff --git a/src/shared/secondary/utils.h b/src/shared/secondary/utils.h
index eba221f..4ae0a8e 100644
--- a/src/shared/secondary/utils.h
+++ b/src/shared/secondary/utils.h
@@ -5,7 +5,8 @@
 #ifndef _SHARED_SECONDARY_UTILS_H_
 #define _SHARED_SECONDARY_UTILS_H_
 
-int parse_resource_uid(char *str, char **port_type, int *port_id);
+int parse_resource_uid(char *str, char **port_type, int *port_id,
+	uint16_t *queue_id);
 
 int spp_atoi(const char *str, int *val);
 
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 02/17] spp_vf: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (18 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 01/17] shared: add support of multi-queue x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 03/17] spp_mirror: " x-fn-spp-ml
                   ` (14 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To achieve hardware offload in secondary process, multi-queue should be
supported.
Multi-queue support is divided into several changes like following:
- add queue number parameter in port command
- parse newly added parameter
- add queue number parameter in status response

For spp_vf, code for forwarder, classifier and merger should be changed.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/vf/classifier.c    | 40 +++++++++++++++++++++++++++-------------
 src/vf/forwarder.c     | 12 +++++++-----
 src/vf/vf_cmd_runner.c | 32 ++++++++++++++++++++------------
 3 files changed, 54 insertions(+), 30 deletions(-)

diff --git a/src/vf/classifier.c b/src/vf/classifier.c
index 8a4c8fc..2d95289 100644
--- a/src/vf/classifier.c
+++ b/src/vf/classifier.c
@@ -199,7 +199,8 @@ log_classification(long clsd_idx, struct rte_mbuf *pkt,
 	else
 		sppwk_port_uid(iface_str,
 				clsd_data[clsd_idx].iface_type,
-				clsd_data[clsd_idx].iface_no_global);
+				clsd_data[clsd_idx].iface_no_global,
+				clsd_data[clsd_idx].queue_no);
 
 	RTE_LOG_DP(DEBUG, VF_CLS,
 			"[%s]Classification(%s:%d). d_addr=%s, "
@@ -227,7 +228,8 @@ log_entry(long clsd_idx, uint16_t vid, const char *mac_addr_str,
 	else
 		sppwk_port_uid(iface_str,
 				clsd_data[clsd_idx].iface_type,
-				clsd_data[clsd_idx].iface_no_global);
+				clsd_data[clsd_idx].iface_no_global,
+				clsd_data[clsd_idx].queue_no);
 
 	RTE_LOG_DP(DEBUG, VF_CLS,
 			"[%s]Entry(%s:%d). vid=%hu, mac_addr=%s, iface=%s\n",
@@ -320,6 +322,7 @@ init_component_info(struct cls_comp_info *cmp_info,
 	if (wk_comp_info->nof_rx == 0) {
 		cls_rx_port_info->iface_type = UNDEF;
 		cls_rx_port_info->iface_no = 0;
+		cls_rx_port_info->queue_no = DEFAULT_QUEUE_ID;
 		cls_rx_port_info->iface_no_global = 0;
 		cls_rx_port_info->ethdev_port_id = 0;
 		cls_rx_port_info->nof_pkts = 0;
@@ -327,6 +330,8 @@ init_component_info(struct cls_comp_info *cmp_info,
 		cls_rx_port_info->iface_type =
 			wk_comp_info->rx_ports[0]->iface_type;
 		cls_rx_port_info->iface_no = 0;
+		cls_rx_port_info->queue_no =
+			wk_comp_info->rx_ports[0]->queue_no;
 		cls_rx_port_info->iface_no_global =
 			wk_comp_info->rx_ports[0]->iface_no;
 		cls_rx_port_info->ethdev_port_id =
@@ -344,6 +349,7 @@ init_component_info(struct cls_comp_info *cmp_info,
 		/* store ports information */
 		cls_tx_ports_info[i].iface_type = tx_port->iface_type;
 		cls_tx_ports_info[i].iface_no = i;
+		cls_tx_ports_info[i].queue_no = tx_port->queue_no;
 		cls_tx_ports_info[i].iface_no_global = tx_port->iface_no;
 		cls_tx_ports_info[i].ethdev_port_id = tx_port->ethdev_port_id;
 		cls_tx_ports_info[i].nof_pkts = 0;
@@ -374,10 +380,10 @@ init_component_info(struct cls_comp_info *cmp_info,
 			mac_cls->default_cls_idx = i;
 			RTE_LOG(INFO, VF_CLS,
 					"default classified. vid=%hu, "
-					"iface_type=%d, iface_no=%d, "
+					"iface_type=%d, iface_no=%d, queue_no=%d, "
 					"ethdev_port_id=%d\n",
 					vid, tx_port->iface_type,
-					tx_port->iface_no,
+					tx_port->iface_no, tx_port->queue_no,
 					tx_port->ethdev_port_id);
 			continue;
 		}
@@ -401,9 +407,10 @@ init_component_info(struct cls_comp_info *cmp_info,
 		RTE_LOG(INFO, VF_CLS,
 				"Add entry to classifier table. "
 				"vid=%hu, mac_addr=%s, iface_type=%d, "
-				"iface_no=%d, ethdev_port_id=%d\n",
+				"iface_no=%d, queue_no=%d, ethdev_port_id=%d\n",
 				vid, mac_addr_str, tx_port->iface_type,
-				tx_port->iface_no, tx_port->ethdev_port_id);
+				tx_port->iface_no, tx_port->queue_no,
+				tx_port->ethdev_port_id);
 	}
 
 	return SPPWK_RET_OK;
@@ -422,8 +429,9 @@ transmit_packets(struct cls_port_info *clsd_data)
 			clsd_data->iface_type, clsd_data->iface_no,
 			0, clsd_data->pkts, clsd_data->nof_pkts);
 #else
-	n_tx = sppwk_eth_vlan_tx_burst(clsd_data->ethdev_port_id, 0,
-			clsd_data->pkts, clsd_data->nof_pkts);
+	n_tx = sppwk_eth_vlan_tx_burst(clsd_data->ethdev_port_id,
+			clsd_data->queue_no, clsd_data->pkts,
+			clsd_data->nof_pkts);
 #endif
 
 	/* free cannot transmit packets */
@@ -467,11 +475,12 @@ push_packet(struct rte_mbuf *pkt, struct cls_port_info *clsd_data)
 	if (unlikely(clsd_data->nof_pkts == MAX_PKT_BURST)) {
 		RTE_LOG(DEBUG, VF_CLS,
 				"transmit packets (buffer is filled). "
-				"iface_type=%d, iface_no={%d,%d}, "
+				"iface_type=%d, iface_no={%d,%d}, queue_no=%d, "
 				"tx_port=%hu, nof_pkts=%hu\n",
 				clsd_data->iface_type,
 				clsd_data->iface_no_global,
 				clsd_data->iface_no,
+				clsd_data->queue_no,
 				clsd_data->ethdev_port_id,
 				clsd_data->nof_pkts);
 		transmit_packets(clsd_data);
@@ -754,8 +763,8 @@ classify_packets(int comp_id)
 			clsd_data_rx->iface_type, clsd_data_rx->iface_no,
 			0, rx_pkts, MAX_PKT_BURST);
 #else
-	n_rx = sppwk_eth_vlan_rx_burst(clsd_data_rx->ethdev_port_id, 0,
-			rx_pkts, MAX_PKT_BURST);
+	n_rx = sppwk_eth_vlan_rx_burst(clsd_data_rx->ethdev_port_id,
+			clsd_data_rx->queue_no, rx_pkts, MAX_PKT_BURST);
 #endif
 	if (unlikely(n_rx == 0))
 		return SPPWK_RET_OK;
@@ -776,8 +785,8 @@ get_classifier_status(unsigned int lcore_id, int id,
 	struct cls_mng_info *mng_info;
 	struct cls_comp_info *cmp_info;
 	struct cls_port_info *port_info;
-	struct sppwk_port_idx rx_ports[RTE_MAX_ETHPORTS];
-	struct sppwk_port_idx tx_ports[RTE_MAX_ETHPORTS];
+	struct sppwk_port_idx rx_ports[RTE_MAX_QUEUES_PER_PORT];
+	struct sppwk_port_idx tx_ports[RTE_MAX_QUEUES_PER_PORT];
 
 	mng_info = cls_mng_info_list + id;
 	if (!is_used_mng_info(mng_info)) {
@@ -796,6 +805,7 @@ get_classifier_status(unsigned int lcore_id, int id,
 		nof_rx = 1;
 		rx_ports[0].iface_type = cmp_info->rx_port_i.iface_type;
 		rx_ports[0].iface_no = cmp_info->rx_port_i.iface_no_global;
+		rx_ports[0].queue_no = cmp_info->rx_port_i.queue_no;
 	}
 
 	memset(tx_ports, 0x00, sizeof(tx_ports));
@@ -803,6 +813,7 @@ get_classifier_status(unsigned int lcore_id, int id,
 	for (i = 0; i < nof_tx; i++) {
 		tx_ports[i].iface_type = port_info[i].iface_type;
 		tx_ports[i].iface_no = port_info[i].iface_no_global;
+		tx_ports[i].queue_no = port_info[i].queue_no;
 	}
 
 	/* Set the information with the function specified by the command. */
@@ -840,6 +851,8 @@ add_mac_entry(struct classifier_table_params *params,
 				mac_cls->default_cls_idx)->iface_type;
 		port.iface_no = (port_info +
 				mac_cls->default_cls_idx)->iface_no_global;
+		port.queue_no = (port_info +
+				mac_cls->default_cls_idx)->queue_no;
 
 		LOG_ENT((long)mac_cls->default_cls_idx, vid,
 				SPPWK_TERM_DEFAULT, cmp_info, port_info);
@@ -863,6 +876,7 @@ add_mac_entry(struct classifier_table_params *params,
 
 		port.iface_type = (port_info + (long)data)->iface_type;
 		port.iface_no = (port_info + (long)data)->iface_no_global;
+		port.queue_no = (port_info + (long)data)->queue_no;
 
 		LOG_ENT((long)data, vid, mac_addr_str, cmp_info, port_info);
 
diff --git a/src/vf/forwarder.c b/src/vf/forwarder.c
index 04bf5c3..66e1682 100644
--- a/src/vf/forwarder.c
+++ b/src/vf/forwarder.c
@@ -62,8 +62,8 @@ get_forwarder_status(unsigned int lcore_id, int id,
 	const char *component_type = NULL;
 	struct forward_info *fwd_info = &g_forward_info[id];
 	struct forward_path *fwd_path = &fwd_info->path[fwd_info->ref_index];
-	struct sppwk_port_idx rx_ports[RTE_MAX_ETHPORTS];
-	struct sppwk_port_idx tx_ports[RTE_MAX_ETHPORTS];
+	struct sppwk_port_idx rx_ports[RTE_MAX_QUEUES_PER_PORT];
+	struct sppwk_port_idx tx_ports[RTE_MAX_QUEUES_PER_PORT];
 
 	if (unlikely(fwd_path->wk_type == SPPWK_TYPE_NONE)) {
 		RTE_LOG(ERR, FORWARD,
@@ -82,12 +82,14 @@ get_forwarder_status(unsigned int lcore_id, int id,
 	for (cnt = 0; cnt < fwd_path->nof_rx; cnt++) {
 		rx_ports[cnt].iface_type = fwd_path->ports[cnt].rx.iface_type;
 		rx_ports[cnt].iface_no = fwd_path->ports[cnt].rx.iface_no;
+		rx_ports[cnt].queue_no = fwd_path->ports[cnt].rx.queue_no;
 	}
 
 	memset(tx_ports, 0x00, sizeof(tx_ports));
 	for (cnt = 0; cnt < fwd_path->nof_tx; cnt++) {
 		tx_ports[cnt].iface_type = fwd_path->ports[cnt].tx.iface_type;
 		tx_ports[cnt].iface_no = fwd_path->ports[cnt].tx.iface_no;
+		tx_ports[cnt].queue_no = fwd_path->ports[cnt].tx.queue_no;
 	}
 
 	/* Set the information with the function specified by the command. */
@@ -217,8 +219,8 @@ forward_packets(int id)
 				rx->iface_type, rx->iface_no, 0,
 				bufs, MAX_PKT_BURST);
 #else
-		nb_rx = sppwk_eth_vlan_rx_burst(rx->ethdev_port_id, 0,
-				bufs, MAX_PKT_BURST);
+		nb_rx = sppwk_eth_vlan_rx_burst(rx->ethdev_port_id,
+				rx->queue_no, bufs, MAX_PKT_BURST);
 #endif
 		if (unlikely(nb_rx == 0))
 			continue;
@@ -231,7 +233,7 @@ forward_packets(int id)
 					tx->iface_no, 0, bufs, nb_rx);
 #else
 			nb_tx = sppwk_eth_vlan_tx_burst(tx->ethdev_port_id,
-					0, bufs, nb_rx);
+					tx->queue_no, bufs, nb_rx);
 #endif
 
 		/* Discard remained packets to release mbuf */
diff --git a/src/vf/vf_cmd_runner.c b/src/vf/vf_cmd_runner.c
index 305c4a2..c5e8278 100644
--- a/src/vf/vf_cmd_runner.c
+++ b/src/vf/vf_cmd_runner.c
@@ -43,8 +43,9 @@ update_cls_table(enum sppwk_action wk_action,
 	struct sppwk_port_info *port_info;
 
 	RTE_LOG(DEBUG, VF_CMD_RUNNER, "Called __func__ with "
-			"type `mac`, mac_addr `%s`, and port `%d:%d`.\n",
-			mac_str, port->iface_type, port->iface_no);
+			"type `mac`, mac_addr `%s`, and port `%d:%d nq %d`.\n",
+			mac_str, port->iface_type, port->iface_no,
+			port->queue_no);
 
 	mac_int64 = sppwk_convert_mac_str_to_int64(mac_str);
 	if (unlikely(mac_int64 == -1)) {
@@ -54,15 +55,18 @@ update_cls_table(enum sppwk_action wk_action,
 	}
 	mac_uint64 = (uint64_t)mac_int64;
 
-	port_info = get_sppwk_port(port->iface_type, port->iface_no);
+	port_info = get_sppwk_port(port->iface_type, port->iface_no,
+			port->queue_no);
 	if (unlikely(port_info == NULL)) {
-		RTE_LOG(ERR, VF_CMD_RUNNER, "Failed to get port %d:%d.\n",
-				port->iface_type, port->iface_no);
+		RTE_LOG(ERR, VF_CMD_RUNNER, "Failed to get port %d:%d nq %d.\n",
+				port->iface_type, port->iface_no,
+				port->queue_no);
 		return SPPWK_RET_NG;
 	}
 	if (unlikely(port_info->iface_type == UNDEF)) {
-		RTE_LOG(ERR, VF_CMD_RUNNER, "Port %d:%d doesn't exist.\n",
-				port->iface_type, port->iface_no);
+		RTE_LOG(ERR, VF_CMD_RUNNER, "Port %d:%d nq %d doesn't exist.\n",
+				port->iface_type, port->iface_no,
+				port->queue_no);
 		return SPPWK_RET_NG;
 	}
 
@@ -88,15 +92,17 @@ update_cls_table(enum sppwk_action wk_action,
 		if (unlikely(port_info->cls_attrs.vlantag.vid !=
 				ETH_VLAN_ID_MAX)) {
 			/* TODO(yasufum) why two vids are required in msg ? */
-			RTE_LOG(ERR, VF_CMD_RUNNER, "Used port %d:%d, vid %d != %d.\n",
+			RTE_LOG(ERR, VF_CMD_RUNNER, "Used port %d:%d nq %d, vid %d != %d.\n",
 					port->iface_type, port->iface_no,
+					port->queue_no,
 					port_info->cls_attrs.vlantag.vid, vid);
 			return SPPWK_RET_NG;
 		}
 		if (unlikely(port_info->cls_attrs.mac_addr != 0)) {
 			/* TODO(yasufum) why two macs are required in msg ? */
-			RTE_LOG(ERR, VF_CMD_RUNNER, "Used port %d:%d, mac %s != %s.\n",
+			RTE_LOG(ERR, VF_CMD_RUNNER, "Used port %d:%d nq %d, mac %s != %s.\n",
 					port->iface_type, port->iface_no,
+					port->queue_no,
 					port_info->cls_attrs.mac_addr_str,
 					mac_str);
 			return SPPWK_RET_NG;
@@ -277,7 +283,8 @@ update_port(enum sppwk_action wk_action,
 	sppwk_get_mng_data(NULL, &comp_info_base, NULL, NULL,
 			&change_component, NULL);
 	comp_info = (comp_info_base + comp_lcore_id);
-	port_info = get_sppwk_port(port->iface_type, port->iface_no);
+	port_info = get_sppwk_port(port->iface_type, port->iface_no,
+			port->queue_no);
 	if (dir == SPPWK_PORT_DIR_RX) {
 		nof_ports = &comp_info->nof_rx;
 		ports = comp_info->rx_ports;
@@ -317,7 +324,7 @@ update_port(enum sppwk_action wk_action,
 			return SPPWK_RET_OK;
 		}
 
-		if (*nof_ports >= RTE_MAX_ETHPORTS) {
+		if (*nof_ports >= RTE_MAX_QUEUES_PER_PORT) {
 			RTE_LOG(ERR, VF_CMD_RUNNER, "Cannot assign port over the "
 				"maximum number.\n");
 			return SPPWK_RET_NG;
@@ -565,7 +572,8 @@ append_classifier_element_value(
 		return ret;
 	}
 
-	sppwk_port_uid(port_str, port->iface_type, port->iface_no);
+	sppwk_port_uid(port_str, port->iface_type, port->iface_no,
+			port->queue_no);
 
 	ret = append_json_str_value(&tmp_buff, "type",
 			CLS_TYPE_A_LIST[cls_type]);
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 03/17] spp_mirror: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (19 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 02/17] spp_vf: " x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 04/17] spp_pcap: " x-fn-spp-ml
                   ` (13 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

By changing common code under shared directory to achieve multi-queue,
existing code of spp_mirror should be changed also for successful
compile.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/mirror/mir_cmd_runner.c |  3 ++-
 src/mirror/spp_mirror.c     | 12 ++++++++----
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/src/mirror/mir_cmd_runner.c b/src/mirror/mir_cmd_runner.c
index fb1695f..5153b65 100644
--- a/src/mirror/mir_cmd_runner.c
+++ b/src/mirror/mir_cmd_runner.c
@@ -156,7 +156,8 @@ update_port(enum sppwk_action wk_action,
 	sppwk_get_mng_data(NULL, &comp_info_base, NULL, NULL,
 			&change_component, NULL);
 	comp_info = (comp_info_base + comp_lcore_id);
-	port_info = get_sppwk_port(port->iface_type, port->iface_no);
+	port_info = get_sppwk_port(port->iface_type, port->iface_no,
+			port->queue_no);
 	if (dir == SPPWK_PORT_DIR_RX) {
 		nof_ports = &comp_info->nof_rx;
 		ports = comp_info->rx_ports;
diff --git a/src/mirror/spp_mirror.c b/src/mirror/spp_mirror.c
index e5852aa..2bfbf66 100644
--- a/src/mirror/spp_mirror.c
+++ b/src/mirror/spp_mirror.c
@@ -340,7 +340,8 @@ mirror_proc(int id)
 	nb_rx = sppwk_eth_ring_stats_rx_burst(rx->ethdev_port_id,
 			rx->iface_type, rx->iface_no, 0, bufs, MAX_PKT_BURST);
 #else
-	nb_rx = rte_eth_rx_burst(rx->ethdev_port_id, 0, bufs, MAX_PKT_BURST);
+	nb_rx = rte_eth_rx_burst(rx->ethdev_port_id, rx->queue_no, bufs,
+			MAX_PKT_BURST);
 #endif
 
 	if (unlikely(nb_rx == 0))
@@ -403,8 +404,8 @@ mirror_proc(int id)
 					tx->ethdev_port_id, tx->iface_type,
 					tx->iface_no, 0, copybufs, cnt);
 #else
-			nb_tx2 = rte_eth_tx_burst(tx->ethdev_port_id, 0,
-					copybufs, cnt);
+			nb_tx2 = rte_eth_tx_burst(tx->ethdev_port_id,
+					tx->queue_no, copybufs, cnt);
 #endif
 	}
 
@@ -415,7 +416,8 @@ mirror_proc(int id)
 		nb_tx1 = sppwk_eth_ring_stats_tx_burst(tx->ethdev_port_id,
 				tx->iface_type, tx->iface_no, 0, bufs, nb_rx);
 #else
-		nb_tx1 = rte_eth_tx_burst(tx->ethdev_port_id, 0, bufs, nb_rx);
+		nb_tx1 = rte_eth_tx_burst(tx->ethdev_port_id, tx->queue_no,
+				bufs, nb_rx);
 #endif
 	nb_tx = nb_tx1;
 
@@ -685,12 +687,14 @@ get_mirror_status(unsigned int lcore_id, int id,
 	for (cnt = 0; cnt < path->nof_rx; cnt++) {
 		rx_ports[cnt].iface_type = path->ports[cnt].rx.iface_type;
 		rx_ports[cnt].iface_no   = path->ports[cnt].rx.iface_no;
+		rx_ports[cnt].queue_no   = path->ports[cnt].rx.queue_no;
 	}
 
 	memset(tx_ports, 0x00, sizeof(tx_ports));
 	for (cnt = 0; cnt < path->nof_tx; cnt++) {
 		tx_ports[cnt].iface_type = path->ports[cnt].tx.iface_type;
 		tx_ports[cnt].iface_no   = path->ports[cnt].tx.iface_no;
+		tx_ports[cnt].queue_no   = path->ports[cnt].tx.queue_no;
 	}
 
 	/* Set the information with the function specified by the command. */
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 04/17] spp_pcap: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (20 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 03/17] spp_mirror: " x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 05/17] spp_primary: " x-fn-spp-ml
                   ` (12 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

By changing common code under shared directory to achieve multi-queue,
existing code of spp_pcap should be changed also for successful compile.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/pcap/cmd_runner.c |   3 +-
 src/pcap/cmd_utils.c  |  73 +++++++++++++++++++++++++-----
 src/pcap/cmd_utils.h  |  19 +++++++-
 src/pcap/spp_pcap.c   | 102 +++++++++++++++++++++++++++++++++++++-----
 4 files changed, 171 insertions(+), 26 deletions(-)

diff --git a/src/pcap/cmd_runner.c b/src/pcap/cmd_runner.c
index 85c02cd..ea6ec4d 100644
--- a/src/pcap/cmd_runner.c
+++ b/src/pcap/cmd_runner.c
@@ -398,7 +398,8 @@ append_port_entry(char **output, const struct sppwk_port_idx *port,
 		return SPPWK_RET_NG;
 	}
 
-	sppwk_port_uid(port_str, port->iface_type, port->iface_no);
+	sppwk_port_uid(port_str, port->iface_type, port->iface_no,
+			port->queue_no);
 	ret = append_json_str_value("port", &tmp_buff, port_str);
 	if (unlikely(ret < SPPWK_RET_OK))
 		return SPPWK_RET_NG;
diff --git a/src/pcap/cmd_utils.c b/src/pcap/cmd_utils.c
index 5aa131b..383f783 100644
--- a/src/pcap/cmd_utils.c
+++ b/src/pcap/cmd_utils.c
@@ -10,6 +10,7 @@
 
 #include "cmd_utils.h"
 #include "shared/secondary/return_codes.h"
+#include "shared/common.h"
 
 #define RTE_LOGTYPE_PCAP_UTILS RTE_LOGTYPE_USER2
 
@@ -149,13 +150,13 @@ stop_process(int signal)
  * It returns NULL value if given type is invalid.
  */
 struct sppwk_port_info *
-get_iface_info(enum port_type iface_type, int iface_no)
+get_iface_info(enum port_type iface_type, int iface_no, int queue_no)
 {
 	struct iface_info *iface_info = g_mng_data_addr.p_iface_info;
 
 	switch (iface_type) {
 	case PHY:
-		return &iface_info->phy[iface_no];
+		return &iface_info->phy[iface_no][queue_no];
 	case RING:
 		return &iface_info->ring[iface_no];
 	default:
@@ -172,16 +173,27 @@ static void
 init_iface_info(void)
 {
 	int port_cnt;  /* increment ether ports */
+	int queue_cnt;  /* increment queues per port */
 	struct iface_info *p_iface_info = g_mng_data_addr.p_iface_info;
 	memset(p_iface_info, 0x00, sizeof(struct iface_info));
 	for (port_cnt = 0; port_cnt < RTE_MAX_ETHPORTS; port_cnt++) {
-		p_iface_info->phy[port_cnt].iface_type = UNDEF;
-		p_iface_info->phy[port_cnt].iface_no   = port_cnt;
-		p_iface_info->phy[port_cnt].ethdev_port_id  = -1;
-		p_iface_info->phy[port_cnt].cls_attrs.vlantag.vid =
-				ETH_VLAN_ID_MAX;
+		for (queue_cnt = 0; queue_cnt < RTE_MAX_QUEUES_PER_PORT;
+				queue_cnt++) {
+			p_iface_info->phy[port_cnt][queue_cnt].iface_type =
+					UNDEF;
+			p_iface_info->phy[port_cnt][queue_cnt].iface_no =
+					port_cnt;
+			p_iface_info->phy[port_cnt][queue_cnt].queue_no =
+					queue_cnt;
+			p_iface_info->phy[port_cnt][queue_cnt]
+					.ethdev_port_id = -1;
+			p_iface_info->phy[port_cnt][queue_cnt]
+					.cls_attrs.vlantag.vid =
+					ETH_VLAN_ID_MAX;
+		}
 		p_iface_info->ring[port_cnt].iface_type = UNDEF;
 		p_iface_info->ring[port_cnt].iface_no   = port_cnt;
+		p_iface_info->ring[port_cnt].queue_no   = DEFAULT_QUEUE_ID;
 		p_iface_info->ring[port_cnt].ethdev_port_id  = -1;
 		p_iface_info->ring[port_cnt].cls_attrs.vlantag.vid =
 				ETH_VLAN_ID_MAX;
@@ -206,7 +218,7 @@ static int
 init_host_port_info(void)
 {
 	int port_type, port_id;
-	int i, ret;
+	int i, ret, queue_id;
 	int nof_phys = 0;
 	char dev_name[RTE_DEV_NAME_MAX_LEN] = { 0 };
 	struct iface_info *p_iface_info = g_mng_data_addr.p_iface_info;
@@ -227,8 +239,13 @@ init_host_port_info(void)
 
 		switch (port_type) {
 		case PHY:
-			p_iface_info->phy[port_id].iface_type = port_type;
-			p_iface_info->phy[port_id].ethdev_port_id = port_id;
+			for (queue_id = 0; queue_id < RTE_MAX_QUEUES_PER_PORT;
+					queue_id++) {
+				p_iface_info->phy[port_id][queue_id]
+						.iface_type = port_type;
+				p_iface_info->phy[port_id][queue_id]
+						.ethdev_port_id = port_id;
+			}
 			break;
 		case RING:
 			p_iface_info->ring[port_id].iface_type = port_type;
@@ -263,9 +280,11 @@ init_mng_data(void)
  * Generate a formatted string of combination from interface type and
  * number and assign to given 'port'
  */
-int sppwk_port_uid(char *port, enum port_type iface_type, int iface_no)
+int sppwk_port_uid(char *port, enum port_type iface_type, int iface_no,
+		int queue_no)
 {
 	const char *iface_type_str;
+	int max_queue;
 
 	switch (iface_type) {
 	case PHY:
@@ -278,7 +297,15 @@ int sppwk_port_uid(char *port, enum port_type iface_type, int iface_no)
 		return SPPWK_RET_NG;
 	}
 
-	sprintf(port, "%s:%d", iface_type_str, iface_no);
+	max_queue = get_port_max_queues(iface_type, iface_no);
+	if (unlikely(max_queue == SPPWK_RET_NG))
+		return SPPWK_RET_NG;
+
+	if (max_queue <= 1)
+		sprintf(port, "%s:%d", iface_type_str, iface_no);
+	else
+		sprintf(port, "%s:%d nq %d",
+			iface_type_str, iface_no, queue_no);
 
 	return SPPWK_RET_OK;
 }
@@ -319,3 +346,25 @@ void spp_get_mng_data_addr(struct iface_info **iface_p,
 		*capture_status_p = g_mng_data_addr.p_capture_status;
 
 }
+
+/* Returns a larger number of queues of RX or TX port as the maximum number */
+int
+get_port_max_queues(const enum port_type iface_type, int iface_no)
+{
+	if (unlikely(iface_no > RTE_MAX_ETHPORTS) || unlikely(iface_no < 0))
+		return SPPWK_RET_NG;
+
+	if (iface_type != PHY)
+		return 1;
+
+	const struct port_info *ports = NULL;
+	const struct rte_memzone *mz;
+	mz = rte_memzone_lookup(MZ_PORT_INFO);
+	ports = mz->addr;
+	int max_q_rx = ports->queue_info[iface_no].rxq;
+	int max_q_tx = ports->queue_info[iface_no].txq;
+	if (max_q_rx > max_q_tx)
+		return max_q_rx;
+	else
+		return max_q_tx;
+}
diff --git a/src/pcap/cmd_utils.h b/src/pcap/cmd_utils.h
index 95b8a8a..94e7087 100644
--- a/src/pcap/cmd_utils.h
+++ b/src/pcap/cmd_utils.h
@@ -87,12 +87,14 @@ void stop_process(int signal);
  *  Interface type to be validated.
  * @param iface_no
  *  Interface number to be validated.
+ * @param queue_no
+ *  Interface queue number to be validated.
  *
  * @retval !NULL  sppwk_port_info.
  * @retval NULL   failed.
  */
 struct sppwk_port_info *
-get_iface_info(enum port_type iface_type, int iface_no);
+get_iface_info(enum port_type iface_type, int iface_no, int queue_no);
 
 /**
  * Setup management info for spp_vf
@@ -105,15 +107,19 @@ struct core_info *get_core_info(unsigned int lcore_id);
 /**
  * Port type to string
  *
+ * TODO(smurakami) Change spp_pcap to use queue_no for supporting multi-queue
+ *
  * @param port String of port type to be converted.
  * @param iface_type Interface type.
  * @param iface_no Interface number.
+ * @param queue_no Queue number of interface.
  * @retval SPPWK_RET_OK If succeeded.
  * @retval SPPWK_RET_NG If failed.
  */
 /* TODO(yasufum) consider to merge to shared. */
 int
-sppwk_port_uid(char *port, enum port_type iface_type, int iface_no);
+sppwk_port_uid(char *port, enum port_type iface_type, int iface_no,
+		int queue_no);
 
 /**
  * Set mange data address
@@ -143,4 +149,13 @@ void spp_get_mng_data_addr(struct iface_info **iface_p,
 			   int **capture_request_p,
 			   int **capture_status_p);
 
+/**
+ * Returns max queue number of the target port.
+ *
+ * @param[in] iface_type Interface type such as PHY or so.
+ * @param[in] iface_no Interface number.
+ * @return Max queue number if succeeded, or SPPWK_RET_NG if failed.
+ */
+int get_port_max_queues(const enum port_type iface_type, int iface_no);
+
 #endif
diff --git a/src/pcap/spp_pcap.c b/src/pcap/spp_pcap.c
index ab4d796..1f20e1e 100644
--- a/src/pcap/spp_pcap.c
+++ b/src/pcap/spp_pcap.c
@@ -185,7 +185,7 @@ usage(const char *progname)
 		" [--fsize MAX_FILE_SIZE]\n"
 		" --client-id CLIENT_ID: My client ID\n"
 		" -s IPADDR:PORT: IP addr and sec port for spp-ctl\n"
-		" -c: Captured port (e.g. 'phy:0' or 'ring:1')\n"
+		" -c: Captured port (e.g. 'phy:0', 'phy:0 nq 1' or 'ring:1')\n"
 		" --out-dir: Output dir (Default is /tmp)\n"
 		" --fsize: Maximum captured file size (Default is 1GiB)\n"
 		, progname);
@@ -209,12 +209,15 @@ parse_fsize(const char *fsize_str, uint64_t *fsize)
 
 /* Parse `-c` option for captured port and get the port type and ID */
 static int
-parse_captured_port(const char *port_str, enum port_type *iface_type,
-			int *iface_no)
+parse_captured_port(const char *port_str, int option_index,
+			const int argcopt, char *argvopt[],
+			enum port_type *iface_type, int *iface_no,
+			int *queue_no)
 {
 	enum port_type type = UNDEF;
 	const char *no_str = NULL;
 	char *endptr = NULL;
+	int q_no = 0; /* Init value is default queue_no */
 
 	/* Find out which type of interface from resource UID */
 	if (strncmp(port_str, SPPWK_PHY_STR ":",
@@ -243,11 +246,45 @@ parse_captured_port(const char *port_str, enum port_type *iface_type,
 		return SPPWK_RET_NG;
 	}
 
+	/* Convert to numeric if queue_no */
+	if (type == PHY && option_index + 1 <= argcopt &&
+		!strcmp(argvopt[option_index], DELIM_PHY_MQ)) {
+
+		no_str = argvopt[option_index + 1];
+
+		q_no = strtol(no_str, &endptr, 0);
+		if (unlikely(no_str == endptr) || unlikely(*endptr != '\0')) {
+			/* No Queue number */
+			RTE_LOG(ERR, SPP_PCAP,
+				"No queue number. (port = %s)\n", port_str);
+			return SPPWK_RET_NG;
+		}
+
+		/* Check max num of queue_no */
+		if (q_no >= get_port_max_queues(type, ret_no)) {
+			RTE_LOG(ERR, SPP_PCAP,
+				"queue_no exceeds the definition of primary.\n"
+				);
+			return SPPWK_RET_NG;
+		}
+
+	} else {
+		if (get_port_max_queues(type, ret_no) > 1) {
+			RTE_LOG(ERR, SPP_PCAP,
+				"Queue_no is required for the specified "
+				"physical port because of multi-queue.\n"
+				);
+			return SPPWK_RET_NG;
+		}
+	}
+
 	*iface_type = type;
 	*iface_no = ret_no;
+	*queue_no = q_no;
 
-	RTE_LOG(DEBUG, SPP_PCAP, "Port = %s => Type = %d No = %d\n",
-					port_str, *iface_type, *iface_no);
+	RTE_LOG(DEBUG, SPP_PCAP,
+		"Port = %s => Type = %d No = %d\n Queue = %d",
+		port_str, *iface_type, *iface_no, *queue_no);
 	return SPPWK_RET_OK;
 }
 
@@ -322,13 +359,23 @@ parse_app_args(int argc, char *argv[])
 			break;
 		case 'c':  /* captured port */
 			strcpy(cap_port_str, optarg);
-			if (parse_captured_port(optarg,
+			if (parse_captured_port(optarg, optind,
+					argcopt, argvopt,
 					&g_pcap_option.port_cap.iface_type,
-					&g_pcap_option.port_cap.iface_no) !=
+					&g_pcap_option.port_cap.iface_no,
+					&g_pcap_option.port_cap.queue_no) !=
 					SPPWK_RET_OK) {
 				usage(progname);
 				return SPPWK_RET_NG;
 			}
+			if (get_port_max_queues(
+				g_pcap_option.port_cap.iface_type,
+				g_pcap_option.port_cap.iface_no) > 1) {
+				snprintf(cap_port_str, PORT_STR_SIZE,
+					"%s nq %d",
+					optarg,
+					g_pcap_option.port_cap.queue_no);
+			}
 			port_flg = 1;
 			break;
 		case 's':  /* server addr */
@@ -384,6 +431,7 @@ spp_pcap_get_core_status(
 		memset(rx_ports, 0x00, sizeof(rx_ports));
 		rx_ports[0].iface_type = g_pcap_option.port_cap.iface_type;
 		rx_ports[0].iface_no   = g_pcap_option.port_cap.iface_no;
+		rx_ports[0].queue_no   = g_pcap_option.port_cap.queue_no;
 		rx_num = 1;
 		strcpy(role_type, "receive");
 	}
@@ -472,7 +520,22 @@ static int file_compression_operation(struct pcap_mng_info *info,
 			iface_type_str = SPPWK_PHY_STR;
 		else
 			iface_type_str = SPPWK_RING_STR;
-		snprintf(info->compress_file_name,
+
+		if (get_port_max_queues(
+			g_pcap_option.port_cap.iface_type,
+			g_pcap_option.port_cap.iface_no) > 1)
+			/* If multi-queue, add queue_no */
+			snprintf(info->compress_file_name,
+					PCAP_FNAME_STRLEN - 1,
+					"spp_pcap.%s.%s%dnq%d.%u.%u.pcap.lz4",
+					g_pcap_option.compress_file_date,
+					iface_type_str,
+					g_pcap_option.port_cap.iface_no,
+					g_pcap_option.port_cap.queue_no,
+					info->thread_no,
+					info->file_no);
+		else
+			snprintf(info->compress_file_name,
 					PCAP_FNAME_STRLEN - 1,
 					"spp_pcap.%s.%s%d.%u.%u.pcap.lz4",
 					g_pcap_option.compress_file_date,
@@ -527,7 +590,22 @@ static int file_compression_operation(struct pcap_mng_info *info,
 			iface_type_str = SPPWK_PHY_STR;
 		else
 			iface_type_str = SPPWK_RING_STR;
-		snprintf(info->compress_file_name,
+
+		if (get_port_max_queues(
+			g_pcap_option.port_cap.iface_type,
+			g_pcap_option.port_cap.iface_no) > 1)
+			/* If multi-queue, add queue_no */
+			snprintf(info->compress_file_name,
+					PCAP_FNAME_STRLEN - 1,
+					"spp_pcap.%s.%s%dnq%d.%u.%u.pcap.lz4",
+					g_pcap_option.compress_file_date,
+					iface_type_str,
+					g_pcap_option.port_cap.iface_no,
+					g_pcap_option.port_cap.queue_no,
+					info->thread_no,
+					info->file_no);
+		else
+			snprintf(info->compress_file_name,
 					PCAP_FNAME_STRLEN - 1,
 					"spp_pcap.%s.%s%d.%u.%u.pcap.lz4",
 					g_pcap_option.compress_file_date,
@@ -773,7 +851,8 @@ static int pcap_proc_receive(int lcore_id)
 	nb_rx = sppwk_eth_ring_stats_rx_burst(rx->ethdev_port_id,
 			rx->iface_type, rx->iface_no, 0, bufs, MAX_PCAP_BURST);
 #else
-	nb_rx = rte_eth_rx_burst(rx->ethdev_port_id, 0, bufs, MAX_PCAP_BURST);
+	nb_rx = rte_eth_rx_burst(rx->ethdev_port_id, rx->queue_no, bufs,
+			MAX_PCAP_BURST);
 #endif
 	if (unlikely(nb_rx == 0))
 		return SPPWK_RET_OK;
@@ -992,7 +1071,8 @@ main(int argc, char *argv[])
 		struct sppwk_port_info *port_cap = &g_pcap_option.port_cap;
 		struct sppwk_port_info *port_info = get_iface_info(
 						port_cap->iface_type,
-						port_cap->iface_no);
+						port_cap->iface_no,
+						port_cap->queue_no);
 		if (port_info == NULL) {
 			RTE_LOG(ERR, SPP_PCAP, "caputre port undefined.\n");
 			break;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 05/17] spp_primary: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (21 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 04/17] spp_pcap: " x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 06/17] spp_primary: add support of rte_flow x-fn-spp-ml
                   ` (11 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

Multi-queue support is provided for both tx and rx of physical NIC.
To initialize physical NIC with multi-queue mode, startup parameters of
the primary process should be newly introduced.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/Makefile |  10 ++++
 src/primary/args.c   | 138 ++++++++++++++++++++++++++++++++++++++++++-
 src/primary/args.h   |   3 +
 src/primary/init.c   |   8 ++-
 src/primary/init.h   |   3 +-
 src/primary/main.c   |  40 ++++++++++---
 6 files changed, 189 insertions(+), 13 deletions(-)

diff --git a/src/primary/Makefile b/src/primary/Makefile
index ace6228..35ae788 100644
--- a/src/primary/Makefile
+++ b/src/primary/Makefile
@@ -20,12 +20,22 @@ APP = spp_primary
 
 # TODO: revise to not use functions in secondary's.
 SPP_SEC_DIR = ../shared/secondary
+SPP_FLOW_DIR = ./flow
+SPP_FLOW_SRC = flow.c attr.c common.c
+SPP_FLOW_PTN_DIR = $(SPP_FLOW_DIR)/pattern
+SPP_FLOW_PTN_SRC = eth.c vlan.c
+SPP_FLOW_ACT_DIR = $(SPP_FLOW_DIR)/action
+SPP_FLOW_ACT_SRC = jump.c queue.c of_push_vlan.c of_set_vlan_vid.c
+SPP_FLOW_ACT_SRC += of_set_vlan_pcp.c
 
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 SRCS-y += ../shared/common.c ../shared/basic_forwarder.c ../shared/port_manager.c
 SRCS-y += $(SPP_SEC_DIR)/add_port.c
 SRCS-y += $(SPP_SEC_DIR)/utils.c
+SRCS-y += $(addprefix $(SPP_FLOW_DIR)/,$(SPP_FLOW_SRC))
+SRCS-y += $(addprefix $(SPP_FLOW_PTN_DIR)/,$(SPP_FLOW_PTN_SRC))
+SRCS-y += $(addprefix $(SPP_FLOW_ACT_DIR)/,$(SPP_FLOW_ACT_SRC))
 
 INC := $(wildcard *.h)
 
diff --git a/src/primary/args.c b/src/primary/args.c
index 346bcbf..e9d1c58 100644
--- a/src/primary/args.c
+++ b/src/primary/args.c
@@ -29,10 +29,12 @@ int do_forwarding;
 enum {
 	CMD_LINE_OPT_MIN_NUM = 256,
 	CMD_OPT_DISP_STATS,
+	CMD_OPT_PORT_NUM, /* For `--port-num` */
 };
 
 struct option lgopts[] = {
 	{"disp-stats", no_argument, NULL, CMD_OPT_DISP_STATS},
+	{"port-num", required_argument, NULL, CMD_OPT_PORT_NUM},
 	{0}
 };
 
@@ -45,9 +47,14 @@ static void
 usage(void)
 {
 	RTE_LOG(INFO, PRIMARY,
-	    "%s [EAL options] -- -p PORTMASK -n NUM_CLIENTS [-s NUM_SOCKETS]\n"
+	    "%s [EAL options] -- -p PORTMASK -n NUM_CLIENTS [-s NUM_SOCKETS]"
+		" [--port-num NUM_PORT"
+		" rxq NUM_RX_QUEUE txq NUM_TX_QUEUE]...\n"
 	    " -p PORTMASK: hexadecimal bitmask of ports to use\n"
 	    " -n NUM_RINGS: number of ring ports used from secondaries\n"
+		" --port-num NUM_PORT: number of ports for multi-queue setting\n"
+		" rxq NUM_RX_QUEUE: number of receive queues\n"
+		" txq NUM_TX_QUEUE number of transmit queues\n"
 	    , progname);
 }
 
@@ -131,6 +138,120 @@ parse_nof_rings(uint16_t *num_clients, const char *clients)
 	return 0;
 }
 
+/* Extract the number of queues from startup option. */
+static int
+parse_nof_queues(struct port_queue *arg_queues, const char *str_port_num,
+		int option_index, uint16_t max_ports, int argc, char *argv[])
+{
+	char *end = NULL;
+	unsigned long temp;
+	uint16_t port_num, rxq, txq;
+
+
+	if (str_port_num == NULL || *str_port_num == '\0') {
+		RTE_LOG(ERR, PRIMARY,
+			"PORT_NUM is not specified(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* Parameter check of port_num */
+	temp = strtoul(str_port_num, &end, 10);
+	if (end == NULL || *end != '\0') {
+		RTE_LOG(ERR, PRIMARY,
+			"PORT_NUM is not a number(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+	port_num = (uint16_t)temp;
+
+	if (port_num > max_ports) {
+		RTE_LOG(ERR, PRIMARY,
+			"PORT_NUM exceeds the number of available ports"
+			"(%s:%d)\n",
+			__func__, __LINE__);
+		return 1;
+	}
+
+	/* Check if both 'rxq' and 'txq' are inclued in parameter string. */
+	if (option_index + 3 > argc) {
+		RTE_LOG(ERR, PRIMARY,
+			"rxq NUM_RX_QUEUE txq NUM_TX_QUEUE is not specified"
+			"(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	if (strcmp(argv[option_index], "rxq")) {
+		RTE_LOG(ERR, PRIMARY,
+			"rxq is not specified in the --port_num option"
+			"(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* Parameter check of rxq */
+	temp = strtoul(argv[option_index + 1], &end, 10);
+	if (end == NULL || *end != '\0' || temp == 0) {
+		RTE_LOG(ERR, PRIMARY,
+			"NUM_RX_QUEUE is not a number(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+	rxq = (uint16_t)temp;
+
+	if (strcmp(argv[option_index + 2], "txq")) {
+		RTE_LOG(ERR, PRIMARY,
+			"txq is not specified in the --port_num option"
+			"(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* Parameter check of txq */
+	temp = strtoul(argv[option_index + 3], &end, 10);
+	if (end == NULL || *end != '\0' || temp == 0) {
+		RTE_LOG(ERR, PRIMARY,
+			"NUM_TX_QUEUE is not a number(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+	txq = (uint16_t)temp;
+
+	arg_queues[port_num].rxq = rxq;
+	arg_queues[port_num].txq = txq;
+
+	return 0;
+}
+
+/**
+ * Set the number of queues for port_id.
+ * If not specified number of queue is set as 1.
+ */
+static int
+set_nof_queues(struct port_info *ports, struct port_queue *arg_queues)
+{
+	int index;
+	uint16_t port_id, rxq, txq;
+
+	for (index = 0; index < ports->num_ports; index++) {
+		port_id = ports->id[index];
+
+		if (arg_queues[port_id].rxq == 0 ||
+			arg_queues[port_id].txq == 0) {
+			rxq = 1;
+			txq = 1;
+		} else {
+			rxq = arg_queues[port_id].rxq;
+			txq = arg_queues[port_id].txq;
+		}
+
+		ports->queue_info[index].rxq = rxq;
+		ports->queue_info[index].txq = txq;
+	}
+	return 0;
+}
+
 /**
  * The application specific arguments follow the DPDK-specific
  * arguments which are stripped by the DPDK init. This function
@@ -143,6 +264,7 @@ parse_app_args(uint16_t max_ports, int argc, char *argv[])
 	int option_index, opt;
 	char **argvopt = argv;
 	int ret;
+	struct port_queue arg_queues[RTE_MAX_ETHPORTS] = { 0 };
 
 	progname = argv[0];
 
@@ -171,6 +293,14 @@ parse_app_args(uint16_t max_ports, int argc, char *argv[])
 				return -1;
 			}
 			break;
+		case CMD_OPT_PORT_NUM:
+			ret = parse_nof_queues(arg_queues, optarg, optind,
+					max_ports, argc, argv);
+			if (ret != 0) {
+				usage();
+				return -1;
+			}
+			break;
 		default:
 			RTE_LOG(ERR,
 				PRIMARY, "ERROR: Unknown option '%c'\n", opt);
@@ -184,5 +314,11 @@ parse_app_args(uint16_t max_ports, int argc, char *argv[])
 		return -1;
 	}
 
+	ret = set_nof_queues(ports, arg_queues);
+	if (ret != 0) {
+		usage();
+		return -1;
+	}
+
 	return 0;
 }
diff --git a/src/primary/args.h b/src/primary/args.h
index 644cd56..f73772c 100644
--- a/src/primary/args.h
+++ b/src/primary/args.h
@@ -13,6 +13,9 @@ extern uint16_t num_rings;
 extern char *server_ip;
 extern int server_port;
 
+/* Return value definition for getopt_long(). Only for long option. */
+#define SPP_LONGOPT_RETVAL_PORT_NUM 1 /* For `--port-num` */
+
 /**
  * Set flg from given argument.
  *
diff --git a/src/primary/init.c b/src/primary/init.c
index 9e47fcf..e9c12f1 100644
--- a/src/primary/init.c
+++ b/src/primary/init.c
@@ -160,7 +160,9 @@ init(int argc, char *argv[])
 	/* now initialise the ports we will use */
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		for (count = 0; count < ports->num_ports; count++) {
-			retval = init_port(ports->id[count], pktmbuf_pool);
+			retval = init_port(ports->id[count], pktmbuf_pool,
+				ports->queue_info[count].rxq,
+				ports->queue_info[count].txq);
 			if (retval != 0)
 				rte_exit(EXIT_FAILURE,
 					"Cannot initialise port %d\n", count);
@@ -254,7 +256,8 @@ check_all_ports_link_status(struct port_info *ports, uint16_t port_num,
  * - start the port and report its status to stdout
  */
 int
-init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool)
+init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool,
+	uint16_t rx_rings, uint16_t tx_rings)
 {
 	/* for port configuration all features are off by default */
 	const struct rte_eth_conf port_conf = {
@@ -262,7 +265,6 @@ init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool)
 			.mq_mode = ETH_MQ_RX_RSS,
 		},
 	};
-	const uint16_t rx_rings = 1, tx_rings = 1;
 	const uint16_t rx_ring_size = RTE_MP_RX_DESC_DEFAULT;
 	const uint16_t tx_ring_size = RTE_MP_TX_DESC_DEFAULT;
 	uint16_t q;
diff --git a/src/primary/init.h b/src/primary/init.h
index 6344377..5384154 100644
--- a/src/primary/init.h
+++ b/src/primary/init.h
@@ -50,6 +50,7 @@ int init(int argc, char *argv[]);
 void check_all_ports_link_status(struct port_info *ports, uint16_t port_num,
 		uint32_t port_mask);
 
-int init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool);
+int init_port(uint16_t port_num, struct rte_mempool *pktmbuf_pool,
+		uint16_t rx_rings, uint16_t tx_rings);
 
 #endif /* ifndef _PRIMARY_INIT_H_ */
diff --git a/src/primary/main.c b/src/primary/main.c
index d3828e8..ca81636 100644
--- a/src/primary/main.c
+++ b/src/primary/main.c
@@ -16,6 +16,7 @@
 #include "args.h"
 #include "init.h"
 #include "primary.h"
+#include "primary/flow/flow.h"
 
 #include "shared/port_manager.h"
 #include "shared/secondary/add_port.h"
@@ -23,10 +24,10 @@
 
 /*
  * Buffer sizes of status message of primary. Total number of size
- * must be equal to MSG_SIZE 2048 defined in `shared/common.h`.
+ * must be equal to MSG_SIZE 32768 defined in `shared/common.h`.
  */
 #define PRI_BUF_SIZE_LCORE 128
-#define PRI_BUF_SIZE_PHY 512
+#define PRI_BUF_SIZE_PHY 30720
 #define PRI_BUF_SIZE_RING (MSG_SIZE - PRI_BUF_SIZE_LCORE - PRI_BUF_SIZE_PHY)
 
 #define SPP_PATH_LEN 1024  /* seems enough for path of spp procs */
@@ -639,9 +640,10 @@ forwarder_status_json(char *str)
 static int
 phy_port_stats_json(char *str)
 {
-	int i;
-	int buf_size = 256;  /* size of temp buffer */
-	char phy_port[buf_size];
+	int i, ret;
+	int buf_size = PRI_BUF_SIZE_PHY - 512;  /* size of temp buffer */
+	char phy_port[PRI_BUF_SIZE_PHY];
+	char flow[buf_size];
 	char buf_phy_ports[PRI_BUF_SIZE_PHY];
 	memset(phy_port, '\0', sizeof(phy_port));
 	memset(buf_phy_ports, '\0', sizeof(buf_phy_ports));
@@ -651,16 +653,29 @@ phy_port_stats_json(char *str)
 		RTE_LOG(DEBUG, PRIMARY, "Size of buf_phy_ports str: %d\n",
 				(int)strlen(buf_phy_ports));
 
-		memset(phy_port, '\0', buf_size);
+		memset(phy_port, '\0', PRI_BUF_SIZE_PHY);
+		memset(flow, '\0', buf_size);
+
+		ret = append_flow_json(i, buf_size, flow);
+		if (ret != 0) {
+			sprintf(buf_phy_ports + strlen(buf_phy_ports) - 1,
+					"%s", "");
+			break;
+		}
 
 		sprintf(phy_port, "{\"id\":%u,\"eth\":\"%s\","
 				"\"rx\":%"PRIu64",\"tx\":%"PRIu64","
-				"\"tx_drop\":%"PRIu64"}",
+				"\"tx_drop\":%"PRIu64","
+				"\"nof_queues\":{\"rx\":%d,\"tx\":%d},"
+				"\"flow\":%s}",
 				ports->id[i],
 				get_printable_mac_addr(ports->id[i]),
 				ports->port_stats[i].rx,
 				ports->port_stats[i].tx,
-				ports->port_stats[i].tx_drop);
+				ports->port_stats[i].tx_drop,
+				ports->queue_info[i].rxq,
+				ports->queue_info[i].txq,
+				flow);
 
 		int cur_buf_size = (int)strlen(buf_phy_ports) +
 			(int)strlen(phy_port);
@@ -948,8 +963,10 @@ parse_command(char *str)
 	char patch_set[64] = { 0 };  /* "{\"src\":\"%s:%d\",\"dst\":...}" */
 	char *p_type;
 	int p_id;
+	char tmp_response[MSG_SIZE];
 
 	memset(sec_name, '\0', 16);
+	memset(tmp_response, '\0', MSG_SIZE);
 
 	/* tokenize the user commands from controller */
 	token_list[max_token] = strtok(str, " ");
@@ -1150,6 +1167,13 @@ parse_command(char *str)
 		sprintf(str, "{%s:%s,%s:%s}",
 				"\"result\"", "\"succeeded\"",
 				"\"command\"", "\"clear\"");
+
+	} else if (!strcmp(token_list[0], "flow")) {
+		RTE_LOG(DEBUG, PRIMARY, "'%s' command received.\n",
+				token_list[0]);
+		ret = parse_flow(token_list, tmp_response);
+		memset(str, '\0', MSG_SIZE);
+		strncpy(str, tmp_response, MSG_SIZE-1);
 	}
 
 	return ret;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 06/17] spp_primary: add support of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (22 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 05/17] spp_primary: " x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 07/17] spp_primary: add common function " x-fn-spp-ml
                   ` (10 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch provides main functions which
defines validate, create, delete and flush flow

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/flow.c | 1045 +++++++++++++++++++++++++++++++++++++++
 src/primary/flow/flow.h |   94 ++++
 2 files changed, 1139 insertions(+)
 create mode 100644 src/primary/flow/flow.c
 create mode 100644 src/primary/flow/flow.h

diff --git a/src/primary/flow/flow.c b/src/primary/flow/flow.c
new file mode 100644
index 0000000..ba28012
--- /dev/null
+++ b/src/primary/flow/flow.c
@@ -0,0 +1,1045 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+#include <rte_common.h>
+#include <rte_ether.h>
+#include <rte_byteorder.h>
+
+#include "shared/common.h"
+#include "shared/secondary/utils.h"
+#include "shared/secondary/spp_worker_th/data_types.h"
+#include "primary/primary.h"
+#include "flow.h"
+#include "attr.h"
+#include "common.h"
+
+#include "primary/flow/pattern/eth.h"
+#include "primary/flow/pattern/vlan.h"
+
+#include "primary/flow/action/jump.h"
+#include "primary/flow/action/queue.h"
+#include "primary/flow/action/of_push_vlan.h"
+#include "primary/flow/action/of_set_vlan_vid.h"
+#include "primary/flow/action/of_set_vlan_pcp.h"
+
+
+/* Flow list for each port */
+static struct port_flow port_list[RTE_MAX_ETHPORTS] = { 0 };
+
+/* Define item operations */
+static struct flow_item_ops flow_item_ops_list[] = {
+	{
+		.str_type = "end",
+		.type = RTE_FLOW_ITEM_TYPE_END,
+		.parse = NULL,
+		.detail_list = NULL
+	},
+	{
+		.str_type = "eth",
+		.type = RTE_FLOW_ITEM_TYPE_ETH,
+		.size = sizeof(struct rte_flow_item_eth),
+		.parse = parse_item_common,
+		.detail_list = eth_ops_list,
+		.status = append_item_eth_json,
+	},
+	{
+		.str_type = "vlan",
+		.type = RTE_FLOW_ITEM_TYPE_VLAN,
+		.size = sizeof(struct rte_flow_item_vlan),
+		.parse = parse_item_common,
+		.detail_list = vlan_ops_list,
+		.status = append_item_vlan_json,
+	},
+};
+
+/* Define action operations */
+static struct flow_action_ops flow_action_ops_list[] = {
+	{
+		.str_type = "end",
+		.type = RTE_FLOW_ACTION_TYPE_END,
+		.size = 0,
+		.parse = NULL,
+		.detail_list = NULL,
+		.status = NULL,
+	},
+	{
+		.str_type = "jump",
+		.type = RTE_FLOW_ACTION_TYPE_JUMP,
+		.size = sizeof(struct rte_flow_action_jump),
+		.parse = parse_action_common,
+		.detail_list = jump_ops_list,
+		.status = append_action_jump_json,
+	},
+	{
+		.str_type = "queue",
+		.type = RTE_FLOW_ACTION_TYPE_QUEUE,
+		.size = sizeof(struct rte_flow_action_queue),
+		.parse = parse_action_common,
+		.detail_list = queue_ops_list,
+		.status = append_action_queue_json,
+	},
+	{
+		.str_type = "of_pop_vlan",
+		.type = RTE_FLOW_ACTION_TYPE_OF_POP_VLAN,
+		.size = 0,
+		.parse = NULL,
+		.detail_list = NULL,
+		.status = append_action_null_json,
+	},
+	{
+		.str_type = "of_push_vlan",
+		.type = RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN,
+		.size = sizeof(struct rte_flow_action_of_push_vlan),
+		.parse = parse_action_common,
+		.detail_list = of_push_vlan_ops_list,
+		.status = append_action_of_push_vlan_json,
+	},
+	{
+		.str_type = "of_set_vlan_vid",
+		.type = RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID,
+		.size = sizeof(struct rte_flow_action_of_set_vlan_vid),
+		.parse = parse_action_common,
+		.detail_list = of_set_vlan_vid_ops_list,
+		.status = append_action_of_set_vlan_vid_json,
+	},
+	{
+		.str_type = "of_set_vlan_pcp",
+		.type = RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP,
+		.size = sizeof(struct rte_flow_action_of_set_vlan_pcp),
+		.parse = parse_action_common,
+		.detail_list = of_set_vlan_pcp_ops_list,
+		.status = append_action_of_set_vlan_pcp_json,
+	},
+};
+
+/* Free memory of "flow_args". */
+static void
+free_flow_args(struct flow_args *input)
+{
+	int i;
+	struct rte_flow_item *pattern;
+	struct rte_flow_action *actions;
+	char **target;
+
+	if ((input->command != VALIDATE) &&
+		(input->command != CREATE))
+		return;
+
+	pattern = input->args.rule.pattern;
+	if (pattern != NULL) {
+		for (i = 0; pattern[i].type != RTE_FLOW_ITEM_TYPE_END; i++) {
+			target = (char **)((char *)(&pattern[i]) +
+				offsetof(struct rte_flow_item, spec));
+			if (*target != NULL)
+				free(*target);
+
+			target = (char **)((char *)(&pattern[i]) +
+				offsetof(struct rte_flow_item, last));
+			if (*target != NULL)
+				free(*target);
+
+			target = (char **)((char *)(&pattern[i]) +
+				offsetof(struct rte_flow_item, mask));
+			if (*target != NULL)
+				free(*target);
+		}
+
+		free(pattern);
+	}
+
+	actions = input->args.rule.actions;
+	if (actions != NULL) {
+		for (i = 0; actions[i].type != RTE_FLOW_ACTION_TYPE_END; i++) {
+			target = (char **)((char *)(&actions[i]) +
+				offsetof(struct rte_flow_action, conf));
+			if (*target != NULL)
+				free(*target);
+		}
+
+		free(actions);
+	}
+}
+
+/*
+ * Create response in JSON format.
+ * `rule_id` must be empty if flow create is failed.
+ */
+static void
+make_response(char *response, const char *result, const char *message,
+	char *rule_id)
+{
+	if (rule_id == NULL)
+		snprintf(response, MSG_SIZE,
+			"{\"result\": \"%s\", \"message\": \"%s\"}",
+			result, message);
+	else
+		snprintf(response, MSG_SIZE,
+			"{\"result\": \"%s\", \"message\": \"%s\", "
+			"\"rule_id\": \"%s\"}",
+			result, message, rule_id);
+}
+
+/* Create error response from rte_flow_error */
+static void
+make_error_response(char *response, const char *message,
+	struct rte_flow_error error, char *rule_id)
+{
+	/* Define description for each error type */
+	static const char *const errstr_list[] = {
+		[RTE_FLOW_ERROR_TYPE_NONE] = "No error",
+		[RTE_FLOW_ERROR_TYPE_UNSPECIFIED] = "Cause unspecified",
+		[RTE_FLOW_ERROR_TYPE_HANDLE] = "Flow rule (handle)",
+		[RTE_FLOW_ERROR_TYPE_ATTR_GROUP] = "Group field",
+		[RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY] = "Priority field",
+		[RTE_FLOW_ERROR_TYPE_ATTR_INGRESS] = "Ingress field",
+		[RTE_FLOW_ERROR_TYPE_ATTR_EGRESS] = "Egress field",
+		[RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER] = "Transfer field",
+		[RTE_FLOW_ERROR_TYPE_ATTR] = "Attributes structure",
+		[RTE_FLOW_ERROR_TYPE_ITEM_NUM] = "Pattern length",
+		[RTE_FLOW_ERROR_TYPE_ITEM_SPEC] = "Item specification",
+		[RTE_FLOW_ERROR_TYPE_ITEM_LAST] = "Item specification range",
+		[RTE_FLOW_ERROR_TYPE_ITEM_MASK] = "Item specification mask",
+		[RTE_FLOW_ERROR_TYPE_ITEM] = "Specific pattern item",
+		[RTE_FLOW_ERROR_TYPE_ACTION_NUM] = "Number of actions",
+		[RTE_FLOW_ERROR_TYPE_ACTION_CONF] = "Action configuration",
+		[RTE_FLOW_ERROR_TYPE_ACTION] = "Specific action",
+	};
+	int err = rte_errno;
+	char msg[512] = "";
+	char cause[32] = "";
+	const char *errstr;
+
+	if ((unsigned int)error.type >= RTE_DIM(errstr_list) ||
+	    !errstr_list[error.type])
+		errstr = "Unknown type";
+	else
+		errstr = errstr_list[error.type];
+
+
+	if (error.cause != NULL)
+		snprintf(cause, sizeof(cause), "cause: %p\\n", error.cause);
+
+	snprintf(msg, sizeof(msg),
+		"%s\\nerror type: %d (%s)\\n"
+		"%serror message: %s\\nrte_errno: %s",
+		message, error.type, errstr, cause,
+		error.message ? error.message : "(no stated reason)",
+		rte_strerror(err));
+	make_response(response, "error", msg, rule_id);
+}
+
+/* Add to array, redeclare memory. */
+static int
+append_object_list(void **list, void *add, size_t obj_size, int num)
+{
+	char *new_list;
+
+	new_list = malloc(obj_size * num);
+	if (new_list == NULL)
+		return -1;
+
+	/* Copy original list*/
+	if (*list != NULL) {
+		memcpy(new_list, *list, obj_size * (num - 1));
+		free(*list);
+	}
+
+	/* Add to list */
+	memcpy(new_list + (obj_size * (num - 1)), add, obj_size);
+
+	*list = (void *)new_list;
+	return 0;
+}
+
+static int
+parse_flow_actions(char *token_list[], int *index,
+	struct rte_flow_action **actions)
+{
+	int ret;
+	int action_count = 0;
+	uint16_t i;
+	char *token;
+	struct flow_action_ops *ops;
+	struct rte_flow_action action;
+
+	if (strcmp(token_list[*index], "actions")) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Invalid parameter is %s(%s:%d)\n",
+			token_list[*index], __func__, __LINE__);
+		return -1;
+	}
+
+	/* Next to word */
+	(*index)++;
+
+	while (token_list[*index] != NULL) {
+		token = token_list[*index];
+
+		for (i = 0; i < RTE_DIM(flow_action_ops_list); i++) {
+			ops = &flow_action_ops_list[i];
+			if (strcmp(token, ops->str_type))
+				continue;
+
+			memset(&action, 0, sizeof(struct rte_flow_action));
+			action.type = ops->type;
+			if (ops->parse != NULL) {
+				ret = ops->parse(token_list, index, &action,
+					ops);
+				if (ret < 0)
+					return -1;
+			} else {
+				(*index)++;
+			}
+			break;
+		}
+
+		/*
+		 * Error occurs if a action string that is not defined in
+		 * str_type of flow_action_ops_list is specified
+		 */
+		if (i == RTE_DIM(flow_action_ops_list)) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Invalid parameter "
+				"is %s action(%s:%d)\n",
+				token, __func__, __LINE__);
+			return -1;
+		}
+
+		/* Add to "actions" list */
+		action_count++;
+		ret = append_object_list((void **)actions, &action,
+			sizeof(struct rte_flow_action), action_count);
+
+		if (!strcmp(token, "end"))
+			break;
+
+		(*index)++;
+	}
+
+	return 0;
+}
+
+static int
+parse_flow_pattern(char *token_list[], int *index,
+	struct rte_flow_item **pattern)
+{
+	int ret;
+	int item_count = 0;
+	uint32_t i;
+	char *token;
+	struct flow_item_ops *ops;
+	struct rte_flow_item item;
+
+	while (token_list[*index] != NULL) {
+		token = token_list[*index];
+
+		for (i = 0; i < RTE_DIM(flow_item_ops_list); i++) {
+			ops = &flow_item_ops_list[i];
+			if (strcmp(token, ops->str_type))
+				continue;
+
+			memset(&item, 0, sizeof(struct rte_flow_item));
+			item.type = ops->type;
+			if (ops->parse != NULL) {
+				ret = ops->parse(token_list, index, &item,
+					ops);
+				if (ret < 0)
+					return -1;
+			}
+			break;
+		}
+
+		/*
+		 * Error occurs if a pattern string that is not defined in
+		 * str_type of flow_item_ops_list is specified
+		 */
+		if (i == RTE_DIM(flow_item_ops_list)) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Invalid parameter "
+				"is %s pattern(%s:%d)\n",
+				token, __func__, __LINE__);
+			return -1;
+		}
+
+		/* Add to "pattern" list */
+		item_count++;
+		ret = append_object_list((void **)pattern, &item,
+			sizeof(struct rte_flow_item), item_count);
+
+		if (!strcmp(token, "end"))
+			break;
+
+		(*index)++;
+	}
+
+	return 0;
+}
+
+static int
+parse_flow_rule(char *token_list[], struct flow_args *input)
+{
+	int ret = 0;
+	int index;
+
+	ret = parse_phy_port_id(token_list[2], &input->port_id);
+	if (ret < 0)
+		return -1;
+
+	/* The next index of the port */
+	index = 3;
+
+	/* Attribute parse */
+	ret = parse_flow_attr(token_list, &index, &input->args.rule.attr);
+	if (ret < 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Failed to parse Attribute(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* The next index of the pattern */
+	index++;
+
+	/* Pattern parse */
+	ret = parse_flow_pattern(token_list, &index,
+		&input->args.rule.pattern);
+	if (ret < 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Failed to parse Pattern(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	/* The next index of the actions */
+	index++;
+
+	/* Actions parse */
+	ret = parse_flow_actions(token_list, &index,
+		&input->args.rule.actions);
+	if (ret < 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Failed to parse Actions(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	return 0;
+}
+
+static int
+parse_flow_destroy(char *token_list[], struct flow_args *input)
+{
+	int ret;
+	char *end;
+
+	ret = parse_phy_port_id(token_list[2], &input->port_id);
+	if (ret < 0)
+		return -1;
+
+	if (token_list[3] == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"rule_id is not specified(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	if (!strcmp(token_list[3], "ALL")) {
+		input->command = FLUSH;
+
+	} else {
+		input->command = DESTROY;
+		input->args.destroy.rule_id = strtoul(token_list[3],
+			&end, 10);
+	}
+
+	return 0;
+}
+
+/** Generate a flow_rule entry from attributes/pattern/actions. */
+static struct flow_rule *
+create_flow_rule(struct rte_flow_attr *attr,
+	struct rte_flow_item *pattern,
+	struct rte_flow_action *actions,
+	struct rte_flow_error *error)
+{
+	const struct rte_flow_conv_rule conv_rule = {
+		.attr_ro = attr,
+		.pattern_ro = pattern,
+		.actions_ro = actions,
+	};
+	struct flow_rule *rule;
+	int ret;
+
+	ret = rte_flow_conv(RTE_FLOW_CONV_OP_RULE, NULL, 0, &conv_rule,
+		error);
+	if (ret < 0)
+		return NULL;
+
+	rule = calloc(1, offsetof(struct flow_rule, rule) + ret);
+	if (!rule) {
+		rte_flow_error_set
+			(error, errno, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+			 "calloc() failed");
+		return NULL;
+	}
+
+	ret = rte_flow_conv(RTE_FLOW_CONV_OP_RULE, &rule->rule, ret, &conv_rule,
+			  error);
+	if (ret >= 0)
+		return rule;
+
+	free(rule);
+	return NULL;
+}
+
+/* Execute rte_flow_validate().*/
+static void
+exec_flow_validate(int port_id,
+	struct rte_flow_attr *attr,
+	struct rte_flow_item *pattern,
+	struct rte_flow_action *actions,
+	char *response)
+{
+	int ret;
+	struct rte_flow_error error;
+
+	memset(&error, 0, sizeof(error));
+
+	ret = rte_flow_validate(port_id, attr, pattern, actions, &error);
+	if (ret != 0)
+		make_error_response(response, "Flow validate error", error,
+			NULL);
+	else
+		make_response(response, "success", "Flow rule validated",
+			NULL);
+}
+
+/* Execute rte_flow_create(). Save flow rules globally */
+static void
+exec_flow_create(int port_id,
+	struct rte_flow_attr *attr,
+	struct rte_flow_item *pattern,
+	struct rte_flow_action *actions,
+	char *response)
+{
+	uint32_t rule_id;
+	char mes[32];
+	char rule_id_str[11] = {0};
+	struct rte_flow_error error;
+	struct rte_flow *flow;
+	struct flow_rule *rule;
+	struct port_flow *port;
+
+	memset(&error, 0, sizeof(error));
+
+	flow = rte_flow_create(port_id, attr, pattern, actions, &error);
+	if (flow == NULL) {
+		make_error_response(response, "Flow create error", error,
+			rule_id_str);
+		return;
+	}
+
+	port = &port_list[port_id];
+	if (port->flow_list != NULL) {
+		if (port->flow_list->rule_id >= UINT32_MAX) {
+			make_response(response, "error",
+				"Rule ID must be less than %"PRIu32,
+				rule_id_str);
+			rte_flow_destroy(port_id, flow, NULL);
+			return;
+		}
+		rule_id = port->flow_list->rule_id + 1;
+	} else {
+		rule_id = 0;
+	}
+
+	rule = create_flow_rule(attr, pattern, actions, &error);
+	if (rule == NULL) {
+		rte_flow_destroy(port_id, flow, NULL);
+		make_error_response(response, "Flow create error", error,
+			rule_id_str);
+		return;
+	}
+
+	/* Keep it globally as a list */
+	rule->rule_id = rule_id;
+	rule->flow_handle = flow;
+
+	if (port->flow_list == NULL)
+		rule->prev = NULL;
+	else
+		rule->prev = port->flow_list;
+
+	port->flow_list = rule;
+
+	sprintf(mes, "Flow rule #%d created", rule_id);
+	sprintf(rule_id_str, "%d", rule_id);
+	make_response(response, "success", mes, rule_id_str);
+}
+
+/* Execute rte_flow_destroy(). Destroying a globally saved flow rule */
+static void
+exec_flow_destroy(int port_id, uint32_t rule_id, char *response)
+{
+	int ret;
+	int found_flg = 0;
+	char mes[64];
+	struct flow_rule *rule, **next_ptr;
+	struct rte_flow_error error;
+
+	memset(&error, 0, sizeof(error));
+
+	ret = is_portid_used(port_id);
+	if (ret != 0) {
+		sprintf(mes, "Invalid port %d", port_id);
+		make_response(response, "error", mes, NULL);
+		return;
+	}
+
+	next_ptr = &(port_list[port_id].flow_list);
+	rule = port_list[port_id].flow_list;
+
+	while (rule != NULL) {
+		if (rule->rule_id != rule_id) {
+			next_ptr = &(rule->prev);
+			rule = rule->prev;
+			continue;
+		}
+
+		ret = rte_flow_destroy(port_id, rule->flow_handle, &error);
+		if (ret != 0) {
+			make_error_response(response, "Flow destroy error",
+				error, NULL);
+			return;
+		}
+
+		/* Remove flow from global list */
+		*next_ptr = rule->prev;
+		free(rule);
+		found_flg = 1;
+
+		sprintf(mes, "Flow rule #%d destroyed", rule_id);
+		make_response(response, "success", mes, NULL);
+		break;
+	}
+
+	/* Rule_id not found */
+	if (found_flg == 0) {
+		sprintf(mes, "Flow rule #%d not found", rule_id);
+		make_response(response, "error", mes, NULL);
+	}
+}
+
+/* Delete all globally saved flow rules */
+static void
+exec_flow_flush(int port_id, char *response)
+{
+	int ret;
+	char mes[64];
+	struct flow_rule *rule;
+	struct rte_flow_error error;
+
+	memset(&error, 0, sizeof(error));
+
+	ret = is_portid_used(port_id);
+	if (ret != 0) {
+		sprintf(mes, "Invalid port %d", port_id);
+		make_response(response, "error", mes, NULL);
+		return;
+	}
+
+	ret = rte_flow_flush(port_id, &error);
+	if (ret != 0)
+		make_error_response(response, "Flow destroy error",
+			error, NULL);
+	else
+		make_response(response, "success", "Flow rule all destroyed",
+			NULL);
+
+	/*
+	 * Even if a failure occurs, flow handle is invalidated,
+	 * so delete flow_list.
+	 */
+
+	while (port_list[port_id].flow_list != NULL) {
+		rule = port_list[port_id].flow_list->prev;
+		free(port_list[port_id].flow_list);
+		port_list[port_id].flow_list = rule;
+	}
+}
+
+static void
+exec_flow(struct flow_args *input, char *response)
+{
+	switch (input->command) {
+	case VALIDATE:
+		exec_flow_validate(input->port_id,
+			&input->args.rule.attr,
+			input->args.rule.pattern,
+			input->args.rule.actions,
+			response);
+		break;
+	case CREATE:
+		exec_flow_create(input->port_id,
+			&input->args.rule.attr,
+			input->args.rule.pattern,
+			input->args.rule.actions,
+			response);
+		break;
+	case DESTROY:
+		exec_flow_destroy(input->port_id,
+			input->args.destroy.rule_id,
+			response);
+		break;
+	case FLUSH:
+		exec_flow_flush(input->port_id, response);
+		break;
+	}
+
+	/* Argument data is no longer needed and freed */
+	free_flow_args(input);
+}
+
+int
+parse_flow(char *token_list[], char *response)
+{
+	int ret = 0;
+	struct flow_args input = { 0 };
+
+	if (token_list[1] == NULL) {
+		ret = -1;
+	} else if (!strcmp(token_list[1], "validate")) {
+		input.command = VALIDATE;
+		ret = parse_flow_rule(token_list, &input);
+
+	} else if (!strcmp(token_list[1], "create")) {
+		input.command = CREATE;
+		ret = parse_flow_rule(token_list, &input);
+
+	} else if (!strcmp(token_list[1], "destroy")) {
+		ret = parse_flow_destroy(token_list, &input);
+
+	} else {
+		ret = -1;
+	}
+
+	if (ret != 0) {
+		free_flow_args(&input);
+		make_response(response, "error",
+			"Flow command invalid argument", NULL);
+		return 0;
+	}
+
+	exec_flow(&input, response);
+
+	return 0;
+}
+
+static int
+append_flow_pattern_json(const struct rte_flow_item *pattern, int buf_size,
+	char *pattern_str)
+{
+	uint32_t i, j;
+	uint32_t nof_elems = 3;
+	int ret = 0;
+	char *tmp_str;
+	const char element_str[][5] = { "spec", "last", "mask" };
+	const struct rte_flow_item *ptn = pattern;
+	struct flow_item_ops *ops;
+	const void *tmp_ptr[nof_elems];
+
+	tmp_str = malloc(buf_size);
+	if (tmp_str == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	while (ptn->type != RTE_FLOW_ITEM_TYPE_END) {
+		memset(tmp_str, 0, buf_size);
+
+		tmp_ptr[0] = ptn->spec;
+		tmp_ptr[1] = ptn->last;
+		tmp_ptr[2] = ptn->mask;
+
+		for (i = 0; i < RTE_DIM(flow_item_ops_list); i++) {
+			ops = &flow_item_ops_list[i];
+			if (ptn->type != ops->type)
+				continue;
+
+			snprintf(tmp_str, buf_size,
+				"{\"type\":\"%s\",",
+				ops->str_type);
+
+			for (j = 0; j < nof_elems; j++) {
+				snprintf(tmp_str + strlen(tmp_str), buf_size,
+					"\"%s\":",
+					element_str[j]);
+
+				if (tmp_ptr[j] != NULL)
+					ret = ops->status(tmp_ptr[j],
+						buf_size - (int)strlen(tmp_str),
+						tmp_str + strlen(tmp_str));
+				else
+					snprintf(tmp_str + strlen(tmp_str),
+						buf_size,
+						"null");
+
+				if (ret != 0)
+					break;
+
+				if (j < nof_elems - 1)
+					tmp_str[strlen(tmp_str)] = ',';
+			}
+
+			tmp_str[strlen(tmp_str)] = '}';
+
+			break;
+		}
+
+		if (ret != 0)
+			break;
+
+		if ((int)strlen(pattern_str) + (int)strlen(tmp_str)
+			> buf_size - 1) {
+			ret = -1;
+			break;
+		}
+		strncat(pattern_str, tmp_str, strlen(tmp_str));
+
+		/*
+		 * If there is the following pattern, add ',' to
+		 * pattern_str
+		 */
+		ptn++;
+		if (ptn->type != RTE_FLOW_ITEM_TYPE_END) {
+			if ((int)strlen(pattern_str) + 1 > buf_size - 1) {
+				ret = -1;
+				break;
+			}
+			pattern_str[strlen(pattern_str)] = ',';
+		}
+	}
+
+	if (tmp_str != NULL)
+		free(tmp_str);
+
+	return ret;
+}
+
+static int
+append_flow_action_json(const struct rte_flow_action *actions, int buf_size,
+	char *actions_str)
+{
+	uint32_t i;
+	int ret = 0;
+	char *tmp_str;
+	const struct rte_flow_action *act = actions;
+	struct flow_action_ops *ops;
+
+	tmp_str = malloc(buf_size);
+	if (tmp_str == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	while (act->type != RTE_FLOW_ACTION_TYPE_END) {
+		memset(tmp_str, 0, buf_size);
+
+		for (i = 0; i < RTE_DIM(flow_action_ops_list); i++) {
+			ops = &flow_action_ops_list[i];
+			if (act->type != ops->type)
+				continue;
+
+			snprintf(tmp_str, buf_size,
+				"{\"type\":\"%s\",\"conf\":",
+				ops->str_type);
+
+			ret = ops->status(act->conf,
+				buf_size - (int)strlen(tmp_str),
+				tmp_str + strlen(tmp_str));
+			tmp_str[strlen(tmp_str)] = '}';
+			break;
+		}
+
+		if (ret != 0)
+			break;
+
+		if ((int)strlen(actions_str) + (int)strlen(tmp_str)
+			> buf_size - 1) {
+			ret = -1;
+			break;
+		}
+		strncat(actions_str, tmp_str, strlen(tmp_str));
+
+		/*
+		 * If there is the following pattern, add ',' to
+		 * actions_str
+		 */
+		act++;
+		if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+			if ((int)strlen(actions_str) + 1 > buf_size - 1) {
+				ret = -1;
+				break;
+			}
+			actions_str[strlen(actions_str)] = ',';
+		}
+	}
+
+	if (tmp_str != NULL)
+		free(tmp_str);
+
+	return ret;
+}
+
+static int
+append_flow_rule_json(struct flow_rule *flow, int buf_size, char *flow_str)
+{
+	int ret = 0;
+	struct rte_flow_conv_rule rule;
+	char *tmp_str, *attr_str, *pattern_str, *actions_str;
+
+	while (1) {
+		tmp_str = malloc(buf_size);
+		attr_str = malloc(buf_size);
+		pattern_str = malloc(buf_size);
+		actions_str = malloc(buf_size);
+		if (tmp_str == NULL || attr_str == NULL
+			|| pattern_str == NULL || actions_str == NULL) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Memory allocation failure(%s:%d)\n",
+				__func__, __LINE__);
+			ret = -1;
+			break;
+		}
+		memset(tmp_str, 0, buf_size);
+		memset(attr_str, 0, buf_size);
+		memset(pattern_str, 0, buf_size);
+		memset(actions_str, 0, buf_size);
+
+		rule = flow->rule;
+
+		ret = append_flow_attr_json(rule.attr_ro, buf_size, attr_str);
+		if (ret != 0)
+			break;
+
+		ret = append_flow_pattern_json(rule.pattern_ro, buf_size,
+			pattern_str);
+		if (ret != 0)
+			break;
+
+		ret = append_flow_action_json(rule.actions_ro, buf_size,
+			actions_str);
+		if (ret != 0)
+			break;
+
+		snprintf(tmp_str, buf_size,
+			"{\"rule_id\":%d,"
+			"\"attr\":%s,"
+			"\"patterns\":[%s],"
+			"\"actions\":[%s]}",
+			flow->rule_id, attr_str, pattern_str, actions_str);
+
+		if ((int)strlen(tmp_str) > buf_size - 1) {
+			ret = -1;
+			break;
+		}
+
+		snprintf(flow_str, buf_size, "%s", tmp_str);
+		break;
+	}
+
+	if (tmp_str != NULL)
+		free(tmp_str);
+	if (attr_str != NULL)
+		free(attr_str);
+	if (pattern_str != NULL)
+		free(pattern_str);
+	if (actions_str != NULL)
+		free(actions_str);
+
+	return ret;
+}
+
+int
+append_flow_json(int port_id, int buf_size, char *output)
+{
+	int ret = 0;
+	int str_size = 0;
+	char *flow_str, *tmp_str;
+	struct flow_rule *flow;
+
+	flow_str = malloc(buf_size);
+	tmp_str = malloc(buf_size);
+	if (flow_str == NULL || tmp_str == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	flow = port_list[port_id].flow_list;
+
+	while (flow != NULL) {
+		memset(flow_str, 0, buf_size);
+
+		ret = append_flow_rule_json(flow, buf_size, flow_str);
+		if (ret != 0)
+			break;
+
+		if (str_size == 0) {
+			snprintf(output, buf_size, "%s", flow_str);
+			str_size += (int)strlen(flow_str);
+
+		} else {
+			str_size += ((int)strlen(flow_str) + 1);
+			if (str_size > buf_size - 1) {
+				ret = -1;
+				break;
+			}
+
+			/*
+			 * Since flow_list is in descending order,
+			 * concatenate the strings in front.
+			 */
+			memset(tmp_str, 0, buf_size);
+			strncpy(tmp_str, output, buf_size);
+			memset(output, 0, buf_size);
+
+			snprintf(output, buf_size, "%s,%s",
+				flow_str, tmp_str);
+		}
+
+		flow = flow->prev;
+	}
+
+	if (ret == 0) {
+		if ((int)strlen("[]") + (int)strlen(flow_str)
+			> buf_size - 1)
+			ret = -1;
+		else {
+			memset(tmp_str, 0, buf_size);
+			strncpy(tmp_str, output, buf_size);
+			memset(output, 0, buf_size);
+
+			snprintf(output, buf_size, "[%s]", tmp_str);
+		}
+	}
+
+	if (ret != 0)
+		RTE_LOG(ERR, SPP_FLOW,
+			"Cannot send all of flow stats(%s:%d)\n",
+			__func__, __LINE__);
+
+	if (flow_str != NULL)
+		free(flow_str);
+	if (tmp_str != NULL)
+		free(tmp_str);
+
+	return ret;
+}
diff --git a/src/primary/flow/flow.h b/src/primary/flow/flow.h
new file mode 100644
index 0000000..ecd4eb3
--- /dev/null
+++ b/src/primary/flow/flow.h
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_H_
+#define _PRIMARY_FLOW_H_
+
+#include <rte_log.h>
+
+#define RTE_LOGTYPE_SPP_FLOW RTE_LOGTYPE_USER1
+
+enum flow_command {
+	VALIDATE = 0,
+	CREATE,
+	DESTROY,
+	FLUSH
+};
+
+/* Parser result of flow command arguments */
+struct flow_args {
+	enum flow_command command;
+	int port_id;
+	union {
+		struct {
+			struct rte_flow_attr attr;
+			struct rte_flow_item *pattern;
+			struct rte_flow_action *actions;
+		} rule; /* validate or create arguments. */
+		struct {
+			uint32_t rule_id;
+		} destroy; /* destroy arguments. */
+	} args;
+};
+
+/* Descriptor for a single flow. */
+struct flow_rule {
+	/* Flow rule ID */
+	uint32_t rule_id;
+
+	/* Previous flow in list. */
+	struct flow_rule *prev;
+
+	/* Opaque flow object returned by PMD. */
+	struct rte_flow *flow_handle;
+
+	/* Saved flow rule description. */
+	struct rte_flow_conv_rule rule;
+};
+
+/* Flow rule list of the port */
+struct port_flow {
+	/* Associated flows */
+	struct flow_rule *flow_list;
+};
+
+/* Detail parse operation for a specific item or action */
+struct flow_detail_ops {
+	const char *token;
+	const size_t offset;
+	const size_t size;
+	int flg_value;
+	int (*parse_detail)(char *str, void *output);
+};
+
+/* Operation for each item type */
+struct flow_item_ops {
+	const char *str_type;
+	enum rte_flow_item_type type;
+	size_t size;
+	int (*parse)(char *token_list[], int *index,
+		struct rte_flow_item *pattern,
+		struct flow_item_ops *ops);
+	struct flow_detail_ops *detail_list;
+	int (*status)(const void *element,
+		int buf_size, char *pattern_str);
+};
+
+/* Operation for each action type */
+struct flow_action_ops {
+	const char *str_type;
+	enum rte_flow_action_type type;
+	size_t size;
+	int (*parse)(char *token_list[], int *index,
+		struct rte_flow_action *action,
+		struct flow_action_ops *ops);
+	struct flow_detail_ops *detail_list;
+	int (*status)(const void *conf,
+		int buf_size, char *action_str);
+};
+
+int parse_flow(char *token_list[], char *response);
+int append_flow_json(int port_id, int buf_size, char *output);
+
+#endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 07/17] spp_primary: add common function of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (23 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 06/17] spp_primary: add support of rte_flow x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 08/17] spp_primary: add attribute " x-fn-spp-ml
                   ` (9 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch provides parse functions those
are commonly used by primary application.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/common.c | 646 ++++++++++++++++++++++++++++++++++++++
 src/primary/flow/common.h |  53 ++++
 src/primary/main.c        |  48 +--
 3 files changed, 725 insertions(+), 22 deletions(-)
 create mode 100644 src/primary/flow/common.c
 create mode 100644 src/primary/flow/common.h

diff --git a/src/primary/flow/common.c b/src/primary/flow/common.c
new file mode 100644
index 0000000..da34619
--- /dev/null
+++ b/src/primary/flow/common.c
@@ -0,0 +1,646 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_ethdev.h>
+
+#include "shared/secondary/spp_worker_th/data_types.h"
+#include "shared/secondary/utils.h"
+#include "flow.h"
+#include "common.h"
+
+/*
+ * Check if port_id is used
+ * Return 0: Port_id used
+ * Return 1: Unused port_id
+ */
+int
+is_portid_used(int port_id)
+{
+	uint16_t pid;
+
+	RTE_ETH_FOREACH_DEV(pid) {
+		if (port_id == pid)
+			return 0;
+	}
+
+	return 1;
+}
+
+/*
+ * Retrieve port ID from source UID of phy port. Return error code if port
+ * type is other than phy.
+ */
+int
+parse_phy_port_id(char *res_uid, int *port_id)
+{
+	int ret;
+	char *port_type;
+	uint16_t queue_id;
+
+	if (res_uid == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"RES UID is NULL(%s:%d)\n", __func__, __LINE__);
+		return -1;
+	}
+
+	ret = parse_resource_uid(res_uid, &port_type, port_id, &queue_id);
+	if (ret < 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Failed to parse RES UID(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	if (strcmp(port_type, SPPWK_PHY_STR) != 0) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"It's not phy type(%s:%d)\n", __func__, __LINE__);
+		return -1;
+	}
+	return 0;
+}
+
+/*
+ * Convert string to rte_ether_addr.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_rte_ether_addr(char *mac_str, void *output)
+{
+	int i = 0;
+	uint8_t byte;
+	char *end, *token;
+	char tmp_mac_str[32] = { 0 };
+	struct rte_ether_addr *mac_addr = output;
+
+	strncpy(tmp_mac_str, mac_str, 32);
+	token = strtok(tmp_mac_str, ":");
+
+	while (token != NULL) {
+		if (i >= RTE_ETHER_ADDR_LEN)
+			return -1;
+
+		byte = (uint8_t)strtoul(token, &end, 16);
+		if (end == NULL || *end != '\0')
+			return -1;
+
+		mac_addr->addr_bytes[i] = byte;
+		i++;
+		token = strtok(NULL, ":");
+	}
+
+	return 0;
+}
+
+/*
+ * Convert string to tci.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_tci(char *tci_str, void *output)
+{
+	char *end;
+	rte_be16_t *tci = output;
+
+	*tci = (rte_be16_t)strtoul(tci_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	return 0;
+}
+
+/*
+ * Convert string to pcp.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_pcp(char *pcp_str, void *output)
+{
+	char *end;
+	uint8_t *pcp = output;
+
+	*pcp = (uint8_t)strtoul(pcp_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	/* 3bit check */
+	if (*pcp > 0x7)
+		return -1;
+
+	return 0;
+}
+
+/*
+ * Set PCP in TCI. TCI is a 16bits value and consists of 3bits PCP,
+ * 1bit DEI and the rest 12bits VID.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+set_pcp_in_tci(char *pcp_str, void *output)
+{
+	int ret;
+	uint8_t pcp = 0;
+	rte_be16_t *tci = output;
+
+	ret = str_to_pcp(pcp_str, &pcp);
+	if (ret != 0)
+		return -1;
+
+	/* Assign to the first 3 bits */
+	pcp = pcp << 1;
+	((char *)tci)[0] = ((char *)tci)[0] | pcp;
+
+	return 0;
+}
+
+/*
+ * Set DEI in TCI. TCI is a 16bits value and consists of 3bits PCP,
+ * 1bit DEI and the rest 12bits VID.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+set_dei_in_tci(char *dei_str, void *output)
+{
+	char *end;
+	uint8_t dei = 0;
+	rte_be16_t *tci = output;
+
+	dei = (uint8_t)strtoul(dei_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	/* 1bit check */
+	if (dei > 0x1)
+		return -1;
+
+	/* Assign to 4th bit */
+	((char *)tci)[0] = ((char *)tci)[0] | dei;
+
+	return 0;
+}
+
+/*
+ * Set VID in TCI. TCI is a 16bits value and consists of 3bits PCP,
+ * 1bit DEI and the rest 12bits VID.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+set_vid_in_tci(char *vid_str, void *output)
+{
+	char *end;
+	rte_be16_t vid = 0;
+	rte_be16_t *tci = output;
+
+	vid = (rte_be16_t)strtoul(vid_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	/* 12bit check */
+	if (vid > 0x0fff)
+		return -1;
+
+	/* Convert vid to big endian if system is little endian. */
+	int i = 1;
+	if (*(char *)&i) { /* check if little endian */
+		uint8_t b1 = ((char *)&vid)[0];
+		uint8_t b2 = ((char *)&vid)[1];
+		((char *)&vid)[0] = b2;
+		((char *)&vid)[1] = b1;
+	}
+
+	/* Assign to 5-16 bit */
+	*tci = *tci | vid;
+
+	return 0;
+}
+
+/*
+ * Convert string to rte_be16_t.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_rte_be16_t(char *target_str, void *output)
+{
+	char *end;
+	rte_be16_t *value = output;
+
+	*value = (rte_be16_t)strtoul(target_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+	/* Convert vid to big endian if system is little endian. */
+	int i = 1;
+	if (*(char *)&i) { /* check if little endian */
+		uint8_t b1 = ((char *)value)[0];
+		uint8_t b2 = ((char *)value)[1];
+		((char *)value)[0] = b2;
+		((char *)value)[1] = b1;
+	}
+
+	return 0;
+}
+
+/*
+ * Convert string to uint16_t.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_uint16_t(char *target_str, void *output)
+{
+	char *end;
+	uint16_t *value = output;
+
+	*value = (uint16_t)strtoul(target_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	return 0;
+}
+
+/*
+ * Convert string to uint32_t.
+ * This function is intended to be called as a function pointer to
+ * 'parse_detail' in 'struct flow_detail_ops'.
+ */
+int
+str_to_uint32_t(char *target_str, void *output)
+{
+	char *end;
+	uint32_t *value = output;
+
+	*value = (uint32_t)strtoul(target_str, &end, 0);
+	if (end == NULL || *end != '\0')
+		return -1;
+
+	return 0;
+}
+
+int
+parse_rte_flow_item_field(char *token_list[], int *index,
+	struct flow_detail_ops *detail_list, size_t size,
+	void **spec, void **last, void **mask
+	)
+{
+	int ret = 0;
+	uint32_t prefix, i, j, bitmask;
+	char *end, *target;
+
+	if (!strcmp(token_list[*index], "is")) {
+		/* Match value perfectly (with full bit-mask). */
+		(*index)++;
+
+		if (*spec == NULL) {
+			ret = malloc_object(spec, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		ret = detail_list->parse_detail(token_list[*index],
+			(char *)(*spec) + detail_list->offset);
+		if (ret != 0) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"parse_detail error (%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+		/* Set full bit-mask */
+		if (*mask == NULL) {
+			ret = malloc_object(mask, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		memset((char *)(*mask) + detail_list->offset, 0xff,
+			detail_list->size);
+
+	} else if (!strcmp(token_list[*index], "spec")) {
+		/* Match value according to configured bit-mask. */
+		(*index)++;
+
+		if (*spec == NULL) {
+			ret = malloc_object(spec, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		ret = detail_list->parse_detail(token_list[*index],
+			(char *)(*spec) + detail_list->offset);
+		if (ret != 0) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"parse_detail error (%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+	} else if (!strcmp(token_list[*index], "last")) {
+		/* Specify upper bound to establish a range. */
+		(*index)++;
+
+		if (*last == NULL) {
+			ret = malloc_object(last, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		ret = detail_list->parse_detail(token_list[*index],
+			(char *)(*last) + detail_list->offset);
+		if (ret != 0) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"parse_detail error (%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+	} else if (!strcmp(token_list[*index], "mask")) {
+		/* Specify bit-mask with relevant bits set to one. */
+		(*index)++;
+
+		if (*mask == NULL) {
+			ret = malloc_object(mask, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		ret = detail_list->parse_detail(token_list[*index],
+			(char *)(*mask) + detail_list->offset);
+		if (ret != 0) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"parse_detail error (%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+	} else if (!strcmp(token_list[*index], "prefix")) {
+		/*
+		 * generate bit-mask with <prefix-length>
+		 * most-significant bits set to one.
+		 */
+		(*index)++;
+
+		if (*mask == NULL) {
+			ret = malloc_object(mask, size);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Failed to alloc memory (%s:%d)\n",
+					__func__, __LINE__);
+				return -1;
+			}
+		}
+
+		prefix = strtoul(token_list[*index], &end, 10);
+		if (end == NULL || *end != '\0') {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Prefix is not a number(%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+		/* Compare prefix (bit) and size (byte). */
+		if (prefix > detail_list->size * 8) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Prefix value is too large(%s:%d)\n",
+				__func__, __LINE__);
+			return -1;
+		}
+
+		target = (char *)(*mask) + detail_list->offset;
+		memset(target, 0, detail_list->size);
+		for (i = 0; i < detail_list->size; i++) {
+			if (prefix <= 0)
+				break;
+
+			bitmask = 0x80;
+
+			for (j = 0; j < 8; j++) {
+				if (prefix <= 0)
+					break;
+
+				target[i] = target[i] | bitmask;
+				bitmask = bitmask >> 1;
+				prefix--;
+			}
+		}
+
+	} else {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Invalid parameter is %s(%s:%d)\n",
+			token_list[*index], __func__, __LINE__);
+		return -1;
+	}
+
+	return 0;
+}
+
+int
+parse_item_common(char *token_list[], int *index,
+	struct rte_flow_item *item,
+	struct flow_item_ops *ops)
+{
+	int ret = 0;
+	int i = 0;
+	//void *spec, *last, *mask;
+	void *spec = NULL, *last = NULL, *mask = NULL;
+	struct flow_detail_ops *detail_list = ops->detail_list;
+
+	/* Next to pattern word */
+	(*index)++;
+
+	while (token_list[*index] != NULL) {
+
+		/* Exit if "/" */
+		if (!strcmp(token_list[*index], "/"))
+			break;
+
+		/* First is value type */
+		i = 0;
+		while (detail_list[i].token != NULL) {
+			if (!strcmp(token_list[*index],
+				detail_list[i].token)) {
+				break;
+			}
+
+			i++;
+		}
+		if (detail_list[i].token == NULL) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Invalid \"%s\" pattern arguments(%s:%d)\n",
+				ops->str_type, __func__, __LINE__);
+			ret = -1;
+			break;
+		}
+
+		/* Parse token value */
+		if (detail_list[i].flg_value == 1) {
+			(*index)++;
+
+			ret = parse_rte_flow_item_field(token_list, index,
+				&detail_list[i], ops->size,
+				&spec, &last, &mask);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Invalid \"%s\" pattern arguments"
+					"(%s:%d)\n",
+					ops->str_type, __func__, __LINE__);
+				ret = -1;
+				break;
+			}
+		}
+
+		(*index)++;
+	}
+
+	/* Free memory allocated in case of failure. */
+	if (ret != 0) {
+		if (spec != NULL)
+			free(spec);
+
+		if (last != NULL)
+			free(last);
+
+		if (mask != NULL)
+			free(mask);
+	}
+
+	/* Parse result to item. */
+	item->spec = spec;
+	item->last = last;
+	item->mask = mask;
+
+	return ret;
+}
+
+int
+parse_action_common(char *token_list[], int *index,
+	struct rte_flow_action *action,
+	struct flow_action_ops *ops)
+{
+	int ret = 0;
+	int i = 0;
+	struct rte_flow_action_queue  *conf;
+	struct flow_detail_ops *detail_list = ops->detail_list;
+
+	conf = malloc(ops->size);
+	if (conf == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+	memset(conf, 0, ops->size);
+
+	/* Next to word */
+	(*index)++;
+
+	while (token_list[*index] != NULL) {
+
+		/* Exit if "/" */
+		if (!strcmp(token_list[*index], "/"))
+			break;
+
+		/* First is value type */
+		i = 0;
+		while (detail_list[i].token != NULL) {
+			if (!strcmp(token_list[*index],
+				detail_list[i].token)) {
+				break;
+			}
+
+			i++;
+		}
+		if (detail_list[i].token == NULL) {
+			RTE_LOG(ERR, SPP_FLOW,
+				"Invalid \"%s\" pattern arguments(%s:%d)\n",
+				ops->str_type, __func__, __LINE__);
+			ret = -1;
+			break;
+		}
+
+		/* Parse token value */
+		if (detail_list[i].flg_value == 1) {
+			(*index)++;
+
+			ret = detail_list[i].parse_detail(
+				token_list[*index],
+				(char *)conf + detail_list[i].offset);
+			if (ret != 0) {
+				RTE_LOG(ERR, SPP_FLOW,
+					"Invalid \"%s\" pattern arguments"
+					"(%s:%d)\n",
+					detail_list[i].token,
+					__func__, __LINE__);
+				ret = -1;
+				break;
+			}
+		}
+
+		(*index)++;
+	}
+
+	/* Free memory allocated in case of failure. */
+	if ((ret != 0) && (conf != NULL))
+		free(conf);
+
+	/* Parse result to action. */
+	action->conf = conf;
+
+	return ret;
+}
+
+/* Append action json, conf field is null */
+int
+append_action_null_json(const void *conf __attribute__ ((unused)),
+	int buf_size, char *action_str)
+{
+	char null_str[] = "null";
+
+	if ((int)strlen(action_str) + (int)strlen(null_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, null_str, strlen(null_str));
+
+	return 0;
+}
+
+int
+malloc_object(void **ptr, size_t size)
+{
+	*ptr = malloc(size);
+	if (*ptr == NULL) {
+		RTE_LOG(ERR, SPP_FLOW,
+			"Memory allocation failure(%s:%d)\n",
+			__func__, __LINE__);
+		return -1;
+	}
+
+	memset(*ptr, 0, size);
+	return 0;
+}
diff --git a/src/primary/flow/common.h b/src/primary/flow/common.h
new file mode 100644
index 0000000..998fd12
--- /dev/null
+++ b/src/primary/flow/common.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_COMMON_H_
+#define _PRIMARY_FLOW_COMMON_H_
+
+int is_portid_used(int port_id);
+
+/* Function for flow command parse */
+int parse_phy_port_id(char *res_uid, int *port_id);
+
+/* Functions for converting string to data */
+int str_to_rte_ether_addr(char *mac_str, void *output);
+int str_to_tci(char *tci_str, void *output);
+int str_to_pcp(char *pcp_str, void *output);
+int str_to_rte_be16_t(char *target_str, void *output);
+int str_to_uint16_t(char *target_str, void *output);
+int str_to_uint32_t(char *target_str, void *output);
+
+/* Functions for setting string to data */
+int set_pcp_in_tci(char *pcp_str, void *output);
+int set_dei_in_tci(char *dei_str, void *output);
+int set_vid_in_tci(char *vid_str, void *output);
+
+/* Parse rte_flow_item for each field */
+int parse_rte_flow_item_field(char *token_list[], int *index,
+	struct flow_detail_ops *detail_list, size_t size,
+	void **spec, void **last, void **mask);
+
+/*
+ * Common parse for item type. Perform detailed parse with
+ * flow_detail_ops according to type.
+ */
+int parse_item_common(char *token_list[], int *index,
+	struct rte_flow_item *item,
+	struct flow_item_ops *ops);
+
+/*
+ * Common parse for action type. Perform detailed parse with
+ * flow_detail_ops according to type.
+ */
+int parse_action_common(char *token_list[], int *index,
+	struct rte_flow_action *action,
+	struct flow_action_ops *ops);
+
+/* Append action json, conf field is null */
+int append_action_null_json(const void *conf, int buf_size, char *action_str);
+
+/* Allocate memory for the size */
+int malloc_object(void **ptr, size_t size);
+
+#endif
diff --git a/src/primary/main.c b/src/primary/main.c
index ca81636..4ef9cb1 100644
--- a/src/primary/main.c
+++ b/src/primary/main.c
@@ -395,9 +395,7 @@ append_lcore_info_json(char *str,
 
 /* TODO(yasufum): change to use shared */
 static int
-append_port_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+append_port_info_json(char *str)
 {
 	unsigned int i;
 	unsigned int has_port = 0;  // for checking having port at last
@@ -405,7 +403,7 @@ append_port_info_json(char *str,
 	sprintf(str + strlen(str), "\"ports\":[");
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
+		if (ports_fwd_array[i][0].in_port_id == PORT_RESET)
 			continue;
 
 		has_port = 1;
@@ -457,9 +455,7 @@ append_port_info_json(char *str,
 
 /* TODO(yasufum): change to use shared */
 static int
-append_patch_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+append_patch_info_json(char *str)
 {
 	unsigned int i;
 	unsigned int has_patch = 0;  // for checking having patch at last
@@ -468,12 +464,12 @@ append_patch_info_json(char *str,
 	sprintf(str + strlen(str), "\"patches\":[");
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
+		if (ports_fwd_array[i][0].in_port_id == PORT_RESET)
 			continue;
 
 		RTE_LOG(INFO, PRIMARY, "Port ID %d\n", i);
 		RTE_LOG(INFO, PRIMARY, "Status %d\n",
-			ports_fwd_array[i].in_port_id);
+			ports_fwd_array[i][0].in_port_id);
 
 		memset(patch_str, '\0', sizeof(patch_str));
 
@@ -533,14 +529,14 @@ append_patch_info_json(char *str,
 		sprintf(patch_str + strlen(patch_str), "\"dst\":");
 
 		RTE_LOG(INFO, PRIMARY, "Out Port ID %d\n",
-				ports_fwd_array[i].out_port_id);
+				ports_fwd_array[i][0].out_port_id);
 
-		if (ports_fwd_array[i].out_port_id == PORT_RESET) {
+		if (ports_fwd_array[i][0].out_port_id == PORT_RESET) {
 			//sprintf(patch_str + strlen(patch_str), "%s", "\"\"");
 			continue;
 		} else {
 			has_patch = 1;
-			unsigned int j = ports_fwd_array[i].out_port_id;
+			unsigned int j = ports_fwd_array[i][0].out_port_id;
 			switch (port_map[j].port_type) {
 			case PHY:
 				RTE_LOG(INFO, PRIMARY, "Type: PHY\n");
@@ -629,8 +625,8 @@ forwarder_status_json(char *str)
 	else
 		sprintf(buf_running + strlen(buf_running), "\"%s\"", "idling");
 
-	append_port_info_json(buf_ports, ports_fwd_array, port_map);
-	append_patch_info_json(buf_patches, ports_fwd_array, port_map);
+	append_port_info_json(buf_ports);
+	append_patch_info_json(buf_patches);
 
 	sprintf(str, "\"forwarder\":{%s,%s,%s}", buf_running, buf_ports,
 			buf_patches);
@@ -871,7 +867,7 @@ add_port(char *p_type, int p_id)
 	 */
 
 	/* Update ports_fwd_array with port id */
-	ports_fwd_array[port_id].in_port_id = port_id;
+	ports_fwd_array[port_id][0].in_port_id = port_id;
 	return 0;
 }
 
@@ -942,7 +938,7 @@ del_port(char *p_type, int p_id)
 	port_id_list[dev_id].port_id = PORT_RESET;
 	port_id_list[dev_id].type = UNDEF;
 
-	forward_array_remove(dev_id);
+	forward_array_remove(dev_id, 0);
 	port_map_init_one(dev_id);
 
 	return 0;
@@ -964,6 +960,7 @@ parse_command(char *str)
 	char *p_type;
 	int p_id;
 	char tmp_response[MSG_SIZE];
+	uint16_t queue_id;
 
 	memset(sec_name, '\0', 16);
 	memset(tmp_response, '\0', MSG_SIZE);
@@ -1033,7 +1030,8 @@ parse_command(char *str)
 		RTE_LOG(DEBUG, PRIMARY, "'%s' command received.\n",
 				token_list[0]);
 
-		ret = parse_resource_uid(token_list[1], &p_type, &p_id);
+		ret = parse_resource_uid(token_list[1], &p_type, &p_id,
+			&queue_id);
 		if (ret < 0) {
 			RTE_LOG(ERR, PRIMARY, "Failed to parse RES UID.\n");
 			return ret;
@@ -1055,7 +1053,8 @@ parse_command(char *str)
 	} else if (!strcmp(token_list[0], "del")) {
 		RTE_LOG(DEBUG, PRIMARY, "Received del command\n");
 
-		ret = parse_resource_uid(token_list[1], &p_type, &p_id);
+		ret = parse_resource_uid(token_list[1], &p_type, &p_id,
+			&queue_id);
 		if (ret < 0) {
 			RTE_LOG(ERR, PRIMARY, "Failed to parse RES UID.\n");
 			return ret;
@@ -1094,13 +1093,15 @@ parse_command(char *str)
 			char *out_p_type;
 			int in_p_id;
 			int out_p_id;
+			uint16_t in_queue_id, out_queue_id;
 
-			parse_resource_uid(token_list[1], &in_p_type, &in_p_id);
+			parse_resource_uid(token_list[1], &in_p_type, &in_p_id,
+				&in_queue_id);
 			in_port = find_port_id(in_p_id,
 					get_port_type(in_p_type));
 
 			parse_resource_uid(token_list[2],
-					&out_p_type, &out_p_id);
+					&out_p_type, &out_p_id, &out_queue_id);
 			out_port = find_port_id(out_p_id,
 					get_port_type(out_p_type));
 
@@ -1128,7 +1129,8 @@ parse_command(char *str)
 				RTE_LOG(ERR, PRIMARY, "%s\n", err_msg);
 			}
 
-			if (add_patch(in_port, out_port) == 0) {
+			if (add_patch(in_port, in_queue_id, out_port,
+				out_queue_id) == 0) {
 				RTE_LOG(INFO, PRIMARY,
 					"Patched '%s:%d' and '%s:%d'\n",
 					in_p_type, in_p_id,
@@ -1323,10 +1325,12 @@ main(int argc, char *argv[])
 			}
 
 			/* Update ports_fwd_array with phy port. */
-			ports_fwd_array[i].in_port_id = i;
+			ports_fwd_array[i][0].in_port_id = i;
+			ports_fwd_array[i][0].in_queue_id = 0;
 			port_map[i].port_type = port_type;
 			port_map[i].id = port_id;
 			port_map[i].stats = &ports->port_stats[i];
+			port_map[i].queue_info = NULL;
 
 			/* TODO(yasufum) convert type of port_type to char */
 			RTE_LOG(DEBUG, PRIMARY, "Add port, type: %d, id: %d\n",
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 08/17] spp_primary: add attribute of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (24 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 07/17] spp_primary: add common function " x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 09/17] spp_primary: add patterns " x-fn-spp-ml
                   ` (8 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch provides support of various
attribute of rte_flow.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/attr.c | 105 ++++++++++++++++++++++++++++++++++++++++
 src/primary/flow/attr.h |  13 +++++
 2 files changed, 118 insertions(+)
 create mode 100644 src/primary/flow/attr.c
 create mode 100644 src/primary/flow/attr.h

diff --git a/src/primary/flow/attr.c b/src/primary/flow/attr.c
new file mode 100644
index 0000000..8be7319
--- /dev/null
+++ b/src/primary/flow/attr.c
@@ -0,0 +1,105 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "attr.h"
+
+int
+parse_flow_attr(char *token_list[], int *index,
+	struct rte_flow_attr *attr)
+{
+	int ret;
+	char *token;
+	char *end;
+	unsigned long temp = 0;
+
+	while (token_list[*index] != NULL) {
+		token = token_list[*index];
+
+		if (!strcmp(token, "group")) {
+			/* "group" requires option argument */
+			if (token_list[*index + 1] == NULL) {
+				ret = -1;
+				break;
+			}
+
+			temp = strtoul(token_list[*index + 1], &end, 10);
+			if (end == NULL || *end != '\0') {
+				ret = -1;
+				break;
+			}
+
+			attr->group = (uint32_t)temp;
+			(*index)++;
+
+		} else if (!strcmp(token, "priority")) {
+			/* "priority" requires option argument */
+			if (token_list[*index + 1] == NULL) {
+				ret = -1;
+				break;
+			}
+
+			temp = strtoul(token_list[*index + 1], &end, 10);
+			if (end == NULL || *end != '\0') {
+				ret = -1;
+				break;
+			}
+
+			attr->priority = (uint32_t)temp;
+			(*index)++;
+
+		} else if (!strcmp(token, "ingress")) {
+			attr->ingress = 1;
+
+		} else if (!strcmp(token, "egress")) {
+			attr->egress = 1;
+
+		} else if (!strcmp(token, "transfer")) {
+			attr->transfer = 1;
+
+		} else if (!strcmp(token, "pattern")) {
+			/* Attribute parameter end */
+			ret = 0;
+			break;
+
+		} else {
+			/* Illegal parameter */
+			ret = -1;
+			break;
+
+		}
+
+		(*index)++;
+	}
+
+	if (token_list[*index] == NULL)
+		ret = -1;
+
+	return ret;
+}
+
+int
+append_flow_attr_json(const struct rte_flow_attr *attr, int buf_size,
+	char *attr_str)
+{
+	char tmp_str[128] = { 0 };
+
+	snprintf(tmp_str, 128,
+		"{\"group\":%d,"
+		"\"priority\":%d,"
+		"\"ingress\":%d,"
+		"\"egress\":%d,"
+		"\"transfer\":%d}",
+		attr->group, attr->priority, attr->ingress,
+		attr->egress, attr->transfer);
+
+	if ((int)strlen(attr_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(attr_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/attr.h b/src/primary/flow/attr.h
new file mode 100644
index 0000000..5fb22e3
--- /dev/null
+++ b/src/primary/flow/attr.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ATTR_H_
+#define _PRIMARY_FLOW_ATTR_H_
+
+int parse_flow_attr(char *token_list[], int *index,
+	struct rte_flow_attr *attr);
+int append_flow_attr_json(const struct rte_flow_attr *attr,
+	int buf_size, char *attr_str);
+
+#endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 09/17] spp_primary: add patterns of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (25 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 08/17] spp_primary: add attribute " x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 10/17] spp_primary: add actions " x-fn-spp-ml
                   ` (7 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch adds the following
patterns of rte_flow.
   - eth
   - vlan
Additional files should be added when new pattern should be
newly supported by SPP.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/pattern/eth.c  | 63 +++++++++++++++++++++++++++++
 src/primary/flow/pattern/eth.h  | 13 ++++++
 src/primary/flow/pattern/vlan.c | 71 +++++++++++++++++++++++++++++++++
 src/primary/flow/pattern/vlan.h | 13 ++++++
 4 files changed, 160 insertions(+)
 create mode 100644 src/primary/flow/pattern/eth.c
 create mode 100644 src/primary/flow/pattern/eth.h
 create mode 100644 src/primary/flow/pattern/vlan.c
 create mode 100644 src/primary/flow/pattern/vlan.h

diff --git a/src/primary/flow/pattern/eth.c b/src/primary/flow/pattern/eth.c
new file mode 100644
index 0000000..d718b77
--- /dev/null
+++ b/src/primary/flow/pattern/eth.c
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "eth.h"
+
+/* Define item "eth" operations */
+struct flow_detail_ops eth_ops_list[] = {
+	{
+		.token = "src",
+		.offset = offsetof(struct rte_flow_item_eth, src),
+		.size = sizeof(struct rte_ether_addr),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_ether_addr,
+	},
+	{
+		.token = "dst",
+		.offset = offsetof(struct rte_flow_item_eth, dst),
+		.size = sizeof(struct rte_ether_addr),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_ether_addr,
+	},
+	{
+		.token = "type",
+		.offset = offsetof(struct rte_flow_item_eth, type),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_be16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_item_eth_json(const void *element, int buf_size, char *pattern_str)
+{
+	const struct rte_flow_item_eth *eth = element;
+	char dst_mac[RTE_ETHER_ADDR_FMT_SIZE] = { 0 };
+	char src_mac[RTE_ETHER_ADDR_FMT_SIZE] = { 0 };
+	char tmp_str[128] = { 0 };
+
+	rte_ether_format_addr(dst_mac, RTE_ETHER_ADDR_FMT_SIZE, &eth->dst);
+	rte_ether_format_addr(src_mac, RTE_ETHER_ADDR_FMT_SIZE, &eth->src);
+
+	snprintf(tmp_str, 128,
+		"{\"dst\":\"%s\","
+		"\"src\":\"%s\","
+		"\"type\":\"0x%04x\"}",
+		dst_mac, src_mac, eth->type);
+
+	if ((int)strlen(pattern_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(pattern_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/pattern/eth.h b/src/primary/flow/pattern/eth.h
new file mode 100644
index 0000000..550813e
--- /dev/null
+++ b/src/primary/flow/pattern/eth.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_PATTERN_ETH_H_
+#define _PRIMARY_FLOW_PATTERN_ETH_H_
+
+extern struct flow_detail_ops eth_ops_list[];
+
+int append_item_eth_json(const void *element, int buf_size,
+	char *pattern_str);
+
+#endif
diff --git a/src/primary/flow/pattern/vlan.c b/src/primary/flow/pattern/vlan.c
new file mode 100644
index 0000000..3ca8ebc
--- /dev/null
+++ b/src/primary/flow/pattern/vlan.c
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "vlan.h"
+
+/* Define item "vlan" operations */
+struct flow_detail_ops vlan_ops_list[] = {
+	{
+		.token = "tci",
+		.offset = offsetof(struct rte_flow_item_vlan, tci),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_tci,
+	},
+	{
+		.token = "pcp",
+		.offset = offsetof(struct rte_flow_item_vlan, tci),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = set_pcp_in_tci,
+	},
+	{
+		.token = "dei",
+		.offset = offsetof(struct rte_flow_item_vlan, tci),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = set_dei_in_tci,
+	},
+	{
+		.token = "vid",
+		.offset = offsetof(struct rte_flow_item_vlan, tci),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = set_vid_in_tci,
+	},
+	{
+		.token = "inner_type",
+		.offset = offsetof(struct rte_flow_item_vlan, inner_type),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_be16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_item_vlan_json(const void *element, int buf_size, char *pattern_str)
+{
+	const struct rte_flow_item_vlan *vlan = element;
+	char tmp_str[128] = { 0 };
+
+	snprintf(tmp_str, 128,
+		"{\"tci\":\"0x%04x\","
+		"\"inner_type\":\"0x%04x\"}",
+		vlan->tci, vlan->inner_type);
+
+	if ((int)strlen(pattern_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(pattern_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/pattern/vlan.h b/src/primary/flow/pattern/vlan.h
new file mode 100644
index 0000000..50aef96
--- /dev/null
+++ b/src/primary/flow/pattern/vlan.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_PATTERN_VLAN_H_
+#define _PRIMARY_FLOW_PATTERN_VLAN_H_
+
+extern struct flow_detail_ops vlan_ops_list[];
+
+int append_item_vlan_json(const void *element, int buf_size,
+	char *pattern_str);
+
+#endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 10/17] spp_primary: add actions of rte_flow
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (26 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 09/17] spp_primary: add patterns " x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 11/17] bin: add parameter for hardrare offload x-fn-spp-ml
                   ` (6 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To support rte_flow in SPP, this patch adds the following
actions of rte_flow.
   - queue
   - jump
   - pop_vlan
   - push_vlan
   - set_vlan_vid
   - set_vlan_pcp
Additional files should be added when new actions should be
newly supported by SPP.

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/primary/flow/action/jump.c            | 42 ++++++++++++++++++++++
 src/primary/flow/action/jump.h            | 12 +++++++
 src/primary/flow/action/of_push_vlan.c    | 44 +++++++++++++++++++++++
 src/primary/flow/action/of_push_vlan.h    | 13 +++++++
 src/primary/flow/action/of_set_vlan_pcp.c | 44 +++++++++++++++++++++++
 src/primary/flow/action/of_set_vlan_pcp.h | 13 +++++++
 src/primary/flow/action/of_set_vlan_vid.c | 44 +++++++++++++++++++++++
 src/primary/flow/action/of_set_vlan_vid.h | 13 +++++++
 src/primary/flow/action/queue.c           | 42 ++++++++++++++++++++++
 src/primary/flow/action/queue.h           | 13 +++++++
 10 files changed, 280 insertions(+)
 create mode 100644 src/primary/flow/action/jump.c
 create mode 100644 src/primary/flow/action/jump.h
 create mode 100644 src/primary/flow/action/of_push_vlan.c
 create mode 100644 src/primary/flow/action/of_push_vlan.h
 create mode 100644 src/primary/flow/action/of_set_vlan_pcp.c
 create mode 100644 src/primary/flow/action/of_set_vlan_pcp.h
 create mode 100644 src/primary/flow/action/of_set_vlan_vid.c
 create mode 100644 src/primary/flow/action/of_set_vlan_vid.h
 create mode 100644 src/primary/flow/action/queue.c
 create mode 100644 src/primary/flow/action/queue.h

diff --git a/src/primary/flow/action/jump.c b/src/primary/flow/action/jump.c
new file mode 100644
index 0000000..b643015
--- /dev/null
+++ b/src/primary/flow/action/jump.c
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "jump.h"
+
+/* Define action "jump" operations */
+struct flow_detail_ops jump_ops_list[] = {
+	{
+		.token = "group",
+		.offset = offsetof(struct rte_flow_action_jump, group),
+		.size = sizeof(uint32_t),
+		.flg_value = 1,
+		.parse_detail = str_to_uint32_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_jump_json(const void *conf, int buf_size, char *action_str)
+{
+	const struct rte_flow_action_jump *jump = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"group\":%d}",
+		jump->group);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/jump.h b/src/primary/flow/action/jump.h
new file mode 100644
index 0000000..f8c6a60
--- /dev/null
+++ b/src/primary/flow/action/jump.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_JUMP_H_
+#define _PRIMARY_FLOW_ACTION_JUMP_H_
+
+extern struct flow_detail_ops jump_ops_list[];
+
+int append_action_jump_json(const void *conf, int buf_size, char *action_str);
+
+#endif
diff --git a/src/primary/flow/action/of_push_vlan.c b/src/primary/flow/action/of_push_vlan.c
new file mode 100644
index 0000000..749490e
--- /dev/null
+++ b/src/primary/flow/action/of_push_vlan.c
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: BSD-3-Claus1
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "of_push_vlan.h"
+
+/* Define action "of_push_vlan" operations */
+struct flow_detail_ops of_push_vlan_ops_list[] = {
+	{
+		.token = "ethertype",
+		.offset = offsetof(struct rte_flow_action_of_push_vlan,
+			ethertype),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_be16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_of_push_vlan_json(const void *conf, int buf_size,
+	char *action_str)
+{
+	const struct rte_flow_action_of_push_vlan *of_push_vlan = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"ethertype\":\"0x%04x\"}",
+		of_push_vlan->ethertype);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/of_push_vlan.h b/src/primary/flow/action/of_push_vlan.h
new file mode 100644
index 0000000..7c37ab1
--- /dev/null
+++ b/src/primary/flow/action/of_push_vlan.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_OF_PUSH_VLAN_H_
+#define _PRIMARY_FLOW_ACTION_OF_PUSH_VLAN_H_
+
+extern struct flow_detail_ops of_push_vlan_ops_list[];
+
+int append_action_of_push_vlan_json(const void *conf, int buf_size,
+	char *action_str);
+
+#endif
diff --git a/src/primary/flow/action/of_set_vlan_pcp.c b/src/primary/flow/action/of_set_vlan_pcp.c
new file mode 100644
index 0000000..33e23d5
--- /dev/null
+++ b/src/primary/flow/action/of_set_vlan_pcp.c
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "of_set_vlan_pcp.h"
+
+/* Define action "of_set_vlan_pcp" operations */
+struct flow_detail_ops of_set_vlan_pcp_ops_list[] = {
+	{
+		.token = "vlan_pcp",
+		.offset = offsetof(struct rte_flow_action_of_set_vlan_pcp,
+			vlan_pcp),
+		.size = sizeof(uint8_t),
+		.flg_value = 1,
+		.parse_detail = str_to_pcp,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_of_set_vlan_pcp_json(const void *conf, int buf_size,
+	char *action_str)
+{
+	const struct rte_flow_action_of_set_vlan_pcp *pcp = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"vlan_pcp\":\"0x%01x\"}",
+		pcp->vlan_pcp);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/of_set_vlan_pcp.h b/src/primary/flow/action/of_set_vlan_pcp.h
new file mode 100644
index 0000000..3f8a4b3
--- /dev/null
+++ b/src/primary/flow/action/of_set_vlan_pcp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_OF_SET_VLAN_PCP_H_
+#define _PRIMARY_FLOW_ACTION_OF_SET_VLAN_PCP_H_
+
+extern struct flow_detail_ops of_set_vlan_pcp_ops_list[];
+
+int append_action_of_set_vlan_pcp_json(const void *conf, int buf_size,
+	char *action_str);
+
+#endif
diff --git a/src/primary/flow/action/of_set_vlan_vid.c b/src/primary/flow/action/of_set_vlan_vid.c
new file mode 100644
index 0000000..930cef5
--- /dev/null
+++ b/src/primary/flow/action/of_set_vlan_vid.c
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "of_set_vlan_vid.h"
+
+/* Define action "of_set_vlan_vid" operations */
+struct flow_detail_ops of_set_vlan_vid_ops_list[] = {
+	{
+		.token = "vlan_vid",
+		.offset = offsetof(struct rte_flow_action_of_set_vlan_vid,
+			vlan_vid),
+		.size = sizeof(rte_be16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_rte_be16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_of_set_vlan_vid_json(const void *conf, int buf_size,
+	char *action_str)
+{
+	const struct rte_flow_action_of_set_vlan_vid *vid = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"vlan_vid\":\"0x%04x\"}",
+		vid->vlan_vid);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/of_set_vlan_vid.h b/src/primary/flow/action/of_set_vlan_vid.h
new file mode 100644
index 0000000..a0abe85
--- /dev/null
+++ b/src/primary/flow/action/of_set_vlan_vid.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_OF_SET_VLAN_VID_H_
+#define _PRIMARY_FLOW_ACTION_OF_SET_VLAN_VID_H_
+
+extern struct flow_detail_ops of_set_vlan_vid_ops_list[];
+
+int append_action_of_set_vlan_vid_json(const void *conf, int buf_size,
+	char *action_str);
+
+#endif
diff --git a/src/primary/flow/action/queue.c b/src/primary/flow/action/queue.c
new file mode 100644
index 0000000..f7bcdce
--- /dev/null
+++ b/src/primary/flow/action/queue.c
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#include <rte_flow.h>
+
+#include "primary/flow/flow.h"
+#include "primary/flow/common.h"
+#include "queue.h"
+
+/* Define action "queue" operations */
+struct flow_detail_ops queue_ops_list[] = {
+	{
+		.token = "index",
+		.offset = offsetof(struct rte_flow_action_queue, index),
+		.size = sizeof(uint16_t),
+		.flg_value = 1,
+		.parse_detail = str_to_uint16_t,
+	},
+	{
+		.token = NULL,
+	},
+};
+
+int
+append_action_queue_json(const void *conf, int buf_size, char *action_str)
+{
+	const struct rte_flow_action_queue *queue = conf;
+	char tmp_str[64] = { 0 };
+
+	snprintf(tmp_str, 64,
+		"{\"index\":%d}",
+		queue->index);
+
+	if ((int)strlen(action_str) + (int)strlen(tmp_str)
+		> buf_size)
+		return -1;
+
+	strncat(action_str, tmp_str, strlen(tmp_str));
+
+	return 0;
+}
diff --git a/src/primary/flow/action/queue.h b/src/primary/flow/action/queue.h
new file mode 100644
index 0000000..22816c7
--- /dev/null
+++ b/src/primary/flow/action/queue.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+
+#ifndef _PRIMARY_FLOW_ACTION_QUEUE_H_
+#define _PRIMARY_FLOW_ACTION_QUEUE_H_
+
+extern struct flow_detail_ops queue_ops_list[];
+
+int append_action_queue_json(const void *conf, int buf_size,
+	char *action_str);
+
+#endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 11/17] bin: add parameter for hardrare offload
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (27 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 10/17] spp_primary: add actions " x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 12/17] cli: add support of hardware offload x-fn-spp-ml
                   ` (5 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch adds config for the following to support hardware offload:
  - multi-queue setting
  - whitelist

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 bin/sample/config.sh | 15 +++++++++++++++
 bin/spp_pri.sh       | 26 +++++++++++++++++++++++++-
 bin/start.sh         |  2 ++
 3 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/bin/sample/config.sh b/bin/sample/config.sh
index ec4d4b8..214757e 100644
--- a/bin/sample/config.sh
+++ b/bin/sample/config.sh
@@ -29,3 +29,18 @@ PRI_PORTMASK=0x03  # total num of ports of spp_primary.
 # Log files are created in 'spp/log/'.
 SPP_CTL_LOG=spp_ctl.log
 PRI_LOG=spp_primary.log
+
+# number of ports for multi-queue setting.
+#PRI_PORT_QUEUE=(
+#    "0 rxq 16 txq 16"
+#)
+
+# Add a PCI device in white list.
+# `dv_flow_en=1` is required for HW offload with Mellanox NIC.
+# Set a nonzero value to enables the DV flow steering assuming it is
+# supported by the driver.
+# https://doc.dpdk.org/guides/nics/mlx5.html
+#PRI_WHITE_LIST=(
+#    "0000:04:00.0,dv_flow_en=1"
+#    "0000:05:00.0"
+#)
diff --git a/bin/spp_pri.sh b/bin/spp_pri.sh
index c0b0e1e..6238ee1 100644
--- a/bin/spp_pri.sh
+++ b/bin/spp_pri.sh
@@ -9,6 +9,8 @@ SPP_PRI_RING=""
 SPP_PRI_TAP=""
 SPP_PRI_MEMIF=""
 SPP_PRI_VDEVS=""
+SPP_PRI_PORT_QUEUE=""
+SPP_PRI_WHITE_LIST=""
 
 SOCK_VHOST="/tmp/sock"
 SOCK_MEMIF="/tmp/spp-memif.sock"
@@ -64,6 +66,26 @@ function setup_vdevs() {
     fi
 }
 
+# Add queue number to port
+function setup_queue_number() {
+    if [ ${#PRI_PORT_QUEUE[@]} ]; then
+        for (( i=0; i < ${#PRI_PORT_QUEUE[@]}; i++)); do
+            SPP_PRI_PORT_QUEUE="
+                ${SPP_PRI_PORT_QUEUE} --port-num ${PRI_PORT_QUEUE[${i}]}"
+        done
+    fi
+}
+
+# Add whitelist
+function setup_whitelist() {
+    if [ ${#PRI_WHITE_LIST[@]} ]; then
+        for (( i=0; i < ${#PRI_WHITE_LIST[@]}; i++)); do
+            SPP_PRI_WHITE_LIST="
+                ${SPP_PRI_WHITE_LIST} -w ${PRI_WHITE_LIST[${i}]}"
+        done
+    fi
+}
+
 # Launch spp_primary.
 function spp_pri() {
     SPP_PRI_BIN=${SPP_DIR}/src/primary/${RTE_TARGET}/spp_primary
@@ -86,10 +108,12 @@ function spp_pri() {
         ${SPP_PRI_TAP} \
         ${SPP_PRI_MEMIF} \
         ${SPP_PRI_VDEVS} \
+        ${SPP_PRI_WHITE_LIST} \
         -- \
         -p ${PRI_PORTMASK} \
         -n ${NUM_RINGS} \
-        -s ${SPP_CTL_IP}:5555"
+        -s ${SPP_CTL_IP}:5555 \
+        ${SPP_PRI_PORT_QUEUE}"
 
     if [ ${DRY_RUN} ]; then
         echo ${cmd}
diff --git a/bin/start.sh b/bin/start.sh
index eecb2ed..28dc3d9 100755
--- a/bin/start.sh
+++ b/bin/start.sh
@@ -45,6 +45,8 @@ function start_spp_pri() {
     setup_tap_vdevs  # setup vdevs of net_tap
     setup_memif_vdevs  # setup vdevs of net_memif
     setup_vdevs  # setup any of vdevs
+    setup_queue_number  # setup port number
+    setup_whitelist  # setup whitelist
     spp_pri  # launch spp_primary
 }
 
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 12/17] cli: add support of hardware offload
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (28 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 11/17] bin: add parameter for hardrare offload x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 13/17] cli: add support of rte_flow in vf x-fn-spp-ml
                   ` (4 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch adds support of flow command in primary process
with the following aspects.
1. adds completion of flow command.
2. display status of flow rules
3. support of the following flow command.
   - pri; flow create
   - pri; flow validate
   - pri; flow destroy

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/cli/commands/pri.py                    |  52 +-
 src/cli/commands/pri_flow.py               | 936 +++++++++++++++++++++
 src/cli/commands/pri_flow_compl_action.py  | 115 +++
 src/cli/commands/pri_flow_compl_pattern.py |  84 ++
 4 files changed, 1180 insertions(+), 7 deletions(-)
 create mode 100644 src/cli/commands/pri_flow.py
 create mode 100644 src/cli/commands/pri_flow_compl_action.py
 create mode 100644 src/cli/commands/pri_flow_compl_pattern.py

diff --git a/src/cli/commands/pri.py b/src/cli/commands/pri.py
index a13137d..dbe56d5 100644
--- a/src/cli/commands/pri.py
+++ b/src/cli/commands/pri.py
@@ -4,6 +4,7 @@
 from .. import spp_common
 from ..shell_lib import common
 from ..spp_common import logger
+from .pri_flow import SppPrimaryFlow
 import os
 import time
 
@@ -20,7 +21,7 @@ class SppPrimary(object):
 
     # All of primary commands used for validation and completion.
     PRI_CMDS = ['status', 'add', 'del', 'forward', 'stop', 'patch',
-                'launch', 'clear']
+                'launch', 'clear', 'flow']
 
     ENV_FILE_PREF = 'SPP_FILE_PREFIX'
 
@@ -46,6 +47,8 @@ class SppPrimary(object):
         temp = temp + "{vhost_cli}"
         self.launch_template = temp
 
+        self.flow = SppPrimaryFlow(spp_ctl_cli)
+
     def _do_if_forwarder_exists(self, status, func, params):
         """Execute command of func if forwarder thread is existing.
 
@@ -126,6 +129,9 @@ class SppPrimary(object):
                 else:
                     print('Error: unknown response for clear.')
 
+        elif subcmd == 'flow':
+            self._run_flow(params)
+
         else:
             print('Invalid pri command!')
 
@@ -185,8 +191,8 @@ class SppPrimary(object):
                 - phy:1
             - stats
               - physical ports:
-                  ID          rx          tx     tx_drop  mac_addr
-                   0    78932932    78932931           1  56:48:4f:53:54:00
+                  ID          rx          tx     tx_drop  rxq  txq  mac_addr
+                   0    78932932    78932931           1   16   16  56:48:...
                    ...
               - ring ports:
                   ID          rx          tx     rx_drop     tx_drop
@@ -228,15 +234,19 @@ class SppPrimary(object):
 
             if 'phy_ports' in json_obj:
                 print('  - physical ports:')
-                print('{s6}ID{s10}rx{s10}tx{s4}tx_drop  mac_addr'.format(
-                      s4=sep*4, s6=sep*6, s10=sep*10))
+                print('{s6}ID{s10}rx{s10}tx{s5}tx_drop'
+                      '  rxq  txq mac_addr'.format(
+                          s5=sep*5, s6=sep*6, s10=sep*10))
 
-                temp = '{s6}{portid:2}  {rx:10}  {tx:10}  {tx_d:10}  {eth}'
+                temp = '{s6}{portid:2}  {rx:10}  {tx:10}  {tx_d:10}' \
+                    ' {rxq:4} {txq:4} {eth}'
                 for pports in json_obj['phy_ports']:
                     print(temp.format(s6=sep*6,
                                       portid=pports['id'], rx=pports['rx'],
                                       tx=pports['tx'],
                                       tx_d=pports['tx_drop'],
+                                      rxq=pports["nof_queues"]["rx"],
+                                      txq=pports["nof_queues"]["tx"],
                                       eth=pports['eth']))
 
             if 'ring_ports' in json_obj:
@@ -560,9 +570,20 @@ class SppPrimary(object):
                         candidates = self._compl_del(tokens[1:])
                     elif tokens[1] == 'patch':
                         candidates = self._compl_patch(tokens[1:])
+                    elif tokens[1] == 'flow':
+                        candidates = self._compl_flow(tokens[1:])
 
+            completions = []
             if not text:
-                completions = candidates
+                for candidate in candidates:
+                    if not candidate.startswith(tokens[len(tokens) - 1]):
+                        completions.append(candidate)
+                        continue
+
+                    if ":" in tokens[len(tokens) - 1]:
+                        completions.append(candidate.split(":")[1])
+                    else:
+                        completions.append(candidate)
             else:
                 completions = [p for p in candidates
                                if p.startswith(text)
@@ -717,6 +738,10 @@ class SppPrimary(object):
 
             return res
 
+    def _compl_flow(self, tokens):
+        """Complete `flow` command."""
+        return self.flow.complete_flow(tokens)
+
     def _get_sec_ids(self):
         sec_ids = []
         res = self.spp_ctl_cli.get('processes')
@@ -745,6 +770,7 @@ class SppPrimary(object):
         """
         prekey = None
         opts_dict = {}
+        index = 0
         for opt in opts_list:
             if opt.startswith('-'):
                 opts_dict[opt] = None
@@ -752,7 +778,15 @@ class SppPrimary(object):
             else:
                 if prekey is not None:
                     opts_dict[prekey] = opt
+
+                    if (opt.startswith("phy:")
+                            and len(opts_list) > index + 2
+                            and opts_list[index + 1] == "nq"):
+                        opts_dict[prekey] += " nq {0}".format(
+                            opts_list[index + 2])
+
                     prekey = None
+            index += 1
         return opts_dict
 
     def _run_add(self, params):
@@ -980,6 +1014,10 @@ class SppPrimary(object):
             else:
                 print('Error: unknown response for launch.')
 
+    def _run_flow(self, params):
+        """Run `flow` command."""
+        self.flow.run_flow(params)
+
     @classmethod
     def help(cls):
         msg = """Send a command to primary process.
diff --git a/src/cli/commands/pri_flow.py b/src/cli/commands/pri_flow.py
new file mode 100644
index 0000000..de91a25
--- /dev/null
+++ b/src/cli/commands/pri_flow.py
@@ -0,0 +1,936 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+import re
+import importlib
+import copy
+from . import pri_flow_compl_pattern as flow_compl_ptn
+from . import pri_flow_compl_action as flow_compl_act
+
+
+class SppPrimaryFlow(object):
+    """Exec SPP primary flow command.
+
+    SppPrimaryFlow class is intended to be used in SppPrimary class as a
+    delegator for running 'flow' subcommand.
+
+    """
+
+    # All of flow commands
+    FLOW_CMDS = ["validate", "create", "destroy", "list", "status"]
+
+    # Attribute commands of flow rule
+    ATTR_CMDS = ["group", "priority", "ingress", "egress", "transfer",
+                 "pattern"]
+
+    # Host, port and api version in URL are complemented by
+    # SppCtlClient class and execute API
+    FLOW_API_VALIDATE = "primary/flow_rules/port_id/{port_id}/validate"
+    FLOW_API_CREATE = "primary/flow_rules/port_id/{port_id}"
+    FLOW_API_DESTROY = "primary/flow_rules/{rule_id}/port_id/{port_id}"
+    FLOW_API_ALL_DESTROY = "primary/flow_rules/port_id/{port_id}"
+
+    # Completion class relevant to the pattern item type
+    PTN_COMPL_CLASSES = {
+        "eth": flow_compl_ptn.ComplEth,
+        "vlan": flow_compl_ptn.ComplVlan,
+    }
+
+    # Completion class relevant to the action type
+    ACT_COMPL_CLASSES = {
+        "jump": flow_compl_act.ComplJump,
+        "of_pop_vlan": flow_compl_act.ComplOfPopVlan,
+        "of_push_vlan": flow_compl_act.ComplOfPushVlan,
+        "of_set_vlan_pcp": flow_compl_act.ComplOfSetVlanPCP,
+        "of_set_vlan_vid": flow_compl_act.ComplOfSetVlanVID,
+        "queue": flow_compl_act.ComplQueue,
+    }
+
+    def __init__(self, spp_ctl_cli):
+        self.spp_ctl_cli = spp_ctl_cli
+
+        # Use as index of tokens list when complementing flow_rule.
+        self._flow_rule_token_index = 0
+
+    def run_flow(self, params):
+        """Run `flow` command."""
+
+        if len(params) == 0:
+            print("Flow sub command is NULL")
+            return None
+
+        elif params[0] not in self.FLOW_CMDS:
+            print("Invalid flow sub command: '{0}'".format(params[0]))
+            return None
+
+        if params[0] == "validate":
+            self._run_flow_validate(params[1:])
+
+        elif params[0] == "create":
+            self._run_flow_create(params[1:])
+
+        elif params[0] == "destroy":
+            self._run_flow_destroy(params[1:])
+
+        elif params[0] == "list":
+            self._run_flow_list(params[1:])
+
+        elif params[0] == "status":
+            self._run_flow_status(params[1:])
+
+    def complete_flow(self, tokens):
+        """Completion for flow commands."""
+        candidates = []
+
+        if len(tokens) == 2:
+            for kw in self.FLOW_CMDS:
+                if kw.startswith(tokens[1]):
+                    candidates.append(kw)
+
+        else:
+            if tokens[1] == "validate":
+                candidates = self._compl_flow_rule(tokens[2:])
+
+            elif tokens[1] == "create":
+                candidates = self._compl_flow_rule(tokens[2:])
+
+            elif tokens[1] == "destroy":
+                candidates = self._compl_flow_destroy(tokens[2:])
+
+            elif tokens[1] == "list":
+                candidates = self._compl_flow_list(tokens[2:])
+
+            elif tokens[1] == "status":
+                candidates = self._compl_flow_status(tokens[2:])
+
+        return candidates
+
+    def _compl_flow_rule(self, tokens):
+        """Completion for validate and create commands."""
+        candidates = []
+
+        self._flow_rule_token_index = 0
+
+        if len(tokens) == 1:
+            candidates = self._create_candidacy_phy_ports(tokens[0])
+            return candidates
+
+        # Next index of RES_UID
+        self._flow_rule_token_index += 1
+
+        # Completion of attribute part in flow rule
+        candidates = self._compl_flow_rule_attribute(tokens)
+        if self._flow_rule_token_index == len(tokens):
+            return candidates
+
+        # Completion of pattern part in flow rule
+        candidates = self._compl_flow_rule_pattern(tokens)
+        if self._flow_rule_token_index == len(tokens):
+            return candidates
+
+        # Completion of action part in flow rule
+        candidates = self._compl_flow_rule_action(tokens)
+        if self._flow_rule_token_index == len(tokens):
+            return candidates
+
+        return candidates
+
+    def _compl_flow_rule_attribute(self, tokens):
+        """Completion for flow rule in attribute."""
+        candidates = []
+
+        while self._flow_rule_token_index < len(tokens):
+            # If "group" is specified, "GROUP_ID" is a candidate
+            if tokens[self._flow_rule_token_index - 1] == "group":
+                candidates = ["GROUP_ID"]
+
+            # If "priority" is specified, "LEVEL" is a candidate
+            elif tokens[self._flow_rule_token_index - 1] == "priority":
+                candidates = ["LEVEL"]
+
+            # If "pattern" is specified, exit attribute completion
+            elif tokens[self._flow_rule_token_index - 1] == "pattern":
+                candidates = []
+                break
+
+            else:
+                candidates = self.ATTR_CMDS
+
+            self._flow_rule_token_index += 1
+
+        return candidates
+
+    def _compl_flow_rule_pattern(self, tokens):
+        """Completion for flow rule in pattern."""
+        candidates = []
+
+        while self._flow_rule_token_index < len(tokens):
+            token = tokens[self._flow_rule_token_index - 1]
+
+            if token in self.PTN_COMPL_CLASSES.keys():
+                try:
+                    item_cls = self.PTN_COMPL_CLASSES[token]
+                    item_instance = item_cls()
+
+                    candidates, index = item_instance.compl_item(
+                        tokens, self._flow_rule_token_index)
+
+                    self._flow_rule_token_index = index
+                    if self._flow_rule_token_index == len(tokens):
+                        break
+
+                    if (tokens[self._flow_rule_token_index - 1] == "/"):
+                        # Type candidate and end token
+                        pattern_list = list(self.PTN_COMPL_CLASSES.keys())
+                        candidates = copy.deepcopy(pattern_list)
+                        candidates.append("end")
+
+                except Exception as _:
+                    candidates = []
+
+            elif (token == "end"):
+                candidates = []
+                break
+
+            else:
+                # Type candidate and end token
+                pattern_list = list(self.PTN_COMPL_CLASSES.keys())
+                candidates = copy.deepcopy(pattern_list)
+                candidates.append("end")
+
+            self._flow_rule_token_index += 1
+
+        return candidates
+
+    def _compl_flow_rule_action(self, tokens):
+        """Completion for flow rule in action."""
+        candidates = []
+
+        if (tokens[self._flow_rule_token_index - 1] == "end"):
+            candidates = ["actions"]
+            self._flow_rule_token_index += 1
+
+        while self._flow_rule_token_index < len(tokens):
+            token = tokens[self._flow_rule_token_index - 1]
+
+            if token in self.ACT_COMPL_CLASSES.keys():
+                try:
+                    action_cls = self.ACT_COMPL_CLASSES[token]
+                    action_instance = action_cls()
+
+                    candidates, index = action_instance.compl_action(
+                        tokens, self._flow_rule_token_index)
+
+                    self._flow_rule_token_index = index
+                    if self._flow_rule_token_index == len(tokens):
+                        break
+
+                    if (tokens[self._flow_rule_token_index - 1] == "/"):
+                        # Type candidate and end token
+                        action_list = list(self.ACT_COMPL_CLASSES.keys())
+                        candidates = copy.deepcopy(action_list)
+                        candidates.append("end")
+
+                except Exception as _:
+                    candidates = []
+
+            elif (tokens[self._flow_rule_token_index - 1] == "end"):
+                candidates = []
+                break
+
+            else:
+                # Type candidate and end token
+                action_list = list(self.ACT_COMPL_CLASSES.keys())
+                candidates = copy.deepcopy(action_list)
+                candidates.append("end")
+
+            self._flow_rule_token_index += 1
+
+        return candidates
+
+    def _compl_flow_destroy(self, tokens):
+        """Completion for destroy command."""
+        candidates = []
+
+        if len(tokens) == 1:
+            candidates = self._create_candidacy_phy_ports(tokens[0])
+
+        elif len(tokens) == 2:
+            candidates.append("ALL")
+
+            rule_ids = self._get_rule_ids(tokens[0])
+            if rule_ids is not None:
+                candidates.extend(rule_ids)
+
+        return candidates
+
+    def _compl_flow_list(self, tokens):
+        """Completion for list command."""
+        candidates = []
+
+        if len(tokens) == 1:
+            candidates = self._create_candidacy_phy_ports(tokens[0])
+
+        return candidates
+
+    def _compl_flow_status(self, tokens):
+        """Completion for status command."""
+        candidates = []
+
+        if len(tokens) == 1:
+            candidates = self._create_candidacy_phy_ports(tokens[0])
+
+        elif len(tokens) == 2:
+            rule_ids = self._get_rule_ids(tokens[0])
+
+            if rule_ids is not None:
+                candidates = rule_ids
+
+        return candidates
+
+    def _run_flow_validate(self, params):
+        """Run `validate` command."""
+        if len(params) == 0:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        if len(params) < 2:
+            print("Flow command invalid argument")
+            return None
+
+        body = self._parse_flow_rule(params[1:])
+        if body is None:
+            print("Flow command invalid argument")
+            return None
+
+        url = self.FLOW_API_VALIDATE.format(port_id=port_id)
+
+        response = self.spp_ctl_cli.post(url, body)
+        if response is None or response.status_code != 200:
+            print("Error: API execution failed for flow validate")
+            return None
+
+        try:
+            res_body = response.json()
+        except Exception as _:
+            print("Error: API response is not json")
+            return None
+
+        message = res_body.get("message")
+        if message is not None:
+            print(message)
+        else:
+            print("Error: result message is None")
+
+    def _run_flow_create(self, params):
+        """Run `create` command."""
+        if len(params) == 0:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        if len(params) < 2:
+            print("Flow command invalid argument")
+            return None
+
+        body = self._parse_flow_rule(params[1:])
+        if body is None:
+            print("Flow command invalid argument")
+            return None
+
+        url = self.FLOW_API_CREATE.format(port_id=port_id)
+
+        response = self.spp_ctl_cli.post(url, body)
+        if response is None or response.status_code != 200:
+            print("Error: API execution failed for flow create")
+            return None
+
+        try:
+            res_body = response.json()
+        except Exception as _:
+            print("Error: API response is not json")
+            return None
+
+        message = res_body.get("message")
+        if message is not None:
+            print(message)
+        else:
+            print("Error: result message is None")
+
+    def _run_flow_destroy(self, params):
+        """Run `destroy` command."""
+        if len(params) == 0:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        if len(params) < 2:
+            print("RULE_ID is NULL")
+            return None
+
+        all_destroy = False
+
+        if params[1] == "ALL":
+            all_destroy = True
+        else:
+            try:
+                rule_id = int(params[1])
+            except Exception as _:
+                print("RULE_ID is invalid")
+                return None
+
+        if all_destroy:
+            url = self.FLOW_API_ALL_DESTROY.format(port_id=port_id)
+        else:
+            url = self.FLOW_API_DESTROY.format(
+                port_id=port_id, rule_id=rule_id)
+
+        response = self.spp_ctl_cli.delete(url)
+        if response is None or response.status_code != 200:
+            print("Error: API execution failed for flow destroy")
+            return None
+
+        try:
+            res_body = response.json()
+        except Exception as _:
+            print("Error: API response is not json")
+            return None
+
+        message = res_body.get("message")
+        if message is not None:
+            print(message)
+        else:
+            print("Error: result message is None")
+
+    def _run_flow_list(self, params):
+        """Run `list` command."""
+        if len(params) != 1:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        status = self._get_pri_status()
+        if (status is None) or ("phy_ports" not in status):
+            print("Failed to get primary status")
+            return None
+
+        target_flow_list = None
+        for phy_port in status.get("phy_ports"):
+            if phy_port.get("id") == port_id:
+                target_flow_list = phy_port.get("flow")
+                break
+
+        if target_flow_list is None:
+            print("'{0}' is invalid".format(params[0]))
+            return None
+
+        self._print_flow_list(target_flow_list)
+
+    def _run_flow_status(self, params):
+        """Run `status` command."""
+        if len(params) == 0:
+            print("RES_UID is NULL")
+            return None
+
+        port_id = self._parse_phy_res_uid_to_port_id(params[0])
+        if port_id is None:
+            print("RES_UID is invalid")
+            return None
+
+        if len(params) < 2:
+            print("RULE_ID is NULL")
+            return None
+
+        try:
+            rule_id = int(params[1])
+        except Exception as _:
+            print("RULE_ID is invalid")
+            return None
+
+        status = self._get_pri_status()
+        if (status is None) or ("phy_ports" not in status):
+            print("Failed to get primary status")
+            return None
+
+        target_flow_list = None
+        for phy_port in status.get("phy_ports"):
+            if phy_port.get("id") == port_id:
+                target_flow_list = phy_port.get("flow")
+                break
+
+        if target_flow_list is None:
+            print("'{0}' is invalid".format(params[0]))
+            return None
+
+        target_flow = None
+        for flow in target_flow_list:
+            if flow.get("rule_id") == rule_id:
+                target_flow = flow
+
+        if target_flow is None:
+            print("RULE_ID:{0} does not exist.".format(params[1]))
+            return None
+
+        self._print_flow_status(target_flow)
+
+    def _get_pri_status(self):
+        """Get primary status."""
+        try:
+            res = self.spp_ctl_cli.get('primary/status')
+            if res is None or res.status_code != 200:
+                print("Error: receive error response from primary")
+                return None
+        except Exception as _:
+            print("Error: there is an error sending the HTTP request")
+            return None
+
+        try:
+            body = res.json()
+        except Exception as _:
+            print("Error: response body of primary status is not json.")
+            return None
+
+        return body
+
+    def _get_candidacy_phy_ports(self):
+        """Get physical_ports candidates
+
+        Return port_id list, for example: ["0", "1"]
+        If physical_ports candidates cannot be returned, return None.
+        """
+        pri_status = self._get_pri_status()
+        if pri_status is None:
+            return None
+
+        if "phy_ports" not in pri_status:
+            return None
+
+        candidates = []
+
+        for port in pri_status["phy_ports"]:
+            if "id" not in port:
+                continue
+
+            candidates.append(str(port["id"]))
+
+        return candidates
+
+    def _get_rule_ids(self, res_uid):
+        """Get rule_ids condidates
+
+        Return rule_ids list, for example: ["0", "1"]
+        If rule_ids candidates cannot be returned, return None.
+        """
+        if not res_uid.startswith("phy:"):
+            return None
+
+        try:
+            port_id = int(re.sub(r"\D", "", res_uid))
+        except Exception as _:
+            return None
+
+        pri_status = self._get_pri_status()
+        if pri_status is None:
+            return None
+
+        if "phy_ports" not in pri_status:
+            return None
+
+        flow_list = None
+        for port in pri_status["phy_ports"]:
+            if port["id"] == port_id:
+                flow_list = port["flow"]
+                break
+
+        if flow_list is None:
+            return None
+
+        candidates = []
+        for flow in flow_list:
+            candidates.append(str(flow["rule_id"]))
+
+        return candidates
+
+    def _create_candidacy_phy_ports(self, text):
+        """Create physical_ports candidate list
+
+        Return phy_ports candidate list, for example: ["phy:0", "phy:1"].
+        If phy_ports candidates cannot be returned, return None.
+        """
+        port_ids = self._get_candidacy_phy_ports()
+        if port_ids is None:
+            return None
+        return ["phy:" + p for p in port_ids]
+
+    def _parse_phy_res_uid_to_port_id(self, res_uid):
+        """Extract port_id from phy RES_UID string
+
+        For example, extract 0 from `phy:0` and return.
+        If it is not RES_UID or not phy, return None.
+        """
+        if type(res_uid) is not str:
+            return None
+
+        tokens = res_uid.split(":")
+
+        if (len(tokens) != 2) or (tokens[0] != "phy"):
+            return None
+
+        try:
+            port_id = int(tokens[1])
+        except Exception as _:
+            return None
+
+        return port_id
+
+    def _parse_flow_rule(self, params):
+        """Parse the flow rule and convert to dict type."""
+        index = 0
+        max_index = len(params) - 1
+        flow_rule = {"rule": {}}
+
+        index = self._parse_flow_rule_attribute(params, index,
+                                                flow_rule["rule"])
+        if index is None or index >= max_index:
+            return None
+
+        index = self._parse_flow_rule_patterns(params, index,
+                                               flow_rule["rule"])
+        if index is None or index >= max_index:
+            return None
+
+        index = self._parse_flow_rule_actions(params, index,
+                                              flow_rule["rule"])
+        if index is None:
+            return None
+
+        return flow_rule
+
+    def _parse_flow_rule_attribute(self, params, index, flow_rule):
+        """Parse attribute of flow rule and convert to dict type.
+
+        Returns the index where "pattern" is specified
+        and the attribute part is completed.
+        It is abnormal that index exceeds "len(params)"
+        because "pattern" is not specified.
+        """
+        try:
+            while index < len(params):
+                if params[index] == "group":
+                    index += 1
+                    flow_rule["group"] = int(params[index])
+
+                elif params[index] == "priority":
+                    index += 1
+                    flow_rule["priority"] = int(params[index])
+
+                elif params[index] == "ingress" or params[index] == "egress":
+                    flow_rule["direction"] = params[index]
+
+                elif params[index] == "transfer":
+                    flow_rule["transfer"] = True
+
+                elif params[index] == "pattern":
+                    break
+
+                index += 1
+
+        except Exception as _:
+            return None
+
+        return index
+
+    def _parse_flow_rule_patterns(self, params, index, flow_rule):
+        """Parse patterns of flow rule and convert to dict type.
+
+        Returns the index where "end" is specified
+        and the pattern part is completed.
+        It is abnormal that index exceeds "len(params)"
+        because "end" is not specified.
+        """
+        sentensce = ""
+        flow_rule["pattern"] = []
+
+        if params[index] != "pattern":
+            return None
+
+        index += 1
+
+        while index < len(params):
+            if params[index] == "/":
+                flow_rule["pattern"].append(sentensce.rstrip())
+                sentensce = ""
+
+            elif params[index] == "end":
+                index += 1
+                break
+
+            else:
+                sentensce += params[index] + " "
+
+            index += 1
+
+        return index
+
+    def _parse_flow_rule_actions(self, params, index, flow_rule):
+        """Parse actions of flow rule and convert to dict type.
+
+        Returns the index where "end" is specified
+        and the action part is completed.
+        It is abnormal that index exceeds "len(params)"
+        because "end" is not specified.
+        """
+        sentensce = ""
+        flow_rule["actions"] = []
+
+        if params[index] != "actions":
+            return None
+
+        index += 1
+
+        while index < len(params):
+            if params[index] == "/":
+                flow_rule["actions"].append(sentensce.rstrip())
+                sentensce = ""
+
+            elif params[index] == "end":
+                index += 1
+                break
+
+            else:
+                sentensce += params[index] + " "
+
+            index += 1
+
+        return index
+
+    def _print_flow_list(self, flow_list):
+        """Print flow information as a list.
+
+        Print example.
+        -----
+        spp > pri; flow list phy:0
+        ID      Group   Prio    Attr    Rule
+        0       1       0       -e-     ETH => OF_PUSH_VLAN OF_SET_VLAN_VID
+        1       1       0       i--     ETH VLAN => QUEUE OF_POP_VLAN
+        2       0       0       i--     ETH => JUMP
+        -----
+        """
+        print("ID      Group   Prio    Attr    Rule")
+
+        for flow in flow_list:
+            print_data = {}
+
+            try:
+                print_data["id"] = str(flow.get("rule_id")).ljust(7)
+
+                attr = flow.get("attr")
+                if attr is None:
+                    continue
+
+                print_data["group"] = str(attr.get("group")).ljust(7)
+                print_data["prio"] = str(attr.get("priority")).ljust(7)
+
+                ingress = "i" if attr.get("ingress") == 1 else "-"
+                egress = "e" if attr.get("egress") == 1 else "-"
+                transfer = "t" if attr.get("transfer") == 1 else "-"
+                print_data["attr"] = "{0}{1}{2}".format(
+                    ingress, egress, transfer).ljust(7)
+
+                patterns = flow.get("patterns")
+                if patterns is None:
+                    continue
+
+                print_data["rule"] = ""
+                for ptn in patterns:
+                    print_data["rule"] += "{0} ".format(
+                        ptn.get("type").upper())
+                print_data["rule"] += "=> "
+
+                actions = flow.get("actions")
+                if actions is None:
+                    continue
+
+                for act in actions:
+                    print_data["rule"] += "{0} ".format(
+                        act.get("type").upper())
+
+                print("{id} {group} {prio} {attr} {rule}".format(**print_data))
+
+            except Exception as _:
+                continue
+
+    def _print_flow_status(self, flow):
+        """Print details of flow information."""
+        # Attribute print
+        self._print_flow_status_attribute(flow.get("attr"))
+
+        # Patterns print
+        self._print_flow_status_patterns(flow.get("patterns"))
+
+        # Actions print
+        self._print_flow_status_actions(flow.get("actions"))
+
+    def _print_flow_status_attribute(self, attr):
+        """Print attribute in the details of flow information.
+
+        Print example.
+        -----
+        Attribute:
+          Group   Priority Ingress Egress Transfer
+          1       0        true    false  false
+        -----
+        """
+        print_data = {}
+
+        print_data["group"] = str(attr.get("group")).ljust(7)
+        print_data["priority"] = str(attr.get("priority")).ljust(8)
+        print_data["ingress"] = ("true" if attr.get("ingress") == 1
+                                 else "false").ljust(7)
+        print_data["egress"] = ("true" if attr.get("egress") == 1
+                                else "false").ljust(6)
+        print_data["transfer"] = ("true" if attr.get("transfer") == 1
+                                  else "false").ljust(8)
+
+        print("Attribute:")
+        print("  Group   Priority Ingress Egress Transfer")
+        print("  {group} {priority} {ingress} {egress} {transfer}".format(
+            **print_data))
+
+    def _print_flow_status_patterns(self, patterns):
+        """Print patterns in the details of flow information.
+
+        Print example.
+        -----
+        Patterns:
+          - eth:
+            - spec:
+              - dst: 10:22:33:44:55:66
+              - src: 00:00:00:00:00:00
+              - type: 0xffff
+            - last:
+            - mask:
+              - dst: ff:ff:ff:ff:ff:ff
+              - src: 00:00:00:00:00:00
+              - type: 0xffff
+          - vlan:
+            - spec:
+              - tci: 0x0064
+              - inner_type: 0x0000
+            - last:
+            - mask:
+              - tci: 0xffff
+              - inner_type: 0x0000
+        -----
+        """
+        item_type_indent = 2
+        item_fields_indent = 4
+
+        try:
+            print("Patterns:")
+
+            for item in patterns:
+                # Type print
+                self._print_key_value(item.get("type"), None, item_type_indent)
+
+                # Spec print
+                self._print_key_value(
+                    "spec", None, item_fields_indent)
+                spec = item.get("spec")
+                if spec is not None:
+                    self._print_item_fields(spec)
+
+                # Last print
+                self._print_key_value(
+                    "last", None, item_fields_indent)
+                last = item.get("last")
+                if last is not None:
+                    self._print_item_fields(last)
+
+                # Mask print
+                self._print_key_value(
+                    "mask", None, item_fields_indent)
+                mask = item.get("mask")
+                if mask is not None:
+                    self._print_item_fields(mask)
+
+        except Exception as _:
+            print("Error: `patterns` structure of json received "
+                  "from spp-ctl is invalid")
+            return
+
+    def _print_flow_status_actions(self, actions):
+        """Print actions in the details of flow information.
+
+        Print example.
+        -----
+        Actions:
+          - queue:
+            - index: 0
+          - of_pop_vlan:
+        -----
+        """
+        act_type_indent = 2
+        act_fields_indent = 4
+
+        try:
+            print("Actions:")
+
+            for act in actions:
+                # Type print
+                self._print_key_value(act.get("type"), None, act_type_indent)
+
+                # Conf print
+                conf = act.get("conf")
+                if conf is not None:
+                    self._print_action_conf(conf)
+
+        except Exception as _:
+            print("Error: `actions` structure of json received "
+                  "from spp-ctl is invalid")
+            return
+
+    def _print_item_fields(self, fields_dic):
+        """Print each field (spec, last or mask) of flow item."""
+        item_elements_indent = 6
+
+        for key, value in fields_dic.items():
+            self._print_key_value(key, value, item_elements_indent)
+
+    def _print_action_conf(self, conf):
+        """Print conf of flow action."""
+        act_elements_indent = 4
+
+        for key, value in conf.items():
+            self._print_key_value(key, value, act_elements_indent)
+
+    def _print_key_value(self, key, value=None, indent_level=0):
+        """Print key / value combination.
+
+        Print in the format "- key: value".
+        If value is None, print in "- key:" format
+        Insert space for indent.
+        """
+        if value is not None:
+            print_str = "- {key}: {value}".format(
+                key=str(key), value=str(value))
+        else:
+            print_str = "- {key}:".format(key=str(key))
+
+        print((" " * indent_level) + print_str)
diff --git a/src/cli/commands/pri_flow_compl_action.py b/src/cli/commands/pri_flow_compl_action.py
new file mode 100644
index 0000000..748029d
--- /dev/null
+++ b/src/cli/commands/pri_flow_compl_action.py
@@ -0,0 +1,115 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+import copy
+
+
+class BaseComplAction(object):
+    """Base class for completion of actions list.
+
+    Each action inherits from this class.
+    Provides base functionality and command completion.
+    """
+
+    def __init__(self, data=None):
+        if data is not None:
+            self.data = data
+
+    def compl_action(self, tokens, index):
+        """Completion for actions list.
+
+        Complement using the information defined in the inherited class.
+        """
+        candidates = []
+
+        while index < len(tokens):
+            if tokens[index - 1] == "/":
+                # Completion processing end when "/" is specified
+                candidates = []
+                break
+
+            elif tokens[index - 1] in self.DATA_FIELDS:
+                tmp_token = self.DATA_FIELDS_VALUES.get(tokens[index - 1])
+                if tmp_token is not None:
+                    candidates = [tmp_token]
+                else:
+                    candidates = []
+
+            else:
+                # Data fields candidate and end token
+                candidates = copy.deepcopy(self.DATA_FIELDS)
+                candidates.append("/")
+
+            index += 1
+
+        return (candidates, index)
+
+
+class ComplJump(BaseComplAction):
+    """Complete action `jump`."""
+
+    # Jump data fields
+    DATA_FIELDS = ["group"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "group": "UNSIGNED_INT"
+    }
+
+
+class ComplOfPopVlan(BaseComplAction):
+    """Complete action `of_pop_vlan`."""
+
+    # of_pop_vlan data fields
+    DATA_FIELDS = []
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+    }
+
+
+class ComplOfPushVlan(BaseComplAction):
+    """Complete action `of_push_vlan`."""
+
+    # of_push_vlan data fields
+    DATA_FIELDS = ["ethertype"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "ethertype": "UNSIGNED_INT"
+    }
+
+
+class ComplOfSetVlanPCP(BaseComplAction):
+    """Complete action `of_set_vlan_pcp`."""
+
+    # of_set_vlan_pcp data fields
+    DATA_FIELDS = ["vlan_pcp"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "vlan_pcp": "UNSIGNED_INT"
+    }
+
+
+class ComplOfSetVlanVID(BaseComplAction):
+    """Complete action `of_set_vlan_vid`."""
+
+    # of_set_vlan_vid data fields
+    DATA_FIELDS = ["vlan_vid"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "vlan_vid": "UNSIGNED_INT"
+    }
+
+
+class ComplQueue(BaseComplAction):
+    """Complete action `queue`."""
+
+    # Queue data fields
+    DATA_FIELDS = ["index"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "index": "UNSIGNED_INT"
+    }
diff --git a/src/cli/commands/pri_flow_compl_pattern.py b/src/cli/commands/pri_flow_compl_pattern.py
new file mode 100644
index 0000000..8f84a0c
--- /dev/null
+++ b/src/cli/commands/pri_flow_compl_pattern.py
@@ -0,0 +1,84 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+import copy
+
+
+class BaseComplPatternItem(object):
+    """Base class for completion of pattern items.
+
+    Each pattern item inherits from this class.
+    Provides base functionality and command completion.
+    """
+
+    # Token after DATA_FIELDS
+    MATCHING_PATTERN = ["is", "spec", "last", "mask", "prefix"]
+
+    def __init__(self, data=None):
+        if data is not None:
+            self.data = data
+
+    def compl_item(self, tokens, index):
+        """Completion for pattern item commands.
+
+        Complement using the information defined in the inherited class.
+        """
+        candidates = []
+
+        while index < len(tokens):
+            if tokens[index - 1] == "/":
+                # Completion processing end when "/" is specified
+                candidates = []
+                break
+
+            elif tokens[index - 1] in self.DATA_FIELDS:
+                candidates = self.MATCHING_PATTERN
+
+            elif tokens[index - 1] in self.MATCHING_PATTERN:
+                if tokens[index - 1] == "prefix":
+                    candidates = ["Prefix"]
+
+                else:
+                    tmp_token = self.DATA_FIELDS_VALUES.get(tokens[index - 2])
+                    if tmp_token is not None:
+                        candidates = [tmp_token]
+                    else:
+                        candidates = []
+
+            else:
+                # Data fields candidate and end token
+                candidates = copy.deepcopy(self.DATA_FIELDS)
+                candidates.append("/")
+
+            index += 1
+
+        return (candidates, index)
+
+
+class ComplEth(BaseComplPatternItem):
+    """Complete pattern item `eth`."""
+
+    # Eth data fields
+    DATA_FIELDS = ["dst", "src", "type"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "dst": "MAC_ADDRESS",
+        "src": "MAC_ADDRESS",
+        "type": "UNSIGNED_INT"
+    }
+
+
+class ComplVlan(BaseComplPatternItem):
+    """Complete pattern item `vlan`."""
+
+    # Vlan data fields
+    DATA_FIELDS = ["tci", "pcp", "dei", "vid", "inner_type"]
+
+    # DATA_FIELDS value candidates
+    DATA_FIELDS_VALUES = {
+        "tci": "UNSIGNED_INT",
+        "pcp": "UNSIGNED_INT",
+        "dei": "UNSIGNED_INT",
+        "vid": "UNSIGNED_INT",
+        "inner_type": "UNSIGNED_INT"
+    }
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 13/17] cli: add support of rte_flow in vf
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (29 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 12/17] cli: add support of hardware offload x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 14/17] cli: add support of rte_flow in mirror x-fn-spp-ml
                   ` (3 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch implements support of multi-queue in vf command.
  - vf; status
  - vf; port
  - vf; classifier_table

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/cli/commands/vf.py | 460 +++++++++++++++++++++++++++--------------
 1 file changed, 301 insertions(+), 159 deletions(-)

diff --git a/src/cli/commands/vf.py b/src/cli/commands/vf.py
index 5b361d1..c05412c 100644
--- a/src/cli/commands/vf.py
+++ b/src/cli/commands/vf.py
@@ -206,8 +206,8 @@ class SppVf(object):
 
         To update status of the instance of SppVf, get the status from
         spp-ctl. This method returns the result as a dict. For considering
-        behaviour of spp_vf, it is enough to return worker's name and core
-        IDs as the status, but might need to be update for future updates.
+        behaviour of spp_vf, it is enough to return worker's name, core IDs and
+        ports as the status, but might need to be update for future updates.
 
         # return worker's name and used core IDs, and all of core IDs.
         {
@@ -216,12 +216,13 @@ class SppVf(object):
             {'name': 'mg1', 'core_id': 6},
             ...
           ],
-          'core_ids': [5, 6, 7, ...]
+          'core_ids': [5, 6, 7, ...],
+          'ports': ['phy:0', 'phy:1', ...]
         }
 
         """
 
-        status = {'workers': [], 'core_ids': []}
+        status = {'workers': [], 'core_ids': [], 'ports': []}
         res = self.spp_ctl_cli.get('vfs/%d' % self.sec_id)
         if res is not None:
             if res.status_code == 200:
@@ -236,6 +237,9 @@ class SppVf(object):
                                             'core_id': wk['core']})
                             status['core_ids'].append(wk['core'])
 
+                if 'ports' in json_obj:
+                    status['ports'] = json_obj.get('ports')
+
         return status
 
     def _run_status(self):
@@ -290,82 +294,136 @@ class SppVf(object):
                     print('Error: unknown response.')
 
     def _run_port(self, params):
-        req_params = None
-        if len(params) == 4:
-            if params[0] == 'add':
-                action = 'attach'
-            elif params[0] == 'del':
-                action = 'detach'
-            else:
-                print('Error: Invalid action.')
-                return None
-
-            req_params = {'action': action, 'port': params[1],
-                          'dir': params[2],
-                          'vlan': {'operation': 'none',
-                                   'id': 'none',
-                                   'pcp': 'none'}}
-
-        elif len(params) == 5:  # delete vlan with 'port add' command
-            # TODO(yasufum) Syntax for deleting vlan should be modified
-            #               because deleting with 'port add' is terrible!
-            action = 'attach'
-            req_params = {'action': action, 'port': params[1],
-                          'dir': params[2],
-                          'vlan': {'operation': 'del',
-                                   'id': 'none',
-                                   'pcp': 'none'}}
-
-        elif len(params) == 7:
-            action = 'attach'
-            if params[4] == 'add_vlantag':
-                op = 'add'
-            elif params[4] == 'del_vlantag':
-                op = 'del'
-            req_params = {'action': action, 'port': params[1],
-                          'dir': params[2],
-                          'vlan': {'operation': op, 'id': int(params[5]),
-                                   'pcp': int(params[6])}}
-        else:
-            print('Error: Invalid syntax.')
-
-        if req_params is not None:
-            res = self.spp_ctl_cli.put('vfs/%d/components/%s/ports'
-                                       % (self.sec_id, params[3]), req_params)
-            if res is not None:
-                error_codes = self.spp_ctl_cli.rest_common_error_codes
-                if res.status_code == 204:
-                    print("Succeeded to %s port" % params[0])
-                elif res.status_code in error_codes:
-                    pass
+        params_index = 0
+        req_params = {}
+        vlan_params = {}
+        name = None
+        flg_mq = False
+
+        while params_index < len(params):
+            if params_index == 0:
+                if params[params_index] == 'add':
+                    req_params["action"] = 'attach'
+                elif params[params_index] == 'del':
+                    req_params["action"] = 'detach'
                 else:
-                    print('Error: unknown response.')
+                    print('Error: Invalid action.')
+                    return None
+
+            elif params_index == 1:
+                req_params["port"] = params[params_index]
+
+                # Check Multi queue
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["port"] += "nq" + params[params_index]
+                        flg_mq = True
+                else:
+                    print("Error: Not enough parameters.")
+                    return None
+
+            elif ((params_index == 2 and flg_mq is False) or
+                    (params_index == 4 and flg_mq is True)):
+                req_params["dir"] = params[params_index]
+
+            elif ((params_index == 3 and flg_mq is False) or
+                    (params_index == 5 and flg_mq is True)):
+                name = params[params_index]
+
+            elif ((params_index == 4 and flg_mq is False) or
+                    (params_index == 6 and flg_mq is True)):
+                if params[params_index] == "add_vlantag":
+                    vlan_params["operation"] = "add"
+                elif params[params_index] == "del_vlantag":
+                    vlan_params["operation"] = "del"
+                else:
+                    print('Error: vlantag is Only add_vlantag or del_vlantag.')
+                    return None
+
+            elif ((params_index == 5 and flg_mq is False) or
+                    (params_index == 7 and flg_mq is True)):
+                try:
+                    vlan_params["id"] = int(params[params_index])
+                except Exception as _:
+                    print('Error: vid is not a number.')
+                    return None
+
+            elif ((params_index == 6 and flg_mq is False) or
+                    (params_index == 8 and flg_mq is True)):
+                try:
+                    vlan_params["pcp"] = int(params[params_index])
+                except Exception as _:
+                    print('Error: pcp is not a number.')
+                    return None
+
+            params_index += 1
+
+        req_params["vlan"] = vlan_params
+        res = self.spp_ctl_cli.put('vfs/%d/components/%s/ports'
+                                   % (self.sec_id, name), req_params)
+        if res is not None:
+            error_codes = self.spp_ctl_cli.rest_common_error_codes
+            if res.status_code == 204:
+                print("Succeeded to %s port" % params[0])
+            elif res.status_code in error_codes:
+                pass
+            else:
+                print('Error: unknown response.')
 
     def _run_cls_table(self, params):
-        req_params = None
-        if len(params) == 4:
-            req_params = {'action': params[0], 'type': params[1],
-                          'mac_address': params[2], 'port': params[3]}
-
-        elif len(params) == 5:
-            req_params = {'action': params[0], 'type': params[1],
-                          'vlan': params[2], 'mac_address': params[3],
-                          'port': params[4]}
-        else:
-            print('Error: Invalid syntax.')
+        params_index = 0
+        req_params = {}
+        flg_vlan = False
 
-        if req_params is not None:
-            req = 'vfs/%d/classifier_table' % self.sec_id
-            res = self.spp_ctl_cli.put(req, req_params)
+        # The list elements are:
+        #       action, type, vlan, mac,  port
+        values = [None, None, None, None, None]
+        values_index = 0
 
-            if res is not None:
-                error_codes = self.spp_ctl_cli.rest_common_error_codes
-                if res.status_code == 204:
-                    print("Succeeded to %s" % params[0])
-                elif res.status_code in error_codes:
-                    pass
-                else:
-                    print('Error: unknown response.')
+        while params_index < len(params):
+
+            if params_index == 0:
+                req_params["action"] = params[params_index]
+
+            elif params_index == 1:
+                req_params["type"] = params[params_index]
+                if (req_params["type"] != "vlan" and
+                        req_params["type"] != "mac"):
+                    print("Error: Type is only vlan or mac")
+                    return None
+
+            elif params_index == 2 and req_params["type"] == "vlan":
+                req_params["vlan"] = params[params_index]
+                flg_vlan = True
+
+            elif ((params_index == 2 and flg_vlan is False) or
+                    (params_index == 3 and flg_vlan is True)):
+                req_params["mac_address"] = params[params_index]
+
+            elif ((params_index == 3 and flg_vlan is False) or
+                    (params_index == 4 and flg_vlan is True)):
+                req_params["port"] = params[params_index]
+
+                # Check Multi queue
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["port"] += "nq" + params[params_index]
+
+            params_index += 1
+
+        req = 'vfs/%d/classifier_table' % self.sec_id
+        res = self.spp_ctl_cli.put(req, req_params)
+
+        if res is not None:
+            error_codes = self.spp_ctl_cli.rest_common_error_codes
+            if res.status_code == 204:
+                print("Succeeded to %s" % params[0])
+            elif res.status_code in error_codes:
+                pass
+            else:
+                print('Error: unknown response.')
 
     def _run_exit(self):
         """Run `exit` command."""
@@ -411,96 +469,180 @@ class SppVf(object):
             return res
 
     def _compl_port(self, sub_tokens):
-        if len(sub_tokens) < 9:
-            subsub_cmds = ['add', 'del']
-            res = []
-            if len(sub_tokens) == 2:
-                for kw in subsub_cmds:
-                    if kw.startswith(sub_tokens[1]):
-                        res.append(kw)
-            elif len(sub_tokens) == 3:
-                if sub_tokens[1] in subsub_cmds:
-                    if 'RES_UID'.startswith(sub_tokens[2]):
-                        res.append('RES_UID')
-            elif len(sub_tokens) == 4:
-                if sub_tokens[1] in subsub_cmds:
-                    for direction in ['rx', 'tx']:
-                        if direction.startswith(sub_tokens[3]):
-                            res.append(direction)
-            elif len(sub_tokens) == 5:
-                if sub_tokens[1] in subsub_cmds:
-                    for kw in self.worker_names:
-                        if kw.startswith(sub_tokens[4]):
-                            res.append(kw)
-            elif len(sub_tokens) == 6:
-                if sub_tokens[1] == 'add':
-                    for kw in ['add_vlantag', 'del_vlantag']:
-                        if kw.startswith(sub_tokens[5]):
-                            res.append(kw)
-            elif len(sub_tokens) == 7:
-                if sub_tokens[1] == 'add' and sub_tokens[5] == 'add_vlantag':
-                    if 'VID'.startswith(sub_tokens[6]):
-                        res.append('VID')
-            elif len(sub_tokens) == 8:
-                if sub_tokens[1] == 'add' and sub_tokens[5] == 'add_vlantag':
-                    if 'PCP'.startswith(sub_tokens[7]):
-                        res.append('PCP')
-            return res
+        res = []
+        index = 0
+
+        # compl_phase "add_del"  : candidate is add or del
+        # compl_phase "res_uid"  : candidate is RES_UID
+        # compl_phase "nq"       : candidate is nq
+        # compl_phase "queue_no" : candidate is queue no
+        # compl_phase "dir"      : candidate is DIR
+        # compl_phase "name"     : candidate is NAME
+        # compl_phase "vlan_tag" : candidate is vlan tag
+        # compl_phase "vid"      : candidate is vid
+        # compl_phase "pcp"      : candidate is pcp
+        # compl_phase None       : candidate is None
+        compl_phase = "add_del"
+        add_or_del = None
+
+        while index < len(sub_tokens):
+            if compl_phase == "nq":
+                queue_no_list = self._get_candidate_phy_queue_no(
+                    sub_tokens[index - 1])
+
+                if queue_no_list is None:
+                    compl_phase = "dir"
+
+            if ((compl_phase == "add_del") and
+                    (sub_tokens[index - 1] == "port")):
+                res = ["add", "del"]
+                compl_phase = "res_uid"
+
+            elif ((compl_phase == "res_uid") and
+                    (sub_tokens[index - 1] == "add" or
+                     sub_tokens[index - 1] == "del")):
+                res = ["RES_UID"]
+                compl_phase = "nq"
+                add_or_del = sub_tokens[index - 1]
+
+            elif compl_phase == "nq":
+                res = ["nq"]
+                compl_phase = "queue_no"
+
+            elif compl_phase == "queue_no":
+                res = queue_no_list
+                compl_phase = "dir"
+
+            elif compl_phase == "dir":
+                res = ["rx", "tx"]
+                compl_phase = "name"
+
+            elif compl_phase == "name":
+                res = ["NAME"]
+                if add_or_del == "add":
+                    compl_phase = "vlan_tag"
+                else:
+                    compl_phase = None
+
+            elif compl_phase == "vlan_tag":
+                res = ["add_vlantag", "del_vlantag"]
+                compl_phase = "vid"
+
+            elif (compl_phase == "vid" and
+                  sub_tokens[index - 1] == "add_vlantag"):
+                res = ["VID"]
+                compl_phase = "pcp"
+
+            elif compl_phase == "pcp":
+                res = ["PCP"]
+                compl_phase = None
+
+            else:
+                res = []
+
+            index += 1
+
+        res = [p for p in res
+               if p.startswith(sub_tokens[len(sub_tokens) - 1])]
+
+        return res
+
+    def _get_candidate_phy_queue_no(self, res_uid):
+        """Get phy queue_no candidate.
+        If res_uid is phy and multi-queue, return queue_no in list type.
+        Otherwise returns None.
+        """
+
+        try:
+            port, _ = res_uid.split(":")
+        except Exception as _:
+            return None
+
+        if port != "phy":
+            return None
+
+        status = self._get_status(self.sec_id)
+
+        queue_no_list = []
+        for port in status["ports"]:
+            if not port.startswith(res_uid):
+                continue
+
+            try:
+                _, queue_no = port.split("nq")
+            except Exception as _:
+                continue
+
+            queue_no_list.append(queue_no.strip(" "))
+
+        if len(queue_no_list) == 0:
+            return None
+
+        return queue_no_list
 
     def _compl_cls_table(self, sub_tokens):
-        if len(sub_tokens) < 7:
-            subsub_cmds = ['add', 'del']
-            res = []
+        res = []
+        index = 0
+
+        # compl_phase "add_del"  : candidate is add or del
+        # compl_phase "vlan_mac" : candidate is vlan or mac
+        # compl_phase "vid"      : candidate is VID
+        # compl_phase "mac_addr" : candidate is MAC_ADDR or default
+        # compl_phase "res_uid"  : candidate is RES_UID
+        # compl_phase "nq"       : candidate is nq
+        # compl_phase "queue_no" : candidate is queue_no
+        # compl_phase None : candidate is None
+        compl_phase = "add_del"
+
+        while index < len(sub_tokens):
+
+            if compl_phase == "vid" and sub_tokens[index - 1] == "mac":
+                compl_phase = "mac_addr"
+
+            if compl_phase == "nq":
+                queue_no_list = self._get_candidate_phy_queue_no(
+                    sub_tokens[index - 1])
+                if queue_no_list is None:
+                    compl_phase = None
+
+            if ((compl_phase == "add_del") and
+                    (sub_tokens[index - 1] == "classifier_table")):
+                res = ["add", "del"]
+                compl_phase = "vlan_mac"
+
+            elif compl_phase == "vlan_mac":
+                res = ["vlan", "mac"]
+                compl_phase = "vid"
+
+            elif compl_phase == "vid" and sub_tokens[index - 1] == "vlan":
+                res = ["VID"]
+                compl_phase = "mac_addr"
+
+            elif compl_phase == "mac_addr":
+                res = ["MAC_ADDR", "default"]
+                compl_phase = "res_uid"
+
+            elif compl_phase == "res_uid":
+                res = ["RES_UID"]
+                compl_phase = "nq"
+
+            elif compl_phase == "nq":
+                res = ["nq"]
+                compl_phase = "queue_no"
+
+            elif compl_phase == "queue_no":
+                res = queue_no_list
+                compl_phase = None
 
-            if len(sub_tokens) == 2:
-                for kw in subsub_cmds:
-                    if kw.startswith(sub_tokens[1]):
-                        res.append(kw)
+            else:
+                res = []
 
-            elif len(sub_tokens) == 3:
-                if sub_tokens[1] in subsub_cmds:
-                    for kw in ['mac', 'vlan']:
-                        if kw.startswith(sub_tokens[2]):
-                            res.append(kw)
+            index += 1
 
-            elif len(sub_tokens) == 4:
-                if sub_tokens[1] == 'add':
-                    if sub_tokens[2] == 'mac':
-                        if 'MAC_ADDR'.startswith(sub_tokens[3]):
-                            res.append('MAC_ADDR')
-                    elif sub_tokens[2] == 'vlan':
-                        if 'VID'.startswith('VID'):
-                            res.append('VID')
-                elif sub_tokens[1] == 'del':
-                    if sub_tokens[2] == 'mac':
-                        if 'MAC_ADDR'.startswith(sub_tokens[3]):
-                            res.append('MAC_ADDR')
-                    if sub_tokens[2] == 'vlan':
-                        if 'VID'.startswith(sub_tokens[3]):
-                                res.append('VID')
+        res = [p for p in res
+               if p.startswith(sub_tokens[len(sub_tokens) - 1])]
 
-            elif len(sub_tokens) == 5:
-                if sub_tokens[1] == 'add':
-                    if sub_tokens[2] == 'mac':
-                        if 'RES_UID'.startswith(sub_tokens[4]):
-                            res.append('RES_UID')
-                    elif sub_tokens[2] == 'vlan':
-                        if 'MAC_ADDR'.startswith(sub_tokens[4]):
-                            res.append('MAC_ADDR')
-                if sub_tokens[1] == 'del':
-                    if sub_tokens[2] == 'mac':
-                        if 'RES_UID'.startswith(sub_tokens[4]):
-                            res.append('RES_UID')
-                    elif sub_tokens[2] == 'vlan':
-                        if 'MAC_ADDR'.startswith(sub_tokens[4]):
-                            res.append('MAC_ADDR')
-
-            elif len(sub_tokens) == 6:
-                if sub_tokens[1] in subsub_cmds and \
-                        sub_tokens[2] == 'vlan':
-                            if 'RES_UID'.startswith(sub_tokens[5]):
-                                res.append('RES_UID')
-            return res
+        return res
 
     @classmethod
     def help(cls):
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 14/17] cli: add support of rte_flow in mirror
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (30 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 13/17] cli: add support of rte_flow in vf x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 15/17] cli: add support of rte_flow in nfv x-fn-spp-ml
                   ` (2 subsequent siblings)
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch implements support of multi-queue in mirror command.
  - mirror; status
  - mirror; port

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/cli/commands/mirror.py | 175 +++++++++++++++++++++++++++++--------
 1 file changed, 137 insertions(+), 38 deletions(-)

diff --git a/src/cli/commands/mirror.py b/src/cli/commands/mirror.py
index d7ea4cd..10c29f9 100644
--- a/src/cli/commands/mirror.py
+++ b/src/cli/commands/mirror.py
@@ -56,7 +56,8 @@ class SppMirror(object):
             self.unused_core_ids = core_ids  # used while completion to exclude
 
             self.workers = mirror_status['workers']
-            self.worker_names = [attr['name'] for attr in mirror_status['workers']]
+            self.worker_names = [attr['name']
+                                 for attr in mirror_status['workers']]
 
         cmd = cmdline.split(' ')[0]
         params = cmdline.split(' ')[1:]
@@ -184,8 +185,9 @@ class SppMirror(object):
 
         To update status of the instance of SppMirror, get the status from
         spp-ctl. This method returns the result as a dict. For considering
-        behaviour of spp_mirror, it is enough to return worker's name and core
-        IDs as the status, but might need to be update for future updates.
+        behaviour of spp_mirror, it is enough to return worker's name, core
+        IDs and ports as the status, but might need to be update for future
+        updates.
 
         # return worker's name and used core IDs, and all of core IDs.
         {
@@ -194,12 +196,13 @@ class SppMirror(object):
             {'name': 'mr2', 'core_id': 6},
             ...
           ],
-          'core_ids': [5, 6, 7, ...]
+          'core_ids': [5, 6, 7, ...],
+          'ports': ['phy:0', 'phy:1', ...]
         }
 
         """
 
-        status = {'workers': [], 'core_ids': []}
+        status = {'workers': [], 'core_ids': [], 'ports': []}
         res = self.spp_ctl_cli.get('mirrors/%d' % self.sec_id)
         if res is not None:
             if res.status_code == 200:
@@ -214,8 +217,45 @@ class SppMirror(object):
                                             'core_id': wk['core']})
                             status['core_ids'].append(wk['core'])
 
+                if 'ports' in json_obj:
+                    status['ports'] = json_obj.get('ports')
+
         return status
 
+    def _get_candidate_phy_queue_no(self, res_uid):
+        """Get phy queue_no candidate.
+
+        If res_uid is phy and multi-queue, return queue_no in list type.
+        Otherwise returns None.
+        """
+
+        try:
+            port, _ = res_uid.split(":")
+        except Exception as _:
+            return None
+
+        if port != "phy":
+            return None
+
+        status = self._get_status(self.sec_id)
+
+        queue_no_list = []
+        for port in status["ports"]:
+            if not port.startswith(res_uid):
+                continue
+
+            try:
+                _, queue_no = port.split("nq")
+            except Exception as _:
+                continue
+
+            queue_no_list.append(queue_no.strip(" "))
+
+        if len(queue_no_list) == 0:
+            return None
+
+        return queue_no_list
+
     def _run_status(self):
         res = self.spp_ctl_cli.get('mirrors/%d' % self.sec_id)
         if res is not None:
@@ -267,20 +307,45 @@ class SppMirror(object):
                     print('Error: unknown response.')
 
     def _run_port(self, params):
-        if len(params) == 4:
-            if params[0] == 'add':
-                action = 'attach'
-            elif params[0] == 'del':
-                action = 'detach'
-            else:
-                print('Error: Invalid action.')
-                return None
+        params_index = 0
+        req_params = {}
+        name = None
+        flg_mq = False
+
+        while params_index < len(params):
+            if params_index == 0:
+                if params[params_index] == 'add':
+                    req_params["action"] = 'attach'
+                elif params[params_index] == 'del':
+                    req_params["action"] = 'detach'
+                else:
+                    print('Error: Invalid action.')
+                    return None
+
+            elif params_index == 1:
+                req_params["port"] = params[params_index]
+
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["port"] += "nq" + params[params_index]
+                        flg_mq = True
+                else:
+                    print("Error: Not enough parameters.")
+                    return None
+
+            elif ((params_index == 2 and flg_mq is False) or
+                    (params_index == 4 and flg_mq is True)):
+                req_params["dir"] = params[params_index]
 
-            req_params = {'action': action, 'port': params[1],
-                          'dir': params[2]}
+            elif ((params_index == 3 and flg_mq is False) or
+                    (params_index == 5 and flg_mq is True)):
+                name = params[params_index]
+
+            params_index += 1
 
         res = self.spp_ctl_cli.put('mirrors/%d/components/%s/ports'
-                                   % (self.sec_id, params[3]), req_params)
+                                   % (self.sec_id, name), req_params)
         if res is not None:
             error_codes = self.spp_ctl_cli.rest_common_error_codes
             if res.status_code == 204:
@@ -334,28 +399,62 @@ class SppMirror(object):
             return res
 
     def _compl_port(self, sub_tokens):
-        if len(sub_tokens) < 9:
-            subsub_cmds = ['add', 'del']
-            res = []
-            if len(sub_tokens) == 2:
-                for kw in subsub_cmds:
-                    if kw.startswith(sub_tokens[1]):
-                        res.append(kw)
-            elif len(sub_tokens) == 3:
-                if sub_tokens[1] in subsub_cmds:
-                    if 'RES_UID'.startswith(sub_tokens[2]):
-                        res.append('RES_UID')
-            elif len(sub_tokens) == 4:
-                if sub_tokens[1] in subsub_cmds:
-                    for direction in ['rx', 'tx']:
-                        if direction.startswith(sub_tokens[3]):
-                            res.append(direction)
-            elif len(sub_tokens) == 5:
-                if sub_tokens[1] in subsub_cmds:
-                    for kw in self.worker_names:
-                        if kw.startswith(sub_tokens[4]):
-                            res.append(kw)
-            return res
+        res = []
+        index = 0
+
+        # compl_phase "add_del" : candidate is add or del
+        # compl_phase "res_uid" : candidate is RES_UID
+        # compl_phase "nq"      : candidate is nq
+        # compl_phase "queue_no": candidate is queue no
+        # compl_phase "dir"     : candidate is DIR
+        # compl_phase "name"    : candidate is NAME
+        # compl_phase None      : candidate is None
+        compl_phase = "add_del"
+
+        while index < len(sub_tokens):
+            if compl_phase == "nq":
+                queue_no_list = self._get_candidate_phy_queue_no(
+                    sub_tokens[index - 1])
+
+                if queue_no_list is None:
+                    compl_phase = "dir"
+
+            if ((compl_phase == "add_del") and
+                    (sub_tokens[index - 1] == "port")):
+                res = ["add", "del"]
+                compl_phase = "res_uid"
+
+            elif ((compl_phase == "res_uid") and
+                    (sub_tokens[index - 1] == "add" or
+                     sub_tokens[index - 1] == "del")):
+                res = ["RES_UID"]
+                compl_phase = "nq"
+
+            elif compl_phase == "nq":
+                res = ["nq"]
+                compl_phase = "queue_no"
+
+            elif compl_phase == "queue_no":
+                res = queue_no_list
+                compl_phase = "dir"
+
+            elif compl_phase == "dir":
+                res = ["rx", "tx"]
+                compl_phase = "name"
+
+            elif compl_phase == "name":
+                res = ["NAME"]
+                compl_phase = None
+
+            else:
+                res = []
+
+            index += 1
+
+        res = [p for p in res
+               if p.startswith(sub_tokens[len(sub_tokens) - 1])]
+
+        return res
 
     @classmethod
     def help(cls):
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 15/17] cli: add support of rte_flow in nfv
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (31 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 14/17] cli: add support of rte_flow in mirror x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 16/17] spp-ctl: add APIs for flow rules x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 17/17] spp_nfv: add support of multi-queue x-fn-spp-ml
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch implements support of multi-queue in nfv command.
  - nfv; status
  - nfv; port

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/cli/commands/nfv.py | 245 +++++++++++++++++++++++++++++-----------
 1 file changed, 179 insertions(+), 66 deletions(-)

diff --git a/src/cli/commands/nfv.py b/src/cli/commands/nfv.py
index 6349823..0feb4ac 100644
--- a/src/cli/commands/nfv.py
+++ b/src/cli/commands/nfv.py
@@ -260,49 +260,123 @@ class SppNfv(object):
     def _compl_patch(self, sub_tokens):
         """Complete `patch` command."""
 
-        # Patch command consists of three tokens max, for instance,
-        # `nfv 1; patch phy:0 ring:1`.
-        if len(sub_tokens) < 4:
-            res = []
-
-            if self.use_cache is False:
-                self.ports, self.patches = self._get_ports_and_patches()
+        res = []
+        candidates = []
+        # index 0 is "port", so from 1
+        index = 1
+
+        # compl_phase "src_res_uid" : candidate is src RES_UID or reset
+        # compl_phase "src_nq"      : candidate is nq
+        # compl_phase "src_q_no"    : candidate is queue no
+        # compl_phase "dst_res_uid" : candidate is dst RES_UID
+        # compl_phase "dst_nq"      : candidate is nq
+        # compl_phase "dst_q_no"    : candidate is queue no
+        # compl_phase None          : candidate is None
+        compl_phase = "src_res_uid"
+
+        if self.use_cache is False:
+            self.ports, self.patches = self._get_ports_and_patches()
+
+        # Get patched ports of src and dst to be used for completion.
+        src_ports = []
+        dst_ports = []
+        for pt in self.patches:
+            src_ports.append(pt['src'])
+            dst_ports.append(pt['dst'])
+
+        while index < len(sub_tokens):
+            if compl_phase == "src_nq" or compl_phase == "dst_nq":
+                if sub_tokens[index - 1] == "reset":
+                    candidates = []
+                    compl_phase = None
+                    continue
+
+                queue_no_list = []
+                for port in self.ports:
+                    split_port = port.split()
+                    if len(split_port) != 3:
+                        continue
+                    if sub_tokens[index - 1] != split_port[0]:
+                        continue
+                    queue_no_list.append(split_port[2])
+
+                if len(queue_no_list) == 0:
+                    if compl_phase == "src_nq":
+                        compl_phase = "dst_res_uid"
+                    elif compl_phase == "dst_nq":
+                        compl_phase = None
+
+            if compl_phase == "src_res_uid":
+                candidates = []
+                for port in self.ports:
+                    if port in src_ports:
+                        continue
+                    if port in candidates:
+                        continue
+                    candidates.append(port.split()[0])
+
+                # If some of ports are patched, `reset` should be included
+                if len(self.patches) != 0:
+                    candidates.append("reset")
+
+                compl_phase = "src_nq"
+
+            elif compl_phase == "src_nq":
+                candidates = ["nq"]
+                compl_phase = "src_q_no"
+
+            elif compl_phase == "src_q_no":
+                candidates = []
+                for queue_no in queue_no_list:
+                    res_uid = "{0} nq {1}".format(
+                        sub_tokens[index - 2], queue_no)
+                    if res_uid in src_ports:
+                        continue
+                    candidates.append(queue_no)
+                compl_phase = "dst_res_uid"
+
+            elif compl_phase == "dst_res_uid":
+                candidates = []
+                for port in self.ports:
+                    if port in dst_ports:
+                        continue
+                    if port in candidates:
+                        continue
+                    candidates.append(port.split()[0])
+
+                compl_phase = "dst_nq"
+
+            elif compl_phase == "dst_nq":
+                candidates = ["nq"]
+                compl_phase = "dst_q_no"
+
+            elif compl_phase == "dst_q_no":
+                candidates = []
+                for queue_no in queue_no_list:
+                    res_uid = "{0} nq {1}".format(
+                        sub_tokens[index - 2], queue_no)
+                    if res_uid in dst_ports:
+                        continue
+                    candidates.append(queue_no)
+                compl_phase = None
 
-            # Get patched ports of src and dst to be used for completion.
-            src_ports = []
-            dst_ports = []
-            for pt in self.patches:
-                src_ports.append(pt['src'])
-                dst_ports.append(pt['dst'])
-
-            # Remove patched ports from candidates.
-            target_idx = len(sub_tokens) - 1  # target is src or dst
-            tmp_ports = self.ports[:]  # candidates
-            if target_idx == 1:  # find src port
-                # If some of ports are patched, `reset` should be included.
-                if self.patches != []:
-                    tmp_ports.append('reset')
-                for pt in src_ports:
-                    tmp_ports.remove(pt)  # remove patched ports
-            else:  # find dst port
-                # If `reset` is given, no need to show dst ports.
-                if sub_tokens[target_idx - 1] == 'reset':
-                    tmp_ports = []
+            else:
+                candidates = []
+                compl_phase = None
+
+            index += 1
+
+        last_index = len(sub_tokens) - 1
+        for candidate in candidates:
+            if candidate.startswith(sub_tokens[last_index]):
+                # Completion does not work correctly if `:` is included in
+                # tokens. Required to create keyword only after `:`.
+                if ':' in sub_tokens[last_index]:  # 'ring:' or 'ring:0'
+                    res.append(candidate.split(':')[1])  # add only after `:`
                 else:
-                    for pt in dst_ports:
-                        tmp_ports.remove(pt)
-
-            # Return candidates.
-            for kw in tmp_ports:
-                if kw.startswith(sub_tokens[target_idx]):
-                    # Completion does not work correctly if `:` is included in
-                    # tokens. Required to create keyword only after `:`.
-                    if ':' in sub_tokens[target_idx]:  # 'ring:' or 'ring:0'
-                        res.append(kw.split(':')[1])  # add only after `:`
-                    else:
-                        res.append(kw)
+                    res.append(candidate)
 
-            return res
+        return res
 
     def _run_status(self):
         """Run `status` command."""
@@ -407,34 +481,73 @@ class SppNfv(object):
     def _run_patch(self, params):
         """Run `patch` command."""
 
+        params_index = 0
+        req_params = {}
+        flg_reset = False
+        flg_mq = False
+
         if len(params) == 0:
-            print('Params are required!')
-        elif params[0] == 'reset':
-            res = self.spp_ctl_cli.delete('nfvs/%d/patches' % self.sec_id)
-            if res is not None:
-                error_codes = self.spp_ctl_cli.rest_common_error_codes
-                if res.status_code == 204:
-                    print('Clear all of patches.')
-                elif res.status_code in error_codes:
-                    pass
-                else:
-                    print('Error: unknown response.')
+            print('Error: Params are required!')
+            return
+
+        while params_index < len(params):
+            if params_index == 0 and params[0] == "reset":
+                flg_reset = True
+                break
+
+            elif params_index == 0:
+                req_params["src"] = params[params_index]
+
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["src"] += "nq" + params[params_index]
+                        flg_mq = True
+
+            elif ((params_index == 1 and flg_mq is False) or
+                    (params_index == 3 and flg_mq is True)):
+                req_params["dst"] = params[params_index]
+
+                if params_index + 2 < len(params):
+                    if params[params_index + 1] == "nq":
+                        params_index += 2
+                        req_params["dst"] += "nq" + params[params_index]
+
+            params_index += 1
+
+        if flg_reset is False:
+            if "src" not in req_params:
+                print("Error: Src port is required!")
+                return
+
+            if "dst" not in req_params:
+                print("Error: Dst port is required!")
+                return
+
+        url = "nfvs/{0}/patches".format(self.sec_id)
+        if flg_reset:
+            res = self.spp_ctl_cli.delete(url)
         else:
-            if len(params) < 2:
-                print('Dst port is required!')
-            else:
-                req_params = {'src': params[0], 'dst': params[1]}
-                res = self.spp_ctl_cli.put(
-                        'nfvs/%d/patches' % self.sec_id, req_params)
-                if res is not None:
-                    error_codes = self.spp_ctl_cli.rest_common_error_codes
-                    if res.status_code == 204:
-                        print('Patch ports (%s -> %s).' % (
-                            params[0], params[1]))
-                    elif res.status_code in error_codes:
-                        pass
-                    else:
-                        print('Error: unknown response.')
+            res = self.spp_ctl_cli.put(url, req_params)
+
+        if res is None:
+            return
+
+        error_codes = self.spp_ctl_cli.rest_common_error_codes
+        if res.status_code in error_codes:
+            pass
+        elif res.status_code != 204:
+            print('Error: unknown response.')
+            return
+
+        if flg_reset:
+            print("Clear all of patches.")
+        else:
+            src = (req_params["src"].replace("nq", " nq ")
+                   if "nq" in req_params["src"] else req_params["src"])
+            dst = (req_params["dst"].replace("nq", " nq ")
+                   if "nq" in req_params["dst"] else req_params["dst"])
+            print("Patch ports ({0} -> {1}).".format(src, dst))
 
     def _run_exit(self):
         """Run `exit` command."""
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 16/17] spp-ctl: add APIs for flow rules
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (32 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 15/17] cli: add support of rte_flow in nfv x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  2020-02-19 11:49 ` [spp] [PATCH v2 17/17] spp_nfv: add support of multi-queue x-fn-spp-ml
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

This patch implements support of flow APIs in spp-ctl.
  - flow validate
  - flow create
  - flow delete
  - flow flush

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/spp-ctl/spp_proc.py   |   4 +
 src/spp-ctl/spp_webapi.py | 200 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 194 insertions(+), 10 deletions(-)

diff --git a/src/spp-ctl/spp_proc.py b/src/spp-ctl/spp_proc.py
index 35919fb..d584f7f 100644
--- a/src/spp-ctl/spp_proc.py
+++ b/src/spp-ctl/spp_proc.py
@@ -317,6 +317,10 @@ class PrimaryProc(SppProc):
     def stop(self):
         return "stop"
 
+    @exec_command
+    def flow(self, command):
+        return command
+
     @exec_command
     def do_launch_sec_proc(self, args):
         proc_name = args['proc_name']
diff --git a/src/spp-ctl/spp_webapi.py b/src/spp-ctl/spp_webapi.py
index 31befe2..d8e6e73 100644
--- a/src/spp-ctl/spp_webapi.py
+++ b/src/spp-ctl/spp_webapi.py
@@ -34,6 +34,20 @@ class KeyInvalid(bottle.HTTPError):
         super(KeyRequired, self).__init__(400, msg)
 
 
+class RequestJSONDecodeHTTPError(bottle.HTTPError):
+
+    def __init__(self):
+        msg = "Not in json format."
+        super().__init__(400, msg)
+
+
+class ResponseJSONDecodeHTTPError(bottle.HTTPError):
+
+    def __init__(self):
+        msg = "Internal Server Error"
+        super().__init__(500, msg)
+
+
 class BaseHandler(bottle.Bottle):
     """Define common methods for each handler."""
 
@@ -55,7 +69,12 @@ class BaseHandler(bottle.Bottle):
             if_type, if_num = port.split(":")
             if if_type not in PORT_TYPES:
                 raise
-            int(if_num)
+            if if_type == "phy" and "nq" in if_num:
+                port_num, queue_num = if_num.split("nq")
+                int(port_num)
+                int(queue_num)
+            else:
+                int(if_num)
         except Exception:
             raise KeyInvalid('port', port)
 
@@ -75,12 +94,15 @@ class BaseHandler(bottle.Bottle):
         def wrapper(*args, **kwargs):
             req = bottle.request
             if req.method in ["POST", "PUT"]:
-                if req.get_header('Content-Type') == "application/json":
-                    body = req.json
-                else:
-                    body = json.loads(req.body.read().decode())
+                try:
+                    if req.get_header('Content-Type') == "application/json":
+                        body = req.json
+                    else:
+                        body = json.loads(req.body.read().decode())
+                    LOG.info("body: %s", body)
+                except Exception:
+                    raise RequestJSONDecodeHTTPError()
                 kwargs['body'] = body
-                LOG.info("body: %s", body)
             return func(*args, **kwargs)
         return wrapper
 
@@ -103,10 +125,15 @@ class BaseHandler(bottle.Bottle):
             ret = func(*args, **kwargs)
             if ret is None:
                 return bottle.HTTPResponse(status=204)
-            else:
-                r = bottle.HTTPResponse(status=200, body=json.dumps(ret))
-                r.content_type = "application/json"
-                return r
+
+            try:
+                body = json.dumps(ret)
+            except Exception:
+                raise ResponseJSONDecodeHTTPError()
+
+            r = bottle.HTTPResponse(status=200, body=body)
+            r.content_type = "application/json"
+            return r
         return wrapper
 
 
@@ -446,12 +473,16 @@ class V1PrimaryHandler(BaseHandler):
 
     def __init__(self, controller):
         super(V1PrimaryHandler, self).__init__(controller)
+        self._initialize()
 
+    def _initialize(self):
         self.set_route()
 
         self.install(self.make_response)
         self.install(self.get_body)
 
+        self.mount("/flow_rules", V1PrimaryFlowHandler(self.ctrl))
+
     def set_route(self):
         self.route('/status', 'GET', callback=self.get_status)
         self.route('/status', 'DELETE', callback=self.clear_status)
@@ -556,6 +587,155 @@ class V1PrimaryHandler(BaseHandler):
         proc.do_exit()
 
 
+class V1PrimaryFlowHandler(V1PrimaryHandler):
+
+    def __init__(self, controller):
+        super().__init__(controller)
+
+    def _initialize(self):
+        self.set_route()
+
+        self.install(self.make_response)
+        self.install(self.get_body)
+
+    def set_route(self):
+        self.route('/port_id/<port_id:int>/validate',
+                   'POST', callback=self.post_flow_validate)
+        self.route('/port_id/<port_id:int>',
+                   'POST', callback=self.post_flow_create)
+        self.route('/port_id/<port_id:int>',
+                   'DELETE', callback=self.delete_flow_all_destroy)
+        self.route('/<rule_id:int>/port_id/<port_id:int>',
+                   'DELETE', callback=self.delete_flow_destroy)
+
+    def post_flow_validate(self, port_id, body):
+        self._check_request_body(body)
+        command = self._create_flow_rule_command(
+            port_id, body.get("rule"), "validate")
+
+        proc = self._get_proc()
+        return proc.flow(command)
+
+    def post_flow_create(self, port_id, body):
+        self._check_request_body(body)
+        command = self._create_flow_rule_command(
+            port_id, body.get("rule"), "create")
+
+        proc = self._get_proc()
+        return proc.flow(command)
+
+    def delete_flow_all_destroy(self, port_id):
+        command = self._gen_flow_destroy(port_id)
+
+        proc = self._get_proc()
+        return proc.flow(command)
+
+    def delete_flow_destroy(self, rule_id, port_id):
+        command = self._gen_flow_destroy(port_id, rule_id)
+
+        proc = self._get_proc()
+        return proc.flow(command)
+
+    def _create_flow_rule_command(self, port_id, rule, sub_command):
+        attr_data = {}
+        data = {}
+
+        # `group`,` priority`, and `transfer` in` attrs` are optional and
+        # may be omitted
+        attr_command = "{group}{priority}{transfer}{direction}"
+
+        attr_data["direction"] = rule.get("direction")
+
+        if "group" in rule:
+            attr_data["group"] = "group {0} ".format(rule.get("group"))
+        else:
+            attr_data["group"] = ""
+
+        if "priority" in rule:
+            attr_data["priority"] = "priority {0} ".format(
+                rule.get("priority"))
+        else:
+            attr_data["priority"] = ""
+
+        if "transfer" in rule:
+            attr_data["transfer"] = "transfer " if rule.get("transfer") else ""
+        else:
+            attr_data["transfer"] = ""
+
+        attrs = attr_command.format(**attr_data)
+
+        command = "flow {sub_command} {res_uid} {attrs} "
+        command += "pattern {pattern} / end "
+        command += "actions {actions} / end"
+
+        data["sub_command"] = sub_command
+        data["res_uid"] = "phy:{0}".format(port_id)
+        data["attrs"] = attrs
+        data["pattern"] = " / ".join(rule.get("pattern"))
+        data["actions"] = " / ".join(rule.get("actions"))
+
+        return command.format(**data)
+
+    def _gen_flow_destroy(self, port_id, rule_id=None):
+        """Delete a flow of given rule ID, or all flows if the ID is None."""
+        if rule_id is not None:
+            target = int(rule_id)
+        else:
+            target = "ALL"
+        return "flow destroy phy:{0} {1}".format(port_id, target)
+
+    def _check_request_body(self, body):
+        self._check_request_body_required_param(body, "rule", dict)
+        rule = body.get("rule")
+
+        self._check_request_body_optional_param(rule, "group", int)
+        self._check_request_body_optional_param(rule, "priority", int)
+        self._check_request_body_required_param(rule, "direction", str)
+        self._check_request_body_optional_param(rule, "transfer", bool)
+        self._check_request_body_required_param(rule, "pattern", list)
+        self._check_request_body_required_param(rule, "actions", list)
+
+        dir = rule.get("direction")
+        if dir != "ingress" and dir != "egress":
+            raise KeyInvalid("direction", dir)
+
+        pattern = rule.get("pattern")
+        for obj in pattern:
+            if obj is None or type(obj) != str:
+                raise KeyInvalid("pattern", pattern)
+
+        actions = rule.get("actions")
+        for obj in actions:
+            if obj is None or type(obj) != str:
+                raise KeyInvalid("actions", actions)
+
+    def _check_request_body_optional_param(self, target, key_name, obj_type):
+        """Check for optional parameter.
+
+        If key_name exists in dict, checking obj_type.
+        If invalid, raise error class. Return True if valid.
+        """
+        if key_name not in target:
+            return True
+        return self._check_request_body_required_param(
+            target, key_name, obj_type)
+
+    def _check_request_body_required_param(self, target, key_name, obj_type):
+        """Check for required parameter.
+
+        key_name must be present and check obj_type.
+        If invalid, raise error class. Return True if valid.
+        """
+        if key_name not in target:
+            raise KeyRequired(key_name)
+
+        obj = target.get(key_name)
+        if obj is None or type(obj) != obj_type:
+            raise KeyInvalid(key_name, obj)
+
+        return True
+
+
 class V1PcapHandler(BaseHandler):
 
     def __init__(self, controller):
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [spp] [PATCH v2 17/17] spp_nfv: add support of multi-queue
  2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
                   ` (33 preceding siblings ...)
  2020-02-19 11:49 ` [spp] [PATCH v2 16/17] spp-ctl: add APIs for flow rules x-fn-spp-ml
@ 2020-02-19 11:49 ` x-fn-spp-ml
  34 siblings, 0 replies; 40+ messages in thread
From: x-fn-spp-ml @ 2020-02-19 11:49 UTC (permalink / raw)
  To: spp, ferruh.yigit, yasufum.o

From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>

To achieve hardware offload in secondary process, multi-queue should be
supported.
Multi-queue support is divided into several changes like following:
- add queue number parameter in patch command
- parse newly added parameter
- add queue number parameter in status response

Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 src/nfv/commands.h   |  71 +++++++---
 src/nfv/main.c       |  18 ++-
 src/nfv/nfv_status.c | 315 +++++++++++++++++++------------------------
 src/nfv/nfv_status.h |  12 +-
 4 files changed, 207 insertions(+), 209 deletions(-)

diff --git a/src/nfv/commands.h b/src/nfv/commands.h
index e5b25be..6b3a935 100644
--- a/src/nfv/commands.h
+++ b/src/nfv/commands.h
@@ -17,7 +17,7 @@
  */
 
 static int
-do_del(char *p_type, int p_id)
+do_del(char *p_type, int p_id, uint16_t queue_id)
 {
 	uint16_t port_id = PORT_RESET;
 
@@ -55,7 +55,7 @@ do_del(char *p_type, int p_id)
 
 	}
 
-	forward_array_remove(port_id);
+	forward_array_remove(port_id, queue_id);
 	port_map_init_one(port_id);
 
 	return 0;
@@ -66,7 +66,7 @@ do_del(char *p_type, int p_id)
  * combination of port type and ID like as 'ring:0'.
  */
 static int
-do_add(char *p_type, int p_id)
+do_add(char *p_type, int p_id, uint16_t queue_id)
 {
 	enum port_type type = UNDEF;
 	uint16_t port_id = PORT_RESET;
@@ -105,9 +105,11 @@ do_add(char *p_type, int p_id)
 	 * other than RING. There is no support to show/clear this stats
 	 * at the moment.
 	 */
+	port_map[port_id].queue_info = NULL;
 
-	/* Update ports_fwd_array with port id */
-	ports_fwd_array[port_id].in_port_id = port_id;
+	/* Update ports_fwd_array with port id and queue id */
+	ports_fwd_array[port_id][queue_id].in_port_id = port_id;
+	ports_fwd_array[port_id][queue_id].in_queue_id = queue_id;
 
 	return 0;
 }
@@ -164,6 +166,7 @@ parse_command(char *str)
 	char port_set[128] = { 0 };
 	char *p_type;
 	int p_id;
+	uint16_t queue_id;
 
 	if (!str)
 		return 0;
@@ -185,12 +188,10 @@ parse_command(char *str)
 		memset(str, '\0', MSG_SIZE);
 		if (cmd == FORWARD)
 			get_sec_stats_json(str, get_client_id(), "running",
-					lcore_id_used,
-					ports_fwd_array, port_map);
+					lcore_id_used);
 		else
 			get_sec_stats_json(str, get_client_id(), "idling",
-					lcore_id_used,
-					ports_fwd_array, port_map);
+					lcore_id_used);
 
 		RTE_ETH_FOREACH_DEV(dev_id) {
 			rte_eth_dev_get_name_by_port(dev_id, dev_name);
@@ -245,11 +246,12 @@ parse_command(char *str)
 	} else if (!strcmp(token_list[0], "add")) {
 		RTE_LOG(DEBUG, SPP_NFV, "Received add command\n");
 
-		ret = parse_resource_uid(token_list[1], &p_type, &p_id);
+		ret = parse_resource_uid(token_list[1], &p_type, &p_id,
+				&queue_id);
 		if (ret < 0)
 			return ret;
 
-		if (do_add(p_type, p_id) < 0) {
+		if (do_add(p_type, p_id, queue_id) < 0) {
 			RTE_LOG(ERR, SPP_NFV, "Failed to do_add()\n");
 			sprintf(result, "%s", "\"failed\"");
 		} else
@@ -282,13 +284,23 @@ parse_command(char *str)
 			char *out_p_type;
 			int in_p_id;
 			int out_p_id;
-
-			parse_resource_uid(token_list[1], &in_p_type, &in_p_id);
+			uint16_t in_queue_id, out_queue_id;
+			int res_uid_str_size = 32;
+			char in_res_uid[res_uid_str_size];
+			char out_res_uid[res_uid_str_size];
+
+			strncpy(in_res_uid, token_list[1],
+				res_uid_str_size - 1);
+			strncpy(out_res_uid, token_list[2],
+				res_uid_str_size - 1);
+
+			parse_resource_uid(token_list[1], &in_p_type, &in_p_id,
+					&in_queue_id);
 			in_port = find_port_id(in_p_id,
 					get_port_type(in_p_type));
 
 			parse_resource_uid(token_list[2],
-					&out_p_type, &out_p_id);
+					&out_p_type, &out_p_id, &out_queue_id);
 			out_port = find_port_id(out_p_id,
 					get_port_type(out_p_type));
 
@@ -314,13 +326,27 @@ parse_command(char *str)
 					"Patch not found, out_port",
 					out_p_type, out_p_id);
 				RTE_LOG(ERR, SPP_NFV, "%s\n", err_msg);
+			} else if (is_valid_port_rxq(in_port, in_queue_id)) {
+				RTE_LOG(ERR, SPP_NFV,
+					"Queue number of in_port"
+					" exceeds definition"
+					" %s:%d nq %d(%s:%d)\n",
+					in_p_type, in_p_id, in_queue_id,
+					__func__, __LINE__);
+			} else if (is_valid_port_txq(out_port, out_queue_id)) {
+				RTE_LOG(ERR, SPP_NFV,
+					"Queue number of out_port"
+					" exceeds definition"
+					" %s:%d nq %d(%s:%d)\n",
+					out_p_type, out_p_id, out_queue_id,
+					__func__, __LINE__);
 			}
 
-			if (add_patch(in_port, out_port) == 0) {
+			if (add_patch(in_port, in_queue_id, out_port,
+				out_queue_id) == 0) {
 				RTE_LOG(INFO, SPP_NFV,
-					"Patched '%s:%d' and '%s:%d'\n",
-					in_p_type, in_p_id,
-					out_p_type, out_p_id);
+					"Patched '%s' and '%s'\n",
+					in_res_uid, out_res_uid);
 				sprintf(result, "%s", "\"succeeded\"");
 			} else {
 				RTE_LOG(ERR, SPP_NFV, "Failed to patch\n");
@@ -328,8 +354,8 @@ parse_command(char *str)
 			}
 
 			sprintf(port_set,
-				"{\"src\":\"%s:%d\",\"dst\":\"%s:%d\"}",
-				in_p_type, in_p_id, out_p_type, out_p_id);
+				"{\"src\":\"%s\",\"dst\":\"%s\"}",
+				in_res_uid, out_res_uid);
 
 			memset(str, '\0', MSG_SIZE);
 			sprintf(str, "{%s:%s,%s:%s,%s:%s}",
@@ -345,11 +371,12 @@ parse_command(char *str)
 
 		cmd = STOP;
 
-		ret = parse_resource_uid(token_list[1], &p_type, &p_id);
+		ret = parse_resource_uid(token_list[1], &p_type, &p_id,
+				&queue_id);
 		if (ret < 0)
 			return ret;
 
-		if (do_del(p_type, p_id) < 0) {
+		if (do_del(p_type, p_id, queue_id) < 0) {
 			RTE_LOG(ERR, SPP_NFV, "Failed to do_del()\n");
 			sprintf(result, "%s", "\"failed\"");
 		} else
diff --git a/src/nfv/main.c b/src/nfv/main.c
index f2c6bfc..32f1074 100644
--- a/src/nfv/main.c
+++ b/src/nfv/main.c
@@ -145,13 +145,14 @@ main(int argc, char *argv[])
 	unsigned int nb_ports;
 	int connected = 0;
 	char str[MSG_SIZE] = { 0 };
-	unsigned int i;
+	unsigned int i, j;
 	int flg_exit;  // used as res of parse_command() to exit if -1
 	int ret;
 	char dev_name[RTE_DEV_NAME_MAX_LEN] = { 0 };
 	int port_type;
 	int nof_phy_port = 0;
 	char log_msg[1024] = { 0 };  /* temporary log message */
+	uint16_t max_queue;
 
 	ret = rte_eal_init(argc, argv);
 	if (ret < 0)
@@ -218,11 +219,22 @@ main(int argc, char *argv[])
 		 * not display to avoid confusion.
 		 */
 
-		/* Update ports_fwd_array with phy port. */
-		ports_fwd_array[i].in_port_id = i;
 		port_map[i].port_type = port_type;
 		port_map[i].id = port_id;
 		port_map[i].stats = &ports->port_stats[i];
+		port_map[i].queue_info = &ports->queue_info[i];
+
+		/* Update ports_fwd_array with phy port. */
+		if (port_map[i].queue_info->rxq >=
+			port_map[i].queue_info->txq)
+			max_queue = port_map[i].queue_info->rxq;
+		else
+			max_queue = port_map[i].queue_info->txq;
+
+		for (j = 0; j < max_queue; j++) {
+			ports_fwd_array[i][j].in_port_id = i;
+			ports_fwd_array[i][j].in_queue_id = j;
+		}
 
 		/* TODO(yasufum) convert from int of port_type to char */
 		RTE_LOG(DEBUG, SPP_NFV, "Add port, type: %d, id: %d\n",
diff --git a/src/nfv/nfv_status.c b/src/nfv/nfv_status.c
index 3947a84..d5d7f44 100644
--- a/src/nfv/nfv_status.c
+++ b/src/nfv/nfv_status.c
@@ -6,6 +6,8 @@
 
 #include <arpa/inet.h>
 #include "shared/common.h"
+#include "shared/basic_forwarder.h"
+#include "shared/port_manager.h"
 #include "nfv_status.h"
 
 /*
@@ -28,9 +30,7 @@
 void
 get_sec_stats_json(char *str, int cli_id,
 		const char *running_stat,
-		uint8_t lcore_id_used[RTE_MAX_LCORE],
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+		uint8_t lcore_id_used[RTE_MAX_LCORE])
 {
 	sprintf(str, "{\"client-id\":%d,", cli_id);
 
@@ -40,10 +40,10 @@ get_sec_stats_json(char *str, int cli_id,
 	append_lcore_info_json(str, lcore_id_used);
 	sprintf(str + strlen(str), ",");
 
-	append_port_info_json(str, ports_fwd_array, port_map);
+	append_port_info_json(str);
 	sprintf(str + strlen(str), ",");
 
-	append_patch_info_json(str, ports_fwd_array, port_map);
+	append_patch_info_json(str);
 	sprintf(str + strlen(str), "}");
 
 	/* Make sure to be terminated with null character. */
@@ -76,53 +76,63 @@ append_lcore_info_json(char *str,
  *     "ports": ["phy:0", "phy:1", "ring:0", "vhost:0"]
  */
 int
-append_port_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+append_port_info_json(char *str)
 {
-	unsigned int i;
+	unsigned int i, j;
 	unsigned int has_port = 0;  // for checking having port at last
+	uint16_t max_queue;
 
 	sprintf(str + strlen(str), "\"ports\":[");
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		max_queue = get_port_max_queues(i);
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
-			continue;
+		for (j = 0; j < max_queue; j++) {
+			if (ports_fwd_array[i][j].in_port_id == PORT_RESET)
+				continue;
 
-		has_port = 1;
-		switch (port_map[i].port_type) {
-		case PHY:
-			sprintf(str + strlen(str), "\"phy:%u\",",
-					port_map[i].id);
-			break;
-		case RING:
-			sprintf(str + strlen(str), "\"ring:%u\",",
-				port_map[i].id);
-			break;
-		case VHOST:
-			sprintf(str + strlen(str), "\"vhost:%u\",",
-				port_map[i].id);
-			break;
-		case PCAP:
-			sprintf(str + strlen(str), "\"pcap:%u\",",
-					port_map[i].id);
-			break;
-		case NULLPMD:
-			sprintf(str + strlen(str), "\"nullpmd:%u\",",
-					port_map[i].id);
-			break;
-		case TAP:
-			sprintf(str + strlen(str), "\"tap:%u\",",
+			has_port = 1;
+			switch (port_map[i].port_type) {
+			case PHY:
+				if (max_queue == 1)
+					sprintf(str + strlen(str),
+						"\"phy:%u\",", port_map[i].id);
+				else
+					sprintf(str + strlen(str),
+						"\"phy:%u nq %u\",",
+						port_map[i].id, j);
+				break;
+			case RING:
+				sprintf(str + strlen(str), "\"ring:%u\",",
 					port_map[i].id);
-			break;
-		case MEMIF:
-			sprintf(str + strlen(str), "\"memif:%u\",",
+				break;
+			case VHOST:
+				sprintf(str + strlen(str), "\"vhost:%u\",",
 					port_map[i].id);
-			break;
-		case UNDEF:
-			/* TODO(yasufum) Need to remove print for undefined ? */
-			sprintf(str + strlen(str), "\"udf\",");
-			break;
+				break;
+			case PCAP:
+				sprintf(str + strlen(str), "\"pcap:%u\",",
+						port_map[i].id);
+				break;
+			case NULLPMD:
+				sprintf(str + strlen(str), "\"nullpmd:%u\",",
+						port_map[i].id);
+				break;
+			case TAP:
+				sprintf(str + strlen(str), "\"tap:%u\",",
+						port_map[i].id);
+				break;
+			case MEMIF:
+				sprintf(str + strlen(str), "\"memif:%u\",",
+						port_map[i].id);
+				break;
+			case UNDEF:
+				/*
+				 * TODO(yasufum) Need to remove print for
+				 * undefined ?
+				 */
+				sprintf(str + strlen(str), "\"udf\",");
+				break;
+			}
 		}
 	}
 
@@ -136,6 +146,57 @@ append_port_info_json(char *str,
 	return 0;
 }
 
+static void
+append_port_string(char *str, enum port_type port_type,
+		uint16_t port_id, uint16_t queue_id, int max_queue)
+{
+	switch (port_type) {
+	case PHY:
+		RTE_LOG(INFO, SHARED, "Type: PHY\n");
+		if (max_queue > 1)
+			sprintf(str, "\"phy:%u nq %u\"", port_id, queue_id);
+		else
+			sprintf(str, "\"phy:%u\"", port_id);
+		break;
+
+	case RING:
+		RTE_LOG(INFO, SHARED, "Type: RING\n");
+		sprintf(str, "\"ring:%u\"", port_id);
+		break;
+
+	case VHOST:
+		RTE_LOG(INFO, SHARED, "Type: VHOST\n");
+		sprintf(str, "\"vhost:%u\"", port_id);
+		break;
+
+	case PCAP:
+		RTE_LOG(INFO, SHARED, "Type: PCAP\n");
+		sprintf(str, "\"pcap:%u\"", port_id);
+		break;
+
+	case NULLPMD:
+		RTE_LOG(INFO, SHARED, "Type: NULLPMD\n");
+		sprintf(str, "\"nullpmd:%u\"", port_id);
+		break;
+
+	case TAP:
+		RTE_LOG(INFO, SHARED, "Type: TAP\n");
+		sprintf(str, "\"tap:%u\"", port_id);
+		break;
+
+	case MEMIF:
+		RTE_LOG(INFO, SHARED, "Type: MEMIF\n");
+		sprintf(str, "\"memif:%u\"", port_id);
+		break;
+
+	case UNDEF:
+		RTE_LOG(INFO, SHARED, "Type: UDF\n");
+		/* TODO(yasufum) Need to remove print for undefined ? */
+		sprintf(str, "\"udf\"");
+		break;
+	}
+}
+
 /*
  * Append patch info to sec status. It is called from get_sec_stats_json()
  * to add a JSON formatted patch info to given 'str'. Here is an example.
@@ -146,152 +207,56 @@ append_port_info_json(char *str,
  *      ]
  */
 int
-append_patch_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map)
+append_patch_info_json(char *str)
 {
-	unsigned int i;
+	unsigned int i, j;
 	unsigned int has_patch = 0;  // for checking having patch at last
+	unsigned int out_port_id;
+	uint16_t out_queue_id;
+	uint16_t in_max_queue, out_max_queue;
 
 	char patch_str[128];
 	sprintf(str + strlen(str), "\"patches\":[");
 	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		in_max_queue = get_port_max_queues(i);
 
-		if (ports_fwd_array[i].in_port_id == PORT_RESET)
-			continue;
-
-		RTE_LOG(INFO, SHARED, "Port ID %d\n", i);
-		RTE_LOG(INFO, SHARED, "Status %d\n",
-			ports_fwd_array[i].in_port_id);
+		for (j = 0; j < in_max_queue; j++) {
 
-		memset(patch_str, '\0', sizeof(patch_str));
+			if (ports_fwd_array[i][j].in_port_id == PORT_RESET ||
+				ports_fwd_array[i][j].out_port_id == PORT_RESET)
+				continue;
 
-		sprintf(patch_str, "{\"src\":");
-
-		switch (port_map[i].port_type) {
-		case PHY:
-			RTE_LOG(INFO, SHARED, "Type: PHY\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"phy:%u\",",
-					port_map[i].id);
-			break;
-		case RING:
-			RTE_LOG(INFO, SHARED, "Type: RING\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"ring:%u\",",
-					port_map[i].id);
-			break;
-		case VHOST:
-			RTE_LOG(INFO, SHARED, "Type: VHOST\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"vhost:%u\",",
-					port_map[i].id);
-			break;
-		case PCAP:
-			RTE_LOG(INFO, SHARED, "Type: PCAP\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"pcap:%u\",",
-					port_map[i].id);
-			break;
-		case NULLPMD:
-			RTE_LOG(INFO, SHARED, "Type: NULLPMD\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"nullpmd:%u\",",
-					port_map[i].id);
-			break;
-		case TAP:
-			RTE_LOG(INFO, SHARED, "Type: TAP\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"tap:%u\",",
-					port_map[i].id);
-			break;
-		case MEMIF:
-			RTE_LOG(INFO, SHARED, "Type: MEMIF\n");
-			sprintf(patch_str + strlen(patch_str),
-					"\"memif:%u\",",
-					port_map[i].id);
-			break;
-		case UNDEF:
-			RTE_LOG(INFO, SHARED, "Type: UDF\n");
-			/* TODO(yasufum) Need to remove print for undefined ? */
-			sprintf(patch_str + strlen(patch_str),
-					"\"udf\",");
-			break;
-		}
-
-		sprintf(patch_str + strlen(patch_str), "\"dst\":");
-
-		RTE_LOG(INFO, SHARED, "Out Port ID %d\n",
-				ports_fwd_array[i].out_port_id);
-
-		if (ports_fwd_array[i].out_port_id == PORT_RESET) {
-			//sprintf(patch_str + strlen(patch_str), "%s", "\"\"");
-			continue;
-		} else {
 			has_patch = 1;
-			unsigned int j = ports_fwd_array[i].out_port_id;
-			switch (port_map[j].port_type) {
-			case PHY:
-				RTE_LOG(INFO, SHARED, "Type: PHY\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"phy:%u\"",
-						port_map[j].id);
-				break;
-			case RING:
-				RTE_LOG(INFO, SHARED, "Type: RING\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"ring:%u\"",
-						port_map[j].id);
-				break;
-			case VHOST:
-				RTE_LOG(INFO, SHARED, "Type: VHOST\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"vhost:%u\"",
-						port_map[j].id);
-				break;
-			case PCAP:
-				RTE_LOG(INFO, SHARED, "Type: PCAP\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"pcap:%u\"",
-						port_map[j].id);
-				break;
-			case NULLPMD:
-				RTE_LOG(INFO, SHARED, "Type: NULLPMD\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"nullpmd:%u\"",
-						port_map[j].id);
-				break;
-			case TAP:
-				RTE_LOG(INFO, SHARED, "Type: TAP\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"tap:%u\"",
-						port_map[j].id);
-				break;
-			case MEMIF:
-				RTE_LOG(INFO, SHARED, "Type: MEMIF\n");
-				sprintf(patch_str + strlen(patch_str),
-						"\"memif:%u\"",
-						port_map[j].id);
-				break;
-			case UNDEF:
-				RTE_LOG(INFO, SHARED, "Type: UDF\n");
-				/*
-				 * TODO(yasufum) Need to remove print for
-				 * undefined ?
-				 */
-				sprintf(patch_str + strlen(patch_str),
-						"\"udf\"");
-				break;
-			}
-		}
-
-		sprintf(patch_str + strlen(patch_str), "},");
+			RTE_LOG(INFO, SHARED, "Port ID %d\n", i);
+			RTE_LOG(INFO, SHARED, "Queue ID %d\n", j);
+			RTE_LOG(INFO, SHARED, "Status %d\n",
+				ports_fwd_array[i][j].in_port_id);
+
+			memset(patch_str, '\0', sizeof(patch_str));
+
+			sprintf(patch_str, "{\"src\":");
+			append_port_string(patch_str + strlen(patch_str),
+				port_map[i].port_type, port_map[i].id,
+				j, in_max_queue);
+			sprintf(patch_str + strlen(patch_str), ",\"dst\":");
+
+			out_port_id = ports_fwd_array[i][j].out_port_id;
+			out_queue_id = ports_fwd_array[i][j].out_queue_id;
+			RTE_LOG(INFO, SHARED, "Out Port ID %d\n", out_port_id);
+			RTE_LOG(INFO, SHARED, "Out Queue ID %d\n",
+				out_queue_id);
+
+			out_max_queue = get_port_max_queues(out_port_id);
+			append_port_string(patch_str + strlen(patch_str),
+				port_map[out_port_id].port_type,
+				port_map[out_port_id].id,
+				out_queue_id, out_max_queue);
+			sprintf(patch_str + strlen(patch_str), "},");
 
-		if (has_patch != 0)
 			sprintf(str + strlen(str), "%s", patch_str);
+		}
 	}
 
-
 	/* Check if it has at least one patch to remove ",". */
 	if (has_patch == 0) {
 		sprintf(str + strlen(str), "]");
diff --git a/src/nfv/nfv_status.h b/src/nfv/nfv_status.h
index 14225cc..cf59dde 100644
--- a/src/nfv/nfv_status.h
+++ b/src/nfv/nfv_status.h
@@ -8,21 +8,15 @@
 /* Get status of spp_nfv or spp_vm as JSON format. */
 void get_sec_stats_json(char *str, int client_id,
 		const char *running_stat,
-		uint8_t lcore_id_used[RTE_MAX_LCORE],
-		struct port *ports_fwd_array,
-		struct port_map *port_map);
+		uint8_t lcore_id_used[RTE_MAX_LCORE]);
 
 int append_lcore_info_json(char *str,
 		uint8_t lcore_id_used[RTE_MAX_LCORE]);
 
 /* Append port info to sec status, called from get_sec_stats_json(). */
-int append_port_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map);
+int append_port_info_json(char *str);
 
 /* Append patch info to sec status, called from get_sec_stats_json(). */
-int append_patch_info_json(char *str,
-		struct port *ports_fwd_array,
-		struct port_map *port_map);
+int append_patch_info_json(char *str);
 
 #endif
-- 
2.17.1


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [spp] (x-fn-spp-ml 118) Re: [PATCH 06/17] spp_primary: add support of rte_flow
  2020-02-19  2:24   ` Yasufumi Ogawa
@ 2020-02-19 11:57     ` Hideyuki Yamashita
  0 siblings, 0 replies; 40+ messages in thread
From: Hideyuki Yamashita @ 2020-02-19 11:57 UTC (permalink / raw)
  To: Yasufumi Ogawa; +Cc: x-fn-spp-ml, ferruh.yigit, spp

Hello Yasufumi,

Thanks for your pointing out and my appologies.
I have sent updated patch set as v2 
which resolvedd the issues you pointed out.

Note that during my test email of v2 patch,
I hvae included you as CC.
So you might receive 2 sets of v2 patches.
But anyway those are completely identical.
So please discard one of it.
Sorry again.


Thanks in advance!
BR,
Hideyuki Yamashita
NTT TechnoCross

> > From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
> >
> > To support rte_flow in SPP, this patch provides main functions which
> > defines validate, create, delete and flush flow
> This patch has several issues from checkpatches.sh. Please fix bellow.
> 
> WARNING:RETURN_VOID: void function return statements are not generally useful
> #604: FILE: src/primary/flow/flow.c:581:
> +	return;
> +}
> 
> ERROR:SWITCH_CASE_INDENT_LEVEL: switch and case should be at the same indent
> #700: FILE: src/primary/flow/flow.c:677:
> +	switch(input->command) {
> +		case VALIDATE:
> [...]
> +		case CREATE:
> [...]
> +		case DESTROY:
> [...]
> +		case FLUSH:
> 
> ERROR:SPACING: space required before the open parenthesis '('
> #700: FILE: src/primary/flow/flow.c:677:
> +	switch(input->command) {
> 
> WARNING:RETURN_VOID: void function return statements are not generally useful
> #729: FILE: src/primary/flow/flow.c:706:
> +	return;
> +}
> 
> Regards,
> Yasufumi
> >
> > Signed-off-by: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
> > Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
> > ---
> >   src/primary/flow/flow.c | 1048 +++++++++++++++++++++++++++++++++++++++
> >   src/primary/flow/flow.h |   94 ++++
> >   2 files changed, 1142 insertions(+)
> >   create mode 100644 src/primary/flow/flow.c
> >   create mode 100644 src/primary/flow/flow.h
> >
> > diff --git a/src/primary/flow/flow.c b/src/primary/flow/flow.c
> > new file mode 100644
> > index 0000000..52a3e59
> > --- /dev/null
> > +++ b/src/primary/flow/flow.c
> > @@ -0,0 +1,1048 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
> > + */
> > +
> > +#include <rte_flow.h>
> > +#include <rte_common.h>
> > +#include <rte_ether.h>
> > +#include <rte_byteorder.h>
> > +
> > +#include "shared/common.h"
> > +#include "shared/secondary/utils.h"
> > +#include "shared/secondary/spp_worker_th/data_types.h"
> > +#include "primary/primary.h"
> > +#include "flow.h"
> > +#include "attr.h"
> > +#include "common.h"
> > +
> > +#include "primary/flow/pattern/eth.h"
> > +#include "primary/flow/pattern/vlan.h"
> > +
> > +#include "primary/flow/action/jump.h"
> > +#include "primary/flow/action/queue.h"
> > +#include "primary/flow/action/of_push_vlan.h"
> > +#include "primary/flow/action/of_set_vlan_vid.h"
> > +#include "primary/flow/action/of_set_vlan_pcp.h"
> > +
> > +
> > +/* Flow list for each port */
> > +static struct port_flow port_list[RTE_MAX_ETHPORTS] = { 0 };
> > +
> > +/* Define item operations */
> > +static struct flow_item_ops flow_item_ops_list[] = {
> > +	{
> > +		.str_type = "end",
> > +		.type = RTE_FLOW_ITEM_TYPE_END,
> > +		.parse = NULL,
> > +		.detail_list = NULL
> > +	},
> > +	{
> > +		.str_type = "eth",
> > +		.type = RTE_FLOW_ITEM_TYPE_ETH,
> > +		.size = sizeof(struct rte_flow_item_eth),
> > +		.parse = parse_item_common,
> > +		.detail_list = eth_ops_list,
> > +		.status = append_item_eth_json,
> > +	},
> > +	{
> > +		.str_type = "vlan",
> > +		.type = RTE_FLOW_ITEM_TYPE_VLAN,
> > +		.size = sizeof(struct rte_flow_item_vlan),
> > +		.parse = parse_item_common,
> > +		.detail_list = vlan_ops_list,
> > +		.status = append_item_vlan_json,
> > +	},
> > +};
> > +
> > +/* Define action operations */
> > +static struct flow_action_ops flow_action_ops_list[] = {
> > +	{
> > +		.str_type = "end",
> > +		.type = RTE_FLOW_ACTION_TYPE_END,
> > +		.size = 0,
> > +		.parse = NULL,
> > +		.detail_list = NULL,
> > +		.status = NULL,
> > +	},
> > +	{
> > +		.str_type = "jump",
> > +		.type = RTE_FLOW_ACTION_TYPE_JUMP,
> > +		.size = sizeof(struct rte_flow_action_jump),
> > +		.parse = parse_action_common,
> > +		.detail_list = jump_ops_list,
> > +		.status = append_action_jump_json,
> > +	},
> > +	{
> > +		.str_type = "queue",
> > +		.type = RTE_FLOW_ACTION_TYPE_QUEUE,
> > +		.size = sizeof(struct rte_flow_action_queue),
> > +		.parse = parse_action_common,
> > +		.detail_list = queue_ops_list,
> > +		.status = append_action_queue_json,
> > +	},
> > +	{
> > +		.str_type = "of_pop_vlan",
> > +		.type = RTE_FLOW_ACTION_TYPE_OF_POP_VLAN,
> > +		.size = 0,
> > +		.parse = NULL,
> > +		.detail_list = NULL,
> > +		.status = append_action_null_json,
> > +	},
> > +	{
> > +		.str_type = "of_push_vlan",
> > +		.type = RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN,
> > +		.size = sizeof(struct rte_flow_action_of_push_vlan),
> > +		.parse = parse_action_common,
> > +		.detail_list = of_push_vlan_ops_list,
> > +		.status = append_action_of_push_vlan_json,
> > +	},
> > +	{
> > +		.str_type = "of_set_vlan_vid",
> > +		.type = RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID,
> > +		.size = sizeof(struct rte_flow_action_of_set_vlan_vid),
> > +		.parse = parse_action_common,
> > +		.detail_list = of_set_vlan_vid_ops_list,
> > +		.status = append_action_of_set_vlan_vid_json,
> > +	},
> > +	{
> > +		.str_type = "of_set_vlan_pcp",
> > +		.type = RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP,
> > +		.size = sizeof(struct rte_flow_action_of_set_vlan_pcp),
> > +		.parse = parse_action_common,
> > +		.detail_list = of_set_vlan_pcp_ops_list,
> > +		.status = append_action_of_set_vlan_pcp_json,
> > +	},
> > +};
> > +
> > +/* Free memory of "flow_args". */
> > +static void
> > +free_flow_args(struct flow_args *input)
> > +{
> > +	int i;
> > +	struct rte_flow_item *pattern;
> > +	struct rte_flow_action *actions;
> > +	char **target;
> > +
> > +	if ((input->command != VALIDATE) &&
> > +		(input->command != CREATE))
> > +		return;
> > +
> > +	pattern = input->args.rule.pattern;
> > +	if (pattern != NULL) {
> > +		for (i = 0; pattern[i].type != RTE_FLOW_ITEM_TYPE_END; i++) {
> > +			target = (char **)((char *)(&pattern[i]) +
> > +				offsetof(struct rte_flow_item, spec));
> > +			if (*target != NULL)
> > +				free(*target);
> > +
> > +			target = (char **)((char *)(&pattern[i]) +
> > +				offsetof(struct rte_flow_item, last));
> > +			if (*target != NULL)
> > +				free(*target);
> > +
> > +			target = (char **)((char *)(&pattern[i]) +
> > +				offsetof(struct rte_flow_item, mask));
> > +			if (*target != NULL)
> > +				free(*target);
> > +		}
> > +
> > +		free(pattern);
> > +	}
> > +
> > +	actions = input->args.rule.actions;
> > +	if (actions != NULL) {
> > +		for (i = 0; actions[i].type != RTE_FLOW_ACTION_TYPE_END; i++) {
> > +			target = (char **)((char *)(&actions[i]) +
> > +				offsetof(struct rte_flow_action, conf));
> > +			if (*target != NULL)
> > +				free(*target);
> > +		}
> > +
> > +		free(actions);
> > +	}
> > +}
> > +
> > +/*
> > + * Create response in JSON format.
> > + * `rule_id` must be empty if flow create is failed.
> > + */
> > +static void
> > +make_response(char *response, const char *result, const char *message,
> > +	char *rule_id)
> > +{
> > +	if (rule_id == NULL)
> > +		snprintf(response, MSG_SIZE,
> > +			"{\"result\": \"%s\", \"message\": \"%s\"}",
> > +			result, message);
> > +	else
> > +		snprintf(response, MSG_SIZE,
> > +			"{\"result\": \"%s\", \"message\": \"%s\", "
> > +			"\"rule_id\": \"%s\"}",
> > +			result, message, rule_id);
> > +}
> > +
> > +/* Create error response from rte_flow_error */
> > +static void
> > +make_error_response(char *response, const char *message,
> > +	struct rte_flow_error error, char *rule_id)
> > +{
> > +	/* Define description for each error type */
> > +	static const char *const errstr_list[] = {
> > +		[RTE_FLOW_ERROR_TYPE_NONE] = "No error",
> > +		[RTE_FLOW_ERROR_TYPE_UNSPECIFIED] = "Cause unspecified",
> > +		[RTE_FLOW_ERROR_TYPE_HANDLE] = "Flow rule (handle)",
> > +		[RTE_FLOW_ERROR_TYPE_ATTR_GROUP] = "Group field",
> > +		[RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY] = "Priority field",
> > +		[RTE_FLOW_ERROR_TYPE_ATTR_INGRESS] = "Ingress field",
> > +		[RTE_FLOW_ERROR_TYPE_ATTR_EGRESS] = "Egress field",
> > +		[RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER] = "Transfer field",
> > +		[RTE_FLOW_ERROR_TYPE_ATTR] = "Attributes structure",
> > +		[RTE_FLOW_ERROR_TYPE_ITEM_NUM] = "Pattern length",
> > +		[RTE_FLOW_ERROR_TYPE_ITEM_SPEC] = "Item specification",
> > +		[RTE_FLOW_ERROR_TYPE_ITEM_LAST] = "Item specification range",
> > +		[RTE_FLOW_ERROR_TYPE_ITEM_MASK] = "Item specification mask",
> > +		[RTE_FLOW_ERROR_TYPE_ITEM] = "Specific pattern item",
> > +		[RTE_FLOW_ERROR_TYPE_ACTION_NUM] = "Number of actions",
> > +		[RTE_FLOW_ERROR_TYPE_ACTION_CONF] = "Action configuration",
> > +		[RTE_FLOW_ERROR_TYPE_ACTION] = "Specific action",
> > +	};
> > +	int err = rte_errno;
> > +	char msg[512] = "";
> > +	char cause[32] = "";
> > +	const char *errstr;
> > +
> > +	if ((unsigned int)error.type >= RTE_DIM(errstr_list) ||
> > +	    !errstr_list[error.type])
> > +		errstr = "Unknown type";
> > +	else
> > +		errstr = errstr_list[error.type];
> > +
> > +
> > +	if (error.cause != NULL)
> > +		snprintf(cause, sizeof(cause), "cause: %p\\n", error.cause);
> > +
> > +	snprintf(msg, sizeof(msg),
> > +		"%s\\nerror type: %d (%s)\\n"
> > +		"%serror message: %s\\nrte_errno: %s",
> > +		message, error.type, errstr, cause,
> > +		error.message ? error.message : "(no stated reason)",
> > +		rte_strerror(err));
> > +	make_response(response, "error", msg, rule_id);
> > +}
> > +
> > +/* Add to array, redeclare memory. */
> > +static int
> > +append_object_list(void **list, void *add, size_t obj_size, int num)
> > +{
> > +	char *new_list;
> > +
> > +	new_list = malloc(obj_size * num);
> > +	if (new_list == NULL)
> > +		return -1;
> > +
> > +	/* Copy original list*/
> > +	if (*list != NULL) {
> > +		memcpy(new_list, *list, obj_size * (num - 1));
> > +		free(*list);
> > +	}
> > +
> > +	/* Add to list */
> > +	memcpy(new_list + (obj_size * (num - 1)), add, obj_size);
> > +
> > +	*list = (void *)new_list;
> > +	return 0;
> > +}
> > +
> > +static int
> > +parse_flow_actions(char *token_list[], int *index,
> > +	struct rte_flow_action **actions)
> > +{
> > +	int ret;
> > +	int action_count = 0;
> > +	uint16_t i;
> > +	char *token;
> > +	struct flow_action_ops *ops;
> > +	struct rte_flow_action action;
> > +
> > +	if (strcmp(token_list[*index], "actions")) {
> > +		RTE_LOG(ERR, SPP_FLOW,
> > +			"Invalid parameter is %s(%s:%d)\n",
> > +			token_list[*index], __func__, __LINE__);
> > +		return -1;
> > +	}
> > +
> > +	/* Next to word */
> > +	(*index)++;
> > +
> > +	while (token_list[*index] != NULL) {
> > +		token = token_list[*index];
> > +
> > +		for (i = 0; i < RTE_DIM(flow_action_ops_list); i++) {
> > +			ops = &flow_action_ops_list[i];
> > +			if (strcmp(token, ops->str_type))
> > +				continue;
> > +
> > +			memset(&action, 0, sizeof(struct rte_flow_action));
> > +			action.type = ops->type;
> > +			if (ops->parse != NULL) {
> > +				ret = ops->parse(token_list, index, &action,
> > +					ops);
> > +				if (ret < 0)
> > +					return -1;
> > +			} else {
> > +				(*index)++;
> > +			}
> > +			break;
> > +		}
> > +
> > +		/*
> > +		 * Error occurs if a action string that is not defined in
> > +		 * str_type of flow_action_ops_list is specified
> > +		 */
> > +		if (i == RTE_DIM(flow_action_ops_list)) {
> > +			RTE_LOG(ERR, SPP_FLOW,
> > +				"Invalid parameter "
> > +				"is %s action(%s:%d)\n",
> > +				token, __func__, __LINE__);
> > +			return -1;
> > +		}
> > +
> > +		/* Add to "actions" list */
> > +		action_count++;
> > +		ret = append_object_list((void **)actions, &action,
> > +			sizeof(struct rte_flow_action), action_count);
> > +
> > +		if (!strcmp(token, "end"))
> > +			break;
> > +
> > +		(*index)++;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int
> > +parse_flow_pattern(char *token_list[], int *index,
> > +	struct rte_flow_item **pattern)
> > +{
> > +	int ret;
> > +	int item_count = 0;
> > +	uint32_t i;
> > +	char *token;
> > +	struct flow_item_ops *ops;
> > +	struct rte_flow_item item;
> > +
> > +	while (token_list[*index] != NULL) {
> > +		token = token_list[*index];
> > +
> > +		for (i = 0; i < RTE_DIM(flow_item_ops_list); i++) {
> > +			ops = &flow_item_ops_list[i];
> > +			if (strcmp(token, ops->str_type))
> > +				continue;
> > +
> > +			memset(&item, 0, sizeof(struct rte_flow_item));
> > +			item.type = ops->type;
> > +			if (ops->parse != NULL) {
> > +				ret = ops->parse(token_list, index, &item,
> > +					ops);
> > +				if (ret < 0)
> > +					return -1;
> > +			}
> > +			break;
> > +		}
> > +
> > +		/*
> > +		 * Error occurs if a pattern string that is not defined in
> > +		 * str_type of flow_item_ops_list is specified
> > +		 */
> > +		if (i == RTE_DIM(flow_item_ops_list)) {
> > +			RTE_LOG(ERR, SPP_FLOW,
> > +				"Invalid parameter "
> > +				"is %s pattern(%s:%d)\n",
> > +				token, __func__, __LINE__);
> > +			return -1;
> > +		}
> > +
> > +		/* Add to "pattern" list */
> > +		item_count++;
> > +		ret = append_object_list((void **)pattern, &item,
> > +			sizeof(struct rte_flow_item), item_count);
> > +
> > +		if (!strcmp(token, "end"))
> > +			break;
> > +
> > +		(*index)++;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int
> > +parse_flow_rule(char *token_list[], struct flow_args *input)
> > +{
> > +	int ret = 0;
> > +	int index;
> > +
> > +	ret = parse_phy_port_id(token_list[2], &input->port_id);
> > +	if (ret < 0)
> > +		return -1;
> > +
> > +	/* The next index of the port */
> > +	index = 3;
> > +
> > +	/* Attribute parse */
> > +	ret = parse_flow_attr(token_list, &index, &input->args.rule.attr);
> > +	if (ret < 0) {
> > +		RTE_LOG(ERR, SPP_FLOW,
> > +			"Failed to parse Attribute(%s:%d)\n",
> > +			__func__, __LINE__);
> > +		return -1;
> > +	}
> > +
> > +	/* The next index of the pattern */
> > +	index++;
> > +
> > +	/* Pattern parse */
> > +	ret = parse_flow_pattern(token_list, &index,
> > +		&input->args.rule.pattern);
> > +	if (ret < 0) {
> > +		RTE_LOG(ERR, SPP_FLOW,
> > +			"Failed to parse Pattern(%s:%d)\n",
> > +			__func__, __LINE__);
> > +		return -1;
> > +	}
> > +
> > +	/* The next index of the actions */
> > +	index++;
> > +
> > +	/* Actions parse */
> > +	ret = parse_flow_actions(token_list, &index,
> > +		&input->args.rule.actions);
> > +	if (ret < 0) {
> > +		RTE_LOG(ERR, SPP_FLOW,
> > +			"Failed to parse Actions(%s:%d)\n",
> > +			__func__, __LINE__);
> > +		return -1;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int
> > +parse_flow_destroy(char *token_list[], struct flow_args *input)
> > +{
> > +	int ret;
> > +	char *end;
> > +
> > +	ret = parse_phy_port_id(token_list[2], &input->port_id);
> > +	if (ret < 0)
> > +		return -1;
> > +
> > +	if (token_list[3] == NULL) {
> > +		RTE_LOG(ERR, SPP_FLOW,
> > +			"rule_id is not specified(%s:%d)\n",
> > +			__func__, __LINE__);
> > +		return -1;
> > +	}
> > +
> > +	if (!strcmp(token_list[3], "ALL")) {
> > +		input->command = FLUSH;
> > +
> > +	} else {
> > +		input->command = DESTROY;
> > +		input->args.destroy.rule_id = strtoul(token_list[3],
> > +			&end, 10);
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +/** Generate a flow_rule entry from attributes/pattern/actions. */
> > +static struct flow_rule *
> > +create_flow_rule(struct rte_flow_attr *attr,
> > +	struct rte_flow_item *pattern,
> > +	struct rte_flow_action *actions,
> > +	struct rte_flow_error *error)
> > +{
> > +	const struct rte_flow_conv_rule conv_rule = {
> > +		.attr_ro = attr,
> > +		.pattern_ro = pattern,
> > +		.actions_ro = actions,
> > +	};
> > +	struct flow_rule *rule;
> > +	int ret;
> > +
> > +	ret = rte_flow_conv(RTE_FLOW_CONV_OP_RULE, NULL, 0, &conv_rule,
> > +		error);
> > +	if (ret < 0)
> > +		return NULL;
> > +
> > +	rule = calloc(1, offsetof(struct flow_rule, rule) + ret);
> > +	if (!rule) {
> > +		rte_flow_error_set
> > +			(error, errno, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> > +			 "calloc() failed");
> > +		return NULL;
> > +	}
> > +
> > +	ret = rte_flow_conv(RTE_FLOW_CONV_OP_RULE, &rule->rule, ret, &conv_rule,
> > +			  error);
> > +	if (ret >= 0)
> > +		return rule;
> > +
> > +	free(rule);
> > +	return NULL;
> > +}
> > +
> > +/* Execute rte_flow_validate().*/
> > +static void
> > +exec_flow_validate(int port_id,
> > +	struct rte_flow_attr *attr,
> > +	struct rte_flow_item *pattern,
> > +	struct rte_flow_action *actions,
> > +	char *response)
> > +{
> > +	int ret;
> > +	struct rte_flow_error error;
> > +
> > +	memset(&error, 0, sizeof(error));
> > +
> > +	ret = rte_flow_validate(port_id, attr, pattern, actions, &error);
> > +	if (ret != 0)
> > +		make_error_response(response, "Flow validate error", error,
> > +			NULL);
> > +	else
> > +		make_response(response, "success", "Flow rule validated",
> > +			NULL);
> > +}
> > +
> > +/* Execute rte_flow_create(). Save flow rules globally */
> > +static void
> > +exec_flow_create(int port_id,
> > +	struct rte_flow_attr *attr,
> > +	struct rte_flow_item *pattern,
> > +	struct rte_flow_action *actions,
> > +	char *response)
> > +{
> > +	uint32_t rule_id;
> > +	char mes[32];
> > +	char rule_id_str[11] = {0};
> > +	struct rte_flow_error error;
> > +	struct rte_flow *flow;
> > +	struct flow_rule *rule;
> > +	struct port_flow *port;
> > +
> > +	memset(&error, 0, sizeof(error));
> > +
> > +	flow = rte_flow_create(port_id, attr, pattern, actions, &error);
> > +	if (flow == NULL) {
> > +		make_error_response(response, "Flow create error", error,
> > +			rule_id_str);
> > +		return;
> > +	}
> > +
> > +	port = &port_list[port_id];
> > +	if (port->flow_list != NULL) {
> > +		if (port->flow_list->rule_id >= UINT32_MAX) {
> > +			make_response(response, "error",
> > +				"Rule ID must be less than %"PRIu32,
> > +				rule_id_str);
> > +			rte_flow_destroy(port_id, flow, NULL);
> > +			return;
> > +		}
> > +		rule_id = port->flow_list->rule_id + 1;
> > +	} else {
> > +		rule_id = 0;
> > +	}
> > +
> > +	rule = create_flow_rule(attr, pattern, actions, &error);
> > +	if (rule == NULL) {
> > +		rte_flow_destroy(port_id, flow, NULL);
> > +		make_error_response(response, "Flow create error", error,
> > +			rule_id_str);
> > +		return;
> > +	}
> > +
> > +	/* Keep it globally as a list */
> > +	rule->rule_id = rule_id;
> > +	rule->flow_handle = flow;
> > +
> > +	if (port->flow_list == NULL)
> > +		rule->prev = NULL;
> > +	else
> > +		rule->prev = port->flow_list;
> > +
> > +	port->flow_list = rule;
> > +
> > +	sprintf(mes, "Flow rule #%d created", rule_id);
> > +	sprintf(rule_id_str, "%d", rule_id);
> > +	make_response(response, "success", mes, rule_id_str);
> > +	return;
> > +}
> > +
> > +/* Execute rte_flow_destroy(). Destroying a globally saved flow rule */
> > +static void
> > +exec_flow_destroy(int port_id, uint32_t rule_id, char *response)
> > +{
> > +	int ret;
> > +	int found_flg = 0;
> > +	char mes[64];
> > +	struct flow_rule *rule, **next_ptr;
> > +	struct rte_flow_error error;
> > +
> > +	memset(&error, 0, sizeof(error));
> > +
> > +	ret = is_portid_used(port_id);
> > +	if (ret != 0) {
> > +		sprintf(mes, "Invalid port %d", port_id);
> > +		make_response(response, "error", mes, NULL);
> > +		return;
> > +	}
> > +
> > +	next_ptr = &(port_list[port_id].flow_list);
> > +	rule = port_list[port_id].flow_list;
> > +
> > +	while (rule != NULL) {
> > +		if (rule->rule_id != rule_id) {
> > +			next_ptr = &(rule->prev);
> > +			rule = rule->prev;
> > +			continue;
> > +		}
> > +
> > +		ret = rte_flow_destroy(port_id, rule->flow_handle, &error);
> > +		if (ret != 0) {
> > +			make_error_response(response, "Flow destroy error",
> > +				error, NULL);
> > +			return;
> > +		}
> > +
> > +		/* Remove flow from global list */
> > +		*next_ptr = rule->prev;
> > +		free(rule);
> > +		found_flg = 1;
> > +
> > +		sprintf(mes, "Flow rule #%d destroyed", rule_id);
> > +		make_response(response, "success", mes, NULL);
> > +		break;
> > +	}
> > +
> > +	/* Rule_id not found */
> > +	if (found_flg == 0) {
> > +		sprintf(mes, "Flow rule #%d not found", rule_id);
> > +		make_response(response, "error", mes, NULL);
> > +	}
> > +}
> > +
> > +/* Delete all globally saved flow rules */
> > +static void
> > +exec_flow_flush(int port_id, char *response)
> > +{
> > +	int ret;
> > +	char mes[64];
> > +	struct flow_rule *rule;
> > +	struct rte_flow_error error;
> > +
> > +	memset(&error, 0, sizeof(error));
> > +
> > +	ret = is_portid_used(port_id);
> > +	if (ret != 0) {
> > +		sprintf(mes, "Invalid port %d", port_id);
> > +		make_response(response, "error", mes, NULL);
> > +		return;
> > +	}
> > +
> > +	ret = rte_flow_flush(port_id, &error);
> > +	if (ret != 0)
> > +		make_error_response(response, "Flow destroy error",
> > +			error, NULL);
> > +	else
> > +		make_response(response, "success", "Flow rule all destroyed",
> > +			NULL);
> > +
> > +	/*
> > +	 * Even if a failure occurs, flow handle is invalidated,
> > +	 * so delete flow_list.
> > +	 */
> > +
> > +	while (port_list[port_id].flow_list != NULL) {
> > +		rule = port_list[port_id].flow_list->prev;
> > +		free(port_list[port_id].flow_list);
> > +		port_list[port_id].flow_list = rule;
> > +	}
> > +}
> > +
> > +static void
> > +exec_flow(struct flow_args *input, char *response)
> > +{
> > +	switch(input->command) {
> > +		case VALIDATE:
> > +			exec_flow_validate(input->port_id,
> > +				&input->args.rule.attr,
> > +				input->args.rule.pattern,
> > +				input->args.rule.actions,
> > +				response);
> > +			break;
> > +		case CREATE:
> > +			exec_flow_create(input->port_id,
> > +				&input->args.rule.attr,
> > +				input->args.rule.pattern,
> > +				input->args.rule.actions,
> > +				response);
> > +			break;
> > +		case DESTROY:
> > +			exec_flow_destroy(input->port_id,
> > +				input->args.destroy.rule_id,
> > +				response);
> > +			break;
> > +		case FLUSH:
> > +			exec_flow_flush(input->port_id, response);
> > +			break;
> > +	}
> > +
> > +	/* Argument data is no longer needed and freed */
> > +	free_flow_args(input);
> > +
> > +	return;
> > +}
> > +
> > +int
> > +parse_flow(char *token_list[], char *response)
> > +{
> > +	int ret = 0;
> > +	struct flow_args input = { 0 };
> > +
> > +	if (token_list[1] == NULL) {
> > +		ret = -1;
> > +	} else if (!strcmp(token_list[1], "validate")) {
> > +		input.command = VALIDATE;
> > +		ret = parse_flow_rule(token_list, &input);
> > +
> > +	} else if (!strcmp(token_list[1], "create")) {
> > +		input.command = CREATE;
> > +		ret = parse_flow_rule(token_list, &input);
> > +
> > +	} else if (!strcmp(token_list[1], "destroy")) {
> > +		ret = parse_flow_destroy(token_list, &input);
> > +
> > +	} else {
> > +		ret = -1;
> > +	}
> > +
> > +	if (ret != 0) {
> > +		free_flow_args(&input);
> > +		make_response(response, "error",
> > +			"Flow command invalid argument", NULL);
> > +		return 0;
> > +	}
> > +
> > +	exec_flow(&input, response);
> > +
> > +	return 0;
> > +}
> > +
> > +static int
> > +append_flow_pattern_json(const struct rte_flow_item *pattern, int buf_size,
> > +	char *pattern_str)
> > +{
> > +	uint32_t i, j;
> > +	uint32_t nof_elems = 3;
> > +	int ret = 0;
> > +	char *tmp_str;
> > +	const char element_str[][5] = { "spec", "last", "mask" };
> > +	const struct rte_flow_item *ptn = pattern;
> > +	struct flow_item_ops *ops;
> > +	const void *tmp_ptr[nof_elems];
> > +
> > +	tmp_str = malloc(buf_size);
> > +	if (tmp_str == NULL) {
> > +		RTE_LOG(ERR, SPP_FLOW,
> > +			"Memory allocation failure(%s:%d)\n",
> > +			__func__, __LINE__);
> > +		return -1;
> > +	}
> > +
> > +	while (ptn->type != RTE_FLOW_ITEM_TYPE_END) {
> > +		memset(tmp_str, 0, buf_size);
> > +
> > +		tmp_ptr[0] = ptn->spec;
> > +		tmp_ptr[1] = ptn->last;
> > +		tmp_ptr[2] = ptn->mask;
> > +
> > +		for (i = 0; i < RTE_DIM(flow_item_ops_list); i++) {
> > +			ops = &flow_item_ops_list[i];
> > +			if (ptn->type != ops->type)
> > +				continue;
> > +
> > +			snprintf(tmp_str, buf_size,
> > +				"{\"type\":\"%s\",",
> > +				ops->str_type);
> > +
> > +			for (j = 0; j < nof_elems; j++) {
> > +				snprintf(tmp_str + strlen(tmp_str), buf_size,
> > +					"\"%s\":",
> > +					element_str[j]);
> > +
> > +				if (tmp_ptr[j] != NULL)
> > +					ret = ops->status(tmp_ptr[j],
> > +						buf_size - (int)strlen(tmp_str),
> > +						tmp_str + strlen(tmp_str));
> > +				else
> > +					snprintf(tmp_str + strlen(tmp_str),
> > +						buf_size,
> > +						"null");
> > +
> > +				if (ret != 0)
> > +					break;
> > +
> > +				if (j < nof_elems - 1)
> > +					tmp_str[strlen(tmp_str)] = ',';
> > +			}
> > +
> > +			tmp_str[strlen(tmp_str)] = '}';
> > +
> > +			break;
> > +		}
> > +
> > +		if (ret != 0)
> > +			break;
> > +
> > +		if ((int)strlen(pattern_str) + (int)strlen(tmp_str)
> > +			> buf_size - 1) {
> > +			ret = -1;
> > +			break;
> > +		}
> > +		strncat(pattern_str, tmp_str, strlen(tmp_str));
> > +
> > +		/*
> > +		 * If there is the following pattern, add ',' to
> > +		 * pattern_str
> > +		 */
> > +		ptn++;
> > +		if (ptn->type != RTE_FLOW_ITEM_TYPE_END) {
> > +			if ((int)strlen(pattern_str) + 1 > buf_size - 1) {
> > +				ret = -1;
> > +				break;
> > +			}
> > +			pattern_str[strlen(pattern_str)] = ',';
> > +		}
> > +	}
> > +
> > +	if (tmp_str != NULL)
> > +		free(tmp_str);
> > +
> > +	return ret;
> > +}
> > +
> > +static int
> > +append_flow_action_json(const struct rte_flow_action *actions, int buf_size,
> > +	char *actions_str)
> > +{
> > +	uint32_t i;
> > +	int ret = 0;
> > +	char *tmp_str;
> > +	const struct rte_flow_action *act = actions;
> > +	struct flow_action_ops *ops;
> > +
> > +	tmp_str = malloc(buf_size);
> > +	if (tmp_str == NULL) {
> > +		RTE_LOG(ERR, SPP_FLOW,
> > +			"Memory allocation failure(%s:%d)\n",
> > +			__func__, __LINE__);
> > +		return -1;
> > +	}
> > +
> > +	while (act->type != RTE_FLOW_ACTION_TYPE_END) {
> > +		memset(tmp_str, 0, buf_size);
> > +
> > +		for (i = 0; i < RTE_DIM(flow_action_ops_list); i++) {
> > +			ops = &flow_action_ops_list[i];
> > +			if (act->type != ops->type)
> > +				continue;
> > +
> > +			snprintf(tmp_str, buf_size,
> > +				"{\"type\":\"%s\",\"conf\":",
> > +				ops->str_type);
> > +
> > +			ret = ops->status(act->conf,
> > +				buf_size - (int)strlen(tmp_str),
> > +				tmp_str + strlen(tmp_str));
> > +			tmp_str[strlen(tmp_str)] = '}';
> > +			break;
> > +		}
> > +
> > +		if (ret != 0)
> > +			break;
> > +
> > +		if ((int)strlen(actions_str) + (int)strlen(tmp_str)
> > +			> buf_size - 1) {
> > +			ret = -1;
> > +			break;
> > +		}
> > +		strncat(actions_str, tmp_str, strlen(tmp_str));
> > +
> > +		/*
> > +		 * If there is the following pattern, add ',' to
> > +		 * actions_str
> > +		 */
> > +		act++;
> > +		if (act->type != RTE_FLOW_ACTION_TYPE_END) {
> > +			if ((int)strlen(actions_str) + 1 > buf_size - 1) {
> > +				ret = -1;
> > +				break;
> > +			}
> > +			actions_str[strlen(actions_str)] = ',';
> > +		}
> > +	}
> > +
> > +	if (tmp_str != NULL)
> > +		free(tmp_str);
> > +
> > +	return ret;
> > +}
> > +
> > +static int
> > +append_flow_rule_json(struct flow_rule *flow, int buf_size, char *flow_str)
> > +{
> > +	int ret = 0;
> > +	struct rte_flow_conv_rule rule;
> > +	char *tmp_str, *attr_str, *pattern_str, *actions_str;
> > +
> > +	while (1) {
> > +		tmp_str = malloc(buf_size);
> > +		attr_str = malloc(buf_size);
> > +		pattern_str = malloc(buf_size);
> > +		actions_str = malloc(buf_size);
> > +		if (tmp_str == NULL || attr_str == NULL
> > +			|| pattern_str == NULL || actions_str == NULL) {
> > +			RTE_LOG(ERR, SPP_FLOW,
> > +				"Memory allocation failure(%s:%d)\n",
> > +				__func__, __LINE__);
> > +			ret = -1;
> > +			break;
> > +		}
> > +		memset(tmp_str, 0, buf_size);
> > +		memset(attr_str, 0, buf_size);
> > +		memset(pattern_str, 0, buf_size);
> > +		memset(actions_str, 0, buf_size);
> > +
> > +		rule = flow->rule;
> > +
> > +		ret = append_flow_attr_json(rule.attr_ro, buf_size, attr_str);
> > +		if (ret != 0)
> > +			break;
> > +
> > +		ret = append_flow_pattern_json(rule.pattern_ro, buf_size,
> > +			pattern_str);
> > +		if (ret != 0)
> > +			break;
> > +
> > +		ret = append_flow_action_json(rule.actions_ro, buf_size,
> > +			actions_str);
> > +		if (ret != 0)
> > +			break;
> > +
> > +		snprintf(tmp_str, buf_size,
> > +			"{\"rule_id\":%d,"
> > +			"\"attr\":%s,"
> > +			"\"patterns\":[%s],"
> > +			"\"actions\":[%s]}",
> > +			flow->rule_id, attr_str, pattern_str, actions_str);
> > +
> > +		if ((int)strlen(tmp_str) > buf_size - 1) {
> > +			ret = -1;
> > +			break;
> > +		}
> > +
> > +		snprintf(flow_str, buf_size, "%s", tmp_str);
> > +		break;
> > +	}
> > +
> > +	if (tmp_str != NULL)
> > +		free(tmp_str);
> > +	if (attr_str != NULL)
> > +		free(attr_str);
> > +	if (pattern_str != NULL)
> > +		free(pattern_str);
> > +	if (actions_str != NULL)
> > +		free(actions_str);
> > +
> > +	return ret;
> > +}
> > +
> > +int
> > +append_flow_json(int port_id, int buf_size, char *output)
> > +{
> > +	int ret = 0;
> > +	int str_size = 0;
> > +	char *flow_str, *tmp_str;
> > +	struct flow_rule *flow;
> > +
> > +	flow_str = malloc(buf_size);
> > +	tmp_str = malloc(buf_size);
> > +	if (flow_str == NULL || tmp_str == NULL) {
> > +		RTE_LOG(ERR, SPP_FLOW,
> > +			"Memory allocation failure(%s:%d)\n",
> > +			__func__, __LINE__);
> > +		return -1;
> > +	}
> > +
> > +	flow = port_list[port_id].flow_list;
> > +
> > +	while (flow != NULL) {
> > +		memset(flow_str, 0, buf_size);
> > +
> > +		ret = append_flow_rule_json(flow, buf_size, flow_str);
> > +		if (ret != 0)
> > +			break;
> > +
> > +		if (str_size == 0) {
> > +			snprintf(output, buf_size, "%s", flow_str);
> > +			str_size += (int)strlen(flow_str);
> > +
> > +		} else {
> > +			str_size += ((int)strlen(flow_str) + 1);
> > +			if (str_size > buf_size - 1) {
> > +				ret = -1;
> > +				break;
> > +			}
> > +
> > +			/*
> > +			 * Since flow_list is in descending order,
> > +			 * concatenate the strings in front.
> > +			 */
> > +			memset(tmp_str, 0, buf_size);
> > +			strncpy(tmp_str, output, buf_size);
> > +			memset(output, 0, buf_size);
> > +
> > +			snprintf(output, buf_size, "%s,%s",
> > +				flow_str, tmp_str);
> > +		}
> > +
> > +		flow = flow->prev;
> > +	}
> > +
> > +	if (ret == 0) {
> > +		if ((int)strlen("[]") + (int)strlen(flow_str)
> > +			> buf_size - 1)
> > +			ret = -1;
> > +		else {
> > +			memset(tmp_str, 0, buf_size);
> > +			strncpy(tmp_str, output, buf_size);
> > +			memset(output, 0, buf_size);
> > +
> > +			snprintf(output, buf_size, "[%s]", tmp_str);
> > +		}
> > +	}
> > +
> > +	if (ret != 0)
> > +		RTE_LOG(ERR, SPP_FLOW,
> > +			"Cannot send all of flow stats(%s:%d)\n",
> > +			__func__, __LINE__);
> > +
> > +	if (flow_str != NULL)
> > +		free(flow_str);
> > +	if (tmp_str != NULL)
> > +		free(tmp_str);
> > +
> > +	return ret;
> > +}
> > diff --git a/src/primary/flow/flow.h b/src/primary/flow/flow.h
> > new file mode 100644
> > index 0000000..ecd4eb3
> > --- /dev/null
> > +++ b/src/primary/flow/flow.h
> > @@ -0,0 +1,94 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
> > + */
> > +
> > +#ifndef _PRIMARY_FLOW_H_
> > +#define _PRIMARY_FLOW_H_
> > +
> > +#include <rte_log.h>
> > +
> > +#define RTE_LOGTYPE_SPP_FLOW RTE_LOGTYPE_USER1
> > +
> > +enum flow_command {
> > +	VALIDATE = 0,
> > +	CREATE,
> > +	DESTROY,
> > +	FLUSH
> > +};
> > +
> > +/* Parser result of flow command arguments */
> > +struct flow_args {
> > +	enum flow_command command;
> > +	int port_id;
> > +	union {
> > +		struct {
> > +			struct rte_flow_attr attr;
> > +			struct rte_flow_item *pattern;
> > +			struct rte_flow_action *actions;
> > +		} rule; /* validate or create arguments. */
> > +		struct {
> > +			uint32_t rule_id;
> > +		} destroy; /* destroy arguments. */
> > +	} args;
> > +};
> > +
> > +/* Descriptor for a single flow. */
> > +struct flow_rule {
> > +	/* Flow rule ID */
> > +	uint32_t rule_id;
> > +
> > +	/* Previous flow in list. */
> > +	struct flow_rule *prev;
> > +
> > +	/* Opaque flow object returned by PMD. */
> > +	struct rte_flow *flow_handle;
> > +
> > +	/* Saved flow rule description. */
> > +	struct rte_flow_conv_rule rule;
> > +};
> > +
> > +/* Flow rule list of the port */
> > +struct port_flow {
> > +	/* Associated flows */
> > +	struct flow_rule *flow_list;
> > +};
> > +
> > +/* Detail parse operation for a specific item or action */
> > +struct flow_detail_ops {
> > +	const char *token;
> > +	const size_t offset;
> > +	const size_t size;
> > +	int flg_value;
> > +	int (*parse_detail)(char *str, void *output);
> > +};
> > +
> > +/* Operation for each item type */
> > +struct flow_item_ops {
> > +	const char *str_type;
> > +	enum rte_flow_item_type type;
> > +	size_t size;
> > +	int (*parse)(char *token_list[], int *index,
> > +		struct rte_flow_item *pattern,
> > +		struct flow_item_ops *ops);
> > +	struct flow_detail_ops *detail_list;
> > +	int (*status)(const void *element,
> > +		int buf_size, char *pattern_str);
> > +};
> > +
> > +/* Operation for each action type */
> > +struct flow_action_ops {
> > +	const char *str_type;
> > +	enum rte_flow_action_type type;
> > +	size_t size;
> > +	int (*parse)(char *token_list[], int *index,
> > +		struct rte_flow_action *action,
> > +		struct flow_action_ops *ops);
> > +	struct flow_detail_ops *detail_list;
> > +	int (*status)(const void *conf,
> > +		int buf_size, char *action_str);
> > +};
> > +
> > +int parse_flow(char *token_list[], char *response);
> > +int append_flow_json(int port_id, int buf_size, char *output);
> > +
> > +#endif
> > 




^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [spp] [PATCH v2 00/17] Adding Hardware offload capability
  2020-02-19 11:49 ` [spp] [PATCH v2 00/17] Adding Hardware offload capability x-fn-spp-ml
@ 2020-02-21  8:17   ` Yasufumi Ogawa
  2020-02-25  5:49     ` [spp] (x-fn-spp-ml 177) " Hideyuki Yamashita
  0 siblings, 1 reply; 40+ messages in thread
From: Yasufumi Ogawa @ 2020-02-21  8:17 UTC (permalink / raw)
  To: x-fn-spp-ml; +Cc: spp, ferruh.yigit

> From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
> 
> This patch set provides hardware offload capability
> for spp.
> Note that related documents will be provided soon.
Applied, thanks.

> 
> Hideyuki Yamashita (17):
>    shared: add support of multi-queue
>    spp_vf: add support of multi-queue
>    spp_mirror: add support of multi-queue
>    spp_pcap: add support of multi-queue
>    spp_primary: add support of multi-queue
>    spp_primary: add support of rte_flow
>    spp_primary: add common function of rte_flow
>    spp_primary: add attribute of rte_flow
>    spp_primary: add patterns of rte_flow
>    spp_primary: add actions of rte_flow
>    bin: add parameter for hardrare offload
>    cli: add support of hardware offload
>    cli: add support of rte_flow in vf
>    cli: add support of rte_flow in mirror
>    cli: add support of rte_flow in nfv
>    spp-ctl: add APIs for flow rules
>    spp_nfv: add support of multi-queue
> 
>   bin/sample/config.sh                          |   15 +
>   bin/spp_pri.sh                                |   26 +-
>   bin/start.sh                                  |    2 +
>   src/cli/commands/mirror.py                    |  175 ++-
>   src/cli/commands/nfv.py                       |  245 ++--
>   src/cli/commands/pri.py                       |   52 +-
>   src/cli/commands/pri_flow.py                  |  936 +++++++++++++++
>   src/cli/commands/pri_flow_compl_action.py     |  115 ++
>   src/cli/commands/pri_flow_compl_pattern.py    |   84 ++
>   src/cli/commands/vf.py                        |  460 +++++---
>   src/mirror/mir_cmd_runner.c                   |    3 +-
>   src/mirror/spp_mirror.c                       |   12 +-
>   src/nfv/commands.h                            |   71 +-
>   src/nfv/main.c                                |   18 +-
>   src/nfv/nfv_status.c                          |  315 +++--
>   src/nfv/nfv_status.h                          |   12 +-
>   src/pcap/cmd_runner.c                         |    3 +-
>   src/pcap/cmd_utils.c                          |   73 +-
>   src/pcap/cmd_utils.h                          |   19 +-
>   src/pcap/spp_pcap.c                           |  102 +-
>   src/primary/Makefile                          |   10 +
>   src/primary/args.c                            |  138 ++-
>   src/primary/args.h                            |    3 +
>   src/primary/flow/action/jump.c                |   42 +
>   src/primary/flow/action/jump.h                |   12 +
>   src/primary/flow/action/of_push_vlan.c        |   44 +
>   src/primary/flow/action/of_push_vlan.h        |   13 +
>   src/primary/flow/action/of_set_vlan_pcp.c     |   44 +
>   src/primary/flow/action/of_set_vlan_pcp.h     |   13 +
>   src/primary/flow/action/of_set_vlan_vid.c     |   44 +
>   src/primary/flow/action/of_set_vlan_vid.h     |   13 +
>   src/primary/flow/action/queue.c               |   42 +
>   src/primary/flow/action/queue.h               |   13 +
>   src/primary/flow/attr.c                       |  105 ++
>   src/primary/flow/attr.h                       |   13 +
>   src/primary/flow/common.c                     |  646 ++++++++++
>   src/primary/flow/common.h                     |   53 +
>   src/primary/flow/flow.c                       | 1045 +++++++++++++++++
>   src/primary/flow/flow.h                       |   94 ++
>   src/primary/flow/pattern/eth.c                |   63 +
>   src/primary/flow/pattern/eth.h                |   13 +
>   src/primary/flow/pattern/vlan.c               |   71 ++
>   src/primary/flow/pattern/vlan.h               |   13 +
>   src/primary/init.c                            |    8 +-
>   src/primary/init.h                            |    3 +-
>   src/primary/main.c                            |   88 +-
>   src/shared/basic_forwarder.c                  |   62 +-
>   src/shared/basic_forwarder.h                  |    2 +-
>   src/shared/common.h                           |   30 +-
>   src/shared/port_manager.c                     |  180 ++-
>   src/shared/port_manager.h                     |   13 +-
>   .../secondary/spp_worker_th/cmd_parser.c      |  137 ++-
>   .../spp_worker_th/cmd_res_formatter.c         |   63 +-
>   .../secondary/spp_worker_th/cmd_utils.c       |  111 +-
>   .../secondary/spp_worker_th/cmd_utils.h       |   19 +-
>   .../secondary/spp_worker_th/data_types.h      |   18 +-
>   .../secondary/spp_worker_th/port_capability.c |    8 +-
>   .../secondary/spp_worker_th/port_capability.h |    4 +-
>   src/shared/secondary/spp_worker_th/vf_deps.h  |   15 +-
>   src/shared/secondary/utils.c                  |   28 +-
>   src/shared/secondary/utils.h                  |    3 +-
>   src/spp-ctl/spp_proc.py                       |    4 +
>   src/spp-ctl/spp_webapi.py                     |  200 +++-
>   src/vf/classifier.c                           |   40 +-
>   src/vf/forwarder.c                            |   12 +-
>   src/vf/vf_cmd_runner.c                        |   32 +-
>   66 files changed, 5621 insertions(+), 744 deletions(-)
>   create mode 100644 src/cli/commands/pri_flow.py
>   create mode 100644 src/cli/commands/pri_flow_compl_action.py
>   create mode 100644 src/cli/commands/pri_flow_compl_pattern.py
>   create mode 100644 src/primary/flow/action/jump.c
>   create mode 100644 src/primary/flow/action/jump.h
>   create mode 100644 src/primary/flow/action/of_push_vlan.c
>   create mode 100644 src/primary/flow/action/of_push_vlan.h
>   create mode 100644 src/primary/flow/action/of_set_vlan_pcp.c
>   create mode 100644 src/primary/flow/action/of_set_vlan_pcp.h
>   create mode 100644 src/primary/flow/action/of_set_vlan_vid.c
>   create mode 100644 src/primary/flow/action/of_set_vlan_vid.h
>   create mode 100644 src/primary/flow/action/queue.c
>   create mode 100644 src/primary/flow/action/queue.h
>   create mode 100644 src/primary/flow/attr.c
>   create mode 100644 src/primary/flow/attr.h
>   create mode 100644 src/primary/flow/common.c
>   create mode 100644 src/primary/flow/common.h
>   create mode 100644 src/primary/flow/flow.c
>   create mode 100644 src/primary/flow/flow.h
>   create mode 100644 src/primary/flow/pattern/eth.c
>   create mode 100644 src/primary/flow/pattern/eth.h
>   create mode 100644 src/primary/flow/pattern/vlan.c
>   create mode 100644 src/primary/flow/pattern/vlan.h
> 

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [spp] (x-fn-spp-ml 177) Re: [PATCH v2 00/17] Adding Hardware offload capability
  2020-02-21  8:17   ` Yasufumi Ogawa
@ 2020-02-25  5:49     ` Hideyuki Yamashita
  0 siblings, 0 replies; 40+ messages in thread
From: Hideyuki Yamashita @ 2020-02-25  5:49 UTC (permalink / raw)
  To: Yasufumi Ogawa; +Cc: x-fn-spp-ml, spp, ferruh.yigit

Yasufumi-san

I will try to send v3 patch set those replaces 
Signed-off-by with appropriate form.

I will send those right after this email.

Please use those instead of v2 patch set.

BR,
Hideyuki Yamashtia
NTT TechnoCross

> > From: Hideyuki Yamashita <yamashita.hideyuki@ntt-tx.co.jp>
> >
> > This patch set provides hardware offload capability
> > for spp.
> > Note that related documents will be provided soon.
> Applied, thanks.
> 
> >
> > Hideyuki Yamashita (17):
> >    shared: add support of multi-queue
> >    spp_vf: add support of multi-queue
> >    spp_mirror: add support of multi-queue
> >    spp_pcap: add support of multi-queue
> >    spp_primary: add support of multi-queue
> >    spp_primary: add support of rte_flow
> >    spp_primary: add common function of rte_flow
> >    spp_primary: add attribute of rte_flow
> >    spp_primary: add patterns of rte_flow
> >    spp_primary: add actions of rte_flow
> >    bin: add parameter for hardrare offload
> >    cli: add support of hardware offload
> >    cli: add support of rte_flow in vf
> >    cli: add support of rte_flow in mirror
> >    cli: add support of rte_flow in nfv
> >    spp-ctl: add APIs for flow rules
> >    spp_nfv: add support of multi-queue
> >
> >   bin/sample/config.sh                          |   15 +
> >   bin/spp_pri.sh                                |   26 +-
> >   bin/start.sh                                  |    2 +
> >   src/cli/commands/mirror.py                    |  175 ++-
> >   src/cli/commands/nfv.py                       |  245 ++--
> >   src/cli/commands/pri.py                       |   52 +-
> >   src/cli/commands/pri_flow.py                  |  936 +++++++++++++++
> >   src/cli/commands/pri_flow_compl_action.py     |  115 ++
> >   src/cli/commands/pri_flow_compl_pattern.py    |   84 ++
> >   src/cli/commands/vf.py                        |  460 +++++---
> >   src/mirror/mir_cmd_runner.c                   |    3 +-
> >   src/mirror/spp_mirror.c                       |   12 +-
> >   src/nfv/commands.h                            |   71 +-
> >   src/nfv/main.c                                |   18 +-
> >   src/nfv/nfv_status.c                          |  315 +++--
> >   src/nfv/nfv_status.h                          |   12 +-
> >   src/pcap/cmd_runner.c                         |    3 +-
> >   src/pcap/cmd_utils.c                          |   73 +-
> >   src/pcap/cmd_utils.h                          |   19 +-
> >   src/pcap/spp_pcap.c                           |  102 +-
> >   src/primary/Makefile                          |   10 +
> >   src/primary/args.c                            |  138 ++-
> >   src/primary/args.h                            |    3 +
> >   src/primary/flow/action/jump.c                |   42 +
> >   src/primary/flow/action/jump.h                |   12 +
> >   src/primary/flow/action/of_push_vlan.c        |   44 +
> >   src/primary/flow/action/of_push_vlan.h        |   13 +
> >   src/primary/flow/action/of_set_vlan_pcp.c     |   44 +
> >   src/primary/flow/action/of_set_vlan_pcp.h     |   13 +
> >   src/primary/flow/action/of_set_vlan_vid.c     |   44 +
> >   src/primary/flow/action/of_set_vlan_vid.h     |   13 +
> >   src/primary/flow/action/queue.c               |   42 +
> >   src/primary/flow/action/queue.h               |   13 +
> >   src/primary/flow/attr.c                       |  105 ++
> >   src/primary/flow/attr.h                       |   13 +
> >   src/primary/flow/common.c                     |  646 ++++++++++
> >   src/primary/flow/common.h                     |   53 +
> >   src/primary/flow/flow.c                       | 1045 +++++++++++++++++
> >   src/primary/flow/flow.h                       |   94 ++
> >   src/primary/flow/pattern/eth.c                |   63 +
> >   src/primary/flow/pattern/eth.h                |   13 +
> >   src/primary/flow/pattern/vlan.c               |   71 ++
> >   src/primary/flow/pattern/vlan.h               |   13 +
> >   src/primary/init.c                            |    8 +-
> >   src/primary/init.h                            |    3 +-
> >   src/primary/main.c                            |   88 +-
> >   src/shared/basic_forwarder.c                  |   62 +-
> >   src/shared/basic_forwarder.h                  |    2 +-
> >   src/shared/common.h                           |   30 +-
> >   src/shared/port_manager.c                     |  180 ++-
> >   src/shared/port_manager.h                     |   13 +-
> >   .../secondary/spp_worker_th/cmd_parser.c      |  137 ++-
> >   .../spp_worker_th/cmd_res_formatter.c         |   63 +-
> >   .../secondary/spp_worker_th/cmd_utils.c       |  111 +-
> >   .../secondary/spp_worker_th/cmd_utils.h       |   19 +-
> >   .../secondary/spp_worker_th/data_types.h      |   18 +-
> >   .../secondary/spp_worker_th/port_capability.c |    8 +-
> >   .../secondary/spp_worker_th/port_capability.h |    4 +-
> >   src/shared/secondary/spp_worker_th/vf_deps.h  |   15 +-
> >   src/shared/secondary/utils.c                  |   28 +-
> >   src/shared/secondary/utils.h                  |    3 +-
> >   src/spp-ctl/spp_proc.py                       |    4 +
> >   src/spp-ctl/spp_webapi.py                     |  200 +++-
> >   src/vf/classifier.c                           |   40 +-
> >   src/vf/forwarder.c                            |   12 +-
> >   src/vf/vf_cmd_runner.c                        |   32 +-
> >   66 files changed, 5621 insertions(+), 744 deletions(-)
> >   create mode 100644 src/cli/commands/pri_flow.py
> >   create mode 100644 src/cli/commands/pri_flow_compl_action.py
> >   create mode 100644 src/cli/commands/pri_flow_compl_pattern.py
> >   create mode 100644 src/primary/flow/action/jump.c
> >   create mode 100644 src/primary/flow/action/jump.h
> >   create mode 100644 src/primary/flow/action/of_push_vlan.c
> >   create mode 100644 src/primary/flow/action/of_push_vlan.h
> >   create mode 100644 src/primary/flow/action/of_set_vlan_pcp.c
> >   create mode 100644 src/primary/flow/action/of_set_vlan_pcp.h
> >   create mode 100644 src/primary/flow/action/of_set_vlan_vid.c
> >   create mode 100644 src/primary/flow/action/of_set_vlan_vid.h
> >   create mode 100644 src/primary/flow/action/queue.c
> >   create mode 100644 src/primary/flow/action/queue.h
> >   create mode 100644 src/primary/flow/attr.c
> >   create mode 100644 src/primary/flow/attr.h
> >   create mode 100644 src/primary/flow/common.c
> >   create mode 100644 src/primary/flow/common.h
> >   create mode 100644 src/primary/flow/flow.c
> >   create mode 100644 src/primary/flow/flow.h
> >   create mode 100644 src/primary/flow/pattern/eth.c
> >   create mode 100644 src/primary/flow/pattern/eth.h
> >   create mode 100644 src/primary/flow/pattern/vlan.c
> >   create mode 100644 src/primary/flow/pattern/vlan.h
> > 



^ permalink raw reply	[flat|nested] 40+ messages in thread

end of thread, other threads:[~2020-02-25  5:50 UTC | newest]

Thread overview: 40+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-18  6:37 [spp] [PATCH 00/17] Adding Hardware offload capability x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 01/17] shared: add support of multi-queue x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 02/17] spp_vf: " x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 03/17] spp_mirror: " x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 04/17] spp_pcap: " x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 05/17] spp_primary: " x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 06/17] spp_primary: add support of rte_flow x-fn-spp-ml
2020-02-19  2:24   ` Yasufumi Ogawa
2020-02-19 11:57     ` [spp] (x-fn-spp-ml 118) " Hideyuki Yamashita
2020-02-18  6:37 ` [spp] [PATCH 07/17] spp_primary: add common function " x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 08/17] spp_primary: add attribute " x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 09/17] spp_primary: add patterns " x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 10/17] spp_primary: add actions " x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 11/17] bin: add parameter for hardrare offload x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 12/17] cli: add support of hardware offload x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 13/17] cli: add support of rte_flow in vf x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 14/17] cli: add support of rte_flow in mirror x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 15/17] cli: add support of rte_flow in nfv x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 16/17] spp-ctl: add APIs for flow rules x-fn-spp-ml
2020-02-18  6:37 ` [spp] [PATCH 17/17] spp_nfv: add support of multi-queue x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 00/17] Adding Hardware offload capability x-fn-spp-ml
2020-02-21  8:17   ` Yasufumi Ogawa
2020-02-25  5:49     ` [spp] (x-fn-spp-ml 177) " Hideyuki Yamashita
2020-02-19 11:49 ` [spp] [PATCH v2 01/17] shared: add support of multi-queue x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 02/17] spp_vf: " x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 03/17] spp_mirror: " x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 04/17] spp_pcap: " x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 05/17] spp_primary: " x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 06/17] spp_primary: add support of rte_flow x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 07/17] spp_primary: add common function " x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 08/17] spp_primary: add attribute " x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 09/17] spp_primary: add patterns " x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 10/17] spp_primary: add actions " x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 11/17] bin: add parameter for hardrare offload x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 12/17] cli: add support of hardware offload x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 13/17] cli: add support of rte_flow in vf x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 14/17] cli: add support of rte_flow in mirror x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 15/17] cli: add support of rte_flow in nfv x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 16/17] spp-ctl: add APIs for flow rules x-fn-spp-ml
2020-02-19 11:49 ` [spp] [PATCH v2 17/17] spp_nfv: add support of multi-queue x-fn-spp-ml

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).