From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24D27A0524 for ; Tue, 25 Feb 2020 06:56:45 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0E9BC1BFAC; Tue, 25 Feb 2020 06:56:45 +0100 (CET) Received: from dish-sg.nttdocomo.co.jp (dish-sg.nttdocomo.co.jp [202.19.227.74]) by dpdk.org (Postfix) with ESMTP id 2F3102C4F for ; Tue, 25 Feb 2020 06:56:42 +0100 (CET) X-dD-Source: Outbound Received: from zssg-mailmd105.ddreams.local (zssg-mailmd900.ddreams.local [10.160.172.63]) by zssg-mailou101.ddreams.local (Postfix) with ESMTP id 6678E12011B; Tue, 25 Feb 2020 14:56:41 +0900 (JST) Received: from zssg-mailmf101.ddreams.local (zssg-mailmf900.ddreams.local [10.160.172.84]) by zssg-mailmd105.ddreams.local (dDREAMS) with ESMTP id <0Q68012O0TUH1690@dDREAMS>; Tue, 25 Feb 2020 14:56:41 +0900 (JST) Received: from zssg-mailmf101.ddreams.local (unknown [127.0.0.1]) by zssg-mailmf101.ddreams.local (Postfix) with ESMTP id 41C597E6034; Tue, 25 Feb 2020 14:56:41 +0900 (JST) Received: from zssg-mailmf101.ddreams.local (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 401998E6060; Tue, 25 Feb 2020 14:56:41 +0900 (JST) Received: from localhost (unknown [127.0.0.1]) by IMSVA (Postfix) with SMTP id 34B918E6051; Tue, 25 Feb 2020 14:56:41 +0900 (JST) X-IMSS-HAND-OFF-DIRECTIVE: localhost:10026 Received: from zssg-mailmf101.ddreams.local (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3B0628E6056; Tue, 25 Feb 2020 14:56:40 +0900 (JST) Received: from davinci.ntt-tx.co.jp (unknown [10.160.183.139]) by zssg-mailmf101.ddreams.local (Postfix) with ESMTP; Tue, 25 Feb 2020 14:56:40 +0900 (JST) From: x-fn-spp-ml@ntt-tx.co.jp To: spp@dpdk.org, ferruh.yigit@intel.com, yasufum.o@gmail.com Date: Tue, 25 Feb 2020 14:56:23 +0900 Message-id: <20200225055639.31616-2-x-fn-spp-ml@ntt-tx.co.jp> X-Mailer: git-send-email 2.18.0 In-reply-to: <20200219112155.13964-1-yamashita.hideyuki@ntt-tx.co.jp> References: <20200219112155.13964-1-yamashita.hideyuki@ntt-tx.co.jp> X-TM-AS-GCONF: 00 Subject: [spp] [PATCH v3 01/17] shared: add support of multi-queue X-BeenThere: spp@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Soft Patch Panel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: spp-bounces@dpdk.org Sender: "spp" From: Hideyuki Yamashita To support multi-queue in SPP,changes like following should be introduced: - add queue number parameter in port command - parse newly added parameter - add queue number parameter in status response Under shared directory, common logic among secondary process exist and those should be affected. Signed-off-by: Hideyuki Yamashita Signed-off-by: Naoki Takada --- src/shared/basic_forwarder.c | 62 +++--- src/shared/basic_forwarder.h | 2 +- src/shared/common.h | 30 ++- src/shared/port_manager.c | 180 ++++++++++++++---- src/shared/port_manager.h | 13 +- .../secondary/spp_worker_th/cmd_parser.c | 137 ++++++++++--- .../spp_worker_th/cmd_res_formatter.c | 63 ++++-- .../secondary/spp_worker_th/cmd_utils.c | 111 ++++++++--- .../secondary/spp_worker_th/cmd_utils.h | 19 +- .../secondary/spp_worker_th/data_types.h | 18 +- .../secondary/spp_worker_th/port_capability.c | 8 +- .../secondary/spp_worker_th/port_capability.h | 4 +- src/shared/secondary/spp_worker_th/vf_deps.h | 15 +- src/shared/secondary/utils.c | 28 ++- src/shared/secondary/utils.h | 3 +- 15 files changed, 535 insertions(+), 158 deletions(-) diff --git a/src/shared/basic_forwarder.c b/src/shared/basic_forwarder.c index 7aefaaa..66f8643 100644 --- a/src/shared/basic_forwarder.c +++ b/src/shared/basic_forwarder.c @@ -5,6 +5,7 @@ #include #include "shared/common.h" #include "shared/basic_forwarder.h" +#include "shared/port_manager.h" void forward(void) @@ -14,42 +15,53 @@ forward(void) int in_port; int out_port; uint16_t buf; - int i; + int i, j; + uint16_t max_queue, in_queue, out_queue; /* Go through every possible port numbers*/ for (i = 0; i < RTE_MAX_ETHPORTS; i++) { - struct rte_mbuf *bufs[MAX_PKT_BURST]; - if (ports_fwd_array[i].in_port_id == PORT_RESET) - continue; + max_queue = get_port_max_queues(i); - if (ports_fwd_array[i].out_port_id == PORT_RESET) - continue; + /* Checks the number of rxq declared for the port */ + for (j = 0; j < max_queue; j++) { - /* if status active, i count is in port*/ - in_port = i; - out_port = ports_fwd_array[i].out_port_id; + struct rte_mbuf *bufs[MAX_PKT_BURST]; - /* Get burst of RX packets, from first port of pair. */ - /*first port rx, second port tx*/ - nb_rx = ports_fwd_array[in_port].rx_func(in_port, 0, bufs, - MAX_PKT_BURST); - if (unlikely(nb_rx == 0)) - continue; + if (ports_fwd_array[i][j].in_port_id == PORT_RESET) + continue; - port_map[in_port].stats->rx += nb_rx; + if (ports_fwd_array[i][j].out_port_id == PORT_RESET) + continue; - /* Send burst of TX packets, to second port of pair. */ - nb_tx = ports_fwd_array[out_port].tx_func(out_port, 0, bufs, - nb_rx); + /* if status active, i count is in port*/ + in_port = i; + in_queue = j; + out_port = ports_fwd_array[i][j].out_port_id; + out_queue = ports_fwd_array[i][j].out_queue_id; - port_map[out_port].stats->tx += nb_tx; + /* Get burst of RX packets, from first port of pair. */ + /*first port rx, second port tx*/ + nb_rx = ports_fwd_array[in_port][in_queue].rx_func( + in_port, in_queue, bufs, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) + continue; - /* Free any unsent packets. */ - if (unlikely(nb_tx < nb_rx)) { - port_map[out_port].stats->tx_drop += nb_rx - nb_tx; - for (buf = nb_tx; buf < nb_rx; buf++) - rte_pktmbuf_free(bufs[buf]); + port_map[in_port].stats->rx += nb_rx; + + /* Send burst of TX packets, to second port of pair. */ + nb_tx = ports_fwd_array[out_port][out_queue].tx_func( + out_port, out_queue, bufs, nb_rx); + + port_map[out_port].stats->tx += nb_tx; + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_rx)) { + port_map[out_port].stats->tx_drop += (nb_rx - + nb_tx); + for (buf = nb_tx; buf < nb_rx; buf++) + rte_pktmbuf_free(bufs[buf]); + } } } } diff --git a/src/shared/basic_forwarder.h b/src/shared/basic_forwarder.h index 2e8225d..915b5cc 100644 --- a/src/shared/basic_forwarder.h +++ b/src/shared/basic_forwarder.h @@ -8,7 +8,7 @@ #include "shared/common.h" struct port_map port_map[RTE_MAX_ETHPORTS]; -struct port ports_fwd_array[RTE_MAX_ETHPORTS]; +struct port ports_fwd_array[RTE_MAX_ETHPORTS][RTE_MAX_QUEUES_PER_PORT]; void forward(void); diff --git a/src/shared/common.h b/src/shared/common.h index b4af73c..d311e82 100644 --- a/src/shared/common.h +++ b/src/shared/common.h @@ -10,7 +10,23 @@ #include #include -#define MSG_SIZE 2048 /* socket buffer len */ +/* + * TODO(tx_h-yamashita): Remove this definition because it was used from + * spp_primary and spp_nfv and its value was 2048, but the value should + * be changed to 32KiB for containing status of spp_primary. + * For spp_nfv, 2048 is still enough. This buff len should be defined in + * each of spp_primary and spp_nfv as appropriate size. + */ +/* + * NOTE: The size of 32768(32Kbyte) is enough to set maximum size of JSON + * response string. This maximum size comes from primary`s `status` command + * response which includes `LCORE`, `PHY` and `RING` information. + * For supporting rte_flow API, `PHY` information may contain a lot of + * flow data which defined by user. Our design choice is that `PHY` + * information size would be 30Kbyte and `LCORE` size plus `PHY` size + * would be 2Kbyte. + */ +#define MSG_SIZE 32768 /* socket buffer max len */ #define SOCK_RESET -1 #define PORT_RESET UINT16_MAX @@ -70,11 +86,19 @@ struct stats { uint64_t tx_drop; } __rte_cache_aligned; +/* rx_queue and tx_queue set to port. */ +struct port_queue { + uint16_t rxq; + uint16_t txq; +}; + struct port_info { uint16_t num_ports; uint16_t id[RTE_MAX_ETHPORTS]; struct stats port_stats[RTE_MAX_ETHPORTS]; struct stats client_stats[MAX_CLIENT]; + /* num of queues per port */ + struct port_queue queue_info[RTE_MAX_ETHPORTS]; }; enum port_type { @@ -93,11 +117,15 @@ struct port_map { enum port_type port_type; struct stats *stats; struct stats default_stats; + /* num of queues per port */ + struct port_queue *queue_info; }; struct port { uint16_t in_port_id; + uint16_t in_queue_id; uint16_t out_port_id; + uint16_t out_queue_id; uint16_t (*rx_func)(uint16_t, uint16_t, struct rte_mbuf **, uint16_t); uint16_t (*tx_func)(uint16_t, uint16_t, struct rte_mbuf **, uint16_t); }; diff --git a/src/shared/port_manager.c b/src/shared/port_manager.c index 534af99..6beeca7 100644 --- a/src/shared/port_manager.c +++ b/src/shared/port_manager.c @@ -16,35 +16,46 @@ struct porttype_map portmap[] = { }; void -forward_array_init_one(unsigned int i) +forward_array_init_one(unsigned int i, unsigned int j) { - ports_fwd_array[i].in_port_id = PORT_RESET; - ports_fwd_array[i].out_port_id = PORT_RESET; + ports_fwd_array[i][j].in_port_id = PORT_RESET; + ports_fwd_array[i][j].in_queue_id = 0; + ports_fwd_array[i][j].out_port_id = PORT_RESET; + ports_fwd_array[i][j].out_queue_id = 0; } /* initialize forward array with default value */ void forward_array_init(void) { - unsigned int i; + unsigned int i, j; /* initialize port forward array*/ - for (i = 0; i < RTE_MAX_ETHPORTS; i++) - forward_array_init_one(i); + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + for (j = 0; j < RTE_MAX_QUEUES_PER_PORT; j++) + forward_array_init_one(i, j); + } } void forward_array_reset(void) { - unsigned int i; + unsigned int i, j; + uint16_t max_queue; /* initialize port forward array*/ for (i = 0; i < RTE_MAX_ETHPORTS; i++) { - if (ports_fwd_array[i].in_port_id != PORT_RESET) { - ports_fwd_array[i].out_port_id = PORT_RESET; - RTE_LOG(INFO, SHARED, "Port ID %d\n", i); - RTE_LOG(INFO, SHARED, "out_port_id %d\n", - ports_fwd_array[i].out_port_id); + + max_queue = get_port_max_queues(i); + + for (j = 0; j < max_queue; j++) { + if (ports_fwd_array[i][j].in_port_id != PORT_RESET) { + ports_fwd_array[i][j].out_port_id = PORT_RESET; + RTE_LOG(INFO, SHARED, "Port ID %d\n", i); + RTE_LOG(INFO, SHARED, "Queue ID %d\n", j); + RTE_LOG(INFO, SHARED, "out_port_id %d\n", + ports_fwd_array[i][j].out_port_id); + } } } } @@ -55,6 +66,7 @@ port_map_init_one(unsigned int i) port_map[i].id = PORT_RESET; port_map[i].port_type = UNDEF; port_map[i].stats = &port_map[i].default_stats; + port_map[i].queue_info = NULL; } void @@ -68,28 +80,46 @@ port_map_init(void) /* Return -1 as an error if given patch is invalid */ int -add_patch(uint16_t in_port, uint16_t out_port) +add_patch(uint16_t in_port, uint16_t in_queue, + uint16_t out_port, uint16_t out_queue) { - if (!is_valid_port(in_port) || !is_valid_port(out_port)) + if (!is_valid_port(in_port, in_queue) || + !is_valid_port(out_port, out_queue)) return -1; + if (!is_valid_port_rxq(in_port, in_queue) || + !is_valid_port_txq(out_port, out_queue)) + return 1; + /* Populate in port data */ - ports_fwd_array[in_port].in_port_id = in_port; - ports_fwd_array[in_port].rx_func = &rte_eth_rx_burst; - ports_fwd_array[in_port].tx_func = &rte_eth_tx_burst; - ports_fwd_array[in_port].out_port_id = out_port; + ports_fwd_array[in_port][in_queue].in_port_id = in_port; + ports_fwd_array[in_port][in_queue].in_queue_id = in_queue; + ports_fwd_array[in_port][in_queue].rx_func = &rte_eth_rx_burst; + ports_fwd_array[in_port][in_queue].tx_func = &rte_eth_tx_burst; + ports_fwd_array[in_port][in_queue].out_port_id = out_port; + ports_fwd_array[in_port][in_queue].out_queue_id = out_queue; /* Populate out port data */ - ports_fwd_array[out_port].in_port_id = out_port; - ports_fwd_array[out_port].rx_func = &rte_eth_rx_burst; - ports_fwd_array[out_port].tx_func = &rte_eth_tx_burst; + ports_fwd_array[out_port][out_queue].in_port_id = out_port; + ports_fwd_array[out_port][out_queue].in_queue_id = out_queue; + ports_fwd_array[out_port][out_queue].rx_func = &rte_eth_rx_burst; + ports_fwd_array[out_port][out_queue].tx_func = &rte_eth_tx_burst; - RTE_LOG(DEBUG, SHARED, "STATUS: in port %d in_port_id %d\n", in_port, - ports_fwd_array[in_port].in_port_id); - RTE_LOG(DEBUG, SHARED, "STATUS: in port %d patch out port id %d\n", - in_port, ports_fwd_array[in_port].out_port_id); - RTE_LOG(DEBUG, SHARED, "STATUS: outport %d in_port_id %d\n", out_port, - ports_fwd_array[out_port].in_port_id); + RTE_LOG(DEBUG, SHARED, "STATUS: in port %d in queue %d" + " in_port_id %d in_queue_id %d\n", + in_port, in_queue, + ports_fwd_array[in_port][in_queue].in_port_id, + ports_fwd_array[in_port][in_queue].in_queue_id); + RTE_LOG(DEBUG, SHARED, "STATUS: in port %d in queue %d" + " patch out_port_id %d out_queue_id %d\n", + in_port, in_queue, + ports_fwd_array[in_port][in_queue].out_port_id, + ports_fwd_array[in_port][in_queue].out_queue_id); + RTE_LOG(DEBUG, SHARED, "STATUS: out port %d out queue %d" + " in_port_id %d in_queue_id %d\n", + out_port, out_queue, + ports_fwd_array[out_port][out_queue].in_port_id, + ports_fwd_array[out_port][out_queue].in_queue_id); return 0; } @@ -119,30 +149,92 @@ find_port_id(int id, enum port_type type) /* Return 0 if invalid */ int -is_valid_port(uint16_t port_id) +is_valid_port(uint16_t port_id, uint16_t queue_id) { + uint16_t max_queue; + if (port_id > RTE_MAX_ETHPORTS) return 0; - return port_map[port_id].id != PORT_RESET; + if (port_map[port_id].id == PORT_RESET) + return 0; + + max_queue = get_port_max_queues(port_id); + if (queue_id >= max_queue) + return 0; + + return 1; +} + +/* + * Check if rxq exceeds the number of queues defined for the port. + * Return 0 if invalid. + */ +int +is_valid_port_rxq(uint16_t port_id, uint16_t rxq) +{ + uint16_t nof_queues; + + if (port_map[port_id].queue_info != NULL) { + nof_queues = port_map[port_id].queue_info->rxq; + } else { + /* default number of queues is 1 */ + nof_queues = 1; + } + if (rxq >= nof_queues) + return 0; + + return 1; +} + +/* + * Check if txq exceeds the number of queues defined for the port. + * Return 0 if invalid. + */ +int +is_valid_port_txq(uint16_t port_id, uint16_t txq) +{ + uint16_t nof_queues; + + if (port_map[port_id].queue_info != NULL) { + nof_queues = port_map[port_id].queue_info->txq; + } else { + /* default number of queues is 1 */ + nof_queues = 1; + } + if (txq >= nof_queues) + return 0; + + return 1; } void -forward_array_remove(int port_id) +forward_array_remove(int port_id, uint16_t queue_id) { - unsigned int i; + unsigned int i, j; + uint16_t max_queue; + int remove_flg = 0; /* Update ports_fwd_array */ - forward_array_init_one(port_id); + forward_array_init_one(port_id, queue_id); for (i = 0; i < RTE_MAX_ETHPORTS; i++) { - if (ports_fwd_array[i].in_port_id == PORT_RESET) - continue; - if (ports_fwd_array[i].out_port_id == port_id) { - ports_fwd_array[i].out_port_id = PORT_RESET; + max_queue = get_port_max_queues(i); + + for (j = 0; j < max_queue; j++) { + if (ports_fwd_array[i][j].in_port_id == PORT_RESET && + (ports_fwd_array[i][j].out_port_id != port_id || + ports_fwd_array[i][j].out_queue_id != queue_id)) + continue; + + ports_fwd_array[i][j].out_port_id = PORT_RESET; + remove_flg = 1; break; } + + if (remove_flg) + break; } } @@ -157,3 +249,19 @@ enum port_type get_port_type(char *portname) } return UNDEF; } + +/* Returns a larger number of queues of RX or TX port as the maximum number */ +uint16_t +get_port_max_queues(uint16_t port_id) +{ + uint16_t max_queue = 1; /* default max_queue is 1 */ + + if (port_map[port_id].queue_info != NULL) { + if (port_map[port_id].queue_info->rxq >= + port_map[port_id].queue_info->txq) + max_queue = port_map[port_id].queue_info->rxq; + else + max_queue = port_map[port_id].queue_info->txq; + } + return max_queue; +} diff --git a/src/shared/port_manager.h b/src/shared/port_manager.h index c529998..8317687 100644 --- a/src/shared/port_manager.h +++ b/src/shared/port_manager.h @@ -17,20 +17,25 @@ struct porttype_map { }; /* initialize forward array with default value */ -void forward_array_init_one(unsigned int i); +void forward_array_init_one(unsigned int i, unsigned int j); void forward_array_init(void); void forward_array_reset(void); -void forward_array_remove(int port_id); +void forward_array_remove(int port_id, uint16_t queue_id); void port_map_init_one(unsigned int i); void port_map_init(void); enum port_type get_port_type(char *portname); -int add_patch(uint16_t in_port, uint16_t out_port); +int add_patch(uint16_t in_port, uint16_t in_queue, + uint16_t out_port, uint16_t out_queue); uint16_t find_port_id(int id, enum port_type type); -int is_valid_port(uint16_t port_id); +int is_valid_port(uint16_t port_id, uint16_t queue_id); +int is_valid_port_rxq(uint16_t port_id, uint16_t rxq); +int is_valid_port_txq(uint16_t port_id, uint16_t txq); + +uint16_t get_port_max_queues(uint16_t port_id); #endif // __SHARED_PORT_MANAGER_H__ diff --git a/src/shared/secondary/spp_worker_th/cmd_parser.c b/src/shared/secondary/spp_worker_th/cmd_parser.c index 3f947f5..6e66f35 100644 --- a/src/shared/secondary/spp_worker_th/cmd_parser.c +++ b/src/shared/secondary/spp_worker_th/cmd_parser.c @@ -120,10 +120,10 @@ const char *PORT_ABILITY_LIST[] = { static int is_used_with_addr( int vid, uint64_t mac_addr, - enum port_type iface_type, int iface_no) + enum port_type iface_type, int iface_no, int queue_no) { struct sppwk_port_info *wk_port = get_sppwk_port( - iface_type, iface_no); + iface_type, iface_no, queue_no); return ((mac_addr == wk_port->cls_attrs.mac_addr) && (vid == wk_port->cls_attrs.vlantag.vid)); @@ -131,25 +131,32 @@ is_used_with_addr( /* Return 1 as true if given port is already used. */ static int -is_added_port(enum port_type iface_type, int iface_no) +is_added_port(enum port_type iface_type, int iface_no, int queue_no) { - struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no); + struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no, + queue_no); return port->iface_type != UNDEF; } /** - * Separate resource UID of combination of iface type and number and assign to - * given argument, iface_type and iface_no. For instance, 'ring:0' is separated - * to 'ring' and '0'. The supported types are `phy`, `vhost` and `ring`. + * Extract `iface_type`, `iface_no` and `queue_no` from `res_uid`. + * Only phy port has `queue_no`, such as 'phy:0nq1', if using multi-queue. + * For other port types other than `phy`, + * it returnes `DEFAULT_QUEUE_ID` as `queue_no`. */ static int parse_resource_uid(const char *res_uid, - enum port_type *iface_type, - int *iface_no) + enum port_type *iface_type, + int *iface_no, + int *queue_no) { enum port_type ptype = UNDEF; + const char *iface_no_and_queue_no_str = NULL; + const char *queue_no_with_separator_str = NULL; const char *iface_no_str = NULL; - char *endptr = NULL; + char *queue_no_endptr = NULL; + char *iface_no_endptr = NULL; + int queue_id, multi_queue_flg; /** * TODO(yasufum) consider this checking of zero value is recommended @@ -158,33 +165,84 @@ parse_resource_uid(const char *res_uid, if (strncmp(res_uid, SPPWK_PHY_STR ":", strlen(SPPWK_PHY_STR)+1) == 0) { ptype = PHY; - iface_no_str = &res_uid[strlen(SPPWK_PHY_STR)+1]; + iface_no_and_queue_no_str = &res_uid[strlen(SPPWK_PHY_STR)+1]; } else if (strncmp(res_uid, SPPWK_VHOST_STR ":", strlen(SPPWK_VHOST_STR)+1) == 0) { ptype = VHOST; - iface_no_str = &res_uid[strlen(SPPWK_VHOST_STR)+1]; + iface_no_and_queue_no_str = &res_uid[strlen(SPPWK_VHOST_STR)+1]; } else if (strncmp(res_uid, SPPWK_RING_STR ":", strlen(SPPWK_RING_STR)+1) == 0) { ptype = RING; - iface_no_str = &res_uid[strlen(SPPWK_RING_STR)+1]; + iface_no_and_queue_no_str = &res_uid[strlen(SPPWK_RING_STR)+1]; + } else { + RTE_LOG(ERR, WK_CMD_PARSER, "Unexpected port type in '%s'.\n", + res_uid); + return SPPWK_RET_NG; + } + + /* Parse queue number. */ + queue_no_with_separator_str = strstr(iface_no_and_queue_no_str, + DELIM_PHY_MQ); + if (queue_no_with_separator_str == NULL) { + iface_no_str = iface_no_and_queue_no_str; + queue_id = DEFAULT_QUEUE_ID; + multi_queue_flg = 0; + } else if (ptype == PHY) { + const char *queue_no_str = + &queue_no_with_separator_str[ + strlen(DELIM_PHY_MQ)]; + queue_id = strtol(queue_no_str, &queue_no_endptr, 0); + if (unlikely(queue_no_str == queue_no_endptr) || + unlikely(*queue_no_endptr != '\0')) { + RTE_LOG(ERR, WK_CMD_PARSER, "No queue number in '%s'.\n", + res_uid); + return SPPWK_RET_NG; + } + iface_no_str = iface_no_and_queue_no_str; + multi_queue_flg = 1; } else { RTE_LOG(ERR, WK_CMD_PARSER, "Unexpected port type in '%s'.\n", res_uid); return SPPWK_RET_NG; } - int port_id = strtol(iface_no_str, &endptr, 0); - if (unlikely(iface_no_str == endptr) || unlikely(*endptr != '\0')) { + /* Parse interface number. */ + int port_id = strtol(iface_no_str, &iface_no_endptr, 0); + if (unlikely(iface_no_str == iface_no_endptr) || + (unlikely(*iface_no_endptr != '\0') && + (unlikely(strstr(iface_no_endptr, + DELIM_PHY_MQ) != iface_no_endptr)))) { RTE_LOG(ERR, WK_CMD_PARSER, "No interface number in '%s'.\n", res_uid); return SPPWK_RET_NG; } + if (unlikely(port_id > RTE_MAX_ETHPORTS) || unlikely(port_id < 0)) { + RTE_LOG(ERR, WK_CMD_PARSER, "Unexpected interface number in '%s'.\n", + res_uid); + return SPPWK_RET_NG; + } + + /** + * Check whether the queue number is specified according + * to the port format. + */ + if (ptype == PHY) { + int port_multi_queue_flg = + (get_port_max_queues(ptype, port_id) > 1); + if (unlikely(multi_queue_flg != port_multi_queue_flg)) { + RTE_LOG(ERR, WK_CMD_PARSER, + "Unexpected queue number format in" + " '%s'.\n", res_uid); + return SPPWK_RET_NG; + } + } *iface_type = ptype; *iface_no = port_id; + *queue_no = queue_id; - RTE_LOG(DEBUG, WK_CMD_PARSER, "Parsed '%s' to '%d' and '%d'.\n", - res_uid, *iface_type, *iface_no); + RTE_LOG(DEBUG, WK_CMD_PARSER, "Parsed '%s' to '%d' and '%d' and '%d'.\n", + res_uid, *iface_type, *iface_no, *queue_no); return SPPWK_RET_OK; } @@ -302,7 +360,8 @@ parse_port_uid(void *output, const char *arg_val) { int ret; struct sppwk_port_idx *port = output; - ret = parse_resource_uid(arg_val, &port->iface_type, &port->iface_no); + ret = parse_resource_uid(arg_val, &port->iface_type, &port->iface_no, + &port->queue_no); if (unlikely(ret != 0)) { RTE_LOG(ERR, WK_CMD_PARSER, "Invalid resource UID '%s'.\n", arg_val); @@ -465,9 +524,11 @@ parse_port(void *output, const char *arg_val, int allow_override) if ((port->wk_action == SPPWK_ACT_ADD) && (sppwk_check_used_port(tmp_port.iface_type, tmp_port.iface_no, + tmp_port.queue_no, SPPWK_PORT_DIR_RX) >= 0) && (sppwk_check_used_port(tmp_port.iface_type, tmp_port.iface_no, + tmp_port.queue_no, SPPWK_PORT_DIR_TX) >= 0)) { RTE_LOG(ERR, WK_CMD_PARSER, "Port `%s` is already used.\n", @@ -478,6 +539,7 @@ parse_port(void *output, const char *arg_val, int allow_override) port->port.iface_type = tmp_port.iface_type; port->port.iface_no = tmp_port.iface_no; + port->port.queue_no = tmp_port.queue_no; return SPPWK_RET_OK; } @@ -495,11 +557,36 @@ parse_port_direction(void *output, const char *arg_val, int allow_override) return SPPWK_RET_NG; } - /* add vlantag command check */ + /* Add queue number command check. */ + if ((port->wk_action == SPPWK_ACT_ADD) && + (port->port.iface_type == PHY)) { + const struct port_info *ports = NULL; + const struct rte_memzone *mz; + mz = rte_memzone_lookup(MZ_PORT_INFO); + ports = mz->addr; + int max_port_queue = -1; + if (ret == SPPWK_PORT_DIR_RX) + max_port_queue = ports->queue_info + [port->port.iface_no].rxq - 1; + else if (ret == SPPWK_PORT_DIR_TX) + max_port_queue = ports->queue_info + [port->port.iface_no].txq - 1; + if (unlikely(port->port.queue_no > max_port_queue)) { + RTE_LOG(ERR, WK_CMD_PARSER, + "The queue number exceeds the %s max value." + " queue=%d, max=%d.\n", + PORT_DIR_LIST[ret], port->port.queue_no, + max_port_queue); + return SPPWK_RET_NG; + } + } + + /* Add vlantag command check. */ if (allow_override == 0) { if ((port->wk_action == SPPWK_ACT_ADD) && (sppwk_check_used_port(port->port.iface_type, - port->port.iface_no, ret) >= 0)) { + port->port.iface_no, + port->port.queue_no, ret) >= 0)) { RTE_LOG(ERR, WK_CMD_PARSER, "Port in used. (port command) val=%s\n", arg_val); @@ -726,7 +813,8 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val, if (ret < SPPWK_RET_OK) return SPPWK_RET_NG; - if (is_added_port(tmp_port.iface_type, tmp_port.iface_no) == 0) { + if (is_added_port(tmp_port.iface_type, tmp_port.iface_no, + tmp_port.queue_no) == 0) { RTE_LOG(ERR, WK_CMD_PARSER, "Port not added. val=%s\n", arg_val); return SPPWK_RET_NG; @@ -737,7 +825,8 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val, if (unlikely(cls_attrs->wk_action == SPPWK_ACT_ADD)) { if (!is_used_with_addr(ETH_VLAN_ID_MAX, 0, - tmp_port.iface_type, tmp_port.iface_no)) { + tmp_port.iface_type, tmp_port.iface_no, + tmp_port.queue_no)) { RTE_LOG(ERR, WK_CMD_PARSER, "Port in used. " "(classifier_table command) val=%s\n", arg_val); @@ -750,7 +839,8 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val, if (!is_used_with_addr(cls_attrs->vid, (uint64_t)mac_addr, - tmp_port.iface_type, tmp_port.iface_no)) { + tmp_port.iface_type, tmp_port.iface_no, + tmp_port.queue_no)) { RTE_LOG(ERR, WK_CMD_PARSER, "Port in used. " "(classifier_table command) val=%s\n", arg_val); @@ -760,6 +850,7 @@ parse_cls_port(void *cls_cmd_attr, const char *arg_val, cls_attrs->port.iface_type = tmp_port.iface_type; cls_attrs->port.iface_no = tmp_port.iface_no; + cls_attrs->port.queue_no = tmp_port.queue_no; return SPPWK_RET_OK; } diff --git a/src/shared/secondary/spp_worker_th/cmd_res_formatter.c b/src/shared/secondary/spp_worker_th/cmd_res_formatter.c index 85b48af..a04bc91 100644 --- a/src/shared/secondary/spp_worker_th/cmd_res_formatter.c +++ b/src/shared/secondary/spp_worker_th/cmd_res_formatter.c @@ -79,9 +79,10 @@ append_error_details_value(const char *name, char **output, void *tmp) /* Check if port is already flushed. */ static int -is_port_flushed(enum port_type iface_type, int iface_no) +is_port_flushed(enum port_type iface_type, int iface_no, int queue_no) { - struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no); + struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no, + queue_no); return port->ethdev_port_id >= 0; } @@ -89,23 +90,43 @@ is_port_flushed(enum port_type iface_type, int iface_no) int append_interface_array(char **output, const enum port_type type) { - int i, port_cnt = 0; + int port_cnt, str_cnt = 0; + uint16_t queue_cnt; char tmp_str[CMD_TAG_APPEND_SIZE]; - for (i = 0; i < RTE_MAX_ETHPORTS; i++) { - if (!is_port_flushed(type, i)) - continue; + for (port_cnt = 0; port_cnt < RTE_MAX_ETHPORTS; port_cnt++) { + int max_queue_port = get_port_max_queues(type, port_cnt); + if (unlikely(max_queue_port < 0)) + return SPPWK_RET_NG; - sprintf(tmp_str, "%s%d", JSON_APPEND_COMMA(port_cnt), i); + for (queue_cnt = 0; queue_cnt < max_queue_port; + queue_cnt++) { + if (!is_port_flushed(type, port_cnt, queue_cnt)) + continue; + + if (max_queue_port <= 1) { + sprintf(tmp_str, "%s\"%d\"", + JSON_APPEND_COMMA(str_cnt), + port_cnt); + } else { + sprintf(tmp_str, "%s\"%d nq %d\"", + JSON_APPEND_COMMA(str_cnt), + port_cnt, queue_cnt); + } - *output = spp_strbuf_append(*output, tmp_str, strlen(tmp_str)); - if (unlikely(*output == NULL)) { - RTE_LOG(ERR, WK_CMD_RES_FMT, - /* TODO(yasufum) replace %d to string. */ - "Failed to add index for type `%d`.\n", type); - return SPPWK_RET_NG; + *output = spp_strbuf_append(*output, + tmp_str, strlen(tmp_str)); + if (unlikely(*output == NULL)) { + RTE_LOG(ERR, WK_CMD_RES_FMT, + /** + * TODO(yasufum) replace %d to string. + */ + "Failed to add index for type `%d`.\n", + type); + return SPPWK_RET_NG; + } + str_cnt++; } - port_cnt++; } return SPPWK_RET_OK; } @@ -195,14 +216,14 @@ append_vlan_block(const char *name, char **output, * It returns a port ID, or error code if it's failed to. */ static int -get_ethdev_port_id(enum port_type iface_type, int iface_no) +get_ethdev_port_id(enum port_type iface_type, int iface_no, int queue_no) { struct iface_info *iface_info = NULL; sppwk_get_mng_data(&iface_info, NULL, NULL, NULL, NULL, NULL); switch (iface_type) { case PHY: - return iface_info->phy[iface_no].ethdev_port_id; + return iface_info->phy[iface_no][queue_no].ethdev_port_id; case RING: return iface_info->ring[iface_no].ethdev_port_id; case VHOST: @@ -227,15 +248,17 @@ append_port_block(char **output, const struct sppwk_port_idx *port, return SPPWK_RET_NG; } - sppwk_port_uid(port_str, port->iface_type, port->iface_no); + sppwk_port_uid(port_str, port->iface_type, port->iface_no, + port->queue_no); ret = append_json_str_value(&tmp_buff, "port", port_str); if (unlikely(ret < SPPWK_RET_OK)) return SPPWK_RET_NG; ret = append_vlan_block("vlan", &tmp_buff, get_ethdev_port_id( - port->iface_type, port->iface_no), - dir); + port->iface_type, port->iface_no, + port->queue_no), + dir); if (unlikely(ret < SPPWK_RET_OK)) return SPPWK_RET_NG; @@ -519,7 +542,7 @@ add_client_id(const char *name, char **output, return append_json_int_value(output, name, get_client_id()); } -/* Add entry of port to a response in JSON such as "phy:0". */ +/* Add entry of port to a response in JSON such as "phy:0nq0". */ int add_interface(const char *name, char **output, void *tmp __attribute__ ((unused))) diff --git a/src/shared/secondary/spp_worker_th/cmd_utils.c b/src/shared/secondary/spp_worker_th/cmd_utils.c index 69d7222..3dea02c 100644 --- a/src/shared/secondary/spp_worker_th/cmd_utils.c +++ b/src/shared/secondary/spp_worker_th/cmd_utils.c @@ -154,13 +154,13 @@ stop_process(int signal) * if given type is invalid. */ struct sppwk_port_info * -get_sppwk_port(enum port_type iface_type, int iface_no) +get_sppwk_port(enum port_type iface_type, int iface_no, int queue_no) { struct iface_info *iface_info = g_mng_data.p_iface_info; switch (iface_type) { case PHY: - return &iface_info->phy[iface_no]; + return &iface_info->phy[iface_no][queue_no]; case VHOST: return &iface_info->vhost[iface_no]; case RING: @@ -229,19 +229,24 @@ log_interface_info(const struct iface_info *iface_info) { const struct sppwk_port_info *port = NULL; int cnt = 0; + int queue_cnt; for (cnt = 0; cnt < RTE_MAX_ETHPORTS; cnt++) { - port = &iface_info->phy[cnt]; - if (port->iface_type == UNDEF) - continue; - - RTE_LOG(DEBUG, WK_CMD_UTILS, - "phy [%d] type=%d, no=%d, port=%d, " - "vid = %u, mac=%08lx(%s)\n", - cnt, port->iface_type, port->iface_no, - port->ethdev_port_id, - port->cls_attrs.vlantag.vid, - port->cls_attrs.mac_addr, - port->cls_attrs.mac_addr_str); + int max_queue_port = get_port_max_queues(PHY, cnt); + for (queue_cnt = 0; queue_cnt < max_queue_port; + queue_cnt++) { + port = &iface_info->phy[cnt][queue_cnt]; + if (port->iface_type == UNDEF) + continue; + + RTE_LOG(DEBUG, WK_CMD_UTILS, + "phy [%d] type=%d, no=%d, port=%d, " + "queue=%d, vid = %u, mac=%08lx(%s)\n", + cnt, port->iface_type, port->iface_no, + port->ethdev_port_id, port->queue_no, + port->cls_attrs.vlantag.vid, + port->cls_attrs.mac_addr, + port->cls_attrs.mac_addr_str); + } } for (cnt = 0; cnt < RTE_MAX_ETHPORTS; cnt++) { port = &iface_info->vhost[cnt]; @@ -356,21 +361,33 @@ static void init_iface_info(void) { int port_cnt; /* increment ether ports */ + int queue_cnt; /* increment ether queue per ports */ struct iface_info *p_iface_info = g_mng_data.p_iface_info; memset(p_iface_info, 0x00, sizeof(struct iface_info)); for (port_cnt = 0; port_cnt < RTE_MAX_ETHPORTS; port_cnt++) { - p_iface_info->phy[port_cnt].iface_type = UNDEF; - p_iface_info->phy[port_cnt].iface_no = port_cnt; - p_iface_info->phy[port_cnt].ethdev_port_id = -1; - p_iface_info->phy[port_cnt].cls_attrs.vlantag.vid = - ETH_VLAN_ID_MAX; + for (queue_cnt = 0; queue_cnt < RTE_MAX_QUEUES_PER_PORT; + queue_cnt++) { + p_iface_info->phy[port_cnt][queue_cnt].iface_type = + UNDEF; + p_iface_info->phy[port_cnt][queue_cnt].iface_no = + port_cnt; + p_iface_info->phy[port_cnt][queue_cnt].queue_no = + queue_cnt; + p_iface_info->phy[port_cnt][queue_cnt] + .ethdev_port_id = -1; + p_iface_info->phy[port_cnt][queue_cnt] + .cls_attrs.vlantag.vid = + ETH_VLAN_ID_MAX; + } p_iface_info->vhost[port_cnt].iface_type = UNDEF; p_iface_info->vhost[port_cnt].iface_no = port_cnt; + p_iface_info->vhost[port_cnt].queue_no = DEFAULT_QUEUE_ID; p_iface_info->vhost[port_cnt].ethdev_port_id = -1; p_iface_info->vhost[port_cnt].cls_attrs.vlantag.vid = ETH_VLAN_ID_MAX; p_iface_info->ring[port_cnt].iface_type = UNDEF; p_iface_info->ring[port_cnt].iface_no = port_cnt; + p_iface_info->ring[port_cnt].queue_no = DEFAULT_QUEUE_ID; p_iface_info->ring[port_cnt].ethdev_port_id = -1; p_iface_info->ring[port_cnt].cls_attrs.vlantag.vid = ETH_VLAN_ID_MAX; @@ -411,7 +428,7 @@ static int init_host_port_info(void) { int port_type, port_id; - int i, ret; + int i, ret, queue_id; int nof_phys = 0; char dev_name[RTE_DEV_NAME_MAX_LEN] = { 0 }; struct iface_info *p_iface_info = g_mng_data.p_iface_info; @@ -433,8 +450,13 @@ init_host_port_info(void) switch (port_type) { case PHY: - p_iface_info->phy[port_id].iface_type = port_type; - p_iface_info->phy[port_id].ethdev_port_id = port_id; + for (queue_id = 0; queue_id < RTE_MAX_QUEUES_PER_PORT; + queue_id++) { + p_iface_info->phy[port_id][queue_id] + .iface_type = port_type; + p_iface_info->phy[port_id][queue_id] + .ethdev_port_id = port_id; + } break; case VHOST: /* NOTE: a vhost can be used by one process. @@ -526,12 +548,14 @@ int sppwk_check_used_port( enum port_type iface_type, int iface_no, + int queue_no, enum sppwk_port_dir dir) { int cnt, port_cnt, max = 0; struct sppwk_comp_info *component = NULL; struct sppwk_port_info **port_array = NULL; - struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no); + struct sppwk_port_info *port = get_sppwk_port(iface_type, iface_no, + queue_no); struct sppwk_comp_info *component_info = g_mng_data.p_component_info; @@ -567,14 +591,14 @@ set_component_change_port(struct sppwk_port_info *port, int ret = 0; if ((dir == SPPWK_PORT_DIR_RX) || (dir == SPPWK_PORT_DIR_BOTH)) { ret = sppwk_check_used_port(port->iface_type, port->iface_no, - SPPWK_PORT_DIR_RX); + port->queue_no, SPPWK_PORT_DIR_RX); if (ret >= 0) *(g_mng_data.p_change_component + ret) = 1; } if ((dir == SPPWK_PORT_DIR_TX) || (dir == SPPWK_PORT_DIR_BOTH)) { ret = sppwk_check_used_port(port->iface_type, port->iface_no, - SPPWK_PORT_DIR_TX); + port->queue_no, SPPWK_PORT_DIR_TX); if (ret >= 0) *(g_mng_data.p_change_component + ret) = 1; } @@ -628,6 +652,28 @@ get_idx_port_info(struct sppwk_port_info *p_info, int nof_ports, return ret; } +/* Returns a larger number of queues of RX or TX port as the maximum number */ +int +get_port_max_queues(const enum port_type iface_type, int iface_no) +{ + if (unlikely(iface_no > RTE_MAX_ETHPORTS) || unlikely(iface_no < 0)) + return SPPWK_RET_NG; + + if (iface_type != PHY) + return 1; + + const struct port_info *ports = NULL; + const struct rte_memzone *mz; + mz = rte_memzone_lookup(MZ_PORT_INFO); + ports = mz->addr; + int max_q_rx = ports->queue_info[iface_no].rxq; + int max_q_tx = ports->queue_info[iface_no].txq; + if (max_q_rx > max_q_tx) + return max_q_rx; + else + return max_q_tx; +} + /* Delete given port info from the port info array. */ int delete_port_info(struct sppwk_port_info *p_info, int nof_ports, @@ -716,8 +762,9 @@ update_lcore_info(void) } } -/* Return port uid such as `phy:0`, `ring:1` or so. */ -int sppwk_port_uid(char *port_uid, enum port_type p_type, int iface_no) +/* Return port uid such as `phy:0nq0`, `ring:1` or so. */ +int sppwk_port_uid(char *port_uid, enum port_type p_type, int iface_no, + int queue_no) { const char *p_type_str; @@ -735,7 +782,15 @@ int sppwk_port_uid(char *port_uid, enum port_type p_type, int iface_no) return SPPWK_RET_NG; } - sprintf(port_uid, "%s:%d", p_type_str, iface_no); + int max_queue_port = get_port_max_queues(p_type, iface_no); + if (unlikely(max_queue_port == SPPWK_RET_NG)) + return SPPWK_RET_NG; + + if (max_queue_port <= 1) + sprintf(port_uid, "%s:%d", p_type_str, iface_no); + else + sprintf(port_uid, "%s:%d nq %d", + p_type_str, iface_no, queue_no); return SPPWK_RET_OK; } diff --git a/src/shared/secondary/spp_worker_th/cmd_utils.h b/src/shared/secondary/spp_worker_th/cmd_utils.h index df1b0dc..eda55c6 100644 --- a/src/shared/secondary/spp_worker_th/cmd_utils.h +++ b/src/shared/secondary/spp_worker_th/cmd_utils.h @@ -174,7 +174,7 @@ void stop_process(int signal); /* Return sppwk_port_info of given type and num of interface. */ struct sppwk_port_info * -get_sppwk_port(enum port_type iface_type, int iface_no); +get_sppwk_port(enum port_type iface_type, int iface_no, int queue_no); /* Output log message for core information */ void log_core_info(const struct core_mng_info *core_info); @@ -221,6 +221,7 @@ int sppwk_is_lcore_updated(unsigned int lcore_id); * * @param iface_type Interface type to be validated. * @param iface_no Interface number to be validated. + * @param queue_no Queue number of interface to be validated. * @param rxtx Value of spp_port_rxtx to be validated. * @retval 0~127 If match component ID * @retval SPPWK_RET_NG If failed. @@ -228,6 +229,7 @@ int sppwk_is_lcore_updated(unsigned int lcore_id); int sppwk_check_used_port( enum port_type iface_type, int iface_no, + int queue_no, enum sppwk_port_dir dir); /** @@ -269,6 +271,15 @@ int sppwk_get_lcore_id(const char *comp_name); int get_idx_port_info(struct sppwk_port_info *p_info, int nof_ports, struct sppwk_port_info *p_info_ary[]); +/** + * Returns max queue number of the target port. + * + * @param[in] iface_type Interface type such as PHY or so. + * @param[in] iface_no Interface number. + * @return Max queue number if succeeded, or SPPWK_RET_NG if failed. + */ +int get_port_max_queues(const enum port_type iface_type, int iface_no); + /** * search matched port_info from array and delete it. * @@ -294,15 +305,17 @@ int update_port_info(void); void update_lcore_info(void); /** - * Return port uid such as `phy:0`, `ring:1` or so. + * Return port uid such as `phy:0nq0`, `ring:1` or so. * * @param[in,out] port_uid String of port type to be converted. * @param[in] iface_type Interface type such as PHY or so. * @param[in] iface_no Interface number. + * @param[in] queue_no Queue number of interface. * @return SPPWK_RET_OK If succeeded, or SPPWK_RET_NG if failed. */ int -sppwk_port_uid(char *port_uid, enum port_type iface_type, int iface_no); +sppwk_port_uid(char *port_uid, enum port_type iface_type, int iface_no, + int queue_no); /** * Change string of MAC address to int64. diff --git a/src/shared/secondary/spp_worker_th/data_types.h b/src/shared/secondary/spp_worker_th/data_types.h index 57bf182..a05eb75 100644 --- a/src/shared/secondary/spp_worker_th/data_types.h +++ b/src/shared/secondary/spp_worker_th/data_types.h @@ -15,9 +15,17 @@ #define SPPWK_VHOST_STR "vhost" #define SPPWK_RING_STR "ring" +/* + * Delimiter for phy port supporting multi queue. + * Example of phy port supporting multi-queue: `phy: 0nq1`. + */ +#define DELIM_PHY_MQ "nq" + /* TODO(yasufum) confirm usage of this value and why it is 4. */ #define PORT_CAPABL_MAX 4 /* Max num of port abilities. */ +#define DEFAULT_QUEUE_ID 0 /* Queue ID is counted up from 0. */ + /* Status of a component on lcore. */ enum sppwk_lcore_status { SPPWK_LCORE_UNUSED, @@ -89,6 +97,7 @@ struct sppwk_cls_attrs { struct sppwk_port_idx { enum port_type iface_type; /**< phy, vhost or ring. */ int iface_no; + int queue_no; }; /* Define detailed port params in addition to `sppwk_port_idx`. */ @@ -96,6 +105,7 @@ struct sppwk_port_info { enum port_type iface_type; /**< phy, vhost or ring */ int iface_no; int ethdev_port_id; /**< Consistent ID of ethdev */ + int queue_no; struct sppwk_cls_attrs cls_attrs; struct sppwk_port_attrs port_attrs[PORT_CAPABL_MAX]; }; @@ -108,8 +118,10 @@ struct sppwk_comp_info { int comp_id; /**< Component ID */ int nof_rx; /**< The number of rx ports */ int nof_tx; /**< The number of tx ports */ - struct sppwk_port_info *rx_ports[RTE_MAX_ETHPORTS]; /**< rx ports */ - struct sppwk_port_info *tx_ports[RTE_MAX_ETHPORTS]; /**< tx ports */ + /**< rx ports */ + struct sppwk_port_info *rx_ports[RTE_MAX_QUEUES_PER_PORT]; + /**< tx ports */ + struct sppwk_port_info *tx_ports[RTE_MAX_QUEUES_PER_PORT]; }; /* Manage number of interfaces and port information as global variable. */ @@ -120,7 +132,7 @@ struct sppwk_comp_info { * or not. */ struct iface_info { - struct sppwk_port_info phy[RTE_MAX_ETHPORTS]; + struct sppwk_port_info phy[RTE_MAX_ETHPORTS][RTE_MAX_QUEUES_PER_PORT]; struct sppwk_port_info vhost[RTE_MAX_ETHPORTS]; struct sppwk_port_info ring[RTE_MAX_ETHPORTS]; }; diff --git a/src/shared/secondary/spp_worker_th/port_capability.c b/src/shared/secondary/spp_worker_th/port_capability.c index 5435db0..5bd759f 100644 --- a/src/shared/secondary/spp_worker_th/port_capability.c +++ b/src/shared/secondary/spp_worker_th/port_capability.c @@ -378,11 +378,11 @@ vlan_operation(uint16_t port_id, struct rte_mbuf **pkts, const uint16_t nb_pkts, /* Wrapper function for rte_eth_rx_burst() with VLAN feature. */ uint16_t sppwk_eth_vlan_rx_burst(uint16_t port_id, - uint16_t queue_id __attribute__ ((unused)), + uint16_t queue_id, struct rte_mbuf **rx_pkts, const uint16_t nb_pkts) { uint16_t nb_rx; - nb_rx = rte_eth_rx_burst(port_id, 0, rx_pkts, nb_pkts); + nb_rx = rte_eth_rx_burst(port_id, queue_id, rx_pkts, nb_pkts); if (unlikely(nb_rx == 0)) return SPPWK_RET_OK; @@ -393,7 +393,7 @@ sppwk_eth_vlan_rx_burst(uint16_t port_id, /* Wrapper function for rte_eth_tx_burst() with VLAN feature. */ uint16_t sppwk_eth_vlan_tx_burst(uint16_t port_id, - uint16_t queue_id __attribute__ ((unused)), + uint16_t queue_id, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) { uint16_t nb_tx; @@ -404,7 +404,7 @@ sppwk_eth_vlan_tx_burst(uint16_t port_id, if (unlikely(nb_tx == 0)) return SPPWK_RET_OK; - return rte_eth_tx_burst(port_id, 0, tx_pkts, nb_tx); + return rte_eth_tx_burst(port_id, queue_id, tx_pkts, nb_tx); } #ifdef SPP_RINGLATENCYSTATS_ENABLE diff --git a/src/shared/secondary/spp_worker_th/port_capability.h b/src/shared/secondary/spp_worker_th/port_capability.h index 284ba30..17d3a16 100644 --- a/src/shared/secondary/spp_worker_th/port_capability.h +++ b/src/shared/secondary/spp_worker_th/port_capability.h @@ -62,7 +62,7 @@ void sppwk_update_port_dir(const struct sppwk_comp_info *comp); * Wrapper function for rte_eth_rx_burst() with VLAN feature. * * @param[in] port_id Etherdev ID. - * @param[in] queue_id RX queue ID, but fixed value 0 in SPP. + * @param[in] queue_id RX queue ID. * @param[in] rx_pkts Pointers to mbuf should be enough to store nb_pkts. * @param nb_pkts Maximum number of RX packets. * @return Number of RX packets as number of pointers to mbuf. @@ -74,7 +74,7 @@ uint16_t sppwk_eth_vlan_rx_burst(uint16_t port_id, uint16_t queue_id, * Wrapper function for rte_eth_tx_burst() with VLAN feature. * * @param port_id Etherdev ID. - * @param[in] queue_id TX queue ID, but fixed value 0 in SPP. + * @param[in] queue_id TX queue ID. * @param[in] tx_pkts Pointers to mbuf should be enough to store nb_pkts. * @param nb_pkts Maximum number of TX packets. * @return Number of TX packets as number of pointers to mbuf. diff --git a/src/shared/secondary/spp_worker_th/vf_deps.h b/src/shared/secondary/spp_worker_th/vf_deps.h index f2ea62a..dadcf7a 100644 --- a/src/shared/secondary/spp_worker_th/vf_deps.h +++ b/src/shared/secondary/spp_worker_th/vf_deps.h @@ -24,7 +24,12 @@ struct mac_classifier { struct rte_hash *cls_tbl; /* Hash table for MAC classification. */ int nof_cls_ports; /* Num of ports classified validly. */ - int cls_ports[RTE_MAX_ETHPORTS]; /* Ports for classification. */ + /** + * Ports for classification. + * For multi-queue case, size of ports should be + * RTE_MAX_QUEUES_PER_PORT. RTE_MAX_ETHPORTS is not enough. + */ + int cls_ports[RTE_MAX_QUEUES_PER_PORT]; int default_cls_idx; /* Default index for classification. */ }; @@ -33,6 +38,7 @@ struct mac_classifier { struct cls_port_info { enum port_type iface_type; int iface_no; /* Index of ports handled by classifier. */ + int queue_no; /* Index of queue per port handled by classifier. */ int iface_no_global; /* ID for interface generated by spp_vf */ uint16_t ethdev_port_id; /* Ethdev port ID. */ uint16_t nof_pkts; /* Number of packets in pkts[]. */ @@ -47,7 +53,12 @@ struct cls_comp_info { int nof_tx_ports; /* Number of TX ports info entries. */ /* Classifier has one RX port and several TX ports. */ struct cls_port_info rx_port_i; /* RX port info classified. */ - struct cls_port_info tx_ports_i[RTE_MAX_ETHPORTS]; /* TX info. */ + /** + * TX info. + * For multi-queue case, size of ports should be + * RTE_MAX_QUEUES_PER_PORT. RTE_MAX_ETHPORTS is not enough. + */ + struct cls_port_info tx_ports_i[RTE_MAX_QUEUES_PER_PORT]; }; int add_core(const char *name, char **output, diff --git a/src/shared/secondary/utils.c b/src/shared/secondary/utils.c index c80a478..7cca4a4 100644 --- a/src/shared/secondary/utils.c +++ b/src/shared/secondary/utils.c @@ -5,6 +5,7 @@ #include #include "shared/common.h" #include "shared/secondary/utils.h" +#include "shared/secondary/spp_worker_th/data_types.h" #define RTE_LOGTYPE_SHARED RTE_LOGTYPE_USER1 @@ -66,15 +67,18 @@ parse_client_id(int *cli_id, const char *cli_id_str) } /** - * Retieve port type and ID from resource UID. For example, resource UID - * 'ring:0' is parsed to retrieve port tyep 'ring' and ID '0'. + * Retieve port type, ID and queue id from resource UID. For example, + * resource UID 'ring:0' is parsed to retrieve port type 'ring' and ID '0'. */ int -parse_resource_uid(char *str, char **port_type, int *port_id) +parse_resource_uid(char *str, char **port_type, int *port_id, + uint16_t *queue_id) { char *token; char delim[] = ":"; char *endp; + uint16_t default_queue = DEFAULT_QUEUE_ID; + char *queue_id_str = NULL; RTE_LOG(DEBUG, SHARED, "Parsing resource UID: '%s'\n", str); if (strstr(str, delim) == NULL) { @@ -86,13 +90,27 @@ parse_resource_uid(char *str, char **port_type, int *port_id) *port_type = strtok(str, delim); token = strtok(NULL, delim); + queue_id_str = strstr(token, DELIM_PHY_MQ); *port_id = strtol(token, &endp, 10); - - if (*endp) { + if ((queue_id_str == NULL && *endp) || + (queue_id_str != NULL && endp != queue_id_str)) { RTE_LOG(ERR, SHARED, "Bad integer value: %s\n", str); return -1; } + if (queue_id_str == NULL) { + *queue_id = default_queue; + } else { + *queue_id = strtol(&queue_id_str[strlen(DELIM_PHY_MQ)], + &endp, 10); + + if (*endp) { + RTE_LOG(ERR, SHARED, + "queue_id is bad integer value: %s\n", str); + return -1; + } + } + return 0; } diff --git a/src/shared/secondary/utils.h b/src/shared/secondary/utils.h index eba221f..4ae0a8e 100644 --- a/src/shared/secondary/utils.h +++ b/src/shared/secondary/utils.h @@ -5,7 +5,8 @@ #ifndef _SHARED_SECONDARY_UTILS_H_ #define _SHARED_SECONDARY_UTILS_H_ -int parse_resource_uid(char *str, char **port_type, int *port_id); +int parse_resource_uid(char *str, char **port_type, int *port_id, + uint16_t *queue_id); int spp_atoi(const char *str, int *val); -- 2.17.1