[dpdk-dev] [PATCH 00/39] adding eventmode helper library

[dpdk-dev] [PATCH 00/39] adding eventmode helper library

[Anoob Joseph]

This series adds support for eventmode helper library and l2fwd-event
application.

First 13 patches creates a new l2fwd application (l2fwd-event). Minor
code reorganization is done to faciliate seamless integration of
eventmode.

Next 22 patches adds eventmode helper library. This library abstracts
the configuration of event device & Rx-Tx event adapters. The library
can be extended to allow users to control all the configuration
exposed by adapters and eth device.

Last 4 patches implements eventmode in l2fwd-event application. With
event device and adapters, fine tuned threads (based on dev
capabilities) can be drafted to maximize performance. Eventmode
library facilitates this and l2fwd-event demonstrates this usage.

With the introduction of eventmode helper library, any poll mode
application can be converted to an eventmode application with simple
steps, enabling multi-core scaling and dynamic load balancing to
various example applications.

Usage:
     ./l2fwd-event -- <EAL args> -- <l2fwd args> -- --transfer-mode 1

The above command would invoke eventmode and with the default conf
loaded, traffic on one port would be delivered to all enabled cores.

Planned features,
1. Eventmode helper library doesn't intialize ethdev. Since all
   applications already do this, eventmode helper would start
   from reconfiguring.
2. All features of eventdev and adapters can be exposed to the user
   using common CL arguments. The framework for achieving the same
   is already in place. It has to be extended to support more
   features.
3. Documentation is pending.

Created new app based on discussions,
http://patchwork.dpdk.org/cover/40884/
https://patches.dpdk.org/patch/40901/

Tested with nicvf eth PMD and event_octeontx event PMD on Marvell's
CN83XX platform.

Anoob Joseph (39):
  examples/l2fwd-event: create copy of l2fwd
  examples/l2fwd-event: move macros to common header
  examples/l2fwd-event: move structures to common header
  examples/l2fwd-event: move global vars to common header
  examples/l2fwd-event: move dataplane code to new file
  examples/l2fwd-event: remove unused header includes
  examples/l2fwd-event: move drain buffers to new function
  examples/l2fwd-event: optimize check for master core
  examples/l2fwd-event: move periodic tasks to new func
  examples/l2fwd-event: do timer updates only on master
  examples/l2fwd-event: move pkt send code to a new func
  examples/l2fwd-event: use fprintf in usage print
  examples/l2fwd-event: improvements to the usage print
  eventdev: add files for eventmode helper
  eventdev: add routines for logging eventmode helper
  eventdev: add eventmode CL options framework
  eventdev: allow application to set ethernet portmask
  eventdev: add framework for eventmode conf
  eventdev: add common initialize routine for eventmode devs
  eventdev: add eventdevice init for eventmode
  eventdev: add eventdev port-lcore link
  eventdev: add option to specify schedule mode for app stage
  eventdev: add placeholder for ethdev init
  eventdev: add Rx adapter init in eventmode
  eventdev: add routine to validate conf
  eventdev: add default conf for event devs field in conf
  eventdev: add default conf for Rx adapter conf
  eventdev: add default conf for event port-lcore link
  eventdev: add routines to display the eventmode conf
  eventdev: add routine to access eventmode link info
  eventdev: add routine to access event queue for eth Tx
  eventdev: add routine to launch eventmode workers
  eventdev: add Tx adapter support
  eventdev: add support for internal ports
  eventdev: display Tx adapter conf
  examples/l2fwd-event: add eventmode for l2fwd
  examples/l2fwd-event: add eventmode worker
  examples/l2fwd-event: add eventmode worker
  examples/l2fwd-event: add eventmode worker

 config/common_base                                 |    1 +
 examples/Makefile                                  |    1 +
 examples/l2fwd-event/Makefile                      |   57 +
 examples/l2fwd-event/l2fwd_common.h                |   63 +
 examples/l2fwd-event/l2fwd_worker.c                | 1121 +++++++++++++
 examples/l2fwd-event/l2fwd_worker.h                |   16 +
 examples/l2fwd-event/main.c                        |  585 +++++++
 examples/l2fwd-event/meson.build                   |   14 +
 lib/librte_eal/common/eal_common_log.c             |    1 +
 lib/librte_eal/common/include/rte_log.h            |    1 +
 lib/librte_eventdev/Makefile                       |    6 +-
 lib/librte_eventdev/meson.build                    |    3 +
 lib/librte_eventdev/rte_eventdev_version.map       |    8 +
 lib/librte_eventdev/rte_eventmode_helper.c         | 1678 ++++++++++++++++++++
 lib/librte_eventdev/rte_eventmode_helper.h         |  241 +++
 .../rte_eventmode_helper_internal.h                |  144 ++
 lib/librte_eventdev/rte_eventmode_helper_prints.c  |  223 +++
 17 files changed, 4161 insertions(+), 2 deletions(-)
 create mode 100644 examples/l2fwd-event/Makefile
 create mode 100644 examples/l2fwd-event/l2fwd_common.h
 create mode 100644 examples/l2fwd-event/l2fwd_worker.c
 create mode 100644 examples/l2fwd-event/l2fwd_worker.h
 create mode 100644 examples/l2fwd-event/main.c
 create mode 100644 examples/l2fwd-event/meson.build
 create mode 100644 lib/librte_eventdev/rte_eventmode_helper.c
 create mode 100644 lib/librte_eventdev/rte_eventmode_helper.h
 create mode 100644 lib/librte_eventdev/rte_eventmode_helper_internal.h
 create mode 100644 lib/librte_eventdev/rte_eventmode_helper_prints.c

-- 
2.7.4

[dpdk-dev] [PATCH 01/39] examples/l2fwd-event: create copy of l2fwd

[Anoob Joseph]

Creating a copy of l2fwd for event additions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/Makefile                |   1 +
 examples/l2fwd-event/Makefile    |  55 +++
 examples/l2fwd-event/main.c      | 752 +++++++++++++++++++++++++++++++++++++++
 examples/l2fwd-event/meson.build |  11 +
 4 files changed, 819 insertions(+)
 create mode 100644 examples/l2fwd-event/Makefile
 create mode 100644 examples/l2fwd-event/main.c
 create mode 100644 examples/l2fwd-event/meson.build

diff --git a/examples/Makefile b/examples/Makefile
index 7562424..aa718ef 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -38,6 +38,7 @@ ifneq ($(PQOS_INSTALL_PATH),)
 DIRS-y += l2fwd-cat
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += l2fwd-crypto
+DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += l2fwd-event
 DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += l2fwd-jobstats
 DIRS-y += l2fwd-keepalive
 DIRS-y += l2fwd-keepalive/ka-agent
diff --git a/examples/l2fwd-event/Makefile b/examples/l2fwd-event/Makefile
new file mode 100644
index 0000000..398f7a1
--- /dev/null
+++ b/examples/l2fwd-event/Makefile
@@ -0,0 +1,55 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+
+# binary name
+APP = l2fwd-event
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+# Build using pkg-config variables if possible
+$(shell pkg-config --exists libdpdk)
+ifeq ($(.SHELLSTATUS),0)
+
+all: shared
+.PHONY: shared static
+shared: build/$(APP)-shared
+	ln -sf $(APP)-shared build/$(APP)
+static: build/$(APP)-static
+	ln -sf $(APP)-static build/$(APP)
+
+PC_FILE := $(shell pkg-config --path libdpdk)
+CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
+LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
+
+build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
+
+build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
+
+build:
+	@mkdir -p $@
+
+.PHONY: clean
+clean:
+	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
+	rmdir --ignore-fail-on-non-empty build
+
+else # Build using legacy build system
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+endif
diff --git a/examples/l2fwd-event/main.c b/examples/l2fwd-event/main.c
new file mode 100644
index 0000000..ef51dea
--- /dev/null
+++ b/examples/l2fwd-event/main.c
@@ -0,0 +1,752 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright (C) 2019 Marvell International Ltd.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <sys/types.h>
+#include <sys/queue.h>
+#include <netinet/in.h>
+#include <setjmp.h>
+#include <stdarg.h>
+#include <ctype.h>
+#include <errno.h>
+#include <getopt.h>
+#include <signal.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_memory.h>
+#include <rte_memcpy.h>
+#include <rte_eal.h>
+#include <rte_launch.h>
+#include <rte_atomic.h>
+#include <rte_cycles.h>
+#include <rte_prefetch.h>
+#include <rte_lcore.h>
+#include <rte_per_lcore.h>
+#include <rte_branch_prediction.h>
+#include <rte_interrupts.h>
+#include <rte_random.h>
+#include <rte_debug.h>
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+
+static volatile bool force_quit;
+
+/* MAC updating enabled by default */
+static int mac_updating = 1;
+
+#define RTE_LOGTYPE_L2FWD RTE_LOGTYPE_USER1
+
+#define MAX_PKT_BURST 32
+#define BURST_TX_DRAIN_US 100 /* TX drain every ~100us */
+#define MEMPOOL_CACHE_SIZE 256
+
+/*
+ * Configurable number of RX/TX ring descriptors
+ */
+#define RTE_TEST_RX_DESC_DEFAULT 1024
+#define RTE_TEST_TX_DESC_DEFAULT 1024
+static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT;
+static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;
+
+/* ethernet addresses of ports */
+static struct rte_ether_addr l2fwd_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+/* mask of enabled ports */
+static uint32_t l2fwd_enabled_port_mask;
+
+/* list of enabled ports */
+static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS];
+
+static unsigned int l2fwd_rx_queue_per_lcore = 1;
+
+#define MAX_RX_QUEUE_PER_LCORE 16
+#define MAX_TX_QUEUE_PER_PORT 16
+struct lcore_queue_conf {
+	unsigned int n_rx_port;
+	unsigned int rx_port_list[MAX_RX_QUEUE_PER_LCORE];
+} __rte_cache_aligned;
+struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];
+
+static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
+
+static struct rte_eth_conf port_conf = {
+	.rxmode = {
+		.split_hdr_size = 0,
+	},
+	.txmode = {
+		.mq_mode = ETH_MQ_TX_NONE,
+	},
+};
+
+struct rte_mempool *l2fwd_pktmbuf_pool;
+
+/* Per-port statistics struct */
+struct l2fwd_port_statistics {
+	uint64_t tx;
+	uint64_t rx;
+	uint64_t dropped;
+} __rte_cache_aligned;
+struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS];
+
+#define MAX_TIMER_PERIOD 86400 /* 1 day max */
+/* A tsc-based timer responsible for triggering statistics printout */
+static uint64_t timer_period = 10; /* default period is 10 seconds */
+
+/* Print out statistics on packets dropped */
+static void
+print_stats(void)
+{
+	uint64_t total_packets_dropped, total_packets_tx, total_packets_rx;
+	unsigned int portid;
+
+	total_packets_dropped = 0;
+	total_packets_tx = 0;
+	total_packets_rx = 0;
+
+	const char clr[] = { 27, '[', '2', 'J', '\0' };
+	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+		/* Clear screen and move to top left */
+	printf("%s%s", clr, topLeft);
+
+	printf("\nPort statistics ====================================");
+
+	for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) {
+		/* skip disabled ports */
+		if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
+			continue;
+		printf("\nStatistics for port %u ------------------------------"
+			   "\nPackets sent: %24"PRIu64
+			   "\nPackets received: %20"PRIu64
+			   "\nPackets dropped: %21"PRIu64,
+			   portid,
+			   port_statistics[portid].tx,
+			   port_statistics[portid].rx,
+			   port_statistics[portid].dropped);
+
+		total_packets_dropped += port_statistics[portid].dropped;
+		total_packets_tx += port_statistics[portid].tx;
+		total_packets_rx += port_statistics[portid].rx;
+	}
+	printf("\nAggregate statistics ==============================="
+		   "\nTotal packets sent: %18"PRIu64
+		   "\nTotal packets received: %14"PRIu64
+		   "\nTotal packets dropped: %15"PRIu64,
+		   total_packets_tx,
+		   total_packets_rx,
+		   total_packets_dropped);
+	printf("\n====================================================\n");
+}
+
+static void
+l2fwd_mac_updating(struct rte_mbuf *m, unsigned int dest_portid)
+{
+	struct rte_ether_hdr *eth;
+	void *tmp;
+
+	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
+
+	/* 02:00:00:00:00:xx */
+	tmp = &eth->d_addr.addr_bytes[0];
+	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
+
+	/* src addr */
+	rte_ether_addr_copy(&l2fwd_ports_eth_addr[dest_portid], &eth->s_addr);
+}
+
+static void
+l2fwd_simple_forward(struct rte_mbuf *m, unsigned int portid)
+{
+	unsigned int dst_port;
+	int sent;
+	struct rte_eth_dev_tx_buffer *buffer;
+
+	dst_port = l2fwd_dst_ports[portid];
+
+	if (mac_updating)
+		l2fwd_mac_updating(m, dst_port);
+
+	buffer = tx_buffer[dst_port];
+	sent = rte_eth_tx_buffer(dst_port, 0, buffer, m);
+	if (sent)
+		port_statistics[dst_port].tx += sent;
+}
+
+/* main processing loop */
+static void
+l2fwd_main_loop(void)
+{
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+	struct rte_mbuf *m;
+	int sent;
+	unsigned int lcore_id;
+	uint64_t prev_tsc, diff_tsc, cur_tsc, timer_tsc;
+	unsigned int i, j, portid, nb_rx;
+	struct lcore_queue_conf *qconf;
+	const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1)
+			/ US_PER_S * BURST_TX_DRAIN_US;
+	struct rte_eth_dev_tx_buffer *buffer;
+
+	prev_tsc = 0;
+	timer_tsc = 0;
+
+	lcore_id = rte_lcore_id();
+	qconf = &lcore_queue_conf[lcore_id];
+
+	if (qconf->n_rx_port == 0) {
+		RTE_LOG(INFO, L2FWD, "lcore %u has nothing to do\n", lcore_id);
+		return;
+	}
+
+	RTE_LOG(INFO, L2FWD, "entering main loop on lcore %u\n", lcore_id);
+
+	for (i = 0; i < qconf->n_rx_port; i++) {
+
+		portid = qconf->rx_port_list[i];
+		RTE_LOG(INFO, L2FWD, " -- lcoreid=%u portid=%u\n", lcore_id,
+			portid);
+
+	}
+
+	while (!force_quit) {
+
+		cur_tsc = rte_rdtsc();
+
+		/*
+		 * TX burst queue drain
+		 */
+		diff_tsc = cur_tsc - prev_tsc;
+		if (unlikely(diff_tsc > drain_tsc)) {
+
+			for (i = 0; i < qconf->n_rx_port; i++) {
+
+				portid =
+					l2fwd_dst_ports[qconf->rx_port_list[i]];
+				buffer = tx_buffer[portid];
+
+				sent = rte_eth_tx_buffer_flush(portid, 0,
+							       buffer);
+				if (sent)
+					port_statistics[portid].tx += sent;
+
+			}
+
+			/* if timer is enabled */
+			if (timer_period > 0) {
+
+				/* advance the timer */
+				timer_tsc += diff_tsc;
+
+				/* if timer has reached its timeout */
+				if (unlikely(timer_tsc >= timer_period)) {
+
+					/* do this only on master core */
+					if (lcore_id ==
+					    rte_get_master_lcore()) {
+						print_stats();
+						/* reset the timer */
+						timer_tsc = 0;
+					}
+				}
+			}
+
+			prev_tsc = cur_tsc;
+		}
+
+		/*
+		 * Read packet from RX queues
+		 */
+		for (i = 0; i < qconf->n_rx_port; i++) {
+
+			portid = qconf->rx_port_list[i];
+			nb_rx = rte_eth_rx_burst(portid, 0,
+						 pkts_burst, MAX_PKT_BURST);
+
+			port_statistics[portid].rx += nb_rx;
+
+			for (j = 0; j < nb_rx; j++) {
+				m = pkts_burst[j];
+				rte_prefetch0(rte_pktmbuf_mtod(m, void *));
+				l2fwd_simple_forward(m, portid);
+			}
+		}
+	}
+}
+
+static int
+l2fwd_launch_one_lcore(__attribute__((unused)) void *dummy)
+{
+	l2fwd_main_loop();
+	return 0;
+}
+
+/* display usage */
+static void
+l2fwd_usage(const char *prgname)
+{
+	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
+	       "  -p PORTMASK: hexadecimal bitmask of ports to configure\n"
+	       "  -q NQ: number of queue (=ports) per lcore (default is 1)\n"
+		   "  -T PERIOD: statistics will be refreshed each PERIOD seconds (0 to disable, 10 default, 86400 maximum)\n"
+		   "  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
+		   "      When enabled:\n"
+		   "       - The source MAC address is replaced by the TX port MAC address\n"
+		   "       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n",
+	       prgname);
+}
+
+static int
+l2fwd_parse_portmask(const char *portmask)
+{
+	char *end = NULL;
+	unsigned long pm;
+
+	/* parse hexadecimal string */
+	pm = strtoul(portmask, &end, 16);
+	if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+
+	if (pm == 0)
+		return -1;
+
+	return pm;
+}
+
+static unsigned int
+l2fwd_parse_nqueue(const char *q_arg)
+{
+	char *end = NULL;
+	unsigned long n;
+
+	/* parse hexadecimal string */
+	n = strtoul(q_arg, &end, 10);
+	if ((q_arg[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return 0;
+	if (n == 0)
+		return 0;
+	if (n >= MAX_RX_QUEUE_PER_LCORE)
+		return 0;
+
+	return n;
+}
+
+static int
+l2fwd_parse_timer_period(const char *q_arg)
+{
+	char *end = NULL;
+	int n;
+
+	/* parse number string */
+	n = strtol(q_arg, &end, 10);
+	if ((q_arg[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+	if (n >= MAX_TIMER_PERIOD)
+		return -1;
+
+	return n;
+}
+
+static const char short_options[] =
+	"p:"  /* portmask */
+	"q:"  /* number of queues */
+	"T:"  /* timer period */
+	;
+
+#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
+#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
+
+enum {
+	/* long options mapped to a short option */
+
+	/* first long only option value must be >= 256, so that we won't
+	 * conflict with short options
+	 */
+	CMD_LINE_OPT_MIN_NUM = 256,
+};
+
+static const struct option lgopts[] = {
+	{ CMD_LINE_OPT_MAC_UPDATING, no_argument, &mac_updating, 1},
+	{ CMD_LINE_OPT_NO_MAC_UPDATING, no_argument, &mac_updating, 0},
+	{NULL, 0, 0, 0}
+};
+
+/* Parse the argument given in the command line of the application */
+static int
+l2fwd_parse_args(int argc, char **argv)
+{
+	int opt, ret, timer_secs;
+	char **argvopt;
+	int option_index;
+	char *prgname = argv[0];
+
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, short_options,
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		/* portmask */
+		case 'p':
+			l2fwd_enabled_port_mask = l2fwd_parse_portmask(optarg);
+			if (l2fwd_enabled_port_mask == 0) {
+				printf("invalid portmask\n");
+				l2fwd_usage(prgname);
+				return -1;
+			}
+			break;
+
+		/* nqueue */
+		case 'q':
+			l2fwd_rx_queue_per_lcore = l2fwd_parse_nqueue(optarg);
+			if (l2fwd_rx_queue_per_lcore == 0) {
+				printf("invalid queue number\n");
+				l2fwd_usage(prgname);
+				return -1;
+			}
+			break;
+
+		/* timer period */
+		case 'T':
+			timer_secs = l2fwd_parse_timer_period(optarg);
+			if (timer_secs < 0) {
+				printf("invalid timer period\n");
+				l2fwd_usage(prgname);
+				return -1;
+			}
+			timer_period = timer_secs;
+			break;
+
+		/* long options */
+		case 0:
+			break;
+
+		default:
+			l2fwd_usage(prgname);
+			return -1;
+		}
+	}
+
+	if (optind >= 0)
+		argv[optind-1] = prgname;
+
+	ret = optind-1;
+	optind = 1; /* reset getopt lib */
+	return ret;
+}
+
+/* Check the link status of all ports in up to 9s, and print them finally */
+static void
+check_all_ports_link_status(uint32_t port_mask)
+{
+#define CHECK_INTERVAL 100 /* 100ms */
+#define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */
+	uint16_t portid;
+	uint8_t count, all_ports_up, print_flag = 0;
+	struct rte_eth_link link;
+
+	printf("\nChecking link status");
+	fflush(stdout);
+	for (count = 0; count <= MAX_CHECK_TIME; count++) {
+		if (force_quit)
+			return;
+		all_ports_up = 1;
+		RTE_ETH_FOREACH_DEV(portid) {
+			if (force_quit)
+				return;
+			if ((port_mask & (1 << portid)) == 0)
+				continue;
+			memset(&link, 0, sizeof(link));
+			rte_eth_link_get_nowait(portid, &link);
+			/* print link status if flag set */
+			if (print_flag == 1) {
+				if (link.link_status)
+					printf(
+					"Port%d Link Up. Speed %u Mbps - %s\n",
+						portid, link.link_speed,
+				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+					("full-duplex") : ("half-duplex\n"));
+				else
+					printf("Port %d Link Down\n", portid);
+				continue;
+			}
+			/* clear all_ports_up flag if any link down */
+			if (link.link_status == ETH_LINK_DOWN) {
+				all_ports_up = 0;
+				break;
+			}
+		}
+		/* after finally printing all link status, get out */
+		if (print_flag == 1)
+			break;
+
+		if (all_ports_up == 0) {
+			printf(".");
+			fflush(stdout);
+			rte_delay_ms(CHECK_INTERVAL);
+		}
+
+		/* set the print_flag if all ports up or timeout */
+		if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) {
+			print_flag = 1;
+			printf("done\n");
+		}
+	}
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\n\nSignal %d received, preparing to exit...\n",
+				signum);
+		force_quit = true;
+	}
+}
+
+int
+main(int argc, char **argv)
+{
+	struct lcore_queue_conf *qconf;
+	int ret;
+	uint16_t nb_ports;
+	uint16_t nb_ports_available = 0;
+	uint16_t portid, last_port;
+	unsigned int lcore_id, rx_lcore_id;
+	unsigned int nb_ports_in_mask = 0;
+	unsigned int nb_lcores = 0;
+	unsigned int nb_mbufs;
+
+	/* init EAL */
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+	argc -= ret;
+	argv += ret;
+
+	force_quit = false;
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	/* parse application arguments (after the EAL ones) */
+	ret = l2fwd_parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid L2FWD arguments\n");
+
+	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
+
+	/* convert to number of cycles */
+	timer_period *= rte_get_timer_hz();
+
+	nb_ports = rte_eth_dev_count_avail();
+	if (nb_ports == 0)
+		rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
+
+	/* check port mask to possible port mask */
+	if (l2fwd_enabled_port_mask & ~((1 << nb_ports) - 1))
+		rte_exit(EXIT_FAILURE, "Invalid portmask; possible (0x%x)\n",
+			(1 << nb_ports) - 1);
+
+	/* reset l2fwd_dst_ports */
+	for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++)
+		l2fwd_dst_ports[portid] = 0;
+	last_port = 0;
+
+	/*
+	 * Each logical core is assigned a dedicated TX queue on each port.
+	 */
+	RTE_ETH_FOREACH_DEV(portid) {
+		/* skip ports that are not enabled */
+		if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
+			continue;
+
+		if (nb_ports_in_mask % 2) {
+			l2fwd_dst_ports[portid] = last_port;
+			l2fwd_dst_ports[last_port] = portid;
+		} else
+			last_port = portid;
+
+		nb_ports_in_mask++;
+	}
+	if (nb_ports_in_mask % 2) {
+		printf("Notice: odd number of ports in portmask.\n");
+		l2fwd_dst_ports[last_port] = last_port;
+	}
+
+	rx_lcore_id = 0;
+	qconf = NULL;
+
+	/* Initialize the port/queue configuration of each logical core */
+	RTE_ETH_FOREACH_DEV(portid) {
+		/* skip ports that are not enabled */
+		if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
+			continue;
+
+		/* get the lcore_id for this port */
+		while (rte_lcore_is_enabled(rx_lcore_id) == 0 ||
+		       lcore_queue_conf[rx_lcore_id].n_rx_port ==
+		       l2fwd_rx_queue_per_lcore) {
+			rx_lcore_id++;
+			if (rx_lcore_id >= RTE_MAX_LCORE)
+				rte_exit(EXIT_FAILURE, "Not enough cores\n");
+		}
+
+		if (qconf != &lcore_queue_conf[rx_lcore_id]) {
+			/* Assigned a new logical core in the loop above. */
+			qconf = &lcore_queue_conf[rx_lcore_id];
+			nb_lcores++;
+		}
+
+		qconf->rx_port_list[qconf->n_rx_port] = portid;
+		qconf->n_rx_port++;
+		printf("Lcore %u: RX port %u\n", rx_lcore_id, portid);
+	}
+
+	nb_mbufs = RTE_MAX(nb_ports * (nb_rxd + nb_txd + MAX_PKT_BURST +
+		nb_lcores * MEMPOOL_CACHE_SIZE), 8192U);
+
+	/* create the mbuf pool */
+	l2fwd_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+		MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+		rte_socket_id());
+	if (l2fwd_pktmbuf_pool == NULL)
+		rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+	/* Initialise each port */
+	RTE_ETH_FOREACH_DEV(portid) {
+		struct rte_eth_rxconf rxq_conf;
+		struct rte_eth_txconf txq_conf;
+		struct rte_eth_conf local_port_conf = port_conf;
+		struct rte_eth_dev_info dev_info;
+
+		/* skip ports that are not enabled */
+		if ((l2fwd_enabled_port_mask & (1 << portid)) == 0) {
+			printf("Skipping disabled port %u\n", portid);
+			continue;
+		}
+		nb_ports_available++;
+
+		/* init port */
+		printf("Initializing port %u... ", portid);
+		fflush(stdout);
+		rte_eth_dev_info_get(portid, &dev_info);
+		if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+			local_port_conf.txmode.offloads |=
+				DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+		ret = rte_eth_dev_configure(portid, 1, 1, &local_port_conf);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE, "Cannot configure device: err=%d, port=%u\n",
+				  ret, portid);
+
+		ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+						       &nb_txd);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE,
+				 "Cannot adjust number of descriptors: err=%d, port=%u\n",
+				 ret, portid);
+
+		rte_eth_macaddr_get(portid, &l2fwd_ports_eth_addr[portid]);
+
+		/* init one RX queue */
+		fflush(stdout);
+		rxq_conf = dev_info.default_rxconf;
+		rxq_conf.offloads = local_port_conf.rxmode.offloads;
+		ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd,
+					     rte_eth_dev_socket_id(portid),
+					     &rxq_conf,
+					     l2fwd_pktmbuf_pool);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup:err=%d, port=%u\n",
+				  ret, portid);
+
+		/* init one TX queue on each port */
+		fflush(stdout);
+		txq_conf = dev_info.default_txconf;
+		txq_conf.offloads = local_port_conf.txmode.offloads;
+		ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+				rte_eth_dev_socket_id(portid),
+				&txq_conf);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d, port=%u\n",
+				ret, portid);
+
+		/* Initialize TX buffers */
+		tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+				RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+				rte_eth_dev_socket_id(portid));
+		if (tx_buffer[portid] == NULL)
+			rte_exit(EXIT_FAILURE, "Cannot allocate buffer for tx on port %u\n",
+					portid);
+
+		rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+		ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+				rte_eth_tx_buffer_count_callback,
+				&port_statistics[portid].dropped);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE,
+			"Cannot set error callback for tx buffer on port %u\n",
+				 portid);
+
+		/* Start device */
+		ret = rte_eth_dev_start(portid);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
+				  ret, portid);
+
+		printf("done:\n");
+
+		rte_eth_promiscuous_enable(portid);
+
+		printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+				portid,
+				l2fwd_ports_eth_addr[portid].addr_bytes[0],
+				l2fwd_ports_eth_addr[portid].addr_bytes[1],
+				l2fwd_ports_eth_addr[portid].addr_bytes[2],
+				l2fwd_ports_eth_addr[portid].addr_bytes[3],
+				l2fwd_ports_eth_addr[portid].addr_bytes[4],
+				l2fwd_ports_eth_addr[portid].addr_bytes[5]);
+
+		/* initialize port stats */
+		memset(&port_statistics, 0, sizeof(port_statistics));
+	}
+
+	if (!nb_ports_available) {
+		rte_exit(EXIT_FAILURE,
+			"All available ports are disabled. Please set portmask.\n");
+	}
+
+	check_all_ports_link_status(l2fwd_enabled_port_mask);
+
+	ret = 0;
+	/* launch per-lcore init on every lcore */
+	rte_eal_mp_remote_launch(l2fwd_launch_one_lcore, NULL, CALL_MASTER);
+	RTE_LCORE_FOREACH_SLAVE(lcore_id) {
+		if (rte_eal_wait_lcore(lcore_id) < 0) {
+			ret = -1;
+			break;
+		}
+	}
+
+	RTE_ETH_FOREACH_DEV(portid) {
+		if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
+			continue;
+		printf("Closing port %d...", portid);
+		rte_eth_dev_stop(portid);
+		rte_eth_dev_close(portid);
+		printf(" Done\n");
+	}
+	printf("Bye...\n");
+
+	return ret;
+}
diff --git a/examples/l2fwd-event/meson.build b/examples/l2fwd-event/meson.build
new file mode 100644
index 0000000..c34e11e
--- /dev/null
+++ b/examples/l2fwd-event/meson.build
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+sources = files(
+	'main.c'
+)
-- 
2.7.4

[dpdk-dev] [PATCH 02/39] examples/l2fwd-event: move macros to common header

[Anoob Joseph]

Moving macros to common header.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_common.h | 25 +++++++++++++++++++++++++
 examples/l2fwd-event/main.c         | 16 ++--------------
 2 files changed, 27 insertions(+), 14 deletions(-)
 create mode 100644 examples/l2fwd-event/l2fwd_common.h

diff --git a/examples/l2fwd-event/l2fwd_common.h b/examples/l2fwd-event/l2fwd_common.h
new file mode 100644
index 0000000..ce13bbb
--- /dev/null
+++ b/examples/l2fwd-event/l2fwd_common.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright (C) 2019 Marvell International Ltd.
+ */
+#ifndef _L2FWD_COMMON_H_
+#define _L2FWD_COMMON_H_
+
+#define RTE_LOGTYPE_L2FWD RTE_LOGTYPE_USER1
+
+#define MAX_PKT_BURST 32
+#define BURST_TX_DRAIN_US 100 /* TX drain every ~100us */
+#define MEMPOOL_CACHE_SIZE 256
+
+/*
+ * Configurable number of RX/TX ring descriptors
+ */
+#define RTE_TEST_RX_DESC_DEFAULT 1024
+#define RTE_TEST_TX_DESC_DEFAULT 1024
+
+#define MAX_RX_QUEUE_PER_LCORE 16
+#define MAX_TX_QUEUE_PER_PORT 16
+
+#define MAX_TIMER_PERIOD 86400 /* 1 day max */
+
+#endif /* _L2FWD_COMMON_H_ */
diff --git a/examples/l2fwd-event/main.c b/examples/l2fwd-event/main.c
index ef51dea..9514485 100644
--- a/examples/l2fwd-event/main.c
+++ b/examples/l2fwd-event/main.c
@@ -40,22 +40,13 @@
 #include <rte_mempool.h>
 #include <rte_mbuf.h>
 
+#include "l2fwd_common.h"
+
 static volatile bool force_quit;
 
 /* MAC updating enabled by default */
 static int mac_updating = 1;
 
-#define RTE_LOGTYPE_L2FWD RTE_LOGTYPE_USER1
-
-#define MAX_PKT_BURST 32
-#define BURST_TX_DRAIN_US 100 /* TX drain every ~100us */
-#define MEMPOOL_CACHE_SIZE 256
-
-/*
- * Configurable number of RX/TX ring descriptors
- */
-#define RTE_TEST_RX_DESC_DEFAULT 1024
-#define RTE_TEST_TX_DESC_DEFAULT 1024
 static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT;
 static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;
 
@@ -70,8 +61,6 @@ static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS];
 
 static unsigned int l2fwd_rx_queue_per_lcore = 1;
 
-#define MAX_RX_QUEUE_PER_LCORE 16
-#define MAX_TX_QUEUE_PER_PORT 16
 struct lcore_queue_conf {
 	unsigned int n_rx_port;
 	unsigned int rx_port_list[MAX_RX_QUEUE_PER_LCORE];
@@ -99,7 +88,6 @@ struct l2fwd_port_statistics {
 } __rte_cache_aligned;
 struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS];
 
-#define MAX_TIMER_PERIOD 86400 /* 1 day max */
 /* A tsc-based timer responsible for triggering statistics printout */
 static uint64_t timer_period = 10; /* default period is 10 seconds */
 
-- 
2.7.4

[dpdk-dev] [PATCH 03/39] examples/l2fwd-event: move structures to common header

[Anoob Joseph]

Moving structures to common header.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_common.h | 12 ++++++++++++
 examples/l2fwd-event/main.c         | 10 ----------
 2 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/examples/l2fwd-event/l2fwd_common.h b/examples/l2fwd-event/l2fwd_common.h
index ce13bbb..a7bb5af 100644
--- a/examples/l2fwd-event/l2fwd_common.h
+++ b/examples/l2fwd-event/l2fwd_common.h
@@ -22,4 +22,16 @@
 
 #define MAX_TIMER_PERIOD 86400 /* 1 day max */
 
+struct lcore_queue_conf {
+	unsigned int n_rx_port;
+	unsigned int rx_port_list[MAX_RX_QUEUE_PER_LCORE];
+} __rte_cache_aligned;
+
+/* Per-port statistics struct */
+struct l2fwd_port_statistics {
+	uint64_t tx;
+	uint64_t rx;
+	uint64_t dropped;
+} __rte_cache_aligned;
+
 #endif /* _L2FWD_COMMON_H_ */
diff --git a/examples/l2fwd-event/main.c b/examples/l2fwd-event/main.c
index 9514485..1551c7f 100644
--- a/examples/l2fwd-event/main.c
+++ b/examples/l2fwd-event/main.c
@@ -61,10 +61,6 @@ static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS];
 
 static unsigned int l2fwd_rx_queue_per_lcore = 1;
 
-struct lcore_queue_conf {
-	unsigned int n_rx_port;
-	unsigned int rx_port_list[MAX_RX_QUEUE_PER_LCORE];
-} __rte_cache_aligned;
 struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];
 
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
@@ -80,12 +76,6 @@ static struct rte_eth_conf port_conf = {
 
 struct rte_mempool *l2fwd_pktmbuf_pool;
 
-/* Per-port statistics struct */
-struct l2fwd_port_statistics {
-	uint64_t tx;
-	uint64_t rx;
-	uint64_t dropped;
-} __rte_cache_aligned;
 struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS];
 
 /* A tsc-based timer responsible for triggering statistics printout */
-- 
2.7.4

[dpdk-dev] [PATCH 04/39] examples/l2fwd-event: move global vars to common header

[Anoob Joseph]

Moving global variables to common header for access from control plane and
data plane code.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_common.h | 26 +++++++++++++++++++++++
 examples/l2fwd-event/main.c         | 41 +++++++++++++++----------------------
 2 files changed, 43 insertions(+), 24 deletions(-)

diff --git a/examples/l2fwd-event/l2fwd_common.h b/examples/l2fwd-event/l2fwd_common.h
index a7bb5af..55226f7 100644
--- a/examples/l2fwd-event/l2fwd_common.h
+++ b/examples/l2fwd-event/l2fwd_common.h
@@ -5,6 +5,10 @@
 #ifndef _L2FWD_COMMON_H_
 #define _L2FWD_COMMON_H_
 
+#include <stdbool.h>
+
+#include <rte_common.h>
+
 #define RTE_LOGTYPE_L2FWD RTE_LOGTYPE_USER1
 
 #define MAX_PKT_BURST 32
@@ -34,4 +38,26 @@ struct l2fwd_port_statistics {
 	uint64_t dropped;
 } __rte_cache_aligned;
 
+volatile bool force_quit;
+
+int mac_updating;
+
+/* ethernet addresses of ports */
+static struct rte_ether_addr l2fwd_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+/* mask of enabled ports */
+static uint32_t l2fwd_enabled_port_mask;
+
+/* list of enabled ports */
+static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS];
+
+struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];
+
+struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
+
+struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS];
+
+/* A tsc-based timer responsible for triggering statistics printout */
+uint64_t timer_period;
+
 #endif /* _L2FWD_COMMON_H_ */
diff --git a/examples/l2fwd-event/main.c b/examples/l2fwd-event/main.c
index 1551c7f..67f2bb0 100644
--- a/examples/l2fwd-event/main.c
+++ b/examples/l2fwd-event/main.c
@@ -42,29 +42,11 @@
 
 #include "l2fwd_common.h"
 
-static volatile bool force_quit;
-
-/* MAC updating enabled by default */
-static int mac_updating = 1;
-
 static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT;
 static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;
 
-/* ethernet addresses of ports */
-static struct rte_ether_addr l2fwd_ports_eth_addr[RTE_MAX_ETHPORTS];
-
-/* mask of enabled ports */
-static uint32_t l2fwd_enabled_port_mask;
-
-/* list of enabled ports */
-static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS];
-
 static unsigned int l2fwd_rx_queue_per_lcore = 1;
 
-struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];
-
-static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
-
 static struct rte_eth_conf port_conf = {
 	.rxmode = {
 		.split_hdr_size = 0,
@@ -76,11 +58,6 @@ static struct rte_eth_conf port_conf = {
 
 struct rte_mempool *l2fwd_pktmbuf_pool;
 
-struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS];
-
-/* A tsc-based timer responsible for triggering statistics printout */
-static uint64_t timer_period = 10; /* default period is 10 seconds */
-
 /* Print out statistics on packets dropped */
 static void
 print_stats(void)
@@ -492,6 +469,20 @@ signal_handler(int signum)
 	}
 }
 
+static void
+l2fwd_init_global_vars(void)
+{
+	force_quit = false;
+
+	/* MAC updating enabled by default */
+	mac_updating = 1;
+
+	/* Default period is 10 seconds */
+	timer_period = 10;
+
+	l2fwd_enabled_port_mask = 0;
+}
+
 int
 main(int argc, char **argv)
 {
@@ -505,6 +496,9 @@ main(int argc, char **argv)
 	unsigned int nb_lcores = 0;
 	unsigned int nb_mbufs;
 
+	/* Set default values for global vars */
+	l2fwd_init_global_vars();
+
 	/* init EAL */
 	ret = rte_eal_init(argc, argv);
 	if (ret < 0)
@@ -512,7 +506,6 @@ main(int argc, char **argv)
 	argc -= ret;
 	argv += ret;
 
-	force_quit = false;
 	signal(SIGINT, signal_handler);
 	signal(SIGTERM, signal_handler);
 
-- 
2.7.4

[dpdk-dev] [PATCH 05/39] examples/l2fwd-event: move dataplane code to new file

[Anoob Joseph]

Splitting control path and data path code to two different files.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/Makefile       |   1 +
 examples/l2fwd-event/l2fwd_worker.c | 231 ++++++++++++++++++++++++++++++++++++
 examples/l2fwd-event/l2fwd_worker.h |  10 ++
 examples/l2fwd-event/main.c         | 189 +----------------------------
 examples/l2fwd-event/meson.build    |   1 +
 5 files changed, 244 insertions(+), 188 deletions(-)
 create mode 100644 examples/l2fwd-event/l2fwd_worker.c
 create mode 100644 examples/l2fwd-event/l2fwd_worker.h

diff --git a/examples/l2fwd-event/Makefile b/examples/l2fwd-event/Makefile
index 398f7a1..d6bdb9e 100644
--- a/examples/l2fwd-event/Makefile
+++ b/examples/l2fwd-event/Makefile
@@ -6,6 +6,7 @@ APP = l2fwd-event
 
 # all source are stored in SRCS-y
 SRCS-y := main.c
+SRCS-y += l2fwd_worker.c
 
 # Build using pkg-config variables if possible
 $(shell pkg-config --exists libdpdk)
diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
new file mode 100644
index 0000000..167fe39
--- /dev/null
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -0,0 +1,231 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright (C) 2019 Marvell International Ltd.
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <sys/types.h>
+#include <sys/queue.h>
+#include <netinet/in.h>
+#include <setjmp.h>
+#include <stdarg.h>
+#include <ctype.h>
+#include <errno.h>
+#include <getopt.h>
+#include <signal.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_memory.h>
+#include <rte_memcpy.h>
+#include <rte_eal.h>
+#include <rte_launch.h>
+#include <rte_atomic.h>
+#include <rte_cycles.h>
+#include <rte_prefetch.h>
+#include <rte_lcore.h>
+#include <rte_per_lcore.h>
+#include <rte_branch_prediction.h>
+#include <rte_interrupts.h>
+#include <rte_random.h>
+#include <rte_debug.h>
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+
+#include "l2fwd_common.h"
+#include "l2fwd_worker.h"
+
+/* Print out statistics on packets dropped */
+static void
+print_stats(void)
+{
+	uint64_t total_packets_dropped, total_packets_tx, total_packets_rx;
+	unsigned int portid;
+
+	total_packets_dropped = 0;
+	total_packets_tx = 0;
+	total_packets_rx = 0;
+
+	const char clr[] = { 27, '[', '2', 'J', '\0' };
+	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+		/* Clear screen and move to top left */
+	printf("%s%s", clr, topLeft);
+
+	printf("\nPort statistics ====================================");
+
+	for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) {
+		/* skip disabled ports */
+		if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
+			continue;
+		printf("\nStatistics for port %u ------------------------------"
+			   "\nPackets sent: %24"PRIu64
+			   "\nPackets received: %20"PRIu64
+			   "\nPackets dropped: %21"PRIu64,
+			   portid,
+			   port_statistics[portid].tx,
+			   port_statistics[portid].rx,
+			   port_statistics[portid].dropped);
+
+		total_packets_dropped += port_statistics[portid].dropped;
+		total_packets_tx += port_statistics[portid].tx;
+		total_packets_rx += port_statistics[portid].rx;
+	}
+	printf("\nAggregate statistics ==============================="
+		   "\nTotal packets sent: %18"PRIu64
+		   "\nTotal packets received: %14"PRIu64
+		   "\nTotal packets dropped: %15"PRIu64,
+		   total_packets_tx,
+		   total_packets_rx,
+		   total_packets_dropped);
+	printf("\n====================================================\n");
+}
+
+static void
+l2fwd_mac_updating(struct rte_mbuf *m, unsigned int dest_portid)
+{
+	struct rte_ether_hdr *eth;
+	void *tmp;
+
+	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
+
+	/* 02:00:00:00:00:xx */
+	tmp = &eth->d_addr.addr_bytes[0];
+	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
+
+	/* src addr */
+	rte_ether_addr_copy(&l2fwd_ports_eth_addr[dest_portid], &eth->s_addr);
+}
+
+static void
+l2fwd_simple_forward(struct rte_mbuf *m, unsigned int portid)
+{
+	unsigned int dst_port;
+	int sent;
+	struct rte_eth_dev_tx_buffer *buffer;
+
+	dst_port = l2fwd_dst_ports[portid];
+
+	if (mac_updating)
+		l2fwd_mac_updating(m, dst_port);
+
+	buffer = tx_buffer[dst_port];
+	sent = rte_eth_tx_buffer(dst_port, 0, buffer, m);
+	if (sent)
+		port_statistics[dst_port].tx += sent;
+}
+
+/* main processing loop */
+static void
+l2fwd_main_loop(void)
+{
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+	struct rte_mbuf *m;
+	int sent;
+	unsigned int lcore_id;
+	uint64_t prev_tsc, diff_tsc, cur_tsc, timer_tsc;
+	unsigned int i, j, portid, nb_rx;
+	struct lcore_queue_conf *qconf;
+	const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1)
+			/ US_PER_S * BURST_TX_DRAIN_US;
+	struct rte_eth_dev_tx_buffer *buffer;
+
+	prev_tsc = 0;
+	timer_tsc = 0;
+
+	lcore_id = rte_lcore_id();
+	qconf = &lcore_queue_conf[lcore_id];
+
+	if (qconf->n_rx_port == 0) {
+		RTE_LOG(INFO, L2FWD, "lcore %u has nothing to do\n", lcore_id);
+		return;
+	}
+
+	RTE_LOG(INFO, L2FWD, "entering main loop on lcore %u\n", lcore_id);
+
+	for (i = 0; i < qconf->n_rx_port; i++) {
+
+		portid = qconf->rx_port_list[i];
+		RTE_LOG(INFO, L2FWD, " -- lcoreid=%u portid=%u\n", lcore_id,
+			portid);
+
+	}
+
+	while (!force_quit) {
+
+		cur_tsc = rte_rdtsc();
+
+		/*
+		 * TX burst queue drain
+		 */
+		diff_tsc = cur_tsc - prev_tsc;
+		if (unlikely(diff_tsc > drain_tsc)) {
+
+			for (i = 0; i < qconf->n_rx_port; i++) {
+
+				portid =
+					l2fwd_dst_ports[qconf->rx_port_list[i]];
+				buffer = tx_buffer[portid];
+
+				sent = rte_eth_tx_buffer_flush(portid, 0,
+							       buffer);
+				if (sent)
+					port_statistics[portid].tx += sent;
+
+			}
+
+			/* if timer is enabled */
+			if (timer_period > 0) {
+
+				/* advance the timer */
+				timer_tsc += diff_tsc;
+
+				/* if timer has reached its timeout */
+				if (unlikely(timer_tsc >= timer_period)) {
+
+					/* do this only on master core */
+					if (lcore_id ==
+					    rte_get_master_lcore()) {
+						print_stats();
+						/* reset the timer */
+						timer_tsc = 0;
+					}
+				}
+			}
+
+			prev_tsc = cur_tsc;
+		}
+
+		/*
+		 * Read packet from RX queues
+		 */
+		for (i = 0; i < qconf->n_rx_port; i++) {
+
+			portid = qconf->rx_port_list[i];
+			nb_rx = rte_eth_rx_burst(portid, 0,
+						 pkts_burst, MAX_PKT_BURST);
+
+			port_statistics[portid].rx += nb_rx;
+
+			for (j = 0; j < nb_rx; j++) {
+				m = pkts_burst[j];
+				rte_prefetch0(rte_pktmbuf_mtod(m, void *));
+				l2fwd_simple_forward(m, portid);
+			}
+		}
+	}
+}
+
+int
+l2fwd_launch_one_lcore(__attribute__((unused)) void *dummy)
+{
+	l2fwd_main_loop();
+	return 0;
+}
diff --git a/examples/l2fwd-event/l2fwd_worker.h b/examples/l2fwd-event/l2fwd_worker.h
new file mode 100644
index 0000000..dc4da5b
--- /dev/null
+++ b/examples/l2fwd-event/l2fwd_worker.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C) 2019 Marvell International Ltd.
+ */
+#ifndef _L2FWD_WORKER_H_
+#define _L2FWD_WORKER_H_
+
+int
+l2fwd_launch_one_lcore(__attribute__((unused)) void *dummy);
+
+#endif /* _L2FWD_WORKER_H_ */
diff --git a/examples/l2fwd-event/main.c b/examples/l2fwd-event/main.c
index 67f2bb0..241a8f2 100644
--- a/examples/l2fwd-event/main.c
+++ b/examples/l2fwd-event/main.c
@@ -41,6 +41,7 @@
 #include <rte_mbuf.h>
 
 #include "l2fwd_common.h"
+#include "l2fwd_worker.h"
 
 static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT;
 static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;
@@ -58,194 +59,6 @@ static struct rte_eth_conf port_conf = {
 
 struct rte_mempool *l2fwd_pktmbuf_pool;
 
-/* Print out statistics on packets dropped */
-static void
-print_stats(void)
-{
-	uint64_t total_packets_dropped, total_packets_tx, total_packets_rx;
-	unsigned int portid;
-
-	total_packets_dropped = 0;
-	total_packets_tx = 0;
-	total_packets_rx = 0;
-
-	const char clr[] = { 27, '[', '2', 'J', '\0' };
-	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
-
-		/* Clear screen and move to top left */
-	printf("%s%s", clr, topLeft);
-
-	printf("\nPort statistics ====================================");
-
-	for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) {
-		/* skip disabled ports */
-		if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
-			continue;
-		printf("\nStatistics for port %u ------------------------------"
-			   "\nPackets sent: %24"PRIu64
-			   "\nPackets received: %20"PRIu64
-			   "\nPackets dropped: %21"PRIu64,
-			   portid,
-			   port_statistics[portid].tx,
-			   port_statistics[portid].rx,
-			   port_statistics[portid].dropped);
-
-		total_packets_dropped += port_statistics[portid].dropped;
-		total_packets_tx += port_statistics[portid].tx;
-		total_packets_rx += port_statistics[portid].rx;
-	}
-	printf("\nAggregate statistics ==============================="
-		   "\nTotal packets sent: %18"PRIu64
-		   "\nTotal packets received: %14"PRIu64
-		   "\nTotal packets dropped: %15"PRIu64,
-		   total_packets_tx,
-		   total_packets_rx,
-		   total_packets_dropped);
-	printf("\n====================================================\n");
-}
-
-static void
-l2fwd_mac_updating(struct rte_mbuf *m, unsigned int dest_portid)
-{
-	struct rte_ether_hdr *eth;
-	void *tmp;
-
-	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
-
-	/* 02:00:00:00:00:xx */
-	tmp = &eth->d_addr.addr_bytes[0];
-	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
-
-	/* src addr */
-	rte_ether_addr_copy(&l2fwd_ports_eth_addr[dest_portid], &eth->s_addr);
-}
-
-static void
-l2fwd_simple_forward(struct rte_mbuf *m, unsigned int portid)
-{
-	unsigned int dst_port;
-	int sent;
-	struct rte_eth_dev_tx_buffer *buffer;
-
-	dst_port = l2fwd_dst_ports[portid];
-
-	if (mac_updating)
-		l2fwd_mac_updating(m, dst_port);
-
-	buffer = tx_buffer[dst_port];
-	sent = rte_eth_tx_buffer(dst_port, 0, buffer, m);
-	if (sent)
-		port_statistics[dst_port].tx += sent;
-}
-
-/* main processing loop */
-static void
-l2fwd_main_loop(void)
-{
-	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
-	struct rte_mbuf *m;
-	int sent;
-	unsigned int lcore_id;
-	uint64_t prev_tsc, diff_tsc, cur_tsc, timer_tsc;
-	unsigned int i, j, portid, nb_rx;
-	struct lcore_queue_conf *qconf;
-	const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1)
-			/ US_PER_S * BURST_TX_DRAIN_US;
-	struct rte_eth_dev_tx_buffer *buffer;
-
-	prev_tsc = 0;
-	timer_tsc = 0;
-
-	lcore_id = rte_lcore_id();
-	qconf = &lcore_queue_conf[lcore_id];
-
-	if (qconf->n_rx_port == 0) {
-		RTE_LOG(INFO, L2FWD, "lcore %u has nothing to do\n", lcore_id);
-		return;
-	}
-
-	RTE_LOG(INFO, L2FWD, "entering main loop on lcore %u\n", lcore_id);
-
-	for (i = 0; i < qconf->n_rx_port; i++) {
-
-		portid = qconf->rx_port_list[i];
-		RTE_LOG(INFO, L2FWD, " -- lcoreid=%u portid=%u\n", lcore_id,
-			portid);
-
-	}
-
-	while (!force_quit) {
-
-		cur_tsc = rte_rdtsc();
-
-		/*
-		 * TX burst queue drain
-		 */
-		diff_tsc = cur_tsc - prev_tsc;
-		if (unlikely(diff_tsc > drain_tsc)) {
-
-			for (i = 0; i < qconf->n_rx_port; i++) {
-
-				portid =
-					l2fwd_dst_ports[qconf->rx_port_list[i]];
-				buffer = tx_buffer[portid];
-
-				sent = rte_eth_tx_buffer_flush(portid, 0,
-							       buffer);
-				if (sent)
-					port_statistics[portid].tx += sent;
-
-			}
-
-			/* if timer is enabled */
-			if (timer_period > 0) {
-
-				/* advance the timer */
-				timer_tsc += diff_tsc;
-
-				/* if timer has reached its timeout */
-				if (unlikely(timer_tsc >= timer_period)) {
-
-					/* do this only on master core */
-					if (lcore_id ==
-					    rte_get_master_lcore()) {
-						print_stats();
-						/* reset the timer */
-						timer_tsc = 0;
-					}
-				}
-			}
-
-			prev_tsc = cur_tsc;
-		}
-
-		/*
-		 * Read packet from RX queues
-		 */
-		for (i = 0; i < qconf->n_rx_port; i++) {
-
-			portid = qconf->rx_port_list[i];
-			nb_rx = rte_eth_rx_burst(portid, 0,
-						 pkts_burst, MAX_PKT_BURST);
-
-			port_statistics[portid].rx += nb_rx;
-
-			for (j = 0; j < nb_rx; j++) {
-				m = pkts_burst[j];
-				rte_prefetch0(rte_pktmbuf_mtod(m, void *));
-				l2fwd_simple_forward(m, portid);
-			}
-		}
-	}
-}
-
-static int
-l2fwd_launch_one_lcore(__attribute__((unused)) void *dummy)
-{
-	l2fwd_main_loop();
-	return 0;
-}
-
 /* display usage */
 static void
 l2fwd_usage(const char *prgname)
diff --git a/examples/l2fwd-event/meson.build b/examples/l2fwd-event/meson.build
index c34e11e..1d2df49 100644
--- a/examples/l2fwd-event/meson.build
+++ b/examples/l2fwd-event/meson.build
@@ -7,5 +7,6 @@
 # DPDK instance, use 'make'
 
 sources = files(
+	'l2fwd_worker.c',
 	'main.c'
 )
-- 
2.7.4

[dpdk-dev] [PATCH 06/39] examples/l2fwd-event: remove unused header includes

[Anoob Joseph]

Retain only the required headers.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_worker.c | 12 ------------
 1 file changed, 12 deletions(-)

diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index 167fe39..942b191 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -13,30 +13,18 @@
 #include <setjmp.h>
 #include <stdarg.h>
 #include <ctype.h>
-#include <errno.h>
-#include <getopt.h>
-#include <signal.h>
 #include <stdbool.h>
 
 #include <rte_common.h>
 #include <rte_log.h>
-#include <rte_malloc.h>
-#include <rte_memory.h>
 #include <rte_memcpy.h>
-#include <rte_eal.h>
-#include <rte_launch.h>
 #include <rte_atomic.h>
 #include <rte_cycles.h>
 #include <rte_prefetch.h>
 #include <rte_lcore.h>
-#include <rte_per_lcore.h>
 #include <rte_branch_prediction.h>
-#include <rte_interrupts.h>
-#include <rte_random.h>
-#include <rte_debug.h>
 #include <rte_ether.h>
 #include <rte_ethdev.h>
-#include <rte_mempool.h>
 #include <rte_mbuf.h>
 
 #include "l2fwd_common.h"
-- 
2.7.4

[dpdk-dev] [PATCH 07/39] examples/l2fwd-event: move drain buffers to new function

[Anoob Joseph]

Modularizing code to aid in smooth integration of eventmode.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_worker.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index 942b191..ebce5c1 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -76,6 +76,24 @@ print_stats(void)
 	printf("\n====================================================\n");
 }
 
+static inline void
+l2fwd_drain_buffers(struct lcore_queue_conf *qconf)
+{
+	unsigned int i, sent;
+	unsigned int portid;
+	struct rte_eth_dev_tx_buffer *buffer;
+
+	for (i = 0; i < qconf->n_rx_port; i++) {
+
+		portid = l2fwd_dst_ports[qconf->rx_port_list[i]];
+		buffer = tx_buffer[portid];
+
+		sent = rte_eth_tx_buffer_flush(portid, 0, buffer);
+		if (sent)
+			port_statistics[portid].tx += sent;
+	}
+}
+
 static void
 l2fwd_mac_updating(struct rte_mbuf *m, unsigned int dest_portid)
 {
@@ -116,14 +134,12 @@ l2fwd_main_loop(void)
 {
 	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
 	struct rte_mbuf *m;
-	int sent;
 	unsigned int lcore_id;
 	uint64_t prev_tsc, diff_tsc, cur_tsc, timer_tsc;
 	unsigned int i, j, portid, nb_rx;
 	struct lcore_queue_conf *qconf;
 	const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1)
 			/ US_PER_S * BURST_TX_DRAIN_US;
-	struct rte_eth_dev_tx_buffer *buffer;
 
 	prev_tsc = 0;
 	timer_tsc = 0;
@@ -156,18 +172,8 @@ l2fwd_main_loop(void)
 		diff_tsc = cur_tsc - prev_tsc;
 		if (unlikely(diff_tsc > drain_tsc)) {
 
-			for (i = 0; i < qconf->n_rx_port; i++) {
-
-				portid =
-					l2fwd_dst_ports[qconf->rx_port_list[i]];
-				buffer = tx_buffer[portid];
-
-				sent = rte_eth_tx_buffer_flush(portid, 0,
-							       buffer);
-				if (sent)
-					port_statistics[portid].tx += sent;
-
-			}
+			/* Drain buffers */
+			l2fwd_drain_buffers(qconf);
 
 			/* if timer is enabled */
 			if (timer_period > 0) {
-- 
2.7.4

[dpdk-dev] [PATCH 08/39] examples/l2fwd-event: optimize check for master core

[Anoob Joseph]

Replacing the check for lcore_id & mastercore_id with the check for a
flag.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_worker.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index ebce5c1..86bdd40 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -140,6 +140,7 @@ l2fwd_main_loop(void)
 	struct lcore_queue_conf *qconf;
 	const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1)
 			/ US_PER_S * BURST_TX_DRAIN_US;
+	int is_master_core;
 
 	prev_tsc = 0;
 	timer_tsc = 0;
@@ -162,6 +163,9 @@ l2fwd_main_loop(void)
 
 	}
 
+	/* Set the flag if master core */
+	is_master_core = (lcore_id == rte_get_master_lcore()) ? 1 : 0;
+
 	while (!force_quit) {
 
 		cur_tsc = rte_rdtsc();
@@ -185,8 +189,7 @@ l2fwd_main_loop(void)
 				if (unlikely(timer_tsc >= timer_period)) {
 
 					/* do this only on master core */
-					if (lcore_id ==
-					    rte_get_master_lcore()) {
+					if (is_master_core) {
 						print_stats();
 						/* reset the timer */
 						timer_tsc = 0;
-- 
2.7.4

[dpdk-dev] [PATCH 09/39] examples/l2fwd-event: move periodic tasks to new func

[Anoob Joseph]

Move the periodic operations (stats flush and drain buffers) to a new
function.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_worker.c | 83 ++++++++++++++++++++-----------------
 examples/l2fwd-event/l2fwd_worker.h |  6 +++
 2 files changed, 52 insertions(+), 37 deletions(-)

diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index 86bdd40..49bbdb6 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -94,6 +94,45 @@ l2fwd_drain_buffers(struct lcore_queue_conf *qconf)
 	}
 }
 
+static inline void
+l2fwd_periodic_drain_stats_monitor(struct lcore_queue_conf *qconf,
+		struct tsc_tracker *t, int is_master_core)
+{
+	uint64_t diff_tsc, cur_tsc;
+
+	cur_tsc = rte_rdtsc();
+
+	/*
+	 * TX burst queue drain
+	 */
+	diff_tsc = cur_tsc - t->prev_tsc;
+	if (unlikely(diff_tsc > t->drain_tsc)) {
+
+		/* Drain buffers */
+		l2fwd_drain_buffers(qconf);
+
+		/* if timer is enabled */
+		if (timer_period > 0) {
+
+			/* advance the timer */
+			t->timer_tsc += diff_tsc;
+
+			/* if timer has reached its timeout */
+			if (unlikely(t->timer_tsc >= timer_period)) {
+
+				/* do this only on master core */
+				if (is_master_core) {
+					print_stats();
+					/* reset the timer */
+					t->timer_tsc = 0;
+				}
+			}
+		}
+
+		t->prev_tsc = cur_tsc;
+	}
+}
+
 static void
 l2fwd_mac_updating(struct rte_mbuf *m, unsigned int dest_portid)
 {
@@ -135,19 +174,18 @@ l2fwd_main_loop(void)
 	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
 	struct rte_mbuf *m;
 	unsigned int lcore_id;
-	uint64_t prev_tsc, diff_tsc, cur_tsc, timer_tsc;
 	unsigned int i, j, portid, nb_rx;
 	struct lcore_queue_conf *qconf;
-	const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1)
-			/ US_PER_S * BURST_TX_DRAIN_US;
 	int is_master_core;
-
-	prev_tsc = 0;
-	timer_tsc = 0;
+	struct tsc_tracker tsc = {0};
 
 	lcore_id = rte_lcore_id();
 	qconf = &lcore_queue_conf[lcore_id];
 
+	/* Set drain tsc */
+	tsc.drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /
+			US_PER_S * BURST_TX_DRAIN_US;
+
 	if (qconf->n_rx_port == 0) {
 		RTE_LOG(INFO, L2FWD, "lcore %u has nothing to do\n", lcore_id);
 		return;
@@ -168,37 +206,8 @@ l2fwd_main_loop(void)
 
 	while (!force_quit) {
 
-		cur_tsc = rte_rdtsc();
-
-		/*
-		 * TX burst queue drain
-		 */
-		diff_tsc = cur_tsc - prev_tsc;
-		if (unlikely(diff_tsc > drain_tsc)) {
-
-			/* Drain buffers */
-			l2fwd_drain_buffers(qconf);
-
-			/* if timer is enabled */
-			if (timer_period > 0) {
-
-				/* advance the timer */
-				timer_tsc += diff_tsc;
-
-				/* if timer has reached its timeout */
-				if (unlikely(timer_tsc >= timer_period)) {
-
-					/* do this only on master core */
-					if (is_master_core) {
-						print_stats();
-						/* reset the timer */
-						timer_tsc = 0;
-					}
-				}
-			}
-
-			prev_tsc = cur_tsc;
-		}
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, &tsc, is_master_core);
 
 		/*
 		 * Read packet from RX queues
diff --git a/examples/l2fwd-event/l2fwd_worker.h b/examples/l2fwd-event/l2fwd_worker.h
index dc4da5b..22d7c5c 100644
--- a/examples/l2fwd-event/l2fwd_worker.h
+++ b/examples/l2fwd-event/l2fwd_worker.h
@@ -4,6 +4,12 @@
 #ifndef _L2FWD_WORKER_H_
 #define _L2FWD_WORKER_H_
 
+struct tsc_tracker {
+	uint64_t prev_tsc;
+	uint64_t timer_tsc;
+	uint64_t drain_tsc;
+};
+
 int
 l2fwd_launch_one_lcore(__attribute__((unused)) void *dummy);
 
-- 
2.7.4

[dpdk-dev] [PATCH 10/39] examples/l2fwd-event: do timer updates only on master

[Anoob Joseph]

The timer updates and checks are required only for stats printing by the
master core. This can be entirely skipped for other cores.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_worker.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index 49bbdb6..58fd5b8 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -111,6 +111,14 @@ l2fwd_periodic_drain_stats_monitor(struct lcore_queue_conf *qconf,
 		/* Drain buffers */
 		l2fwd_drain_buffers(qconf);
 
+		t->prev_tsc = cur_tsc;
+
+		/* Skip the timer based stats prints if not master core */
+		if (!is_master_core)
+			return;
+
+		/* On master core */
+
 		/* if timer is enabled */
 		if (timer_period > 0) {
 
@@ -120,16 +128,13 @@ l2fwd_periodic_drain_stats_monitor(struct lcore_queue_conf *qconf,
 			/* if timer has reached its timeout */
 			if (unlikely(t->timer_tsc >= timer_period)) {
 
-				/* do this only on master core */
-				if (is_master_core) {
-					print_stats();
-					/* reset the timer */
-					t->timer_tsc = 0;
-				}
+				/* Print stats */
+				print_stats();
+
+				/* reset the timer */
+				t->timer_tsc = 0;
 			}
 		}
-
-		t->prev_tsc = cur_tsc;
 	}
 }
 
-- 
2.7.4

[dpdk-dev] [PATCH 11/39] examples/l2fwd-event: move pkt send code to a new func

[Anoob Joseph]

Modularizing code to aid in smooth integration of eventmode.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_worker.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index 58fd5b8..1a7ee2b 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -154,22 +154,30 @@ l2fwd_mac_updating(struct rte_mbuf *m, unsigned int dest_portid)
 	rte_ether_addr_copy(&l2fwd_ports_eth_addr[dest_portid], &eth->s_addr);
 }
 
+static inline void
+l2fwd_send_pkt(struct rte_mbuf *tx_pkt, unsigned int port_id)
+{
+	int sent;
+	struct rte_eth_dev_tx_buffer *buffer;
+
+	buffer = tx_buffer[port_id];
+	sent = rte_eth_tx_buffer(port_id, 0, buffer, tx_pkt);
+	if (sent)
+		port_statistics[port_id].tx += sent;
+}
+
 static void
 l2fwd_simple_forward(struct rte_mbuf *m, unsigned int portid)
 {
 	unsigned int dst_port;
-	int sent;
-	struct rte_eth_dev_tx_buffer *buffer;
 
 	dst_port = l2fwd_dst_ports[portid];
 
 	if (mac_updating)
 		l2fwd_mac_updating(m, dst_port);
 
-	buffer = tx_buffer[dst_port];
-	sent = rte_eth_tx_buffer(dst_port, 0, buffer, m);
-	if (sent)
-		port_statistics[dst_port].tx += sent;
+	/* Send packet */
+	l2fwd_send_pkt(m, dst_port);
 }
 
 /* main processing loop */
-- 
2.7.4

[dpdk-dev] [PATCH 12/39] examples/l2fwd-event: use fprintf in usage print

[Anoob Joseph]

Following the convention of l3fwd, using fprintf instead of printf for
printing usage.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/main.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/examples/l2fwd-event/main.c b/examples/l2fwd-event/main.c
index 241a8f2..3e6ba0f 100644
--- a/examples/l2fwd-event/main.c
+++ b/examples/l2fwd-event/main.c
@@ -63,7 +63,9 @@ struct rte_mempool *l2fwd_pktmbuf_pool;
 static void
 l2fwd_usage(const char *prgname)
 {
-	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
+	fprintf(stderr, "%s [EAL options] --"
+		" -p PORTMASK"
+		" [-q NQ]\n"
 	       "  -p PORTMASK: hexadecimal bitmask of ports to configure\n"
 	       "  -q NQ: number of queue (=ports) per lcore (default is 1)\n"
 		   "  -T PERIOD: statistics will be refreshed each PERIOD seconds (0 to disable, 10 default, 86400 maximum)\n"
-- 
2.7.4

[dpdk-dev] [PATCH 13/39] examples/l2fwd-event: improvements to the usage print

[Anoob Joseph]

Fixed alignment and split the usage print to aid easy addition of
eventmode usage prints.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/main.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/examples/l2fwd-event/main.c b/examples/l2fwd-event/main.c
index 3e6ba0f..4e83b41 100644
--- a/examples/l2fwd-event/main.c
+++ b/examples/l2fwd-event/main.c
@@ -65,15 +65,20 @@ l2fwd_usage(const char *prgname)
 {
 	fprintf(stderr, "%s [EAL options] --"
 		" -p PORTMASK"
-		" [-q NQ]\n"
-	       "  -p PORTMASK: hexadecimal bitmask of ports to configure\n"
-	       "  -q NQ: number of queue (=ports) per lcore (default is 1)\n"
-		   "  -T PERIOD: statistics will be refreshed each PERIOD seconds (0 to disable, 10 default, 86400 maximum)\n"
-		   "  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
-		   "      When enabled:\n"
-		   "       - The source MAC address is replaced by the TX port MAC address\n"
-		   "       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n",
-	       prgname);
+		" [-q NQ]",
+		prgname);
+
+	fprintf(stderr, "\n\n");
+
+	fprintf(stderr,
+		"  -p PORTMASK: hexadecimal bitmask of ports to configure\n"
+		"  -q NQ: number of queue (=ports) per lcore (default is 1)\n"
+		"  -T PERIOD: statistics will be refreshed each PERIOD seconds (0 to disable, 10 default, 86400 maximum)\n"
+		"  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
+		"      When enabled:\n"
+		"       - The source MAC address is replaced by the TX port MAC address\n"
+		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
+		"\n");
 }
 
 static int
-- 
2.7.4

[dpdk-dev] [PATCH 14/39] eventdev: add files for eventmode helper

[Anoob Joseph]

Adding files for eventmode helper

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/Makefile                        |  5 +++--
 lib/librte_eventdev/meson.build                     |  2 ++
 lib/librte_eventdev/rte_eventmode_helper.c          |  7 +++++++
 lib/librte_eventdev/rte_eventmode_helper.h          | 15 +++++++++++++++
 lib/librte_eventdev/rte_eventmode_helper_internal.h |  6 ++++++
 5 files changed, 33 insertions(+), 2 deletions(-)
 create mode 100644 lib/librte_eventdev/rte_eventmode_helper.c
 create mode 100644 lib/librte_eventdev/rte_eventmode_helper.h
 create mode 100644 lib/librte_eventdev/rte_eventmode_helper_internal.h

diff --git a/lib/librte_eventdev/Makefile b/lib/librte_eventdev/Makefile
index 53079f4..de2b322 100644
--- a/lib/librte_eventdev/Makefile
+++ b/lib/librte_eventdev/Makefile
@@ -1,6 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2016 Cavium, Inc
-#
+# Copyright (C) 2019 Marvell International Ltd.
 
 include $(RTE_SDK)/mk/rte.vars.mk
 
@@ -24,6 +23,7 @@ LDLIBS += -lrte_mbuf -lrte_cryptodev -lpthread
 
 # library source files
 SRCS-y += rte_eventdev.c
+SRCS-y += rte_eventmode_helper.c
 SRCS-y += rte_event_ring.c
 SRCS-y += rte_event_eth_rx_adapter.c
 SRCS-y += rte_event_timer_adapter.c
@@ -35,6 +35,7 @@ SYMLINK-y-include += rte_eventdev.h
 SYMLINK-y-include += rte_eventdev_pmd.h
 SYMLINK-y-include += rte_eventdev_pmd_pci.h
 SYMLINK-y-include += rte_eventdev_pmd_vdev.h
+SYMLINK-y-include += rte_eventmode_helper.h
 SYMLINK-y-include += rte_event_ring.h
 SYMLINK-y-include += rte_event_eth_rx_adapter.h
 SYMLINK-y-include += rte_event_timer_adapter.h
diff --git a/lib/librte_eventdev/meson.build b/lib/librte_eventdev/meson.build
index 6cfe60e..73cd89e 100644
--- a/lib/librte_eventdev/meson.build
+++ b/lib/librte_eventdev/meson.build
@@ -11,6 +11,7 @@ else
 endif
 
 sources = files('rte_eventdev.c',
+		'rte_eventmode_helper.c',
 		'rte_event_ring.c',
 		'rte_event_eth_rx_adapter.c',
 		'rte_event_timer_adapter.c',
@@ -20,6 +21,7 @@ headers = files('rte_eventdev.h',
 		'rte_eventdev_pmd.h',
 		'rte_eventdev_pmd_pci.h',
 		'rte_eventdev_pmd_vdev.h',
+		'rte_eventmode_helper.h',
 		'rte_event_ring.h',
 		'rte_event_eth_rx_adapter.h',
 		'rte_event_timer_adapter.h',
diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
new file mode 100644
index 0000000..f47970e
--- /dev/null
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C) 2019 Marvell International Ltd.
+ */
+
+#include <rte_eventmode_helper.h>
+
+#include "rte_eventmode_helper_internal.h"
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
new file mode 100644
index 0000000..d32cd00
--- /dev/null
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C) 2019 Marvell International Ltd.
+ */
+#ifndef _RTE_EVENTMODE_HELPER_H_
+#define _RTE_EVENTMODE_HELPER_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_EVENTMODE_HELPER_H_ */
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
new file mode 100644
index 0000000..0da7003
--- /dev/null
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -0,0 +1,6 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C) 2019 Marvell International Ltd.
+ */
+#ifndef _RTE_EVENTMODE_HELPER_INTERNAL_H_
+#define _RTE_EVENTMODE_HELPER_INTERNAL_H_
+#endif /* _RTE_EVENTMODE_HELPER_INTERNAL_H_ */
-- 
2.7.4

[dpdk-dev] [PATCH 15/39] eventdev: add routines for logging eventmode helper

[Anoob Joseph]

Adding routines for logging eventmode helper.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 config/common_base                                 |  1 +
 lib/librte_eal/common/eal_common_log.c             |  1 +
 lib/librte_eal/common/include/rte_log.h            |  1 +
 .../rte_eventmode_helper_internal.h                | 29 ++++++++++++++++++++++
 4 files changed, 32 insertions(+)

diff --git a/config/common_base b/config/common_base
index 8576973..96c2537 100644
--- a/config/common_base
+++ b/config/common_base
@@ -685,6 +685,7 @@ CONFIG_RTE_LIBRTE_PMD_ZLIB=n
 #
 CONFIG_RTE_LIBRTE_EVENTDEV=y
 CONFIG_RTE_LIBRTE_EVENTDEV_DEBUG=n
+CONFIG_RTE_LIBRTE_EVENTMODE_HELPER_DEBUG=n
 CONFIG_RTE_EVENT_MAX_DEVS=16
 CONFIG_RTE_EVENT_MAX_QUEUES_PER_DEV=64
 CONFIG_RTE_EVENT_TIMER_ADAPTER_NUM_MAX=32
diff --git a/lib/librte_eal/common/eal_common_log.c b/lib/librte_eal/common/eal_common_log.c
index c714a4b..bc34dc2 100644
--- a/lib/librte_eal/common/eal_common_log.c
+++ b/lib/librte_eal/common/eal_common_log.c
@@ -324,6 +324,7 @@ static const struct logtype logtype_strings[] = {
 	{RTE_LOGTYPE_EFD,        "lib.efd"},
 	{RTE_LOGTYPE_EVENTDEV,   "lib.eventdev"},
 	{RTE_LOGTYPE_GSO,        "lib.gso"},
+	{RTE_LOGTYPE_EVENTMODE,  "lib.eventmode"},
 	{RTE_LOGTYPE_USER1,      "user1"},
 	{RTE_LOGTYPE_USER2,      "user2"},
 	{RTE_LOGTYPE_USER3,      "user3"},
diff --git a/lib/librte_eal/common/include/rte_log.h b/lib/librte_eal/common/include/rte_log.h
index cbb4184..78d595c 100644
--- a/lib/librte_eal/common/include/rte_log.h
+++ b/lib/librte_eal/common/include/rte_log.h
@@ -62,6 +62,7 @@ extern struct rte_logs rte_logs;
 #define RTE_LOGTYPE_EFD       18 /**< Log related to EFD. */
 #define RTE_LOGTYPE_EVENTDEV  19 /**< Log related to eventdev. */
 #define RTE_LOGTYPE_GSO       20 /**< Log related to GSO. */
+#define RTE_LOGTYPE_EVENTMODE 21 /**< Log related to eventmode. */
 
 /* these log types can be used in an application */
 #define RTE_LOGTYPE_USER1     24 /**< User-defined log type 1. */
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 0da7003..7af7758 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -3,4 +3,33 @@
  */
 #ifndef _RTE_EVENTMODE_HELPER_INTERNAL_H_
 #define _RTE_EVENTMODE_HELPER_INTERNAL_H_
+
+#include <rte_log.h>
+
+/* Logging macros */
+
+#define RTE_EM_HLPR_LOG_ERR(...) \
+	RTE_LOG(ERR, EVENTMODE, \
+		RTE_FMT("%s(): " RTE_FMT_HEAD(__VA_ARGS__ ,) "\n", \
+			__func__, RTE_FMT_TAIL(__VA_ARGS__ ,)))
+
+#define RTE_EM_HLPR_LOG_WARNING(...) \
+	RTE_LOG(WARNING, EVENTMODE, \
+		RTE_FMT("%s(): " RTE_FMT_HEAD(__VA_ARGS__ ,) "\n", \
+			__func__, RTE_FMT_TAIL(__VA_ARGS__ ,)))
+
+#define RTE_EM_HLPR_LOG_INFO(...) \
+	RTE_LOG(INFO, EVENTMODE, \
+		RTE_FMT(RTE_FMT_HEAD(__VA_ARGS__ ,) "\n", \
+			RTE_FMT_TAIL(__VA_ARGS__ ,)))
+
+#ifdef RTE_LIBRTE_EVENTMODE_HELPER_DEBUG
+#define RTE_EM_HLPR_LOG_DEBUG(...) \
+	RTE_LOG(DEBUG, EVENTMODE, \
+		RTE_FMT("%s() line %u: " RTE_FMT_HEAD(__VA_ARGS__ ,) "\n", \
+			__func__, __LINE__, RTE_FMT_TAIL(__VA_ARGS__ ,)))
+#else
+#define RTE_EM_HLPR_LOG_DEBUG(...) (void)0
+#endif
+
 #endif /* _RTE_EVENTMODE_HELPER_INTERNAL_H_ */
-- 
2.7.4

[dpdk-dev] [PATCH 16/39] eventdev: add eventmode CL options framework

[Anoob Joseph]

Adding usage prints and CL parsing routines for eventmode. Option to
select packet transfer mode is also added.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventdev_version.map |   2 +
 lib/librte_eventdev/rte_eventmode_helper.c   | 128 +++++++++++++++++++++++++++
 lib/librte_eventdev/rte_eventmode_helper.h   |  51 +++++++++++
 3 files changed, 181 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventdev_version.map b/lib/librte_eventdev/rte_eventdev_version.map
index 95fd089..1199064 100644
--- a/lib/librte_eventdev/rte_eventdev_version.map
+++ b/lib/librte_eventdev/rte_eventdev_version.map
@@ -128,4 +128,6 @@ EXPERIMENTAL {
 
 	rte_event_eth_rx_adapter_cb_register;
 	rte_event_eth_rx_adapter_stats_get;
+	rte_eventmode_helper_print_options_list;
+	rte_eventmode_helper_print_options_description;
 };
diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index f47970e..8119306 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -1,7 +1,135 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright (C) 2019 Marvell International Ltd.
  */
+#include <getopt.h>
 
 #include <rte_eventmode_helper.h>
+#include <rte_malloc.h>
 
 #include "rte_eventmode_helper_internal.h"
+
+#define CMD_LINE_OPT_TRANSFER_MODE	"transfer-mode"
+
+static const char short_options[] =
+	""
+	;
+
+enum {
+	/* long options mapped to a short option */
+
+	/* first long only option value must be >= 256, so that we won't
+	 * conflict with short options
+	 */
+	CMD_LINE_OPT_MIN_NUM = 256,
+	CMD_LINE_OPT_TRANSFER_MODE_NUM,
+};
+
+static const struct option lgopts[] = {
+	{CMD_LINE_OPT_TRANSFER_MODE, 1, 0, CMD_LINE_OPT_TRANSFER_MODE_NUM},
+	{NULL, 0, 0, 0}
+};
+
+/* Internal functions */
+
+static int32_t
+internal_parse_decimal(const char *str)
+{
+	char *end = NULL;
+	unsigned long num;
+
+	num = strtoul(str, &end, 10);
+	if ((str[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+
+	return num;
+}
+
+/* Global functions */
+
+void __rte_experimental
+rte_eventmode_helper_print_options_list(void)
+{
+	fprintf(stderr, " --"
+		" [--transfer-mode MODE]"
+		);
+}
+
+void __rte_experimental
+rte_eventmode_helper_print_options_description(void)
+{
+	fprintf(stderr,
+		"  --transfer-mode MODE\n"
+		"               0: Packet transfer via polling (default)\n"
+		"               1: Packet transfer via eventdev\n"
+		"\n");
+}
+
+static int
+em_parse_transfer_mode(struct rte_eventmode_helper_conf *conf,
+		const char *optarg)
+{
+	int32_t parsed_dec;
+
+	parsed_dec = internal_parse_decimal(optarg);
+	if (parsed_dec != RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL &&
+	    parsed_dec != RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT) {
+		RTE_EM_HLPR_LOG_ERR("Unsupported packet transfer mode");
+		return -1;
+	}
+	conf->mode = parsed_dec;
+	return 0;
+}
+
+static void
+em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)
+{
+	/* Set default conf */
+
+	/* Packet transfer mode: poll */
+	conf->mode = RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL;
+}
+
+struct rte_eventmode_helper_conf *
+rte_eventmode_helper_parse_args(int argc, char **argv)
+{
+	int32_t opt, ret;
+	struct rte_eventmode_helper_conf *conf = NULL;
+
+	/* Allocate memory for conf */
+	conf = rte_zmalloc("eventmode-helper-conf",
+			sizeof(struct rte_eventmode_helper_conf),
+			RTE_CACHE_LINE_SIZE);
+	if (conf == NULL) {
+		RTE_EM_HLPR_LOG_ERR(
+			"Failed allocating memory for eventmode helper conf");
+			goto err;
+	}
+
+	/* Initialize conf with default values */
+	em_initialize_helper_conf(conf);
+
+	while ((opt = getopt_long(argc, argv, short_options,
+				lgopts, NULL)) != EOF) {
+		switch (opt) {
+
+		/* Packet transfer mode */
+		case CMD_LINE_OPT_TRANSFER_MODE_NUM:
+			ret = em_parse_transfer_mode(conf, optarg);
+			if (ret < 0) {
+				RTE_EM_HLPR_LOG_ERR(
+					"Invalid packet transfer mode");
+				goto err;
+			}
+			break;
+		default:
+			goto err;
+		}
+	}
+	return conf;
+
+err:
+	if (conf != NULL)
+		rte_free(conf);
+
+	return NULL;
+}
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index d32cd00..2a0cb30 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -8,6 +8,57 @@
 extern "C" {
 #endif
 
+#include <rte_compat.h>
+
+/* Packet transfer mode of the application */
+enum rte_eventmode_helper_pkt_transfer_mode {
+	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL = 0,
+	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT,
+};
+
+struct rte_eventmode_helper_conf {
+	enum rte_eventmode_helper_pkt_transfer_mode mode;
+		/**< Packet transfer mode of the application */
+	void *mode_params;
+		/**< Mode specific parameters */
+};
+
+/* Common helper functions for command line parsing */
+
+/**
+ * Print event mode options list
+ *
+ */
+void __rte_experimental
+rte_eventmode_helper_print_options_list(void);
+
+/**
+ * Print event mode options description
+ *
+ */
+void __rte_experimental
+rte_eventmode_helper_print_options_description(void);
+
+/**
+ * Parse event mode arguments
+ *
+ * The application can call this function in it's argument parsing routine to
+ * parse the event mode specific args and create the conf accordingly. This
+ * function is to be executed on the MASTER lcore only.
+ *
+ * @param argc
+ *   A non-negative value. If it is greater than 0, the array members
+ *   for argv[0] through argv[argc] (non-inclusive) shall contain pointers
+ *   to strings.
+ * @param argv
+ *   An array of strings. The contents of the array, as well as the strings
+ *   which are pointed to by the array, may be modified by this function.
+ * @return
+ *   Configuration generated by parsing the event mode args.
+ */
+struct rte_eventmode_helper_conf *
+rte_eventmode_helper_parse_args(int argc, char **argv);
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.7.4

[dpdk-dev] [PATCH 17/39] eventdev: allow application to set ethernet portmask

[Anoob Joseph]

Application would be required to restrict helper functions to use only
certain ports. The field eth_portmask field in the conf could be used
for this.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c | 3 +++
 lib/librte_eventdev/rte_eventmode_helper.h | 6 ++++++
 2 files changed, 9 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index 8119306..dc2c934 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -87,6 +87,9 @@ em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)
 
 	/* Packet transfer mode: poll */
 	conf->mode = RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL;
+
+	/* Keep all ethernet ports enabled by default */
+	conf->eth_portmask = -1;
 }
 
 struct rte_eventmode_helper_conf *
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index 2a0cb30..77e69d0 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -8,6 +8,7 @@
 extern "C" {
 #endif
 
+#include <rte_common.h>
 #include <rte_compat.h>
 
 /* Packet transfer mode of the application */
@@ -19,6 +20,11 @@ enum rte_eventmode_helper_pkt_transfer_mode {
 struct rte_eventmode_helper_conf {
 	enum rte_eventmode_helper_pkt_transfer_mode mode;
 		/**< Packet transfer mode of the application */
+	uint32_t eth_portmask;
+		/**<
+		 * Mask of the eth ports to be used. This portmask would be
+		 * checked while initializing devices using helper routines.
+		 */
 	void *mode_params;
 		/**< Mode specific parameters */
 };
-- 
2.7.4

[dpdk-dev] [PATCH 18/39] eventdev: add framework for eventmode conf

[Anoob Joseph]

Adding eventmode conf which would have all required configuration for
the event mode.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c          | 16 ++++++++++++++++
 lib/librte_eventdev/rte_eventmode_helper_internal.h |  5 +++++
 2 files changed, 21 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index dc2c934..38f1a2b 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -97,6 +97,7 @@ rte_eventmode_helper_parse_args(int argc, char **argv)
 {
 	int32_t opt, ret;
 	struct rte_eventmode_helper_conf *conf = NULL;
+	struct eventmode_conf *em_conf = NULL;
 
 	/* Allocate memory for conf */
 	conf = rte_zmalloc("eventmode-helper-conf",
@@ -108,9 +109,21 @@ rte_eventmode_helper_parse_args(int argc, char **argv)
 			goto err;
 	}
 
+	/* Allocate memory for event mode params */
+	conf->mode_params = rte_zmalloc("eventmode-helper-mode-params",
+			sizeof(struct eventmode_conf),
+			RTE_CACHE_LINE_SIZE);
+	if (conf->mode_params == NULL) {
+		RTE_EM_HLPR_LOG_ERR(
+			"Failed allocating memory for event mode params");
+		goto err;
+	}
+
 	/* Initialize conf with default values */
 	em_initialize_helper_conf(conf);
 
+	em_conf = (struct eventmode_conf *)(conf->mode_params);
+
 	while ((opt = getopt_long(argc, argv, short_options,
 				lgopts, NULL)) != EOF) {
 		switch (opt) {
@@ -131,6 +144,9 @@ rte_eventmode_helper_parse_args(int argc, char **argv)
 	return conf;
 
 err:
+	if (em_conf != NULL)
+		rte_free(em_conf);
+
 	if (conf != NULL)
 		rte_free(conf);
 
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 7af7758..2ee7711 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -32,4 +32,9 @@
 #define RTE_EM_HLPR_LOG_DEBUG(...) (void)0
 #endif
 
+/* Eventmode conf data */
+struct eventmode_conf {
+	uint64_t dummy;
+};
+
 #endif /* _RTE_EVENTMODE_HELPER_INTERNAL_H_ */
-- 
2.7.4

[dpdk-dev] [PATCH 19/39] eventdev: add common initialize routine for eventmode devs

[Anoob Joseph]

Adding framework for common initialization routine for event mode.
Event mode would involve initialization of multiple devices, like
eventdev, ethdev etc and this routine would be the placeholder for all
initialization to come in.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventdev_version.map |  2 ++
 lib/librte_eventdev/rte_eventmode_helper.c   | 51 +++++++++++++++++++++++++++-
 lib/librte_eventdev/rte_eventmode_helper.h   | 26 +++++++++++++-
 3 files changed, 77 insertions(+), 2 deletions(-)

diff --git a/lib/librte_eventdev/rte_eventdev_version.map b/lib/librte_eventdev/rte_eventdev_version.map
index 1199064..e156fa0 100644
--- a/lib/librte_eventdev/rte_eventdev_version.map
+++ b/lib/librte_eventdev/rte_eventdev_version.map
@@ -130,4 +130,6 @@ EXPERIMENTAL {
 	rte_event_eth_rx_adapter_stats_get;
 	rte_eventmode_helper_print_options_list;
 	rte_eventmode_helper_print_options_description;
+	rte_eventmode_helper_parse_args;
+	rte_eventmode_helper_initialize_devs;
 };
diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index 38f1a2b..d1a569b 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -3,6 +3,7 @@
  */
 #include <getopt.h>
 
+#include <rte_ethdev.h>
 #include <rte_eventmode_helper.h>
 #include <rte_malloc.h>
 
@@ -92,7 +93,7 @@ em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)
 	conf->eth_portmask = -1;
 }
 
-struct rte_eventmode_helper_conf *
+struct rte_eventmode_helper_conf * __rte_experimental
 rte_eventmode_helper_parse_args(int argc, char **argv)
 {
 	int32_t opt, ret;
@@ -152,3 +153,51 @@ rte_eventmode_helper_parse_args(int argc, char **argv)
 
 	return NULL;
 }
+
+int32_t __rte_experimental
+rte_eventmode_helper_initialize_devs(
+		struct rte_eventmode_helper_conf *mode_conf)
+{
+	int ret;
+	uint16_t portid;
+
+	if (mode_conf == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid conf");
+		return -1;
+	}
+
+	if (mode_conf->mode != RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT)
+		return 0;
+
+	if (mode_conf->mode_params == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid mode params");
+		return -1;
+	}
+
+	/* Stop eth devices before setting up adapter */
+	RTE_ETH_FOREACH_DEV(portid) {
+
+		/* Use only the ports enabled */
+		if ((mode_conf->eth_portmask & (1 << portid)) == 0)
+			continue;
+
+		rte_eth_dev_stop(portid);
+	}
+
+	/* Start eth devices after setting up adapter */
+	RTE_ETH_FOREACH_DEV(portid) {
+
+		/* Use only the ports enabled */
+		if ((mode_conf->eth_portmask & (1 << portid)) == 0)
+			continue;
+
+		ret = rte_eth_dev_start(portid);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error starting eth dev %d", portid);
+			return -1;
+		}
+	}
+
+	return 0;
+}
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index 77e69d0..0eafed3 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -62,9 +62,33 @@ rte_eventmode_helper_print_options_description(void);
  * @return
  *   Configuration generated by parsing the event mode args.
  */
-struct rte_eventmode_helper_conf *
+struct rte_eventmode_helper_conf * __rte_experimental
 rte_eventmode_helper_parse_args(int argc, char **argv);
 
+/* Helper functions for initialization, & launching workers */
+
+/**
+ * Initialize event mode devices
+ *
+ * Application could call this function to get the event device, eth device
+ * and eth rx adapter initialized according to the conf populated using the
+ * command line args.
+ *
+ * Application is expected to initialize the eth device and then the eventmode
+ * helper subsystem will stop & start eth device according to it's requirement.
+ * So call to this function should be done after the eth device is successfully
+ * initialized.
+ *
+ * @param mode_conf
+ *   Configuration of the mode in which app is doing packet handling
+ * @return
+ *  - 0 on success.
+ *  - (<0) on failure.
+ */
+int32_t __rte_experimental
+rte_eventmode_helper_initialize_devs(
+		struct rte_eventmode_helper_conf *mode_conf);
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.7.4

[dpdk-dev] [PATCH 20/39] eventdev: add eventdevice init for eventmode

[Anoob Joseph]

Adding routines to initialize event devs. The internal conf structure
would be used to track device configuration.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c         | 131 +++++++++++++++++++++
 .../rte_eventmode_helper_internal.h                |  16 ++-
 2 files changed, 146 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index d1a569b..bebcfb9 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -4,6 +4,7 @@
 #include <getopt.h>
 
 #include <rte_ethdev.h>
+#include <rte_eventdev.h>
 #include <rte_eventmode_helper.h>
 #include <rte_malloc.h>
 
@@ -154,12 +155,134 @@ rte_eventmode_helper_parse_args(int argc, char **argv)
 	return NULL;
 }
 
+/* Setup eventmode devs */
+
+static int
+rte_eventmode_helper_initialize_eventdev(struct eventmode_conf *em_conf)
+{
+	int ret = -1;
+	uint8_t i, j;
+	struct rte_event_dev_config eventdev_conf;
+	struct rte_event_dev_info evdev_default_conf;
+	struct rte_event_queue_conf eventq_conf = {0};
+	struct eventdev_params *eventdev_config;
+	int nb_eventdev = em_conf->nb_eventdev;
+	int nb_eventqueue;
+	uint8_t eventdev_id;
+
+	for (i = 0; i < nb_eventdev; i++) {
+
+		/* Get eventdev config */
+		eventdev_config = &(em_conf->eventdev_config[i]);
+
+		/* Get event dev ID */
+		eventdev_id = eventdev_config->eventdev_id;
+
+		/* Get the number of queues */
+		nb_eventqueue = eventdev_config->nb_eventqueue;
+
+		/* One queue is reserved for the final stage (doing eth tx) */
+		/* TODO handles only one Tx adapter. Fix this */
+		nb_eventqueue += 1;
+
+		/* Reset the default conf */
+		memset(&evdev_default_conf, 0,
+			sizeof(struct rte_event_dev_info));
+
+		/* Get default conf of eventdev */
+		ret = rte_event_dev_info_get(eventdev_id, &evdev_default_conf);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error in getting event device info[devID:%d]",
+				eventdev_id);
+			return ret;
+		}
+
+		memset(&eventdev_conf, 0, sizeof(struct rte_event_dev_config));
+		eventdev_conf.nb_events_limit =
+				evdev_default_conf.max_num_events;
+		eventdev_conf.nb_event_queues = nb_eventqueue;
+		eventdev_conf.nb_event_ports =
+				eventdev_config->nb_eventport;
+		eventdev_conf.nb_event_queue_flows =
+				evdev_default_conf.max_event_queue_flows;
+		eventdev_conf.nb_event_port_dequeue_depth =
+				evdev_default_conf.max_event_port_dequeue_depth;
+		eventdev_conf.nb_event_port_enqueue_depth =
+				evdev_default_conf.max_event_port_enqueue_depth;
+
+		/* Configure event device */
+		ret = rte_event_dev_configure(eventdev_id, &eventdev_conf);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error in configuring event device");
+			return ret;
+		}
+
+		/* Configure event queues */
+		for (j = 0; j < nb_eventqueue; j++) {
+
+			memset(&eventq_conf, 0,
+					sizeof(struct rte_event_queue_conf));
+
+			/* Read the requested conf */
+
+			/* Per event dev queues can be ATQ or SINGLE LINK */
+			eventq_conf.event_queue_cfg =
+					eventdev_config->ev_queue_mode;
+
+			/* Set schedule type as ATOMIC */
+			eventq_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+
+			/* Set max atomic flows to 1024 */
+			eventq_conf.nb_atomic_flows = 1024;
+			eventq_conf.nb_atomic_order_sequences = 1024;
+
+			/* Setup the queue */
+			ret = rte_event_queue_setup(eventdev_id, j,
+					&eventq_conf);
+			if (ret < 0) {
+				RTE_EM_HLPR_LOG_ERR(
+					"Error in event queue setup");
+				return ret;
+			}
+		}
+
+		/* Configure event ports */
+		for (j = 0; j <  eventdev_config->nb_eventport; j++) {
+			ret = rte_event_port_setup(eventdev_id, j, NULL);
+			if (ret < 0) {
+				RTE_EM_HLPR_LOG_ERR(
+					"Error setting up event port");
+				return ret;
+			}
+		}
+	}
+
+	/* Start event devices */
+	for (i = 0; i < nb_eventdev; i++) {
+
+		/* Get eventdev config */
+		eventdev_config = &(em_conf->eventdev_config[i]);
+
+		ret = rte_event_dev_start(eventdev_config->eventdev_id);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error in starting event device[devID: %d]",
+				eventdev_config->eventdev_id);
+			return ret;
+		}
+	}
+	return 0;
+}
+
 int32_t __rte_experimental
 rte_eventmode_helper_initialize_devs(
 		struct rte_eventmode_helper_conf *mode_conf)
 {
 	int ret;
 	uint16_t portid;
+	struct eventmode_conf *em_conf;
 
 	if (mode_conf == NULL) {
 		RTE_EM_HLPR_LOG_ERR("Invalid conf");
@@ -174,6 +297,9 @@ rte_eventmode_helper_initialize_devs(
 		return -1;
 	}
 
+	/* Get eventmode conf */
+	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
+
 	/* Stop eth devices before setting up adapter */
 	RTE_ETH_FOREACH_DEV(portid) {
 
@@ -184,6 +310,11 @@ rte_eventmode_helper_initialize_devs(
 		rte_eth_dev_stop(portid);
 	}
 
+	/* Setup eventdev */
+	ret = rte_eventmode_helper_initialize_eventdev(em_conf);
+	if (ret != 0)
+		return ret;
+
 	/* Start eth devices after setting up adapter */
 	RTE_ETH_FOREACH_DEV(portid) {
 
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 2ee7711..0b4ec59 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -32,9 +32,23 @@
 #define RTE_EM_HLPR_LOG_DEBUG(...) (void)0
 #endif
 
+/* Max event devices supported */
+#define EVENT_MODE_MAX_EVENT_DEVS RTE_EVENT_MAX_DEVS
+
+/* Event dev params */
+struct eventdev_params {
+	uint8_t eventdev_id;
+	uint8_t nb_eventqueue;
+	uint8_t nb_eventport;
+	uint8_t ev_queue_mode;
+};
+
 /* Eventmode conf data */
 struct eventmode_conf {
-	uint64_t dummy;
+	int nb_eventdev;
+		/**< No of event devs */
+	struct eventdev_params eventdev_config[EVENT_MODE_MAX_EVENT_DEVS];
+		/**< Per event dev conf */
 };
 
 #endif /* _RTE_EVENTMODE_HELPER_INTERNAL_H_ */
-- 
2.7.4

[dpdk-dev] [PATCH 21/39] eventdev: add eventdev port-lcore link

[Anoob Joseph]

Adding eventdev port-lcore link. In addition, this will also specify
which event queue need to be connected to the event port.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c         | 22 ++++++++++++++++++++++
 lib/librte_eventdev/rte_eventmode_helper.h         | 12 ++++++++++++
 .../rte_eventmode_helper_internal.h                | 12 ++++++++++++
 3 files changed, 46 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index bebcfb9..ec0be44 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -165,10 +165,12 @@ rte_eventmode_helper_initialize_eventdev(struct eventmode_conf *em_conf)
 	struct rte_event_dev_config eventdev_conf;
 	struct rte_event_dev_info evdev_default_conf;
 	struct rte_event_queue_conf eventq_conf = {0};
+	struct rte_eventmode_helper_event_link_info *link;
 	struct eventdev_params *eventdev_config;
 	int nb_eventdev = em_conf->nb_eventdev;
 	int nb_eventqueue;
 	uint8_t eventdev_id;
+	uint8_t *queue = NULL;
 
 	for (i = 0; i < nb_eventdev; i++) {
 
@@ -259,6 +261,26 @@ rte_eventmode_helper_initialize_eventdev(struct eventmode_conf *em_conf)
 		}
 	}
 
+	/* Make event queue - event port link */
+	for (j = 0; j <  em_conf->nb_link; j++) {
+
+		/* Get link info */
+		link = &(em_conf->link[j]);
+
+		/* Get event dev ID */
+		eventdev_id = link->eventdev_id;
+
+		queue = &(link->eventq_id);
+
+		/* Link queue to port */
+		ret = rte_event_port_link(eventdev_id, link->event_portid,
+				queue, NULL, 1);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR("Error in event port linking");
+			return ret;
+		}
+	}
+
 	/* Start event devices */
 	for (i = 0; i < nb_eventdev; i++) {
 
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index 0eafed3..1ae6a15 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -29,6 +29,18 @@ struct rte_eventmode_helper_conf {
 		/**< Mode specific parameters */
 };
 
+/* Event-lcore link conf */
+struct rte_eventmode_helper_event_link_info {
+	uint8_t eventdev_id;
+		/**< Event device ID */
+	uint8_t event_portid;
+		/**< Event port ID */
+	uint8_t eventq_id;
+		/**< Event queue to be linked to the port */
+	uint8_t lcore_id;
+		/**< Lcore to be polling on this port */
+};
+
 /* Common helper functions for command line parsing */
 
 /**
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 0b4ec59..ee41833 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -35,6 +35,13 @@
 /* Max event devices supported */
 #define EVENT_MODE_MAX_EVENT_DEVS RTE_EVENT_MAX_DEVS
 
+/* Max event queues supported per event device */
+#define EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV RTE_EVENT_MAX_QUEUES_PER_DEV
+
+/* Max event-lcore links */
+#define EVENT_MODE_MAX_LCORE_LINKS \
+	(EVENT_MODE_MAX_EVENT_DEVS * EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV)
+
 /* Event dev params */
 struct eventdev_params {
 	uint8_t eventdev_id;
@@ -49,6 +56,11 @@ struct eventmode_conf {
 		/**< No of event devs */
 	struct eventdev_params eventdev_config[EVENT_MODE_MAX_EVENT_DEVS];
 		/**< Per event dev conf */
+	uint8_t nb_link;
+		/**< No of links */
+	struct rte_eventmode_helper_event_link_info
+			link[EVENT_MODE_MAX_LCORE_LINKS];
+		/**< Per link conf */
 };
 
 #endif /* _RTE_EVENTMODE_HELPER_INTERNAL_H_ */
-- 
2.7.4

[dpdk-dev] [PATCH 22/39] eventdev: add option to specify schedule mode for app stage

[Anoob Joseph]

Scheduling mode for each event queue is dependent on the same of app
stage. Configure event queue taking this also into account.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c         | 24 ++++++++++++++++++++--
 .../rte_eventmode_helper_internal.h                |  8 ++++++++
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index ec0be44..30bb357 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -85,6 +85,8 @@ em_parse_transfer_mode(struct rte_eventmode_helper_conf *conf,
 static void
 em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)
 {
+	struct eventmode_conf *em_conf = NULL;
+
 	/* Set default conf */
 
 	/* Packet transfer mode: poll */
@@ -92,6 +94,13 @@ em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)
 
 	/* Keep all ethernet ports enabled by default */
 	conf->eth_portmask = -1;
+
+	/* Get eventmode conf */
+	em_conf = (struct eventmode_conf *)(conf->mode_params);
+
+	/* Schedule type: ordered */
+	/* FIXME */
+	em_conf->ext_params.sched_type = RTE_SCHED_TYPE_ORDERED;
 }
 
 struct rte_eventmode_helper_conf * __rte_experimental
@@ -233,8 +242,19 @@ rte_eventmode_helper_initialize_eventdev(struct eventmode_conf *em_conf)
 			eventq_conf.event_queue_cfg =
 					eventdev_config->ev_queue_mode;
 
-			/* Set schedule type as ATOMIC */
-			eventq_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+			/*
+			 * All queues need to be set with sched_type as
+			 * schedule type for the application stage. One queue
+			 * would be reserved for the final eth tx stage. This
+			 * will be an atomic queue.
+			 */
+			if (j == nb_eventqueue-1) {
+				eventq_conf.schedule_type =
+					RTE_SCHED_TYPE_ATOMIC;
+			} else {
+				eventq_conf.schedule_type =
+					em_conf->ext_params.sched_type;
+			}
 
 			/* Set max atomic flows to 1024 */
 			eventq_conf.nb_atomic_flows = 1024;
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index ee41833..2a6cd90 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -61,6 +61,14 @@ struct eventmode_conf {
 	struct rte_eventmode_helper_event_link_info
 			link[EVENT_MODE_MAX_LCORE_LINKS];
 		/**< Per link conf */
+	union {
+		struct {
+			uint64_t sched_type			: 2;
+		/**< Schedule type */
+		};
+		uint64_t u64;
+	} ext_params;
+		/**< 64 bit field to specify extended params */
 };
 
 #endif /* _RTE_EVENTMODE_HELPER_INTERNAL_H_ */
-- 
2.7.4

[dpdk-dev] [PATCH 23/39] eventdev: add placeholder for ethdev init

[Anoob Joseph]

Presently, all the applications would do ethdev init and then pass
control to eventmode helper init. So not doing any "real"
initialization. But this would be expanded once applications are
modified to pass the eth init task also to the helper routine.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index 30bb357..8faa4ea 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -318,6 +318,14 @@ rte_eventmode_helper_initialize_eventdev(struct eventmode_conf *em_conf)
 	return 0;
 }
 
+static int
+rte_eventmode_helper_initialize_ethdev(struct eventmode_conf *em_conf)
+{
+	RTE_SET_USED(em_conf);
+
+	return 0;
+}
+
 int32_t __rte_experimental
 rte_eventmode_helper_initialize_devs(
 		struct rte_eventmode_helper_conf *mode_conf)
@@ -357,6 +365,11 @@ rte_eventmode_helper_initialize_devs(
 	if (ret != 0)
 		return ret;
 
+	/* Setup ethdev */
+	ret = rte_eventmode_helper_initialize_ethdev(em_conf);
+	if (ret != 0)
+		return ret;
+
 	/* Start eth devices after setting up adapter */
 	RTE_ETH_FOREACH_DEV(portid) {
 
-- 
2.7.4

[dpdk-dev] [PATCH 24/39] eventdev: add Rx adapter init in eventmode

[Anoob Joseph]

Adding rx adapter conf. The helper init routine would be initializing
the rx adapter according to the conf.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c         | 123 +++++++++++++++++++++
 .../rte_eventmode_helper_internal.h                |  27 +++++
 2 files changed, 150 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index 8faa4ea..a57f837 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -6,6 +6,7 @@
 #include <rte_ethdev.h>
 #include <rte_eventdev.h>
 #include <rte_eventmode_helper.h>
+#include <rte_event_eth_rx_adapter.h>
 #include <rte_malloc.h>
 
 #include "rte_eventmode_helper_internal.h"
@@ -326,6 +327,123 @@ rte_eventmode_helper_initialize_ethdev(struct eventmode_conf *em_conf)
 	return 0;
 }
 
+static int
+rx_adapter_configure(struct eventmode_conf *em_conf,
+	struct rx_adapter_conf *adapter)
+{
+	int j;
+	int ret;
+	uint8_t eventdev_id;
+	uint32_t service_id;
+	struct adapter_connection_info *conn;
+	struct rte_event_port_conf port_conf = {0};
+	struct rte_event_eth_rx_adapter_queue_conf queue_conf = {0};
+	struct rte_event_dev_info evdev_default_conf = {0};
+
+	/* Get event dev ID */
+	eventdev_id = adapter->eventdev_id;
+
+	/* Create rx_adapter */
+
+	/* Get default configuration of event dev */
+	ret = rte_event_dev_info_get(eventdev_id, &evdev_default_conf);
+	if (ret < 0) {
+		RTE_EM_HLPR_LOG_ERR(
+			"Error in getting event device info[devID:%d]",
+			eventdev_id);
+		return ret;
+	}
+
+	/* Setup port conf */
+	port_conf.new_event_threshold = 1200;
+	port_conf.dequeue_depth =
+			evdev_default_conf.max_event_port_dequeue_depth;
+	port_conf.enqueue_depth =
+			evdev_default_conf.max_event_port_enqueue_depth;
+
+	/* Create Rx adapter */
+	ret = rte_event_eth_rx_adapter_create(adapter->adapter_id,
+			adapter->eventdev_id,
+			&port_conf);
+	if (ret < 0) {
+		RTE_EM_HLPR_LOG_ERR("Error in rx adapter creation");
+		return ret;
+	}
+
+	/* Setup various connections in the adapter */
+
+	queue_conf.rx_queue_flags =
+			RTE_EVENT_ETH_RX_ADAPTER_QUEUE_FLOW_ID_VALID;
+
+	for (j = 0; j < adapter->nb_connections; j++) {
+		/* Get connection */
+		conn = &(adapter->conn[j]);
+
+		/* Setup queue conf */
+		queue_conf.ev.queue_id = conn->eventq_id;
+		queue_conf.ev.sched_type = em_conf->ext_params.sched_type;
+
+		/* Set flow ID as ethdev ID */
+		queue_conf.ev.flow_id = conn->ethdev_id;
+
+		/* Add queue to the adapter */
+		ret = rte_event_eth_rx_adapter_queue_add(
+				adapter->adapter_id,
+				conn->ethdev_id,
+				conn->ethdev_rx_qid,
+				&queue_conf);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error in adding eth queue in Rx adapter");
+			return ret;
+		}
+	}
+
+	/* Get the service ID used by rx adapter */
+	ret = rte_event_eth_rx_adapter_service_id_get(adapter->adapter_id,
+						      &service_id);
+	if (ret != -ESRCH && ret != 0) {
+		RTE_EM_HLPR_LOG_ERR(
+			"Error getting service ID used by Rx adapter");
+		return ret;
+	}
+
+	/*
+	 * TODO
+	 * Rx core will invoke the service when required. The runstate check
+	 * is not required.
+	 *
+	 */
+	rte_service_set_runstate_mapped_check(service_id, 0);
+
+	/* Start adapter */
+	ret = rte_event_eth_rx_adapter_start(adapter->adapter_id);
+	if (ret) {
+		RTE_EM_HLPR_LOG_ERR("Error in starting rx adapter");
+		return ret;
+	}
+
+	return 0;
+}
+
+static int
+rte_eventmode_helper_initialize_rx_adapter(struct eventmode_conf *em_conf)
+{
+	int i, ret;
+	struct rx_adapter_conf *adapter;
+
+	/* Configure rx adapters */
+	for (i = 0; i < em_conf->nb_rx_adapter; i++) {
+		adapter = &(em_conf->rx_adapter[i]);
+		ret = rx_adapter_configure(em_conf, adapter);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR("Rx adapter configuration failed");
+			return ret;
+		}
+	}
+	return 0;
+}
+
 int32_t __rte_experimental
 rte_eventmode_helper_initialize_devs(
 		struct rte_eventmode_helper_conf *mode_conf)
@@ -370,6 +488,11 @@ rte_eventmode_helper_initialize_devs(
 	if (ret != 0)
 		return ret;
 
+	/* Setup Rx adapter */
+	ret = rte_eventmode_helper_initialize_rx_adapter(em_conf);
+	if (ret != 0)
+		return ret;
+
 	/* Start eth devices after setting up adapter */
 	RTE_ETH_FOREACH_DEV(portid) {
 
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 2a6cd90..9c68605 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -35,6 +35,12 @@
 /* Max event devices supported */
 #define EVENT_MODE_MAX_EVENT_DEVS RTE_EVENT_MAX_DEVS
 
+/* Max Rx adapters supported */
+#define EVENT_MODE_MAX_RX_ADAPTERS RTE_EVENT_MAX_DEVS
+
+/* Max Rx adapter connections */
+#define EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER 16
+
 /* Max event queues supported per event device */
 #define EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV RTE_EVENT_MAX_QUEUES_PER_DEV
 
@@ -50,12 +56,33 @@ struct eventdev_params {
 	uint8_t ev_queue_mode;
 };
 
+/* Rx adapter connection info */
+struct adapter_connection_info {
+	uint8_t ethdev_id;
+	uint8_t eventq_id;
+	int32_t ethdev_rx_qid;
+};
+
+/* Rx adapter conf */
+struct rx_adapter_conf {
+	int32_t eventdev_id;
+	int32_t adapter_id;
+	uint32_t rx_core_id;
+	uint8_t nb_connections;
+	struct adapter_connection_info
+			conn[EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER];
+};
+
 /* Eventmode conf data */
 struct eventmode_conf {
 	int nb_eventdev;
 		/**< No of event devs */
 	struct eventdev_params eventdev_config[EVENT_MODE_MAX_EVENT_DEVS];
 		/**< Per event dev conf */
+	uint8_t nb_rx_adapter;
+		/**< No of Rx adapters */
+	struct rx_adapter_conf rx_adapter[EVENT_MODE_MAX_RX_ADAPTERS];
+		/**< Rx adapter conf */
 	uint8_t nb_link;
 		/**< No of links */
 	struct rte_eventmode_helper_event_link_info
-- 
2.7.4

[dpdk-dev] [PATCH 25/39] eventdev: add routine to validate conf

[Anoob Joseph]

Adding routine to validate event mode conf. This function will verify
the conf requested by the user and would populate other fields with
default values. Presently, the function acts as placeholder for the
above mentioned actions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c | 33 ++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index a57f837..4dbb94a 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -165,6 +165,32 @@ rte_eventmode_helper_parse_args(int argc, char **argv)
 	return NULL;
 }
 
+/* Pre-process conf before using for init*/
+
+static int
+rte_eventmode_validate_user_params(struct eventmode_conf *em_conf)
+{
+	/* TODO */
+	/* Check sanity of the conf requested by user */
+
+	RTE_SET_USED(em_conf);
+
+	return 0;
+}
+
+static int
+rte_eventmode_helper_validate_conf(struct eventmode_conf *em_conf)
+{
+	int ret;
+
+	/* After parsing all args, verify that the conf can be allowed */
+	ret = rte_eventmode_validate_user_params(em_conf);
+	if (ret != 0)
+		return ret;
+
+	return 0;
+}
+
 /* Setup eventmode devs */
 
 static int
@@ -468,6 +494,13 @@ rte_eventmode_helper_initialize_devs(
 	/* Get eventmode conf */
 	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
 
+	/* Validate the conf requested */
+	if (rte_eventmode_helper_validate_conf(em_conf) != 0) {
+		RTE_EM_HLPR_LOG_ERR(
+			"Failed while validating the conf requested");
+		return -1;
+	}
+
 	/* Stop eth devices before setting up adapter */
 	RTE_ETH_FOREACH_DEV(portid) {
 
-- 
2.7.4

[dpdk-dev] [PATCH 26/39] eventdev: add default conf for event devs field in conf

[Anoob Joseph]

Generate a default conf for event devs, if it's not specified in the
conf. This routine will check the available event devices and it's
properties and sets the conf accordingly.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c | 67 ++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index 4dbb94a..1729561 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -179,6 +179,63 @@ rte_eventmode_validate_user_params(struct eventmode_conf *em_conf)
 }
 
 static int
+rte_eventmode_helper_set_default_conf_eventdev(struct eventmode_conf *em_conf)
+{
+	int i, ret;
+	int nb_eventdev;
+	struct eventdev_params *eventdev_config;
+	struct rte_event_dev_info dev_info;
+
+	/* Get the number of event devices */
+	nb_eventdev = rte_event_dev_count();
+
+	if (nb_eventdev == 0) {
+		RTE_EM_HLPR_LOG_ERR("No event devices detected");
+		return -1;
+	}
+
+	for (i = 0; i < nb_eventdev; i++) {
+
+		/* Get the event dev conf */
+		eventdev_config = &(em_conf->eventdev_config[i]);
+
+		/* Read event device info */
+		ret = rte_event_dev_info_get(i, &dev_info);
+
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Failed reading event device info (err:%d)",
+				ret);
+			return ret;
+		}
+
+		/* Check if enough ports are available */
+		if (dev_info.max_event_ports < 2) {
+			RTE_EM_HLPR_LOG_ERR("Not enough ports available");
+			return -1;
+		}
+
+		/* Save number of queues & ports available */
+		eventdev_config->eventdev_id = i;
+		eventdev_config->nb_eventqueue = dev_info.max_event_queues;
+		eventdev_config->nb_eventport = dev_info.max_event_ports;
+		eventdev_config->ev_queue_mode =
+				RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+
+		/* One port is required for eth Rx adapter */
+		eventdev_config->nb_eventport -= 1;
+
+		/* One port is reserved for eth Tx adapter */
+		eventdev_config->nb_eventport -= 1;
+
+		/* Update the number of eventdevs */
+		em_conf->nb_eventdev++;
+	}
+
+	return 0;
+}
+
+static int
 rte_eventmode_helper_validate_conf(struct eventmode_conf *em_conf)
 {
 	int ret;
@@ -188,6 +245,16 @@ rte_eventmode_helper_validate_conf(struct eventmode_conf *em_conf)
 	if (ret != 0)
 		return ret;
 
+	/*
+	 * See if event devs are specified. Else probe the event devices
+	 * and initialize the conf with all ports & queues available
+	 */
+	if (em_conf->nb_eventdev == 0) {
+		ret = rte_eventmode_helper_set_default_conf_eventdev(em_conf);
+		if (ret != 0)
+			return ret;
+	}
+
 	return 0;
 }
 
-- 
2.7.4

[dpdk-dev] [PATCH 27/39] eventdev: add default conf for Rx adapter conf

[Anoob Joseph]

Generate a default conf for Rx adapter, if not specified in the conf.
This routine will check the available eth ports and event queues, and
maps them 1:1. So one eth port will be connected to one event queue.
This way, event queue ID could be used to figure out the port on which
the packet came in.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c         | 127 +++++++++++++++++++++
 .../rte_eventmode_helper_internal.h                |   4 +
 2 files changed, 131 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index 1729561..a24d654 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -47,6 +47,30 @@ internal_parse_decimal(const char *str)
 	return num;
 }
 
+static inline unsigned int
+internal_get_next_rx_core(struct eventmode_conf *em_conf,
+		unsigned int prev_core)
+{
+	unsigned int next_core;
+
+get_next_core:
+	/* Get the next core */
+	next_core = rte_get_next_lcore(prev_core, 0, 0);
+
+	/* Check if we have reached max lcores */
+	if (next_core == RTE_MAX_LCORE)
+		return next_core;
+
+	/* Only some cores would be marked as rx cores. Skip others */
+	if (!(em_conf->eth_core_mask & (1 << next_core))) {
+		prev_core = next_core;
+		goto get_next_core;
+	}
+
+	return next_core;
+}
+
+
 /* Global functions */
 
 void __rte_experimental
@@ -87,6 +111,7 @@ static void
 em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)
 {
 	struct eventmode_conf *em_conf = NULL;
+	unsigned int rx_core_id;
 
 	/* Set default conf */
 
@@ -102,6 +127,12 @@ em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)
 	/* Schedule type: ordered */
 	/* FIXME */
 	em_conf->ext_params.sched_type = RTE_SCHED_TYPE_ORDERED;
+	/* Set rx core. Use first core other than master core as Rx core */
+	rx_core_id = rte_get_next_lcore(0, /* curr core */
+					1, /* skip master core */
+					0  /* wrap */);
+
+	em_conf->eth_core_mask = (1 << rx_core_id);
 }
 
 struct rte_eventmode_helper_conf * __rte_experimental
@@ -236,6 +267,89 @@ rte_eventmode_helper_set_default_conf_eventdev(struct eventmode_conf *em_conf)
 }
 
 static int
+rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf *em_conf)
+{
+	int nb_eth_dev;
+	int i;
+	int adapter_id;
+	int eventdev_id;
+	int conn_id;
+	struct rx_adapter_conf *adapter;
+	struct adapter_connection_info *conn;
+	struct eventdev_params *eventdev_config;
+
+	/* Create one adapter with all eth queues mapped to event queues 1:1 */
+
+	if (em_conf->nb_eventdev == 0) {
+		RTE_EM_HLPR_LOG_ERR("No event devs registered");
+		return -1;
+	}
+
+	/* Get the number of eth devs */
+	nb_eth_dev = rte_eth_dev_count_avail();
+
+	/* Use the first event dev */
+	eventdev_config = &(em_conf->eventdev_config[0]);
+
+	/* Get eventdev ID */
+	eventdev_id = eventdev_config->eventdev_id;
+	adapter_id = 0;
+
+	/* Get adapter conf */
+	adapter = &(em_conf->rx_adapter[adapter_id]);
+
+	/* Set adapter conf */
+	adapter->eventdev_id = eventdev_id;
+	adapter->adapter_id = adapter_id;
+	adapter->rx_core_id = internal_get_next_rx_core(em_conf, -1);
+
+	/*
+	 * All queues of one eth device (port) will be mapped to one event
+	 * queue. Each port will have an individual connection.
+	 *
+	 */
+
+	/* Make sure there is enough event queues for 1:1 mapping */
+	if (nb_eth_dev > eventdev_config->nb_eventqueue) {
+		RTE_EM_HLPR_LOG_ERR(
+			"Not enough event queues for 1:1 mapping "
+			"[eth devs: %d, event queues: %d]\n",
+			nb_eth_dev,
+			eventdev_config->nb_eventqueue);
+		return -1;
+	}
+
+	for (i = 0; i < nb_eth_dev; i++) {
+
+		/* Use only the ports enabled */
+		if ((em_conf->eth_portmask & (1 << i)) == 0)
+			continue;
+
+		/* Get the connection id */
+		conn_id = adapter->nb_connections;
+
+		/* Get the connection */
+		conn = &(adapter->conn[conn_id]);
+
+		/* Set 1:1 mapping between eth ports & event queues*/
+		conn->ethdev_id = i;
+		conn->eventq_id = i;
+
+		/* Add all eth queues of one eth port to one event queue */
+		conn->ethdev_rx_qid = -1;
+
+		/* Update no of connections */
+		adapter->nb_connections++;
+
+	}
+
+	/* We have setup one adapter */
+	em_conf->nb_rx_adapter = 1;
+
+	return 0;
+}
+
+static int
 rte_eventmode_helper_validate_conf(struct eventmode_conf *em_conf)
 {
 	int ret;
@@ -255,6 +369,16 @@ rte_eventmode_helper_validate_conf(struct eventmode_conf *em_conf)
 			return ret;
 	}
 
+	/*
+	 * See if rx adapters are specified. Else generate a default conf
+	 * with one rx adapter and all eth queue - event queue mapped.
+	 */
+	if (em_conf->nb_rx_adapter == 0) {
+		ret = rte_eventmode_helper_set_default_conf_rx_adapter(em_conf);
+		if (ret != 0)
+			return ret;
+	}
+
 	return 0;
 }
 
@@ -561,6 +685,9 @@ rte_eventmode_helper_initialize_devs(
 	/* Get eventmode conf */
 	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
 
+	/* Eventmode conf would need eth portmask */
+	em_conf->eth_portmask = mode_conf->eth_portmask;
+
 	/* Validate the conf requested */
 	if (rte_eventmode_helper_validate_conf(em_conf) != 0) {
 		RTE_EM_HLPR_LOG_ERR(
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 9c68605..7cc5776 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -88,6 +88,10 @@ struct eventmode_conf {
 	struct rte_eventmode_helper_event_link_info
 			link[EVENT_MODE_MAX_LCORE_LINKS];
 		/**< Per link conf */
+	uint32_t eth_core_mask;
+		/**< Core mask of cores to be used for software Rx and Tx */
+	uint32_t eth_portmask;
+		/**< Mask of the eth ports to be used */
 	union {
 		struct {
 			uint64_t sched_type			: 2;
-- 
2.7.4

[dpdk-dev] [PATCH 28/39] eventdev: add default conf for event port-lcore link

[Anoob Joseph]

Generate a default conf for event port-lcore link, if not specified in
the conf. This routine will check the number of available ports and then
create links according to the number of cores available.

This patch also adds a new entry in the eventmode conf to denote that
all queues is to be linked with every port. This enables one core to
receive packets from every port.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c         | 109 ++++++++++++++++++++-
 .../rte_eventmode_helper_internal.h                |   5 +
 2 files changed, 113 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index a24d654..191eb77 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -70,6 +70,28 @@ internal_get_next_rx_core(struct eventmode_conf *em_conf,
 	return next_core;
 }
 
+static inline unsigned int
+internal_get_next_active_core(struct eventmode_conf *em_conf,
+		unsigned int prev_core)
+{
+	unsigned int next_core;
+
+get_next_core:
+	/* Get the next core */
+	next_core = rte_get_next_lcore(prev_core, 0, 0);
+
+	/* Check if we have reached max lcores */
+	if (next_core == RTE_MAX_LCORE)
+		return next_core;
+
+	/* Some cores would be reserved as rx cores. Skip them */
+	if (em_conf->eth_core_mask & (1 << next_core)) {
+		prev_core = next_core;
+		goto get_next_core;
+	}
+
+	return next_core;
+}
 
 /* Global functions */
 
@@ -350,6 +372,74 @@ rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf *em_conf)
 }
 
 static int
+rte_eventmode_helper_set_default_conf_link(struct eventmode_conf *em_conf)
+{
+	int i, j;
+	struct eventdev_params *eventdev_config;
+	unsigned int lcore_id = -1;
+	int link_index;
+	struct rte_eventmode_helper_event_link_info *link;
+
+	/*
+	 * Create a 1:1 mapping from event ports to cores. If the number
+	 * of event ports is lesser than the cores, some cores won't
+	 * execute worker. If event ports are more, then some ports won't
+	 * be used.
+	 *
+	 */
+
+	/*
+	 * The event queue-port mapping is done according to the link. Since
+	 * we are falling back to the default link conf, enabling
+	 * "all_ev_queue_to_ev_port" mode flag. This will map all queues to the
+	 * port.
+	 */
+	em_conf->ext_params.all_ev_queue_to_ev_port = 1;
+
+	for (i = 0; i < em_conf->nb_eventdev; i++) {
+
+		/* Get event dev conf */
+		eventdev_config = &(em_conf->eventdev_config[i]);
+
+		/* Loop through the ports */
+		for (j = 0; j < eventdev_config->nb_eventport; j++) {
+
+			/* Get next active core id */
+			lcore_id = internal_get_next_active_core(em_conf,
+					lcore_id);
+
+			if (lcore_id == RTE_MAX_LCORE) {
+				/* Reached max cores */
+				return 0;
+			}
+
+			/* Save the current combination as one link */
+
+			/* Get the index */
+			link_index = em_conf->nb_link;
+
+			/* Get the corresponding link */
+			link = &(em_conf->link[link_index]);
+
+			/* Save link */
+			link->eventdev_id = eventdev_config->eventdev_id;
+			link->event_portid = j;
+			link->lcore_id = lcore_id;
+
+			/*
+			 * Not setting eventq_id as by default all queues
+			 * need to be mapped to the port, and is controlled
+			 * by the operating mode.
+			 */
+
+			/* Update number of links */
+			em_conf->nb_link++;
+		}
+	}
+	return 0;
+}
+
+static int
 rte_eventmode_helper_validate_conf(struct eventmode_conf *em_conf)
 {
 	int ret;
@@ -379,6 +469,16 @@ rte_eventmode_helper_validate_conf(struct eventmode_conf *em_conf)
 			return ret;
 	}
 
+	/*
+	 * See if links are specified. Else generate a default conf for
+	 * the event ports used.
+	 */
+	if (em_conf->nb_link == 0) {
+		ret = rte_eventmode_helper_set_default_conf_link(em_conf);
+		if (ret != 0)
+			return ret;
+	}
+
 	return 0;
 }
 
@@ -508,7 +608,14 @@ rte_eventmode_helper_initialize_eventdev(struct eventmode_conf *em_conf)
 		/* Get event dev ID */
 		eventdev_id = link->eventdev_id;
 
-		queue = &(link->eventq_id);
+		/*
+		 * If "all_ev_queue_to_ev_port" params flag is selected, all
+		 * queues need to be mapped to the port.
+		 */
+		if (em_conf->ext_params.all_ev_queue_to_ev_port)
+			queue = NULL;
+		else
+			queue = &(link->eventq_id);
 
 		/* Link queue to port */
 		ret = rte_event_port_link(eventdev_id, link->event_portid,
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 7cc5776..499cf5d 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -96,6 +96,11 @@ struct eventmode_conf {
 		struct {
 			uint64_t sched_type			: 2;
 		/**< Schedule type */
+			uint64_t all_ev_queue_to_ev_port	: 1;
+		/**<
+		 * When enabled, all event queues need to be mapped to
+		 * each event port
+		 */
 		};
 		uint64_t u64;
 	} ext_params;
-- 
2.7.4

[dpdk-dev] [PATCH 29/39] eventdev: add routines to display the eventmode conf

[Anoob Joseph]

Adding routines to display the eventmode configuration. This gives an
overview of the devices used with the configuration.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/Makefile                      |   1 +
 lib/librte_eventdev/meson.build                   |   1 +
 lib/librte_eventdev/rte_eventdev_version.map      |   1 +
 lib/librte_eventdev/rte_eventmode_helper.c        |   3 +
 lib/librte_eventdev/rte_eventmode_helper.h        |  11 ++
 lib/librte_eventdev/rte_eventmode_helper_prints.c | 161 ++++++++++++++++++++++
 6 files changed, 178 insertions(+)
 create mode 100644 lib/librte_eventdev/rte_eventmode_helper_prints.c

diff --git a/lib/librte_eventdev/Makefile b/lib/librte_eventdev/Makefile
index de2b322..103d1e9 100644
--- a/lib/librte_eventdev/Makefile
+++ b/lib/librte_eventdev/Makefile
@@ -24,6 +24,7 @@ LDLIBS += -lrte_mbuf -lrte_cryptodev -lpthread
 # library source files
 SRCS-y += rte_eventdev.c
 SRCS-y += rte_eventmode_helper.c
+SRCS-y += rte_eventmode_helper_prints.c
 SRCS-y += rte_event_ring.c
 SRCS-y += rte_event_eth_rx_adapter.c
 SRCS-y += rte_event_timer_adapter.c
diff --git a/lib/librte_eventdev/meson.build b/lib/librte_eventdev/meson.build
index 73cd89e..ce7a483 100644
--- a/lib/librte_eventdev/meson.build
+++ b/lib/librte_eventdev/meson.build
@@ -12,6 +12,7 @@ endif
 
 sources = files('rte_eventdev.c',
 		'rte_eventmode_helper.c',
+		'rte_eventmode_helper_prints.c',
 		'rte_event_ring.c',
 		'rte_event_eth_rx_adapter.c',
 		'rte_event_timer_adapter.c',
diff --git a/lib/librte_eventdev/rte_eventdev_version.map b/lib/librte_eventdev/rte_eventdev_version.map
index e156fa0..8b78a68 100644
--- a/lib/librte_eventdev/rte_eventdev_version.map
+++ b/lib/librte_eventdev/rte_eventdev_version.map
@@ -132,4 +132,5 @@ EXPERIMENTAL {
 	rte_eventmode_helper_print_options_description;
 	rte_eventmode_helper_parse_args;
 	rte_eventmode_helper_initialize_devs;
+	rte_eventmode_helper_display_conf;
 };
diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index 191eb77..a333c4a 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -802,6 +802,9 @@ rte_eventmode_helper_initialize_devs(
 		return -1;
 	}
 
+	/* Display the current conf */
+	rte_eventmode_helper_display_conf(mode_conf);
+
 	/* Stop eth devices before setting up adapter */
 	RTE_ETH_FOREACH_DEV(portid) {
 
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index 1ae6a15..d4941be 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -101,6 +101,17 @@ int32_t __rte_experimental
 rte_eventmode_helper_initialize_devs(
 		struct rte_eventmode_helper_conf *mode_conf);
 
+/**
+ * Display event mode conf
+ *
+ * Parse the conf and display the current configuration.
+ *
+ * @param mode_conf
+ *   Configuration of the mode in which app is doing packet handling
+ */
+void __rte_experimental
+rte_eventmode_helper_display_conf(struct rte_eventmode_helper_conf *mode_conf);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/librte_eventdev/rte_eventmode_helper_prints.c b/lib/librte_eventdev/rte_eventmode_helper_prints.c
new file mode 100644
index 0000000..3114d29
--- /dev/null
+++ b/lib/librte_eventdev/rte_eventmode_helper_prints.c
@@ -0,0 +1,161 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C) 2019 Marvell International Ltd.
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+#include <rte_eventmode_helper.h>
+#include "rte_eventmode_helper_internal.h"
+
+static void
+rte_eventmode_display_operating_mode(struct eventmode_conf *em_conf)
+{
+	char sched_types[][32] = {
+		"RTE_SCHED_TYPE_ORDERED",
+		"RTE_SCHED_TYPE_ATOMIC",
+		"RTE_SCHED_TYPE_PARALLEL",
+	};
+	RTE_EM_HLPR_LOG_INFO("Operating mode:");
+
+	RTE_EM_HLPR_LOG_INFO("\tScheduling type: \t%s",
+		sched_types[em_conf->ext_params.sched_type]);
+
+	RTE_EM_HLPR_LOG_INFO("");
+}
+
+static void
+rte_eventmode_display_event_dev_conf(struct eventmode_conf *em_conf)
+{
+	int i;
+	char print_buf[256] = { 0 };
+	char queue_mode[][32] = {
+		"",
+		"ATQ (ALL TYPE QUEUE)",
+		"SINGLE LINK",
+	};
+
+	RTE_EM_HLPR_LOG_INFO("Event Device Configuration:");
+
+	for (i = 0; i < em_conf->nb_eventdev; i++) {
+		sprintf(print_buf,
+			"\tDev ID: %-2d \tQueues: %-2d \tPorts: %-2d",
+			em_conf->eventdev_config[i].eventdev_id,
+			em_conf->eventdev_config[i].nb_eventqueue,
+			em_conf->eventdev_config[i].nb_eventport);
+		sprintf(print_buf + strlen(print_buf),
+			"\tQueue mode: %s",
+			queue_mode[em_conf->eventdev_config[i].ev_queue_mode]);
+		RTE_EM_HLPR_LOG_INFO("%s", print_buf);
+	}
+	RTE_EM_HLPR_LOG_INFO("");
+}
+
+static void
+rte_eventmode_display_rx_adapter_conf(struct eventmode_conf *em_conf)
+{
+	int i, j;
+	int nb_rx_adapter = em_conf->nb_rx_adapter;
+	struct rx_adapter_conf *adapter;
+	struct adapter_connection_info *conn;
+	char print_buf[256] = { 0 };
+
+	RTE_EM_HLPR_LOG_INFO("Rx adapters configured: %d", nb_rx_adapter);
+
+	for (i = 0; i < nb_rx_adapter; i++) {
+		adapter = &(em_conf->rx_adapter[i]);
+		RTE_EM_HLPR_LOG_INFO(
+			"\tRx adaper ID: %-2d\tConnections: %-2d\tEvent dev ID: %-2d"
+			"\tRx core: %-2d",
+			adapter->adapter_id,
+			adapter->nb_connections,
+			adapter->eventdev_id,
+			adapter->rx_core_id);
+
+		for (j = 0; j < adapter->nb_connections; j++) {
+			conn = &(adapter->conn[j]);
+
+			sprintf(print_buf,
+				"\t\tEthdev ID: %-2d", conn->ethdev_id);
+
+			if (conn->ethdev_rx_qid == -1)
+				sprintf(print_buf + strlen(print_buf),
+					"\tEth rx queue: %-2s", "ALL");
+			else
+				sprintf(print_buf + strlen(print_buf),
+					"\tEth rx queue: %-2d",
+					conn->ethdev_rx_qid);
+
+			sprintf(print_buf + strlen(print_buf),
+				"\tEvent queue: %-2d", conn->eventq_id);
+			RTE_EM_HLPR_LOG_INFO("%s", print_buf);
+		}
+	}
+	RTE_EM_HLPR_LOG_INFO("");
+}
+
+static void
+rte_eventmode_display_link_conf(struct eventmode_conf *em_conf)
+{
+	int i;
+	struct rte_eventmode_helper_event_link_info *link;
+	char print_buf[256] = { 0 };
+
+	RTE_EM_HLPR_LOG_INFO("Links configured: %d", em_conf->nb_link);
+
+	for (i = 0; i < em_conf->nb_link; i++) {
+		link = &(em_conf->link[i]);
+
+		sprintf(print_buf,
+			"\tEvent dev ID: %-2d\tEvent port: %-2d",
+			link->eventdev_id,
+			link->event_portid);
+
+		if (em_conf->ext_params.all_ev_queue_to_ev_port)
+			sprintf(print_buf + strlen(print_buf),
+				"Event queue: %-2s\t", "ALL");
+		else
+			sprintf(print_buf + strlen(print_buf),
+				"Event queue: %-2d\t", link->eventq_id);
+
+		sprintf(print_buf + strlen(print_buf),
+			"Lcore: %-2d", link->lcore_id);
+		RTE_EM_HLPR_LOG_INFO("%s", print_buf);
+	}
+	RTE_EM_HLPR_LOG_INFO("");
+}
+
+void __rte_experimental
+rte_eventmode_helper_display_conf(struct rte_eventmode_helper_conf *mode_conf)
+{
+	struct eventmode_conf *em_conf;
+
+	if (mode_conf == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid conf");
+		return;
+	}
+
+	if (mode_conf->mode != RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT)
+		return;
+
+	if (mode_conf->mode_params == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid mode params");
+		return;
+	}
+
+	/* Get eventmode conf */
+	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
+
+	/* Display user exposed operating modes */
+	rte_eventmode_display_operating_mode(em_conf);
+
+	/* Display event device conf */
+	rte_eventmode_display_event_dev_conf(em_conf);
+
+	/* Display Rx adapter conf */
+	rte_eventmode_display_rx_adapter_conf(em_conf);
+
+	/* Display event-lcore link */
+	rte_eventmode_display_link_conf(em_conf);
+}
+
-- 
2.7.4

[dpdk-dev] [PATCH 30/39] eventdev: add routine to access eventmode link info

[Anoob Joseph]

When the application is drafted for single stage eventmode, it will be
efficient to have the loop in the application space, rather than passing
it on to the helper. But application would need to have info on the
links to be able to do that efficiently. This function exposes the links
to that application.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventdev_version.map |  1 +
 lib/librte_eventdev/rte_eventmode_helper.c   | 80 ++++++++++++++++++++++++++++
 lib/librte_eventdev/rte_eventmode_helper.h   | 24 +++++++++
 3 files changed, 105 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventdev_version.map b/lib/librte_eventdev/rte_eventdev_version.map
index 8b78a68..8137cb5 100644
--- a/lib/librte_eventdev/rte_eventdev_version.map
+++ b/lib/librte_eventdev/rte_eventdev_version.map
@@ -133,4 +133,5 @@ EXPERIMENTAL {
 	rte_eventmode_helper_parse_args;
 	rte_eventmode_helper_initialize_devs;
 	rte_eventmode_helper_display_conf;
+	rte_eventmode_helper_get_event_lcore_links;
 };
diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index a333c4a..6c853f6 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -847,3 +847,83 @@ rte_eventmode_helper_initialize_devs(
 
 	return 0;
 }
+
+/* Helper functions for eventmode workers */
+
+uint8_t __rte_experimental
+rte_eventmode_helper_get_event_lcore_links(uint32_t lcore_id,
+		struct rte_eventmode_helper_conf *mode_conf,
+		struct rte_eventmode_helper_event_link_info **links)
+{
+	int i;
+	int index = 0;
+	uint8_t lcore_nb_link = 0;
+	struct rte_eventmode_helper_event_link_info *link;
+	struct rte_eventmode_helper_event_link_info *link_cache;
+	struct eventmode_conf *em_conf = NULL;
+	size_t cache_size;
+	size_t single_link_size;
+
+	if (mode_conf == NULL || links == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid args");
+		return 0;
+	}
+
+	/* Get eventmode conf */
+	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
+
+	if (em_conf == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid event mode conf");
+		return 0;
+	}
+
+	/* Get the number of links registered */
+	for (i = 0; i < em_conf->nb_link; i++) {
+
+		/* Get link */
+		link = &(em_conf->link[i]);
+
+		/* Check if we have link intended for this lcore */
+		if (link->lcore_id == lcore_id) {
+
+			/* Update the number of links for this core */
+			lcore_nb_link++;
+
+		}
+	}
+
+	/* Compute size of one entry to be copied */
+	single_link_size = sizeof(struct rte_eventmode_helper_event_link_info);
+
+	/* Compute size of the buffer required */
+	cache_size = lcore_nb_link *
+			sizeof(struct rte_eventmode_helper_event_link_info);
+
+	/* Allocate memory for caching the links */
+	link_cache = rte_zmalloc("eventmode-event-lcore-links", cache_size,
+			RTE_CACHE_LINE_SIZE);
+
+	/* Get the number of links registered */
+	for (i = 0; i < em_conf->nb_link; i++) {
+
+		/* Get link */
+		link = &(em_conf->link[i]);
+
+		/* Check if we have link intended for this lcore */
+		if (link->lcore_id == lcore_id) {
+
+			/* Cache the link */
+			memcpy(&link_cache[index], link, single_link_size);
+
+			/* Update index */
+			index++;
+		}
+	}
+
+	/* Update the links for application to use the cached links */
+	*links = link_cache;
+
+	/* Return the number of cached links */
+	return lcore_nb_link;
+}
+
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index d4941be..925b660 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -112,6 +112,30 @@ rte_eventmode_helper_initialize_devs(
 void __rte_experimental
 rte_eventmode_helper_display_conf(struct rte_eventmode_helper_conf *mode_conf);
 
+/**
+ * Get event dev - lcore links
+ *
+ * Since the execution loop is in the application, the application would need
+ * the info on which event port to be polled by an lcore etc. This helper
+ * function would help the application in doing so. The 'links' would point
+ * to the memory allocated for the links list, and the application should
+ * release this, once the use is over.
+ *
+ * @param lcore_id
+ *   ID of the lcore for which the links list need to be populated
+ * @param mode_conf
+ *   Configuration of the mode in which app is doing packet handling
+ * @param links
+ *   Used to pass the pointer of the memory allocated by the helper to the
+ *   application
+ * @return
+ *   Number of links found for the lcore
+ */
+uint8_t __rte_experimental
+rte_eventmode_helper_get_event_lcore_links(uint32_t lcore_id,
+		struct rte_eventmode_helper_conf *mode_conf,
+		struct rte_eventmode_helper_event_link_info **links);
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.7.4

[dpdk-dev] [PATCH 31/39] eventdev: add routine to access event queue for eth Tx

[Anoob Joseph]

When the application is drafted for single stage eventmode, it will be
efficient to have the loop in the application space, rather than passing
it on to the helper.

When the application's stage is in ORDERED sched mode, the application
will have to change the sched type of the event to ATOMIC before sending
it, to ensure ingress ordering is maintained. Since, it is application
who would do the tx, this info is required in it's space.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventdev_version.map |  1 +
 lib/librte_eventdev/rte_eventmode_helper.c   | 53 ++++++++++++++++++++++++++++
 lib/librte_eventdev/rte_eventmode_helper.h   | 21 +++++++++++
 3 files changed, 75 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventdev_version.map b/lib/librte_eventdev/rte_eventdev_version.map
index 8137cb5..3cf926a 100644
--- a/lib/librte_eventdev/rte_eventdev_version.map
+++ b/lib/librte_eventdev/rte_eventdev_version.map
@@ -134,4 +134,5 @@ EXPERIMENTAL {
 	rte_eventmode_helper_initialize_devs;
 	rte_eventmode_helper_display_conf;
 	rte_eventmode_helper_get_event_lcore_links;
+	rte_eventmode_helper_get_tx_queue;
 };
diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index 6c853f6..e7670e0 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -93,6 +93,24 @@ internal_get_next_active_core(struct eventmode_conf *em_conf,
 	return next_core;
 }
 
+static struct eventdev_params *
+internal_get_eventdev_params(struct eventmode_conf *em_conf,
+		uint8_t eventdev_id)
+{
+	int i;
+
+	for (i = 0; i < em_conf->nb_eventdev; i++) {
+		if (em_conf->eventdev_config[i].eventdev_id == eventdev_id)
+			break;
+	}
+
+	/* No match */
+	if (i == em_conf->nb_eventdev)
+		return NULL;
+
+	return &(em_conf->eventdev_config[i]);
+}
+
 /* Global functions */
 
 void __rte_experimental
@@ -927,3 +945,38 @@ rte_eventmode_helper_get_event_lcore_links(uint32_t lcore_id,
 	return lcore_nb_link;
 }
 
+uint8_t __rte_experimental
+rte_eventmode_helper_get_tx_queue(struct rte_eventmode_helper_conf *mode_conf,
+		uint8_t eventdev_id)
+{
+	struct eventdev_params *eventdev_config;
+	struct eventmode_conf *em_conf;
+
+	if (mode_conf == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid conf");
+		return (uint8_t)(-1);
+	}
+
+	if (mode_conf->mode_params == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid mode params");
+		return (uint8_t)(-1);
+	}
+
+	/* Get eventmode conf */
+	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
+
+	/* Get event device conf */
+	eventdev_config = internal_get_eventdev_params(em_conf, eventdev_id);
+
+	if (eventdev_config == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Error reading eventdev conf");
+		return (uint8_t)(-1);
+	}
+
+	/*
+	 * The last queue would be reserved to be used as atomic queue for the
+	 * last stage (eth packet tx stage)
+	 */
+	return eventdev_config->nb_eventqueue - 1;
+}
+
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index 925b660..cd6d708 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -136,6 +136,27 @@ rte_eventmode_helper_get_event_lcore_links(uint32_t lcore_id,
 		struct rte_eventmode_helper_conf *mode_conf,
 		struct rte_eventmode_helper_event_link_info **links);
 
+/**
+ * Get eventdev tx queue
+ *
+ * If the application uses event device which does not support internal port
+ * then it needs to submit the events to an atomic Tx queue before final
+ * transmission. The Tx queue will be atomic to make sure that ingress order of
+ * the packets is maintained. This Tx queue will be created internally by the
+ * eventmode helper subsystem, and application will need it's queue ID when it
+ * is running the execution loop.
+ *
+ * @param mode_conf
+ *   Configuration of the mode in which app is doing packet handling
+ * @param eventdev_id
+ *   Event device ID
+ * @return
+ *   Tx queue ID
+ */
+uint8_t __rte_experimental
+rte_eventmode_helper_get_tx_queue(struct rte_eventmode_helper_conf *mode_conf,
+		uint8_t eventdev_id);
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.7.4

[dpdk-dev] [PATCH 32/39] eventdev: add routine to launch eventmode workers

[Anoob Joseph]

With eventmode, workers could be drafted differently according to the
capabilities of the underlying event device. The added function would
receive an array of such workers and probes the eventmode properties to
choose the worker.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventdev_version.map       |   1 +
 lib/librte_eventdev/rte_eventmode_helper.c         | 240 +++++++++++++++++++++
 lib/librte_eventdev/rte_eventmode_helper.h         |  49 +++++
 .../rte_eventmode_helper_internal.h                |   3 +
 4 files changed, 293 insertions(+)

diff --git a/lib/librte_eventdev/rte_eventdev_version.map b/lib/librte_eventdev/rte_eventdev_version.map
index 3cf926a..665836e 100644
--- a/lib/librte_eventdev/rte_eventdev_version.map
+++ b/lib/librte_eventdev/rte_eventdev_version.map
@@ -135,4 +135,5 @@ EXPERIMENTAL {
 	rte_eventmode_helper_display_conf;
 	rte_eventmode_helper_get_event_lcore_links;
 	rte_eventmode_helper_get_tx_queue;
+	rte_eventmode_helper_launch_worker;
 };
diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index e7670e0..77a5a4e 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -2,6 +2,7 @@
  * Copyright (C) 2019 Marvell International Ltd.
  */
 #include <getopt.h>
+#include <stdbool.h>
 
 #include <rte_ethdev.h>
 #include <rte_eventdev.h>
@@ -13,6 +14,8 @@
 
 #define CMD_LINE_OPT_TRANSFER_MODE	"transfer-mode"
 
+static volatile bool eth_core_running;
+
 static const char short_options[] =
 	""
 	;
@@ -111,6 +114,16 @@ internal_get_eventdev_params(struct eventmode_conf *em_conf,
 	return &(em_conf->eventdev_config[i]);
 }
 
+static inline bool
+internal_dev_has_burst_mode(uint8_t dev_id)
+{
+	struct rte_event_dev_info dev_info;
+
+	rte_event_dev_info_get(dev_id, &dev_info);
+	return (dev_info.event_dev_cap & RTE_EVENT_DEV_CAP_BURST_MODE) ?
+			true : false;
+}
+
 /* Global functions */
 
 void __rte_experimental
@@ -980,3 +993,230 @@ rte_eventmode_helper_get_tx_queue(struct rte_eventmode_helper_conf *mode_conf,
 	return eventdev_config->nb_eventqueue - 1;
 }
 
+/* Helper functions for launching workers */
+
+static int32_t
+rte_eventmode_helper_start_worker_eth_core(struct eventmode_conf *em_conf,
+		uint32_t lcore_id)
+{
+	uint32_t service_id[EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE];
+	struct rx_adapter_conf *rx_adapter;
+	int service_count = 0;
+	int adapter_id;
+	int32_t ret;
+	int i;
+
+	RTE_EM_HLPR_LOG_INFO(
+		"Entering eth_core processing on lcore %u", lcore_id);
+
+	/*
+	 * Need to parse adapter conf to see which of all Rx adapters need
+	 * to be handled by this core.
+	 */
+	for (i = 0; i < em_conf->nb_rx_adapter; i++) {
+		/* Check if we have exceeded the max allowed */
+		if (service_count > EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Exceeded the max allowed adapters per rx core");
+			break;
+		}
+
+		rx_adapter = &(em_conf->rx_adapter[i]);
+		if (rx_adapter->rx_core_id != lcore_id)
+			continue;
+
+		/* Adapter need to be handled by this core */
+		adapter_id = rx_adapter->adapter_id;
+
+		/* Get the service ID for the adapters */
+		ret = rte_event_eth_rx_adapter_service_id_get(adapter_id,
+				&(service_id[service_count]));
+
+		if (ret != -ESRCH && ret != 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error getting service ID used by Rx adapter");
+			return ret;
+		}
+
+		/* Update service count */
+		service_count++;
+	}
+
+	eth_core_running = true;
+
+	while (eth_core_running) {
+		for (i = 0; i < service_count; i++) {
+			/* Initiate adapter service */
+			rte_service_run_iter_on_app_lcore(service_id[i], 0);
+		}
+	}
+
+	return 0;
+}
+
+static int32_t
+rte_eventmode_helper_stop_worker_eth_core(void)
+{
+	if (eth_core_running) {
+		RTE_EM_HLPR_LOG_INFO("Stopping rx cores\n");
+		eth_core_running = false;
+	}
+	return 0;
+}
+
+static struct rte_eventmode_helper_app_worker_params *
+rte_eventmode_helper_find_worker(uint32_t lcore_id,
+		struct eventmode_conf *em_conf,
+		struct rte_eventmode_helper_app_worker_params *app_wrkrs,
+		uint8_t nb_wrkr_param)
+{
+	struct rte_eventmode_helper_event_link_info *link = NULL;
+	uint8_t eventdev_id;
+	struct eventdev_params *eventdev_config;
+	int i;
+	struct rte_eventmode_helper_app_worker_params curr_conf = {
+			{{0} }, NULL};
+	struct rte_eventmode_helper_app_worker_params *tmp_wrkr;
+
+	/*
+	 * Event device to be used will be derived from the first lcore-event
+	 * link.
+	 *
+	 * Assumption: All lcore-event links tied to a core would be using the
+	 * same event device. in other words, one core would be polling on
+	 * queues of a single event device only.
+	 */
+
+	/* Get a link for this lcore */
+	for (i = 0; i < em_conf->nb_link; i++) {
+		link = &(em_conf->link[i]);
+		if (link->lcore_id == lcore_id)
+			break;
+	}
+
+	if (link == NULL) {
+		RTE_EM_HLPR_LOG_ERR(
+			"No valid link found for lcore(%d)", lcore_id);
+		return NULL;
+	}
+
+	/* Get event dev ID */
+	eventdev_id = link->eventdev_id;
+
+	/* Get the corresponding eventdev config */
+	eventdev_config = internal_get_eventdev_params(em_conf, eventdev_id);
+
+	/* Populate the curr_conf with the capabilities */
+
+	/* Check for burst mode */
+	if (internal_dev_has_burst_mode(eventdev_id))
+		curr_conf.cap.burst = RTE_EVENTMODE_HELPER_RX_TYPE_BURST;
+	else
+		curr_conf.cap.burst = RTE_EVENTMODE_HELPER_RX_TYPE_NON_BURST;
+
+	/* Now parse the passed list and see if we have matching capabilities */
+
+	/* Initialize the pointer used to traverse the list */
+	tmp_wrkr = app_wrkrs;
+
+	for (i = 0; i < nb_wrkr_param; i++, tmp_wrkr++) {
+
+		/* Skip this if capabilities are not matching */
+		if (tmp_wrkr->cap.u64 != curr_conf.cap.u64)
+			continue;
+
+		/* If the checks pass, we have a match */
+		return tmp_wrkr;
+	}
+
+	/* TODO required for ATQ */
+	RTE_SET_USED(eventdev_config);
+
+	return NULL;
+}
+
+static int
+rte_eventmode_helper_verify_match_worker(
+	struct rte_eventmode_helper_app_worker_params *match_wrkr)
+{
+	/* Verify registered worker */
+	if (match_wrkr->worker_thread == NULL) {
+		RTE_EM_HLPR_LOG_ERR("No worker registered for second stage");
+		return 0;
+	}
+
+	/* Success */
+	return 1;
+}
+
+void __rte_experimental
+rte_eventmode_helper_launch_worker(struct rte_eventmode_helper_conf *mode_conf,
+		struct rte_eventmode_helper_app_worker_params *app_wrkr,
+		uint8_t nb_wrkr_param)
+{
+	struct rte_eventmode_helper_app_worker_params *match_wrkr;
+	uint32_t lcore_id;
+	struct eventmode_conf *em_conf;
+
+	if (mode_conf == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid conf");
+		return;
+	}
+
+	if (mode_conf->mode_params == NULL) {
+		RTE_EM_HLPR_LOG_ERR("Invalid mode params");
+		return;
+	}
+
+	/* Get eventmode conf */
+	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
+
+	/* Get core ID */
+	lcore_id = rte_lcore_id();
+
+	/* TODO check capability for rx core */
+
+	/* Check if this is rx core */
+	if (em_conf->eth_core_mask & (1 << lcore_id)) {
+		rte_eventmode_helper_start_worker_eth_core(em_conf, lcore_id);
+		return;
+	}
+
+	if (app_wrkr == NULL || nb_wrkr_param == 0) {
+		RTE_EM_HLPR_LOG_ERR("Invalid args");
+		return;
+	}
+
+	/*
+	 * This is a regular worker thread. The application would be
+	 * registering multiple workers with various capabilities. The
+	 * worker to be run will be selected by the capabilities of the
+	 * event device configured.
+	 */
+
+	/* Get the first matching worker for the event device */
+	match_wrkr = rte_eventmode_helper_find_worker(lcore_id,
+			em_conf,
+			app_wrkr,
+			nb_wrkr_param);
+
+	if (match_wrkr == NULL) {
+		RTE_EM_HLPR_LOG_ERR(
+			"No matching worker registered for lcore %d", lcore_id);
+		goto clean_and_exit;
+	}
+
+	/* Verify sanity of the matched worker */
+	if (rte_eventmode_helper_verify_match_worker(match_wrkr) != 1) {
+		RTE_EM_HLPR_LOG_ERR("Error in validating the matched worker");
+		goto clean_and_exit;
+	}
+
+	/* Launch the worker thread */
+	match_wrkr->worker_thread(mode_conf);
+
+clean_and_exit:
+
+	/* Flag eth_cores to stop, if started */
+	rte_eventmode_helper_stop_worker_eth_core();
+}
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index cd6d708..1235ca4 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -17,6 +17,20 @@ enum rte_eventmode_helper_pkt_transfer_mode {
 	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT,
 };
 
+/* Event mode packet rx types */
+enum rte_eventmode_helper_rx_types {
+	RTE_EVENTMODE_HELPER_RX_TYPE_INVALID = 0,
+	RTE_EVENTMODE_HELPER_RX_TYPE_NON_BURST,
+	RTE_EVENTMODE_HELPER_RX_TYPE_BURST,
+	RTE_EVENTMODE_HELPER_RX_TYPE_MAX = 16
+};
+
+/* Event mode packet tx types */
+enum rte_eventmode_helper_tx_types {
+	RTE_EVETNMODE_HELPER_TX_TYPE_INVALID = 0,
+	RTE_EVENTMODE_HELPER_TX_TYPE_MAX = 16
+};
+
 struct rte_eventmode_helper_conf {
 	enum rte_eventmode_helper_pkt_transfer_mode mode;
 		/**< Packet transfer mode of the application */
@@ -41,6 +55,20 @@ struct rte_eventmode_helper_event_link_info {
 		/**< Lcore to be polling on this port */
 };
 
+/* Workers registered by the application */
+struct rte_eventmode_helper_app_worker_params {
+	union {
+		struct {
+			uint64_t burst : 4;
+			/**< Specify status of rx type burst */
+		};
+		uint64_t u64;
+	} cap;
+			/**< Capabilities of this worker */
+	void (*worker_thread)(void *mode_conf);
+			/**< Worker thread */
+};
+
 /* Common helper functions for command line parsing */
 
 /**
@@ -157,6 +185,27 @@ uint8_t __rte_experimental
 rte_eventmode_helper_get_tx_queue(struct rte_eventmode_helper_conf *mode_conf,
 		uint8_t eventdev_id);
 
+/**
+ * Launch eventmode worker
+ *
+ * The application can request the eventmode helper subsystem to launch the
+ * worker based on the capabilities of event device and the options selected
+ * while initializing the eventmode.
+ *
+ * @param mode_conf
+ *   Configuration of the mode in which app is doing packet handling
+ * @param app_wrkr
+ *   List of all the workers registered by application, along with it's
+ *   capabilities
+ * @param nb_wrkr_param
+ *   Number of workers passed by the application
+ *
+ */
+void __rte_experimental
+rte_eventmode_helper_launch_worker(struct rte_eventmode_helper_conf *mode_conf,
+		struct rte_eventmode_helper_app_worker_params *app_wrkr,
+		uint8_t nb_wrkr_param);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 499cf5d..906766c 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -48,6 +48,9 @@
 #define EVENT_MODE_MAX_LCORE_LINKS \
 	(EVENT_MODE_MAX_EVENT_DEVS * EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV)
 
+/* Max adapters that one Rx core can handle */
+#define EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE EVENT_MODE_MAX_RX_ADAPTERS
+
 /* Event dev params */
 struct eventdev_params {
 	uint8_t eventdev_id;
-- 
2.7.4

[dpdk-dev] [PATCH 33/39] eventdev: add Tx adapter support

[Anoob Joseph]

Adding support for Tx adapter.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c         | 349 ++++++++++++++++++++-
 lib/librte_eventdev/rte_eventmode_helper.h         |  24 ++
 .../rte_eventmode_helper_internal.h                |  30 ++
 lib/librte_eventdev/rte_eventmode_helper_prints.c  |   9 +
 4 files changed, 399 insertions(+), 13 deletions(-)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index 77a5a4e..f237cab 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -8,6 +8,7 @@
 #include <rte_eventdev.h>
 #include <rte_eventmode_helper.h>
 #include <rte_event_eth_rx_adapter.h>
+#include <rte_event_eth_tx_adapter.h>
 #include <rte_malloc.h>
 
 #include "rte_eventmode_helper_internal.h"
@@ -50,25 +51,51 @@ internal_parse_decimal(const char *str)
 	return num;
 }
 
+static int
+internal_get_enabled_cores(unsigned int core_mask)
+{
+	int i;
+	int count = 0;
+
+	RTE_LCORE_FOREACH(i) {
+		/* Check if this core is enabled in core_mask*/
+		if (core_mask & (1 << i)) {
+			/* We have enabled core */
+			count++;
+		}
+	}
+	return count;
+}
+
 static inline unsigned int
-internal_get_next_rx_core(struct eventmode_conf *em_conf,
-		unsigned int prev_core)
+internal_get_next_eth_core(struct eventmode_conf *em_conf)
 {
 	unsigned int next_core;
+	static unsigned int prev_core = -1;
+
+	/*
+	 * Make sure we have atleast one eth core running, else the following
+	 * logic would lead to an infinite loop.
+	 */
+	if (internal_get_enabled_cores(em_conf->eth_core_mask) == 0) {
+		RTE_EM_HLPR_LOG_INFO("No enabled eth core found");
+		return RTE_MAX_LCORE;
+	}
 
 get_next_core:
 	/* Get the next core */
-	next_core = rte_get_next_lcore(prev_core, 0, 0);
+	next_core = rte_get_next_lcore(prev_core, 0, 1);
 
 	/* Check if we have reached max lcores */
 	if (next_core == RTE_MAX_LCORE)
 		return next_core;
 
+	/* Update prev_core */
+	prev_core = next_core;
+
 	/* Only some cores would be marked as rx cores. Skip others */
-	if (!(em_conf->eth_core_mask & (1 << next_core))) {
-		prev_core = next_core;
+	if (!(em_conf->eth_core_mask & (1 << next_core)))
 		goto get_next_core;
-	}
 
 	return next_core;
 }
@@ -164,7 +191,7 @@ static void
 em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)
 {
 	struct eventmode_conf *em_conf = NULL;
-	unsigned int rx_core_id;
+	unsigned int eth_core_id;
 
 	/* Set default conf */
 
@@ -180,12 +207,21 @@ em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)
 	/* Schedule type: ordered */
 	/* FIXME */
 	em_conf->ext_params.sched_type = RTE_SCHED_TYPE_ORDERED;
-	/* Set rx core. Use first core other than master core as Rx core */
-	rx_core_id = rte_get_next_lcore(0, /* curr core */
-					1, /* skip master core */
-					0  /* wrap */);
+	/* Set two cores as eth cores for Rx & Tx */
+
+	/* Use first core other than master core as Rx core */
+	eth_core_id = rte_get_next_lcore(0,	/* curr core */
+					 1,	/* skip master core */
+					 0	/* wrap */);
 
-	em_conf->eth_core_mask = (1 << rx_core_id);
+	em_conf->eth_core_mask = (1 << eth_core_id);
+
+	/* Use next core as Tx core */
+	eth_core_id = rte_get_next_lcore(eth_core_id,	/* curr core */
+					 1,		/* skip master core */
+					 0		/* wrap */);
+
+	em_conf->eth_core_mask |= (1 << eth_core_id);
 }
 
 struct rte_eventmode_helper_conf * __rte_experimental
@@ -354,7 +390,7 @@ rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf *em_conf)
 	/* Set adapter conf */
 	adapter->eventdev_id = eventdev_id;
 	adapter->adapter_id = adapter_id;
-	adapter->rx_core_id = internal_get_next_rx_core(em_conf, -1);
+	adapter->rx_core_id = internal_get_next_eth_core(em_conf);
 
 	/*
 	 * All queues of one eth device (port) will be mapped to one event
@@ -403,6 +439,100 @@ rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf *em_conf)
 }
 
 static int
+rte_eventmode_helper_set_default_conf_tx_adapter(struct eventmode_conf *em_conf)
+{
+	int nb_eth_dev;
+	int eventdev_id;
+	int adapter_id;
+	int i;
+	int conn_id;
+	struct eventdev_params *eventdev_config;
+	struct tx_adapter_conf *tx_adapter;
+	struct tx_adapter_connection_info *conn;
+
+	/*
+	 * Create one Tx adapter with all eth queues mapped to event queues
+	 * 1:1.
+	 */
+
+	if (em_conf->nb_eventdev == 0) {
+		RTE_EM_HLPR_LOG_ERR("No event devs registered");
+		return -1;
+	}
+
+	/* Get the number of eth devs */
+	nb_eth_dev = rte_eth_dev_count_avail();
+
+	/* Use the first event dev */
+	eventdev_config = &(em_conf->eventdev_config[0]);
+
+	/* Get eventdev ID */
+	eventdev_id = eventdev_config->eventdev_id;
+	adapter_id = 0;
+
+	/* Get adapter conf */
+	tx_adapter = &(em_conf->tx_adapter[adapter_id]);
+
+	/* Set adapter conf */
+	tx_adapter->eventdev_id = eventdev_id;
+	tx_adapter->adapter_id = adapter_id;
+
+	/* TODO: Tx core is required only when internal port is not present */
+
+	tx_adapter->tx_core_id = internal_get_next_eth_core(em_conf);
+
+	/*
+	 * Application would need to use one event queue per adapter for
+	 * submitting packets for Tx. Reserving the last queue available
+	 * and decrementing the total available event queues for this
+	 */
+
+	/* Queue numbers start at 0 */
+	tx_adapter->tx_ev_queue = eventdev_config->nb_eventqueue - 1;
+
+	/* Update the number of event queues available in eventdev */
+	eventdev_config->nb_eventqueue--;
+
+	/*
+	 * All Tx queues of the eth device (port) will be mapped to the event
+	 * device.
+	 */
+
+	/* Set defaults for connections */
+
+	/*
+	 * One eth device (port) would be one connection. All Tx queues of
+	 * the device would be mapped to the Tx adapter.
+	 */
+
+	for (i = 0; i < nb_eth_dev; i++) {
+
+		/* Use only the ports enabled */
+		if ((em_conf->eth_portmask & (1 << i)) == 0)
+			continue;
+
+		/* Get the connection id */
+		conn_id = tx_adapter->nb_connections;
+
+		/* Get the connection */
+		conn = &(tx_adapter->conn[conn_id]);
+
+		/* Add ethdev to connections */
+		conn->ethdev_id = i;
+
+		/* Add all eth tx queues to adapter */
+		conn->ethdev_tx_qid = -1;
+
+		/* Update no of connections */
+		tx_adapter->nb_connections++;
+	}
+
+	/* We have setup one adapter */
+	em_conf->nb_tx_adapter = 1;
+	return 0;
+}
+
+static int
 rte_eventmode_helper_set_default_conf_link(struct eventmode_conf *em_conf)
 {
 	int i, j;
@@ -501,6 +631,16 @@ rte_eventmode_helper_validate_conf(struct eventmode_conf *em_conf)
 	}
 
 	/*
+	 * See if tx adapters are specified. Else generate a default conf
+	 * with one tx adapter.
+	 */
+	if (em_conf->nb_tx_adapter == 0) {
+		ret = rte_eventmode_helper_set_default_conf_tx_adapter(em_conf);
+		if (ret != 0)
+			return ret;
+	}
+
+	/*
 	 * See if links are specified. Else generate a default conf for
 	 * the event ports used.
 	 */
@@ -799,6 +939,150 @@ rte_eventmode_helper_initialize_rx_adapter(struct eventmode_conf *em_conf)
 	return 0;
 }
 
+static int
+tx_adapter_configure(struct eventmode_conf *em_conf,
+	struct tx_adapter_conf *adapter)
+{
+	int ret, j;
+	uint8_t tx_port_id = 0;
+	uint8_t eventdev_id;
+	uint32_t service_id;
+	struct rte_event_port_conf port_conf = {0};
+	struct rte_event_dev_info evdev_default_conf = {0};
+	struct tx_adapter_connection_info *conn;
+	struct eventdev_params *eventdev_config;
+
+	/* Get event dev ID */
+	eventdev_id = adapter->eventdev_id;
+
+	/* Get event device conf */
+	eventdev_config = internal_get_eventdev_params(em_conf, eventdev_id);
+
+	/* Create Tx adapter */
+
+	/* Get default configuration of event dev */
+	ret = rte_event_dev_info_get(eventdev_id, &evdev_default_conf);
+	if (ret < 0) {
+		RTE_EM_HLPR_LOG_ERR(
+			"Error in getting event device info[devID:%d]",
+			eventdev_id);
+		return ret;
+	}
+
+	/* Setup port conf */
+	port_conf.new_event_threshold =
+			evdev_default_conf.max_num_events;
+	port_conf.dequeue_depth =
+			evdev_default_conf.max_event_port_dequeue_depth;
+	port_conf.enqueue_depth =
+			evdev_default_conf.max_event_port_enqueue_depth;
+
+	/* Create Tx adapter */
+	ret = rte_event_eth_tx_adapter_create(adapter->adapter_id,
+			adapter->eventdev_id,
+			&port_conf);
+	if (ret < 0) {
+		RTE_EM_HLPR_LOG_ERR("Error in Tx adapter creation");
+		return ret;
+	}
+
+	/* Setup various connections in the adapter */
+	for (j = 0; j < adapter->nb_connections; j++) {
+
+		/* Get connection */
+		conn = &(adapter->conn[j]);
+
+		/* Add queue to the adapter */
+		ret = rte_event_eth_tx_adapter_queue_add(
+				adapter->adapter_id,
+				conn->ethdev_id,
+				conn->ethdev_tx_qid);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error in adding eth queue in Tx adapter");
+			return ret;
+		}
+	}
+
+	/* Get event port used by the adapter */
+	ret = rte_event_eth_tx_adapter_event_port_get(
+			adapter->adapter_id,
+			&tx_port_id);
+	if (ret) {
+		RTE_EM_HLPR_LOG_ERR("Failed to get Tx adapter port ID");
+		return ret;
+	}
+
+	/*
+	 * TODO: event queue for Tx adapter is required only if the
+	 * INTERNAL PORT is not present.
+	 */
+
+	/*
+	 * Tx event queue would be reserved for Tx adapter. Need to unlink
+	 * this queue from all other ports
+	 *
+	 */
+	for (j = 0; j < eventdev_config->nb_eventport; j++) {
+		rte_event_port_unlink(eventdev_id, j,
+				      &(adapter->tx_ev_queue), 1);
+	}
+
+	ret = rte_event_port_link(
+			eventdev_id,
+			tx_port_id,
+			&(adapter->tx_ev_queue),
+			NULL, 1);
+	if (ret != 1) {
+		RTE_EM_HLPR_LOG_ERR("Failed to link event queue to port");
+		return ret;
+	}
+
+	/* Get the service ID used by Tx adapter */
+	ret = rte_event_eth_tx_adapter_service_id_get(adapter->adapter_id,
+						      &service_id);
+	if (ret != -ESRCH && ret != 0) {
+		RTE_EM_HLPR_LOG_ERR(
+			"Error getting service ID used by adapter");
+		return ret;
+	}
+
+	/*
+	 * TODO
+	 * Tx core will invoke the service when required. The runstate check
+	 * is not required.
+	 *
+	 */
+	rte_service_set_runstate_mapped_check(service_id, 0);
+
+	/* Start adapter */
+	ret = rte_event_eth_tx_adapter_start(adapter->adapter_id);
+	if (ret) {
+		RTE_EM_HLPR_LOG_ERR("Error in starting Tx adapter");
+		return ret;
+	}
+
+	return 0;
+}
+
+static int
+rte_eventmode_helper_initialize_tx_adapter(struct eventmode_conf *em_conf)
+{
+	int i, ret;
+	struct tx_adapter_conf *adapter;
+
+	/* Configure Tx adapters */
+	for (i = 0; i < em_conf->nb_tx_adapter; i++) {
+		adapter = &(em_conf->tx_adapter[i]);
+		ret = tx_adapter_configure(em_conf, adapter);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR("Tx adapter configuration failed");
+			return ret;
+		}
+	}
+	return 0;
+}
+
 int32_t __rte_experimental
 rte_eventmode_helper_initialize_devs(
 		struct rte_eventmode_helper_conf *mode_conf)
@@ -861,6 +1145,11 @@ rte_eventmode_helper_initialize_devs(
 	if (ret != 0)
 		return ret;
 
+	/* Setup Tx adapter */
+	ret = rte_eventmode_helper_initialize_tx_adapter(em_conf);
+	if (ret != 0)
+		return ret;
+
 	/* Start eth devices after setting up adapter */
 	RTE_ETH_FOREACH_DEV(portid) {
 
@@ -1001,6 +1290,7 @@ rte_eventmode_helper_start_worker_eth_core(struct eventmode_conf *em_conf,
 {
 	uint32_t service_id[EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE];
 	struct rx_adapter_conf *rx_adapter;
+	struct tx_adapter_conf *tx_adapter;
 	int service_count = 0;
 	int adapter_id;
 	int32_t ret;
@@ -1042,6 +1332,39 @@ rte_eventmode_helper_start_worker_eth_core(struct eventmode_conf *em_conf,
 		service_count++;
 	}
 
+	/*
+	 * Need to parse adapter conf to see which all Tx adapters need to be
+	 * handled this core.
+	 */
+	for (i = 0; i < em_conf->nb_tx_adapter; i++) {
+		/* Check if we have exceeded the max allowed */
+		if (service_count > EVENT_MODE_MAX_ADAPTERS_PER_TX_CORE) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Exceeded the max allowed adapters per Tx core");
+			break;
+		}
+
+		tx_adapter = &(em_conf->tx_adapter[i]);
+		if (tx_adapter->tx_core_id != lcore_id)
+			continue;
+
+		/* Adapter need to be handled by this core */
+		adapter_id = tx_adapter->adapter_id;
+
+		/* Get the service ID for the adapters */
+		ret = rte_event_eth_tx_adapter_service_id_get(adapter_id,
+				&(service_id[service_count]));
+
+		if (ret != -ESRCH && ret != 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error getting service ID used by Tx adapter");
+			return ret;
+		}
+
+		/* Update service count */
+		service_count++;
+	}
+
 	eth_core_running = true;
 
 	while (eth_core_running) {
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index 1235ca4..2212622 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -10,6 +10,14 @@ extern "C" {
 
 #include <rte_common.h>
 #include <rte_compat.h>
+#include <rte_config.h>
+#include <rte_event_eth_tx_adapter.h>
+
+/* Flag to indicate that the event device used by all adapters is same */
+#define RTE_EM_HELPER_TX_EV_LINK_COMMON_EVENT_DEV	(1 << 0)
+
+/* Flag to indicate that the event queue to be used for all adapters is same */
+#define RTE_EM_HELPER_TX_EV_LINK_COMMON_EVENT_QUEUE	(1 << 1)
 
 /* Packet transfer mode of the application */
 enum rte_eventmode_helper_pkt_transfer_mode {
@@ -55,6 +63,22 @@ struct rte_eventmode_helper_event_link_info {
 		/**< Lcore to be polling on this port */
 };
 
+/*
+ * Tx event queue - port link conf
+ *
+ * Application would need to know which event queue would correspond to which
+ * eth port, so that it can send out accordingly.
+ */
+struct rte_eventmode_helper_tx_ev_link_conf {
+	uint8_t flags;
+	struct {
+		uint8_t eventdev_id;
+			/**< Event device ID */
+		uint8_t tx_ev_queue;
+			/**< Tx event queue */
+	} ports[RTE_MAX_ETHPORTS];
+};
+
 /* Workers registered by the application */
 struct rte_eventmode_helper_app_worker_params {
 	union {
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 906766c..1daca22 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -38,9 +38,15 @@
 /* Max Rx adapters supported */
 #define EVENT_MODE_MAX_RX_ADAPTERS RTE_EVENT_MAX_DEVS
 
+/* Max Tx adapters supported */
+#define EVENT_MODE_MAX_TX_ADAPTERS RTE_EVENT_MAX_DEVS
+
 /* Max Rx adapter connections */
 #define EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER 16
 
+/* Max Tx adapter connections */
+#define EVENT_MODE_MAX_CONNECTIONS_PER_TX_ADAPTER 16
+
 /* Max event queues supported per event device */
 #define EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV RTE_EVENT_MAX_QUEUES_PER_DEV
 
@@ -51,6 +57,9 @@
 /* Max adapters that one Rx core can handle */
 #define EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE EVENT_MODE_MAX_RX_ADAPTERS
 
+/* Max adapters that one Tx core can handle */
+#define EVENT_MODE_MAX_ADAPTERS_PER_TX_CORE EVENT_MODE_MAX_TX_ADAPTERS
+
 /* Event dev params */
 struct eventdev_params {
 	uint8_t eventdev_id;
@@ -76,6 +85,23 @@ struct rx_adapter_conf {
 			conn[EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER];
 };
 
+/* Tx adapter connection info */
+struct tx_adapter_connection_info {
+	uint8_t ethdev_id;
+	int32_t ethdev_tx_qid;
+};
+
+/* Tx adapter conf */
+struct tx_adapter_conf {
+	int32_t eventdev_id;
+	int32_t adapter_id;
+	uint32_t tx_core_id;
+	uint8_t nb_connections;
+	struct tx_adapter_connection_info
+			conn[EVENT_MODE_MAX_CONNECTIONS_PER_TX_ADAPTER];
+	uint8_t tx_ev_queue;
+};
+
 /* Eventmode conf data */
 struct eventmode_conf {
 	int nb_eventdev;
@@ -86,6 +112,10 @@ struct eventmode_conf {
 		/**< No of Rx adapters */
 	struct rx_adapter_conf rx_adapter[EVENT_MODE_MAX_RX_ADAPTERS];
 		/**< Rx adapter conf */
+	uint8_t nb_tx_adapter;
+		/**< No of Tx adapters */
+	struct tx_adapter_conf tx_adapter[EVENT_MODE_MAX_TX_ADAPTERS];
+		/** Tx adapter conf */
 	uint8_t nb_link;
 		/**< No of links */
 	struct rte_eventmode_helper_event_link_info
diff --git a/lib/librte_eventdev/rte_eventmode_helper_prints.c b/lib/librte_eventdev/rte_eventmode_helper_prints.c
index 3114d29..387302a 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_prints.c
+++ b/lib/librte_eventdev/rte_eventmode_helper_prints.c
@@ -95,6 +95,12 @@ rte_eventmode_display_rx_adapter_conf(struct eventmode_conf *em_conf)
 }
 
 static void
+rte_eventmode_display_tx_adapter_conf(struct eventmode_conf *em_conf)
+{
+	RTE_SET_USED(em_conf);
+}
+
+static void
 rte_eventmode_display_link_conf(struct eventmode_conf *em_conf)
 {
 	int i;
@@ -155,6 +161,9 @@ rte_eventmode_helper_display_conf(struct rte_eventmode_helper_conf *mode_conf)
 	/* Display Rx adapter conf */
 	rte_eventmode_display_rx_adapter_conf(em_conf);
 
+	/* Display Tx adapter conf */
+	rte_eventmode_display_tx_adapter_conf(em_conf);
+
 	/* Display event-lcore link */
 	rte_eventmode_display_link_conf(em_conf);
 }
-- 
2.7.4

[dpdk-dev] [PATCH 34/39] eventdev: add support for internal ports

[Anoob Joseph]

For eventdev-ethdev combinations having Tx internal port & Rx internal
port, usage of ethcore is not needed.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c         | 213 +++++++++++++++++----
 lib/librte_eventdev/rte_eventmode_helper.h         |   4 +
 .../rte_eventmode_helper_internal.h                |   1 +
 lib/librte_eventdev/rte_eventmode_helper_prints.c  |  19 +-
 4 files changed, 192 insertions(+), 45 deletions(-)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index f237cab..2451cac 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -142,6 +142,38 @@ internal_get_eventdev_params(struct eventmode_conf *em_conf,
 }
 
 static inline bool
+internal_dev_has_rx_internal_port(uint8_t eventdev_id)
+{
+	int j;
+	bool flag = true;
+
+	RTE_ETH_FOREACH_DEV(j) {
+		uint32_t caps = 0;
+
+		rte_event_eth_rx_adapter_caps_get(eventdev_id, j, &caps);
+		if (!(caps & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT))
+			flag = false;
+	}
+	return flag;
+}
+
+static inline bool
+internal_dev_has_tx_internal_port(uint8_t eventdev_id)
+{
+	int j;
+	bool flag = true;
+
+	RTE_ETH_FOREACH_DEV(j) {
+		uint32_t caps = 0;
+
+		rte_event_eth_tx_adapter_caps_get(eventdev_id, j, &caps);
+		if (!(caps & RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT))
+			flag = false;
+	}
+	return flag;
+}
+
+static inline bool
 internal_dev_has_burst_mode(uint8_t dev_id)
 {
 	struct rte_event_dev_info dev_info;
@@ -303,6 +335,8 @@ rte_eventmode_helper_set_default_conf_eventdev(struct eventmode_conf *em_conf)
 {
 	int i, ret;
 	int nb_eventdev;
+	int nb_eth_dev;
+	int lcore_count;
 	struct eventdev_params *eventdev_config;
 	struct rte_event_dev_info dev_info;
 
@@ -314,6 +348,17 @@ rte_eventmode_helper_set_default_conf_eventdev(struct eventmode_conf *em_conf)
 		return -1;
 	}
 
+	/* Get the number of eth devs */
+	nb_eth_dev = rte_eth_dev_count_avail();
+
+	if (nb_eth_dev == 0) {
+		RTE_EM_HLPR_LOG_ERR("No eth devices detected");
+		return -1;
+	}
+
+	/* Get the number of lcores */
+	lcore_count = rte_lcore_count();
+
 	for (i = 0; i < nb_eventdev; i++) {
 
 		/* Get the event dev conf */
@@ -340,13 +385,19 @@ rte_eventmode_helper_set_default_conf_eventdev(struct eventmode_conf *em_conf)
 		eventdev_config->nb_eventqueue = dev_info.max_event_queues;
 		eventdev_config->nb_eventport = dev_info.max_event_ports;
 		eventdev_config->ev_queue_mode =
-				RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+				RTE_EVENT_QUEUE_CFG_ALL_TYPES;
 
-		/* One port is required for eth Rx adapter */
-		eventdev_config->nb_eventport -= 1;
+		/* Check if there are more queues than required */
+		if (eventdev_config->nb_eventqueue > nb_eth_dev + 1) {
+			/* One queue is reserved for Tx */
+			eventdev_config->nb_eventqueue = nb_eth_dev + 1;
+		}
 
-		/* One port is reserved for eth Tx adapter */
-		eventdev_config->nb_eventport -= 1;
+		/* Check if there are more ports than required */
+		if (eventdev_config->nb_eventport > lcore_count) {
+			/* One port per lcore is enough */
+			eventdev_config->nb_eventport = lcore_count;
+		}
 
 		/* Update the number of eventdevs */
 		em_conf->nb_eventdev++;
@@ -355,6 +406,37 @@ rte_eventmode_helper_set_default_conf_eventdev(struct eventmode_conf *em_conf)
 	return 0;
 }
 
+static void
+rte_eventmode_helper_do_capability_check(struct eventmode_conf *em_conf)
+{
+	struct eventdev_params *eventdev_config;
+	uint32_t eventdev_id;
+	int all_internal_ports = 1;
+	int i;
+
+	for (i = 0; i < em_conf->nb_eventdev; i++) {
+
+		/* Get the event dev conf */
+		eventdev_config = &(em_conf->eventdev_config[i]);
+		eventdev_id = eventdev_config->eventdev_id;
+
+		/* Check if event device has internal port for Rx & Tx */
+		if (internal_dev_has_rx_internal_port(eventdev_id) &&
+		    internal_dev_has_tx_internal_port(eventdev_id)) {
+			eventdev_config->all_internal_ports = 1;
+		} else {
+			all_internal_ports = 0;
+		}
+	}
+
+	/*
+	 * If Rx & Tx internal ports are supported by all event devices then
+	 * eth cores won't be required. Override the eth core mask requested.
+	 */
+	if (all_internal_ports)
+		em_conf->eth_core_mask = 0;
+}
+
 static int
 rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf *em_conf)
 {
@@ -363,9 +445,12 @@ rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf *em_conf)
 	int adapter_id;
 	int eventdev_id;
 	int conn_id;
+	int nb_eventqueue;
 	struct rx_adapter_conf *adapter;
 	struct adapter_connection_info *conn;
 	struct eventdev_params *eventdev_config;
+	bool rx_internal_port = true;
+	uint32_t caps = 0;
 
 	/* Create one adapter with all eth queues mapped to event queues 1:1 */
 
@@ -390,7 +475,14 @@ rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf *em_conf)
 	/* Set adapter conf */
 	adapter->eventdev_id = eventdev_id;
 	adapter->adapter_id = adapter_id;
-	adapter->rx_core_id = internal_get_next_eth_core(em_conf);
+
+	/*
+	 * If event device does not have internal ports for passing
+	 * packets then one queue is reserved for Tx path
+	 */
+	nb_eventqueue = eventdev_config->all_internal_ports ?
+			eventdev_config->nb_eventqueue :
+			eventdev_config->nb_eventqueue - 1;
 
 	/*
 	 * All queues of one eth device (port) will be mapped to one event
@@ -399,12 +491,11 @@ rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf *em_conf)
 	 */
 
 	/* Make sure there is enough event queues for 1:1 mapping */
-	if (nb_eth_dev > eventdev_config->nb_eventqueue) {
+	if (nb_eth_dev > nb_eventqueue) {
 		RTE_EM_HLPR_LOG_ERR(
 			"Not enough event queues for 1:1 mapping "
 			"[eth devs: %d, event queues: %d]\n",
-			nb_eth_dev,
-			eventdev_config->nb_eventqueue);
+			nb_eth_dev, nb_eventqueue);
 		return -1;
 	}
 
@@ -427,11 +518,24 @@ rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf *em_conf)
 		/* Add all eth queues of one eth port to one event queue */
 		conn->ethdev_rx_qid = -1;
 
+		/* Get Rx adapter capabilities */
+		rte_event_eth_rx_adapter_caps_get(eventdev_id, i, &caps);
+		if (!(caps & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT))
+			rx_internal_port = false;
+
 		/* Update no of connections */
 		adapter->nb_connections++;
 
 	}
 
+	if (rx_internal_port) {
+		/* Rx core is not required */
+		adapter->rx_core_id = -1;
+	} else {
+		/* Rx core is required */
+		adapter->rx_core_id = internal_get_next_eth_core(em_conf);
+	}
+
 	/* We have setup one adapter */
 	em_conf->nb_rx_adapter = 1;
 
@@ -449,6 +553,8 @@ rte_eventmode_helper_set_default_conf_tx_adapter(struct eventmode_conf *em_conf)
 	struct eventdev_params *eventdev_config;
 	struct tx_adapter_conf *tx_adapter;
 	struct tx_adapter_connection_info *conn;
+	bool tx_internal_port = true;
+	uint32_t caps = 0;
 
 	/*
 	 * Create one Tx adapter with all eth queues mapped to event queues
@@ -477,22 +583,6 @@ rte_eventmode_helper_set_default_conf_tx_adapter(struct eventmode_conf *em_conf)
 	tx_adapter->eventdev_id = eventdev_id;
 	tx_adapter->adapter_id = adapter_id;
 
-	/* TODO: Tx core is required only when internal port is not present */
-
-	tx_adapter->tx_core_id = internal_get_next_eth_core(em_conf);
-
-	/*
-	 * Application would need to use one event queue per adapter for
-	 * submitting packets for Tx. Reserving the last queue available
-	 * and decrementing the total available event queues for this
-	 */
-
-	/* Queue numbers start at 0 */
-	tx_adapter->tx_ev_queue = eventdev_config->nb_eventqueue - 1;
-
-	/* Update the number of event queues available in eventdev */
-	eventdev_config->nb_eventqueue--;
-
 	/*
 	 * All Tx queues of the eth device (port) will be mapped to the event
 	 * device.
@@ -523,10 +613,30 @@ rte_eventmode_helper_set_default_conf_tx_adapter(struct eventmode_conf *em_conf)
 		/* Add all eth tx queues to adapter */
 		conn->ethdev_tx_qid = -1;
 
+		/* Get Rx adapter capabilities */
+		rte_event_eth_tx_adapter_caps_get(eventdev_id, i, &caps);
+		if (!(caps & RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT))
+			tx_internal_port = false;
+
 		/* Update no of connections */
 		tx_adapter->nb_connections++;
 	}
 
+	if (tx_internal_port) {
+		/* Tx core is not required */
+		tx_adapter->tx_core_id = -1;
+	} else {
+		/* Tx core is required */
+		tx_adapter->tx_core_id = internal_get_next_eth_core(em_conf);
+
+		/*
+		 * Application would need to use one event queue per adapter for
+		 * submitting packets for Tx. Reserving the last queue available
+		 */
+		/* Queue numbers start at 0 */
+		tx_adapter->tx_ev_queue = eventdev_config->nb_eventqueue - 1;
+	}
+
 	/* We have setup one adapter */
 	em_conf->nb_tx_adapter = 1;
 	return 0;
@@ -620,6 +730,9 @@ rte_eventmode_helper_validate_conf(struct eventmode_conf *em_conf)
 			return ret;
 	}
 
+	/* Perform capability check for the selected event devices*/
+	rte_eventmode_helper_do_capability_check(em_conf);
+
 	/*
 	 * See if rx adapters are specified. Else generate a default conf
 	 * with one rx adapter and all eth queue - event queue mapped.
@@ -681,10 +794,6 @@ rte_eventmode_helper_initialize_eventdev(struct eventmode_conf *em_conf)
 		/* Get the number of queues */
 		nb_eventqueue = eventdev_config->nb_eventqueue;
 
-		/* One queue is reserved for the final stage (doing eth tx) */
-		/* TODO handles only one Tx adapter. Fix this */
-		nb_eventqueue += 1;
-
 		/* Reset the default conf */
 		memset(&evdev_default_conf, 0,
 			sizeof(struct rte_event_dev_info));
@@ -730,14 +839,15 @@ rte_eventmode_helper_initialize_eventdev(struct eventmode_conf *em_conf)
 			/* Per event dev queues can be ATQ or SINGLE LINK */
 			eventq_conf.event_queue_cfg =
 					eventdev_config->ev_queue_mode;
-
 			/*
 			 * All queues need to be set with sched_type as
-			 * schedule type for the application stage. One queue
-			 * would be reserved for the final eth tx stage. This
-			 * will be an atomic queue.
+			 * schedule type for the application stage. One
+			 * queue would be reserved for the final eth tx
+			 * stage if event device does not have internal
+			 * ports. This will be an atomic queue.
 			 */
-			if (j == nb_eventqueue-1) {
+			if (!eventdev_config->all_internal_ports &&
+			    j == nb_eventqueue-1) {
 				eventq_conf.schedule_type =
 					RTE_SCHED_TYPE_ATOMIC;
 			} else {
@@ -867,8 +977,10 @@ rx_adapter_configure(struct eventmode_conf *em_conf,
 
 	/* Setup various connections in the adapter */
 
+#ifdef UNSELECT
 	queue_conf.rx_queue_flags =
 			RTE_EVENT_ETH_RX_ADAPTER_QUEUE_FLOW_ID_VALID;
+#endif /* UNSELECT */
 
 	for (j = 0; j < adapter->nb_connections; j++) {
 		/* Get connection */
@@ -877,9 +989,12 @@ rx_adapter_configure(struct eventmode_conf *em_conf,
 		/* Setup queue conf */
 		queue_conf.ev.queue_id = conn->eventq_id;
 		queue_conf.ev.sched_type = em_conf->ext_params.sched_type;
+		queue_conf.ev.event_type = RTE_EVENT_TYPE_ETHDEV;
 
+#ifdef UNSELECT
 		/* Set flow ID as ethdev ID */
 		queue_conf.ev.flow_id = conn->ethdev_id;
+#endif /* UNSELECT */
 
 		/* Add queue to the adapter */
 		ret = rte_event_eth_rx_adapter_queue_add(
@@ -945,8 +1060,8 @@ tx_adapter_configure(struct eventmode_conf *em_conf,
 {
 	int ret, j;
 	uint8_t tx_port_id = 0;
-	uint8_t eventdev_id;
 	uint32_t service_id;
+	uint8_t eventdev_id;
 	struct rte_event_port_conf port_conf = {0};
 	struct rte_event_dev_info evdev_default_conf = {0};
 	struct tx_adapter_connection_info *conn;
@@ -1004,6 +1119,18 @@ tx_adapter_configure(struct eventmode_conf *em_conf,
 		}
 	}
 
+	/*
+	 * Check if Tx core is assigned. If Tx core is not assigned, then
+	 * the adapter would be having internal port for submitting packets
+	 * for Tx and so Tx event queue & port setup is not required
+	 */
+	if (adapter->tx_core_id == (uint32_t) (-1)) {
+		/* Internal port is present */
+		goto skip_tx_queue_port_setup;
+	}
+
+	/* Setup Tx queue & port */
+
 	/* Get event port used by the adapter */
 	ret = rte_event_eth_tx_adapter_event_port_get(
 			adapter->adapter_id,
@@ -1014,11 +1141,6 @@ tx_adapter_configure(struct eventmode_conf *em_conf,
 	}
 
 	/*
-	 * TODO: event queue for Tx adapter is required only if the
-	 * INTERNAL PORT is not present.
-	 */
-
-	/*
 	 * Tx event queue would be reserved for Tx adapter. Need to unlink
 	 * this queue from all other ports
 	 *
@@ -1028,6 +1150,7 @@ tx_adapter_configure(struct eventmode_conf *em_conf,
 				      &(adapter->tx_ev_queue), 1);
 	}
 
+	/* Link Tx event queue to Tx port */
 	ret = rte_event_port_link(
 			eventdev_id,
 			tx_port_id,
@@ -1055,6 +1178,8 @@ tx_adapter_configure(struct eventmode_conf *em_conf,
 	 */
 	rte_service_set_runstate_mapped_check(service_id, 0);
 
+skip_tx_queue_port_setup:
+
 	/* Start adapter */
 	ret = rte_event_eth_tx_adapter_start(adapter->adapter_id);
 	if (ret) {
@@ -1437,6 +1562,14 @@ rte_eventmode_helper_find_worker(uint32_t lcore_id,
 	else
 		curr_conf.cap.burst = RTE_EVENTMODE_HELPER_RX_TYPE_NON_BURST;
 
+	/* Check for Tx internal port */
+	if (internal_dev_has_tx_internal_port(eventdev_id))
+		curr_conf.cap.tx_internal_port =
+				RTE_EVENTMODE_HELPER_TX_TYPE_INTERNAL_PORT;
+	else
+		curr_conf.cap.tx_internal_port =
+				RTE_EVENTMODE_HELPER_TX_TYPE_NO_INTERNAL_PORT;
+
 	/* Now parse the passed list and see if we have matching capabilities */
 
 	/* Initialize the pointer used to traverse the list */
diff --git a/lib/librte_eventdev/rte_eventmode_helper.h b/lib/librte_eventdev/rte_eventmode_helper.h
index 2212622..f705eec 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.h
+++ b/lib/librte_eventdev/rte_eventmode_helper.h
@@ -36,6 +36,8 @@ enum rte_eventmode_helper_rx_types {
 /* Event mode packet tx types */
 enum rte_eventmode_helper_tx_types {
 	RTE_EVETNMODE_HELPER_TX_TYPE_INVALID = 0,
+	RTE_EVENTMODE_HELPER_TX_TYPE_INTERNAL_PORT,
+	RTE_EVENTMODE_HELPER_TX_TYPE_NO_INTERNAL_PORT,
 	RTE_EVENTMODE_HELPER_TX_TYPE_MAX = 16
 };
 
@@ -85,6 +87,8 @@ struct rte_eventmode_helper_app_worker_params {
 		struct {
 			uint64_t burst : 4;
 			/**< Specify status of rx type burst */
+			uint64_t tx_internal_port : 4;
+			/**< Specify whether tx internal port is available */
 		};
 		uint64_t u64;
 	} cap;
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 1daca22..44796e3 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -66,6 +66,7 @@ struct eventdev_params {
 	uint8_t nb_eventqueue;
 	uint8_t nb_eventport;
 	uint8_t ev_queue_mode;
+	uint8_t all_internal_ports;
 };
 
 /* Rx adapter connection info */
diff --git a/lib/librte_eventdev/rte_eventmode_helper_prints.c b/lib/librte_eventdev/rte_eventmode_helper_prints.c
index 387302a..0a34f43 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_prints.c
+++ b/lib/librte_eventdev/rte_eventmode_helper_prints.c
@@ -64,13 +64,22 @@ rte_eventmode_display_rx_adapter_conf(struct eventmode_conf *em_conf)
 
 	for (i = 0; i < nb_rx_adapter; i++) {
 		adapter = &(em_conf->rx_adapter[i]);
-		RTE_EM_HLPR_LOG_INFO(
-			"\tRx adaper ID: %-2d\tConnections: %-2d\tEvent dev ID: %-2d"
-			"\tRx core: %-2d",
+		sprintf(print_buf,
+			"\tRx adaper ID: %-2d\tConnections: %-2d\tEvent dev ID: %-2d",
 			adapter->adapter_id,
 			adapter->nb_connections,
-			adapter->eventdev_id,
-			adapter->rx_core_id);
+			adapter->eventdev_id);
+		if (adapter->rx_core_id == (uint32_t)-1)
+			sprintf(print_buf + strlen(print_buf),
+				"\tRx core: %-2s", "[INTERNAL PORT]");
+		else if (adapter->rx_core_id == RTE_MAX_LCORE)
+			sprintf(print_buf + strlen(print_buf),
+				"\tRx core: %-2s", "[NONE]");
+		else
+			sprintf(print_buf + strlen(print_buf),
+				"\tRx core: %-2d", adapter->rx_core_id);
+
+		RTE_EM_HLPR_LOG_INFO("%s", print_buf);
 
 		for (j = 0; j < adapter->nb_connections; j++) {
 			conn = &(adapter->conn[j]);
-- 
2.7.4

[dpdk-dev] [PATCH 35/39] eventdev: display Tx adapter conf

[Anoob Joseph]

Adding routines to display Tx adapter configuration.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper_prints.c | 46 ++++++++++++++++++++++-
 1 file changed, 45 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eventdev/rte_eventmode_helper_prints.c b/lib/librte_eventdev/rte_eventmode_helper_prints.c
index 0a34f43..8416bbb 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_prints.c
+++ b/lib/librte_eventdev/rte_eventmode_helper_prints.c
@@ -106,7 +106,51 @@ rte_eventmode_display_rx_adapter_conf(struct eventmode_conf *em_conf)
 static void
 rte_eventmode_display_tx_adapter_conf(struct eventmode_conf *em_conf)
 {
-	RTE_SET_USED(em_conf);
+	int i, j;
+	int nb_tx_adapter = em_conf->nb_tx_adapter;
+	struct tx_adapter_conf *adapter;
+	struct tx_adapter_connection_info *conn;
+	char print_buf[256] = { 0 };
+
+	RTE_EM_HLPR_LOG_INFO("Tx adapters configured: %d", nb_tx_adapter);
+
+	for (i = 0; i < nb_tx_adapter; i++) {
+		adapter = &(em_conf->tx_adapter[i]);
+		sprintf(print_buf,
+			"\tTx adapter ID: %-2d\tConnections: %-2d\tEvent dev ID: %-2d",
+			adapter->adapter_id,
+			adapter->nb_connections,
+			adapter->eventdev_id);
+		if (adapter->tx_core_id == (uint32_t)-1)
+			sprintf(print_buf + strlen(print_buf),
+				"\tTx core: %-2s", "[INTERNAL PORT]");
+		else if (adapter->tx_core_id == RTE_MAX_LCORE)
+			sprintf(print_buf + strlen(print_buf),
+				"\tTx core: %-2s", "[NONE]");
+		else
+			sprintf(print_buf + strlen(print_buf),
+				"\tTx core: %-2d,\tInput event queue: %-2d",
+				adapter->tx_core_id, adapter->tx_ev_queue);
+
+		RTE_EM_HLPR_LOG_INFO("%s", print_buf);
+
+		for (j = 0; j < adapter->nb_connections; j++) {
+			conn = &(adapter->conn[j]);
+
+			sprintf(print_buf,
+				"\t\tEthdev ID: %-2d", conn->ethdev_id);
+
+			if (conn->ethdev_tx_qid == -1)
+				sprintf(print_buf + strlen(print_buf),
+					"\tEth tx queue: %-2s", "ALL");
+			else
+				sprintf(print_buf + strlen(print_buf),
+					"\tEth tx queue: %-2d",
+					conn->ethdev_tx_qid);
+			RTE_EM_HLPR_LOG_INFO("%s", print_buf);
+		}
+	}
+	RTE_EM_HLPR_LOG_INFO("");
 }
 
 static void
-- 
2.7.4

[dpdk-dev] [PATCH 36/39] examples/l2fwd-event: add eventmode for l2fwd

[Anoob Joseph]

Adding eventmode support in l2fwd. This uses rte_eventmode_helper APIs
to setup and use the eventmode capabilties.

Adding non-burst no Tx internal-port eventmode worker.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/Makefile       |   1 +
 examples/l2fwd-event/l2fwd_worker.c | 314 ++++++++++++++++++++++++++++++++++--
 examples/l2fwd-event/main.c         |  64 ++++++--
 examples/l2fwd-event/meson.build    |   2 +
 4 files changed, 360 insertions(+), 21 deletions(-)

diff --git a/examples/l2fwd-event/Makefile b/examples/l2fwd-event/Makefile
index d6bdb9e..dbb793f 100644
--- a/examples/l2fwd-event/Makefile
+++ b/examples/l2fwd-event/Makefile
@@ -49,6 +49,7 @@ RTE_TARGET ?= x86_64-native-linuxapp-gcc
 
 include $(RTE_SDK)/mk/rte.vars.mk
 
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 CFLAGS += -O3
 CFLAGS += $(WERROR_FLAGS)
 
diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index 1a7ee2b..92eae02 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -25,15 +25,38 @@
 #include <rte_branch_prediction.h>
 #include <rte_ether.h>
 #include <rte_ethdev.h>
+#include <rte_eventdev.h>
+#include <rte_eventmode_helper.h>
+#include <rte_malloc.h>
 #include <rte_mbuf.h>
 
 #include "l2fwd_common.h"
 #include "l2fwd_worker.h"
 
+/* Reset eth stats */
+static void
+reset_eth_stats(int is_master_core)
+{
+	int portid;
+
+	/* Only master core need to do this */
+	if (!is_master_core)
+		return;
+
+	/* Reset stats */
+	for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) {
+		/* skip disabled ports */
+		if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
+			continue;
+		rte_eth_stats_reset(portid);
+	}
+}
+
 /* Print out statistics on packets dropped */
 static void
 print_stats(void)
 {
+	struct rte_eth_stats stats;
 	uint64_t total_packets_dropped, total_packets_tx, total_packets_rx;
 	unsigned int portid;
 
@@ -53,19 +76,21 @@ print_stats(void)
 		/* skip disabled ports */
 		if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
 			continue;
+		rte_eth_stats_get(portid, &stats);
 		printf("\nStatistics for port %u ------------------------------"
 			   "\nPackets sent: %24"PRIu64
 			   "\nPackets received: %20"PRIu64
 			   "\nPackets dropped: %21"PRIu64,
 			   portid,
-			   port_statistics[portid].tx,
-			   port_statistics[portid].rx,
-			   port_statistics[portid].dropped);
+			   stats.opackets,
+			   stats.ipackets,
+			   stats.oerrors);
 
-		total_packets_dropped += port_statistics[portid].dropped;
-		total_packets_tx += port_statistics[portid].tx;
-		total_packets_rx += port_statistics[portid].rx;
+		total_packets_dropped += stats.oerrors;
+		total_packets_tx += stats.opackets;
+		total_packets_rx += stats.ipackets;
 	}
+
 	printf("\nAggregate statistics ==============================="
 		   "\nTotal packets sent: %18"PRIu64
 		   "\nTotal packets received: %14"PRIu64
@@ -138,6 +163,16 @@ l2fwd_periodic_drain_stats_monitor(struct lcore_queue_conf *qconf,
 	}
 }
 
+static inline void
+l2fwd_drain_loop(struct lcore_queue_conf *qconf, struct tsc_tracker *t,
+		int is_master_core)
+{
+	while (!force_quit) {
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, t, is_master_core);
+	}
+}
+
 static void
 l2fwd_mac_updating(struct rte_mbuf *m, unsigned int dest_portid)
 {
@@ -180,9 +215,40 @@ l2fwd_simple_forward(struct rte_mbuf *m, unsigned int portid)
 	l2fwd_send_pkt(m, dst_port);
 }
 
-/* main processing loop */
+static inline void
+l2fwd_event_pre_forward(struct rte_event *ev, unsigned int portid)
+{
+	unsigned int dst_port;
+	struct rte_mbuf *m;
+
+	/* Get the mbuf */
+	m = ev->mbuf;
+
+	/* Get the destination port from the tables */
+	dst_port = l2fwd_dst_ports[portid];
+
+	/* Save the destination port in the mbuf */
+	m->port = dst_port;
+
+	/* Use tx queue 0 */
+	rte_event_eth_tx_adapter_txq_set(m, 0);
+
+	/* Perform work */
+	if (mac_updating)
+		l2fwd_mac_updating(m, dst_port);
+}
+
+static inline void
+l2fwd_event_switch_to_tx_queue(struct rte_event *ev, uint8_t tx_queue_id)
+{
+	ev->event_type = RTE_EVENT_TYPE_CPU;
+	ev->op = RTE_EVENT_OP_FORWARD;
+	ev->queue_id = tx_queue_id;
+}
+
+/* poll mode processing loop */
 static void
-l2fwd_main_loop(void)
+l2fwd_poll_mode_worker(void)
 {
 	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
 	struct rte_mbuf *m;
@@ -242,9 +308,237 @@ l2fwd_main_loop(void)
 	}
 }
 
+/*
+ * Event mode exposes various operating modes depending on the
+ * capabilities of the event device and the operating mode
+ * selected.
+ */
+
+/* Workers registered */
+#define L2FWD_EVENTMODE_WORKERS		1
+
+/*
+ * Event mode worker
+ * Operating mode : non-burst no internal port (regular tx worker)
+ */
+static void
+l2fwd_eventmode_non_burst_no_internal_port(void *args)
+{
+	struct rte_event ev;
+	struct rte_mbuf *pkt;
+	struct rte_eventmode_helper_conf *mode_conf;
+	struct rte_eventmode_helper_event_link_info *links = NULL;
+	unsigned int lcore_nb_link = 0;
+	uint32_t lcore_id;
+	unsigned int i, nb_rx = 0;
+	unsigned int portid;
+	struct lcore_queue_conf *qconf;
+	int is_master_core;
+	struct tsc_tracker tsc = {0};
+	uint8_t tx_queue;
+
+	/* Get core ID */
+	lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, L2FWD,
+		"Launching event mode non-burst worker no internal port "
+		"(regular tx worker) on lcore %d\n", lcore_id);
+
+	/* Set the flag if master core */
+	is_master_core = (lcore_id == rte_get_master_lcore()) ? 1 : 0;
+
+	/* Get qconf for this core */
+	qconf = &lcore_queue_conf[lcore_id];
+
+	/* Set drain tsc */
+	tsc.drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /
+			US_PER_S * BURST_TX_DRAIN_US;
+
+	/* Mode conf will be passed as args */
+	mode_conf = (struct rte_eventmode_helper_conf *)args;
+
+	/* Get the links configured for this lcore */
+	lcore_nb_link = rte_eventmode_helper_get_event_lcore_links(lcore_id,
+			mode_conf, &links);
+
+	/* Check if we have links registered for this lcore */
+	if (lcore_nb_link == 0) {
+		/* No links registered. The core could do periodic drains */
+		l2fwd_drain_loop(qconf, &tsc, is_master_core);
+		goto clean_and_exit;
+	}
+
+	/* We have valid links */
+
+	/* Reset stats before proceeding */
+	reset_eth_stats(is_master_core);
+
+	/*
+	 * There is no internal port between ethdev and eventdev. So the worker
+	 * thread needs to submit event to a designated tx queue. Internally
+	 * eth core would receive events from multiple worker threads and send
+	 * out packets on wire.
+	 */
+	tx_queue = rte_eventmode_helper_get_tx_queue(mode_conf,
+						     links[0].eventdev_id);
+	/* See if it's single link */
+	if (lcore_nb_link == 1)
+		goto single_link_loop;
+	else
+		goto multi_link_loop;
+
+single_link_loop:
+
+	RTE_LOG(INFO, L2FWD, " -- lcoreid=%u event_port_id=%u\n", lcore_id,
+			links[0].event_portid);
+
+	while (!force_quit) {
+
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, &tsc, is_master_core);
+
+		/* Read packet from event queues */
+		nb_rx = rte_event_dequeue_burst(links[0].eventdev_id,
+				links[0].event_portid,
+				&ev,	/* events */
+				1,	/* nb_events */
+				0	/* timeout_ticks */);
+
+		if (nb_rx == 0)
+			continue;
+
+		portid = ev.queue_id;
+		port_statistics[portid].rx++;
+		pkt = ev.mbuf;
+
+		rte_prefetch0(rte_pktmbuf_mtod(pkt, void *));
+
+		/* Process packet */
+		l2fwd_event_pre_forward(&ev, portid);
+
+		/*
+		 * Internal port is not available, the packet needs
+		 * to be enqueued to the designated event queue.
+		 */
+
+		/* Prepare event for submission to tx event queue */
+		l2fwd_event_switch_to_tx_queue(&ev, tx_queue);
+
+		/* Submit the updated event for tx stage */
+		rte_event_enqueue_burst(links[0].eventdev_id,
+				links[0].event_portid,
+				&ev,	/* events */
+				1	/* nb_events */);
+	}
+	goto clean_and_exit;
+
+multi_link_loop:
+
+	for (i = 0; i < lcore_nb_link; i++) {
+		RTE_LOG(INFO, L2FWD, " -- lcoreid=%u event_port_id=%u\n",
+				lcore_id, links[i].event_portid);
+	}
+
+	while (!force_quit) {
+
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, &tsc, is_master_core);
+
+		for (i = 0; i < lcore_nb_link; i++) {
+			/* Read packet from event queues */
+			nb_rx = rte_event_dequeue_burst(links[i].eventdev_id,
+					links[i].event_portid,
+					&ev,	/* events */
+					1,	/* nb_events */
+					0	/* timeout_ticks */);
+
+			if (nb_rx == 0)
+				continue;
+
+			portid = ev.queue_id;
+			port_statistics[portid].rx++;
+			pkt = ev.mbuf;
+
+			rte_prefetch0(rte_pktmbuf_mtod(pkt, void *));
+
+			/* Process packet */
+			l2fwd_event_pre_forward(&ev, portid);
+
+			/*
+			 * Internal port is not available, the packet needs
+			 * to be enqueued to the designated event queue.
+			 */
+
+			/* Prepare event for submission to tx event queue */
+			l2fwd_event_switch_to_tx_queue(&ev, tx_queue);
+
+			/* Submit the updated event for tx stage */
+			rte_event_enqueue_burst(links[i].eventdev_id,
+					links[i].event_portid,
+					&ev,	/* events */
+					1	/* nb_events */);
+		}
+	}
+	goto clean_and_exit;
+
+clean_and_exit:
+	if (links != NULL)
+		rte_free(links);
+}
+
+static uint8_t
+l2fwd_eventmode_populate_wrkr_params(
+		struct rte_eventmode_helper_app_worker_params *wrkrs)
+{
+	uint8_t nb_wrkr_param = 0;
+	struct rte_eventmode_helper_app_worker_params *wrkr;
+
+	/* Save workers */
+	wrkr = wrkrs;
+
+	/* Non-burst no internal port (regular tx worker) */
+	wrkr->cap.burst = RTE_EVENTMODE_HELPER_RX_TYPE_NON_BURST;
+	wrkr->cap.tx_internal_port =
+			RTE_EVENTMODE_HELPER_TX_TYPE_NO_INTERNAL_PORT;
+	wrkr->worker_thread = l2fwd_eventmode_non_burst_no_internal_port;
+
+	nb_wrkr_param++;
+	return nb_wrkr_param;
+}
+
+static void
+l2fwd_eventmode_worker(struct rte_eventmode_helper_conf *mode_conf)
+{
+	struct rte_eventmode_helper_app_worker_params
+			l2fwd_wrkr[L2FWD_EVENTMODE_WORKERS] = {
+					{{{0} }, NULL } };
+	uint8_t nb_wrkr_param;
+
+	/* Populate l2fwd_wrkr params */
+	nb_wrkr_param = l2fwd_eventmode_populate_wrkr_params(l2fwd_wrkr);
+
+	/*
+	 * The helper function will launch the correct worker after checking the
+	 * event device's capabilities.
+	 */
+	rte_eventmode_helper_launch_worker(mode_conf, l2fwd_wrkr,
+			nb_wrkr_param);
+}
+
 int
-l2fwd_launch_one_lcore(__attribute__((unused)) void *dummy)
+l2fwd_launch_one_lcore(void *args)
 {
-	l2fwd_main_loop();
+	struct rte_eventmode_helper_conf *mode_conf;
+
+	mode_conf = (struct rte_eventmode_helper_conf *)args;
+
+	if (mode_conf->mode == RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL) {
+		/* App is initialized to run in poll mode */
+		l2fwd_poll_mode_worker();
+	} else if (mode_conf->mode ==
+			RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT) {
+		/* App is initialized to run in event mode */
+		l2fwd_eventmode_worker(mode_conf);
+	}
 	return 0;
 }
diff --git a/examples/l2fwd-event/main.c b/examples/l2fwd-event/main.c
index 4e83b41..62228e6 100644
--- a/examples/l2fwd-event/main.c
+++ b/examples/l2fwd-event/main.c
@@ -39,6 +39,7 @@
 #include <rte_ethdev.h>
 #include <rte_mempool.h>
 #include <rte_mbuf.h>
+#include <rte_eventmode_helper.h>
 
 #include "l2fwd_common.h"
 #include "l2fwd_worker.h"
@@ -68,6 +69,8 @@ l2fwd_usage(const char *prgname)
 		" [-q NQ]",
 		prgname);
 
+	rte_eventmode_helper_print_options_list();
+
 	fprintf(stderr, "\n\n");
 
 	fprintf(stderr,
@@ -78,7 +81,9 @@ l2fwd_usage(const char *prgname)
 		"      When enabled:\n"
 		"       - The source MAC address is replaced by the TX port MAC address\n"
 		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
-		"\n");
+		"");
+
+	rte_eventmode_helper_print_options_description();
 }
 
 static int
@@ -158,12 +163,14 @@ static const struct option lgopts[] = {
 
 /* Parse the argument given in the command line of the application */
 static int
-l2fwd_parse_args(int argc, char **argv)
+l2fwd_parse_args(int argc, char **argv,
+		struct rte_eventmode_helper_conf **mode_conf)
 {
-	int opt, ret, timer_secs;
+	int opt, timer_secs;
 	char **argvopt;
 	int option_index;
 	char *prgname = argv[0];
+	int options_parsed = 0;
 
 	argvopt = argv;
 
@@ -212,12 +219,31 @@ l2fwd_parse_args(int argc, char **argv)
 		}
 	}
 
-	if (optind >= 0)
-		argv[optind-1] = prgname;
+	/* Update argc & argv to move to event mode options */
+	options_parsed = optind-1;
+	argc -= options_parsed;
+	argv += options_parsed;
 
-	ret = optind-1;
-	optind = 1; /* reset getopt lib */
-	return ret;
+	/* Reset getopt lib */
+	optind = 1;
+
+	/* Check for event mode parameters and get the conf prepared*/
+	*mode_conf = rte_eventmode_helper_parse_args(argc, argv);
+	if (*mode_conf == NULL) {
+		l2fwd_usage(prgname);
+		return -1;
+	}
+
+	/* Add the number of options parsed */
+	options_parsed += optind-1;
+
+	if (options_parsed >= 0)
+		argv[options_parsed] = prgname;
+
+	/* Reset getopt lib */
+	optind = 1;
+
+	return options_parsed;
 }
 
 /* Check the link status of all ports in up to 9s, and print them finally */
@@ -315,6 +341,7 @@ main(int argc, char **argv)
 	unsigned int nb_ports_in_mask = 0;
 	unsigned int nb_lcores = 0;
 	unsigned int nb_mbufs;
+	struct rte_eventmode_helper_conf *mode_conf = NULL;
 
 	/* Set default values for global vars */
 	l2fwd_init_global_vars();
@@ -329,8 +356,12 @@ main(int argc, char **argv)
 	signal(SIGINT, signal_handler);
 	signal(SIGTERM, signal_handler);
 
-	/* parse application arguments (after the EAL ones) */
-	ret = l2fwd_parse_args(argc, argv);
+	/*
+	 * Parse application arguments (after the EAL ones). This would parse
+	 * the event mode options too, and would set the conf pointer
+	 * accordingly.
+	 */
+	ret = l2fwd_parse_args(argc, argv, &mode_conf);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE, "Invalid L2FWD arguments\n");
 
@@ -519,9 +550,20 @@ main(int argc, char **argv)
 
 	check_all_ports_link_status(l2fwd_enabled_port_mask);
 
+	/*
+	 * Set the enabled port mask in helper conf to be used by helper
+	 * sub-system. This would be used while intializing devices using
+	 * helper sub-system.
+	 */
+	mode_conf->eth_portmask = l2fwd_enabled_port_mask;
+
+	/* Initialize eventmode components */
+	rte_eventmode_helper_initialize_devs(mode_conf);
+
 	ret = 0;
 	/* launch per-lcore init on every lcore */
-	rte_eal_mp_remote_launch(l2fwd_launch_one_lcore, NULL, CALL_MASTER);
+	rte_eal_mp_remote_launch(l2fwd_launch_one_lcore, (void *)mode_conf,
+			CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(lcore_id) {
 		if (rte_eal_wait_lcore(lcore_id) < 0) {
 			ret = -1;
diff --git a/examples/l2fwd-event/meson.build b/examples/l2fwd-event/meson.build
index 1d2df49..10e6e66 100644
--- a/examples/l2fwd-event/meson.build
+++ b/examples/l2fwd-event/meson.build
@@ -6,6 +6,8 @@
 # To build this example as a standalone application with an already-installed
 # DPDK instance, use 'make'
 
+allow_experimental_apis = true
+deps += ['eventdev']
 sources = files(
 	'l2fwd_worker.c',
 	'main.c'
-- 
2.7.4

[dpdk-dev] [PATCH 37/39] examples/l2fwd-event: add eventmode worker

[Anoob Joseph]

Adding non-burst Tx internal-port eventmode worker.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_worker.c | 164 +++++++++++++++++++++++++++++++++++-
 1 file changed, 163 insertions(+), 1 deletion(-)

diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index 92eae02..0a413a4 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -315,7 +315,7 @@ l2fwd_poll_mode_worker(void)
  */
 
 /* Workers registered */
-#define L2FWD_EVENTMODE_WORKERS		1
+#define L2FWD_EVENTMODE_WORKERS		2
 
 /*
  * Event mode worker
@@ -486,6 +486,159 @@ l2fwd_eventmode_non_burst_no_internal_port(void *args)
 		rte_free(links);
 }
 
+/*
+ * Event mode worker
+ * Operating mode : non-burst tx internal port
+ */
+static void
+l2fwd_eventmode_non_burst_tx_internal_port(void *args)
+{
+	struct rte_event ev;
+	struct rte_mbuf *pkt;
+	struct rte_eventmode_helper_conf *mode_conf;
+	struct rte_eventmode_helper_event_link_info *links = NULL;
+	unsigned int lcore_nb_link = 0;
+	uint32_t lcore_id;
+	unsigned int i, nb_rx = 0;
+	unsigned int portid;
+	struct lcore_queue_conf *qconf;
+	int is_master_core;
+	struct tsc_tracker tsc = {0};
+
+	/* Get core ID */
+	lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, L2FWD,
+		"Launching event mode non-burst worker internal port "
+		"on lcore %d\n", lcore_id);
+
+	/* Set the flag if master core */
+	is_master_core = (lcore_id == rte_get_master_lcore()) ? 1 : 0;
+
+	/* Get qconf for this core */
+	qconf = &lcore_queue_conf[lcore_id];
+
+	/* Set drain tsc */
+	tsc.drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /
+			US_PER_S * BURST_TX_DRAIN_US;
+
+	/* Mode conf will be passed as args */
+	mode_conf = (struct rte_eventmode_helper_conf *)args;
+
+	/* Get the links configured for this lcore */
+	lcore_nb_link = rte_eventmode_helper_get_event_lcore_links(lcore_id,
+			mode_conf, &links);
+
+	/* Check if we have links registered for this lcore */
+	if (lcore_nb_link == 0) {
+		/* No links registered. The core could do periodic drains */
+		l2fwd_drain_loop(qconf, &tsc, is_master_core);
+		goto clean_and_exit;
+	}
+
+	/* We have valid links */
+
+	/* Reset stats before proceeding */
+	reset_eth_stats(is_master_core);
+
+	/* See if it's single link */
+	if (lcore_nb_link == 1)
+		goto single_link_loop;
+	else
+		goto multi_link_loop;
+
+single_link_loop:
+
+	RTE_LOG(INFO, L2FWD, " -- lcoreid=%u event_port_id=%u\n", lcore_id,
+		links[0].event_portid);
+
+	while (!force_quit) {
+
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, &tsc, is_master_core);
+
+		/* Read packet from event queues */
+		nb_rx = rte_event_dequeue_burst(links[0].eventdev_id,
+				links[0].event_portid,
+				&ev,     /* events */
+				1,       /* nb_events */
+				0        /* timeout_ticks */);
+
+		if (nb_rx == 0)
+			continue;
+
+		portid = ev.queue_id;
+		port_statistics[portid].rx++;
+		pkt = ev.mbuf;
+
+		rte_prefetch0(rte_pktmbuf_mtod(pkt, void *));
+
+		/* Process packet */
+		l2fwd_event_pre_forward(&ev, portid);
+
+		/*
+		 * Since tx internal port is available, events can be
+		 * directly enqueued to the adapter and it would be
+		 * internally submitted to the eth device.
+		 */
+		rte_event_eth_tx_adapter_enqueue(links[0].eventdev_id,
+				links[0].event_portid,
+				&ev,	/* events */
+				1	/* nb_events */);
+	}
+	goto clean_and_exit;
+
+multi_link_loop:
+
+	for (i = 0; i < lcore_nb_link; i++) {
+		RTE_LOG(INFO, L2FWD, " -- lcoreid=%u event_port_id=%u\n",
+			lcore_id, links[i].event_portid);
+	}
+
+	while (!force_quit) {
+
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, &tsc, is_master_core);
+
+		for (i = 0; i < lcore_nb_link; i++) {
+			/* Read packet from event queues */
+			nb_rx = rte_event_dequeue_burst(links[i].eventdev_id,
+					links[i].event_portid,
+					&ev,     /* events */
+					1,       /* nb_events */
+					0        /* timeout_ticks */);
+
+			if (nb_rx == 0)
+				continue;
+
+			portid = ev.queue_id;
+			port_statistics[portid].rx++;
+			pkt = ev.mbuf;
+
+			rte_prefetch0(rte_pktmbuf_mtod(pkt, void *));
+
+			/* Process packet */
+			l2fwd_event_pre_forward(&ev, portid);
+
+			/*
+			 * Since tx internal port is available, events can be
+			 * directly enqueued to the adapter and it would be
+			 * internally submitted to the eth device.
+			 */
+			rte_event_eth_tx_adapter_enqueue(links[i].eventdev_id,
+					links[i].event_portid,
+					&ev,	/* events */
+					1	/* nb_events */);
+
+		}
+	}
+	goto clean_and_exit;
+
+clean_and_exit:
+	if (links != NULL)
+		rte_free(links);
+}
+
 static uint8_t
 l2fwd_eventmode_populate_wrkr_params(
 		struct rte_eventmode_helper_app_worker_params *wrkrs)
@@ -503,6 +656,15 @@ l2fwd_eventmode_populate_wrkr_params(
 	wrkr->worker_thread = l2fwd_eventmode_non_burst_no_internal_port;
 
 	nb_wrkr_param++;
+	wrkr++;
+
+	/* Non-burst tx internal port */
+	wrkr->cap.burst = RTE_EVENTMODE_HELPER_RX_TYPE_NON_BURST;
+	wrkr->cap.tx_internal_port =
+			RTE_EVENTMODE_HELPER_TX_TYPE_INTERNAL_PORT;
+	wrkr->worker_thread = l2fwd_eventmode_non_burst_tx_internal_port;
+
+	nb_wrkr_param++;
 	return nb_wrkr_param;
 }
 
-- 
2.7.4

[dpdk-dev] [PATCH 38/39] examples/l2fwd-event: add eventmode worker

[Anoob Joseph]

Adding burst no Tx internal-port eventmode worker

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_worker.c | 219 +++++++++++++++++++++++++++++++++++-
 1 file changed, 218 insertions(+), 1 deletion(-)

diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index 0a413a4..251c546 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -315,7 +315,7 @@ l2fwd_poll_mode_worker(void)
  */
 
 /* Workers registered */
-#define L2FWD_EVENTMODE_WORKERS		2
+#define L2FWD_EVENTMODE_WORKERS		3
 
 /*
  * Event mode worker
@@ -639,6 +639,214 @@ l2fwd_eventmode_non_burst_tx_internal_port(void *args)
 		rte_free(links);
 }
 
+/*
+ * Event mode worker
+ * Operating mode : burst no internal port (regular tx worker)
+ */
+static void
+l2fwd_eventmode_burst_no_internal_port(void *args)
+{
+	struct rte_event ev[MAX_PKT_BURST];
+	struct rte_mbuf *pkt;
+	struct rte_eventmode_helper_conf *mode_conf;
+	struct rte_eventmode_helper_event_link_info *links = NULL;
+	unsigned int lcore_nb_link = 0;
+	uint32_t lcore_id;
+	unsigned int i, j, left, nb_rx = 0;
+	unsigned int portid;
+	struct lcore_queue_conf *qconf;
+	int is_master_core;
+	struct rte_event_port_conf event_port_conf;
+	uint16_t deq_len = 0, enq_len = 0;
+	uint8_t tx_queue;
+	struct tsc_tracker tsc = {0};
+
+	/* Get core ID */
+	lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, L2FWD,
+		"Launching event mode burst worker no internal port "
+		"(regular tx worker) on lcore %d\n", lcore_id);
+
+	/* Set the flag if master core */
+	is_master_core = (lcore_id == rte_get_master_lcore()) ? 1 : 0;
+
+	/* Get qconf for this core */
+	qconf = &lcore_queue_conf[lcore_id];
+
+	/* Set drain tsc */
+	tsc.drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /
+			US_PER_S * BURST_TX_DRAIN_US;
+
+	/* Mode conf will be passed as args */
+	mode_conf = (struct rte_eventmode_helper_conf *)args;
+
+	/* Get the links configured for this lcore */
+	lcore_nb_link = rte_eventmode_helper_get_event_lcore_links(lcore_id,
+			mode_conf, &links);
+
+	/* Check if we have links registered for this lcore */
+	if (lcore_nb_link == 0) {
+		/* No links registered. The core could do periodic drains */
+		l2fwd_drain_loop(qconf, &tsc, is_master_core);
+		goto clean_and_exit;
+	}
+
+	/* We have valid links */
+
+	/* Reset stats before proceeding */
+	reset_eth_stats(is_master_core);
+
+	/*
+	 * There is no internal port between ethdev and eventdev. So the worker
+	 * thread needs to submit event to a designated tx queue. Internally
+	 * eth core would receive events from multiple worker threads and send
+	 * out packets on wire.
+	 */
+	tx_queue = rte_eventmode_helper_get_tx_queue(mode_conf,
+						     links[0].eventdev_id);
+
+	/* Get the burst size of the event device */
+
+	/* Get the default conf of the first link */
+	rte_event_port_default_conf_get(links[0].eventdev_id,
+			links[0].event_portid,
+			&event_port_conf);
+
+	/* Save the burst size */
+	deq_len = event_port_conf.dequeue_depth;
+	enq_len = event_port_conf.enqueue_depth;
+
+	/* Dequeue and enqueue length should not exceed MAX_PKT_BURST */
+	if (deq_len > MAX_PKT_BURST)
+		deq_len = MAX_PKT_BURST;
+	if (enq_len > MAX_PKT_BURST)
+		enq_len = MAX_PKT_BURST;
+
+	/* See if it's single link */
+	if (lcore_nb_link == 1)
+		goto single_link_loop;
+	else
+		goto multi_link_loop;
+
+single_link_loop:
+
+	RTE_LOG(INFO, L2FWD, " -- lcoreid=%u event_port_id=%u\n", lcore_id,
+		links[0].event_portid);
+
+	while (!force_quit) {
+
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, &tsc, is_master_core);
+
+		/* Read packet from event queues */
+		nb_rx = rte_event_dequeue_burst(links[0].eventdev_id,
+				links[0].event_portid,
+				ev,             /* events */
+				deq_len,        /* nb_events */
+				0               /* timeout_ticks */);
+
+		if (nb_rx == 0)
+			continue;
+
+		for (j = 0; j < nb_rx; j++) {
+
+			portid = ev[j].queue_id;
+			port_statistics[portid].rx++;
+			pkt = ev[j].mbuf;
+
+			rte_prefetch0(rte_pktmbuf_mtod(pkt, void *));
+
+			/* Process packet */
+			l2fwd_event_pre_forward(&(ev[j]), portid);
+
+			/*
+			 * Internal port is not available, the packet needs
+			 * to be enqueued to the designated event queue.
+			 */
+
+			/* Prepare event for submission to tx event queue */
+			l2fwd_event_switch_to_tx_queue(&(ev[j]), tx_queue);
+		}
+
+		for (j = 0, left = nb_rx;
+			j < (nb_rx + enq_len - 1)/enq_len; j++) {
+
+			/* Submit the updated events for tx stage */
+			left -= rte_event_enqueue_burst(links[0].eventdev_id,
+					links[0].event_portid,
+					&(ev[j*enq_len]), /* events */
+					left > enq_len ?
+					enq_len : left /* nb_events */);
+		}
+	}
+	goto clean_and_exit;
+
+multi_link_loop:
+
+	for (i = 0; i < lcore_nb_link; i++) {
+		RTE_LOG(INFO, L2FWD, " -- lcoreid=%u event_port_id=%u\n",
+			lcore_id, links[i].event_portid);
+	}
+
+	while (!force_quit) {
+
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, &tsc, is_master_core);
+
+		for (i = 0; i < lcore_nb_link; i++) {
+			/* Read packet from event queues */
+			nb_rx = rte_event_dequeue_burst(links[i].eventdev_id,
+					links[i].event_portid,
+					ev,             /* events */
+					deq_len,        /* nb_events */
+					0               /* timeout_ticks */);
+
+			if (nb_rx == 0)
+				continue;
+
+			for (j = 0; j < nb_rx; j++) {
+
+				portid = ev[j].queue_id;
+				port_statistics[portid].rx++;
+				pkt = ev[j].mbuf;
+
+				rte_prefetch0(rte_pktmbuf_mtod(pkt, void *));
+
+				/* Process packet */
+				l2fwd_event_pre_forward(&(ev[j]), portid);
+
+				/*
+				 * Internal port is not available, the packet
+				 * needs to be enqueued to the designated event
+				 * queue.
+				 */
+
+				/* Update the scheduling type for tx stage */
+				l2fwd_event_switch_to_tx_queue(&(ev[j]),
+						tx_queue);
+			}
+
+			for (j = 0, left = nb_rx;
+				j < (nb_rx + enq_len - 1)/enq_len; j++) {
+
+				/* Submit the updated events for tx stage */
+				left -= rte_event_enqueue_burst(
+					links[i].eventdev_id,
+					links[i].event_portid,
+					&(ev[j*enq_len]), /* events */
+					left > enq_len ?
+					enq_len : left /* nb_events */);
+			}
+		}
+	}
+	goto clean_and_exit;
+
+clean_and_exit:
+	if (links != NULL)
+		rte_free(links);
+}
+
 static uint8_t
 l2fwd_eventmode_populate_wrkr_params(
 		struct rte_eventmode_helper_app_worker_params *wrkrs)
@@ -665,6 +873,15 @@ l2fwd_eventmode_populate_wrkr_params(
 	wrkr->worker_thread = l2fwd_eventmode_non_burst_tx_internal_port;
 
 	nb_wrkr_param++;
+	wrkr++;
+
+	/* Burst no internal port (regular tx worker) */
+	wrkr->cap.burst = RTE_EVENTMODE_HELPER_RX_TYPE_BURST;
+	wrkr->cap.tx_internal_port =
+			RTE_EVENTMODE_HELPER_TX_TYPE_NO_INTERNAL_PORT;
+	wrkr->worker_thread = l2fwd_eventmode_burst_no_internal_port;
+
+	nb_wrkr_param++;
 	return nb_wrkr_param;
 }
 
-- 
2.7.4

[dpdk-dev] [PATCH 39/39] examples/l2fwd-event: add eventmode worker

[Anoob Joseph]

Adding burst Tx internal port eventmode worker.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 examples/l2fwd-event/l2fwd_worker.c | 200 +++++++++++++++++++++++++++++++++++-
 1 file changed, 199 insertions(+), 1 deletion(-)

diff --git a/examples/l2fwd-event/l2fwd_worker.c b/examples/l2fwd-event/l2fwd_worker.c
index 251c546..1e38210 100644
--- a/examples/l2fwd-event/l2fwd_worker.c
+++ b/examples/l2fwd-event/l2fwd_worker.c
@@ -315,7 +315,7 @@ l2fwd_poll_mode_worker(void)
  */
 
 /* Workers registered */
-#define L2FWD_EVENTMODE_WORKERS		3
+#define L2FWD_EVENTMODE_WORKERS		4
 
 /*
  * Event mode worker
@@ -847,6 +847,195 @@ l2fwd_eventmode_burst_no_internal_port(void *args)
 		rte_free(links);
 }
 
+/*
+ * Event mode worker
+ * Operating mode : burst tx internal port
+ */
+static void
+l2fwd_eventmode_burst_tx_internal_port(void *args)
+{
+	struct rte_event ev[MAX_PKT_BURST];
+	struct rte_mbuf *pkt;
+	struct rte_eventmode_helper_conf *mode_conf;
+	struct rte_eventmode_helper_event_link_info *links = NULL;
+	unsigned int lcore_nb_link = 0;
+	uint32_t lcore_id;
+	unsigned int i, j, left, nb_rx = 0;
+	unsigned int portid;
+	struct lcore_queue_conf *qconf;
+	int is_master_core;
+	struct rte_event_port_conf event_port_conf;
+	uint16_t deq_len = 0, enq_len = 0;
+	struct tsc_tracker tsc = {0};
+
+	/* Get core ID */
+	lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, L2FWD,
+		"Launching event mode burst worker internal port "
+		"lcore %d\n", lcore_id);
+
+	/* Set the flag if master core */
+	is_master_core = (lcore_id == rte_get_master_lcore()) ? 1 : 0;
+
+	/* Get qconf for this core */
+	qconf = &lcore_queue_conf[lcore_id];
+
+	/* Set drain tsc */
+	tsc.drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /
+			US_PER_S * BURST_TX_DRAIN_US;
+
+	/* Mode conf will be passed as args */
+	mode_conf = (struct rte_eventmode_helper_conf *)args;
+
+	/* Get the links configured for this lcore */
+	lcore_nb_link = rte_eventmode_helper_get_event_lcore_links(lcore_id,
+			mode_conf, &links);
+
+	/* Check if we have links registered for this lcore */
+	if (lcore_nb_link == 0) {
+		/* No links registered. The core could do periodic drains */
+		l2fwd_drain_loop(qconf, &tsc, is_master_core);
+		goto clean_and_exit;
+	}
+
+	/* We have valid links */
+
+	/* Reset stats before proceeding */
+	reset_eth_stats(is_master_core);
+
+	/* Get the burst size of the event device */
+
+	/* Get the default conf of the first link */
+	rte_event_port_default_conf_get(links[0].eventdev_id,
+			links[0].event_portid,
+			&event_port_conf);
+
+	/* Save the burst size */
+	deq_len = event_port_conf.dequeue_depth;
+	enq_len = event_port_conf.enqueue_depth;
+
+	/* Dequeue and enqueue length should not exceed MAX_PKT_BURST */
+	if (deq_len > MAX_PKT_BURST)
+		deq_len = MAX_PKT_BURST;
+	if (enq_len > MAX_PKT_BURST)
+		enq_len = MAX_PKT_BURST;
+
+	/* See if it's single link */
+	if (lcore_nb_link == 1)
+		goto single_link_loop;
+	else
+		goto multi_link_loop;
+
+single_link_loop:
+
+	RTE_LOG(INFO, L2FWD, " -- lcoreid=%u event_port_id=%u\n", lcore_id,
+		links[0].event_portid);
+
+	while (!force_quit) {
+
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, &tsc, is_master_core);
+
+		/* Read packet from event queues */
+		nb_rx = rte_event_dequeue_burst(links[0].eventdev_id,
+				links[0].event_portid,
+				ev,             /* events */
+				deq_len,        /* nb_events */
+				0               /* timeout_ticks */);
+
+		if (nb_rx == 0)
+			continue;
+
+		for (j = 0; j < nb_rx; j++) {
+
+			portid = ev[j].queue_id;
+			port_statistics[portid].rx++;
+			pkt = ev[j].mbuf;
+
+			rte_prefetch0(rte_pktmbuf_mtod(pkt, void *));
+
+			/* Process packet */
+			l2fwd_event_pre_forward(&(ev[j]), portid);
+		}
+
+		/*
+		 * Since tx internal port is available, events can be
+		 * directly enqueued to the adapter and it would be
+		 * internally submitted to the eth device.
+		 */
+		for (j = 0, left = nb_rx;
+			j < (nb_rx + enq_len - 1)/enq_len; j++) {
+
+			left -= rte_event_eth_tx_adapter_enqueue(
+					links[0].eventdev_id,
+					links[0].event_portid,
+					&ev[j*enq_len],	/* events */
+					left > enq_len ?
+					enq_len : left /* nb_events */);
+		}
+	}
+	goto clean_and_exit;
+
+multi_link_loop:
+
+	for (i = 0; i < lcore_nb_link; i++) {
+		RTE_LOG(INFO, L2FWD, " -- lcoreid=%u event_port_id=%u\n",
+			lcore_id, links[i].event_portid);
+	}
+
+	while (!force_quit) {
+
+		/* Do periodic operations (buffer drain & stats monitor) */
+		l2fwd_periodic_drain_stats_monitor(qconf, &tsc, is_master_core);
+
+		for (i = 0; i < lcore_nb_link; i++) {
+			/* Read packet from event queues */
+			nb_rx = rte_event_dequeue_burst(links[i].eventdev_id,
+					links[i].event_portid,
+					ev,             /* events */
+					deq_len,        /* nb_events */
+					0               /* timeout_ticks */);
+
+			if (nb_rx == 0)
+				continue;
+
+			for (j = 0; j < nb_rx; j++) {
+
+				portid = ev[j].queue_id;
+				port_statistics[portid].rx++;
+				pkt = ev[j].mbuf;
+
+				rte_prefetch0(rte_pktmbuf_mtod(pkt, void *));
+
+				/* Process packet */
+				l2fwd_event_pre_forward(&(ev[j]), portid);
+			}
+
+			/*
+			 * Since tx internal port is available, events can be
+			 * directly enqueued to the adapter and it would be
+			 * internally submitted to the eth device.
+			 */
+			for (j = 0, left = nb_rx;
+				j < (nb_rx + enq_len - 1)/enq_len; j++) {
+
+				left -= rte_event_eth_tx_adapter_enqueue(
+					links[i].eventdev_id,
+					links[i].event_portid,
+					&ev[j*enq_len],	/* events */
+					left > enq_len ?
+					enq_len : left /* nb_events */);
+			}
+		}
+	}
+	goto clean_and_exit;
+
+clean_and_exit:
+	if (links != NULL)
+		rte_free(links);
+}
+
 static uint8_t
 l2fwd_eventmode_populate_wrkr_params(
 		struct rte_eventmode_helper_app_worker_params *wrkrs)
@@ -882,6 +1071,15 @@ l2fwd_eventmode_populate_wrkr_params(
 	wrkr->worker_thread = l2fwd_eventmode_burst_no_internal_port;
 
 	nb_wrkr_param++;
+	wrkr++;
+
+	/* Burst tx internal port */
+	wrkr->cap.burst = RTE_EVENTMODE_HELPER_RX_TYPE_BURST;
+	wrkr->cap.tx_internal_port =
+			RTE_EVENTMODE_HELPER_TX_TYPE_INTERNAL_PORT;
+	wrkr->worker_thread = l2fwd_eventmode_burst_tx_internal_port;
+
+	nb_wrkr_param++;
 	return nb_wrkr_param;
 }
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH 00/39] adding eventmode helper library

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 11:02 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: [PATCH 00/39] adding eventmode helper library
> 
> This series adds support for eventmode helper library and l2fwd-event
> application.
> 
> First 13 patches creates a new l2fwd application (l2fwd-event). Minor code
> reorganization is done to faciliate seamless integration of eventmode.
> 
> Next 22 patches adds eventmode helper library. This library abstracts the
> configuration of event device & Rx-Tx event adapters. The library can be
> extended to allow users to control all the configuration exposed by adapters
> and eth device.
> 
> Last 4 patches implements eventmode in l2fwd-event application. With
> event device and adapters, fine tuned threads (based on dev
> capabilities) can be drafted to maximize performance. Eventmode library
> facilitates this and l2fwd-event demonstrates this usage.
> 
> With the introduction of eventmode helper library, any poll mode application
> can be converted to an eventmode application with simple steps, enabling
> multi-core scaling and dynamic load balancing to various example
> applications.


Anyone planning to review this changes?
I will spend time to review this. Requesting the review from other eventdev stake holders.

Re: [dpdk-dev] [PATCH 04/39] examples/l2fwd-event: move global vars to common header

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 11:02 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: [PATCH 04/39] examples/l2fwd-event: move global vars to common
> header
> 
> Moving global variables to common header for access from control plane and
> data plane code.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---
>  examples/l2fwd-event/l2fwd_common.h | 26
> +++++++++++++++++++++++
>  examples/l2fwd-event/main.c         | 41 +++++++++++++++---------------------
> -
>  2 files changed, 43 insertions(+), 24 deletions(-)
> 
> diff --git a/examples/l2fwd-event/l2fwd_common.h b/examples/l2fwd-
> event/l2fwd_common.h
> index a7bb5af..55226f7 100644
> --- a/examples/l2fwd-event/l2fwd_common.h
> +++ b/examples/l2fwd-event/l2fwd_common.h
> @@ -5,6 +5,10 @@
>  #ifndef _L2FWD_COMMON_H_
>  #define _L2FWD_COMMON_H_
> 
> +#include <stdbool.h>
> +
> +#include <rte_common.h>
> +
>  #define RTE_LOGTYPE_L2FWD RTE_LOGTYPE_USER1
> 
>  #define MAX_PKT_BURST 32
> @@ -34,4 +38,26 @@ struct l2fwd_port_statistics {
>  	uint64_t dropped;
>  } __rte_cache_aligned;
> 
> +volatile bool force_quit;
> +
> +int mac_updating;
> +
> +/* ethernet addresses of ports */
> +static struct rte_ether_addr l2fwd_ports_eth_addr[RTE_MAX_ETHPORTS];
> +
> +/* mask of enabled ports */
> +static uint32_t l2fwd_enabled_port_mask;
> +
> +/* list of enabled ports */
> +static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS];
> +
> +struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];
> +
> +struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
> +
> +struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS];
> +
> +/* A tsc-based timer responsible for triggering statistics printout */
> +uint64_t timer_period;

Instead of moving global variables to other header file, IMO, it is better
to create a structure with context and share with workers with
rte_eal_mp_remote_launch() or so.

Re: [dpdk-dev] [PATCH 04/39] examples/l2fwd-event: move global vars to common header

[Anoob Joseph]

Hi Jerin,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran
> Sent: Friday, June 7, 2019 3:33 PM
> To: Anoob Joseph <anoobj@marvell.com>; Nikhil Rao <nikhil.rao@intel.com>;
> Erik Gabriel Carrillo <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal <hemant.agrawal@nxp.com>;
> Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: RE: [PATCH 04/39] examples/l2fwd-event: move global vars to common
> header
> 
> > -----Original Message-----
> > From: Anoob Joseph <anoobj@marvell.com>
> > Sent: Monday, June 3, 2019 11:02 PM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> > <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> > <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> > <abhinandan.gujjar@intel.com>; Bruce Richardson
> > <bruce.richardson@intel.com>; Pablo de Lara
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> > <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> > <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>;
> > Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> > <harry.van.haaren@intel.com>; Mattias Rönnblom
> > <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> > Subject: [PATCH 04/39] examples/l2fwd-event: move global vars to
> > common header
> >
> > Moving global variables to common header for access from control plane
> > and data plane code.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > ---
> >  examples/l2fwd-event/l2fwd_common.h | 26
> > +++++++++++++++++++++++
> >  examples/l2fwd-event/main.c         | 41 +++++++++++++++---------------------
> > -
> >  2 files changed, 43 insertions(+), 24 deletions(-)
> >
> > diff --git a/examples/l2fwd-event/l2fwd_common.h b/examples/l2fwd-
> > event/l2fwd_common.h index a7bb5af..55226f7 100644
> > --- a/examples/l2fwd-event/l2fwd_common.h
> > +++ b/examples/l2fwd-event/l2fwd_common.h
> > @@ -5,6 +5,10 @@
> >  #ifndef _L2FWD_COMMON_H_
> >  #define _L2FWD_COMMON_H_
> >
> > +#include <stdbool.h>
> > +
> > +#include <rte_common.h>
> > +
> >  #define RTE_LOGTYPE_L2FWD RTE_LOGTYPE_USER1
> >
> >  #define MAX_PKT_BURST 32
> > @@ -34,4 +38,26 @@ struct l2fwd_port_statistics {
> >  	uint64_t dropped;
> >  } __rte_cache_aligned;
> >
> > +volatile bool force_quit;
> > +
> > +int mac_updating;
> > +
> > +/* ethernet addresses of ports */
> > +static struct rte_ether_addr l2fwd_ports_eth_addr[RTE_MAX_ETHPORTS];
> > +
> > +/* mask of enabled ports */
> > +static uint32_t l2fwd_enabled_port_mask;
> > +
> > +/* list of enabled ports */
> > +static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS];
> > +

[Anoob] Static has to be removed from all the above vars. Will fix in the next version.

> > +struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];
> > +
> > +struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
> > +
> > +struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS];
> > +
> > +/* A tsc-based timer responsible for triggering statistics printout
> > +*/ uint64_t timer_period;
> 
> Instead of moving global variables to other header file, IMO, it is better to
> create a structure with context and share with workers with
> rte_eal_mp_remote_launch() or so.

[Anoob] That would make the design a bit different from regular l2fwd. Even in l2fwd, all these variable are present. Be it global or static.

Another option is to have the vars declared as extern in the l2fwd_worker.c and remove the additions in the header. If that approach is fine, we can keep the changes between l2fwd & l2fwd-event minimal. Please share your thoughts on which approach would be better.

Re: [dpdk-dev] [PATCH 04/39] examples/l2fwd-event: move global vars to common header

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Anoob Joseph
> Sent: Friday, June 7, 2019 4:15 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Narayana Prasad Raju Athreya <pathreya@marvell.com>; dev@dpdk.org;
> Lukas Bartosik <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal <hemant.agrawal@nxp.com>;
> Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: RE: [PATCH 04/39] examples/l2fwd-event: move global vars to common
> header
> 
> Hi Jerin,

Hi Anoob,

> 
> Please see inline.
> 
> Thanks,
> Anoob
> 
> > -----Original Message-----
> > From: Jerin Jacob Kollanukkaran
> > Sent: Friday, June 7, 2019 3:33 PM
> > To: Anoob Joseph <anoobj@marvell.com>; Nikhil Rao
> > <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> > <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> > <abhinandan.gujjar@intel.com>; Bruce Richardson
> > <bruce.richardson@intel.com>; Pablo de Lara
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> > <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> > <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>;
> > Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> > <harry.van.haaren@intel.com>; Mattias Rönnblom
> > <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> > Subject: RE: [PATCH 04/39] examples/l2fwd-event: move global vars to
> > common header
> >
> > > -----Original Message-----
> > > From: Anoob Joseph <anoobj@marvell.com>
> > > Sent: Monday, June 3, 2019 11:02 PM
> > > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> > > <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> > > <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> > > <abhinandan.gujjar@intel.com>; Bruce Richardson
> > > <bruce.richardson@intel.com>; Pablo de Lara
> > > <pablo.de.lara.guarch@intel.com>
> > > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> > > <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> > > <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> > > <pbhagavatula@marvell.com>; Hemant Agrawal
> > <hemant.agrawal@nxp.com>;
> > > Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> > > <harry.van.haaren@intel.com>; Mattias Rönnblom
> > > <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> > > Subject: [PATCH 04/39] examples/l2fwd-event: move global vars to
> > > common header
> > >
> > > Moving global variables to common header for access from control
> > > plane and data plane code.
> > >
> > > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > > ---
> > >  examples/l2fwd-event/l2fwd_common.h | 26
> > > +++++++++++++++++++++++
> > >  examples/l2fwd-event/main.c         | 41 +++++++++++++++---------------------
> > > -
> > >  2 files changed, 43 insertions(+), 24 deletions(-)
> > >
> > > diff --git a/examples/l2fwd-event/l2fwd_common.h b/examples/l2fwd-
> > > event/l2fwd_common.h index a7bb5af..55226f7 100644
> > > --- a/examples/l2fwd-event/l2fwd_common.h
> > > +++ b/examples/l2fwd-event/l2fwd_common.h
> > > @@ -5,6 +5,10 @@
> > >  #ifndef _L2FWD_COMMON_H_
> > >  #define _L2FWD_COMMON_H_
> > >
> > > +#include <stdbool.h>
> > > +
> > > +#include <rte_common.h>
> > > +
> > >  #define RTE_LOGTYPE_L2FWD RTE_LOGTYPE_USER1
> > >
> > >  #define MAX_PKT_BURST 32
> > > @@ -34,4 +38,26 @@ struct l2fwd_port_statistics {
> > >  	uint64_t dropped;
> > >  } __rte_cache_aligned;
> > >
> > > +volatile bool force_quit;
> > > +
> > > +int mac_updating;
> > > +
> > > +/* ethernet addresses of ports */
> > > +static struct rte_ether_addr
> > > +l2fwd_ports_eth_addr[RTE_MAX_ETHPORTS];
> > > +
> > > +/* mask of enabled ports */
> > > +static uint32_t l2fwd_enabled_port_mask;
> > > +
> > > +/* list of enabled ports */
> > > +static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS];
> > > +
> 
> [Anoob] Static has to be removed from all the above vars. Will fix in the next
> version.

OK

> > > +struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];
> > > +
> > > +struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
> > > +
> > > +struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS];
> > > +
> > > +/* A tsc-based timer responsible for triggering statistics printout
> > > +*/ uint64_t timer_period;
> >
> > Instead of moving global variables to other header file, IMO, it is
> > better to create a structure with context and share with workers with
> > rte_eal_mp_remote_launch() or so.
> 
> [Anoob] That would make the design a bit different from regular l2fwd. Even in
> l2fwd, all these variable are present. Be it global or static.
> 
> Another option is to have the vars declared as extern in the l2fwd_worker.c and
> remove the additions in the header. If that approach is fine, we can keep the
> changes between l2fwd & l2fwd-event minimal. Please share your thoughts on
> which approach would be better.

I think, we can avoid global variable to communicate between
Workers. I think, we don't need worry about diff between l2fwd and l2fwd-event.
If worried about maintaining the patches splitting then I can think we can
squash a few l2fwd app patches if required to add the context structure.
It will be scalable approach.

Re: [dpdk-dev] [PATCH 18/39] eventdev: add framework for eventmode conf

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 11:02 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: [PATCH 18/39] eventdev: add framework for eventmode conf
> 
> Adding eventmode conf which would have all required configuration for the
> event mode.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---
>  lib/librte_eventdev/rte_eventmode_helper.c          | 16 ++++++++++++++++
>  lib/librte_eventdev/rte_eventmode_helper_internal.h |  5 +++++
>  2 files changed, 21 insertions(+)
> 
> diff --git a/lib/librte_eventdev/rte_eventmode_helper.c
> b/lib/librte_eventdev/rte_eventmode_helper.c
> index dc2c934..38f1a2b 100644
> --- a/lib/librte_eventdev/rte_eventmode_helper.c
> +++ b/lib/librte_eventdev/rte_eventmode_helper.c
> @@ -97,6 +97,7 @@ rte_eventmode_helper_parse_args(int argc, char
> **argv)  {
>  	int32_t opt, ret;
>  	struct rte_eventmode_helper_conf *conf = NULL;
> +	struct eventmode_conf *em_conf = NULL;
> 
>  	/* Allocate memory for conf */
>  	conf = rte_zmalloc("eventmode-helper-conf",
> @@ -108,9 +109,21 @@ rte_eventmode_helper_parse_args(int argc, char
> **argv)
>  			goto err;
>  	}
> 
> +	/* Allocate memory for event mode params */
> +	conf->mode_params = rte_zmalloc("eventmode-helper-mode-
> params",

Use hugepage alloc memory only fastpath. Malloc() would be fine here.

Re: [dpdk-dev] [PATCH 14/39] eventdev: add files for eventmode helper

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 11:02 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: [PATCH 14/39] eventdev: add files for eventmode helper
> 
> Adding files for eventmode helper
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---
>  lib/librte_eventdev/Makefile                        |  5 +++--
>  lib/librte_eventdev/meson.build                     |  2 ++
>  lib/librte_eventdev/rte_eventmode_helper.c          |  7 +++++++
>  lib/librte_eventdev/rte_eventmode_helper.h          | 15 +++++++++++++++

s/rte_eventmode_helper/rte_event_helper/gc. See below

>  lib/librte_eventdev/rte_eventmode_helper_internal.h |  6 ++++++
>  5 files changed, 33 insertions(+), 2 deletions(-)  create mode 100644
> lib/librte_eventdev/rte_eventmode_helper.c
> +#ifndef _RTE_EVENTMODE_HELPER_H_
> +#define _RTE_EVENTMODE_HELPER_H_

In order to shortened the macros, enums, APIs, I think,
We can change to rte_event_helper from rte_eventmode_here.

> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* _RTE_EVENTMODE_HELPER_H_ */
> diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h
> b/lib/librte_eventdev/rte_eventmode_helper_internal.h
> new file mode 100644
> index 0000000..0da7003
> --- /dev/null
> +++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
> @@ -0,0 +1,6 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (C) 2019 Marvell International Ltd.
> + */
> +#ifndef _RTE_EVENTMODE_HELPER_INTERNAL_H_ #define


Please have this patch for Helper API definition(spec) and remaining patches
for implementation. That would help contract between helper and application.


> +_RTE_EVENTMODE_HELPER_INTERNAL_H_
> +#endif /* _RTE_EVENTMODE_HELPER_INTERNAL_H_ */
> --
> 2.7.4

Re: [dpdk-dev] [PATCH 15/39] eventdev: add routines for logging eventmode helper

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 11:02 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: [PATCH 15/39] eventdev: add routines for logging eventmode
> helper
> 
> Adding routines for logging eventmode helper.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---
>  config/common_base                                 |  1 +
>  lib/librte_eal/common/eal_common_log.c             |  1 +
>  lib/librte_eal/common/include/rte_log.h            |  1 +
>  .../rte_eventmode_helper_internal.h                | 29
> ++++++++++++++++++++++
>  4 files changed, 32 insertions(+)
> 
> diff --git a/config/common_base b/config/common_base index
> 8576973..96c2537 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -685,6 +685,7 @@ CONFIG_RTE_LIBRTE_PMD_ZLIB=n  #
> CONFIG_RTE_LIBRTE_EVENTDEV=y
> CONFIG_RTE_LIBRTE_EVENTDEV_DEBUG=n
> +CONFIG_RTE_LIBRTE_EVENTMODE_HELPER_DEBUG=n
>  CONFIG_RTE_EVENT_MAX_DEVS=16
>  CONFIG_RTE_EVENT_MAX_QUEUES_PER_DEV=64
>  CONFIG_RTE_EVENT_TIMER_ADAPTER_NUM_MAX=32
> diff --git a/lib/librte_eal/common/eal_common_log.c
> b/lib/librte_eal/common/eal_common_log.c
> index c714a4b..bc34dc2 100644
> --- a/lib/librte_eal/common/eal_common_log.c
> +++ b/lib/librte_eal/common/eal_common_log.c
> @@ -324,6 +324,7 @@ static const struct logtype logtype_strings[] = {
>  	{RTE_LOGTYPE_EFD,        "lib.efd"},
>  	{RTE_LOGTYPE_EVENTDEV,   "lib.eventdev"},
>  	{RTE_LOGTYPE_GSO,        "lib.gso"},
> +	{RTE_LOGTYPE_EVENTMODE,  "lib.eventmode"},

EAL log type, typically adds for subsystem. Please reuse EVENTDEV_DEBUG
Or introduce dynamic debugging. 

Re: [dpdk-dev] [PATCH 16/39] eventdev: add eventmode CL options framework

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 11:02 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: [PATCH 16/39] eventdev: add eventmode CL options framework
> 
> Adding usage prints and CL parsing routines for eventmode. Option to select
> packet transfer mode is also added.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---
> +#include <rte_compat.h>
> +
> +/* Packet transfer mode of the application */ enum
> +rte_eventmode_helper_pkt_transfer_mode {
> +	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL = 0,
> +	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT,
> +};

Need to mark all public structues as EXPERIMENTAL.
Please grep for EXPERIMENTAL in libeventdev or any other library

> +struct rte_eventmode_helper_conf {
> +	enum rte_eventmode_helper_pkt_transfer_mode mode;
> +		/**< Packet transfer mode of the application */
> +	void *mode_params;
> +		/**< Mode specific parameters */
> +};

Please make this event_helper object as 'opaque'  and move to internal so that it does have
ABI policies and future new features can be added.

Re: [dpdk-dev] [PATCH 19/39] eventdev: add common initialize routine for eventmode devs

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 11:02 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: [PATCH 19/39] eventdev: add common initialize routine for
> eventmode devs
> 
> Adding framework for common initialization routine for event mode.
> Event mode would involve initialization of multiple devices, like eventdev,
> ethdev etc and this routine would be the placeholder for all initialization to
> come in.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---
> 
> +/* Helper functions for initialization, & launching workers */
> +
> +/**
> + * Initialize event mode devices
> + *
> + * Application could call this function to get the event device, eth
> +device
> + * and eth rx adapter initialized according to the conf populated using
> +the
> + * command line args.
> + *
> + * Application is expected to initialize the eth device and then the
> +eventmode
> + * helper subsystem will stop & start eth device according to it's
> requirement.
> + * So call to this function should be done after the eth device is
> +successfully
> + * initialized.
> + *
> + * @param mode_conf
> + *   Configuration of the mode in which app is doing packet handling
> + * @return
> + *  - 0 on success.
> + *  - (<0) on failure.
> + */
> +int32_t __rte_experimental
> +rte_eventmode_helper_initialize_devs(
> +		struct rte_eventmode_helper_conf *mode_conf);
> +

# Prefer to change to rte_event_helper_init() and 
introduce the counter part for the same(rte_event_helper_uninit() or rte_event_helper_fini())
# introduce params structure taking another paraments input instead of new APIs.
# let library return rte_event_helper_conf* object for further operations.


>  #ifdef __cplusplus
>  }
>  #endif
> --
> 2.7.4

Re: [dpdk-dev] [PATCH 25/39] eventdev: add routine to validate conf

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 11:02 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: [PATCH 25/39] eventdev: add routine to validate conf
> 
> Adding routine to validate event mode conf. This function will verify the conf
> requested by the user and would populate other fields with default values.
> Presently, the function acts as placeholder for the above mentioned actions.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---
>  lib/librte_eventdev/rte_eventmode_helper.c | 33
> ++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 
> diff --git a/lib/librte_eventdev/rte_eventmode_helper.c
> b/lib/librte_eventdev/rte_eventmode_helper.c
> index a57f837..4dbb94a 100644
> --- a/lib/librte_eventdev/rte_eventmode_helper.c
> +++ b/lib/librte_eventdev/rte_eventmode_helper.c
> @@ -165,6 +165,32 @@ rte_eventmode_helper_parse_args(int argc, char
> **argv)
>  	return NULL;
>  }
> 
> +/* Pre-process conf before using for init*/
> +
> +static int
> +rte_eventmode_validate_user_params(struct eventmode_conf
> *em_conf) {

No need to start with rte_eventmode_... for internal functions

Re: [dpdk-dev] [PATCH 32/39] eventdev: add routine to launch eventmode workers

[Carrillo, Erik G]

Hi Anoob,

I've listed a few notes in-line.

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 12:33 PM
> To: Jerin Jacob <jerinj@marvell.com>; Rao, Nikhil <nikhil.rao@intel.com>;
> Carrillo, Erik G <erik.g.carrillo@intel.com>; Gujjar, Abhinandan S
> <abhinandan.gujjar@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad
> <pathreya@marvell.com>; dev@dpdk.org; Lukasz Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh <pbhagavatula@marvell.com>;
> Hemant Agrawal <hemant.agrawal@nxp.com>; Nipun Gupta
> <nipun.gupta@nxp.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Ma, Liang J <liang.j.ma@intel.com>
> Subject: [PATCH 32/39] eventdev: add routine to launch eventmode workers
> 
> With eventmode, workers could be drafted differently according to the
> capabilities of the underlying event device. The added function would
> receive an array of such workers and probes the eventmode properties to
> choose the worker.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---

<...snipped...>

> +
> +/* Event mode packet tx types */
> +enum rte_eventmode_helper_tx_types {
> +	RTE_EVETNMODE_HELPER_TX_TYPE_INVALID = 0,

A couple of characters are transposed in the above.

> +	RTE_EVENTMODE_HELPER_TX_TYPE_MAX = 16
> +};
> +
>  struct rte_eventmode_helper_conf {
>  	enum rte_eventmode_helper_pkt_transfer_mode mode;
>  		/**< Packet transfer mode of the application */ @@ -41,6
> +55,20 @@ struct rte_eventmode_helper_event_link_info {
>  		/**< Lcore to be polling on this port */  };
>

I believe anonymous unions and structures should be annotated with RTE_STD_C11 below and in other places throughout the series.

> +/* Workers registered by the application */ struct
> +rte_eventmode_helper_app_worker_params {
> +	union {
> +		struct {
> +			uint64_t burst : 4;
> +			/**< Specify status of rx type burst */
> +		};
> +		uint64_t u64;
> +	} cap;
> +			/**< Capabilities of this worker */
> +	void (*worker_thread)(void *mode_conf);
> +			/**< Worker thread */
> +};
> +
>  /* Common helper functions for command line parsing */
> 
>  /**
> @@ -157,6 +185,27 @@ uint8_t __rte_experimental
> rte_eventmode_helper_get_tx_queue(struct rte_eventmode_helper_conf
> *mode_conf,
>  		uint8_t eventdev_id);
> 

The doxygen documentation for __rte_experimental functions in general should have:

* @warning                                                                     
* @b EXPERIMENTAL: this API may change without prior notice

as well.

> +/**
> + * Launch eventmode worker
> + *
> + * The application can request the eventmode helper subsystem to launch
> +the
> + * worker based on the capabilities of event device and the options
> +selected
> + * while initializing the eventmode.
> + *
> + * @param mode_conf
> + *   Configuration of the mode in which app is doing packet handling
> + * @param app_wrkr
> + *   List of all the workers registered by application, along with it's
> + *   capabilities
> + * @param nb_wrkr_param
> + *   Number of workers passed by the application
> + *
> + */
> +void __rte_experimental
> +rte_eventmode_helper_launch_worker(struct
> rte_eventmode_helper_conf *mode_conf,
> +		struct rte_eventmode_helper_app_worker_params
> *app_wrkr,
> +		uint8_t nb_wrkr_param);
> +
>  #ifdef __cplusplus
>  }
>  #endif

<...snipped...>

Regards,
Erik

Re: [dpdk-dev] [PATCH 32/39] eventdev: add routine to launch eventmode workers

[Carrillo, Erik G]

Hi Anoob,

One other observation in-line:

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 12:33 PM
> To: Jerin Jacob <jerinj@marvell.com>; Rao, Nikhil <nikhil.rao@intel.com>;
> Carrillo, Erik G <erik.g.carrillo@intel.com>; Gujjar, Abhinandan S
> <abhinandan.gujjar@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad
> <pathreya@marvell.com>; dev@dpdk.org; Lukasz Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh <pbhagavatula@marvell.com>;
> Hemant Agrawal <hemant.agrawal@nxp.com>; Nipun Gupta
> <nipun.gupta@nxp.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Ma, Liang J <liang.j.ma@intel.com>
> Subject: [PATCH 32/39] eventdev: add routine to launch eventmode workers
> 
> With eventmode, workers could be drafted differently according to the
> capabilities of the underlying event device. The added function would
> receive an array of such workers and probes the eventmode properties to
> choose the worker.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---

<...Snipped...>

> +
> +void __rte_experimental
> +rte_eventmode_helper_launch_worker(struct
> rte_eventmode_helper_conf *mode_conf,
> +		struct rte_eventmode_helper_app_worker_params
> *app_wrkr,
> +		uint8_t nb_wrkr_param)
> +{
> +	struct rte_eventmode_helper_app_worker_params *match_wrkr;
> +	uint32_t lcore_id;
> +	struct eventmode_conf *em_conf;
> +
> +	if (mode_conf == NULL) {
> +		RTE_EM_HLPR_LOG_ERR("Invalid conf");
> +		return;
> +	}
> +
> +	if (mode_conf->mode_params == NULL) {
> +		RTE_EM_HLPR_LOG_ERR("Invalid mode params");
> +		return;
> +	}
> +
> +	/* Get eventmode conf */
> +	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
> +
> +	/* Get core ID */
> +	lcore_id = rte_lcore_id();
> +
> +	/* TODO check capability for rx core */
> +
> +	/* Check if this is rx core */
> +	if (em_conf->eth_core_mask & (1 << lcore_id)) {

In the above, eth_core_mask is a uint32_t, but RTE_MAX_LCORE=128 in the common config.

> +		rte_eventmode_helper_start_worker_eth_core(em_conf,
> lcore_id);
> +		return;
> +	}
> +
> +	if (app_wrkr == NULL || nb_wrkr_param == 0) {
> +		RTE_EM_HLPR_LOG_ERR("Invalid args");
> +		return;
> +	}
> +
> +	/*
> +	 * This is a regular worker thread. The application would be
> +	 * registering multiple workers with various capabilities. The
> +	 * worker to be run will be selected by the capabilities of the
> +	 * event device configured.
> +	 */
> +
> +	/* Get the first matching worker for the event device */
> +	match_wrkr = rte_eventmode_helper_find_worker(lcore_id,
> +			em_conf,
> +			app_wrkr,
> +			nb_wrkr_param);
> +
> +	if (match_wrkr == NULL) {
> +		RTE_EM_HLPR_LOG_ERR(
> +			"No matching worker registered for lcore %d",
> lcore_id);
> +		goto clean_and_exit;
> +	}
> +
> +	/* Verify sanity of the matched worker */
> +	if (rte_eventmode_helper_verify_match_worker(match_wrkr) != 1)
> {
> +		RTE_EM_HLPR_LOG_ERR("Error in validating the matched
> worker");
> +		goto clean_and_exit;
> +	}
> +
> +	/* Launch the worker thread */
> +	match_wrkr->worker_thread(mode_conf);
> +
> +clean_and_exit:
> +
> +	/* Flag eth_cores to stop, if started */
> +	rte_eventmode_helper_stop_worker_eth_core();
> +}

<...Snipped...>

Regards,
Erik

Re: [dpdk-dev] [PATCH 24/39] eventdev: add Rx adapter init in eventmode

[Carrillo, Erik G]

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, June 3, 2019 12:32 PM
> To: Jerin Jacob <jerinj@marvell.com>; Rao, Nikhil <nikhil.rao@intel.com>;
> Carrillo, Erik G <erik.g.carrillo@intel.com>; Gujjar, Abhinandan S
> <abhinandan.gujjar@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad
> <pathreya@marvell.com>; dev@dpdk.org; Lukasz Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh <pbhagavatula@marvell.com>;
> Hemant Agrawal <hemant.agrawal@nxp.com>; Nipun Gupta
> <nipun.gupta@nxp.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Ma, Liang J <liang.j.ma@intel.com>
> Subject: [PATCH 24/39] eventdev: add Rx adapter init in eventmode
> 
> Adding rx adapter conf. The helper init routine would be initializing the rx
> adapter according to the conf.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---

<...Snipped...>

> diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h
> b/lib/librte_eventdev/rte_eventmode_helper_internal.h
> index 2a6cd90..9c68605 100644
> --- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
> +++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
> @@ -35,6 +35,12 @@
>  /* Max event devices supported */
>  #define EVENT_MODE_MAX_EVENT_DEVS RTE_EVENT_MAX_DEVS
> 
> +/* Max Rx adapters supported */
> +#define EVENT_MODE_MAX_RX_ADAPTERS RTE_EVENT_MAX_DEVS
> +
> +/* Max Rx adapter connections */
> +#define EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER 16
> +
>  /* Max event queues supported per event device */  #define
> EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV
> RTE_EVENT_MAX_QUEUES_PER_DEV
> 
> @@ -50,12 +56,33 @@ struct eventdev_params {
>  	uint8_t ev_queue_mode;
>  };
> 

Should the struct below be named "rx_adapter_connection_info" since you add "tx_adapter_connection_info" in a later patch?

> +/* Rx adapter connection info */
> +struct adapter_connection_info {
> +	uint8_t ethdev_id;
> +	uint8_t eventq_id;
> +	int32_t ethdev_rx_qid;
> +};
> +
> +/* Rx adapter conf */
> +struct rx_adapter_conf {
> +	int32_t eventdev_id;
> +	int32_t adapter_id;
> +	uint32_t rx_core_id;
> +	uint8_t nb_connections;
> +	struct adapter_connection_info
> +
> 	conn[EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER];
> +};
> +

<...Snipped...>

Regards,
Erik

Re: [dpdk-dev] [EXT] Re: [PATCH 24/39] eventdev: add Rx adapter init in eventmode

[Anoob Joseph]

Hi Erik,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Carrillo, Erik G
> Sent: Monday, June 10, 2019 8:26 PM
> To: Anoob Joseph <anoobj@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Rao, Nikhil <nikhil.rao@intel.com>; Gujjar,
> Abhinandan S <abhinandan.gujjar@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: Narayana Prasad Raju Athreya <pathreya@marvell.com>; dev@dpdk.org;
> Lukas Bartosik <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Van
> Haaren, Harry <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Ma, Liang J <liang.j.ma@intel.com>
> Subject: [EXT] Re: [dpdk-dev] [PATCH 24/39] eventdev: add Rx adapter init in
> eventmode
> 
> External Email
> 
> ----------------------------------------------------------------------
> > -----Original Message-----
> > From: Anoob Joseph <anoobj@marvell.com>
> > Sent: Monday, June 3, 2019 12:32 PM
> > To: Jerin Jacob <jerinj@marvell.com>; Rao, Nikhil
> > <nikhil.rao@intel.com>; Carrillo, Erik G <erik.g.carrillo@intel.com>;
> > Gujjar, Abhinandan S <abhinandan.gujjar@intel.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad
> > <pathreya@marvell.com>; dev@dpdk.org; Lukasz Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh <pbhagavatula@marvell.com>;
> > Hemant Agrawal <hemant.agrawal@nxp.com>; Nipun Gupta
> > <nipun.gupta@nxp.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>;
> > Mattias Rönnblom <mattias.ronnblom@ericsson.com>; Ma, Liang J
> > <liang.j.ma@intel.com>
> > Subject: [PATCH 24/39] eventdev: add Rx adapter init in eventmode
> >
> > Adding rx adapter conf. The helper init routine would be initializing
> > the rx adapter according to the conf.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > ---
> 
> <...Snipped...>
> 
> > diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h
> > b/lib/librte_eventdev/rte_eventmode_helper_internal.h
> > index 2a6cd90..9c68605 100644
> > --- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
> > +++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
> > @@ -35,6 +35,12 @@
> >  /* Max event devices supported */
> >  #define EVENT_MODE_MAX_EVENT_DEVS RTE_EVENT_MAX_DEVS
> >
> > +/* Max Rx adapters supported */
> > +#define EVENT_MODE_MAX_RX_ADAPTERS RTE_EVENT_MAX_DEVS
> > +
> > +/* Max Rx adapter connections */
> > +#define EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER 16
> > +
> >  /* Max event queues supported per event device */  #define
> > EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV
> > RTE_EVENT_MAX_QUEUES_PER_DEV
> >
> > @@ -50,12 +56,33 @@ struct eventdev_params {
> >  	uint8_t ev_queue_mode;
> >  };
> >
> 
> Should the struct below be named "rx_adapter_connection_info" since you
> add "tx_adapter_connection_info" in a later patch?

[Anoob] Yes. Will make this change in v2.

> 
> > +/* Rx adapter connection info */
> > +struct adapter_connection_info {
> > +	uint8_t ethdev_id;
> > +	uint8_t eventq_id;
> > +	int32_t ethdev_rx_qid;
> > +};
> > +
> > +/* Rx adapter conf */
> > +struct rx_adapter_conf {
> > +	int32_t eventdev_id;
> > +	int32_t adapter_id;
> > +	uint32_t rx_core_id;
> > +	uint8_t nb_connections;
> > +	struct adapter_connection_info
> > +
> > 	conn[EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER];
> > +};
> > +
> 
> <...Snipped...>
> 
> Regards,
> Erik

Re: [dpdk-dev] [EXT] [PATCH 16/39] eventdev: add eventmode CL options framework

[Sunil Kumar Kori]

Regards
Sunil Kumar Kori

>-----Original Message-----
>From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
>Sent: Monday, June 3, 2019 11:02 PM
>To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
><nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
>Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
><bruce.richardson@intel.com>; Pablo de Lara
><pablo.de.lara.guarch@intel.com>
>Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
><pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
><lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
><pbhagavatula@marvell.com>; Hemant Agrawal
><hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
>van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
><mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
>Subject: [EXT] [dpdk-dev] [PATCH 16/39] eventdev: add eventmode CL options
>framework
>
>External Email
>
>----------------------------------------------------------------------
>Adding usage prints and CL parsing routines for eventmode. Option to select
>packet transfer mode is also added.
>
>Signed-off-by: Anoob Joseph <anoobj@marvell.com>
>Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
>---
> lib/librte_eventdev/rte_eventdev_version.map |   2 +
> lib/librte_eventdev/rte_eventmode_helper.c   | 128
>+++++++++++++++++++++++++++
> lib/librte_eventdev/rte_eventmode_helper.h   |  51 +++++++++++
> 3 files changed, 181 insertions(+)
>
>diff --git a/lib/librte_eventdev/rte_eventdev_version.map
>b/lib/librte_eventdev/rte_eventdev_version.map
>index 95fd089..1199064 100644
>--- a/lib/librte_eventdev/rte_eventdev_version.map
>+++ b/lib/librte_eventdev/rte_eventdev_version.map
>@@ -128,4 +128,6 @@ EXPERIMENTAL {
>
> 	rte_event_eth_rx_adapter_cb_register;
> 	rte_event_eth_rx_adapter_stats_get;
>+	rte_eventmode_helper_print_options_list;
>+	rte_eventmode_helper_print_options_description;
> };
>diff --git a/lib/librte_eventdev/rte_eventmode_helper.c
>b/lib/librte_eventdev/rte_eventmode_helper.c
>index f47970e..8119306 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper.c
>+++ b/lib/librte_eventdev/rte_eventmode_helper.c
>@@ -1,7 +1,135 @@
> /* SPDX-License-Identifier: BSD-3-Clause
>  * Copyright (C) 2019 Marvell International Ltd.
>  */
>+#include <getopt.h>
>
> #include <rte_eventmode_helper.h>
>+#include <rte_malloc.h>
>
> #include "rte_eventmode_helper_internal.h"
>+
>+#define CMD_LINE_OPT_TRANSFER_MODE	"transfer-mode"
>+
>+static const char short_options[] =
>+	""
>+	;
>+
>+enum {
>+	/* long options mapped to a short option */
>+
>+	/* first long only option value must be >= 256, so that we won't
>+	 * conflict with short options
>+	 */
>+	CMD_LINE_OPT_MIN_NUM = 256,
>+	CMD_LINE_OPT_TRANSFER_MODE_NUM,
>+};
>+
>+static const struct option lgopts[] = {
>+	{CMD_LINE_OPT_TRANSFER_MODE, 1, 0,
>CMD_LINE_OPT_TRANSFER_MODE_NUM},
>+	{NULL, 0, 0, 0}
>+};
>+
>+/* Internal functions */
>+
>+static int32_t
>+internal_parse_decimal(const char *str) {
>+	char *end = NULL;
>+	unsigned long num;
>+
>+	num = strtoul(str, &end, 10);
>+	if ((str[0] == '\0') || (end == NULL) || (*end != '\0'))
>+		return -1;
>+
>+	return num;
>+}
>+
>+/* Global functions */
>+
>+void __rte_experimental
>+rte_eventmode_helper_print_options_list(void)
>+{
>+	fprintf(stderr, " --"
>+		" [--transfer-mode MODE]"
>+		);
>+}
>+
>+void __rte_experimental
>+rte_eventmode_helper_print_options_description(void)
>+{
>+	fprintf(stderr,
>+		"  --transfer-mode MODE\n"
>+		"               0: Packet transfer via polling (default)\n"
>+		"               1: Packet transfer via eventdev\n"
>+		"\n");
>+}
>+

Instead of exposing rte_eventmode_helper_print_options_* , we can maintain a page where all event_helper options are mentioned. Also application usage function can be updated like

        	fprintf(stderr, "%s [EAL options] --"
              " -p PORTMASK"
              " [-q NQ]",
	" -- [event helper options]"
              prgname);

 Advantages: Both functions will be removed from set of APIs and usage function will not be changed much. Suggesting the same methodology as used for EAL options.

>+static int
>+em_parse_transfer_mode(struct rte_eventmode_helper_conf *conf,
>+		const char *optarg)
>+{
>+	int32_t parsed_dec;
>+
>+	parsed_dec = internal_parse_decimal(optarg);
>+	if (parsed_dec !=
>RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL &&
>+	    parsed_dec !=
>RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT) {
>+		RTE_EM_HLPR_LOG_ERR("Unsupported packet transfer
>mode");
>+		return -1;
>+	}
>+	conf->mode = parsed_dec;
>+	return 0;
>+}
>+
>+static void
>+em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf) {
>+	/* Set default conf */
>+
>+	/* Packet transfer mode: poll */
>+	conf->mode =
>RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL;
>+}
>+
>+struct rte_eventmode_helper_conf *
>+rte_eventmode_helper_parse_args(int argc, char **argv) {
>+	int32_t opt, ret;
>+	struct rte_eventmode_helper_conf *conf = NULL;
>+
>+	/* Allocate memory for conf */
>+	conf = rte_zmalloc("eventmode-helper-conf",
>+			sizeof(struct rte_eventmode_helper_conf),
>+			RTE_CACHE_LINE_SIZE);
>+	if (conf == NULL) {
>+		RTE_EM_HLPR_LOG_ERR(
>+			"Failed allocating memory for eventmode helper
>conf");
>+			goto err;
>+	}
>+


Memory allocation for conf and conf->mode_params can be done in single alloc operation as given below:

	size = sizeof(struct rte_eventmode_helper_conf) + sizeof(struct eventmode_conf);
	conf = malloc(size);
	conf->mode_params = conf + 1;
	
Advantages: one NULL check will be avoided. To release the memory, need to free one pointer only. line of source code will be reduced.

>+	/* Initialize conf with default values */
>+	em_initialize_helper_conf(conf);
>+
>+	while ((opt = getopt_long(argc, argv, short_options,
>+				lgopts, NULL)) != EOF) {
>+		switch (opt) {
>+
>+		/* Packet transfer mode */
>+		case CMD_LINE_OPT_TRANSFER_MODE_NUM:
>+			ret = em_parse_transfer_mode(conf, optarg);
>+			if (ret < 0) {
>+				RTE_EM_HLPR_LOG_ERR(
>+					"Invalid packet transfer mode");
>+				goto err;
>+			}
>+			break;
>+		default:
>+			goto err;
>+		}
>+	}
>+	return conf;
>+
>+err:
>+	if (conf != NULL)
>+		rte_free(conf);
>+
>+	return NULL;
>+}
>diff --git a/lib/librte_eventdev/rte_eventmode_helper.h
>b/lib/librte_eventdev/rte_eventmode_helper.h
>index d32cd00..2a0cb30 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper.h
>+++ b/lib/librte_eventdev/rte_eventmode_helper.h
>@@ -8,6 +8,57 @@
> extern "C" {
> #endif
>
>+#include <rte_compat.h>
>+
>+/* Packet transfer mode of the application */ enum
>+rte_eventmode_helper_pkt_transfer_mode {
>+	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL = 0,
>+	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT,
>+};
>+
>+struct rte_eventmode_helper_conf {
>+	enum rte_eventmode_helper_pkt_transfer_mode mode;
>+		/**< Packet transfer mode of the application */
>+	void *mode_params;
>+		/**< Mode specific parameters */
>+};
>+
>+/* Common helper functions for command line parsing */
>+
>+/**
>+ * Print event mode options list
>+ *
>+ */
>+void __rte_experimental
>+rte_eventmode_helper_print_options_list(void);
>+
>+/**
>+ * Print event mode options description
>+ *
>+ */
>+void __rte_experimental
>+rte_eventmode_helper_print_options_description(void);
>+
>+/**
>+ * Parse event mode arguments
>+ *
>+ * The application can call this function in it's argument parsing
>+routine to
>+ * parse the event mode specific args and create the conf accordingly.
>+This
>+ * function is to be executed on the MASTER lcore only.
>+ *
>+ * @param argc
>+ *   A non-negative value. If it is greater than 0, the array members
>+ *   for argv[0] through argv[argc] (non-inclusive) shall contain pointers
>+ *   to strings.
>+ * @param argv
>+ *   An array of strings. The contents of the array, as well as the strings
>+ *   which are pointed to by the array, may be modified by this function.
>+ * @return
>+ *   Configuration generated by parsing the event mode args.
>+ */
>+struct rte_eventmode_helper_conf *
>+rte_eventmode_helper_parse_args(int argc, char **argv);
>+
> #ifdef __cplusplus
> }
> #endif
>--
>2.7.4

Re: [dpdk-dev] [EXT] [PATCH 19/39] eventdev: add common initialize routine for eventmode devs

[Sunil Kumar Kori]

Regards
Sunil Kumar Kori

>-----Original Message-----
>From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
>Sent: Monday, June 3, 2019 11:02 PM
>To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
><nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
>Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
><bruce.richardson@intel.com>; Pablo de Lara
><pablo.de.lara.guarch@intel.com>
>Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
><pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
><lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
><pbhagavatula@marvell.com>; Hemant Agrawal
><hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
>van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
><mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
>Subject: [EXT] [dpdk-dev] [PATCH 19/39] eventdev: add common initialize
>routine for eventmode devs
>
>External Email
>
>----------------------------------------------------------------------
>Adding framework for common initialization routine for event mode.
>Event mode would involve initialization of multiple devices, like eventdev,
>ethdev etc and this routine would be the placeholder for all initialization to
>come in.
>
>Signed-off-by: Anoob Joseph <anoobj@marvell.com>
>Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
>---
> lib/librte_eventdev/rte_eventdev_version.map |  2 ++
> lib/librte_eventdev/rte_eventmode_helper.c   | 51
>+++++++++++++++++++++++++++-
> lib/librte_eventdev/rte_eventmode_helper.h   | 26 +++++++++++++-
> 3 files changed, 77 insertions(+), 2 deletions(-)
>
>diff --git a/lib/librte_eventdev/rte_eventdev_version.map
>b/lib/librte_eventdev/rte_eventdev_version.map
>index 1199064..e156fa0 100644
>--- a/lib/librte_eventdev/rte_eventdev_version.map
>+++ b/lib/librte_eventdev/rte_eventdev_version.map
>@@ -130,4 +130,6 @@ EXPERIMENTAL {
> 	rte_event_eth_rx_adapter_stats_get;
> 	rte_eventmode_helper_print_options_list;
> 	rte_eventmode_helper_print_options_description;
>+	rte_eventmode_helper_parse_args;
>+	rte_eventmode_helper_initialize_devs;
> };
>diff --git a/lib/librte_eventdev/rte_eventmode_helper.c
>b/lib/librte_eventdev/rte_eventmode_helper.c
>index 38f1a2b..d1a569b 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper.c
>+++ b/lib/librte_eventdev/rte_eventmode_helper.c
>@@ -3,6 +3,7 @@
>  */
> #include <getopt.h>
>
>+#include <rte_ethdev.h>
> #include <rte_eventmode_helper.h>
> #include <rte_malloc.h>
>
>@@ -92,7 +93,7 @@ em_initialize_helper_conf(struct
>rte_eventmode_helper_conf *conf)
> 	conf->eth_portmask = -1;
> }
>
>-struct rte_eventmode_helper_conf *
>+struct rte_eventmode_helper_conf * __rte_experimental
> rte_eventmode_helper_parse_args(int argc, char **argv)  {
> 	int32_t opt, ret;
>@@ -152,3 +153,51 @@ rte_eventmode_helper_parse_args(int argc, char
>**argv)
>
> 	return NULL;
> }
>+
>+int32_t __rte_experimental
>+rte_eventmode_helper_initialize_devs(
>+		struct rte_eventmode_helper_conf *mode_conf) {
>+	int ret;
>+	uint16_t portid;
>+
>+	if (mode_conf == NULL) {
>+		RTE_EM_HLPR_LOG_ERR("Invalid conf");
>+		return -1;
>+	}
>+
>+	if (mode_conf->mode !=
>RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT)
>+		return 0;
>+
>+	if (mode_conf->mode_params == NULL) {
>+		RTE_EM_HLPR_LOG_ERR("Invalid mode params");
>+		return -1;
Better to return standard error number like EINVAL etc instead of having -1 or 0.
Consider it for all applicable places.

>+	}
>+
>+	/* Stop eth devices before setting up adapter */
>+	RTE_ETH_FOREACH_DEV(portid) {
>+
>+		/* Use only the ports enabled */
>+		if ((mode_conf->eth_portmask & (1 << portid)) == 0)
>+			continue;
>+
>+		rte_eth_dev_stop(portid);
>+	}
>+
>+	/* Start eth devices after setting up adapter */
>+	RTE_ETH_FOREACH_DEV(portid) {
>+
>+		/* Use only the ports enabled */
>+		if ((mode_conf->eth_portmask & (1 << portid)) == 0)
>+			continue;
>+
>+		ret = rte_eth_dev_start(portid);
>+		if (ret < 0) {
>+			RTE_EM_HLPR_LOG_ERR(
>+				"Error starting eth dev %d", portid);
>+			return -1;
>+		}
>+	}
>+
>+	return 0;
>+}
>diff --git a/lib/librte_eventdev/rte_eventmode_helper.h
>b/lib/librte_eventdev/rte_eventmode_helper.h
>index 77e69d0..0eafed3 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper.h
>+++ b/lib/librte_eventdev/rte_eventmode_helper.h
>@@ -62,9 +62,33 @@
>rte_eventmode_helper_print_options_description(void);
>  * @return
>  *   Configuration generated by parsing the event mode args.
>  */
>-struct rte_eventmode_helper_conf *
>+struct rte_eventmode_helper_conf * __rte_experimental
> rte_eventmode_helper_parse_args(int argc, char **argv);
>
>+/* Helper functions for initialization, & launching workers */
>+
>+/**
>+ * Initialize event mode devices
>+ *
>+ * Application could call this function to get the event device, eth
>+device
>+ * and eth rx adapter initialized according to the conf populated using
>+the
>+ * command line args.
>+ *
>+ * Application is expected to initialize the eth device and then the
>+eventmode
>+ * helper subsystem will stop & start eth device according to it's requirement.
>+ * So call to this function should be done after the eth device is
>+successfully
>+ * initialized.
Comments should be updated for its default value of conf->eth_portmask.

>+ *
>+ * @param mode_conf
>+ *   Configuration of the mode in which app is doing packet handling
>+ * @return
>+ *  - 0 on success.
>+ *  - (<0) on failure.
>+ */
>+int32_t __rte_experimental
>+rte_eventmode_helper_initialize_devs(
>+		struct rte_eventmode_helper_conf *mode_conf);
>+
> #ifdef __cplusplus
> }
> #endif
>--
>2.7.4

Re: [dpdk-dev] [EXT] [PATCH 32/39] eventdev: add routine to launch eventmode workers

[Sunil Kumar Kori]

Regards
Sunil Kumar Kori

>-----Original Message-----
>From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
>Sent: Monday, June 3, 2019 11:03 PM
>To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
><nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
>Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
><bruce.richardson@intel.com>; Pablo de Lara
><pablo.de.lara.guarch@intel.com>
>Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
><pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
><lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
><pbhagavatula@marvell.com>; Hemant Agrawal
><hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
>van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
><mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
>Subject: [EXT] [dpdk-dev] [PATCH 32/39] eventdev: add routine to launch
>eventmode workers
>
>External Email
>
>----------------------------------------------------------------------
>With eventmode, workers could be drafted differently according to the
>capabilities of the underlying event device. The added function would receive
>an array of such workers and probes the eventmode properties to choose the
>worker.
>
>Signed-off-by: Anoob Joseph <anoobj@marvell.com>
>Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
>---
> lib/librte_eventdev/rte_eventdev_version.map       |   1 +
> lib/librte_eventdev/rte_eventmode_helper.c         | 240
>+++++++++++++++++++++
> lib/librte_eventdev/rte_eventmode_helper.h         |  49 +++++
> .../rte_eventmode_helper_internal.h                |   3 +
> 4 files changed, 293 insertions(+)
>
>diff --git a/lib/librte_eventdev/rte_eventdev_version.map
>b/lib/librte_eventdev/rte_eventdev_version.map
>index 3cf926a..665836e 100644
>--- a/lib/librte_eventdev/rte_eventdev_version.map
>+++ b/lib/librte_eventdev/rte_eventdev_version.map
>@@ -135,4 +135,5 @@ EXPERIMENTAL {
> 	rte_eventmode_helper_display_conf;
> 	rte_eventmode_helper_get_event_lcore_links;
> 	rte_eventmode_helper_get_tx_queue;
>+	rte_eventmode_helper_launch_worker;
> };
>diff --git a/lib/librte_eventdev/rte_eventmode_helper.c
>b/lib/librte_eventdev/rte_eventmode_helper.c
>index e7670e0..77a5a4e 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper.c
>+++ b/lib/librte_eventdev/rte_eventmode_helper.c
>@@ -2,6 +2,7 @@
>  * Copyright (C) 2019 Marvell International Ltd.
>  */
> #include <getopt.h>
>+#include <stdbool.h>
>
> #include <rte_ethdev.h>
> #include <rte_eventdev.h>
>@@ -13,6 +14,8 @@
>
> #define CMD_LINE_OPT_TRANSFER_MODE	"transfer-mode"
>
>+static volatile bool eth_core_running;
>+
> static const char short_options[] =
> 	""
> 	;
>@@ -111,6 +114,16 @@ internal_get_eventdev_params(struct
>eventmode_conf *em_conf,
> 	return &(em_conf->eventdev_config[i]);  }
>
>+static inline bool
>+internal_dev_has_burst_mode(uint8_t dev_id) {
>+	struct rte_event_dev_info dev_info;
>+
>+	rte_event_dev_info_get(dev_id, &dev_info);
>+	return (dev_info.event_dev_cap &
>RTE_EVENT_DEV_CAP_BURST_MODE) ?
>+			true : false;
>+}
>+
> /* Global functions */
>
> void __rte_experimental
>@@ -980,3 +993,230 @@ rte_eventmode_helper_get_tx_queue(struct
>rte_eventmode_helper_conf *mode_conf,
> 	return eventdev_config->nb_eventqueue - 1;  }
>
>+/* Helper functions for launching workers */
>+
>+static int32_t
>+rte_eventmode_helper_start_worker_eth_core(struct eventmode_conf
>*em_conf,
>+		uint32_t lcore_id)

Internal functions must not be prefixed with "rte_",

>+{
>+	uint32_t service_id[EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE];
>+	struct rx_adapter_conf *rx_adapter;
>+	int service_count = 0;
>+	int adapter_id;
>+	int32_t ret;
>+	int i;
>+
>+	RTE_EM_HLPR_LOG_INFO(
>+		"Entering eth_core processing on lcore %u", lcore_id);
>+
>+	/*
>+	 * Need to parse adapter conf to see which of all Rx adapters need
>+	 * to be handled by this core.
>+	 */
>+	for (i = 0; i < em_conf->nb_rx_adapter; i++) {
>+		/* Check if we have exceeded the max allowed */
>+		if (service_count >
>EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE) {
>+			RTE_EM_HLPR_LOG_ERR(
>+				"Exceeded the max allowed adapters per rx
>core");
>+			break;
>+		}
>+
>+		rx_adapter = &(em_conf->rx_adapter[i]);
>+		if (rx_adapter->rx_core_id != lcore_id)
>+			continue;
>+
>+		/* Adapter need to be handled by this core */
>+		adapter_id = rx_adapter->adapter_id;
>+
>+		/* Get the service ID for the adapters */
>+		ret = rte_event_eth_rx_adapter_service_id_get(adapter_id,
>+				&(service_id[service_count]));
>+
>+		if (ret != -ESRCH && ret != 0) {
>+			RTE_EM_HLPR_LOG_ERR(
>+				"Error getting service ID used by Rx adapter");
>+			return ret;
>+		}
>+
>+		/* Update service count */
>+		service_count++;
>+	}
>+
>+	eth_core_running = true;
>+
>+	while (eth_core_running) {
>+		for (i = 0; i < service_count; i++) {
>+			/* Initiate adapter service */
>+			rte_service_run_iter_on_app_lcore(service_id[i], 0);
>+		}
>+	}
>+
>+	return 0;
>+}
>+
>+static int32_t
>+rte_eventmode_helper_stop_worker_eth_core(void)
>+{
>+	if (eth_core_running) {
>+		RTE_EM_HLPR_LOG_INFO("Stopping rx cores\n");
>+		eth_core_running = false;
>+	}
>+	return 0;
>+}
>+
>+static struct rte_eventmode_helper_app_worker_params *
>+rte_eventmode_helper_find_worker(uint32_t lcore_id,
>+		struct eventmode_conf *em_conf,
>+		struct rte_eventmode_helper_app_worker_params
>*app_wrkrs,
>+		uint8_t nb_wrkr_param)
>+{
>+	struct rte_eventmode_helper_event_link_info *link = NULL;
>+	uint8_t eventdev_id;
>+	struct eventdev_params *eventdev_config;
>+	int i;
>+	struct rte_eventmode_helper_app_worker_params curr_conf = {
>+			{{0} }, NULL};
>+	struct rte_eventmode_helper_app_worker_params *tmp_wrkr;
>+
>+	/*
>+	 * Event device to be used will be derived from the first lcore-event
>+	 * link.
>+	 *
>+	 * Assumption: All lcore-event links tied to a core would be using the
>+	 * same event device. in other words, one core would be polling on
>+	 * queues of a single event device only.
>+	 */
>+
>+	/* Get a link for this lcore */
>+	for (i = 0; i < em_conf->nb_link; i++) {
>+		link = &(em_conf->link[i]);
>+		if (link->lcore_id == lcore_id)
>+			break;
>+	}
>+
>+	if (link == NULL) {
>+		RTE_EM_HLPR_LOG_ERR(
>+			"No valid link found for lcore(%d)", lcore_id);
>+		return NULL;
>+	}
>+
>+	/* Get event dev ID */
>+	eventdev_id = link->eventdev_id;
>+
>+	/* Get the corresponding eventdev config */
>+	eventdev_config = internal_get_eventdev_params(em_conf,
>eventdev_id);
>+
>+	/* Populate the curr_conf with the capabilities */
>+
>+	/* Check for burst mode */
>+	if (internal_dev_has_burst_mode(eventdev_id))
>+		curr_conf.cap.burst =
>RTE_EVENTMODE_HELPER_RX_TYPE_BURST;
>+	else
>+		curr_conf.cap.burst =
>RTE_EVENTMODE_HELPER_RX_TYPE_NON_BURST;
>+
>+	/* Now parse the passed list and see if we have matching capabilities
>+*/
>+
>+	/* Initialize the pointer used to traverse the list */
>+	tmp_wrkr = app_wrkrs;
>+
>+	for (i = 0; i < nb_wrkr_param; i++, tmp_wrkr++) {
>+
>+		/* Skip this if capabilities are not matching */
>+		if (tmp_wrkr->cap.u64 != curr_conf.cap.u64)
>+			continue;
>+
>+		/* If the checks pass, we have a match */
>+		return tmp_wrkr;
>+	}
>+
>+	/* TODO required for ATQ */
>+	RTE_SET_USED(eventdev_config);
>+
>+	return NULL;
>+}
>+
>+static int
>+rte_eventmode_helper_verify_match_worker(
>+	struct rte_eventmode_helper_app_worker_params *match_wrkr) {
>+	/* Verify registered worker */
>+	if (match_wrkr->worker_thread == NULL) {
>+		RTE_EM_HLPR_LOG_ERR("No worker registered for second
>stage");
>+		return 0;
>+	}
>+
>+	/* Success */
>+	return 1;
>+}
>+
>+void __rte_experimental
>+rte_eventmode_helper_launch_worker(struct rte_eventmode_helper_conf
>*mode_conf,
>+		struct rte_eventmode_helper_app_worker_params
>*app_wrkr,
>+		uint8_t nb_wrkr_param)
>+{
>+	struct rte_eventmode_helper_app_worker_params *match_wrkr;
>+	uint32_t lcore_id;
>+	struct eventmode_conf *em_conf;
>+
>+	if (mode_conf == NULL) {
>+		RTE_EM_HLPR_LOG_ERR("Invalid conf");
>+		return;
>+	}
>+
>+	if (mode_conf->mode_params == NULL) {
>+		RTE_EM_HLPR_LOG_ERR("Invalid mode params");
>+		return;
>+	}
>+
>+	/* Get eventmode conf */
>+	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
>+
>+	/* Get core ID */
>+	lcore_id = rte_lcore_id();
>+
>+	/* TODO check capability for rx core */
>+
>+	/* Check if this is rx core */
>+	if (em_conf->eth_core_mask & (1 << lcore_id)) {
>+		rte_eventmode_helper_start_worker_eth_core(em_conf,
>lcore_id);
>+		return;
>+	}
>+
>+	if (app_wrkr == NULL || nb_wrkr_param == 0) {
>+		RTE_EM_HLPR_LOG_ERR("Invalid args");
>+		return;
>+	}
>+
>+	/*
>+	 * This is a regular worker thread. The application would be
>+	 * registering multiple workers with various capabilities. The
>+	 * worker to be run will be selected by the capabilities of the
>+	 * event device configured.
>+	 */
>+
>+	/* Get the first matching worker for the event device */
>+	match_wrkr = rte_eventmode_helper_find_worker(lcore_id,
>+			em_conf,
>+			app_wrkr,
>+			nb_wrkr_param);
>+
>+	if (match_wrkr == NULL) {
>+		RTE_EM_HLPR_LOG_ERR(
>+			"No matching worker registered for lcore %d",
>lcore_id);
>+		goto clean_and_exit;
>+	}
>+
>+	/* Verify sanity of the matched worker */
>+	if (rte_eventmode_helper_verify_match_worker(match_wrkr) != 1) {
>+		RTE_EM_HLPR_LOG_ERR("Error in validating the matched
>worker");
>+		goto clean_and_exit;
>+	}
>+
>+	/* Launch the worker thread */
>+	match_wrkr->worker_thread(mode_conf);
>+
>+clean_and_exit:
>+
>+	/* Flag eth_cores to stop, if started */
>+	rte_eventmode_helper_stop_worker_eth_core();
>+}
>diff --git a/lib/librte_eventdev/rte_eventmode_helper.h
>b/lib/librte_eventdev/rte_eventmode_helper.h
>index cd6d708..1235ca4 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper.h
>+++ b/lib/librte_eventdev/rte_eventmode_helper.h
>@@ -17,6 +17,20 @@ enum rte_eventmode_helper_pkt_transfer_mode {
> 	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT,
> };
>
>+/* Event mode packet rx types */
>+enum rte_eventmode_helper_rx_types {
>+	RTE_EVENTMODE_HELPER_RX_TYPE_INVALID = 0,
>+	RTE_EVENTMODE_HELPER_RX_TYPE_NON_BURST,
>+	RTE_EVENTMODE_HELPER_RX_TYPE_BURST,
>+	RTE_EVENTMODE_HELPER_RX_TYPE_MAX = 16
>+};
>+
>+/* Event mode packet tx types */
>+enum rte_eventmode_helper_tx_types {
>+	RTE_EVETNMODE_HELPER_TX_TYPE_INVALID = 0,
>+	RTE_EVENTMODE_HELPER_TX_TYPE_MAX = 16
>+};
>+
> struct rte_eventmode_helper_conf {
> 	enum rte_eventmode_helper_pkt_transfer_mode mode;
> 		/**< Packet transfer mode of the application */ @@ -41,6
>+55,20 @@ struct rte_eventmode_helper_event_link_info {
> 		/**< Lcore to be polling on this port */  };
>
>+/* Workers registered by the application */ struct
>+rte_eventmode_helper_app_worker_params {
>+	union {
>+		struct {
>+			uint64_t burst : 4;
>+			/**< Specify status of rx type burst */
>+		};
>+		uint64_t u64;
>+	} cap;
>+			/**< Capabilities of this worker */
>+	void (*worker_thread)(void *mode_conf);
>+			/**< Worker thread */
>+};
>+
> /* Common helper functions for command line parsing */
>
> /**
>@@ -157,6 +185,27 @@ uint8_t __rte_experimental
>rte_eventmode_helper_get_tx_queue(struct rte_eventmode_helper_conf
>*mode_conf,
> 		uint8_t eventdev_id);
>
>+/**
>+ * Launch eventmode worker
>+ *
>+ * The application can request the eventmode helper subsystem to launch
>+the
>+ * worker based on the capabilities of event device and the options
>+selected
>+ * while initializing the eventmode.
>+ *
>+ * @param mode_conf
>+ *   Configuration of the mode in which app is doing packet handling
>+ * @param app_wrkr
>+ *   List of all the workers registered by application, along with it's
>+ *   capabilities
>+ * @param nb_wrkr_param
>+ *   Number of workers passed by the application
>+ *
>+ */
>+void __rte_experimental
>+rte_eventmode_helper_launch_worker(struct rte_eventmode_helper_conf
>*mode_conf,
>+		struct rte_eventmode_helper_app_worker_params
>*app_wrkr,
>+		uint8_t nb_wrkr_param);
>+
> #ifdef __cplusplus
> }
> #endif
>diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h
>b/lib/librte_eventdev/rte_eventmode_helper_internal.h
>index 499cf5d..906766c 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
>+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
>@@ -48,6 +48,9 @@
> #define EVENT_MODE_MAX_LCORE_LINKS \
> 	(EVENT_MODE_MAX_EVENT_DEVS *
>EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV)
>
>+/* Max adapters that one Rx core can handle */ #define
>+EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE
>EVENT_MODE_MAX_RX_ADAPTERS
>+
> /* Event dev params */
> struct eventdev_params {
> 	uint8_t eventdev_id;
>--
>2.7.4

Re: [dpdk-dev] [EXT] [PATCH 33/39] eventdev: add Tx adapter support

[Sunil Kumar Kori]

Regards
Sunil Kumar Kori

>-----Original Message-----
>From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
>Sent: Monday, June 3, 2019 11:03 PM
>To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
><nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
>Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
><bruce.richardson@intel.com>; Pablo de Lara
><pablo.de.lara.guarch@intel.com>
>Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
><pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
><lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
><pbhagavatula@marvell.com>; Hemant Agrawal
><hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
>van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
><mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
>Subject: [EXT] [dpdk-dev] [PATCH 33/39] eventdev: add Tx adapter support
>
>External Email
>
>----------------------------------------------------------------------
>Adding support for Tx adapter.
>
>Signed-off-by: Anoob Joseph <anoobj@marvell.com>
>Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
>---
> lib/librte_eventdev/rte_eventmode_helper.c         | 349
>++++++++++++++++++++-
> lib/librte_eventdev/rte_eventmode_helper.h         |  24 ++
> .../rte_eventmode_helper_internal.h                |  30 ++
> lib/librte_eventdev/rte_eventmode_helper_prints.c  |   9 +
> 4 files changed, 399 insertions(+), 13 deletions(-)
>
>diff --git a/lib/librte_eventdev/rte_eventmode_helper.c
>b/lib/librte_eventdev/rte_eventmode_helper.c
>index 77a5a4e..f237cab 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper.c
>+++ b/lib/librte_eventdev/rte_eventmode_helper.c
>@@ -8,6 +8,7 @@
> #include <rte_eventdev.h>
> #include <rte_eventmode_helper.h>
> #include <rte_event_eth_rx_adapter.h>
>+#include <rte_event_eth_tx_adapter.h>
> #include <rte_malloc.h>
>
> #include "rte_eventmode_helper_internal.h"
>@@ -50,25 +51,51 @@ internal_parse_decimal(const char *str)
> 	return num;
> }
>
>+static int
>+internal_get_enabled_cores(unsigned int core_mask) {
>+	int i;
>+	int count = 0;
>+
>+	RTE_LCORE_FOREACH(i) {
>+		/* Check if this core is enabled in core_mask*/
>+		if (core_mask & (1 << i)) {
>+			/* We have enabled core */
>+			count++;
>+		}
>+	}
>+	return count;
>+}
>+
> static inline unsigned int
>-internal_get_next_rx_core(struct eventmode_conf *em_conf,
>-		unsigned int prev_core)
>+internal_get_next_eth_core(struct eventmode_conf *em_conf)
> {
> 	unsigned int next_core;
>+	static unsigned int prev_core = -1;
>+
>+	/*
>+	 * Make sure we have atleast one eth core running, else the following
>+	 * logic would lead to an infinite loop.
>+	 */
>+	if (internal_get_enabled_cores(em_conf->eth_core_mask) == 0) {
>+		RTE_EM_HLPR_LOG_INFO("No enabled eth core found");
>+		return RTE_MAX_LCORE;
>+	}
>
> get_next_core:
> 	/* Get the next core */
>-	next_core = rte_get_next_lcore(prev_core, 0, 0);
>+	next_core = rte_get_next_lcore(prev_core, 0, 1);
>
> 	/* Check if we have reached max lcores */
> 	if (next_core == RTE_MAX_LCORE)
> 		return next_core;
>
>+	/* Update prev_core */
>+	prev_core = next_core;
>+
> 	/* Only some cores would be marked as rx cores. Skip others */
>-	if (!(em_conf->eth_core_mask & (1 << next_core))) {
>-		prev_core = next_core;
>+	if (!(em_conf->eth_core_mask & (1 << next_core)))
> 		goto get_next_core;
>-	}
>
> 	return next_core;
> }
>@@ -164,7 +191,7 @@ static void
> em_initialize_helper_conf(struct rte_eventmode_helper_conf *conf)  {
> 	struct eventmode_conf *em_conf = NULL;
>-	unsigned int rx_core_id;
>+	unsigned int eth_core_id;
>
> 	/* Set default conf */
>
>@@ -180,12 +207,21 @@ em_initialize_helper_conf(struct
>rte_eventmode_helper_conf *conf)
> 	/* Schedule type: ordered */
> 	/* FIXME */
> 	em_conf->ext_params.sched_type = RTE_SCHED_TYPE_ORDERED;
>-	/* Set rx core. Use first core other than master core as Rx core */
>-	rx_core_id = rte_get_next_lcore(0, /* curr core */
>-					1, /* skip master core */
>-					0  /* wrap */);
>+	/* Set two cores as eth cores for Rx & Tx */
>+
>+	/* Use first core other than master core as Rx core */
>+	eth_core_id = rte_get_next_lcore(0,	/* curr core */
>+					 1,	/* skip master core */
>+					 0	/* wrap */);
>
>-	em_conf->eth_core_mask = (1 << rx_core_id);
>+	em_conf->eth_core_mask = (1 << eth_core_id);
>+
>+	/* Use next core as Tx core */
>+	eth_core_id = rte_get_next_lcore(eth_core_id,	/* curr core */
>+					 1,		/* skip master core */
>+					 0		/* wrap */);
>+
>+	em_conf->eth_core_mask |= (1 << eth_core_id);
> }
>
> struct rte_eventmode_helper_conf * __rte_experimental @@ -354,7 +390,7
>@@ rte_eventmode_helper_set_default_conf_rx_adapter(struct
>eventmode_conf *em_conf)
> 	/* Set adapter conf */
> 	adapter->eventdev_id = eventdev_id;
> 	adapter->adapter_id = adapter_id;
>-	adapter->rx_core_id = internal_get_next_rx_core(em_conf, -1);
>+	adapter->rx_core_id = internal_get_next_eth_core(em_conf);
>
> 	/*
> 	 * All queues of one eth device (port) will be mapped to one event
>@@ -403,6 +439,100 @@
>rte_eventmode_helper_set_default_conf_rx_adapter(struct eventmode_conf
>*em_conf)  }
>
> static int
>+rte_eventmode_helper_set_default_conf_tx_adapter(struct
>eventmode_conf
>+*em_conf) {
>+	int nb_eth_dev;
>+	int eventdev_id;
>+	int adapter_id;
>+	int i;
>+	int conn_id;
>+	struct eventdev_params *eventdev_config;
>+	struct tx_adapter_conf *tx_adapter;
>+	struct tx_adapter_connection_info *conn;
>+
>+	/*
>+	 * Create one Tx adapter with all eth queues mapped to event queues
>+	 * 1:1.
>+	 */
>+
>+	if (em_conf->nb_eventdev == 0) {
>+		RTE_EM_HLPR_LOG_ERR("No event devs registered");
>+		return -1;
>+	}
>+
>+	/* Get the number of eth devs */
>+	nb_eth_dev = rte_eth_dev_count_avail();
Why it is used ? Instead em_conf->eth_portmask cannot be used to get number of devices within the application ? 
Also if em_conf->eth_portmask can be used then rte_eth_dev_count_avail can be removed at
multiple places in file.

>+
>+	/* Use the first event dev */
>+	eventdev_config = &(em_conf->eventdev_config[0]);
>+
>+	/* Get eventdev ID */
>+	eventdev_id = eventdev_config->eventdev_id;
>+	adapter_id = 0;
>+
>+	/* Get adapter conf */
>+	tx_adapter = &(em_conf->tx_adapter[adapter_id]);
>+
>+	/* Set adapter conf */
>+	tx_adapter->eventdev_id = eventdev_id;
>+	tx_adapter->adapter_id = adapter_id;
>+
>+	/* TODO: Tx core is required only when internal port is not present */
>+
>+	tx_adapter->tx_core_id = internal_get_next_eth_core(em_conf);
>+
>+	/*
>+	 * Application would need to use one event queue per adapter for
>+	 * submitting packets for Tx. Reserving the last queue available
>+	 * and decrementing the total available event queues for this
>+	 */
>+
>+	/* Queue numbers start at 0 */
>+	tx_adapter->tx_ev_queue = eventdev_config->nb_eventqueue - 1;
>+
>+	/* Update the number of event queues available in eventdev */
>+	eventdev_config->nb_eventqueue--;
>+
>+	/*
>+	 * All Tx queues of the eth device (port) will be mapped to the event
>+	 * device.
>+	 */
>+
>+	/* Set defaults for connections */
>+
>+	/*
>+	 * One eth device (port) would be one connection. All Tx queues of
>+	 * the device would be mapped to the Tx adapter.
>+	 */
>+
>+	for (i = 0; i < nb_eth_dev; i++) {
>+
>+		/* Use only the ports enabled */
>+		if ((em_conf->eth_portmask & (1 << i)) == 0)
>+			continue;
>+
>+		/* Get the connection id */
>+		conn_id = tx_adapter->nb_connections;
>+
>+		/* Get the connection */
>+		conn = &(tx_adapter->conn[conn_id]);
>+
>+		/* Add ethdev to connections */
>+		conn->ethdev_id = i;
>+
>+		/* Add all eth tx queues to adapter */
>+		conn->ethdev_tx_qid = -1;
>+
>+		/* Update no of connections */
>+		tx_adapter->nb_connections++;
>+	}
>+
>+	/* We have setup one adapter */
>+	em_conf->nb_tx_adapter = 1;
>+	return 0;
>+}
>+
>+static int
> rte_eventmode_helper_set_default_conf_link(struct eventmode_conf
>*em_conf)  {
> 	int i, j;
>@@ -501,6 +631,16 @@ rte_eventmode_helper_validate_conf(struct
>eventmode_conf *em_conf)
> 	}
>
> 	/*
>+	 * See if tx adapters are specified. Else generate a default conf
>+	 * with one tx adapter.
>+	 */
>+	if (em_conf->nb_tx_adapter == 0) {
>+		ret =
>rte_eventmode_helper_set_default_conf_tx_adapter(em_conf);
>+		if (ret != 0)
>+			return ret;
>+	}
>+
>+	/*
> 	 * See if links are specified. Else generate a default conf for
> 	 * the event ports used.
> 	 */
>@@ -799,6 +939,150 @@ rte_eventmode_helper_initialize_rx_adapter(struct
>eventmode_conf *em_conf)
> 	return 0;
> }
>
>+static int
>+tx_adapter_configure(struct eventmode_conf *em_conf,
>+	struct tx_adapter_conf *adapter)
>+{
>+	int ret, j;
>+	uint8_t tx_port_id = 0;
>+	uint8_t eventdev_id;
>+	uint32_t service_id;
>+	struct rte_event_port_conf port_conf = {0};
>+	struct rte_event_dev_info evdev_default_conf = {0};
>+	struct tx_adapter_connection_info *conn;
>+	struct eventdev_params *eventdev_config;
>+
>+	/* Get event dev ID */
>+	eventdev_id = adapter->eventdev_id;
>+
>+	/* Get event device conf */
>+	eventdev_config = internal_get_eventdev_params(em_conf,
>eventdev_id);
>+
>+	/* Create Tx adapter */
>+
>+	/* Get default configuration of event dev */
>+	ret = rte_event_dev_info_get(eventdev_id, &evdev_default_conf);
>+	if (ret < 0) {
>+		RTE_EM_HLPR_LOG_ERR(
>+			"Error in getting event device info[devID:%d]",
>+			eventdev_id);
>+		return ret;
>+	}
>+
>+	/* Setup port conf */
>+	port_conf.new_event_threshold =
>+			evdev_default_conf.max_num_events;
>+	port_conf.dequeue_depth =
>+
>	evdev_default_conf.max_event_port_dequeue_depth;
>+	port_conf.enqueue_depth =
>+
>	evdev_default_conf.max_event_port_enqueue_depth;
>+
>+	/* Create Tx adapter */
>+	ret = rte_event_eth_tx_adapter_create(adapter->adapter_id,
>+			adapter->eventdev_id,
>+			&port_conf);
>+	if (ret < 0) {
>+		RTE_EM_HLPR_LOG_ERR("Error in Tx adapter creation");
>+		return ret;
>+	}
>+
>+	/* Setup various connections in the adapter */
>+	for (j = 0; j < adapter->nb_connections; j++) {
>+
>+		/* Get connection */
>+		conn = &(adapter->conn[j]);
>+
>+		/* Add queue to the adapter */
>+		ret = rte_event_eth_tx_adapter_queue_add(
>+				adapter->adapter_id,
>+				conn->ethdev_id,
>+				conn->ethdev_tx_qid);
>+		if (ret < 0) {
>+			RTE_EM_HLPR_LOG_ERR(
>+				"Error in adding eth queue in Tx adapter");
>+			return ret;
>+		}
>+	}
>+
>+	/* Get event port used by the adapter */
>+	ret = rte_event_eth_tx_adapter_event_port_get(
>+			adapter->adapter_id,
>+			&tx_port_id);
>+	if (ret) {
>+		RTE_EM_HLPR_LOG_ERR("Failed to get Tx adapter port ID");
>+		return ret;
>+	}
>+
>+	/*
>+	 * TODO: event queue for Tx adapter is required only if the
>+	 * INTERNAL PORT is not present.
>+	 */
>+
>+	/*
>+	 * Tx event queue would be reserved for Tx adapter. Need to unlink
>+	 * this queue from all other ports
>+	 *
>+	 */
>+	for (j = 0; j < eventdev_config->nb_eventport; j++) {
>+		rte_event_port_unlink(eventdev_id, j,
>+				      &(adapter->tx_ev_queue), 1);
>+	}
>+
>+	ret = rte_event_port_link(
>+			eventdev_id,
>+			tx_port_id,
>+			&(adapter->tx_ev_queue),
>+			NULL, 1);
>+	if (ret != 1) {
>+		RTE_EM_HLPR_LOG_ERR("Failed to link event queue to port");
>+		return ret;
>+	}
>+
>+	/* Get the service ID used by Tx adapter */
>+	ret = rte_event_eth_tx_adapter_service_id_get(adapter->adapter_id,
>+						      &service_id);
>+	if (ret != -ESRCH && ret != 0) {
>+		RTE_EM_HLPR_LOG_ERR(
>+			"Error getting service ID used by adapter");
>+		return ret;
>+	}
>+
>+	/*
>+	 * TODO
>+	 * Tx core will invoke the service when required. The runstate check
>+	 * is not required.
>+	 *
>+	 */
>+	rte_service_set_runstate_mapped_check(service_id, 0);
>+
>+	/* Start adapter */
>+	ret = rte_event_eth_tx_adapter_start(adapter->adapter_id);
>+	if (ret) {
>+		RTE_EM_HLPR_LOG_ERR("Error in starting Tx adapter");
>+		return ret;
>+	}
>+
>+	return 0;
>+}
>+
>+static int
>+rte_eventmode_helper_initialize_tx_adapter(struct eventmode_conf
>+*em_conf) {
>+	int i, ret;
>+	struct tx_adapter_conf *adapter;
>+
>+	/* Configure Tx adapters */
>+	for (i = 0; i < em_conf->nb_tx_adapter; i++) {
>+		adapter = &(em_conf->tx_adapter[i]);
>+		ret = tx_adapter_configure(em_conf, adapter);
>+		if (ret < 0) {
>+			RTE_EM_HLPR_LOG_ERR("Tx adapter configuration
>failed");
>+			return ret;
>+		}
>+	}
>+	return 0;
>+}
>+
> int32_t __rte_experimental
> rte_eventmode_helper_initialize_devs(
> 		struct rte_eventmode_helper_conf *mode_conf) @@ -861,6
>+1145,11 @@ rte_eventmode_helper_initialize_devs(
> 	if (ret != 0)
> 		return ret;
>
>+	/* Setup Tx adapter */
>+	ret = rte_eventmode_helper_initialize_tx_adapter(em_conf);
>+	if (ret != 0)
>+		return ret;
>+
> 	/* Start eth devices after setting up adapter */
> 	RTE_ETH_FOREACH_DEV(portid) {
>
>@@ -1001,6 +1290,7 @@
>rte_eventmode_helper_start_worker_eth_core(struct eventmode_conf
>*em_conf,  {
> 	uint32_t service_id[EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE];
> 	struct rx_adapter_conf *rx_adapter;
>+	struct tx_adapter_conf *tx_adapter;
> 	int service_count = 0;
> 	int adapter_id;
> 	int32_t ret;
>@@ -1042,6 +1332,39 @@
>rte_eventmode_helper_start_worker_eth_core(struct eventmode_conf
>*em_conf,
> 		service_count++;
> 	}
>
>+	/*
>+	 * Need to parse adapter conf to see which all Tx adapters need to be
>+	 * handled this core.
>+	 */
>+	for (i = 0; i < em_conf->nb_tx_adapter; i++) {
>+		/* Check if we have exceeded the max allowed */
>+		if (service_count >
>EVENT_MODE_MAX_ADAPTERS_PER_TX_CORE) {
>+			RTE_EM_HLPR_LOG_ERR(
>+				"Exceeded the max allowed adapters per Tx
>core");
>+			break;
>+		}
>+
>+		tx_adapter = &(em_conf->tx_adapter[i]);
>+		if (tx_adapter->tx_core_id != lcore_id)
>+			continue;
>+
>+		/* Adapter need to be handled by this core */
>+		adapter_id = tx_adapter->adapter_id;
>+
>+		/* Get the service ID for the adapters */
>+		ret = rte_event_eth_tx_adapter_service_id_get(adapter_id,
>+				&(service_id[service_count]));
>+
>+		if (ret != -ESRCH && ret != 0) {
>+			RTE_EM_HLPR_LOG_ERR(
>+				"Error getting service ID used by Tx adapter");
>+			return ret;
>+		}
>+
>+		/* Update service count */
>+		service_count++;
>+	}
>+
> 	eth_core_running = true;
>
> 	while (eth_core_running) {
>diff --git a/lib/librte_eventdev/rte_eventmode_helper.h
>b/lib/librte_eventdev/rte_eventmode_helper.h
>index 1235ca4..2212622 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper.h
>+++ b/lib/librte_eventdev/rte_eventmode_helper.h
>@@ -10,6 +10,14 @@ extern "C" {
>
> #include <rte_common.h>
> #include <rte_compat.h>
>+#include <rte_config.h>
>+#include <rte_event_eth_tx_adapter.h>
>+
>+/* Flag to indicate that the event device used by all adapters is same */
>+#define RTE_EM_HELPER_TX_EV_LINK_COMMON_EVENT_DEV	(1 <<
>0)
>+
>+/* Flag to indicate that the event queue to be used for all adapters is same */
>+#define RTE_EM_HELPER_TX_EV_LINK_COMMON_EVENT_QUEUE	(1 <<
>1)
>
> /* Packet transfer mode of the application */  enum
>rte_eventmode_helper_pkt_transfer_mode { @@ -55,6 +63,22 @@ struct
>rte_eventmode_helper_event_link_info {
> 		/**< Lcore to be polling on this port */  };
>
>+/*
>+ * Tx event queue - port link conf
>+ *
>+ * Application would need to know which event queue would correspond to
>+which
>+ * eth port, so that it can send out accordingly.
>+ */
>+struct rte_eventmode_helper_tx_ev_link_conf {
>+	uint8_t flags;
>+	struct {
>+		uint8_t eventdev_id;
>+			/**< Event device ID */
>+		uint8_t tx_ev_queue;
>+			/**< Tx event queue */
>+	} ports[RTE_MAX_ETHPORTS];
>+};
>+
> /* Workers registered by the application */  struct
>rte_eventmode_helper_app_worker_params {
> 	union {
>diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h
>b/lib/librte_eventdev/rte_eventmode_helper_internal.h
>index 906766c..1daca22 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
>+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
>@@ -38,9 +38,15 @@
> /* Max Rx adapters supported */
> #define EVENT_MODE_MAX_RX_ADAPTERS RTE_EVENT_MAX_DEVS
>
>+/* Max Tx adapters supported */
>+#define EVENT_MODE_MAX_TX_ADAPTERS RTE_EVENT_MAX_DEVS
>+
> /* Max Rx adapter connections */
> #define EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER 16
>
>+/* Max Tx adapter connections */
>+#define EVENT_MODE_MAX_CONNECTIONS_PER_TX_ADAPTER 16
>+
> /* Max event queues supported per event device */  #define
>EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV
>RTE_EVENT_MAX_QUEUES_PER_DEV
>
>@@ -51,6 +57,9 @@
> /* Max adapters that one Rx core can handle */  #define
>EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE
>EVENT_MODE_MAX_RX_ADAPTERS
>
>+/* Max adapters that one Tx core can handle */ #define
>+EVENT_MODE_MAX_ADAPTERS_PER_TX_CORE
>EVENT_MODE_MAX_TX_ADAPTERS
>+
> /* Event dev params */
> struct eventdev_params {
> 	uint8_t eventdev_id;
>@@ -76,6 +85,23 @@ struct rx_adapter_conf {
>
>	conn[EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER];
> };
>
>+/* Tx adapter connection info */
>+struct tx_adapter_connection_info {
>+	uint8_t ethdev_id;
>+	int32_t ethdev_tx_qid;
>+};
>+
>+/* Tx adapter conf */
>+struct tx_adapter_conf {
>+	int32_t eventdev_id;
>+	int32_t adapter_id;
>+	uint32_t tx_core_id;
>+	uint8_t nb_connections;
>+	struct tx_adapter_connection_info
>+
>	conn[EVENT_MODE_MAX_CONNECTIONS_PER_TX_ADAPTER];
>+	uint8_t tx_ev_queue;
>+};
>+
> /* Eventmode conf data */
> struct eventmode_conf {
> 	int nb_eventdev;
>@@ -86,6 +112,10 @@ struct eventmode_conf {
> 		/**< No of Rx adapters */
> 	struct rx_adapter_conf
>rx_adapter[EVENT_MODE_MAX_RX_ADAPTERS];
> 		/**< Rx adapter conf */
>+	uint8_t nb_tx_adapter;
>+		/**< No of Tx adapters */
>+	struct tx_adapter_conf
>tx_adapter[EVENT_MODE_MAX_TX_ADAPTERS];
>+		/** Tx adapter conf */
> 	uint8_t nb_link;
> 		/**< No of links */
> 	struct rte_eventmode_helper_event_link_info
>diff --git a/lib/librte_eventdev/rte_eventmode_helper_prints.c
>b/lib/librte_eventdev/rte_eventmode_helper_prints.c
>index 3114d29..387302a 100644
>--- a/lib/librte_eventdev/rte_eventmode_helper_prints.c
>+++ b/lib/librte_eventdev/rte_eventmode_helper_prints.c
>@@ -95,6 +95,12 @@ rte_eventmode_display_rx_adapter_conf(struct
>eventmode_conf *em_conf)  }
>
> static void
>+rte_eventmode_display_tx_adapter_conf(struct eventmode_conf *em_conf)
>{
>+	RTE_SET_USED(em_conf);
>+}
>+
>+static void
> rte_eventmode_display_link_conf(struct eventmode_conf *em_conf)  {
> 	int i;
>@@ -155,6 +161,9 @@ rte_eventmode_helper_display_conf(struct
>rte_eventmode_helper_conf *mode_conf)
> 	/* Display Rx adapter conf */
> 	rte_eventmode_display_rx_adapter_conf(em_conf);
>
>+	/* Display Tx adapter conf */
>+	rte_eventmode_display_tx_adapter_conf(em_conf);
>+
> 	/* Display event-lcore link */
> 	rte_eventmode_display_link_conf(em_conf);
> }
>--
>2.7.4

Re: [dpdk-dev] [PATCH 00/39] adding eventmode helper library

[Mattias Rönnblom]

On 2019-06-07 11:48, Jerin Jacob Kollanukkaran wrote:
>> -----Original Message-----
>> From: Anoob Joseph <anoobj@marvell.com>
>> Sent: Monday, June 3, 2019 11:02 PM
>> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
>> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
>> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
>> <bruce.richardson@intel.com>; Pablo de Lara
>> <pablo.de.lara.guarch@intel.com>
>> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
>> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
>> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
>> <pbhagavatula@marvell.com>; Hemant Agrawal
>> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
>> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
>> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
>> Subject: [PATCH 00/39] adding eventmode helper library
>>
>> This series adds support for eventmode helper library and l2fwd-event
>> application.
>>
>> First 13 patches creates a new l2fwd application (l2fwd-event). Minor code
>> reorganization is done to faciliate seamless integration of eventmode.
>>
>> Next 22 patches adds eventmode helper library. This library abstracts the
>> configuration of event device & Rx-Tx event adapters. The library can be
>> extended to allow users to control all the configuration exposed by adapters
>> and eth device.
>>
>> Last 4 patches implements eventmode in l2fwd-event application. With
>> event device and adapters, fine tuned threads (based on dev
>> capabilities) can be drafted to maximize performance. Eventmode library
>> facilitates this and l2fwd-event demonstrates this usage.
>>
>> With the introduction of eventmode helper library, any poll mode application
>> can be converted to an eventmode application with simple steps, enabling
>> multi-core scaling and dynamic load balancing to various example
>> applications.
> 
> 
> Anyone planning to review this changes?
> I will spend time to review this. Requesting the review from other eventdev stake holders.
> 

A more extensive description of the purpose of the eventmode helper 
library would be helpful.

Is this supposed to be a generic framework for real-world applications, 
or only something to simplify DPDK the implementation of DPDK example 
programs and similar?

Re: [dpdk-dev] [EXT] Re: [PATCH 00/39] adding eventmode helper library

[Anoob Joseph]

Hi Mattias,
 
> A more extensive description of the purpose of the eventmode helper 
> library would be helpful.
> 
> Is this supposed to be a generic framework for real-world 
> applications, or only something to simplify DPDK the implementation of 
> DPDK example programs and similar?
 
This is intended as a generic framework, but the initial targets would be limited to DPDK example applications.
 
For any application to use an event device for dynamic load balancing, it has to configure the event device and the adapters. Configuring the adapters would involve providing various parameters based on which the dynamic scheduling should happen. But requiring the application to do all this configuration would make the application complicated as well as the same code has to be repeated for a new application. Event mode helper tries to solve that.
 
All the complex configuration would be implemented by the helper library and the helper library would provide a default conf as well. 
 
These patches facilitate event mode configuration in a easy to use manner. My idea is that, for a poll mode DPDK example to operate in event mode, a couple of helper functions and a lean worker thread should suffice. So even complex DPDK examples and real world applications will benefit from this helper library. We plan to propose a change to ipsec-secgw to operate in event mode once this proposal is merged.
 
I'll update the cover-letter to add above details when sending v2.

Thanks,
Anoob

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Mattias Rönnblom
> Sent: Tuesday, June 11, 2019 4:14 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Anoob Joseph
> <anoobj@marvell.com>; Nikhil Rao <nikhil.rao@intel.com>; Erik Gabriel
> Carrillo <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Narayana Prasad Raju Athreya <pathreya@marvell.com>; dev@dpdk.org;
> Lukas Bartosik <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Liang Ma
> <liang.j.ma@intel.com>
> Subject: [EXT] Re: [dpdk-dev] [PATCH 00/39] adding eventmode helper
> library
> 
> External Email
> 
> ----------------------------------------------------------------------
> On 2019-06-07 11:48, Jerin Jacob Kollanukkaran wrote:
> >> -----Original Message-----
> >> From: Anoob Joseph <anoobj@marvell.com>
> >> Sent: Monday, June 3, 2019 11:02 PM
> >> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> >> <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> >> <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> >> <abhinandan.gujjar@intel.com>; Bruce Richardson
> >> <bruce.richardson@intel.com>; Pablo de Lara
> >> <pablo.de.lara.guarch@intel.com>
> >> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> >> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> >> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> >> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>;
> >> Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> >> <harry.van.haaren@intel.com>; Mattias Rönnblom
> >> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> >> Subject: [PATCH 00/39] adding eventmode helper library
> >>
> >> This series adds support for eventmode helper library and l2fwd-event
> >> application.
> >>
> >> First 13 patches creates a new l2fwd application (l2fwd-event). Minor
> >> code reorganization is done to faciliate seamless integration of
> eventmode.
> >>
> >> Next 22 patches adds eventmode helper library. This library abstracts
> >> the configuration of event device & Rx-Tx event adapters. The library
> >> can be extended to allow users to control all the configuration
> >> exposed by adapters and eth device.
> >>
> >> Last 4 patches implements eventmode in l2fwd-event application. With
> >> event device and adapters, fine tuned threads (based on dev
> >> capabilities) can be drafted to maximize performance. Eventmode
> >> library facilitates this and l2fwd-event demonstrates this usage.
> >>
> >> With the introduction of eventmode helper library, any poll mode
> >> application can be converted to an eventmode application with simple
> >> steps, enabling multi-core scaling and dynamic load balancing to
> >> various example applications.
> >
> >
> > Anyone planning to review this changes?
> > I will spend time to review this. Requesting the review from other
> eventdev stake holders.
> >
> 
> A more extensive description of the purpose of the eventmode helper
> library would be helpful.
> 
> Is this supposed to be a generic framework for real-world applications, or
> only something to simplify DPDK the implementation of DPDK example
> programs and similar?

Re: [dpdk-dev] [PATCH 15/39] eventdev: add routines for logging eventmode helper

[Anoob Joseph]

Hi Jerin,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran
> Sent: Monday, June 10, 2019 3:43 PM
> To: Anoob Joseph <anoobj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: RE: [PATCH 15/39] eventdev: add routines for logging eventmode
> helper
> 
> > -----Original Message-----
> > From: Anoob Joseph <anoobj@marvell.com>
> > Sent: Monday, June 3, 2019 11:02 PM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> > <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> > <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> > <abhinandan.gujjar@intel.com>; Bruce Richardson
> > <bruce.richardson@intel.com>; Pablo de Lara
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> > <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> > <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>;
> > Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> > <harry.van.haaren@intel.com>; Mattias Rönnblom
> > <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> > Subject: [PATCH 15/39] eventdev: add routines for logging eventmode
> > helper
> >
> > Adding routines for logging eventmode helper.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > ---
> >  config/common_base                                 |  1 +
> >  lib/librte_eal/common/eal_common_log.c             |  1 +
> >  lib/librte_eal/common/include/rte_log.h            |  1 +
> >  .../rte_eventmode_helper_internal.h                | 29
> > ++++++++++++++++++++++
> >  4 files changed, 32 insertions(+)
> >
> > diff --git a/config/common_base b/config/common_base index
> > 8576973..96c2537 100644
> > --- a/config/common_base
> > +++ b/config/common_base
> > @@ -685,6 +685,7 @@ CONFIG_RTE_LIBRTE_PMD_ZLIB=n  #
> > CONFIG_RTE_LIBRTE_EVENTDEV=y
> CONFIG_RTE_LIBRTE_EVENTDEV_DEBUG=n
> > +CONFIG_RTE_LIBRTE_EVENTMODE_HELPER_DEBUG=n
> >  CONFIG_RTE_EVENT_MAX_DEVS=16
> >  CONFIG_RTE_EVENT_MAX_QUEUES_PER_DEV=64
> >  CONFIG_RTE_EVENT_TIMER_ADAPTER_NUM_MAX=32
> > diff --git a/lib/librte_eal/common/eal_common_log.c
> > b/lib/librte_eal/common/eal_common_log.c
> > index c714a4b..bc34dc2 100644
> > --- a/lib/librte_eal/common/eal_common_log.c
> > +++ b/lib/librte_eal/common/eal_common_log.c
> > @@ -324,6 +324,7 @@ static const struct logtype logtype_strings[] = {
> >  	{RTE_LOGTYPE_EFD,        "lib.efd"},
> >  	{RTE_LOGTYPE_EVENTDEV,   "lib.eventdev"},
> >  	{RTE_LOGTYPE_GSO,        "lib.gso"},
> > +	{RTE_LOGTYPE_EVENTMODE,  "lib.eventmode"},
> 
> EAL log type, typically adds for subsystem. Please reuse EVENTDEV_DEBUG
> Or introduce dynamic debugging.

[Anoob] Will make this change in v2. In that case, this patch can be dropped.

Re: [dpdk-dev] [PATCH 16/39] eventdev: add eventmode CL options framework

[Anoob Joseph]

Hi Jerin,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran
> Sent: Monday, June 10, 2019 3:49 PM
> To: Anoob Joseph <anoobj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: RE: [PATCH 16/39] eventdev: add eventmode CL options framework
> 
> > -----Original Message-----
> > From: Anoob Joseph <anoobj@marvell.com>
> > Sent: Monday, June 3, 2019 11:02 PM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> > <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> > <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> > <abhinandan.gujjar@intel.com>; Bruce Richardson
> > <bruce.richardson@intel.com>; Pablo de Lara
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> > <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> > <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>;
> > Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> > <harry.van.haaren@intel.com>; Mattias Rönnblom
> > <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> > Subject: [PATCH 16/39] eventdev: add eventmode CL options framework
> >
> > Adding usage prints and CL parsing routines for eventmode. Option to
> > select packet transfer mode is also added.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > ---
> > +#include <rte_compat.h>
> > +
> > +/* Packet transfer mode of the application */ enum
> > +rte_eventmode_helper_pkt_transfer_mode {
> > +	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_POLL = 0,
> > +	RTE_EVENTMODE_HELPER_PKT_TRANSFER_MODE_EVENT,
> > +};
> 
> Need to mark all public structues as EXPERIMENTAL.
> Please grep for EXPERIMENTAL in libeventdev or any other library

[Anoob] Will fix in v2.

> 
> > +struct rte_eventmode_helper_conf {
> > +	enum rte_eventmode_helper_pkt_transfer_mode mode;
> > +		/**< Packet transfer mode of the application */
> > +	void *mode_params;
> > +		/**< Mode specific parameters */
> > +};
> 
> Please make this event_helper object as 'opaque'  and move to internal so
> that it does have ABI policies and future new features can be added.
> 

[Anoob] The member, "mode_params" is the opaque pointer which is used internally. There are two fields which need to be communicated between application and helper library.

1. mode: whether the application is running in event-mode or poll-mode. Both application and helper library would need this. This field is set by the helper while parsing the eventmode args.
2. eth_portmask: helper library needs this info from application to know which all ethernet ports it can use. This is determined by the application and has to be communicated to the helper library. This can be made as params passed to the application, but eth_portmask will be the only member in it, as of now.

With the above use case, do you recommend any different structure layout?

Re: [dpdk-dev] [PATCH 19/39] eventdev: add common initialize routine for eventmode devs

[Anoob Joseph]

Hi Jerin,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran
> Sent: Monday, June 10, 2019 3:54 PM
> To: Anoob Joseph <anoobj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: RE: [PATCH 19/39] eventdev: add common initialize routine for
> eventmode devs
> 
> > -----Original Message-----
> > From: Anoob Joseph <anoobj@marvell.com>
> > Sent: Monday, June 3, 2019 11:02 PM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> > <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> > <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> > <abhinandan.gujjar@intel.com>; Bruce Richardson
> > <bruce.richardson@intel.com>; Pablo de Lara
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> > <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> > <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>;
> > Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> > <harry.van.haaren@intel.com>; Mattias Rönnblom
> > <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> > Subject: [PATCH 19/39] eventdev: add common initialize routine for
> > eventmode devs
> >
> > Adding framework for common initialization routine for event mode.
> > Event mode would involve initialization of multiple devices, like
> > eventdev, ethdev etc and this routine would be the placeholder for all
> > initialization to come in.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > ---
> >
> > +/* Helper functions for initialization, & launching workers */
> > +
> > +/**
> > + * Initialize event mode devices
> > + *
> > + * Application could call this function to get the event device, eth
> > +device
> > + * and eth rx adapter initialized according to the conf populated
> > +using the
> > + * command line args.
> > + *
> > + * Application is expected to initialize the eth device and then the
> > +eventmode
> > + * helper subsystem will stop & start eth device according to it's
> > requirement.
> > + * So call to this function should be done after the eth device is
> > +successfully
> > + * initialized.
> > + *
> > + * @param mode_conf
> > + *   Configuration of the mode in which app is doing packet handling
> > + * @return
> > + *  - 0 on success.
> > + *  - (<0) on failure.
> > + */
> > +int32_t __rte_experimental
> > +rte_eventmode_helper_initialize_devs(
> > +		struct rte_eventmode_helper_conf *mode_conf);
> > +
> 
> # Prefer to change to rte_event_helper_init() and introduce the counter part

[Anoob] This routine is doing init of devs (not the helper library). So are you okay with renaming it to rte_event_helper_devs_init() and rte_event_helper_devs_uninit(), and retain the functionality as before?

> for the same(rte_event_helper_uninit() or rte_event_helper_fini()) #
> introduce params structure taking another paraments input instead of new
> APIs.
> # let library return rte_event_helper_conf* object for further operations.

[Anoob] Parse args will parse the command line arguments and return the conf. Based on the conf we setup the required devices (eventdev, Rx adapter & Tx adapter, can be expanded to handle ethdev also). 

> 
> 
> >  #ifdef __cplusplus
> >  }
> >  #endif
> > --
> > 2.7.4

Re: [dpdk-dev] [PATCH 25/39] eventdev: add routine to validate conf

[Anoob Joseph]

Hi Jerin,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran
> Sent: Monday, June 10, 2019 3:55 PM
> To: Anoob Joseph <anoobj@marvell.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Erik Gabriel Carrillo <erik.g.carrillo@intel.com>;
> Abhinandan Gujjar <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: RE: [PATCH 25/39] eventdev: add routine to validate conf
> 
> > -----Original Message-----
> > From: Anoob Joseph <anoobj@marvell.com>
> > Sent: Monday, June 3, 2019 11:02 PM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> > <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> > <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> > <abhinandan.gujjar@intel.com>; Bruce Richardson
> > <bruce.richardson@intel.com>; Pablo de Lara
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> > <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> > <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>;
> > Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> > <harry.van.haaren@intel.com>; Mattias Rönnblom
> > <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> > Subject: [PATCH 25/39] eventdev: add routine to validate conf
> >
> > Adding routine to validate event mode conf. This function will verify
> > the conf requested by the user and would populate other fields with
> default values.
> > Presently, the function acts as placeholder for the above mentioned
> actions.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > ---
> >  lib/librte_eventdev/rte_eventmode_helper.c | 33
> > ++++++++++++++++++++++++++++++
> >  1 file changed, 33 insertions(+)
> >
> > diff --git a/lib/librte_eventdev/rte_eventmode_helper.c
> > b/lib/librte_eventdev/rte_eventmode_helper.c
> > index a57f837..4dbb94a 100644
> > --- a/lib/librte_eventdev/rte_eventmode_helper.c
> > +++ b/lib/librte_eventdev/rte_eventmode_helper.c
> > @@ -165,6 +165,32 @@ rte_eventmode_helper_parse_args(int argc, char
> > **argv)
> >  	return NULL;
> >  }
> >
> > +/* Pre-process conf before using for init*/
> > +
> > +static int
> > +rte_eventmode_validate_user_params(struct eventmode_conf
> > *em_conf) {
> 
> No need to start with rte_eventmode_... for internal functions

[Anoob] Will fix in v2.

Re: [dpdk-dev] [EXT] Re: [PATCH 32/39] eventdev: add routine to launch eventmode workers

[Anoob Joseph]

Hi Erik,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Carrillo, Erik G
> Sent: Monday, June 10, 2019 8:02 PM
> To: Anoob Joseph <anoobj@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Rao, Nikhil <nikhil.rao@intel.com>; Gujjar,
> Abhinandan S <abhinandan.gujjar@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: Narayana Prasad Raju Athreya <pathreya@marvell.com>; dev@dpdk.org;
> Lukas Bartosik <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Van
> Haaren, Harry <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Ma, Liang J <liang.j.ma@intel.com>
> Subject: [EXT] Re: [dpdk-dev] [PATCH 32/39] eventdev: add routine to launch
> eventmode workers
> 
> External Email
> 
> ----------------------------------------------------------------------
> Hi Anoob,
> 
> I've listed a few notes in-line.
> 
> > -----Original Message-----
> > From: Anoob Joseph <anoobj@marvell.com>
> > Sent: Monday, June 3, 2019 12:33 PM
> > To: Jerin Jacob <jerinj@marvell.com>; Rao, Nikhil
> > <nikhil.rao@intel.com>; Carrillo, Erik G <erik.g.carrillo@intel.com>;
> > Gujjar, Abhinandan S <abhinandan.gujjar@intel.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad
> > <pathreya@marvell.com>; dev@dpdk.org; Lukasz Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh <pbhagavatula@marvell.com>;
> > Hemant Agrawal <hemant.agrawal@nxp.com>; Nipun Gupta
> > <nipun.gupta@nxp.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>;
> > Mattias Rönnblom <mattias.ronnblom@ericsson.com>; Ma, Liang J
> > <liang.j.ma@intel.com>
> > Subject: [PATCH 32/39] eventdev: add routine to launch eventmode
> > workers
> >
> > With eventmode, workers could be drafted differently according to the
> > capabilities of the underlying event device. The added function would
> > receive an array of such workers and probes the eventmode properties
> > to choose the worker.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > ---
> 
> <...snipped...>
> 
> > +
> > +/* Event mode packet tx types */
> > +enum rte_eventmode_helper_tx_types {
> > +	RTE_EVETNMODE_HELPER_TX_TYPE_INVALID = 0,
> 
> A couple of characters are transposed in the above.

[Anoob] Will fix it in v2.
> 
> > +	RTE_EVENTMODE_HELPER_TX_TYPE_MAX = 16 };
> > +
> >  struct rte_eventmode_helper_conf {
> >  	enum rte_eventmode_helper_pkt_transfer_mode mode;
> >  		/**< Packet transfer mode of the application */ @@ -41,6
> > +55,20 @@ struct rte_eventmode_helper_event_link_info {
> >  		/**< Lcore to be polling on this port */  };
> >
> 
> I believe anonymous unions and structures should be annotated with
> RTE_STD_C11 below and in other places throughout the series.

[Anoob] Will fix in v2.

> 
> > +/* Workers registered by the application */ struct
> > +rte_eventmode_helper_app_worker_params {
> > +	union {
> > +		struct {
> > +			uint64_t burst : 4;
> > +			/**< Specify status of rx type burst */
> > +		};
> > +		uint64_t u64;
> > +	} cap;
> > +			/**< Capabilities of this worker */
> > +	void (*worker_thread)(void *mode_conf);
> > +			/**< Worker thread */
> > +};
> > +
> >  /* Common helper functions for command line parsing */
> >
> >  /**
> > @@ -157,6 +185,27 @@ uint8_t __rte_experimental
> > rte_eventmode_helper_get_tx_queue(struct
> rte_eventmode_helper_conf
> > *mode_conf,
> >  		uint8_t eventdev_id);
> >
> 
> The doxygen documentation for __rte_experimental functions in general
> should have:
> 
> * @warning
> * @b EXPERIMENTAL: this API may change without prior notice
> 
> as well.
> 

[Anoob] Will add in v2.

> > +/**
> > + * Launch eventmode worker
> > + *
> > + * The application can request the eventmode helper subsystem to
> > +launch the
> > + * worker based on the capabilities of event device and the options
> > +selected
> > + * while initializing the eventmode.
> > + *
> > + * @param mode_conf
> > + *   Configuration of the mode in which app is doing packet handling
> > + * @param app_wrkr
> > + *   List of all the workers registered by application, along with it's
> > + *   capabilities
> > + * @param nb_wrkr_param
> > + *   Number of workers passed by the application
> > + *
> > + */
> > +void __rte_experimental
> > +rte_eventmode_helper_launch_worker(struct
> > rte_eventmode_helper_conf *mode_conf,
> > +		struct rte_eventmode_helper_app_worker_params
> > *app_wrkr,
> > +		uint8_t nb_wrkr_param);
> > +
> >  #ifdef __cplusplus
> >  }
> >  #endif
> 
> <...snipped...>
> 
> Regards,
> Erik

Re: [dpdk-dev] [EXT] Re: [PATCH 00/39] adding eventmode helper library

[Mattias Rönnblom]

On 2019-06-14 11:18, Anoob Joseph wrote:
> Hi Mattias,
>   
>> A more extensive description of the purpose of the eventmode helper
>> library would be helpful.
>>
>> Is this supposed to be a generic framework for real-world
>> applications, or only something to simplify DPDK the implementation of
>> DPDK example programs and similar?
>   
> This is intended as a generic framework, but the initial targets would be limited to DPDK example applications.
>   
> For any application to use an event device for dynamic load balancing, it has to configure the event device and the adapters. Configuring the adapters would involve providing various parameters based on which the dynamic scheduling should happen. But requiring the application to do all this configuration would make the application complicated as well as the same code has to be repeated for a new application. Event mode helper tries to solve that.
>   
> All the complex configuration would be implemented by the helper library and the helper library would provide a default conf as well.
> 

The task of configuring eventdev and its adaptors, and ethernet devices 
is a daunting task indeed. If we could simplify that, that would be great.

However, the flexibility and many of the parameters are there for a 
reason (those there aren't should be deprecated). I would expect a 
real-world application to tweak quite a few of them. I know our 
applications do.

I worry I have is that if you put eventmode (in its current form) 
forward as a generic framework, applications might start using it, only 
to realize it's not flexible enough, and then eventmode is just an extra 
layer, increasing rather than reducing complexity. Or even worse, the 
application's developers are forced to do a big-bang switch over to 
using the event and ethernet device APIs directly, in case they can't 
patch DPDK to work around the 
eventmode-assumption-that-didn't-hold-for-them.

You could always add flexibility to the framework (as you encounter a 
need for it), but then it will grow in complexity as well.

A less ambitious approach would be to instead do a properly modularized, 
non-trivial eventdev example application, for the applications to start 
off from, instead of a generic library.

I would expect it to be very difficult to design a truly generic 
application framework for eventdev-based applications. Such a framework 
would tie everything that's needed in a non-trivial application 
together. If successful, it would be a huge step toward making DPDK an 
operating system for packet processing applications.

What event devices have you tested with?

Re: [dpdk-dev] [EXT] Re: [PATCH 00/39] adding eventmode helper library

[Anoob Joseph]

Hi Mattias,

Please see my response inline.

Thanks,
Anoob

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Mattias Rönnblom
> Sent: Monday, June 17, 2019 6:54 PM
> To: Anoob Joseph <anoobj@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Nikhil Rao <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Narayana Prasad Raju Athreya <pathreya@marvell.com>; dev@dpdk.org;
> Lukas Bartosik <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Liang Ma
> <liang.j.ma@intel.com>
> Subject: Re: [dpdk-dev] [EXT] Re: [PATCH 00/39] adding eventmode helper
> library
> 
> On 2019-06-14 11:18, Anoob Joseph wrote:
> > Hi Mattias,
> >
> >> A more extensive description of the purpose of the eventmode helper
> >> library would be helpful.
> >>
> >> Is this supposed to be a generic framework for real-world
> >> applications, or only something to simplify DPDK the implementation
> >> of DPDK example programs and similar?
> >
> > This is intended as a generic framework, but the initial targets would be
> limited to DPDK example applications.
> >
> > For any application to use an event device for dynamic load balancing, it has
> to configure the event device and the adapters. Configuring the adapters
> would involve providing various parameters based on which the dynamic
> scheduling should happen. But requiring the application to do all this
> configuration would make the application complicated as well as the same
> code has to be repeated for a new application. Event mode helper tries to
> solve that.
> >
> > All the complex configuration would be implemented by the helper library
> and the helper library would provide a default conf as well.
> >
> 
> The task of configuring eventdev and its adaptors, and ethernet devices is a
> daunting task indeed. If we could simplify that, that would be great.
> 
> However, the flexibility and many of the parameters are there for a reason
> (those there aren't should be deprecated). I would expect a real-world
> application to tweak quite a few of them. I know our applications do.
> 
> I worry I have is that if you put eventmode (in its current form) forward as a
> generic framework, applications might start using it, only to realize it's not
> flexible enough, and then eventmode is just an extra layer, increasing rather
> than reducing complexity. Or even worse, the application's developers are
> forced to do a big-bang switch over to using the event and ethernet device
> APIs directly, in case they can't patch DPDK to work around the eventmode-
> assumption-that-didn't-hold-for-them.
> 
> You could always add flexibility to the framework (as you encounter a need
> for it), but then it will grow in complexity as well.
> 
> A less ambitious approach would be to instead do a properly modularized,
> non-trivial eventdev example application, for the applications to start off
> from, instead of a generic library.
> 
> I would expect it to be very difficult to design a truly generic application
> framework for eventdev-based applications. Such a framework would tie
> everything that's needed in a non-trivial application together. If successful, it
> would be a huge step toward making DPDK an operating system for packet
> processing applications.

[Anoob] The idea here is not to deprecate any event dev APIs. I do agree that all the configuration exposed by eventdev & adapters are required for various requirements in the real world applications. But the requirement to understand & use all this configuration is making the applications complicated and causes significant effort from anyone who would want to get started with event mode. The idea of helper is to allow an easy framework for applications to get started with eventmode, and then use various options from C/L or config file (both planned) to override the configuration as required. DPDK has components like crypto-scheduler which abstracts lot of configuration and simplify usage from application's perspective. This effort is on similar lines.

My patchset is a followup to http://patches.dpdk.org/patch/37955 , wherein the approach of introducing a helper library for event mode was mooted. The initial patch proposed additions in one application, and that involved huge code additions just for doing the configuration.

The helper library will be experimental while we add event-mode support for other applications like l3fwd & ipsec-secgw. I expect the helper library to be complete over the course of those applications also using the helper library.

> 
> What event devices have you tested with?

[Anoob] Eventmode helper is tested with the following combinations, 
    1. event-octeontx event PMD & nicvf eth PMD
    2. event-octeontx event PMD & eth-octeontx eth PMD

Re: [dpdk-dev] [PATCH 18/39] eventdev: add framework for eventmode conf

[Anoob Joseph]

Hi Jerin,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran
> Sent: Monday, June 10, 2019 3:36 PM
> To: Anoob Joseph <anoobj@marvell.com>; Nikhil Rao <nikhil.rao@intel.com>;
> Erik Gabriel Carrillo <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> <abhinandan.gujjar@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Pablo de Lara
> <pablo.de.lara.guarch@intel.com>
> Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal <hemant.agrawal@nxp.com>;
> Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> Subject: RE: [PATCH 18/39] eventdev: add framework for eventmode conf
> 
> > -----Original Message-----
> > From: Anoob Joseph <anoobj@marvell.com>
> > Sent: Monday, June 3, 2019 11:02 PM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nikhil Rao
> > <nikhil.rao@intel.com>; Erik Gabriel Carrillo
> > <erik.g.carrillo@intel.com>; Abhinandan Gujjar
> > <abhinandan.gujjar@intel.com>; Bruce Richardson
> > <bruce.richardson@intel.com>; Pablo de Lara
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad Raju Athreya
> > <pathreya@marvell.com>; dev@dpdk.org; Lukas Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> > <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>;
> > Nipun Gupta <nipun.gupta@nxp.com>; Harry van Haaren
> > <harry.van.haaren@intel.com>; Mattias Rönnblom
> > <mattias.ronnblom@ericsson.com>; Liang Ma <liang.j.ma@intel.com>
> > Subject: [PATCH 18/39] eventdev: add framework for eventmode conf
> >
> > Adding eventmode conf which would have all required configuration for
> > the event mode.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > ---
> >  lib/librte_eventdev/rte_eventmode_helper.c          | 16 ++++++++++++++++
> >  lib/librte_eventdev/rte_eventmode_helper_internal.h |  5 +++++
> >  2 files changed, 21 insertions(+)
> >
> > diff --git a/lib/librte_eventdev/rte_eventmode_helper.c
> > b/lib/librte_eventdev/rte_eventmode_helper.c
> > index dc2c934..38f1a2b 100644
> > --- a/lib/librte_eventdev/rte_eventmode_helper.c
> > +++ b/lib/librte_eventdev/rte_eventmode_helper.c
> > @@ -97,6 +97,7 @@ rte_eventmode_helper_parse_args(int argc, char
> > **argv)  {
> >  	int32_t opt, ret;
> >  	struct rte_eventmode_helper_conf *conf = NULL;
> > +	struct eventmode_conf *em_conf = NULL;
> >
> >  	/* Allocate memory for conf */
> >  	conf = rte_zmalloc("eventmode-helper-conf",
> > @@ -108,9 +109,21 @@ rte_eventmode_helper_parse_args(int argc, char
> > **argv)
> >  			goto err;
> >  	}
> >
> > +	/* Allocate memory for event mode params */
> > +	conf->mode_params = rte_zmalloc("eventmode-helper-mode-
> > params",
> 
> Use hugepage alloc memory only fastpath. Malloc() would be fine here.
> 

[Anoob] Will fix this in v2.

Re: [dpdk-dev] [PATCH 32/39] eventdev: add routine to launch eventmode workers

[Anoob Joseph]

Hi Erik,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Carrillo, Erik G <erik.g.carrillo@intel.com>
> Sent: Monday, June 10, 2019 8:16 PM
> To: Anoob Joseph <anoobj@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Rao, Nikhil <nikhil.rao@intel.com>; Gujjar,
> Abhinandan S <abhinandan.gujjar@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: Narayana Prasad Raju Athreya <pathreya@marvell.com>; dev@dpdk.org;
> Lukas Bartosik <lbartosik@marvell.com>; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>; Van
> Haaren, Harry <harry.van.haaren@intel.com>; Mattias Rönnblom
> <mattias.ronnblom@ericsson.com>; Ma, Liang J <liang.j.ma@intel.com>
> Subject: RE: [PATCH 32/39] eventdev: add routine to launch eventmode
> workers
> 
> Hi Anoob,
> 
> One other observation in-line:
> 
> > -----Original Message-----
> > From: Anoob Joseph <anoobj@marvell.com>
> > Sent: Monday, June 3, 2019 12:33 PM
> > To: Jerin Jacob <jerinj@marvell.com>; Rao, Nikhil
> > <nikhil.rao@intel.com>; Carrillo, Erik G <erik.g.carrillo@intel.com>;
> > Gujjar, Abhinandan S <abhinandan.gujjar@intel.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> > <pablo.de.lara.guarch@intel.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Narayana Prasad
> > <pathreya@marvell.com>; dev@dpdk.org; Lukasz Bartosik
> > <lbartosik@marvell.com>; Pavan Nikhilesh <pbhagavatula@marvell.com>;
> > Hemant Agrawal <hemant.agrawal@nxp.com>; Nipun Gupta
> > <nipun.gupta@nxp.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>;
> > Mattias Rönnblom <mattias.ronnblom@ericsson.com>; Ma, Liang J
> > <liang.j.ma@intel.com>
> > Subject: [PATCH 32/39] eventdev: add routine to launch eventmode
> > workers
> >
> > With eventmode, workers could be drafted differently according to the
> > capabilities of the underlying event device. The added function would
> > receive an array of such workers and probes the eventmode properties
> > to choose the worker.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> > ---
> 
> <...Snipped...>
> 
> > +
> > +void __rte_experimental
> > +rte_eventmode_helper_launch_worker(struct
> > rte_eventmode_helper_conf *mode_conf,
> > +		struct rte_eventmode_helper_app_worker_params
> > *app_wrkr,
> > +		uint8_t nb_wrkr_param)
> > +{
> > +	struct rte_eventmode_helper_app_worker_params *match_wrkr;
> > +	uint32_t lcore_id;
> > +	struct eventmode_conf *em_conf;
> > +
> > +	if (mode_conf == NULL) {
> > +		RTE_EM_HLPR_LOG_ERR("Invalid conf");
> > +		return;
> > +	}
> > +
> > +	if (mode_conf->mode_params == NULL) {
> > +		RTE_EM_HLPR_LOG_ERR("Invalid mode params");
> > +		return;
> > +	}
> > +
> > +	/* Get eventmode conf */
> > +	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
> > +
> > +	/* Get core ID */
> > +	lcore_id = rte_lcore_id();
> > +
> > +	/* TODO check capability for rx core */
> > +
> > +	/* Check if this is rx core */
> > +	if (em_conf->eth_core_mask & (1 << lcore_id)) {
> 
> In the above, eth_core_mask is a uint32_t, but RTE_MAX_LCORE=128 in the
> common config.

[Anoob] Will fix this in v2. Will use rte_bitmask APIs instead.
 
> 
> > +		rte_eventmode_helper_start_worker_eth_core(em_conf,
> > lcore_id);
> > +		return;
> > +	}
> > +
> > +	if (app_wrkr == NULL || nb_wrkr_param == 0) {
> > +		RTE_EM_HLPR_LOG_ERR("Invalid args");
> > +		return;
> > +	}
> > +
> > +	/*
> > +	 * This is a regular worker thread. The application would be
> > +	 * registering multiple workers with various capabilities. The
> > +	 * worker to be run will be selected by the capabilities of the
> > +	 * event device configured.
> > +	 */
> > +
> > +	/* Get the first matching worker for the event device */
> > +	match_wrkr = rte_eventmode_helper_find_worker(lcore_id,
> > +			em_conf,
> > +			app_wrkr,
> > +			nb_wrkr_param);
> > +
> > +	if (match_wrkr == NULL) {
> > +		RTE_EM_HLPR_LOG_ERR(
> > +			"No matching worker registered for lcore %d",
> > lcore_id);
> > +		goto clean_and_exit;
> > +	}
> > +
> > +	/* Verify sanity of the matched worker */
> > +	if (rte_eventmode_helper_verify_match_worker(match_wrkr) != 1)
> > {
> > +		RTE_EM_HLPR_LOG_ERR("Error in validating the matched
> > worker");
> > +		goto clean_and_exit;
> > +	}
> > +
> > +	/* Launch the worker thread */
> > +	match_wrkr->worker_thread(mode_conf);
> > +
> > +clean_and_exit:
> > +
> > +	/* Flag eth_cores to stop, if started */
> > +	rte_eventmode_helper_stop_worker_eth_core();
> > +}
> 
> <...Snipped...>
> 
> Regards,
> Erik

[dpdk-dev] [PATCH 20/39] eventdev: add eventdevice init for eventmode

[Anoob Joseph]

Adding routines to initialize event devs. The internal conf structure
would be used to track device configuration.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 lib/librte_eventdev/rte_eventmode_helper.c         | 131 +++++++++++++++++++++
 .../rte_eventmode_helper_internal.h                |  16 ++-
 2 files changed, 146 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eventdev/rte_eventmode_helper.c b/lib/librte_eventdev/rte_eventmode_helper.c
index d1a569b..bebcfb9 100644
--- a/lib/librte_eventdev/rte_eventmode_helper.c
+++ b/lib/librte_eventdev/rte_eventmode_helper.c
@@ -4,6 +4,7 @@
 #include <getopt.h>
 
 #include <rte_ethdev.h>
+#include <rte_eventdev.h>
 #include <rte_eventmode_helper.h>
 #include <rte_malloc.h>
 
@@ -154,12 +155,134 @@ rte_eventmode_helper_parse_args(int argc, char **argv)
 	return NULL;
 }
 
+/* Setup eventmode devs */
+
+static int
+rte_eventmode_helper_initialize_eventdev(struct eventmode_conf *em_conf)
+{
+	int ret = -1;
+	uint8_t i, j;
+	struct rte_event_dev_config eventdev_conf;
+	struct rte_event_dev_info evdev_default_conf;
+	struct rte_event_queue_conf eventq_conf = {0};
+	struct eventdev_params *eventdev_config;
+	int nb_eventdev = em_conf->nb_eventdev;
+	int nb_eventqueue;
+	uint8_t eventdev_id;
+
+	for (i = 0; i < nb_eventdev; i++) {
+
+		/* Get eventdev config */
+		eventdev_config = &(em_conf->eventdev_config[i]);
+
+		/* Get event dev ID */
+		eventdev_id = eventdev_config->eventdev_id;
+
+		/* Get the number of queues */
+		nb_eventqueue = eventdev_config->nb_eventqueue;
+
+		/* One queue is reserved for the final stage (doing eth tx) */
+		/* TODO handles only one Tx adapter. Fix this */
+		nb_eventqueue += 1;
+
+		/* Reset the default conf */
+		memset(&evdev_default_conf, 0,
+			sizeof(struct rte_event_dev_info));
+
+		/* Get default conf of eventdev */
+		ret = rte_event_dev_info_get(eventdev_id, &evdev_default_conf);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error in getting event device info[devID:%d]",
+				eventdev_id);
+			return ret;
+		}
+
+		memset(&eventdev_conf, 0, sizeof(struct rte_event_dev_config));
+		eventdev_conf.nb_events_limit =
+				evdev_default_conf.max_num_events;
+		eventdev_conf.nb_event_queues = nb_eventqueue;
+		eventdev_conf.nb_event_ports =
+				eventdev_config->nb_eventport;
+		eventdev_conf.nb_event_queue_flows =
+				evdev_default_conf.max_event_queue_flows;
+		eventdev_conf.nb_event_port_dequeue_depth =
+				evdev_default_conf.max_event_port_dequeue_depth;
+		eventdev_conf.nb_event_port_enqueue_depth =
+				evdev_default_conf.max_event_port_enqueue_depth;
+
+		/* Configure event device */
+		ret = rte_event_dev_configure(eventdev_id, &eventdev_conf);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error in configuring event device");
+			return ret;
+		}
+
+		/* Configure event queues */
+		for (j = 0; j < nb_eventqueue; j++) {
+
+			memset(&eventq_conf, 0,
+					sizeof(struct rte_event_queue_conf));
+
+			/* Read the requested conf */
+
+			/* Per event dev queues can be ATQ or SINGLE LINK */
+			eventq_conf.event_queue_cfg =
+					eventdev_config->ev_queue_mode;
+
+			/* Set schedule type as ATOMIC */
+			eventq_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+
+			/* Set max atomic flows to 1024 */
+			eventq_conf.nb_atomic_flows = 1024;
+			eventq_conf.nb_atomic_order_sequences = 1024;
+
+			/* Setup the queue */
+			ret = rte_event_queue_setup(eventdev_id, j,
+					&eventq_conf);
+			if (ret < 0) {
+				RTE_EM_HLPR_LOG_ERR(
+					"Error in event queue setup");
+				return ret;
+			}
+		}
+
+		/* Configure event ports */
+		for (j = 0; j <  eventdev_config->nb_eventport; j++) {
+			ret = rte_event_port_setup(eventdev_id, j, NULL);
+			if (ret < 0) {
+				RTE_EM_HLPR_LOG_ERR(
+					"Error setting up event port");
+				return ret;
+			}
+		}
+	}
+
+	/* Start event devices */
+	for (i = 0; i < nb_eventdev; i++) {
+
+		/* Get eventdev config */
+		eventdev_config = &(em_conf->eventdev_config[i]);
+
+		ret = rte_event_dev_start(eventdev_config->eventdev_id);
+		if (ret < 0) {
+			RTE_EM_HLPR_LOG_ERR(
+				"Error in starting event device[devID: %d]",
+				eventdev_config->eventdev_id);
+			return ret;
+		}
+	}
+	return 0;
+}
+
 int32_t __rte_experimental
 rte_eventmode_helper_initialize_devs(
 		struct rte_eventmode_helper_conf *mode_conf)
 {
 	int ret;
 	uint16_t portid;
+	struct eventmode_conf *em_conf;
 
 	if (mode_conf == NULL) {
 		RTE_EM_HLPR_LOG_ERR("Invalid conf");
@@ -174,6 +297,9 @@ rte_eventmode_helper_initialize_devs(
 		return -1;
 	}
 
+	/* Get eventmode conf */
+	em_conf = (struct eventmode_conf *)(mode_conf->mode_params);
+
 	/* Stop eth devices before setting up adapter */
 	RTE_ETH_FOREACH_DEV(portid) {
 
@@ -184,6 +310,11 @@ rte_eventmode_helper_initialize_devs(
 		rte_eth_dev_stop(portid);
 	}
 
+	/* Setup eventdev */
+	ret = rte_eventmode_helper_initialize_eventdev(em_conf);
+	if (ret != 0)
+		return ret;
+
 	/* Start eth devices after setting up adapter */
 	RTE_ETH_FOREACH_DEV(portid) {
 
diff --git a/lib/librte_eventdev/rte_eventmode_helper_internal.h b/lib/librte_eventdev/rte_eventmode_helper_internal.h
index 2ee7711..0b4ec59 100644
--- a/lib/librte_eventdev/rte_eventmode_helper_internal.h
+++ b/lib/librte_eventdev/rte_eventmode_helper_internal.h
@@ -32,9 +32,23 @@
 #define RTE_EM_HLPR_LOG_DEBUG(...) (void)0
 #endif
 
+/* Max event devices supported */
+#define EVENT_MODE_MAX_EVENT_DEVS RTE_EVENT_MAX_DEVS
+
+/* Event dev params */
+struct eventdev_params {
+	uint8_t eventdev_id;
+	uint8_t nb_eventqueue;
+	uint8_t nb_eventport;
+	uint8_t ev_queue_mode;
+};
+
 /* Eventmode conf data */
 struct eventmode_conf {
-	uint64_t dummy;
+	int nb_eventdev;
+		/**< No of event devs */
+	struct eventdev_params eventdev_config[EVENT_MODE_MAX_EVENT_DEVS];
+		/**< Per event dev conf */
 };
 
 #endif /* _RTE_EVENTMODE_HELPER_INTERNAL_H_ */
-- 
2.7.4