[dpdk-dev] [PATCH v6] app/testpmd: add forwarding mode to simulate a noisy neighbour

[dpdk-dev] [PATCH v6] app/testpmd: add forwarding mode to simulate a noisy neighbour

[Jens Freimann]

This adds a new forwarding mode to testpmd to simulate
more realistic behavior of a guest machine engaged in receiving
and sending packets performing Virtual Network Function (VNF).

The goal is to enable a simple way of measuring performance impact on
cache and memory footprint utilization from various VNF co-located on
the same host machine. For this it does:

* Buffer packets in a FIFO:

Create a fifo to buffer received packets. Once it flows over put
those packets into the actual tx queue. The fifo is created per tx
queue and its size can be set with the --noisy-tx-sw-buffer-flushtime
commandline parameter.

A second commandline parameter is used to set a timeout in
milliseconds after which the fifo is flushed.

--noisy-tx-sw-buffer-size [packet numbers]
Keep the mbuf in a FIFO and forward the over flooding packets from the
FIFO. This queue is per TX-queue (after all other packet processing).

--noisy-tx-sw-buffer-flushtime [delay]
Flush the packet queue if no packets have been seen during
[delay]. As long as packets are seen, the timer is reset.

Add several options to simulate route lookups (memory reads) in tables
that can be quite large, as well as route hit statistics update.
These options simulates the while stack traversal and
will trash the cache. Memory access is random.

* simulate route lookups:

Allocate a buffer and perform reads and writes on it as specified by
commandline options:

--noisy-lkup-memory [size]
Size of the VNF internal memory (MB), in which the random
read/write will be done, allocated by rte_malloc (hugepages).

--noisy-lkup-num-writes [num]
Number of random writes in memory per packet should be
performed, simulating hit-flags update. 64 bits per write,
all write in different cache lines.

--noisy-lkup-num-reads [num]
Number of random reads in memory per packet should be
performed, simulating FIB/table lookups. 64 bits per read,
all write in different cache lines.

--noisy-lkup-num-reads-writes [num]
Number of random reads and writes in memory per packet should
be performed, simulating stats update. 64 bits per read-write, all
reads and writes in different cache lines.

Signed-off-by: Jens Freimann <jfreimann@redhat.com>
---
v5->v6:
 * fix patch description
 * fix comment for pkt_burst_noisy_vnf
 * check if flush needed when no packets were received
 * free dropped packets  
 * remove redundant else-if
 * do memory access simulation in all cases
 * change order of free'd data structures in noisy_fwd_end
 * only allocate one noisy_config struct per port 
 * check for return of rte_ring_create()
 * change checking of input parameters in noisy_fwd_begin(). Decided to
   allow to set paramters to 0 (which is the default)
 * did not change use of "=N" in documentation as suggested by Kevin
   because it is how it's done for most other parameters
 * make error message match code in checking of flush time parameter
 * don't add whitespace in testpmd.h 

v4->v5:
 * try to minimize impact in common code. Instead implement fwd_begin and fwd_end
 * simplify actual fwd function
 * remove unnecessary casts (Kevin)
 * use more meaningful names for parameters and global variables (Kevin)
 * free ring and vnf_mem as well (Kevin)
 * squash documentation and code into single patch (Bernard)
 * fix patch subject to "app/testpmd" (Bernard)

 app/test-pmd/Makefile                       |   1 +
 app/test-pmd/meson.build                    |   1 +
 app/test-pmd/noisy_vnf.c                    | 279 ++++++++++++++++++++
 app/test-pmd/parameters.c                   |  60 +++++
 app/test-pmd/testpmd.c                      |  35 +++
 app/test-pmd/testpmd.h                      |   8 +
 doc/guides/testpmd_app_ug/run_app.rst       |  33 +++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   7 +-
 8 files changed, 422 insertions(+), 2 deletions(-)
 create mode 100644 app/test-pmd/noisy_vnf.c

diff --git a/app/test-pmd/Makefile b/app/test-pmd/Makefile
index 2b4d604b8..e2581ca66 100644
--- a/app/test-pmd/Makefile
+++ b/app/test-pmd/Makefile
@@ -33,6 +33,7 @@ SRCS-y += rxonly.c
 SRCS-y += txonly.c
 SRCS-y += csumonly.c
 SRCS-y += icmpecho.c
+SRCS-y += noisy_vnf.c
 SRCS-$(CONFIG_RTE_LIBRTE_IEEE1588) += ieee1588fwd.c
 SRCS-$(CONFIG_RTE_LIBRTE_BPF) += bpf_cmd.c
 
diff --git a/app/test-pmd/meson.build b/app/test-pmd/meson.build
index a0b3be07f..9ef6ed957 100644
--- a/app/test-pmd/meson.build
+++ b/app/test-pmd/meson.build
@@ -17,6 +17,7 @@ sources = files('cmdline.c',
 	'iofwd.c',
 	'macfwd.c',
 	'macswap.c',
+	'noisy_vnf.c',
 	'parameters.c',
 	'rxonly.c',
 	'testpmd.c',
diff --git a/app/test-pmd/noisy_vnf.c b/app/test-pmd/noisy_vnf.c
new file mode 100644
index 000000000..de9022074
--- /dev/null
+++ b/app/test-pmd/noisy_vnf.c
@@ -0,0 +1,279 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Red Hat Corp.
+ */
+
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdbool.h>
+#include <string.h>
+#include <errno.h>
+#include <stdint.h>
+#include <unistd.h>
+#include <inttypes.h>
+
+#include <sys/queue.h>
+#include <sys/stat.h>
+
+#include <rte_common.h>
+#include <rte_log.h>
+#include <rte_debug.h>
+#include <rte_cycles.h>
+#include <rte_memory.h>
+#include <rte_launch.h>
+#include <rte_eal.h>
+#include <rte_per_lcore.h>
+#include <rte_lcore.h>
+#include <rte_memcpy.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_ethdev.h>
+#include <rte_flow.h>
+#include <rte_malloc.h>
+
+#include "testpmd.h"
+
+struct noisy_config {
+	struct rte_ring *f;
+	uint64_t prev_time;
+	char *vnf_mem;
+	bool do_buffering;
+	bool do_flush;
+	bool do_sim;
+};
+
+struct noisy_config *noisy_cfg[RTE_MAX_ETHPORTS];
+
+static inline void
+do_write(char *vnf_mem)
+{
+	uint64_t i = rte_rand();
+	uint64_t w = rte_rand();
+
+	vnf_mem[i % ((noisy_lkup_mem_sz * 1024 * 1024) /
+			RTE_CACHE_LINE_SIZE)] = w;
+}
+
+static inline void
+do_read(char *vnf_mem)
+{
+	uint64_t i = rte_rand();
+	uint64_t r;
+
+	r = vnf_mem[i % ((noisy_lkup_mem_sz * 1024 * 1024) /
+			RTE_CACHE_LINE_SIZE)];
+	r++;
+}
+
+static inline void
+do_readwrite(char *vnf_mem)
+{
+	do_read(vnf_mem);
+	do_write(vnf_mem);
+}
+
+/*
+ * Simulate route lookups as defined by commandline parameters
+ */
+static void
+sim_memory_lookups(struct noisy_config *ncf, uint16_t nb_pkts)
+{
+	uint16_t i, j;
+
+	if (!ncf->do_sim)
+		return;
+
+	for (i = 0; i < nb_pkts; i++) {
+		for (j = 0; j < noisy_lkup_num_writes; j++)
+			do_write(ncf->vnf_mem);
+		for (j = 0; j < noisy_lkup_num_reads; j++)
+			do_read(ncf->vnf_mem);
+		for (j = 0; j < noisy_lkup_num_reads_writes; j++)
+			do_readwrite(ncf->vnf_mem);
+	}
+}
+
+static uint16_t
+do_retry(uint16_t nb_rx, uint16_t nb_tx, struct rte_mbuf **pkts,
+	 struct fwd_stream *fs)
+{
+	uint32_t retry = 0;
+
+	while (nb_tx < nb_rx && retry++ < burst_tx_retry_num) {
+		rte_delay_us(burst_tx_delay_time);
+		nb_tx += rte_eth_tx_burst(fs->tx_port, fs->tx_queue,
+				&pkts[nb_tx], nb_rx - nb_tx);
+	}
+
+	return nb_tx;
+}
+
+static uint32_t
+drop_pkts(struct rte_mbuf **pkts, uint16_t nb_rx, uint16_t nb_tx)
+{
+	if (nb_tx < nb_rx) {
+		do {
+			rte_pktmbuf_free(pkts[nb_tx]);
+		} while (++nb_tx < nb_rx);
+	}
+
+	return nb_rx - nb_tx;
+}
+
+/*
+ * Forwarding of packets in noisy VNF mode.  Forward packets but perform
+ * memory operations first as specified on cmdline.
+ *
+ * Depending on which commandline parameters are specified we have
+ * different cases to handle:
+ *
+ * 1. No FIFO size was given, so we don't do buffering of incoming
+ *    packets.  This case is pretty much what iofwd does but in this case
+ *    we also do simulation of memory accesses (depending on which
+ *    parameters were specified for it).
+ * 2. User wants do buffer packets in a FIFO and sent out overflowing
+ *    packets.
+ * 3. User wants a FIFO and specifies a time in ms to flush all packets
+ *    out of the FIFO
+ * 4. Cases 2 and 3 combined
+ */
+static void
+pkt_burst_noisy_vnf(struct fwd_stream *fs)
+{
+	const uint64_t freq_khz = rte_get_timer_hz() / 1000;
+	struct noisy_config *ncf = noisy_cfg[fs->rx_port];
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+	struct rte_mbuf *tmp_pkts[MAX_PKT_BURST];
+	uint16_t nb_deqd = 0;
+	uint16_t nb_rx = 0;
+	uint16_t nb_tx = 0;
+	uint16_t nb_enqd;
+	unsigned int fifo_free;
+	uint64_t delta_ms;
+	bool needs_flush = false;
+	uint64_t now;
+
+	nb_rx = rte_eth_rx_burst(fs->rx_port, fs->rx_queue,
+			pkts_burst, nb_pkt_per_burst);
+	if (unlikely(nb_rx == 0))
+		goto flush;
+	fs->rx_packets += nb_rx;
+
+	if (!ncf->do_buffering) {
+		sim_memory_lookups(ncf, nb_rx);
+		nb_tx = rte_eth_tx_burst(fs->tx_port, fs->tx_queue,
+				pkts_burst, nb_rx);
+		if (unlikely(nb_tx < nb_rx) && fs->retry_enabled)
+			nb_tx += do_retry(nb_rx, nb_tx, pkts_burst, fs);
+		fs->tx_packets += nb_tx;
+		fs->fwd_dropped += drop_pkts(pkts_burst, nb_rx, nb_tx);
+		return;
+	}
+
+	fifo_free = rte_ring_free_count(ncf->f);
+	if (fifo_free >= nb_rx) {
+		nb_enqd = rte_ring_enqueue_burst(ncf->f,
+				(void **) pkts_burst, nb_rx, NULL);
+		if (nb_enqd < nb_rx)
+			fs->fwd_dropped += drop_pkts(pkts_burst, nb_rx, nb_enqd);
+	} else {
+		nb_deqd = rte_ring_dequeue_burst(ncf->f,
+				(void **) tmp_pkts, nb_rx, NULL);
+		nb_enqd = rte_ring_enqueue_burst(ncf->f,
+				(void **) pkts_burst, nb_deqd, NULL);
+		if (nb_deqd > 0) {
+			nb_tx = rte_eth_tx_burst(fs->tx_port,
+					fs->tx_queue, tmp_pkts,
+					nb_deqd);
+			if (unlikely(nb_tx < nb_rx) && fs->retry_enabled)
+				nb_tx += do_retry(nb_rx, nb_tx, tmp_pkts, fs);
+			fs->fwd_dropped += drop_pkts(tmp_pkts, nb_deqd, nb_tx);
+		}
+	}
+
+	sim_memory_lookups(ncf, nb_enqd);
+
+flush:
+	if (ncf->do_flush) {
+		if (!ncf->prev_time)
+			now = ncf->prev_time = rte_get_timer_cycles();
+		else
+			now = rte_get_timer_cycles();
+		delta_ms = (now - ncf->prev_time) / freq_khz;
+		needs_flush = delta_ms >= noisy_tx_sw_buf_flush_time &&
+				noisy_tx_sw_buf_flush_time > 0 && !nb_tx;
+	}
+	while (needs_flush && !rte_ring_empty(ncf->f)) {
+		unsigned int sent;
+		nb_deqd = rte_ring_dequeue_burst(ncf->f, (void **)tmp_pkts,
+				MAX_PKT_BURST, NULL);
+		sent = rte_eth_tx_burst(fs->tx_port, fs->tx_queue,
+					 tmp_pkts, nb_deqd);
+		if (unlikely(sent < nb_deqd) && fs->retry_enabled)
+			nb_tx += do_retry(nb_rx, nb_tx, tmp_pkts, fs);
+		fs->fwd_dropped += drop_pkts(tmp_pkts, nb_deqd, sent);
+		ncf->prev_time = rte_get_timer_cycles();
+	}
+}
+
+#define NOISY_STRSIZE 256
+#define NOISY_RING "noisy_ring_%d\n"
+
+static void
+noisy_fwd_end(portid_t pi)
+{
+	rte_ring_free(noisy_cfg[pi]->f);
+	rte_free(noisy_cfg[pi]->vnf_mem);
+	rte_free(noisy_cfg[pi]);
+}
+
+static void
+noisy_fwd_begin(portid_t pi)
+{
+	struct noisy_config *n;
+	char name[NOISY_STRSIZE];
+
+	noisy_cfg[pi] = rte_zmalloc("testpmd noisy fifo and timers",
+				sizeof(struct noisy_config),
+				RTE_CACHE_LINE_SIZE);
+	if (noisy_cfg == NULL) {
+		rte_exit(EXIT_FAILURE,
+			 "rte_zmalloc(%d) struct noisy_config) \
+			 failed\n", (int) pi);
+	}
+	n = noisy_cfg[pi];
+	n->do_buffering = noisy_tx_sw_bufsz > 0;
+	n->do_sim = noisy_lkup_num_writes + noisy_lkup_num_reads +
+		    noisy_lkup_num_reads_writes;
+	n->do_flush = noisy_tx_sw_buf_flush_time > 0;
+
+	if (n->do_buffering) {
+		snprintf(name, NOISY_STRSIZE, NOISY_RING, pi);
+		n->f = rte_ring_create(name, noisy_tx_sw_bufsz,
+				rte_socket_id(), 0);
+		if (!n->f)
+			rte_exit(EXIT_FAILURE,
+				 "rte_ring_create(%d), size %d) \
+				 failed\n", (int) pi,
+				 noisy_tx_sw_bufsz);
+	}
+	if (noisy_lkup_mem_sz > 0) {
+		n->vnf_mem = (char *) rte_zmalloc("vnf sim memory",
+				 noisy_lkup_mem_sz * 1024 * 1024,
+				 RTE_CACHE_LINE_SIZE);
+		if (!n->vnf_mem)
+			rte_exit(EXIT_FAILURE,
+				 "rte_zmalloc(%" PRIu64 ") for vnf \
+				 memory) failed\n", noisy_lkup_mem_sz);
+	} else if (n->do_sim) {
+		rte_exit(EXIT_FAILURE, "--noisy-lkup-memory-size \
+			 must be > 0\n");
+	}
+}
+
+struct fwd_engine noisy_vnf_engine = {
+	.fwd_mode_name  = "noisy",
+	.port_fwd_begin = noisy_fwd_begin,
+	.port_fwd_end   = noisy_fwd_end,
+	.packet_fwd     = pkt_burst_noisy_vnf,
+};
+
diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index 9220e1c1b..3231b0c51 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c
@@ -625,6 +625,12 @@ launch_args_parse(int argc, char** argv)
 		{ "vxlan-gpe-port",		1, 0, 0 },
 		{ "mlockall",			0, 0, 0 },
 		{ "no-mlockall",		0, 0, 0 },
+		{ "noisy-tx-sw-buffer-size",	1, 0, 0 },
+		{ "noisy-tx-sw-buffer-flushtime",1, 0, 0 },
+		{ "noisy-lkup-memory",		1, 0, 0 },
+		{ "noisy-lkup-num-writes",	1, 0, 0 },
+		{ "noisy-lkup-num-reads",	1, 0, 0 },
+		{ "noisy-lkup-num-reads-writes",1, 0, 0 },
 		{ 0, 0, 0, 0 },
 	};
 
@@ -1145,6 +1151,60 @@ launch_args_parse(int argc, char** argv)
 				do_mlockall = 1;
 			if (!strcmp(lgopts[opt_idx].name, "no-mlockall"))
 				do_mlockall = 0;
+			if (!strcmp(lgopts[opt_idx].name,
+				    "noisy-tx-sw-buffer-size")) {
+				n = atoi(optarg);
+				if (n >= 0)
+					noisy_tx_sw_bufsz = n;
+				else
+					rte_exit(EXIT_FAILURE,
+						"noisy-tx-sw-buffer-size must be >= 0\n");
+			}
+			if (!strcmp(lgopts[opt_idx].name,
+				    "noisy-tx-sw-buffer-flushtime")) {
+				n = atoi(optarg);
+				if (n >= 0)
+					noisy_tx_sw_buf_flush_time = n;
+				else
+					rte_exit(EXIT_FAILURE,
+						 "noisy-tx-sw-buffer-flushtime must be >= 0\n");
+			}
+			if (!strcmp(lgopts[opt_idx].name,
+				    "noisy-lkup-memory")) {
+				n = atoi(optarg);
+				if (n > 0)
+					noisy_lkup_mem_sz = n;
+				else
+					rte_exit(EXIT_FAILURE,
+						 "noisy-lkup-memory must be > 0\n");
+			}
+			if (!strcmp(lgopts[opt_idx].name,
+				    "noisy-lkup-num-writes")) {
+				n = atoi(optarg);
+				if (n > 0)
+					noisy_lkup_num_writes = n;
+				else
+					rte_exit(EXIT_FAILURE,
+						 "noisy-lkup-num-writes must be > 0\n");
+			}
+			if (!strcmp(lgopts[opt_idx].name,
+				    "noisy-lkup-num-reads")) {
+				n = atoi(optarg);
+				if (n > 0)
+					noisy_lkup_num_reads = n;
+				else
+					rte_exit(EXIT_FAILURE,
+						 "noisy-lkup-num-reads must be > 0\n");
+			}
+			if (!strcmp(lgopts[opt_idx].name,
+				    "noisy-lkup-num-reads-writes")) {
+				n = atoi(optarg);
+				if (n > 0)
+					noisy_lkup_num_reads_writes = n;
+				else
+					rte_exit(EXIT_FAILURE,
+						 "noisy-lkup-num-reads-writes must be > 0\n");
+			}
 			break;
 		case 'h':
 			usage(argv[0]);
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 571ecb4ac..3eeeaa090 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -157,6 +157,7 @@ struct fwd_engine * fwd_engines[] = {
 	&tx_only_engine,
 	&csum_fwd_engine,
 	&icmp_echo_engine,
+	&noisy_vnf_engine,
 #if defined RTE_LIBRTE_PMD_SOFTNIC
 	&softnic_fwd_engine,
 #endif
@@ -252,6 +253,40 @@ int16_t tx_free_thresh = RTE_PMD_PARAM_UNSET;
  */
 int16_t tx_rs_thresh = RTE_PMD_PARAM_UNSET;
 
+/*
+ * Configurable value of buffered packets before sending.
+ */
+uint16_t noisy_tx_sw_bufsz;
+
+/*
+ * Configurable value of packet buffer timeout.
+ */
+uint16_t noisy_tx_sw_buf_flush_time;
+
+/*
+ * Configurable value for size of VNF internal memory area
+ * used for simulating noisy neighbour behaviour
+ */
+uint64_t noisy_lkup_mem_sz;
+
+/*
+ * Configurable value of number of random writes done in
+ * VNF simulation memory area.
+ */
+uint64_t noisy_lkup_num_writes;
+
+/*
+ * Configurable value of number of random reads done in
+ * VNF simulation memory area.
+ */
+uint64_t noisy_lkup_num_reads;
+
+/*
+ * Configurable value of number of random reads/writes done in
+ * VNF simulation memory area.
+ */
+uint64_t noisy_lkup_num_reads_writes;
+
 /*
  * Receive Side Scaling (RSS) configuration.
  */
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index a1f661472..38b5dc15d 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -243,6 +243,7 @@ extern struct fwd_engine rx_only_engine;
 extern struct fwd_engine tx_only_engine;
 extern struct fwd_engine csum_fwd_engine;
 extern struct fwd_engine icmp_echo_engine;
+extern struct fwd_engine noisy_vnf_engine;
 #ifdef SOFTNIC
 extern struct fwd_engine softnic_fwd_engine;
 #endif
@@ -375,6 +376,13 @@ extern int8_t rx_drop_en;
 extern int16_t tx_free_thresh;
 extern int16_t tx_rs_thresh;
 
+extern uint16_t noisy_tx_sw_bufsz;
+extern uint16_t noisy_tx_sw_buf_flush_time;
+extern uint64_t noisy_lkup_mem_sz;
+extern uint64_t noisy_lkup_num_writes;
+extern uint64_t noisy_lkup_num_reads;
+extern uint64_t noisy_lkup_num_reads_writes;
+
 extern uint8_t dcb_config;
 extern uint8_t dcb_test;
 
diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst
index f301c2b6f..9eb650232 100644
--- a/doc/guides/testpmd_app_ug/run_app.rst
+++ b/doc/guides/testpmd_app_ug/run_app.rst
@@ -340,6 +340,7 @@ The commandline options are:
        icmpecho
        ieee1588
        tm
+       noisy
 
 *   ``--rss-ip``
 
@@ -498,3 +499,35 @@ The commandline options are:
 *   ``--no-mlockall``
 
     Disable locking all memory.
+
+*   ``--noisy-tx-sw-buffer-size``
+
+    Set the number of maximum elements  of the FIFO queue to be created
+    for buffering packets. Only available with the noisy forwarding mode.
+    The default value is 0.
+
+*   ``--noisy-tx-sw-buffer-flushtime=N``
+
+    Set the time before packets in the FIFO queue is flushed.
+    Only available with the noisy forwarding mode. The default value is 0.
+
+*   ``--noisy-lkup-memory=N``
+
+    Set the size of the noisy neighbour simulation memory buffer in MB to N.
+    Only available with the noisy forwarding mode. The default value is 0.
+
+
+*   ``--noisy-lkup-num-reads=N``
+
+    Set the number of reads to be done in noisy neighbour simulation memory buffer to N.
+    Only available with the noisy forwarding mode. The default value is 0.
+
+*   ``--noisy-lkup-num-writes=N``
+
+    Set the number of writes to be done in noisy neighbour simulation memory buffer to N.
+    Only available with the noisy forwarding mode. The default value is 0.
+
+*   ``--noisy-lkup-num-reads-writes=N``
+
+    Set the number of r/w accesses to be done in noisy neighbour simulation memory buffer to N.
+    Only available with the noisy forwarding mode. The default value is 0.
diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
index dde205a2b..e2f9db2b5 100644
--- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
+++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
@@ -289,7 +289,7 @@ set fwd
 Set the packet forwarding mode::
 
    testpmd> set fwd (io|mac|macswap|flowgen| \
-                     rxonly|txonly|csum|icmpecho) (""|retry)
+                     rxonly|txonly|csum|icmpecho|noisy) (""|retry)
 
 ``retry`` can be specified for forwarding engines except ``rx_only``.
 
@@ -323,8 +323,11 @@ The available information categories are:
 * ``softnic``: Demonstrates the softnic forwarding operation. In this mode, packet forwarding is
   similar to I/O mode except for the fact that packets are loopback to the softnic ports only. Therefore, portmask parameter should be set to softnic port only. The various software based custom NIC pipelines specified through the softnic firmware (DPDK packet framework script) can be tested in this mode. Furthermore, it allows to build 5-level hierarchical QoS scheduler as a default option that can be enabled through CLI once testpmd application is initialised. The user can modify the default scheduler hierarchy or can specify the new QoS Scheduler hierarchy through CLI. Requires ``CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y``.
 
-Example::
+* ``noisy``: Noisy neighbour simulation.
+  Simulate more realistic behavior of a guest machine engaged in receiving
+  and sending packets performing Virtual Network Function (VNF).
 
+Example::
    testpmd> set fwd rxonly
 
    Set rxonly packet forwarding mode
-- 
2.17.1

Re: [dpdk-dev] [PATCH v6] app/testpmd: add forwarding mode to simulate a noisy neighbour

[Kevin Traynor]

On 09/18/2018 10:35 AM, Jens Freimann wrote:
> This adds a new forwarding mode to testpmd to simulate
> more realistic behavior of a guest machine engaged in receiving
> and sending packets performing Virtual Network Function (VNF).
> 
> The goal is to enable a simple way of measuring performance impact on
> cache and memory footprint utilization from various VNF co-located on
> the same host machine. For this it does:
> 
> * Buffer packets in a FIFO:
> 
> Create a fifo to buffer received packets. Once it flows over put
> those packets into the actual tx queue. The fifo is created per tx
> queue and its size can be set with the --noisy-tx-sw-buffer-flushtime
> commandline parameter.
> 
> A second commandline parameter is used to set a timeout in
> milliseconds after which the fifo is flushed.
> 
> --noisy-tx-sw-buffer-size [packet numbers]
> Keep the mbuf in a FIFO and forward the over flooding packets from the
> FIFO. This queue is per TX-queue (after all other packet processing).
> 
> --noisy-tx-sw-buffer-flushtime [delay]
> Flush the packet queue if no packets have been seen during
> [delay]. As long as packets are seen, the timer is reset.
> 
> Add several options to simulate route lookups (memory reads) in tables
> that can be quite large, as well as route hit statistics update.
> These options simulates the while stack traversal and
> will trash the cache. Memory access is random.
> 
> * simulate route lookups:
> 
> Allocate a buffer and perform reads and writes on it as specified by
> commandline options:
> 
> --noisy-lkup-memory [size]
> Size of the VNF internal memory (MB), in which the random
> read/write will be done, allocated by rte_malloc (hugepages).
> 
> --noisy-lkup-num-writes [num]
> Number of random writes in memory per packet should be
> performed, simulating hit-flags update. 64 bits per write,
> all write in different cache lines.
> 
> --noisy-lkup-num-reads [num]
> Number of random reads in memory per packet should be
> performed, simulating FIB/table lookups. 64 bits per read,
> all write in different cache lines.
> 
> --noisy-lkup-num-reads-writes [num]
> Number of random reads and writes in memory per packet should
> be performed, simulating stats update. 64 bits per read-write, all
> reads and writes in different cache lines.
> 
> Signed-off-by: Jens Freimann <jfreimann@redhat.com>
> ---

Hi Jens, thanks for the new version. A small few remaining comments below,

Kevin.

<snip>

> +
> +static void
> +noisy_fwd_begin(portid_t pi)
> +{
> +	struct noisy_config *n;
> +	char name[NOISY_STRSIZE];
> +
> +	noisy_cfg[pi] = rte_zmalloc("testpmd noisy fifo and timers",
> +				sizeof(struct noisy_config),
> +				RTE_CACHE_LINE_SIZE);
> +	if (noisy_cfg == NULL) {

Looks like it should be 'if (noisy_cfg[pi] == NULL)'

> +		rte_exit(EXIT_FAILURE,
> +			 "rte_zmalloc(%d) struct noisy_config) \
> +			 failed\n", (int) pi);
> +	}
> +	n = noisy_cfg[pi];
> +	n->do_buffering = noisy_tx_sw_bufsz > 0;
> +	n->do_sim = noisy_lkup_num_writes + noisy_lkup_num_reads +
> +		    noisy_lkup_num_reads_writes;
> +	n->do_flush = noisy_tx_sw_buf_flush_time > 0;
> +
> +	if (n->do_buffering) {
> +		snprintf(name, NOISY_STRSIZE, NOISY_RING, pi);
> +		n->f = rte_ring_create(name, noisy_tx_sw_bufsz,
> +				rte_socket_id(), 0);
> +		if (!n->f)
> +			rte_exit(EXIT_FAILURE,
> +				 "rte_ring_create(%d), size %d) \
> +				 failed\n", (int) pi,
> +				 noisy_tx_sw_bufsz);
> +	}
> +	if (noisy_lkup_mem_sz > 0) {
> +		n->vnf_mem = (char *) rte_zmalloc("vnf sim memory",
> +				 noisy_lkup_mem_sz * 1024 * 1024,
> +				 RTE_CACHE_LINE_SIZE);
> +		if (!n->vnf_mem)
> +			rte_exit(EXIT_FAILURE,
> +				 "rte_zmalloc(%" PRIu64 ") for vnf \
> +				 memory) failed\n", noisy_lkup_mem_sz);
> +	} else if (n->do_sim) {
> +		rte_exit(EXIT_FAILURE, "--noisy-lkup-memory-size \
> +			 must be > 0\n");
> +	}
> +}
> +
> +struct fwd_engine noisy_vnf_engine = {
> +	.fwd_mode_name  = "noisy",
> +	.port_fwd_begin = noisy_fwd_begin,
> +	.port_fwd_end   = noisy_fwd_end,
> +	.packet_fwd     = pkt_burst_noisy_vnf,
> +};
> +

new blank line at EOF.
+
warning: 1 line adds whitespace errors.

> diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
> index 9220e1c1b..3231b0c51 100644
> --- a/app/test-pmd/parameters.c
> +++ b/app/test-pmd/parameters.c
> @@ -625,6 +625,12 @@ launch_args_parse(int argc, char** argv)
>  		{ "vxlan-gpe-port",		1, 0, 0 },
>  		{ "mlockall",			0, 0, 0 },
>  		{ "no-mlockall",		0, 0, 0 },
> +		{ "noisy-tx-sw-buffer-size",	1, 0, 0 },
> +		{ "noisy-tx-sw-buffer-flushtime",1, 0, 0 },
> +		{ "noisy-lkup-memory",		1, 0, 0 },
> +		{ "noisy-lkup-num-writes",	1, 0, 0 },
> +		{ "noisy-lkup-num-reads",	1, 0, 0 },
> +		{ "noisy-lkup-num-reads-writes",1, 0, 0 },
>  		{ 0, 0, 0, 0 },
>  	};
>  
> @@ -1145,6 +1151,60 @@ launch_args_parse(int argc, char** argv)
>  				do_mlockall = 1;
>  			if (!strcmp(lgopts[opt_idx].name, "no-mlockall"))
>  				do_mlockall = 0;
> +			if (!strcmp(lgopts[opt_idx].name,
> +				    "noisy-tx-sw-buffer-size")) {
> +				n = atoi(optarg);
> +				if (n >= 0)
> +					noisy_tx_sw_bufsz = n;
> +				else
> +					rte_exit(EXIT_FAILURE,
> +						"noisy-tx-sw-buffer-size must be >= 0\n");
> +			}
> +			if (!strcmp(lgopts[opt_idx].name,
> +				    "noisy-tx-sw-buffer-flushtime")) {
> +				n = atoi(optarg);
> +				if (n >= 0)
> +					noisy_tx_sw_buf_flush_time = n;
> +				else
> +					rte_exit(EXIT_FAILURE,
> +						 "noisy-tx-sw-buffer-flushtime must be >= 0\n");
> +			}
> +			if (!strcmp(lgopts[opt_idx].name,
> +				    "noisy-lkup-memory")) {
> +				n = atoi(optarg);
> +				if (n > 0)

I thought this and below ones would also be '>=' also?

> +					noisy_lkup_mem_sz = n;
> +				else
> +					rte_exit(EXIT_FAILURE,
> +						 "noisy-lkup-memory must be > 0\n");
> +			}
> +			if (!strcmp(lgopts[opt_idx].name,
> +				    "noisy-lkup-num-writes")) {
> +				n = atoi(optarg);
> +				if (n > 0)
> +					noisy_lkup_num_writes = n;
> +				else
> +					rte_exit(EXIT_FAILURE,
> +						 "noisy-lkup-num-writes must be > 0\n");
> +			}
> +			if (!strcmp(lgopts[opt_idx].name,
> +				    "noisy-lkup-num-reads")) {
> +				n = atoi(optarg);
> +				if (n > 0)
> +					noisy_lkup_num_reads = n;
> +				else
> +					rte_exit(EXIT_FAILURE,
> +						 "noisy-lkup-num-reads must be > 0\n");
> +			}
> +			if (!strcmp(lgopts[opt_idx].name,
> +				    "noisy-lkup-num-reads-writes")) {
> +				n = atoi(optarg);
> +				if (n > 0)
> +					noisy_lkup_num_reads_writes = n;
> +				else
> +					rte_exit(EXIT_FAILURE,
> +						 "noisy-lkup-num-reads-writes must be > 0\n");
> +			}
>  			break;
>  		case 'h':
>  			usage(argv[0]);

Re: [dpdk-dev] [PATCH v6] app/testpmd: add forwarding mode to simulate a noisy neighbour

[Jens Freimann]

On Thu, Sep 20, 2018 at 05:50:39PM +0100, Kevin Traynor wrote:
>On 09/18/2018 10:35 AM, Jens Freimann wrote:
>> This adds a new forwarding mode to testpmd to simulate
>> more realistic behavior of a guest machine engaged in receiving
>> and sending packets performing Virtual Network Function (VNF).
>>
>> The goal is to enable a simple way of measuring performance impact on
>> cache and memory footprint utilization from various VNF co-located on
>> the same host machine. For this it does:
>>
>> * Buffer packets in a FIFO:
>>
>> Create a fifo to buffer received packets. Once it flows over put
>> those packets into the actual tx queue. The fifo is created per tx
>> queue and its size can be set with the --noisy-tx-sw-buffer-flushtime
>> commandline parameter.
>>
>> A second commandline parameter is used to set a timeout in
>> milliseconds after which the fifo is flushed.
>>
>> --noisy-tx-sw-buffer-size [packet numbers]
>> Keep the mbuf in a FIFO and forward the over flooding packets from the
>> FIFO. This queue is per TX-queue (after all other packet processing).
>>
>> --noisy-tx-sw-buffer-flushtime [delay]
>> Flush the packet queue if no packets have been seen during
>> [delay]. As long as packets are seen, the timer is reset.
>>
>> Add several options to simulate route lookups (memory reads) in tables
>> that can be quite large, as well as route hit statistics update.
>> These options simulates the while stack traversal and
>> will trash the cache. Memory access is random.
>>
>> * simulate route lookups:
>>
>> Allocate a buffer and perform reads and writes on it as specified by
>> commandline options:
>>
>> --noisy-lkup-memory [size]
>> Size of the VNF internal memory (MB), in which the random
>> read/write will be done, allocated by rte_malloc (hugepages).
>>
>> --noisy-lkup-num-writes [num]
>> Number of random writes in memory per packet should be
>> performed, simulating hit-flags update. 64 bits per write,
>> all write in different cache lines.
>>
>> --noisy-lkup-num-reads [num]
>> Number of random reads in memory per packet should be
>> performed, simulating FIB/table lookups. 64 bits per read,
>> all write in different cache lines.
>>
>> --noisy-lkup-num-reads-writes [num]
>> Number of random reads and writes in memory per packet should
>> be performed, simulating stats update. 64 bits per read-write, all
>> reads and writes in different cache lines.
>>
>> Signed-off-by: Jens Freimann <jfreimann@redhat.com>
>> ---
>
>Hi Jens, thanks for the new version. A small few remaining comments below,
>
>Kevin.
>
><snip>
>
>> +
>> +static void
>> +noisy_fwd_begin(portid_t pi)
>> +{
>> +	struct noisy_config *n;
>> +	char name[NOISY_STRSIZE];
>> +
>> +	noisy_cfg[pi] = rte_zmalloc("testpmd noisy fifo and timers",
>> +				sizeof(struct noisy_config),
>> +				RTE_CACHE_LINE_SIZE);
>> +	if (noisy_cfg == NULL) {
>
>Looks like it should be 'if (noisy_cfg[pi] == NULL)'

yep, fixed.

>> +		rte_exit(EXIT_FAILURE,
>> +			 "rte_zmalloc(%d) struct noisy_config) \
>> +			 failed\n", (int) pi);
>> +	}
>> +	n = noisy_cfg[pi];
>> +	n->do_buffering = noisy_tx_sw_bufsz > 0;
>> +	n->do_sim = noisy_lkup_num_writes + noisy_lkup_num_reads +
>> +		    noisy_lkup_num_reads_writes;
>> +	n->do_flush = noisy_tx_sw_buf_flush_time > 0;
>> +
>> +	if (n->do_buffering) {
>> +		snprintf(name, NOISY_STRSIZE, NOISY_RING, pi);
>> +		n->f = rte_ring_create(name, noisy_tx_sw_bufsz,
>> +				rte_socket_id(), 0);
>> +		if (!n->f)
>> +			rte_exit(EXIT_FAILURE,
>> +				 "rte_ring_create(%d), size %d) \
>> +				 failed\n", (int) pi,
>> +				 noisy_tx_sw_bufsz);
>> +	}
>> +	if (noisy_lkup_mem_sz > 0) {
>> +		n->vnf_mem = (char *) rte_zmalloc("vnf sim memory",
>> +				 noisy_lkup_mem_sz * 1024 * 1024,
>> +				 RTE_CACHE_LINE_SIZE);
>> +		if (!n->vnf_mem)
>> +			rte_exit(EXIT_FAILURE,
>> +				 "rte_zmalloc(%" PRIu64 ") for vnf \
>> +				 memory) failed\n", noisy_lkup_mem_sz);
>> +	} else if (n->do_sim) {
>> +		rte_exit(EXIT_FAILURE, "--noisy-lkup-memory-size \
>> +			 must be > 0\n");
>> +	}
>> +}
>> +
>> +struct fwd_engine noisy_vnf_engine = {
>> +	.fwd_mode_name  = "noisy",
>> +	.port_fwd_begin = noisy_fwd_begin,
>> +	.port_fwd_end   = noisy_fwd_end,
>> +	.packet_fwd     = pkt_burst_noisy_vnf,
>> +};
>> +
>
>new blank line at EOF.
>+
>warning: 1 line adds whitespace errors.

hmpf

>> diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
>> index 9220e1c1b..3231b0c51 100644
>> --- a/app/test-pmd/parameters.c
>> +++ b/app/test-pmd/parameters.c
>> @@ -625,6 +625,12 @@ launch_args_parse(int argc, char** argv)
>>  		{ "vxlan-gpe-port",		1, 0, 0 },
>>  		{ "mlockall",			0, 0, 0 },
>>  		{ "no-mlockall",		0, 0, 0 },
>> +		{ "noisy-tx-sw-buffer-size",	1, 0, 0 },
>> +		{ "noisy-tx-sw-buffer-flushtime",1, 0, 0 },
>> +		{ "noisy-lkup-memory",		1, 0, 0 },
>> +		{ "noisy-lkup-num-writes",	1, 0, 0 },
>> +		{ "noisy-lkup-num-reads",	1, 0, 0 },
>> +		{ "noisy-lkup-num-reads-writes",1, 0, 0 },
>>  		{ 0, 0, 0, 0 },
>>  	};
>>
>> @@ -1145,6 +1151,60 @@ launch_args_parse(int argc, char** argv)
>>  				do_mlockall = 1;
>>  			if (!strcmp(lgopts[opt_idx].name, "no-mlockall"))
>>  				do_mlockall = 0;
>> +			if (!strcmp(lgopts[opt_idx].name,
>> +				    "noisy-tx-sw-buffer-size")) {
>> +				n = atoi(optarg);
>> +				if (n >= 0)
>> +					noisy_tx_sw_bufsz = n;
>> +				else
>> +					rte_exit(EXIT_FAILURE,
>> +						"noisy-tx-sw-buffer-size must be >= 0\n");
>> +			}
>> +			if (!strcmp(lgopts[opt_idx].name,
>> +				    "noisy-tx-sw-buffer-flushtime")) {
>> +				n = atoi(optarg);
>> +				if (n >= 0)
>> +					noisy_tx_sw_buf_flush_time = n;
>> +				else
>> +					rte_exit(EXIT_FAILURE,
>> +						 "noisy-tx-sw-buffer-flushtime must be >= 0\n");
>> +			}
>> +			if (!strcmp(lgopts[opt_idx].name,
>> +				    "noisy-lkup-memory")) {
>> +				n = atoi(optarg);
>> +				if (n > 0)
>
>I thought this and below ones would also be '>=' also?

I wasn't sure if it's a good idea or if I should choose a different
default... and then forgot about it. sorry. I checked that it's safe
to allow 0, will keep the defaults at 0 and allow it to be set here.

Thanks for the review Kevin!

regards,
Jens