* [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet
@ 2014-08-25 6:12 Cunming Liang
2014-08-25 6:12 ` [dpdk-dev] [PATCH 1/5] app/test: unit test for rx and tx cycles/packet Cunming Liang
` (6 more replies)
0 siblings, 7 replies; 65+ messages in thread
From: Cunming Liang @ 2014-08-25 6:12 UTC (permalink / raw)
To: dev
It provides unit test to measure cycles/packet in NIC loopback mode.
It simply gives the average cycles of IO used per packet without test equipment.
When doing the test, make sure the link is UP.
There's two stream control mode support, one is continues, another is burst.
The former continues to forward the injected packets until reaching a certain amount of number.
The latter one stop when all the injected packets are received.
In burst stream, now measure two situations, with or without desc. cache conflict.
By default, it runs in continues stream mode to measure the whole rxtx.
Usage Example:
1. Run unit test app in interactive mode
app/test -c f -n 4 -- -i
2. Set stream control mode, by default is continuous
set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit]
3. If choose continuous stream, there are another two options can configure
3.1 choose rx/tx pair, default is vector
set_rxtx_mode [vector|scalar|full|hybrid]
Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without INC_VEC=y in config
3.2 choose the area of masurement, default is rxtx
set_rxtx_anchor [rxtx|rxonly|txonly]
4. Run and wait for the result
pmd_perf_autotest
For who simply just want to see how much cycles cost per packet.
Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it.
Nothing else needs to configure.
Using other options when you understand and what to measures more.
Cunming Liang (5):
app/test: unit test for rx and tx cycles/packet
app/test: measure standalone rx or tx cycles/packet
ixgbe/vpmd: add fix to store unaligned mbuf point array
app/test: add unit test to measure RX burst cycles
app/test: allow to create packets in different sizes
app/test/Makefile | 1 +
app/test/commands.c | 114 +++++
app/test/packet_burst_generator.c | 26 +-
app/test/packet_burst_generator.h | 11 +-
app/test/test.h | 8 +
app/test/test_link_bonding.c | 39 +-
app/test/test_pmd_perf.c | 918 ++++++++++++++++++++++++++++++++++
lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 +
lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 4 +-
9 files changed, 1092 insertions(+), 35 deletions(-)
create mode 100644 app/test/test_pmd_perf.c
--
1.8.1.4
^ permalink raw reply [flat|nested] 65+ messages in thread* [dpdk-dev] [PATCH 1/5] app/test: unit test for rx and tx cycles/packet 2014-08-25 6:12 [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Cunming Liang @ 2014-08-25 6:12 ` Cunming Liang 2014-08-25 6:12 ` [dpdk-dev] [PATCH 2/5] app/test: measure standalone rx or " Cunming Liang ` (5 subsequent siblings) 6 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-08-25 6:12 UTC (permalink / raw) To: dev Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/Makefile | 1 + app/test/commands.c | 41 +++ app/test/packet_burst_generator.c | 4 +- app/test/test.h | 6 + app/test/test_pmd_perf.c | 620 ++++++++++++++++++++++++++++++++++++ lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 6 files changed, 676 insertions(+), 2 deletions(-) create mode 100644 app/test/test_pmd_perf.c diff --git a/app/test/Makefile b/app/test/Makefile index 0024737..8418b2e 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -104,6 +104,7 @@ SRCS-$(CONFIG_RTE_APP_TEST) += test_distributor_perf.c SRCS-$(CONFIG_RTE_APP_TEST) += test_devargs.c SRCS-$(CONFIG_RTE_APP_TEST) += virtual_pmd.c SRCS-$(CONFIG_RTE_APP_TEST) += packet_burst_generator.c +SRCS-$(CONFIG_RTE_APP_TEST) += test_pmd_perf.c ifeq ($(CONFIG_RTE_APP_TEST),y) SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c diff --git a/app/test/commands.c b/app/test/commands.c index 5f23420..d0e583e 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -204,6 +204,8 @@ static void cmd_autotest_parsed(void *parsed_result, if (!strcmp(res->autotest, "kvargs_autotest")) ret |= test_kvargs(); #endif /* RTE_LIBRTE_KVARGS */ + if (!strcmp(res->autotest, "pmd_perf_autotest")) + ret = test_pmd_perf(); if (ret == 0) printf("Test OK\n"); @@ -251,6 +253,7 @@ cmdline_parse_token_string_t cmd_autotest_autotest = #ifdef RTE_LIBRTE_KVARGS "kvargs_autotest#" #endif + "pmd_perf_autotest#" "common_autotest#" "distributor_autotest#distributor_perf_autotest"); @@ -451,12 +454,50 @@ cmdline_parse_inst_t cmd_quit = { /****************/ +/****************/ + +struct cmd_set_rxtx_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t mode; +}; + +static void cmd_set_rxtx_parsed(void *parsed_result, struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_result *res = parsed_result; + if (test_set_rxtx_conf(res->mode) < 0) + cmdline_printf(cl, "Cannot find such mode\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, set, + "set_rxtx_mode"); + +cmdline_parse_token_string_t cmd_set_rxtx_mode = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, mode, NULL); + +cmdline_parse_inst_t cmd_set_rxtx = { + .f = cmd_set_rxtx_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx routine: " + "set_rxtx <mode>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_set, + (void *)&cmd_set_rxtx_mode, + NULL, + }, +}; + +/****************/ + + cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, (cmdline_parse_inst_t *)&cmd_dump, (cmdline_parse_inst_t *)&cmd_dump_one, (cmdline_parse_inst_t *)&cmd_set_ring, (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_set_rxtx, NULL, }; diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c index 5d539f1..943ea98 100644 --- a/app/test/packet_burst_generator.c +++ b/app/test/packet_burst_generator.c @@ -190,12 +190,12 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, */ #define RTE_MAX_SEGS_PER_PKT 255 /**< pkt.nb_segs is a 8-bit unsigned char. */ -#define TXONLY_DEF_PACKET_LEN 64 +#define TXONLY_DEF_PACKET_LEN 60 #define TXONLY_DEF_PACKET_LEN_128 128 uint16_t tx_pkt_length = TXONLY_DEF_PACKET_LEN; uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT] = { - TXONLY_DEF_PACKET_LEN_128, + TXONLY_DEF_PACKET_LEN, }; uint8_t tx_pkt_nb_segs = 1; diff --git a/app/test/test.h b/app/test/test.h index 181c38e..ad01eec 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -123,6 +123,9 @@ int unit_test_suite_runner(struct unit_test_suite *suite); #define RECURSIVE_ENV_VAR "RTE_TEST_RECURSIVE" +#include <cmdline_parse.h> +#include <cmdline_parse_string.h> + extern const char *prgname; int main(int argc, char **argv); @@ -180,6 +183,9 @@ int test_distributor_perf(void); int test_kvargs(void); int test_devargs(void); int test_link_bonding(void); +int test_pmd_perf(void); +int test_set_rxtx_conf(cmdline_fixed_string_t mode); + int test_pci_run; diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c new file mode 100644 index 0000000..ee527d3 --- /dev/null +++ b/app/test/test_pmd_perf.c @@ -0,0 +1,620 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + + +#include <stdio.h> +#include <inttypes.h> +#include <signal.h> +#include <unistd.h> +#include <rte_cycles.h> +#include <rte_ethdev.h> +#include <rte_byteorder.h> +#include "packet_burst_generator.h" +#include "test.h" + +#define NB_ETHPORTS_USED (1) +#define NB_SOCKETS (2) +#define MEMPOOL_CACHE_SIZE 250 +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MAX_PKT_BURST (32) +#define RTE_TEST_RX_DESC_DEFAULT (128) +#define RTE_TEST_TX_DESC_DEFAULT (512) +#define RTE_PORT_ALL (~(uint8_t)0x0) + +/* how long test would take at full line rate */ +#define RTE_TEST_DURATION (2) + +/* + * RX and TX Prefetch, Host, and Write-back threshold values should be + * carefully set for optimal performance. Consult the network + * controller's datasheet and supporting DPDK documentation for guidance + * on how these parameters should be set. + */ +#define RX_PTHRESH 8 /**< Default values of RX prefetch threshold reg. */ +#define RX_HTHRESH 8 /**< Default values of RX host threshold reg. */ +#define RX_WTHRESH 0 /**< Default values of RX write-back threshold reg. */ + +/* + * These default values are optimized for use with the Intel(R) 82599 10 GbE + * Controller and the DPDK ixgbe PMD. Consider using other values for other + * network controllers and/or network drivers. + */ +#define TX_PTHRESH 32 /**< Default values of TX prefetch threshold reg. */ +#define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */ +#define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */ + + +#define NB_MBUF RTE_MAX( \ + (unsigned)(nb_ports*nb_rx_queue*RTE_TEST_RX_DESC_DEFAULT + \ + nb_ports*nb_lcores*MAX_PKT_BURST + \ + nb_ports*nb_tx_queue*RTE_TEST_TX_DESC_DEFAULT + \ + nb_lcores*MEMPOOL_CACHE_SIZE), \ + (unsigned)8192) + + +static struct rte_mempool *mbufpool[NB_SOCKETS]; +/* ethernet addresses of ports */ +static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS]; + +static struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .hw_vlan_strip = 0, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 1, /* enable loopback */ +}; + +static struct rte_eth_rxconf rx_conf = { + .rx_thresh = { + .pthresh = RX_PTHRESH, + .hthresh = RX_HTHRESH, + .wthresh = RX_WTHRESH, + }, + .rx_free_thresh = 32, +}; + +static struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = TX_PTHRESH, + .hthresh = TX_HTHRESH, + .wthresh = TX_WTHRESH, + }, + .tx_free_thresh = 32, /* Use PMD default values */ + .tx_rs_thresh = 32, /* Use PMD default values */ + .txq_flags = (ETH_TXQ_FLAGS_NOMULTSEGS | + ETH_TXQ_FLAGS_NOVLANOFFL | + ETH_TXQ_FLAGS_NOXSUMSCTP | + ETH_TXQ_FLAGS_NOXSUMUDP | + ETH_TXQ_FLAGS_NOXSUMTCP) +}; + +enum { + LCORE_INVALID = 0, + LCORE_AVAIL, + LCORE_USED, +}; + +struct lcore_conf { + uint8_t status; + uint8_t socketid; + uint16_t nb_ports; + uint8_t portlist[RTE_MAX_ETHPORTS]; +} __rte_cache_aligned; + +struct lcore_conf lcore_conf[RTE_MAX_LCORE]; + +static uint64_t link_mbps; + +/* Check the link status of all ports in up to 3s, and print them finally */ +static void +check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +{ +#define CHECK_INTERVAL 100 /* 100ms */ +#define MAX_CHECK_TIME 30 /* 3s (30 * 100ms) in total */ + uint8_t portid, count, all_ports_up, print_flag = 0; + struct rte_eth_link link; + + printf("Checking link statuses...\n"); + fflush(stdout); + for (count = 0; count <= MAX_CHECK_TIME; count++) { + all_ports_up = 1; + for (portid = 0; portid < port_num; portid++) { + if ((port_mask & (1 << portid)) == 0) + continue; + memset(&link, 0, sizeof(link)); + rte_eth_link_get_nowait(portid, &link); + /* print link status if flag set */ + if (print_flag == 1) { + if (link.link_status) { + printf("Port %d Link Up - speed %u " + "Mbps - %s\n", (uint8_t)portid, + (unsigned)link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + if (link_mbps == 0) + link_mbps = link.link_speed; + } else + printf("Port %d Link Down\n", + (uint8_t)portid); + continue; + } + /* clear all_ports_up flag if any link down */ + if (link.link_status == 0) { + all_ports_up = 0; + break; + } + } + /* after finally printing all link status, get out */ + if (print_flag == 1) + break; + + if (all_ports_up == 0) { + fflush(stdout); + rte_delay_ms(CHECK_INTERVAL); + } + + /* set the print_flag if all ports up or timeout */ + if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) + print_flag = 1; + } +} + +static void +print_ethaddr(const char *name, const struct ether_addr *eth_addr) +{ + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, + eth_addr->addr_bytes[0], + eth_addr->addr_bytes[1], + eth_addr->addr_bytes[2], + eth_addr->addr_bytes[3], + eth_addr->addr_bytes[4], + eth_addr->addr_bytes[5]); +} + +static int +init_traffic(struct rte_mempool *mp, + struct rte_mbuf **pkts_burst, uint32_t burst_size) +{ + struct ether_hdr pkt_eth_hdr; + struct ipv4_hdr pkt_ipv4_hdr; + struct udp_hdr pkt_udp_hdr; + uint32_t pktlen; + static uint8_t src_mac[] = { 0x00, 0xFF, 0xAA, 0xFF, 0xAA, 0xFF }; + static uint8_t dst_mac[] = { 0x00, 0xAA, 0xFF, 0xAA, 0xFF, 0xAA }; + + + initialize_eth_header(&pkt_eth_hdr, + (struct ether_addr *)src_mac, + (struct ether_addr *)dst_mac, 0, 0); + pkt_eth_hdr.ether_type = rte_cpu_to_be_16(ETHER_TYPE_IPv4); + + pktlen = initialize_ipv4_header(&pkt_ipv4_hdr, + IPV4_ADDR(10, 0, 0, 1), + IPV4_ADDR(10, 0, 0, 2), 26); + printf("IPv4 pktlen %u\n", pktlen); + + pktlen = initialize_udp_header(&pkt_udp_hdr, 0, 0, 18); + + printf("UDP pktlen %u\n", pktlen); + + return generate_packet_burst(mp, pkts_burst, &pkt_eth_hdr, + 0, &pkt_ipv4_hdr, 1, + &pkt_udp_hdr, burst_size); +} + +static int +init_lcores(void) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + lcore_conf[lcore_id].socketid = + rte_lcore_to_socket_id(lcore_id); + if (rte_lcore_is_enabled(lcore_id) == 0) { + lcore_conf[lcore_id].status = LCORE_INVALID; + continue; + } else + lcore_conf[lcore_id].status = LCORE_AVAIL; + } + return 0; +} + +static int +init_mbufpool(unsigned nb_mbuf) +{ + int socketid; + unsigned lcore_id; + char s[64]; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + socketid = rte_lcore_to_socket_id(lcore_id); + if (socketid >= NB_SOCKETS) { + rte_exit(EXIT_FAILURE, + "Socket %d of lcore %u is out of range %d\n", + socketid, lcore_id, NB_SOCKETS); + } + if (mbufpool[socketid] == NULL) { + snprintf(s, sizeof(s), "mbuf_pool_%d", socketid); + mbufpool[socketid] = + rte_mempool_create(s, nb_mbuf, MBUF_SIZE, + MEMPOOL_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + socketid, 0); + if (mbufpool[socketid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot init mbuf pool on socket %d\n", + socketid); + else + printf("Allocated mbuf pool on socket %d\n", + socketid); + } + } + return 0; +} + +static uint16_t +alloc_lcore(uint16_t socketid) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (LCORE_AVAIL != lcore_conf[lcore_id].status || + lcore_conf[lcore_id].socketid != socketid || + lcore_id == rte_get_master_lcore()) + continue; + lcore_conf[lcore_id].status = LCORE_USED; + lcore_conf[lcore_id].nb_ports = 0; + return lcore_id; + } + + return (uint16_t)-1; +} + +volatile uint64_t stop; +uint64_t count; +uint64_t drop; +uint64_t idle; + +static void +reset_count(void) +{ + count = 0; + drop = 0; + idle = 0; +} + +static void +stats_display(uint8_t port_id) +{ + struct rte_eth_stats stats; + rte_eth_stats_get(port_id, &stats); + + printf(" RX-packets: %-10"PRIu64" RX-missed: %-10"PRIu64" RX-bytes: " + "%-"PRIu64"\n", + stats.ipackets, stats.imissed, stats.ibytes); + printf(" RX-badcrc: %-10"PRIu64" RX-badlen: %-10"PRIu64" RX-errors: " + "%-"PRIu64"\n", + stats.ibadcrc, stats.ibadlen, stats.ierrors); + printf(" RX-nombuf: %-10"PRIu64"\n", + stats.rx_nombuf); + printf(" TX-packets: %-10"PRIu64" TX-errors: %-10"PRIu64" TX-bytes: " + "%-"PRIu64"\n", + stats.opackets, stats.oerrors, stats.obytes); +} + +static void +signal_handler(int signum) +{ + /* When we receive a USR1 signal, print stats */ + if (signum == SIGUSR1) { + printf("Force Stop!\n"); + stop = 1; + } + if (signum == SIGUSR2) + stats_display(0); +} + +#define MAX_TRAFIC_BURST (4096) +struct rte_mbuf *tx_burst[MAX_TRAFIC_BURST]; + +/* main processing loop */ +static int +main_loop(__rte_unused void *args) +{ +#define PACKET_SIZE 64 +#define FRAME_GAP 12 +#define MAC_PREAMBLE 8 + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + unsigned lcore_id; + unsigned i, portid, nb_rx = 0, nb_tx = 0; + struct lcore_conf *conf; + uint64_t prev_tsc, cur_tsc; + int pkt_per_port; + uint64_t packets_per_second, total_packets; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; + + int idx = 0; + for (i = 0; i < conf->nb_ports; i++) { + int num = pkt_per_port; + portid = conf->portlist[i]; + printf("inject %d packet to port %d\n", num, portid); + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + nb_tx = rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + num -= nb_tx; + idx += nb_tx; + } + } + printf("Total packets inject to prime ports = %u\n", idx); + + packets_per_second = (link_mbps * 1000 * 1000) / + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); + printf("Each port will do %"PRIu64" packets per second\n", + +packets_per_second); + + total_packets = RTE_TEST_DURATION * conf->nb_ports * packets_per_second; + printf("Test will stop after at least %"PRIu64" packets received\n", + + total_packets); + + prev_tsc = rte_rdtsc(); + + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_packets)) + break; + } + + cur_tsc = rte_rdtsc(); + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + int nb_free = pkt_per_port; + do { /* dry out */ + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + nb_tx = 0; + while (nb_tx < nb_rx) + rte_pktmbuf_free(pkts_burst[nb_tx++]); + nb_free -= nb_rx; + } while (nb_free != 0); + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); + } + + if (count == 0) + return -1; + + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); + + return 0; +} + +int +test_pmd_perf(void) +{ + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; + uint16_t portid; + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; + int socketid = -1; + int ret; + + printf("Start PMD RXTX cycles cost test.\n"); + + signal(SIGUSR1, signal_handler); + signal(SIGUSR2, signal_handler); + + nb_ports = rte_eth_dev_count(); + if (nb_ports < NB_ETHPORTS_USED) { + printf("At least %u port(s) used for perf. test\n", + NB_ETHPORTS_USED); + return -1; + } + + if (nb_ports > RTE_MAX_ETHPORTS) + nb_ports = RTE_MAX_ETHPORTS; + + nb_lcores = rte_lcore_count(); + + memset(lcore_conf, 0, sizeof(lcore_conf)); + init_lcores(); + + init_mbufpool(NB_MBUF); + + reset_count(); + num = 0; + for (portid = 0; portid < nb_ports; portid++) { + if (socketid == -1) { + socketid = rte_eth_dev_socket_id(portid); + slave_id = alloc_lcore(socketid); + if (slave_id == (uint16_t)-1) { + printf("No avail lcore to run test\n"); + return -1; + } + printf("Performance test runs on lcore %u socket %u\n", + slave_id, socketid); + } + + if (socketid != rte_eth_dev_socket_id(portid)) { + printf("Skip port %d\n", portid); + continue; + } + + /* port configure */ + ret = rte_eth_dev_configure(portid, nb_rx_queue, + nb_tx_queue, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot configure device: err=%d, port=%d\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); + printf("Port %u ", portid); + print_ethaddr("Address:", &ports_eth_addr[portid]); + printf("\n"); + + /* tx queue setup */ + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + socketid, &tx_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup: err=%d, " + "port=%d\n", ret, portid); + + /* rx queue steup */ + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, + socketid, &rx_conf, + mbufpool[socketid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," + "port=%d\n", ret, portid); + + /* Start device */ + stop = 0; + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start: err=%d, port=%d\n", + ret, portid); + + /* always eanble promiscuous */ + rte_eth_promiscuous_enable(portid); + + lcore_conf[slave_id].portlist[num++] = portid; + lcore_conf[slave_id].nb_ports++; + } + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); + + rte_eal_remote_launch(main_loop, NULL, slave_id); + if (rte_eal_wait_lcore(slave_id) < 0) + return -1; + + /* port tear down */ + for (portid = 0; portid < nb_ports; portid++) { + if (socketid != rte_eth_dev_socket_id(portid)) + continue; + + rte_eth_dev_stop(portid); + } + + return 0; +} + +int +test_set_rxtx_conf(cmdline_fixed_string_t mode) +{ + printf("mode is %s\n", mode); + + if (!strcmp(mode, "vector")) { + /* vector rx, tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "scalar")) { + /* bulk alloc rx, simple tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 128; + tx_conf.tx_free_thresh = 128; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "hybrid")) { + /* bulk alloc rx, vector tx + * when vec macro not define, + * using the same rx/tx as scalar + */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "full")) { + /* full feature rx,tx pair */ + tx_conf.txq_flags = 0x0; /* must condition */ + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 1; /* must condition */ + return 0; + } + + return -1; +} diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c index 59122a1..c581dbf 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c +++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c @@ -1582,6 +1582,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev) ixgbe_dev_clear_queues(dev); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + /* Clear recorded link status */ memset(&link, 0, sizeof(link)); rte_ixgbe_dev_atomic_write_link_status(dev, &link); @@ -2853,6 +2856,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) */ ixgbevf_set_vfta_all(dev,0); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + ixgbe_dev_clear_queues(dev); } -- 1.8.1.4 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH 2/5] app/test: measure standalone rx or tx cycles/packet 2014-08-25 6:12 [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Cunming Liang 2014-08-25 6:12 ` [dpdk-dev] [PATCH 1/5] app/test: unit test for rx and tx cycles/packet Cunming Liang @ 2014-08-25 6:12 ` Cunming Liang 2014-08-25 6:12 ` [dpdk-dev] [PATCH 3/5] ixgbe/vpmd: add fix to store unaligned mbuf point array Cunming Liang ` (4 subsequent siblings) 6 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-08-25 6:12 UTC (permalink / raw) To: dev Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/commands.c | 38 ++++++++++ app/test/test.h | 1 + app/test/test_pmd_perf.c | 175 +++++++++++++++++++++++++++++++++++++++-------- 3 files changed, 185 insertions(+), 29 deletions(-) diff --git a/app/test/commands.c b/app/test/commands.c index d0e583e..8721e25 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -490,6 +490,43 @@ cmdline_parse_inst_t cmd_set_rxtx = { /****************/ +/****************/ + +struct cmd_set_rxtx_anchor { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_anchor_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_anchor *res = parsed_result; + if (test_set_rxtx_anchor(res->type) < 0) + cmdline_printf(cl, "Cannot find such anchor\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, set, + "set_rxtx_anchor"); + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_anchor = { + .f = cmd_set_rxtx_anchor_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx anchor: " + "set_rxtx_anchor <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_anchor_set, + (void *)&cmd_set_rxtx_anchor_type, + NULL, + }, +}; + +/****************/ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, @@ -498,6 +535,7 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_ring, (cmdline_parse_inst_t *)&cmd_quit, (cmdline_parse_inst_t *)&cmd_set_rxtx, + (cmdline_parse_inst_t *)&cmd_set_rxtx_anchor, NULL, }; diff --git a/app/test/test.h b/app/test/test.h index ad01eec..c24ce9f 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -185,6 +185,7 @@ int test_devargs(void); int test_link_bonding(void); int test_pmd_perf(void); int test_set_rxtx_conf(cmdline_fixed_string_t mode); +int test_set_rxtx_anchor(cmdline_fixed_string_t type); int test_pci_run; diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c index ee527d3..5d02fac 100644 --- a/app/test/test_pmd_perf.c +++ b/app/test/test_pmd_perf.c @@ -364,6 +364,125 @@ signal_handler(int signum) #define MAX_TRAFIC_BURST (4096) struct rte_mbuf *tx_burst[MAX_TRAFIC_BURST]; +uint64_t (*do_measure)(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts); + +static uint64_t +measure_rxtx(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t prev_tsc, cur_tsc; + + prev_tsc = rte_rdtsc(); + + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + cur_tsc = rte_rdtsc(); + + return cur_tsc - prev_tsc; +} + +static uint64_t +measure_rxonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + + cur_tsc = rte_rdtsc(); + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + diff_tsc += rte_rdtsc() - cur_tsc; + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +static uint64_t +measure_txonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + printf("do tx measure\n"); + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + + cur_tsc = rte_rdtsc(); + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + diff_tsc += rte_rdtsc() - cur_tsc; + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + /* main processing loop */ static int main_loop(__rte_unused void *args) @@ -375,8 +494,8 @@ main_loop(__rte_unused void *args) unsigned lcore_id; unsigned i, portid, nb_rx = 0, nb_tx = 0; struct lcore_conf *conf; - uint64_t prev_tsc, cur_tsc; int pkt_per_port; + uint64_t diff_tsc; uint64_t packets_per_second, total_packets; lcore_id = rte_lcore_id(); @@ -410,32 +529,7 @@ main_loop(__rte_unused void *args) printf("Test will stop after at least %"PRIu64" packets received\n", + total_packets); - prev_tsc = rte_rdtsc(); - - while (likely(!stop)) { - for (i = 0; i < conf->nb_ports; i++) { - portid = conf->portlist[i]; - nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, - pkts_burst, MAX_PKT_BURST); - if (unlikely(nb_rx == 0)) { - idle++; - continue; - } - - count += nb_rx; - nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); - if (unlikely(nb_tx < nb_rx)) { - drop += (nb_rx - nb_tx); - do { - rte_pktmbuf_free(pkts_burst[nb_tx]); - } while (++nb_tx < nb_rx); - } - } - if (unlikely(count >= total_packets)) - break; - } - - cur_tsc = rte_rdtsc(); + diff_tsc = do_measure(conf, pkts_burst, total_packets); for (i = 0; i < conf->nb_ports; i++) { portid = conf->portlist[i]; @@ -455,7 +549,7 @@ main_loop(__rte_unused void *args) return -1; printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); - printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); + printf("Result: %ld cycles per packet\n", diff_tsc / count); return 0; } @@ -559,6 +653,10 @@ test_pmd_perf(void) init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); + /* do both rxtx by default */ + if (NULL == do_measure) + do_measure = measure_rxtx; + rte_eal_remote_launch(main_loop, NULL, slave_id); if (rte_eal_wait_lcore(slave_id) < 0) return -1; @@ -577,7 +675,7 @@ test_pmd_perf(void) int test_set_rxtx_conf(cmdline_fixed_string_t mode) { - printf("mode is %s\n", mode); + printf("mode switch to %s\n", mode); if (!strcmp(mode, "vector")) { /* vector rx, tx */ @@ -618,3 +716,22 @@ test_set_rxtx_conf(cmdline_fixed_string_t mode) return -1; } + +int +test_set_rxtx_anchor(cmdline_fixed_string_t type) +{ + printf("type switch to %s\n", type); + + if (!strcmp(type, "rxtx")) { + do_measure = measure_rxtx; + return 0; + } else if (!strcmp(type, "rxonly")) { + do_measure = measure_rxonly; + return 0; + } else if (!strcmp(type, "txonly")) { + do_measure = measure_txonly; + return 0; + } + + return -1; +} -- 1.8.1.4 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH 3/5] ixgbe/vpmd: add fix to store unaligned mbuf point array 2014-08-25 6:12 [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Cunming Liang 2014-08-25 6:12 ` [dpdk-dev] [PATCH 1/5] app/test: unit test for rx and tx cycles/packet Cunming Liang 2014-08-25 6:12 ` [dpdk-dev] [PATCH 2/5] app/test: measure standalone rx or " Cunming Liang @ 2014-08-25 6:12 ` Cunming Liang 2014-08-25 6:12 ` [dpdk-dev] [PATCH 4/5] app/test: add unit test to measure RX burst cycles Cunming Liang ` (3 subsequent siblings) 6 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-08-25 6:12 UTC (permalink / raw) To: dev Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c b/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c index fe39ca2..92e07de 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c @@ -277,7 +277,7 @@ ixgbe_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, descs[3] = _mm_loadu_si128((__m128i *)(rxdp + 3)); /* B.2 copy 2 mbuf point into rx_pkts */ - _mm_store_si128((__m128i *)&rx_pkts[pos], mbp1); + _mm_storeu_si128((__m128i *)&rx_pkts[pos], mbp1); /* B.1 load 1 mbuf point */ mbp2 = _mm_loadu_si128((__m128i *)&sw_ring[pos+2]); @@ -288,7 +288,7 @@ ixgbe_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, descs[0] = _mm_loadu_si128((__m128i *)(rxdp)); /* B.2 copy 2 mbuf point into rx_pkts */ - _mm_store_si128((__m128i *)&rx_pkts[pos+2], mbp2); + _mm_storeu_si128((__m128i *)&rx_pkts[pos+2], mbp2); /* avoid compiler reorder optimization */ rte_compiler_barrier(); -- 1.8.1.4 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH 4/5] app/test: add unit test to measure RX burst cycles 2014-08-25 6:12 [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Cunming Liang ` (2 preceding siblings ...) 2014-08-25 6:12 ` [dpdk-dev] [PATCH 3/5] ixgbe/vpmd: add fix to store unaligned mbuf point array Cunming Liang @ 2014-08-25 6:12 ` Cunming Liang 2014-08-25 6:12 ` [dpdk-dev] [PATCH 5/5] app/test: allow to create packets in different sizes Cunming Liang ` (2 subsequent siblings) 6 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-08-25 6:12 UTC (permalink / raw) To: dev Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- app/test/commands.c | 43 +++++++++- app/test/test.h | 1 + app/test/test_pmd_perf.c | 214 +++++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 237 insertions(+), 21 deletions(-) diff --git a/app/test/commands.c b/app/test/commands.c index 8721e25..f222398 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -454,8 +454,6 @@ cmdline_parse_inst_t cmd_quit = { /****************/ -/****************/ - struct cmd_set_rxtx_result { cmdline_fixed_string_t set; cmdline_fixed_string_t mode; @@ -490,8 +488,6 @@ cmdline_parse_inst_t cmd_set_rxtx = { /****************/ -/****************/ - struct cmd_set_rxtx_anchor { cmdline_fixed_string_t set; cmdline_fixed_string_t type; @@ -528,6 +524,44 @@ cmdline_parse_inst_t cmd_set_rxtx_anchor = { /****************/ +/* for stream control */ +struct cmd_set_rxtx_sc { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_sc_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_sc *res = parsed_result; + if (test_set_rxtx_sc(res->type) < 0) + cmdline_printf(cl, "Cannot find such stream control\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_sc_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, set, + "set_rxtx_sc"); + +cmdline_parse_token_string_t cmd_set_rxtx_sc_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_sc = { + .f = cmd_set_rxtx_sc_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx stream control: " + "set_rxtx_sc <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_sc_set, + (void *)&cmd_set_rxtx_sc_type, + NULL, + }, +}; + +/****************/ + + cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, (cmdline_parse_inst_t *)&cmd_dump, @@ -536,6 +570,7 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_quit, (cmdline_parse_inst_t *)&cmd_set_rxtx, (cmdline_parse_inst_t *)&cmd_set_rxtx_anchor, + (cmdline_parse_inst_t *)&cmd_set_rxtx_sc, NULL, }; diff --git a/app/test/test.h b/app/test/test.h index c24ce9f..4c10afe 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -186,6 +186,7 @@ int test_link_bonding(void); int test_pmd_perf(void); int test_set_rxtx_conf(cmdline_fixed_string_t mode); int test_set_rxtx_anchor(cmdline_fixed_string_t type); +int test_set_rxtx_sc(cmdline_fixed_string_t type); int test_pci_run; diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c index 5d02fac..e11fa41 100644 --- a/app/test/test_pmd_perf.c +++ b/app/test/test_pmd_perf.c @@ -39,6 +39,8 @@ #include <rte_cycles.h> #include <rte_ethdev.h> #include <rte_byteorder.h> +#include <rte_atomic.h> +#include <rte_malloc.h> #include "packet_burst_generator.h" #include "test.h" @@ -73,13 +75,15 @@ #define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */ #define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */ +#define MAX_TRAFFIC_BURST 2048 #define NB_MBUF RTE_MAX( \ - (unsigned)(nb_ports*nb_rx_queue*RTE_TEST_RX_DESC_DEFAULT + \ - nb_ports*nb_lcores*MAX_PKT_BURST + \ - nb_ports*nb_tx_queue*RTE_TEST_TX_DESC_DEFAULT + \ - nb_lcores*MEMPOOL_CACHE_SIZE), \ - (unsigned)8192) + (unsigned)(nb_ports*nb_rx_queue*nb_rxd + \ + nb_ports*nb_lcores*MAX_PKT_BURST + \ + nb_ports*nb_tx_queue*nb_txd + \ + nb_lcores*MEMPOOL_CACHE_SIZE + \ + nb_ports*MAX_TRAFFIC_BURST), \ + (unsigned)8192) static struct rte_mempool *mbufpool[NB_SOCKETS]; @@ -147,6 +151,14 @@ struct lcore_conf lcore_conf[RTE_MAX_LCORE]; static uint64_t link_mbps; +enum { + SC_CONTINUOUS = 0, + SC_BURST_POLL_FIRST, + SC_BURST_XMIT_FIRST, +}; + +static uint32_t sc_flag; + /* Check the link status of all ports in up to 3s, and print them finally */ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) @@ -361,8 +373,7 @@ signal_handler(int signum) stats_display(0); } -#define MAX_TRAFIC_BURST (4096) -struct rte_mbuf *tx_burst[MAX_TRAFIC_BURST]; +struct rte_mbuf **tx_burst; uint64_t (*do_measure)(struct lcore_conf *conf, struct rte_mbuf *pkts_burst[], @@ -503,7 +514,7 @@ main_loop(__rte_unused void *args) if (conf->status != LCORE_USED) return 0; - pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; + pkt_per_port = MAX_TRAFFIC_BURST; int idx = 0; for (i = 0; i < conf->nb_ports; i++) { @@ -554,12 +565,137 @@ main_loop(__rte_unused void *args) return 0; } +rte_atomic64_t start; + +/* main processing loop */ +static inline int +poll_burst(void *args) +{ +#define MAX_IDLE (1000) + unsigned lcore_id; + struct rte_mbuf **pkts_burst; + uint64_t diff_tsc, cur_tsc; + uint16_t next[RTE_MAX_ETHPORTS]; + struct lcore_conf *conf; + uint32_t pkt_per_port = *((uint32_t *)args); + unsigned i, portid, nb_rx = 0; + uint64_t total; + uint64_t timeout = MAX_IDLE; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + total = pkt_per_port * conf->nb_ports; + printf("start to receive total expect %ld\n", total); + + pkts_burst = (struct rte_mbuf **) + rte_calloc_socket("poll_burst", + total, sizeof(void *), + CACHE_LINE_SIZE, conf->socketid); + if (!pkts_burst) + return -1; + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + next[portid] = i * pkt_per_port; + } + + while (!rte_atomic64_read(&start)) + ; + + cur_tsc = rte_rdtsc(); + while (total) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + &pkts_burst[next[portid]], + MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + timeout--; + if (unlikely(timeout == 0)) + goto timeout; + continue; + } + next[portid] += nb_rx; + total -= nb_rx; + } + } +timeout: + diff_tsc = rte_rdtsc() - cur_tsc; + + printf("%ld packets lost, IDLE %ld times\n", total, MAX_IDLE - timeout); + + /* clean up */ + total = pkt_per_port * conf->nb_ports - total; + for (i = 0; i < total; i++) + rte_pktmbuf_free(pkts_burst[i]); + + rte_free(pkts_burst); + + return diff_tsc / total; +} + +/* main processing loop */ +static int +exec_burst(uint32_t flags, int lcore) +{ + unsigned i, portid, nb_tx = 0; + struct lcore_conf *conf; + uint32_t pkt_per_port; + int num, idx = 0; + int diff_tsc; + + conf = &lcore_conf[lcore]; + + pkt_per_port = MAX_TRAFFIC_BURST; + num = pkt_per_port; + + rte_atomic64_init(&start); + + /* start polling thread, but not actually poll yet */ + rte_eal_remote_launch(poll_burst, + (void *)&pkt_per_port, lcore); + + /* Only when polling first */ + if (flags == SC_BURST_POLL_FIRST) + rte_atomic64_set(&start, 1); + + /* start xmit */ + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + idx += nb_tx; + } + num -= nb_tx; + } + + sleep(5); + + /* only when polling second */ + if (flags == SC_BURST_XMIT_FIRST) + rte_atomic64_set(&start, 1); + + /* wait for polling finished */ + diff_tsc = rte_eal_wait_lcore(lcore); + if (diff_tsc < 0) + return -1; + + printf("Result: %d cycles per packet\n", diff_tsc); + + return 0; +} + int test_pmd_perf(void) { uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; - uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; - uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; + uint16_t nb_rxd = MAX_TRAFFIC_BURST; + uint16_t nb_txd = MAX_TRAFFIC_BURST; uint16_t portid; uint16_t nb_rx_queue = 1, nb_tx_queue = 1; int socketid = -1; @@ -587,6 +723,12 @@ test_pmd_perf(void) init_mbufpool(NB_MBUF); + if (sc_flag == SC_CONTINUOUS) { + nb_rxd = RTE_TEST_RX_DESC_DEFAULT; + nb_txd = RTE_TEST_TX_DESC_DEFAULT; + } + printf("CONFIG RXD=%d TXD=%d\n", nb_rxd, nb_txd); + reset_count(); num = 0; for (portid = 0; portid < nb_ports; portid++) { @@ -651,15 +793,34 @@ test_pmd_perf(void) } check_all_ports_link_status(nb_ports, RTE_PORT_ALL); - init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); + if (tx_burst == NULL) { + tx_burst = (struct rte_mbuf **) + rte_calloc_socket("tx_buff", + MAX_TRAFFIC_BURST * nb_ports, + sizeof(void *), + CACHE_LINE_SIZE, socketid); + if (!tx_burst) + return -1; + } - /* do both rxtx by default */ - if (NULL == do_measure) - do_measure = measure_rxtx; + init_traffic(mbufpool[socketid], + tx_burst, MAX_TRAFFIC_BURST * nb_ports); - rte_eal_remote_launch(main_loop, NULL, slave_id); - if (rte_eal_wait_lcore(slave_id) < 0) - return -1; + printf("Generate %d packets @socket %d\n", + MAX_TRAFFIC_BURST * nb_ports, socketid); + + if (sc_flag == SC_CONTINUOUS) { + /* do both rxtx by default */ + if (NULL == do_measure) + do_measure = measure_rxtx; + + rte_eal_remote_launch(main_loop, NULL, slave_id); + + if (rte_eal_wait_lcore(slave_id) < 0) + return -1; + } else if (sc_flag == SC_BURST_POLL_FIRST || + sc_flag == SC_BURST_XMIT_FIRST) + exec_burst(sc_flag, slave_id); /* port tear down */ for (portid = 0; portid < nb_ports; portid++) { @@ -735,3 +896,22 @@ test_set_rxtx_anchor(cmdline_fixed_string_t type) return -1; } + +int +test_set_rxtx_sc(cmdline_fixed_string_t type) +{ + printf("stream control switch to %s\n", type); + + if (!strcmp(type, "continuous")) { + sc_flag = SC_CONTINUOUS; + return 0; + } else if (!strcmp(type, "poll_before_xmit")) { + sc_flag = SC_BURST_POLL_FIRST; + return 0; + } else if (!strcmp(type, "poll_after_xmit")) { + sc_flag = SC_BURST_XMIT_FIRST; + return 0; + } + + return -1; +} -- 1.8.1.4 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH 5/5] app/test: allow to create packets in different sizes 2014-08-25 6:12 [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Cunming Liang ` (3 preceding siblings ...) 2014-08-25 6:12 ` [dpdk-dev] [PATCH 4/5] app/test: add unit test to measure RX burst cycles Cunming Liang @ 2014-08-25 6:12 ` Cunming Liang 2014-09-03 1:46 ` [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Zhan, Zhaochen 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 0/4] " Cunming Liang 6 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-08-25 6:12 UTC (permalink / raw) To: dev adding support to allow packet burst generator to create packets of differenct sizes Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- app/test/packet_burst_generator.c | 26 +++++++++----------------- app/test/packet_burst_generator.h | 11 ++++++++--- app/test/test_link_bonding.c | 39 ++++++++++++++++++++++++++------------- app/test/test_pmd_perf.c | 3 ++- 4 files changed, 45 insertions(+), 34 deletions(-) diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c index 943ea98..bf78362 100644 --- a/app/test/packet_burst_generator.c +++ b/app/test/packet_burst_generator.c @@ -190,20 +190,12 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, */ #define RTE_MAX_SEGS_PER_PKT 255 /**< pkt.nb_segs is a 8-bit unsigned char. */ -#define TXONLY_DEF_PACKET_LEN 60 -#define TXONLY_DEF_PACKET_LEN_128 128 - -uint16_t tx_pkt_length = TXONLY_DEF_PACKET_LEN; -uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT] = { - TXONLY_DEF_PACKET_LEN, -}; - -uint8_t tx_pkt_nb_segs = 1; - int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst) + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs) { int i, nb_pkt = 0; size_t eth_hdr_size; @@ -220,9 +212,9 @@ nomore_mbuf: break; } - pkt->pkt.data_len = tx_pkt_seg_lengths[0]; + pkt->pkt.data_len = pkt_len; pkt_seg = pkt; - for (i = 1; i < tx_pkt_nb_segs; i++) { + for (i = 1; i < nb_pkt_segs; i++) { pkt_seg->pkt.next = rte_pktmbuf_alloc(mp); if (pkt_seg->pkt.next == NULL) { pkt->pkt.nb_segs = i; @@ -230,7 +222,7 @@ nomore_mbuf: goto nomore_mbuf; } pkt_seg = pkt_seg->pkt.next; - pkt_seg->pkt.data_len = tx_pkt_seg_lengths[i]; + pkt_seg->pkt.data_len = pkt_len; } pkt_seg->pkt.next = NULL; /* Last segment of packet. */ @@ -258,8 +250,8 @@ nomore_mbuf: * Complete first mbuf of packet and append it to the * burst of packets to be transmitted. */ - pkt->pkt.nb_segs = tx_pkt_nb_segs; - pkt->pkt.pkt_len = tx_pkt_length; + pkt->pkt.nb_segs = nb_pkt_segs; + pkt->pkt.pkt_len = pkt_len; pkt->pkt.vlan_macip.f.l2_len = eth_hdr_size; if (ipv4) { diff --git a/app/test/packet_burst_generator.h b/app/test/packet_burst_generator.h index 5b3cd6c..fe992ac 100644 --- a/app/test/packet_burst_generator.h +++ b/app/test/packet_burst_generator.h @@ -47,10 +47,13 @@ extern "C" { #define IPV4_ADDR(a, b, c, d)(((a & 0xff) << 24) | ((b & 0xff) << 16) | \ ((c & 0xff) << 8) | (d & 0xff)) +#define PACKET_BURST_GEN_PKT_LEN 60 +#define PACKET_BURST_GEN_PKT_LEN_128 128 void initialize_eth_header(struct ether_hdr *eth_hdr, struct ether_addr *src_mac, - struct ether_addr *dst_mac, uint8_t vlan_enabled, uint16_t van_id); + struct ether_addr *dst_mac, uint8_t vlan_enabled, + uint16_t van_id); uint16_t initialize_udp_header(struct udp_hdr *udp_hdr, uint16_t src_port, @@ -67,8 +70,10 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst); + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs); #ifdef __cplusplus } diff --git a/app/test/test_link_bonding.c b/app/test/test_link_bonding.c index 5c1303e..758f1b1 100644 --- a/app/test/test_link_bonding.c +++ b/app/test/test_link_bonding.c @@ -1189,9 +1189,12 @@ generate_test_burst(struct rte_mbuf **pkts_burst, uint16_t burst_size, } /* Generate burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, vlan, ip_hdr, ipv4, - test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, + vlan, ip_hdr, ipv4, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN_128, 1); if (generated_burst_size != burst_size) { printf("Failed to generate packet burst"); return -1; @@ -1778,9 +1781,12 @@ test_activebackup_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; @@ -2432,8 +2438,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 1 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[0][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[0]) != burst_size[0]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[0], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[0]) return -1; initialize_eth_header(test_params->pkt_eth_hdr, @@ -2441,8 +2449,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 2 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[1][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[1]) != burst_size[1]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[1], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[1]) return -1; /* Send burst 1 on bonded port */ @@ -3259,9 +3269,12 @@ test_broadcast_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c index e11fa41..ea35b38 100644 --- a/app/test/test_pmd_perf.c +++ b/app/test/test_pmd_perf.c @@ -253,7 +253,8 @@ init_traffic(struct rte_mempool *mp, return generate_packet_burst(mp, pkts_burst, &pkt_eth_hdr, 0, &pkt_ipv4_hdr, 1, - &pkt_udp_hdr, burst_size); + &pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); } static int -- 1.8.1.4 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet 2014-08-25 6:12 [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Cunming Liang ` (4 preceding siblings ...) 2014-08-25 6:12 ` [dpdk-dev] [PATCH 5/5] app/test: allow to create packets in different sizes Cunming Liang @ 2014-09-03 1:46 ` Zhan, Zhaochen 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 0/4] " Cunming Liang 6 siblings, 0 replies; 65+ messages in thread From: Zhan, Zhaochen @ 2014-09-03 1:46 UTC (permalink / raw) To: Liang, Cunming, dev > It provides unit test to measure cycles/packet in NIC loopback mode. > It simply gives the average cycles of IO used per packet without test equipment. > When doing the test, make sure the link is UP. Tested-by: Zhaochen Zhan <zhaochen.zhan@intel.com> This patch has been verified on ixgbe and it is ready to be integrated to dpdk.org. For e1000 and i40e, lacking loopback supporting, this patch can't support them for now. ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v2 0/4] app/test: unit test to measure cycles per packet 2014-08-25 6:12 [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Cunming Liang ` (5 preceding siblings ...) 2014-09-03 1:46 ` [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Zhan, Zhaochen @ 2014-10-10 12:29 ` Cunming Liang 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet Cunming Liang ` (5 more replies) 6 siblings, 6 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-10 12:29 UTC (permalink / raw) To: dev v2 update: Rebase code to the latest master branch. -------------- Tested-by: Zhaochen Zhan <zhaochen.zhan@intel.com> This patch has been verified on ixgbe and it is ready to be integrated to dpdk.org. For e1000 and i40e, lacking loopback supporting, this patch can't support them for now. It provides unit test to measure cycles/packet in NIC loopback mode. It simply gives the average cycles of IO used per packet without test equipment. When doing the test, make sure the link is UP. There's two stream control mode support, one is continues, another is burst. The former continues to forward the injected packets until reaching a certain amount of number. The latter one stop when all the injected packets are received. In burst stream, now measure two situations, with or without desc. cache conflict. By default, it runs in continues stream mode to measure the whole rxtx. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] 3. If choose continuous stream, there are another two options can configure 3.1 choose rx/tx pair, default is vector set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without INC_VEC=y in config 3.2 choose the area of masurement, default is rxtx set_rxtx_anchor [rxtx|rxonly|txonly] 4. Run and wait for the result pmd_perf_autotest For who simply just want to see how much cycles cost per packet. Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. Nothing else needs to configure. Using other options when you understand and what to measures more. *** BLURB HERE *** Cunming Liang (4): app/test: unit test for rx and tx cycles/packet app/test: measure standalone rx or tx cycles/packet app/test: add unit test to measure RX burst cycles app/test: allow to create packets in different sizes app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/packet_burst_generator.c | 26 +- app/test/packet_burst_generator.h | 11 +- app/test/test.h | 6 + app/test/test_link_bonding.c | 39 +- app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 8 files changed, 1089 insertions(+), 33 deletions(-) create mode 100644 app/test/test_pmd_perf.c -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 0/4] " Cunming Liang @ 2014-10-10 12:29 ` Cunming Liang 2014-10-10 17:52 ` Neil Horman 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 2/4] app/test: measure standalone rx or " Cunming Liang ` (4 subsequent siblings) 5 siblings, 1 reply; 65+ messages in thread From: Cunming Liang @ 2014-10-10 12:29 UTC (permalink / raw) To: dev It provides unit test to measure cycles/packet in NIC loopback mode. It simply gives the average cycles of IO used per packet without test equipment. When doing the test, make sure the link is UP. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Run and wait for the result pmd_perf_autotest There's option to choose rx/tx pair, default is vector. set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without INC_VEC=y in config Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/Makefile | 1 + app/test/commands.c | 38 +++ app/test/packet_burst_generator.c | 4 +- app/test/test.h | 4 + app/test/test_pmd_perf.c | 626 +++++++++++++++++++++++++++++++++++ lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 6 files changed, 677 insertions(+), 2 deletions(-) create mode 100644 app/test/test_pmd_perf.c diff --git a/app/test/Makefile b/app/test/Makefile index 6af6d76..ebfa0ba 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c SRCS-y += test_ring.c SRCS-y += test_ring_perf.c +SRCS-y += test_pmd_perf.c ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) SRCS-y += test_table.c diff --git a/app/test/commands.c b/app/test/commands.c index a9e36b1..f1e746e 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -310,12 +310,50 @@ cmdline_parse_inst_t cmd_quit = { /****************/ +/****************/ + +struct cmd_set_rxtx_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t mode; +}; + +static void cmd_set_rxtx_parsed(void *parsed_result, struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_result *res = parsed_result; + if (test_set_rxtx_conf(res->mode) < 0) + cmdline_printf(cl, "Cannot find such mode\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, set, + "set_rxtx_mode"); + +cmdline_parse_token_string_t cmd_set_rxtx_mode = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, mode, NULL); + +cmdline_parse_inst_t cmd_set_rxtx = { + .f = cmd_set_rxtx_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx routine: " + "set_rxtx <mode>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_set, + (void *)&cmd_set_rxtx_mode, + NULL, + }, +}; + +/****************/ + + cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, (cmdline_parse_inst_t *)&cmd_dump, (cmdline_parse_inst_t *)&cmd_dump_one, (cmdline_parse_inst_t *)&cmd_set_ring, (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_set_rxtx, NULL, }; diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c index 9e747a4..9100b1d 100644 --- a/app/test/packet_burst_generator.c +++ b/app/test/packet_burst_generator.c @@ -191,12 +191,12 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, */ #define RTE_MAX_SEGS_PER_PKT 255 /**< pkt.nb_segs is a 8-bit unsigned char. */ -#define TXONLY_DEF_PACKET_LEN 64 +#define TXONLY_DEF_PACKET_LEN 60 #define TXONLY_DEF_PACKET_LEN_128 128 uint16_t tx_pkt_length = TXONLY_DEF_PACKET_LEN; uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT] = { - TXONLY_DEF_PACKET_LEN_128, + TXONLY_DEF_PACKET_LEN, }; uint8_t tx_pkt_nb_segs = 1; diff --git a/app/test/test.h b/app/test/test.h index 98ab804..76033a8 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -125,6 +125,9 @@ int unit_test_suite_runner(struct unit_test_suite *suite); #define RECURSIVE_ENV_VAR "RTE_TEST_RECURSIVE" +#include <cmdline_parse.h> +#include <cmdline_parse_string.h> + extern const char *prgname; int commands_init(void); @@ -137,6 +140,7 @@ int test_pci_run; int test_mp_secondary(void); int test_ivshmem(void); +int test_set_rxtx_conf(cmdline_fixed_string_t mode); typedef int (test_callback)(void); TAILQ_HEAD(test_commands_list, test_command); diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c new file mode 100644 index 0000000..bca4a9a --- /dev/null +++ b/app/test/test_pmd_perf.c @@ -0,0 +1,626 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + + +#include <stdio.h> +#include <inttypes.h> +#include <signal.h> +#include <unistd.h> +#include <rte_cycles.h> +#include <rte_ethdev.h> +#include <rte_byteorder.h> +#include "packet_burst_generator.h" +#include "test.h" + +#define NB_ETHPORTS_USED (1) +#define NB_SOCKETS (2) +#define MEMPOOL_CACHE_SIZE 250 +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MAX_PKT_BURST (32) +#define RTE_TEST_RX_DESC_DEFAULT (128) +#define RTE_TEST_TX_DESC_DEFAULT (512) +#define RTE_PORT_ALL (~(uint8_t)0x0) + +/* how long test would take at full line rate */ +#define RTE_TEST_DURATION (2) + +/* + * RX and TX Prefetch, Host, and Write-back threshold values should be + * carefully set for optimal performance. Consult the network + * controller's datasheet and supporting DPDK documentation for guidance + * on how these parameters should be set. + */ +#define RX_PTHRESH 8 /**< Default values of RX prefetch threshold reg. */ +#define RX_HTHRESH 8 /**< Default values of RX host threshold reg. */ +#define RX_WTHRESH 0 /**< Default values of RX write-back threshold reg. */ + +/* + * These default values are optimized for use with the Intel(R) 82599 10 GbE + * Controller and the DPDK ixgbe PMD. Consider using other values for other + * network controllers and/or network drivers. + */ +#define TX_PTHRESH 32 /**< Default values of TX prefetch threshold reg. */ +#define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */ +#define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */ + + +#define NB_MBUF RTE_MAX( \ + (unsigned)(nb_ports*nb_rx_queue*RTE_TEST_RX_DESC_DEFAULT + \ + nb_ports*nb_lcores*MAX_PKT_BURST + \ + nb_ports*nb_tx_queue*RTE_TEST_TX_DESC_DEFAULT + \ + nb_lcores*MEMPOOL_CACHE_SIZE), \ + (unsigned)8192) + + +static struct rte_mempool *mbufpool[NB_SOCKETS]; +/* ethernet addresses of ports */ +static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS]; + +static struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .hw_vlan_strip = 0, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 1, /* enable loopback */ +}; + +static struct rte_eth_rxconf rx_conf = { + .rx_thresh = { + .pthresh = RX_PTHRESH, + .hthresh = RX_HTHRESH, + .wthresh = RX_WTHRESH, + }, + .rx_free_thresh = 32, +}; + +static struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = TX_PTHRESH, + .hthresh = TX_HTHRESH, + .wthresh = TX_WTHRESH, + }, + .tx_free_thresh = 32, /* Use PMD default values */ + .tx_rs_thresh = 32, /* Use PMD default values */ + .txq_flags = (ETH_TXQ_FLAGS_NOMULTSEGS | + ETH_TXQ_FLAGS_NOVLANOFFL | + ETH_TXQ_FLAGS_NOXSUMSCTP | + ETH_TXQ_FLAGS_NOXSUMUDP | + ETH_TXQ_FLAGS_NOXSUMTCP) +}; + +enum { + LCORE_INVALID = 0, + LCORE_AVAIL, + LCORE_USED, +}; + +struct lcore_conf { + uint8_t status; + uint8_t socketid; + uint16_t nb_ports; + uint8_t portlist[RTE_MAX_ETHPORTS]; +} __rte_cache_aligned; + +struct lcore_conf lcore_conf[RTE_MAX_LCORE]; + +static uint64_t link_mbps; + +/* Check the link status of all ports in up to 3s, and print them finally */ +static void +check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +{ +#define CHECK_INTERVAL 100 /* 100ms */ +#define MAX_CHECK_TIME 30 /* 3s (30 * 100ms) in total */ + uint8_t portid, count, all_ports_up, print_flag = 0; + struct rte_eth_link link; + + printf("Checking link statuses...\n"); + fflush(stdout); + for (count = 0; count <= MAX_CHECK_TIME; count++) { + all_ports_up = 1; + for (portid = 0; portid < port_num; portid++) { + if ((port_mask & (1 << portid)) == 0) + continue; + memset(&link, 0, sizeof(link)); + rte_eth_link_get_nowait(portid, &link); + /* print link status if flag set */ + if (print_flag == 1) { + if (link.link_status) { + printf("Port %d Link Up - speed %u " + "Mbps - %s\n", (uint8_t)portid, + (unsigned)link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + if (link_mbps == 0) + link_mbps = link.link_speed; + } else + printf("Port %d Link Down\n", + (uint8_t)portid); + continue; + } + /* clear all_ports_up flag if any link down */ + if (link.link_status == 0) { + all_ports_up = 0; + break; + } + } + /* after finally printing all link status, get out */ + if (print_flag == 1) + break; + + if (all_ports_up == 0) { + fflush(stdout); + rte_delay_ms(CHECK_INTERVAL); + } + + /* set the print_flag if all ports up or timeout */ + if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) + print_flag = 1; + } +} + +static void +print_ethaddr(const char *name, const struct ether_addr *eth_addr) +{ + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, + eth_addr->addr_bytes[0], + eth_addr->addr_bytes[1], + eth_addr->addr_bytes[2], + eth_addr->addr_bytes[3], + eth_addr->addr_bytes[4], + eth_addr->addr_bytes[5]); +} + +static int +init_traffic(struct rte_mempool *mp, + struct rte_mbuf **pkts_burst, uint32_t burst_size) +{ + struct ether_hdr pkt_eth_hdr; + struct ipv4_hdr pkt_ipv4_hdr; + struct udp_hdr pkt_udp_hdr; + uint32_t pktlen; + static uint8_t src_mac[] = { 0x00, 0xFF, 0xAA, 0xFF, 0xAA, 0xFF }; + static uint8_t dst_mac[] = { 0x00, 0xAA, 0xFF, 0xAA, 0xFF, 0xAA }; + + + initialize_eth_header(&pkt_eth_hdr, + (struct ether_addr *)src_mac, + (struct ether_addr *)dst_mac, 0, 0); + pkt_eth_hdr.ether_type = rte_cpu_to_be_16(ETHER_TYPE_IPv4); + + pktlen = initialize_ipv4_header(&pkt_ipv4_hdr, + IPV4_ADDR(10, 0, 0, 1), + IPV4_ADDR(10, 0, 0, 2), 26); + printf("IPv4 pktlen %u\n", pktlen); + + pktlen = initialize_udp_header(&pkt_udp_hdr, 0, 0, 18); + + printf("UDP pktlen %u\n", pktlen); + + return generate_packet_burst(mp, pkts_burst, &pkt_eth_hdr, + 0, &pkt_ipv4_hdr, 1, + &pkt_udp_hdr, burst_size); +} + +static int +init_lcores(void) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + lcore_conf[lcore_id].socketid = + rte_lcore_to_socket_id(lcore_id); + if (rte_lcore_is_enabled(lcore_id) == 0) { + lcore_conf[lcore_id].status = LCORE_INVALID; + continue; + } else + lcore_conf[lcore_id].status = LCORE_AVAIL; + } + return 0; +} + +static int +init_mbufpool(unsigned nb_mbuf) +{ + int socketid; + unsigned lcore_id; + char s[64]; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + socketid = rte_lcore_to_socket_id(lcore_id); + if (socketid >= NB_SOCKETS) { + rte_exit(EXIT_FAILURE, + "Socket %d of lcore %u is out of range %d\n", + socketid, lcore_id, NB_SOCKETS); + } + if (mbufpool[socketid] == NULL) { + snprintf(s, sizeof(s), "mbuf_pool_%d", socketid); + mbufpool[socketid] = + rte_mempool_create(s, nb_mbuf, MBUF_SIZE, + MEMPOOL_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + socketid, 0); + if (mbufpool[socketid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot init mbuf pool on socket %d\n", + socketid); + else + printf("Allocated mbuf pool on socket %d\n", + socketid); + } + } + return 0; +} + +static uint16_t +alloc_lcore(uint16_t socketid) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (LCORE_AVAIL != lcore_conf[lcore_id].status || + lcore_conf[lcore_id].socketid != socketid || + lcore_id == rte_get_master_lcore()) + continue; + lcore_conf[lcore_id].status = LCORE_USED; + lcore_conf[lcore_id].nb_ports = 0; + return lcore_id; + } + + return (uint16_t)-1; +} + +volatile uint64_t stop; +uint64_t count; +uint64_t drop; +uint64_t idle; + +static void +reset_count(void) +{ + count = 0; + drop = 0; + idle = 0; +} + +static void +stats_display(uint8_t port_id) +{ + struct rte_eth_stats stats; + rte_eth_stats_get(port_id, &stats); + + printf(" RX-packets: %-10"PRIu64" RX-missed: %-10"PRIu64" RX-bytes: " + "%-"PRIu64"\n", + stats.ipackets, stats.imissed, stats.ibytes); + printf(" RX-badcrc: %-10"PRIu64" RX-badlen: %-10"PRIu64" RX-errors: " + "%-"PRIu64"\n", + stats.ibadcrc, stats.ibadlen, stats.ierrors); + printf(" RX-nombuf: %-10"PRIu64"\n", + stats.rx_nombuf); + printf(" TX-packets: %-10"PRIu64" TX-errors: %-10"PRIu64" TX-bytes: " + "%-"PRIu64"\n", + stats.opackets, stats.oerrors, stats.obytes); +} + +static void +signal_handler(int signum) +{ + /* When we receive a USR1 signal, print stats */ + if (signum == SIGUSR1) { + printf("Force Stop!\n"); + stop = 1; + } + if (signum == SIGUSR2) + stats_display(0); +} + +#define MAX_TRAFIC_BURST (4096) +struct rte_mbuf *tx_burst[MAX_TRAFIC_BURST]; + +/* main processing loop */ +static int +main_loop(__rte_unused void *args) +{ +#define PACKET_SIZE 64 +#define FRAME_GAP 12 +#define MAC_PREAMBLE 8 + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + unsigned lcore_id; + unsigned i, portid, nb_rx = 0, nb_tx = 0; + struct lcore_conf *conf; + uint64_t prev_tsc, cur_tsc; + int pkt_per_port; + uint64_t packets_per_second, total_packets; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; + + int idx = 0; + for (i = 0; i < conf->nb_ports; i++) { + int num = pkt_per_port; + portid = conf->portlist[i]; + printf("inject %d packet to port %d\n", num, portid); + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + nb_tx = rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + num -= nb_tx; + idx += nb_tx; + } + } + printf("Total packets inject to prime ports = %u\n", idx); + + packets_per_second = (link_mbps * 1000 * 1000) / + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); + printf("Each port will do %"PRIu64" packets per second\n", + +packets_per_second); + + total_packets = RTE_TEST_DURATION * conf->nb_ports * packets_per_second; + printf("Test will stop after at least %"PRIu64" packets received\n", + + total_packets); + + prev_tsc = rte_rdtsc(); + + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_packets)) + break; + } + + cur_tsc = rte_rdtsc(); + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + int nb_free = pkt_per_port; + do { /* dry out */ + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + nb_tx = 0; + while (nb_tx < nb_rx) + rte_pktmbuf_free(pkts_burst[nb_tx++]); + nb_free -= nb_rx; + } while (nb_free != 0); + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); + } + + if (count == 0) + return -1; + + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); + + return 0; +} + +static int +test_pmd_perf(void) +{ + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; + uint16_t portid; + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; + int socketid = -1; + int ret; + + printf("Start PMD RXTX cycles cost test.\n"); + + signal(SIGUSR1, signal_handler); + signal(SIGUSR2, signal_handler); + + nb_ports = rte_eth_dev_count(); + if (nb_ports < NB_ETHPORTS_USED) { + printf("At least %u port(s) used for perf. test\n", + NB_ETHPORTS_USED); + return -1; + } + + if (nb_ports > RTE_MAX_ETHPORTS) + nb_ports = RTE_MAX_ETHPORTS; + + nb_lcores = rte_lcore_count(); + + memset(lcore_conf, 0, sizeof(lcore_conf)); + init_lcores(); + + init_mbufpool(NB_MBUF); + + reset_count(); + num = 0; + for (portid = 0; portid < nb_ports; portid++) { + if (socketid == -1) { + socketid = rte_eth_dev_socket_id(portid); + slave_id = alloc_lcore(socketid); + if (slave_id == (uint16_t)-1) { + printf("No avail lcore to run test\n"); + return -1; + } + printf("Performance test runs on lcore %u socket %u\n", + slave_id, socketid); + } + + if (socketid != rte_eth_dev_socket_id(portid)) { + printf("Skip port %d\n", portid); + continue; + } + + /* port configure */ + ret = rte_eth_dev_configure(portid, nb_rx_queue, + nb_tx_queue, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot configure device: err=%d, port=%d\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); + printf("Port %u ", portid); + print_ethaddr("Address:", &ports_eth_addr[portid]); + printf("\n"); + + /* tx queue setup */ + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + socketid, &tx_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup: err=%d, " + "port=%d\n", ret, portid); + + /* rx queue steup */ + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, + socketid, &rx_conf, + mbufpool[socketid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," + "port=%d\n", ret, portid); + + /* Start device */ + stop = 0; + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start: err=%d, port=%d\n", + ret, portid); + + /* always eanble promiscuous */ + rte_eth_promiscuous_enable(portid); + + lcore_conf[slave_id].portlist[num++] = portid; + lcore_conf[slave_id].nb_ports++; + } + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); + + rte_eal_remote_launch(main_loop, NULL, slave_id); + if (rte_eal_wait_lcore(slave_id) < 0) + return -1; + + /* port tear down */ + for (portid = 0; portid < nb_ports; portid++) { + if (socketid != rte_eth_dev_socket_id(portid)) + continue; + + rte_eth_dev_stop(portid); + } + + return 0; +} + +int +test_set_rxtx_conf(cmdline_fixed_string_t mode) +{ + printf("mode is %s\n", mode); + + if (!strcmp(mode, "vector")) { + /* vector rx, tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "scalar")) { + /* bulk alloc rx, simple tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 128; + tx_conf.tx_free_thresh = 128; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "hybrid")) { + /* bulk alloc rx, vector tx + * when vec macro not define, + * using the same rx/tx as scalar + */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "full")) { + /* full feature rx,tx pair */ + tx_conf.txq_flags = 0x0; /* must condition */ + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 1; /* must condition */ + return 0; + } + + return -1; +} + +static struct test_command pmd_perf_cmd = { + .command = "pmd_perf_autotest", + .callback = test_pmd_perf, +}; +REGISTER_TEST_COMMAND(pmd_perf_cmd); diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c index 215e563..8b87f98 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c +++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c @@ -1586,6 +1586,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev) ixgbe_dev_clear_queues(dev); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + /* Clear recorded link status */ memset(&link, 0, sizeof(link)); rte_ixgbe_dev_atomic_write_link_status(dev, &link); @@ -2852,6 +2855,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) */ ixgbevf_set_vfta_all(dev,0); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + ixgbe_dev_clear_queues(dev); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet Cunming Liang @ 2014-10-10 17:52 ` Neil Horman 2014-10-12 11:10 ` Liang, Cunming 0 siblings, 1 reply; 65+ messages in thread From: Neil Horman @ 2014-10-10 17:52 UTC (permalink / raw) To: Cunming Liang; +Cc: dev On Fri, Oct 10, 2014 at 08:29:58PM +0800, Cunming Liang wrote: > It provides unit test to measure cycles/packet in NIC loopback mode. > It simply gives the average cycles of IO used per packet without test equipment. > When doing the test, make sure the link is UP. > > Usage Example: > 1. Run unit test app in interactive mode > app/test -c f -n 4 -- -i > 2. Run and wait for the result > pmd_perf_autotest > > There's option to choose rx/tx pair, default is vector. > set_rxtx_mode [vector|scalar|full|hybrid] > Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without INC_VEC=y in config > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > Acked-by: Bruce Richardson <bruce.richardson@intel.com> Notes inline > --- > app/test/Makefile | 1 + > app/test/commands.c | 38 +++ > app/test/packet_burst_generator.c | 4 +- > app/test/test.h | 4 + > app/test/test_pmd_perf.c | 626 +++++++++++++++++++++++++++++++++++ > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > 6 files changed, 677 insertions(+), 2 deletions(-) > create mode 100644 app/test/test_pmd_perf.c > > diff --git a/app/test/Makefile b/app/test/Makefile > index 6af6d76..ebfa0ba 100644 > --- a/app/test/Makefile > +++ b/app/test/Makefile > @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c > > SRCS-y += test_ring.c > SRCS-y += test_ring_perf.c > +SRCS-y += test_pmd_perf.c > > ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) > SRCS-y += test_table.c > diff --git a/app/test/commands.c b/app/test/commands.c > index a9e36b1..f1e746e 100644 > --- a/app/test/commands.c > +++ b/app/test/commands.c > @@ -310,12 +310,50 @@ cmdline_parse_inst_t cmd_quit = { > > +#define NB_ETHPORTS_USED (1) > +#define NB_SOCKETS (2) > +#define MEMPOOL_CACHE_SIZE 250 > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) Don't you want to size this in accordance with the amount of data your sending (64 Bytes as noted above)? > +static void > +print_ethaddr(const char *name, const struct ether_addr *eth_addr) > +{ > + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, > + eth_addr->addr_bytes[0], > + eth_addr->addr_bytes[1], > + eth_addr->addr_bytes[2], > + eth_addr->addr_bytes[3], > + eth_addr->addr_bytes[4], > + eth_addr->addr_bytes[5]); > +} > + This was copieed from print_ethaddr. Seems like a good candidate for a common function in rte_ether.h > +} > + > +static void > +signal_handler(int signum) > +{ > + /* When we receive a USR1 signal, print stats */ I think you mean SIGUSR2, below, SIGUSR1 tears the test down and exits the program > + if (signum == SIGUSR1) { SIGINT instead. Thats the common practice. > + printf("Force Stop!\n"); > + stop = 1; > + } > + if (signum == SIGUSR2) > + stats_display(0); > +} > +/* main processing loop */ > +static int > +main_loop(__rte_unused void *args) > +{ > +#define PACKET_SIZE 64 > +#define FRAME_GAP 12 > +#define MAC_PREAMBLE 8 > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > + unsigned lcore_id; > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > + struct lcore_conf *conf; > + uint64_t prev_tsc, cur_tsc; > + int pkt_per_port; > + uint64_t packets_per_second, total_packets; > + > + lcore_id = rte_lcore_id(); > + conf = &lcore_conf[lcore_id]; > + if (conf->status != LCORE_USED) > + return 0; > + > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > + > + int idx = 0; > + for (i = 0; i < conf->nb_ports; i++) { > + int num = pkt_per_port; > + portid = conf->portlist[i]; > + printf("inject %d packet to port %d\n", num, portid); > + while (num) { > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > + nb_tx = rte_eth_tx_burst(portid, 0, > + &tx_burst[idx], nb_tx); > + num -= nb_tx; > + idx += nb_tx; > + } > + } > + printf("Total packets inject to prime ports = %u\n", idx); > + > + packets_per_second = (link_mbps * 1000 * 1000) / > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); > + printf("Each port will do %"PRIu64" packets per second\n", > + +packets_per_second); > + > + total_packets = RTE_TEST_DURATION * conf->nb_ports * packets_per_second; > + printf("Test will stop after at least %"PRIu64" packets received\n", > + + total_packets); > + > + prev_tsc = rte_rdtsc(); > + > + while (likely(!stop)) { > + for (i = 0; i < conf->nb_ports; i++) { > + portid = conf->portlist[i]; > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > + pkts_burst, MAX_PKT_BURST); > + if (unlikely(nb_rx == 0)) { > + idle++; > + continue; > + } > + > + count += nb_rx; Doesn't take into consideration error conditions. rte_eth_rx_burst can return -ENOTSUP > + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); Ditto with -ENOTSUP > + if (unlikely(nb_tx < nb_rx)) { What makes this unlikely? Seems like a perfectly reasonable condition to happen to me. If the network is busy, its completely likely that you will receive more frames than you send, if you elect to receive all frames. > + drop += (nb_rx - nb_tx); > + do { > + rte_pktmbuf_free(pkts_burst[nb_tx]); Defer this, it skews your timing > + } while (++nb_tx < nb_rx); > + } > + } > + if (unlikely(count >= total_packets)) > + break; Whats the reasoning here? Do you only ever expect to receive frames that you send? If so, seems like this should call for a big warning to get printed. > + } > + > + cur_tsc = rte_rdtsc(); > + > + for (i = 0; i < conf->nb_ports; i++) { > + portid = conf->portlist[i]; > + int nb_free = pkt_per_port; > + do { /* dry out */ > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > + pkts_burst, MAX_PKT_BURST); > + nb_tx = 0; > + while (nb_tx < nb_rx) > + rte_pktmbuf_free(pkts_burst[nb_tx++]); > + nb_free -= nb_rx; > + } while (nb_free != 0); > + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); > + } > + Whats the purpose of this? Are you trying to flush the device? Wouldn't it be enough just to stop the interface? > + if (count == 0) > + return -1; > + > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > + Bad math here. Theres no guarantee that the tsc hasn't wrapped (potentially more than once) depending on your test length. you need to check the tsc before and after each burst and record an average of deltas instead, accounting in each instance for the possibility of wrap. > + return 0; > +} > + > +static int > +test_pmd_perf(void) > +{ > + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; > + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; > + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; > + uint16_t portid; > + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; > + int socketid = -1; > + int ret; > + > + printf("Start PMD RXTX cycles cost test.\n"); > + > + signal(SIGUSR1, signal_handler); Again SIGINT here. > + signal(SIGUSR2, signal_handler); > + > + nb_ports = rte_eth_dev_count(); > + if (nb_ports < NB_ETHPORTS_USED) { > + printf("At least %u port(s) used for perf. test\n", > + NB_ETHPORTS_USED); > + return -1; > + } > + > + if (nb_ports > RTE_MAX_ETHPORTS) > + nb_ports = RTE_MAX_ETHPORTS; > + > + nb_lcores = rte_lcore_count(); > + > + memset(lcore_conf, 0, sizeof(lcore_conf)); > + init_lcores(); > + > + init_mbufpool(NB_MBUF); > + > + reset_count(); > + num = 0; > + for (portid = 0; portid < nb_ports; portid++) { > + if (socketid == -1) { > + socketid = rte_eth_dev_socket_id(portid); > + slave_id = alloc_lcore(socketid); > + if (slave_id == (uint16_t)-1) { > + printf("No avail lcore to run test\n"); > + return -1; > + } > + printf("Performance test runs on lcore %u socket %u\n", > + slave_id, socketid); > + } > + > + if (socketid != rte_eth_dev_socket_id(portid)) { > + printf("Skip port %d\n", portid); > + continue; > + } > + > + /* port configure */ > + ret = rte_eth_dev_configure(portid, nb_rx_queue, > + nb_tx_queue, &port_conf); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, > + "Cannot configure device: err=%d, port=%d\n", > + ret, portid); > + > + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); > + printf("Port %u ", portid); > + print_ethaddr("Address:", &ports_eth_addr[portid]); > + printf("\n"); > + > + /* tx queue setup */ > + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, > + socketid, &tx_conf); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, > + "rte_eth_tx_queue_setup: err=%d, " > + "port=%d\n", ret, portid); > + > + /* rx queue steup */ > + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, > + socketid, &rx_conf, > + mbufpool[socketid]); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," > + "port=%d\n", ret, portid); > + > + /* Start device */ > + stop = 0; > + ret = rte_eth_dev_start(portid); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, > + "rte_eth_dev_start: err=%d, port=%d\n", > + ret, portid); > + > + /* always eanble promiscuous */ > + rte_eth_promiscuous_enable(portid); > + > + lcore_conf[slave_id].portlist[num++] = portid; > + lcore_conf[slave_id].nb_ports++; > + } > + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > + > + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); > + > + rte_eal_remote_launch(main_loop, NULL, slave_id); > + if (rte_eal_wait_lcore(slave_id) < 0) > + return -1; > + > + /* port tear down */ > + for (portid = 0; portid < nb_ports; portid++) { > + if (socketid != rte_eth_dev_socket_id(portid)) > + continue; > + > + rte_eth_dev_stop(portid); > + } > + Clean up your allocated memory/lcores/etc? Neil ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-10 17:52 ` Neil Horman @ 2014-10-12 11:10 ` Liang, Cunming [not found] ` <E115CCD9D858EF4F90C690B0DCB4D8972262B720@IRSMSX108.ger.corp.intel.com> 2014-10-21 10:33 ` Neil Horman 0 siblings, 2 replies; 65+ messages in thread From: Liang, Cunming @ 2014-10-12 11:10 UTC (permalink / raw) To: Neil Horman; +Cc: dev Hi Neil, Very appreciate your comments. I add inline reply, will send v3 asap when we get alignment. BRs, Liang Cunming > -----Original Message----- > From: Neil Horman [mailto:nhorman@tuxdriver.com] > Sent: Saturday, October 11, 2014 1:52 AM > To: Liang, Cunming > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet > > On Fri, Oct 10, 2014 at 08:29:58PM +0800, Cunming Liang wrote: > > It provides unit test to measure cycles/packet in NIC loopback mode. > > It simply gives the average cycles of IO used per packet without test equipment. > > When doing the test, make sure the link is UP. > > > > Usage Example: > > 1. Run unit test app in interactive mode > > app/test -c f -n 4 -- -i > > 2. Run and wait for the result > > pmd_perf_autotest > > > > There's option to choose rx/tx pair, default is vector. > > set_rxtx_mode [vector|scalar|full|hybrid] > > Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without > INC_VEC=y in config > > > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > > Acked-by: Bruce Richardson <bruce.richardson@intel.com> > > Notes inline > > > --- > > app/test/Makefile | 1 + > > app/test/commands.c | 38 +++ > > app/test/packet_burst_generator.c | 4 +- > > app/test/test.h | 4 + > > app/test/test_pmd_perf.c | 626 > +++++++++++++++++++++++++++++++++++ > > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > > 6 files changed, 677 insertions(+), 2 deletions(-) > > create mode 100644 app/test/test_pmd_perf.c > > > > diff --git a/app/test/Makefile b/app/test/Makefile > > index 6af6d76..ebfa0ba 100644 > > --- a/app/test/Makefile > > +++ b/app/test/Makefile > > @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c > > > > SRCS-y += test_ring.c > > SRCS-y += test_ring_perf.c > > +SRCS-y += test_pmd_perf.c > > > > ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) > > SRCS-y += test_table.c > > diff --git a/app/test/commands.c b/app/test/commands.c > > index a9e36b1..f1e746e 100644 > > --- a/app/test/commands.c > > +++ b/app/test/commands.c > > @@ -310,12 +310,50 @@ cmdline_parse_inst_t cmd_quit = { > > > > +#define NB_ETHPORTS_USED (1) > > +#define NB_SOCKETS (2) > > +#define MEMPOOL_CACHE_SIZE 250 > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + > RTE_PKTMBUF_HEADROOM) > Don't you want to size this in accordance with the amount of data your sending > (64 Bytes as noted above)? [Liang, Cunming] The case is designed to measure small packet IO cost with normal mbuf size. Even if decreasing the size, it won't gain significant cycles. > > > +static void > > +print_ethaddr(const char *name, const struct ether_addr *eth_addr) > > +{ > > + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, > > + eth_addr->addr_bytes[0], > > + eth_addr->addr_bytes[1], > > + eth_addr->addr_bytes[2], > > + eth_addr->addr_bytes[3], > > + eth_addr->addr_bytes[4], > > + eth_addr->addr_bytes[5]); > > +} > > + > This was copieed from print_ethaddr. Seems like a good candidate for a common > function in rte_ether.h [Liang, Cunming] Agree with you, some of samples now use it with the same copy. I'll rework it. Adding 'ether_format_addr' in rte_ether.h only for format the 48bits address output. And leaving other prints for application customization. > > > > +} > > + > > +static void > > +signal_handler(int signum) > > +{ > > + /* When we receive a USR1 signal, print stats */ > I think you mean SIGUSR2, below, SIGUSR1 tears the test down and exits the > program [Liang, Cunming] Thanks, it's a typo. > > > + if (signum == SIGUSR1) { > SIGINT instead. Thats the common practice. [Liang, Cunming] I understood your opinion. The considerations I'm not using SIGINT instead are: 1. We unset ISIG in c_lflag of term. CRTL+C won't trigger SIGINT in command interactive. It always has to explicitly send signal. No matter SIGUSR1 or SIGINT. 2. By SIGINT semantic, expect to terminate the process. Here I expect to force stop this case, but still alive in command line. After it stopped, it can run again or start to run other test cases. So I keep SIGINT, SIGUSR1 in different behavior. 3. It should be rarely used. Only when exception timeout, I leave this backdoor for automation test control. For manual test, we can easily force kill the process. > > > + printf("Force Stop!\n"); > > + stop = 1; > > + } > > + if (signum == SIGUSR2) > > + stats_display(0); > > +} > > +/* main processing loop */ > > +static int > > +main_loop(__rte_unused void *args) > > +{ > > +#define PACKET_SIZE 64 > > +#define FRAME_GAP 12 > > +#define MAC_PREAMBLE 8 > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > + unsigned lcore_id; > > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > > + struct lcore_conf *conf; > > + uint64_t prev_tsc, cur_tsc; > > + int pkt_per_port; > > + uint64_t packets_per_second, total_packets; > > + > > + lcore_id = rte_lcore_id(); > > + conf = &lcore_conf[lcore_id]; > > + if (conf->status != LCORE_USED) > > + return 0; > > + > > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > > + > > + int idx = 0; > > + for (i = 0; i < conf->nb_ports; i++) { > > + int num = pkt_per_port; > > + portid = conf->portlist[i]; > > + printf("inject %d packet to port %d\n", num, portid); > > + while (num) { > > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > > + nb_tx = rte_eth_tx_burst(portid, 0, > > + &tx_burst[idx], nb_tx); > > + num -= nb_tx; > > + idx += nb_tx; > > + } > > + } > > + printf("Total packets inject to prime ports = %u\n", idx); > > + > > + packets_per_second = (link_mbps * 1000 * 1000) / > > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); > > + printf("Each port will do %"PRIu64" packets per second\n", > > + +packets_per_second); > > + > > + total_packets = RTE_TEST_DURATION * conf->nb_ports * > packets_per_second; > > + printf("Test will stop after at least %"PRIu64" packets received\n", > > + + total_packets); > > + > > + prev_tsc = rte_rdtsc(); > > + > > + while (likely(!stop)) { > > + for (i = 0; i < conf->nb_ports; i++) { > > + portid = conf->portlist[i]; > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > + pkts_burst, MAX_PKT_BURST); > > + if (unlikely(nb_rx == 0)) { > > + idle++; > > + continue; > > + } > > + > > + count += nb_rx; > Doesn't take into consideration error conditions. rte_eth_rx_burst can return > -ENOTSUP [Liang, Cunming] It returns -ENOTSUP when turning on ETHDEV_DEBUG CONFIG. The error is used to avoid no function call register. When ETHDEV_DEBUG turn off, the NULL function call cause segfault directly. So I think it's a library internal error. In such library exceptional case, I prefer not expecting sample/application to condition check library functional error. > > > + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); > Ditto with -ENOTSUP > > > + if (unlikely(nb_tx < nb_rx)) { > What makes this unlikely? Seems like a perfectly reasonable condition to happen > to me. If the network is busy, its completely likely that you will receive more > frames than you send, if you elect to receive all frames. [Liang, Cunming] For this case, NIC works in MAC loopback mode. It firstly injects numbers of packets to NIC. Then NIC will loopback all packets to ingress side. The code here will receive the packets, and send them back to NIC Packets looping inside all come from initial injection. As the total number of injected packets is much less than in-chip queue size, the tx egress queue shouldn't block desc. ring update. So when receiving packets in line rate and nb_tx < nb_rx, it means tx cannot archive line rate. When it happens, the cycles/packets result make no sense, as the bottle neck is NIC. The drop counter can record it. > > > + drop += (nb_rx - nb_tx); > > + do { > > + rte_pktmbuf_free(pkts_burst[nb_tx]); > Defer this, it skews your timing [Liang, Cunming] Agree with you, I ever thought about it. This test cases is designed to measure pure IO RX/TX routine. When doing tx burst with result nb_tx < nb_rx, we either repeat re-send or drop it. Each way introduces noise(adding additional control code), resending much times even cost more than free it. The cycles/packets is useful when there's no packet drop, otherwise it gives the hint where the problem comes from (by idle or drop). > > > + } while (++nb_tx < nb_rx); > > + } > > + } > > + if (unlikely(count >= total_packets)) > > + break; > Whats the reasoning here? Do you only ever expect to receive frames that you > send? If so, seems like this should call for a big warning to get printed. [Liang, Cunming] The loop exits when the pre-calculated total_packets are received. As the nb_rx is unpredictable, the packet counter may large equal than total_packets the last time. The reason unlikely used here is because the condition becomes true only the last time. > > > + } > > + > > + cur_tsc = rte_rdtsc(); > > + > > + for (i = 0; i < conf->nb_ports; i++) { > > + portid = conf->portlist[i]; > > + int nb_free = pkt_per_port; > > + do { /* dry out */ > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > + pkts_burst, MAX_PKT_BURST); > > + nb_tx = 0; > > + while (nb_tx < nb_rx) > > + rte_pktmbuf_free(pkts_burst[nb_tx++]); > > + nb_free -= nb_rx; > > + } while (nb_free != 0); > > + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); > > + } > > + > Whats the purpose of this? Are you trying to flush the device? Wouldn't it be > enough just to stop the interface? [Liang, Cunming] If we only run the cases once and exit, it doesn't matter. But it's designed to run multi-times without exit, for the purpose of warming up or for getting average number. So stopping device is not enough, we have to release the flying packets. > > > + if (count == 0) > > + return -1; > > + > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > > + > Bad math here. Theres no guarantee that the tsc hasn't wrapped (potentially > more than once) depending on your test length. you need to check the tsc before > and after each burst and record an average of deltas instead, accounting in each > instance for the possibility of wrap. [Liang, Cunming] I'm not sure catch your point correctly. I think both cur_tsc and prev_tsc are 64 bits width. For 3GHz, I think it won't wrapped so quick. As it's uint64_t, so even get wrapped, the delta should still be correct. > > > + return 0; > > +} > > + > > +static int > > +test_pmd_perf(void) > > +{ > > + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; > > + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; > > + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; > > + uint16_t portid; > > + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; > > + int socketid = -1; > > + int ret; > > + > > + printf("Start PMD RXTX cycles cost test.\n"); > > + > > + signal(SIGUSR1, signal_handler); > Again SIGINT here. > > > + signal(SIGUSR2, signal_handler); > > + > > + nb_ports = rte_eth_dev_count(); > > + if (nb_ports < NB_ETHPORTS_USED) { > > + printf("At least %u port(s) used for perf. test\n", > > + NB_ETHPORTS_USED); > > + return -1; > > + } > > + > > + if (nb_ports > RTE_MAX_ETHPORTS) > > + nb_ports = RTE_MAX_ETHPORTS; > > + > > + nb_lcores = rte_lcore_count(); > > + > > + memset(lcore_conf, 0, sizeof(lcore_conf)); > > + init_lcores(); > > + > > + init_mbufpool(NB_MBUF); > > + > > + reset_count(); > > + num = 0; > > + for (portid = 0; portid < nb_ports; portid++) { > > + if (socketid == -1) { > > + socketid = rte_eth_dev_socket_id(portid); > > + slave_id = alloc_lcore(socketid); > > + if (slave_id == (uint16_t)-1) { > > + printf("No avail lcore to run test\n"); > > + return -1; > > + } > > + printf("Performance test runs on lcore %u socket %u\n", > > + slave_id, socketid); > > + } > > + > > + if (socketid != rte_eth_dev_socket_id(portid)) { > > + printf("Skip port %d\n", portid); > > + continue; > > + } > > + > > + /* port configure */ > > + ret = rte_eth_dev_configure(portid, nb_rx_queue, > > + nb_tx_queue, &port_conf); > > + if (ret < 0) > > + rte_exit(EXIT_FAILURE, > > + "Cannot configure device: err=%d, port=%d\n", > > + ret, portid); > > + > > + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); > > + printf("Port %u ", portid); > > + print_ethaddr("Address:", &ports_eth_addr[portid]); > > + printf("\n"); > > + > > + /* tx queue setup */ > > + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, > > + socketid, &tx_conf); > > + if (ret < 0) > > + rte_exit(EXIT_FAILURE, > > + "rte_eth_tx_queue_setup: err=%d, " > > + "port=%d\n", ret, portid); > > + > > + /* rx queue steup */ > > + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, > > + socketid, &rx_conf, > > + mbufpool[socketid]); > > + if (ret < 0) > > + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," > > + "port=%d\n", ret, portid); > > + > > + /* Start device */ > > + stop = 0; > > + ret = rte_eth_dev_start(portid); > > + if (ret < 0) > > + rte_exit(EXIT_FAILURE, > > + "rte_eth_dev_start: err=%d, port=%d\n", > > + ret, portid); > > + > > + /* always eanble promiscuous */ > > + rte_eth_promiscuous_enable(portid); > > + > > + lcore_conf[slave_id].portlist[num++] = portid; > > + lcore_conf[slave_id].nb_ports++; > > + } > > + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > > + > > + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); > > + > > + rte_eal_remote_launch(main_loop, NULL, slave_id); > > + if (rte_eal_wait_lcore(slave_id) < 0) > > + return -1; > > + > > + /* port tear down */ > > + for (portid = 0; portid < nb_ports; portid++) { > > + if (socketid != rte_eth_dev_socket_id(portid)) > > + continue; > > + > > + rte_eth_dev_stop(portid); > > + } > > + > Clean up your allocated memory/lcores/etc? [Liang, Cunming] It do cleanup on the beginning of case. "Init_lcores","init_mbufpool","reset_count" guarantee the data clean before each testing. And mbufpool only allocated once even if we run multiple times. > > Neil ^ permalink raw reply [flat|nested] 65+ messages in thread
[parent not found: <E115CCD9D858EF4F90C690B0DCB4D8972262B720@IRSMSX108.ger.corp.intel.com>]
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet [not found] ` <E115CCD9D858EF4F90C690B0DCB4D8972262B720@IRSMSX108.ger.corp.intel.com> @ 2014-10-14 0:54 ` Liang, Cunming 0 siblings, 0 replies; 65+ messages in thread From: Liang, Cunming @ 2014-10-14 0:54 UTC (permalink / raw) To: De Lara Guarch, Pablo; +Cc: dev > -----Original Message----- > From: De Lara Guarch, Pablo > Sent: Monday, October 13, 2014 8:56 PM > To: Liang, Cunming; Neil Horman > Cc: dev@dpdk.org > Subject: RE: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > cycles/packet > > > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Liang, Cunming > > Sent: Sunday, October 12, 2014 12:11 PM > > To: Neil Horman > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > cycles/packet > > > > Hi Neil, > > > > Very appreciate your comments. > > I add inline reply, will send v3 asap when we get alignment. > > > > BRs, > > Liang Cunming > > > > > -----Original Message----- > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > Sent: Saturday, October 11, 2014 1:52 AM > > > To: Liang, Cunming > > > Cc: dev@dpdk.org > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > cycles/packet > > > > > > On Fri, Oct 10, 2014 at 08:29:58PM +0800, Cunming Liang wrote: > > > > It provides unit test to measure cycles/packet in NIC loopback mode. > > > > It simply gives the average cycles of IO used per packet without test > > equipment. > > > > When doing the test, make sure the link is UP. > > > > > > > > Usage Example: > > > > 1. Run unit test app in interactive mode > > > > app/test -c f -n 4 -- -i > > > > 2. Run and wait for the result > > > > pmd_perf_autotest > > > > > > > > There's option to choose rx/tx pair, default is vector. > > > > set_rxtx_mode [vector|scalar|full|hybrid] > > > > Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without > > > INC_VEC=y in config > > > > > > > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > > > > Acked-by: Bruce Richardson <bruce.richardson@intel.com> > > > > > > Notes inline > > > > > > > --- > > > > app/test/Makefile | 1 + > > > > app/test/commands.c | 38 +++ > > > > app/test/packet_burst_generator.c | 4 +- > > > > app/test/test.h | 4 + > > > > app/test/test_pmd_perf.c | 626 > > > +++++++++++++++++++++++++++++++++++ > > > > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > > > > 6 files changed, 677 insertions(+), 2 deletions(-) > > > > create mode 100644 app/test/test_pmd_perf.c > > > > > > > > diff --git a/app/test/Makefile b/app/test/Makefile > > > > index 6af6d76..ebfa0ba 100644 > > > > --- a/app/test/Makefile > > > > +++ b/app/test/Makefile > > > > @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c > > > > > > > > SRCS-y += test_ring.c > > > > SRCS-y += test_ring_perf.c > > > > +SRCS-y += test_pmd_perf.c > > > > > > > > ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) > > > > SRCS-y += test_table.c > > > > diff --git a/app/test/commands.c b/app/test/commands.c > > > > index a9e36b1..f1e746e 100644 > > > > --- a/app/test/commands.c > > > > +++ b/app/test/commands.c > > > > @@ -310,12 +310,50 @@ cmdline_parse_inst_t cmd_quit = { > > > > > > > > +#define NB_ETHPORTS_USED (1) > > > > +#define NB_SOCKETS (2) > > > > +#define MEMPOOL_CACHE_SIZE 250 > > > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + > > > RTE_PKTMBUF_HEADROOM) > > > Don't you want to size this in accordance with the amount of data your > > sending > > > (64 Bytes as noted above)? > > [Liang, Cunming] The case is designed to measure small packet IO cost with > > normal mbuf size. > > Even if decreasing the size, it won't gain significant cycles. > > > > > > > +static void > > > > +print_ethaddr(const char *name, const struct ether_addr *eth_addr) > > > > +{ > > > > + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, > > > > + eth_addr->addr_bytes[0], > > > > + eth_addr->addr_bytes[1], > > > > + eth_addr->addr_bytes[2], > > > > + eth_addr->addr_bytes[3], > > > > + eth_addr->addr_bytes[4], > > > > + eth_addr->addr_bytes[5]); > > > > +} > > > > + > > > This was copieed from print_ethaddr. Seems like a good candidate for a > > common > > > function in rte_ether.h > > [Liang, Cunming] Agree with you, some of samples now use it with the same > > copy. > > I'll rework it. Adding 'ether_format_addr' in rte_ether.h only for format the > > 48bits address output. > > And leaving other prints for application customization. > > > > > > > > > > +} > > > > + > > > > +static void > > > > +signal_handler(int signum) > > > > +{ > > > > + /* When we receive a USR1 signal, print stats */ > > > I think you mean SIGUSR2, below, SIGUSR1 tears the test down and exits > > the > > > program > > [Liang, Cunming] Thanks, it's a typo. > > > > > > > + if (signum == SIGUSR1) { > > > SIGINT instead. Thats the common practice. > > [Liang, Cunming] I understood your opinion. > > The considerations I'm not using SIGINT instead are: > > 1. We unset ISIG in c_lflag of term. CRTL+C won't trigger SIGINT in command > > interactive. > > It always has to explicitly send signal. No matter SIGUSR1 or SIGINT. > > 2. By SIGINT semantic, expect to terminate the process. > > Here I expect to force stop this case, but still alive in command line. > > After it stopped, it can run again or start to run other test cases. > > So I keep SIGINT, SIGUSR1 in different behavior. > > 3. It should be rarely used. > > Only when exception timeout, I leave this backdoor for automation test > > control. > > For manual test, we can easily force kill the process. > > > > > > > > > + printf("Force Stop!\n"); > > > > + stop = 1; > > > > + } > > > > + if (signum == SIGUSR2) > > > > + stats_display(0); > > > > +} > > > > +/* main processing loop */ > > > > +static int > > > > +main_loop(__rte_unused void *args) > > > > +{ > > > > +#define PACKET_SIZE 64 > > > > +#define FRAME_GAP 12 > > > > +#define MAC_PREAMBLE 8 > > > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > > > + unsigned lcore_id; > > > > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > > > > + struct lcore_conf *conf; > > > > + uint64_t prev_tsc, cur_tsc; > > > > + int pkt_per_port; > > > > + uint64_t packets_per_second, total_packets; > > > > + > > > > + lcore_id = rte_lcore_id(); > > > > + conf = &lcore_conf[lcore_id]; > > > > + if (conf->status != LCORE_USED) > > > > + return 0; > > > > + > > > > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > > > > + > > > > + int idx = 0; > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > + int num = pkt_per_port; > > > > + portid = conf->portlist[i]; > > > > + printf("inject %d packet to port %d\n", num, portid); > > > > + while (num) { > > > > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > > > > + nb_tx = rte_eth_tx_burst(portid, 0, > > > > + &tx_burst[idx], nb_tx); > > > > + num -= nb_tx; > > > > + idx += nb_tx; > > > > + } > > > > + } > > > > + printf("Total packets inject to prime ports = %u\n", idx); > > > > + > > > > + packets_per_second = (link_mbps * 1000 * 1000) / > > > > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * > > CHAR_BIT); > > > > + printf("Each port will do %"PRIu64" packets per second\n", > > > > + +packets_per_second); > > > > + > > > > + total_packets = RTE_TEST_DURATION * conf->nb_ports * > > > packets_per_second; > > > > + printf("Test will stop after at least %"PRIu64" packets received\n", > > > > + + total_packets); > > > > + > > > > + prev_tsc = rte_rdtsc(); > > > > + > > > > + while (likely(!stop)) { > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > + portid = conf->portlist[i]; > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > + pkts_burst, > > MAX_PKT_BURST); > > > > + if (unlikely(nb_rx == 0)) { > > > > + idle++; > > > > + continue; > > > > + } > > > > + > > > > + count += nb_rx; > > > Doesn't take into consideration error conditions. rte_eth_rx_burst can > > return > > > -ENOTSUP > > [Liang, Cunming] It returns -ENOTSUP when turning on ETHDEV_DEBUG > > CONFIG. > > The error is used to avoid no function call register. > > When ETHDEV_DEBUG turn off, the NULL function call cause segfault > > directly. > > So I think it's a library internal error. > > In such library exceptional case, I prefer not expecting sample/application to > > condition check library functional error. > > > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, > > nb_rx); > > > Ditto with -ENOTSUP > > > > > > > + if (unlikely(nb_tx < nb_rx)) { > > > What makes this unlikely? Seems like a perfectly reasonable condition to > > happen > > > to me. If the network is busy, its completely likely that you will receive > > more > > > frames than you send, if you elect to receive all frames. > > [Liang, Cunming] For this case, NIC works in MAC loopback mode. > > It firstly injects numbers of packets to NIC. Then NIC will loopback all packets > > to ingress side. > > The code here will receive the packets, and send them back to NIC > > Packets looping inside all come from initial injection. > > As the total number of injected packets is much less than in-chip queue size, > > the tx egress queue shouldn't block desc. ring update. > > So when receiving packets in line rate and nb_tx < nb_rx, it means tx cannot > > archive line rate. > > When it happens, the cycles/packets result make no sense, as the bottle > > neck is NIC. > > The drop counter can record it. > > > > > > > + drop += (nb_rx - nb_tx); > > > > + do { > > > > + > > rte_pktmbuf_free(pkts_burst[nb_tx]); > > > Defer this, it skews your timing > > [Liang, Cunming] Agree with you, I ever thought about it. > > This test cases is designed to measure pure IO RX/TX routine. > > When doing tx burst with result nb_tx < nb_rx, we either repeat re-send or > > drop it. > > Each way introduces noise(adding additional control code), resending much > > times even cost more than free it. > > The cycles/packets is useful when there's no packet drop, otherwise it gives > > the hint where the problem comes from (by idle or drop). > > > > > > > + } while (++nb_tx < nb_rx); > > > > + } > > > > + } > > > > + if (unlikely(count >= total_packets)) > > > > + break; > > > Whats the reasoning here? Do you only ever expect to receive frames that > > you > > > send? If so, seems like this should call for a big warning to get printed. > > [Liang, Cunming] The loop exits when the pre-calculated total_packets are > > received. > > As the nb_rx is unpredictable, the packet counter may large equal than > > total_packets the last time. > > The reason unlikely used here is because the condition becomes true only > > the last time. > > > > > > > + } > > > > + > > > > + cur_tsc = rte_rdtsc(); > > > > + > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > + portid = conf->portlist[i]; > > > > + int nb_free = pkt_per_port; > > > > + do { /* dry out */ > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > + pkts_burst, > > MAX_PKT_BURST); > > > > + nb_tx = 0; > > > > + while (nb_tx < nb_rx) > > > > + rte_pktmbuf_free(pkts_burst[nb_tx++]); > > > > + nb_free -= nb_rx; > > > > + } while (nb_free != 0); > > > > + printf("free %d mbuf left in port %u\n", pkt_per_port, > > portid); > > > > + } > > > > + > > > Whats the purpose of this? Are you trying to flush the device? Wouldn't it > > be > > > enough just to stop the interface? > > [Liang, Cunming] If we only run the cases once and exit, it doesn't matter. > > But it's designed to run multi-times without exit, for the purpose of warming > > up or for getting average number. > > So stopping device is not enough, we have to release the flying packets. > > > > > > > > > + if (count == 0) > > > > + return -1; > > > > + > > > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > > > > + > > > Bad math here. Theres no guarantee that the tsc hasn't wrapped > > (potentially > > > more than once) depending on your test length. you need to check the tsc > > before > > > and after each burst and record an average of deltas instead, accounting in > > each > > > instance for the possibility of wrap. > > [Liang, Cunming] I'm not sure catch your point correctly. > > I think both cur_tsc and prev_tsc are 64 bits width. > > For 3GHz, I think it won't wrapped so quick. > > As it's uint64_t, so even get wrapped, the delta should still be correct. > > You need to change those %lu to %PRIu64, or you will get a compilation error > when > using 32-bit targets, since those variables are uint64_t. There are other parts of > the > code with same problem, as well. [Liang, Cunming] Make sense to me. Thanks. > > > > > > > > > + return 0; > > > > +} > > > > + > > > > +static int > > > > +test_pmd_perf(void) > > > > +{ > > > > + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; > > > > + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; > > > > + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; > > > > + uint16_t portid; > > > > + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; > > > > + int socketid = -1; > > > > + int ret; > > > > + > > > > + printf("Start PMD RXTX cycles cost test.\n"); > > > > + > > > > + signal(SIGUSR1, signal_handler); > > > Again SIGINT here. > > > > > > > + signal(SIGUSR2, signal_handler); > > > > + > > > > + nb_ports = rte_eth_dev_count(); > > > > + if (nb_ports < NB_ETHPORTS_USED) { > > > > + printf("At least %u port(s) used for perf. test\n", > > > > + NB_ETHPORTS_USED); > > > > + return -1; > > > > + } > > > > + > > > > + if (nb_ports > RTE_MAX_ETHPORTS) > > > > + nb_ports = RTE_MAX_ETHPORTS; > > > > + > > > > + nb_lcores = rte_lcore_count(); > > > > + > > > > + memset(lcore_conf, 0, sizeof(lcore_conf)); > > > > + init_lcores(); > > > > + > > > > + init_mbufpool(NB_MBUF); > > > > + > > > > + reset_count(); > > > > + num = 0; > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > + if (socketid == -1) { > > > > + socketid = rte_eth_dev_socket_id(portid); > > > > + slave_id = alloc_lcore(socketid); > > > > + if (slave_id == (uint16_t)-1) { > > > > + printf("No avail lcore to run test\n"); > > > > + return -1; > > > > + } > > > > + printf("Performance test runs on lcore %u socket > > %u\n", > > > > + slave_id, socketid); > > > > + } > > > > + > > > > + if (socketid != rte_eth_dev_socket_id(portid)) { > > > > + printf("Skip port %d\n", portid); > > > > + continue; > > > > + } > > > > + > > > > + /* port configure */ > > > > + ret = rte_eth_dev_configure(portid, nb_rx_queue, > > > > + nb_tx_queue, &port_conf); > > > > + if (ret < 0) > > > > + rte_exit(EXIT_FAILURE, > > > > + "Cannot configure device: err=%d, > > port=%d\n", > > > > + ret, portid); > > > > + > > > > + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); > > > > + printf("Port %u ", portid); > > > > + print_ethaddr("Address:", &ports_eth_addr[portid]); > > > > + printf("\n"); > > > > + > > > > + /* tx queue setup */ > > > > + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, > > > > + socketid, &tx_conf); > > > > + if (ret < 0) > > > > + rte_exit(EXIT_FAILURE, > > > > + "rte_eth_tx_queue_setup: err=%d, " > > > > + "port=%d\n", ret, portid); > > > > + > > > > + /* rx queue steup */ > > > > + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, > > > > + socketid, &rx_conf, > > > > + mbufpool[socketid]); > > > > + if (ret < 0) > > > > + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: > > err=%d," > > > > + "port=%d\n", ret, portid); > > > > + > > > > + /* Start device */ > > > > + stop = 0; > > > > + ret = rte_eth_dev_start(portid); > > > > + if (ret < 0) > > > > + rte_exit(EXIT_FAILURE, > > > > + "rte_eth_dev_start: err=%d, port=%d\n", > > > > + ret, portid); > > > > + > > > > + /* always eanble promiscuous */ > > > > + rte_eth_promiscuous_enable(portid); > > > > + > > > > + lcore_conf[slave_id].portlist[num++] = portid; > > > > + lcore_conf[slave_id].nb_ports++; > > > > + } > > > > + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > > > > + > > > > + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); > > > > + > > > > + rte_eal_remote_launch(main_loop, NULL, slave_id); > > > > + if (rte_eal_wait_lcore(slave_id) < 0) > > > > + return -1; > > > > + > > > > + /* port tear down */ > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > + if (socketid != rte_eth_dev_socket_id(portid)) > > > > + continue; > > > > + > > > > + rte_eth_dev_stop(portid); > > > > + } > > > > + > > > Clean up your allocated memory/lcores/etc? > > [Liang, Cunming] It do cleanup on the beginning of case. > > "Init_lcores","init_mbufpool","reset_count" guarantee the data clean before > > each testing. > > And mbufpool only allocated once even if we run multiple times. > > > > > > Neil ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-12 11:10 ` Liang, Cunming [not found] ` <E115CCD9D858EF4F90C690B0DCB4D8972262B720@IRSMSX108.ger.corp.intel.com> @ 2014-10-21 10:33 ` Neil Horman 2014-10-21 10:43 ` Richardson, Bruce 2014-10-21 13:17 ` Liang, Cunming 1 sibling, 2 replies; 65+ messages in thread From: Neil Horman @ 2014-10-21 10:33 UTC (permalink / raw) To: Liang, Cunming; +Cc: dev On Sun, Oct 12, 2014 at 11:10:39AM +0000, Liang, Cunming wrote: > Hi Neil, > > Very appreciate your comments. > I add inline reply, will send v3 asap when we get alignment. > > BRs, > Liang Cunming > > > -----Original Message----- > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > Sent: Saturday, October 11, 2014 1:52 AM > > To: Liang, Cunming > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet > > > > On Fri, Oct 10, 2014 at 08:29:58PM +0800, Cunming Liang wrote: > > > It provides unit test to measure cycles/packet in NIC loopback mode. > > > It simply gives the average cycles of IO used per packet without test equipment. > > > When doing the test, make sure the link is UP. > > > > > > Usage Example: > > > 1. Run unit test app in interactive mode > > > app/test -c f -n 4 -- -i > > > 2. Run and wait for the result > > > pmd_perf_autotest > > > > > > There's option to choose rx/tx pair, default is vector. > > > set_rxtx_mode [vector|scalar|full|hybrid] > > > Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without > > INC_VEC=y in config > > > > > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > > > Acked-by: Bruce Richardson <bruce.richardson@intel.com> > > > > Notes inline > > > > > --- > > > app/test/Makefile | 1 + > > > app/test/commands.c | 38 +++ > > > app/test/packet_burst_generator.c | 4 +- > > > app/test/test.h | 4 + > > > app/test/test_pmd_perf.c | 626 > > +++++++++++++++++++++++++++++++++++ > > > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > > > 6 files changed, 677 insertions(+), 2 deletions(-) > > > create mode 100644 app/test/test_pmd_perf.c > > > > > > diff --git a/app/test/Makefile b/app/test/Makefile > > > index 6af6d76..ebfa0ba 100644 > > > --- a/app/test/Makefile > > > +++ b/app/test/Makefile > > > @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c > > > > > > SRCS-y += test_ring.c > > > SRCS-y += test_ring_perf.c > > > +SRCS-y += test_pmd_perf.c > > > > > > ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) > > > SRCS-y += test_table.c > > > diff --git a/app/test/commands.c b/app/test/commands.c > > > index a9e36b1..f1e746e 100644 > > > --- a/app/test/commands.c > > > +++ b/app/test/commands.c > > > @@ -310,12 +310,50 @@ cmdline_parse_inst_t cmd_quit = { > > > > > > +#define NB_ETHPORTS_USED (1) > > > +#define NB_SOCKETS (2) > > > +#define MEMPOOL_CACHE_SIZE 250 > > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + > > RTE_PKTMBUF_HEADROOM) > > Don't you want to size this in accordance with the amount of data your sending > > (64 Bytes as noted above)? > [Liang, Cunming] The case is designed to measure small packet IO cost with normal mbuf size. > Even if decreasing the size, it won't gain significant cycles. > > That presumes a non-memory constrained system, doesn't it? I suppose in the end as long as you have consistency, its not overly relevant, but it seems like you'll want to add data sizing as a feature to this eventually (i.e. the ability to test performance for larger frames sizes), at which point you'll need to make this non-static anyway. > > > +static void > > > +print_ethaddr(const char *name, const struct ether_addr *eth_addr) > > > +{ > > > + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, > > > + eth_addr->addr_bytes[0], > > > + eth_addr->addr_bytes[1], > > > + eth_addr->addr_bytes[2], > > > + eth_addr->addr_bytes[3], > > > + eth_addr->addr_bytes[4], > > > + eth_addr->addr_bytes[5]); > > > +} > > > + > > This was copieed from print_ethaddr. Seems like a good candidate for a common > > function in rte_ether.h > [Liang, Cunming] Agree with you, some of samples now use it with the same copy. > I'll rework it. Adding 'ether_format_addr' in rte_ether.h only for format the 48bits address output. > And leaving other prints for application customization. > > Sounds good. > > > > > +} > > > + > > > +static void > > > +signal_handler(int signum) > > > +{ > > > + /* When we receive a USR1 signal, print stats */ > > I think you mean SIGUSR2, below, SIGUSR1 tears the test down and exits the > > program > [Liang, Cunming] Thanks, it's a typo. > > > > > + if (signum == SIGUSR1) { > > SIGINT instead. Thats the common practice. > [Liang, Cunming] I understood your opinion. > The considerations I'm not using SIGINT instead are: > 1. We unset ISIG in c_lflag of term. CRTL+C won't trigger SIGINT in command interactive. > It always has to explicitly send signal. No matter SIGUSR1 or SIGINT. > 2. By SIGINT semantic, expect to terminate the process. > Here I expect to force stop this case, but still alive in command line. > After it stopped, it can run again or start to run other test cases. > So I keep SIGINT, SIGUSR1 in different behavior. > 3. It should be rarely used. > Only when exception timeout, I leave this backdoor for automation test control. > For manual test, we can easily force kill the process. > Hmm, ok, that sounds reasonable. > > > > > + printf("Force Stop!\n"); > > > + stop = 1; > > > + } > > > + if (signum == SIGUSR2) > > > + stats_display(0); > > > +} > > > +/* main processing loop */ > > > +static int > > > +main_loop(__rte_unused void *args) > > > +{ > > > +#define PACKET_SIZE 64 > > > +#define FRAME_GAP 12 > > > +#define MAC_PREAMBLE 8 > > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > > + unsigned lcore_id; > > > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > > > + struct lcore_conf *conf; > > > + uint64_t prev_tsc, cur_tsc; > > > + int pkt_per_port; > > > + uint64_t packets_per_second, total_packets; > > > + > > > + lcore_id = rte_lcore_id(); > > > + conf = &lcore_conf[lcore_id]; > > > + if (conf->status != LCORE_USED) > > > + return 0; > > > + > > > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > > > + > > > + int idx = 0; > > > + for (i = 0; i < conf->nb_ports; i++) { > > > + int num = pkt_per_port; > > > + portid = conf->portlist[i]; > > > + printf("inject %d packet to port %d\n", num, portid); > > > + while (num) { > > > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > > > + nb_tx = rte_eth_tx_burst(portid, 0, > > > + &tx_burst[idx], nb_tx); > > > + num -= nb_tx; > > > + idx += nb_tx; > > > + } > > > + } > > > + printf("Total packets inject to prime ports = %u\n", idx); > > > + > > > + packets_per_second = (link_mbps * 1000 * 1000) / > > > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); > > > + printf("Each port will do %"PRIu64" packets per second\n", > > > + +packets_per_second); > > > + > > > + total_packets = RTE_TEST_DURATION * conf->nb_ports * > > packets_per_second; > > > + printf("Test will stop after at least %"PRIu64" packets received\n", > > > + + total_packets); > > > + > > > + prev_tsc = rte_rdtsc(); > > > + > > > + while (likely(!stop)) { > > > + for (i = 0; i < conf->nb_ports; i++) { > > > + portid = conf->portlist[i]; > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > + pkts_burst, MAX_PKT_BURST); > > > + if (unlikely(nb_rx == 0)) { > > > + idle++; > > > + continue; > > > + } > > > + > > > + count += nb_rx; > > Doesn't take into consideration error conditions. rte_eth_rx_burst can return > > -ENOTSUP > [Liang, Cunming] It returns -ENOTSUP when turning on ETHDEV_DEBUG CONFIG. > The error is used to avoid no function call register. > When ETHDEV_DEBUG turn off, the NULL function call cause segfault directly. > So I think it's a library internal error. No, look at FUNC_PTR_OR_ERR_RET. If the passed in function pointer is null, -ENOTSUPP will be returned to the application, you need to handle the error condition. > In such library exceptional case, I prefer not expecting sample/application to condition check library functional error. But you're assertion about the library handling the exception is wrong. > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); > > Ditto with -ENOTSUP > > > > > + if (unlikely(nb_tx < nb_rx)) { > > What makes this unlikely? Seems like a perfectly reasonable condition to happen > > to me. If the network is busy, its completely likely that you will receive more > > frames than you send, if you elect to receive all frames. > [Liang, Cunming] For this case, NIC works in MAC loopback mode. > It firstly injects numbers of packets to NIC. Then NIC will loopback all packets to ingress side. > The code here will receive the packets, and send them back to NIC > Packets looping inside all come from initial injection. > As the total number of injected packets is much less than in-chip queue size, the tx egress queue shouldn't block desc. ring update. > So when receiving packets in line rate and nb_tx < nb_rx, it means tx cannot archive line rate. > When it happens, the cycles/packets result make no sense, as the bottle neck is NIC. > The drop counter can record it. Ok. > > > > > + drop += (nb_rx - nb_tx); > > > + do { > > > + rte_pktmbuf_free(pkts_burst[nb_tx]); > > Defer this, it skews your timing > [Liang, Cunming] Agree with you, I ever thought about it. > This test cases is designed to measure pure IO RX/TX routine. > When doing tx burst with result nb_tx < nb_rx, we either repeat re-send or drop it. > Each way introduces noise(adding additional control code), resending much times even cost more than free it. > The cycles/packets is useful when there's no packet drop, otherwise it gives the hint where the problem comes from (by idle or drop). I'm not sure what you're asserting here. Are you suggesting that you want to include the time it takes to free memory buffers in your testing? That seems dubious at best to me. If you want to measure I/O, thats great, but memory management of packet buffers is a separate operation from that I/O > > > > > + } while (++nb_tx < nb_rx); > > > + } > > > + } > > > + if (unlikely(count >= total_packets)) > > > + break; > > Whats the reasoning here? Do you only ever expect to receive frames that you > > send? If so, seems like this should call for a big warning to get printed. > [Liang, Cunming] The loop exits when the pre-calculated total_packets are received. > As the nb_rx is unpredictable, the packet counter may large equal than total_packets the last time. > The reason unlikely used here is because the condition becomes true only the last time. ok > > > > > + } > > > + > > > + cur_tsc = rte_rdtsc(); > > > + > > > + for (i = 0; i < conf->nb_ports; i++) { > > > + portid = conf->portlist[i]; > > > + int nb_free = pkt_per_port; > > > + do { /* dry out */ > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > + pkts_burst, MAX_PKT_BURST); > > > + nb_tx = 0; > > > + while (nb_tx < nb_rx) > > > + rte_pktmbuf_free(pkts_burst[nb_tx++]); > > > + nb_free -= nb_rx; > > > + } while (nb_free != 0); > > > + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); > > > + } > > > + > > Whats the purpose of this? Are you trying to flush the device? Wouldn't it be > > enough just to stop the interface? > [Liang, Cunming] If we only run the cases once and exit, it doesn't matter. > But it's designed to run multi-times without exit, for the purpose of warming up or for getting average number. > So stopping device is not enough, we have to release the flying packets. > Ok > > > > > + if (count == 0) > > > + return -1; > > > + > > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > > > + > > Bad math here. Theres no guarantee that the tsc hasn't wrapped (potentially > > more than once) depending on your test length. you need to check the tsc before > > and after each burst and record an average of deltas instead, accounting in each > > instance for the possibility of wrap. > [Liang, Cunming] I'm not sure catch your point correctly. > I think both cur_tsc and prev_tsc are 64 bits width. > For 3GHz, I think it won't wrapped so quick. > As it's uint64_t, so even get wrapped, the delta should still be correct. But theres no guarantee that the tsc starts at zero when you begin your test. The system may have been up for a long time and near wrapping already. Regardless, you need to account for the possibility that cur_tsc is smaller than prev_tsc, or this breaks. > > > > > + return 0; > > > +} > > > + > > > +static int > > > +test_pmd_perf(void) > > > +{ > > > + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; > > > + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; > > > + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; > > > + uint16_t portid; > > > + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; > > > + int socketid = -1; > > > + int ret; > > > + > > > + printf("Start PMD RXTX cycles cost test.\n"); > > > + > > > + signal(SIGUSR1, signal_handler); > > Again SIGINT here. > > > > > + signal(SIGUSR2, signal_handler); > > > + > > > + nb_ports = rte_eth_dev_count(); > > > + if (nb_ports < NB_ETHPORTS_USED) { > > > + printf("At least %u port(s) used for perf. test\n", > > > + NB_ETHPORTS_USED); > > > + return -1; > > > + } > > > + > > > + if (nb_ports > RTE_MAX_ETHPORTS) > > > + nb_ports = RTE_MAX_ETHPORTS; > > > + > > > + nb_lcores = rte_lcore_count(); > > > + > > > + memset(lcore_conf, 0, sizeof(lcore_conf)); > > > + init_lcores(); > > > + > > > + init_mbufpool(NB_MBUF); > > > + > > > + reset_count(); > > > + num = 0; > > > + for (portid = 0; portid < nb_ports; portid++) { > > > + if (socketid == -1) { > > > + socketid = rte_eth_dev_socket_id(portid); > > > + slave_id = alloc_lcore(socketid); > > > + if (slave_id == (uint16_t)-1) { > > > + printf("No avail lcore to run test\n"); > > > + return -1; > > > + } > > > + printf("Performance test runs on lcore %u socket %u\n", > > > + slave_id, socketid); > > > + } > > > + > > > + if (socketid != rte_eth_dev_socket_id(portid)) { > > > + printf("Skip port %d\n", portid); > > > + continue; > > > + } > > > + > > > + /* port configure */ > > > + ret = rte_eth_dev_configure(portid, nb_rx_queue, > > > + nb_tx_queue, &port_conf); > > > + if (ret < 0) > > > + rte_exit(EXIT_FAILURE, > > > + "Cannot configure device: err=%d, port=%d\n", > > > + ret, portid); > > > + > > > + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); > > > + printf("Port %u ", portid); > > > + print_ethaddr("Address:", &ports_eth_addr[portid]); > > > + printf("\n"); > > > + > > > + /* tx queue setup */ > > > + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, > > > + socketid, &tx_conf); > > > + if (ret < 0) > > > + rte_exit(EXIT_FAILURE, > > > + "rte_eth_tx_queue_setup: err=%d, " > > > + "port=%d\n", ret, portid); > > > + > > > + /* rx queue steup */ > > > + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, > > > + socketid, &rx_conf, > > > + mbufpool[socketid]); > > > + if (ret < 0) > > > + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," > > > + "port=%d\n", ret, portid); > > > + > > > + /* Start device */ > > > + stop = 0; > > > + ret = rte_eth_dev_start(portid); > > > + if (ret < 0) > > > + rte_exit(EXIT_FAILURE, > > > + "rte_eth_dev_start: err=%d, port=%d\n", > > > + ret, portid); > > > + > > > + /* always eanble promiscuous */ > > > + rte_eth_promiscuous_enable(portid); > > > + > > > + lcore_conf[slave_id].portlist[num++] = portid; > > > + lcore_conf[slave_id].nb_ports++; > > > + } > > > + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > > > + > > > + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); > > > + > > > + rte_eal_remote_launch(main_loop, NULL, slave_id); > > > + if (rte_eal_wait_lcore(slave_id) < 0) > > > + return -1; > > > + > > > + /* port tear down */ > > > + for (portid = 0; portid < nb_ports; portid++) { > > > + if (socketid != rte_eth_dev_socket_id(portid)) > > > + continue; > > > + > > > + rte_eth_dev_stop(portid); > > > + } > > > + > > Clean up your allocated memory/lcores/etc? > [Liang, Cunming] It do cleanup on the beginning of case. > "Init_lcores","init_mbufpool","reset_count" guarantee the data clean before each testing. > And mbufpool only allocated once even if we run multiple times. Its a janitorial issue. Before the program exits, you need to free any resources that you've allocated. Neil > > > > Neil > > ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-21 10:33 ` Neil Horman @ 2014-10-21 10:43 ` Richardson, Bruce 2014-10-21 13:37 ` Neil Horman 2014-10-21 13:17 ` Liang, Cunming 1 sibling, 1 reply; 65+ messages in thread From: Richardson, Bruce @ 2014-10-21 10:43 UTC (permalink / raw) To: Neil Horman, Liang, Cunming; +Cc: dev > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > Sent: Tuesday, October 21, 2014 11:33 AM > To: Liang, Cunming > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > cycles/packet > > > > > > > > + if (count == 0) > > > > + return -1; > > > > + > > > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > > > > + > > > Bad math here. Theres no guarantee that the tsc hasn't wrapped (potentially > > > more than once) depending on your test length. you need to check the tsc > before > > > and after each burst and record an average of deltas instead, accounting in > each > > > instance for the possibility of wrap. > > [Liang, Cunming] I'm not sure catch your point correctly. > > I think both cur_tsc and prev_tsc are 64 bits width. > > For 3GHz, I think it won't wrapped so quick. > > As it's uint64_t, so even get wrapped, the delta should still be correct. > But theres no guarantee that the tsc starts at zero when you begin your test. > The system may have been up for a long time and near wrapping already. > Regardless, you need to account for the possibility that cur_tsc is smaller > than prev_tsc, or this breaks. > The tsc. is 64-bit and so only wraps around every couple of hundred years or so on a 2GHz machine, so I don't think it's necessary to handle that case. /Bruce ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-21 10:43 ` Richardson, Bruce @ 2014-10-21 13:37 ` Neil Horman 2014-10-21 22:43 ` Ananyev, Konstantin 0 siblings, 1 reply; 65+ messages in thread From: Neil Horman @ 2014-10-21 13:37 UTC (permalink / raw) To: Richardson, Bruce; +Cc: dev On Tue, Oct 21, 2014 at 10:43:03AM +0000, Richardson, Bruce wrote: > > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > > Sent: Tuesday, October 21, 2014 11:33 AM > > To: Liang, Cunming > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > cycles/packet > > > > > > > > > > > + if (count == 0) > > > > > + return -1; > > > > > + > > > > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > > > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > > > > > + > > > > Bad math here. Theres no guarantee that the tsc hasn't wrapped (potentially > > > > more than once) depending on your test length. you need to check the tsc > > before > > > > and after each burst and record an average of deltas instead, accounting in > > each > > > > instance for the possibility of wrap. > > > [Liang, Cunming] I'm not sure catch your point correctly. > > > I think both cur_tsc and prev_tsc are 64 bits width. > > > For 3GHz, I think it won't wrapped so quick. > > > As it's uint64_t, so even get wrapped, the delta should still be correct. > > But theres no guarantee that the tsc starts at zero when you begin your test. > > The system may have been up for a long time and near wrapping already. > > Regardless, you need to account for the possibility that cur_tsc is smaller > > than prev_tsc, or this breaks. > > > > The tsc. is 64-bit and so only wraps around every couple of hundred years or so on a 2GHz machine, so I don't think it's necessary to handle that case. > But that presumes that no one has written the TSC via IA32_TIME_STAMP_COUNTER. Assuming that something will never wrap just seems like bad practice here. We should have a general purpose macro to handle wrapping counters like this, if not for this case specficially, then in general. Neil > /Bruce > ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-21 13:37 ` Neil Horman @ 2014-10-21 22:43 ` Ananyev, Konstantin 0 siblings, 0 replies; 65+ messages in thread From: Ananyev, Konstantin @ 2014-10-21 22:43 UTC (permalink / raw) To: Neil Horman, Richardson, Bruce; +Cc: dev > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > Sent: Tuesday, October 21, 2014 2:38 PM > To: Richardson, Bruce > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet > > On Tue, Oct 21, 2014 at 10:43:03AM +0000, Richardson, Bruce wrote: > > > > > > > -----Original Message----- > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > > > Sent: Tuesday, October 21, 2014 11:33 AM > > > To: Liang, Cunming > > > Cc: dev@dpdk.org > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > > cycles/packet > > > > > > > > > > > > > > + if (count == 0) > > > > > > + return -1; > > > > > > + > > > > > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > > > > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > > > > > > + > > > > > Bad math here. Theres no guarantee that the tsc hasn't wrapped (potentially > > > > > more than once) depending on your test length. you need to check the tsc > > > before > > > > > and after each burst and record an average of deltas instead, accounting in > > > each > > > > > instance for the possibility of wrap. > > > > [Liang, Cunming] I'm not sure catch your point correctly. > > > > I think both cur_tsc and prev_tsc are 64 bits width. > > > > For 3GHz, I think it won't wrapped so quick. > > > > As it's uint64_t, so even get wrapped, the delta should still be correct. > > > But theres no guarantee that the tsc starts at zero when you begin your test. > > > The system may have been up for a long time and near wrapping already. > > > Regardless, you need to account for the possibility that cur_tsc is smaller > > > than prev_tsc, or this breaks. > > > > > > > The tsc. is 64-bit and so only wraps around every couple of hundred years or so on a 2GHz machine, so I don't think it's necessary to > handle that case. > > > But that presumes that no one has written the TSC via IA32_TIME_STAMP_COUNTER. Then the test app would just print the wrong number :) I suppose user will just repeat the test. Again, imagine someone wrmsr(TSC), but set the value just much bigger than current. Then your statistics will be incorrect, but you have no wait to figure that out. I think that trying to handle all such hypothetical situations is a pure overkill. > Assuming that something will never wrap just seems like bad practice here. We > should have a general purpose macro to handle wrapping counters like this, if > not for this case specficially, then in general. > > Neil > > > /Bruce > > ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-21 10:33 ` Neil Horman 2014-10-21 10:43 ` Richardson, Bruce @ 2014-10-21 13:17 ` Liang, Cunming 2014-10-22 14:03 ` Neil Horman 1 sibling, 1 reply; 65+ messages in thread From: Liang, Cunming @ 2014-10-21 13:17 UTC (permalink / raw) To: Neil Horman; +Cc: dev > -----Original Message----- > From: Neil Horman [mailto:nhorman@tuxdriver.com] > Sent: Tuesday, October 21, 2014 6:33 PM > To: Liang, Cunming > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > cycles/packet > > On Sun, Oct 12, 2014 at 11:10:39AM +0000, Liang, Cunming wrote: > > Hi Neil, > > > > Very appreciate your comments. > > I add inline reply, will send v3 asap when we get alignment. > > > > BRs, > > Liang Cunming > > > > > -----Original Message----- > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > Sent: Saturday, October 11, 2014 1:52 AM > > > To: Liang, Cunming > > > Cc: dev@dpdk.org > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > cycles/packet > > > > > > On Fri, Oct 10, 2014 at 08:29:58PM +0800, Cunming Liang wrote: > > > > It provides unit test to measure cycles/packet in NIC loopback mode. > > > > It simply gives the average cycles of IO used per packet without test > equipment. > > > > When doing the test, make sure the link is UP. > > > > > > > > Usage Example: > > > > 1. Run unit test app in interactive mode > > > > app/test -c f -n 4 -- -i > > > > 2. Run and wait for the result > > > > pmd_perf_autotest > > > > > > > > There's option to choose rx/tx pair, default is vector. > > > > set_rxtx_mode [vector|scalar|full|hybrid] > > > > Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without > > > INC_VEC=y in config > > > > > > > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > > > > Acked-by: Bruce Richardson <bruce.richardson@intel.com> > > > > > > Notes inline > > > > > > > --- > > > > app/test/Makefile | 1 + > > > > app/test/commands.c | 38 +++ > > > > app/test/packet_burst_generator.c | 4 +- > > > > app/test/test.h | 4 + > > > > app/test/test_pmd_perf.c | 626 > > > +++++++++++++++++++++++++++++++++++ > > > > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > > > > 6 files changed, 677 insertions(+), 2 deletions(-) > > > > create mode 100644 app/test/test_pmd_perf.c > > > > > > > > diff --git a/app/test/Makefile b/app/test/Makefile > > > > index 6af6d76..ebfa0ba 100644 > > > > --- a/app/test/Makefile > > > > +++ b/app/test/Makefile > > > > @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c > > > > > > > > SRCS-y += test_ring.c > > > > SRCS-y += test_ring_perf.c > > > > +SRCS-y += test_pmd_perf.c > > > > > > > > ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) > > > > SRCS-y += test_table.c > > > > diff --git a/app/test/commands.c b/app/test/commands.c > > > > index a9e36b1..f1e746e 100644 > > > > --- a/app/test/commands.c > > > > +++ b/app/test/commands.c > > > > @@ -310,12 +310,50 @@ cmdline_parse_inst_t cmd_quit = { > > > > > > > > +#define NB_ETHPORTS_USED (1) > > > > +#define NB_SOCKETS (2) > > > > +#define MEMPOOL_CACHE_SIZE 250 > > > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + > > > RTE_PKTMBUF_HEADROOM) > > > Don't you want to size this in accordance with the amount of data your > sending > > > (64 Bytes as noted above)? > > [Liang, Cunming] The case is designed to measure small packet IO cost with > normal mbuf size. > > Even if decreasing the size, it won't gain significant cycles. > > > > That presumes a non-memory constrained system, doesn't it? I suppose in the > end > as long as you have consistency, its not overly relevant, but it seems like > you'll want to add data sizing as a feature to this eventually (i.e. the ability > to test performance for larger frames sizes), at which point you'll need to make > this non-static anyway. [Liang, Cunming] For a normal Ethernet packet(w/o jumbo frame), packet size is 1518B. As in really network, there won't have huge number of jumbo frames. The mbuf size 2048 is a reasonable value to cover most of the packet size. It's also be chosen by lots of NIC as the default receiving buffer size in DMA register. In case larger than the size, it need do scatter and gather but lose some performance. The unit test won't measure size from 64 to 9600, won't plan to measure scatter-gather rx/tx. It focus on 64B packet size and taking the mbuf size being used the most often. > > > > > +static void > > > > +print_ethaddr(const char *name, const struct ether_addr *eth_addr) > > > > +{ > > > > + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, > > > > + eth_addr->addr_bytes[0], > > > > + eth_addr->addr_bytes[1], > > > > + eth_addr->addr_bytes[2], > > > > + eth_addr->addr_bytes[3], > > > > + eth_addr->addr_bytes[4], > > > > + eth_addr->addr_bytes[5]); > > > > +} > > > > + > > > This was copieed from print_ethaddr. Seems like a good candidate for a > common > > > function in rte_ether.h > > [Liang, Cunming] Agree with you, some of samples now use it with the same > copy. > > I'll rework it. Adding 'ether_format_addr' in rte_ether.h only for format the > 48bits address output. > > And leaving other prints for application customization. > > > > Sounds good. > > > > > > > > +} > > > > + > > > > +static void > > > > +signal_handler(int signum) > > > > +{ > > > > + /* When we receive a USR1 signal, print stats */ > > > I think you mean SIGUSR2, below, SIGUSR1 tears the test down and exits the > > > program > > [Liang, Cunming] Thanks, it's a typo. > > > > > > > + if (signum == SIGUSR1) { > > > SIGINT instead. Thats the common practice. > > [Liang, Cunming] I understood your opinion. > > The considerations I'm not using SIGINT instead are: > > 1. We unset ISIG in c_lflag of term. CRTL+C won't trigger SIGINT in command > interactive. > > It always has to explicitly send signal. No matter SIGUSR1 or SIGINT. > > 2. By SIGINT semantic, expect to terminate the process. > > Here I expect to force stop this case, but still alive in command line. > > After it stopped, it can run again or start to run other test cases. > > So I keep SIGINT, SIGUSR1 in different behavior. > > 3. It should be rarely used. > > Only when exception timeout, I leave this backdoor for automation test > control. > > For manual test, we can easily force kill the process. > > > Hmm, ok, that sounds reasonable. > > > > > > > > + printf("Force Stop!\n"); > > > > + stop = 1; > > > > + } > > > > + if (signum == SIGUSR2) > > > > + stats_display(0); > > > > +} > > > > +/* main processing loop */ > > > > +static int > > > > +main_loop(__rte_unused void *args) > > > > +{ > > > > +#define PACKET_SIZE 64 > > > > +#define FRAME_GAP 12 > > > > +#define MAC_PREAMBLE 8 > > > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > > > + unsigned lcore_id; > > > > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > > > > + struct lcore_conf *conf; > > > > + uint64_t prev_tsc, cur_tsc; > > > > + int pkt_per_port; > > > > + uint64_t packets_per_second, total_packets; > > > > + > > > > + lcore_id = rte_lcore_id(); > > > > + conf = &lcore_conf[lcore_id]; > > > > + if (conf->status != LCORE_USED) > > > > + return 0; > > > > + > > > > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > > > > + > > > > + int idx = 0; > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > + int num = pkt_per_port; > > > > + portid = conf->portlist[i]; > > > > + printf("inject %d packet to port %d\n", num, portid); > > > > + while (num) { > > > > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > > > > + nb_tx = rte_eth_tx_burst(portid, 0, > > > > + &tx_burst[idx], nb_tx); > > > > + num -= nb_tx; > > > > + idx += nb_tx; > > > > + } > > > > + } > > > > + printf("Total packets inject to prime ports = %u\n", idx); > > > > + > > > > + packets_per_second = (link_mbps * 1000 * 1000) / > > > > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); > > > > + printf("Each port will do %"PRIu64" packets per second\n", > > > > + +packets_per_second); > > > > + > > > > + total_packets = RTE_TEST_DURATION * conf->nb_ports * > > > packets_per_second; > > > > + printf("Test will stop after at least %"PRIu64" packets received\n", > > > > + + total_packets); > > > > + > > > > + prev_tsc = rte_rdtsc(); > > > > + > > > > + while (likely(!stop)) { > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > + portid = conf->portlist[i]; > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > + pkts_burst, MAX_PKT_BURST); > > > > + if (unlikely(nb_rx == 0)) { > > > > + idle++; > > > > + continue; > > > > + } > > > > + > > > > + count += nb_rx; > > > Doesn't take into consideration error conditions. rte_eth_rx_burst can > return > > > -ENOTSUP > > [Liang, Cunming] It returns -ENOTSUP when turning on ETHDEV_DEBUG > CONFIG. > > The error is used to avoid no function call register. > > When ETHDEV_DEBUG turn off, the NULL function call cause segfault directly. > > So I think it's a library internal error. > No, look at FUNC_PTR_OR_ERR_RET. If the passed in function pointer is null, > -ENOTSUPP will be returned to the application, you need to handle the error > condition. [Liang, Cunming] The runtime rte_eth_rx_burst is a inline function in rte_ethdev.h. The one you're talking about is the one defined in rte_ethdev.c which is a extern non-inline function. It works only when RTE_LIBRTE_ETHDEV_DEBUG turns on. If we always turns such library internal checking on, it lose performance. So I insist it's a library internal error checking, doesn't need to take care in application level. > > > In such library exceptional case, I prefer not expecting sample/application to > condition check library functional error. > But you're assertion about the library handling the exception is wrong. > > > > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); > > > Ditto with -ENOTSUP > > > > > > > + if (unlikely(nb_tx < nb_rx)) { > > > What makes this unlikely? Seems like a perfectly reasonable condition to > happen > > > to me. If the network is busy, its completely likely that you will receive more > > > frames than you send, if you elect to receive all frames. > > [Liang, Cunming] For this case, NIC works in MAC loopback mode. > > It firstly injects numbers of packets to NIC. Then NIC will loopback all packets to > ingress side. > > The code here will receive the packets, and send them back to NIC > > Packets looping inside all come from initial injection. > > As the total number of injected packets is much less than in-chip queue size, > the tx egress queue shouldn't block desc. ring update. > > So when receiving packets in line rate and nb_tx < nb_rx, it means tx cannot > archive line rate. > > When it happens, the cycles/packets result make no sense, as the bottle neck is > NIC. > > The drop counter can record it. > Ok. > > > > > > > > + drop += (nb_rx - nb_tx); > > > > + do { > > > > + rte_pktmbuf_free(pkts_burst[nb_tx]); > > > Defer this, it skews your timing > > [Liang, Cunming] Agree with you, I ever thought about it. > > This test cases is designed to measure pure IO RX/TX routine. > > When doing tx burst with result nb_tx < nb_rx, we either repeat re-send or > drop it. > > Each way introduces noise(adding additional control code), resending much > times even cost more than free it. > > The cycles/packets is useful when there's no packet drop, otherwise it gives the > hint where the problem comes from (by idle or drop). > I'm not sure what you're asserting here. Are you suggesting that you want to > include the time it takes to free memory buffers in your testing? That seems > dubious at best to me. If you want to measure I/O, thats great, but memory > management of packet buffers is a separate operation from that I/O [Liang, Cunming] Agree with you. I means it doesn't need to take care of the mbuf free cost. As I said in previous reply, it rarely happens in line rate. The cycle measurement doesn't make much sense if happens. On that time the unit test just notify it happens, and keep safe free all the unsent mbuf. > > > > > > > > + } while (++nb_tx < nb_rx); > > > > + } > > > > + } > > > > + if (unlikely(count >= total_packets)) > > > > + break; > > > Whats the reasoning here? Do you only ever expect to receive frames that > you > > > send? If so, seems like this should call for a big warning to get printed. > > [Liang, Cunming] The loop exits when the pre-calculated total_packets are > received. > > As the nb_rx is unpredictable, the packet counter may large equal than > total_packets the last time. > > The reason unlikely used here is because the condition becomes true only the > last time. > ok > > > > > > > > + } > > > > + > > > > + cur_tsc = rte_rdtsc(); > > > > + > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > + portid = conf->portlist[i]; > > > > + int nb_free = pkt_per_port; > > > > + do { /* dry out */ > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > + pkts_burst, MAX_PKT_BURST); > > > > + nb_tx = 0; > > > > + while (nb_tx < nb_rx) > > > > + rte_pktmbuf_free(pkts_burst[nb_tx++]); > > > > + nb_free -= nb_rx; > > > > + } while (nb_free != 0); > > > > + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); > > > > + } > > > > + > > > Whats the purpose of this? Are you trying to flush the device? Wouldn't it > be > > > enough just to stop the interface? > > [Liang, Cunming] If we only run the cases once and exit, it doesn't matter. > > But it's designed to run multi-times without exit, for the purpose of warming > up or for getting average number. > > So stopping device is not enough, we have to release the flying packets. > > > Ok > > > > > > > > + if (count == 0) > > > > + return -1; > > > > + > > > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > > > > + > > > Bad math here. Theres no guarantee that the tsc hasn't wrapped > (potentially > > > more than once) depending on your test length. you need to check the tsc > before > > > and after each burst and record an average of deltas instead, accounting in > each > > > instance for the possibility of wrap. > > [Liang, Cunming] I'm not sure catch your point correctly. > > I think both cur_tsc and prev_tsc are 64 bits width. > > For 3GHz, I think it won't wrapped so quick. > > As it's uint64_t, so even get wrapped, the delta should still be correct. > But theres no guarantee that the tsc starts at zero when you begin your test. > The system may have been up for a long time and near wrapping already. > Regardless, you need to account for the possibility that cur_tsc is smaller > than prev_tsc, or this breaks. [Liang, Cunming] In case prev_tsc near wrapping, and cur_tsc get wrapped. As they are unsigned, it still ok. e.g. cur_tsc=0x2, prev_tsc=0xFFFFFFFFFFFFFFFC Delta=cur_tsc-prev_tsc is 6, which is still correct. And for uint64_t, we need to start computer for hundreds of years. > > > > > > > > + return 0; > > > > +} > > > > + > > > > +static int > > > > +test_pmd_perf(void) > > > > +{ > > > > + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; > > > > + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; > > > > + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; > > > > + uint16_t portid; > > > > + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; > > > > + int socketid = -1; > > > > + int ret; > > > > + > > > > + printf("Start PMD RXTX cycles cost test.\n"); > > > > + > > > > + signal(SIGUSR1, signal_handler); > > > Again SIGINT here. > > > > > > > + signal(SIGUSR2, signal_handler); > > > > + > > > > + nb_ports = rte_eth_dev_count(); > > > > + if (nb_ports < NB_ETHPORTS_USED) { > > > > + printf("At least %u port(s) used for perf. test\n", > > > > + NB_ETHPORTS_USED); > > > > + return -1; > > > > + } > > > > + > > > > + if (nb_ports > RTE_MAX_ETHPORTS) > > > > + nb_ports = RTE_MAX_ETHPORTS; > > > > + > > > > + nb_lcores = rte_lcore_count(); > > > > + > > > > + memset(lcore_conf, 0, sizeof(lcore_conf)); > > > > + init_lcores(); > > > > + > > > > + init_mbufpool(NB_MBUF); > > > > + > > > > + reset_count(); > > > > + num = 0; > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > + if (socketid == -1) { > > > > + socketid = rte_eth_dev_socket_id(portid); > > > > + slave_id = alloc_lcore(socketid); > > > > + if (slave_id == (uint16_t)-1) { > > > > + printf("No avail lcore to run test\n"); > > > > + return -1; > > > > + } > > > > + printf("Performance test runs on lcore %u socket %u\n", > > > > + slave_id, socketid); > > > > + } > > > > + > > > > + if (socketid != rte_eth_dev_socket_id(portid)) { > > > > + printf("Skip port %d\n", portid); > > > > + continue; > > > > + } > > > > + > > > > + /* port configure */ > > > > + ret = rte_eth_dev_configure(portid, nb_rx_queue, > > > > + nb_tx_queue, &port_conf); > > > > + if (ret < 0) > > > > + rte_exit(EXIT_FAILURE, > > > > + "Cannot configure device: err=%d, port=%d\n", > > > > + ret, portid); > > > > + > > > > + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); > > > > + printf("Port %u ", portid); > > > > + print_ethaddr("Address:", &ports_eth_addr[portid]); > > > > + printf("\n"); > > > > + > > > > + /* tx queue setup */ > > > > + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, > > > > + socketid, &tx_conf); > > > > + if (ret < 0) > > > > + rte_exit(EXIT_FAILURE, > > > > + "rte_eth_tx_queue_setup: err=%d, " > > > > + "port=%d\n", ret, portid); > > > > + > > > > + /* rx queue steup */ > > > > + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, > > > > + socketid, &rx_conf, > > > > + mbufpool[socketid]); > > > > + if (ret < 0) > > > > + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," > > > > + "port=%d\n", ret, portid); > > > > + > > > > + /* Start device */ > > > > + stop = 0; > > > > + ret = rte_eth_dev_start(portid); > > > > + if (ret < 0) > > > > + rte_exit(EXIT_FAILURE, > > > > + "rte_eth_dev_start: err=%d, port=%d\n", > > > > + ret, portid); > > > > + > > > > + /* always eanble promiscuous */ > > > > + rte_eth_promiscuous_enable(portid); > > > > + > > > > + lcore_conf[slave_id].portlist[num++] = portid; > > > > + lcore_conf[slave_id].nb_ports++; > > > > + } > > > > + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > > > > + > > > > + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); > > > > + > > > > + rte_eal_remote_launch(main_loop, NULL, slave_id); > > > > + if (rte_eal_wait_lcore(slave_id) < 0) > > > > + return -1; > > > > + > > > > + /* port tear down */ > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > + if (socketid != rte_eth_dev_socket_id(portid)) > > > > + continue; > > > > + > > > > + rte_eth_dev_stop(portid); > > > > + } > > > > + > > > Clean up your allocated memory/lcores/etc? > > [Liang, Cunming] It do cleanup on the beginning of case. > > "Init_lcores","init_mbufpool","reset_count" guarantee the data clean before > each testing. > > And mbufpool only allocated once even if we run multiple times. > Its a janitorial issue. Before the program exits, you need to free any > resources that you've allocated. [Liang, Cunming] Yes, I do but only not for the mbuf pool. Firstly the case exit is not equal to program exit. I expect the re-enter will use the same mempool, so that can do cache warm up. Secondly, DPDK doesn't supply really mempool release. The program exit, the memzone then free. > > Neil > > > > > > > Neil > > > > ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-21 13:17 ` Liang, Cunming @ 2014-10-22 14:03 ` Neil Horman 2014-10-22 14:48 ` Liang, Cunming 2014-10-22 14:53 ` Ananyev, Konstantin 0 siblings, 2 replies; 65+ messages in thread From: Neil Horman @ 2014-10-22 14:03 UTC (permalink / raw) To: Liang, Cunming; +Cc: dev On Tue, Oct 21, 2014 at 01:17:01PM +0000, Liang, Cunming wrote: > > > > -----Original Message----- > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > Sent: Tuesday, October 21, 2014 6:33 PM > > To: Liang, Cunming > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > cycles/packet > > > > On Sun, Oct 12, 2014 at 11:10:39AM +0000, Liang, Cunming wrote: > > > Hi Neil, > > > > > > Very appreciate your comments. > > > I add inline reply, will send v3 asap when we get alignment. > > > > > > BRs, > > > Liang Cunming > > > > > > > -----Original Message----- > > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > > Sent: Saturday, October 11, 2014 1:52 AM > > > > To: Liang, Cunming > > > > Cc: dev@dpdk.org > > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > cycles/packet > > > > > > > > On Fri, Oct 10, 2014 at 08:29:58PM +0800, Cunming Liang wrote: > > > > > It provides unit test to measure cycles/packet in NIC loopback mode. > > > > > It simply gives the average cycles of IO used per packet without test > > equipment. > > > > > When doing the test, make sure the link is UP. > > > > > > > > > > Usage Example: > > > > > 1. Run unit test app in interactive mode > > > > > app/test -c f -n 4 -- -i > > > > > 2. Run and wait for the result > > > > > pmd_perf_autotest > > > > > > > > > > There's option to choose rx/tx pair, default is vector. > > > > > set_rxtx_mode [vector|scalar|full|hybrid] > > > > > Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without > > > > INC_VEC=y in config > > > > > > > > > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > > > > > Acked-by: Bruce Richardson <bruce.richardson@intel.com> > > > > > > > > Notes inline > > > > > > > > > --- > > > > > app/test/Makefile | 1 + > > > > > app/test/commands.c | 38 +++ > > > > > app/test/packet_burst_generator.c | 4 +- > > > > > app/test/test.h | 4 + > > > > > app/test/test_pmd_perf.c | 626 > > > > +++++++++++++++++++++++++++++++++++ > > > > > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > > > > > 6 files changed, 677 insertions(+), 2 deletions(-) > > > > > create mode 100644 app/test/test_pmd_perf.c > > > > > > > > > > diff --git a/app/test/Makefile b/app/test/Makefile > > > > > index 6af6d76..ebfa0ba 100644 > > > > > --- a/app/test/Makefile > > > > > +++ b/app/test/Makefile > > > > > @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c > > > > > > > > > > SRCS-y += test_ring.c > > > > > SRCS-y += test_ring_perf.c > > > > > +SRCS-y += test_pmd_perf.c > > > > > > > > > > ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) > > > > > SRCS-y += test_table.c > > > > > diff --git a/app/test/commands.c b/app/test/commands.c > > > > > index a9e36b1..f1e746e 100644 > > > > > --- a/app/test/commands.c > > > > > +++ b/app/test/commands.c > > > > > @@ -310,12 +310,50 @@ cmdline_parse_inst_t cmd_quit = { > > > > > > > > > > +#define NB_ETHPORTS_USED (1) > > > > > +#define NB_SOCKETS (2) > > > > > +#define MEMPOOL_CACHE_SIZE 250 > > > > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + > > > > RTE_PKTMBUF_HEADROOM) > > > > Don't you want to size this in accordance with the amount of data your > > sending > > > > (64 Bytes as noted above)? > > > [Liang, Cunming] The case is designed to measure small packet IO cost with > > normal mbuf size. > > > Even if decreasing the size, it won't gain significant cycles. > > > > > > That presumes a non-memory constrained system, doesn't it? I suppose in the > > end > > as long as you have consistency, its not overly relevant, but it seems like > > you'll want to add data sizing as a feature to this eventually (i.e. the ability > > to test performance for larger frames sizes), at which point you'll need to make > > this non-static anyway. > [Liang, Cunming] For a normal Ethernet packet(w/o jumbo frame), packet size is 1518B. > As in really network, there won't have huge number of jumbo frames. > The mbuf size 2048 is a reasonable value to cover most of the packet size. > It's also be chosen by lots of NIC as the default receiving buffer size in DMA register. > In case larger than the size, it need do scatter and gather but lose some performance. > The unit test won't measure size from 64 to 9600, won't plan to measure scatter-gather rx/tx. > It focus on 64B packet size and taking the mbuf size being used the most often. Fine. > > > > > > > +static void > > > > > +print_ethaddr(const char *name, const struct ether_addr *eth_addr) > > > > > +{ > > > > > + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, > > > > > + eth_addr->addr_bytes[0], > > > > > + eth_addr->addr_bytes[1], > > > > > + eth_addr->addr_bytes[2], > > > > > + eth_addr->addr_bytes[3], > > > > > + eth_addr->addr_bytes[4], > > > > > + eth_addr->addr_bytes[5]); > > > > > +} > > > > > + > > > > This was copieed from print_ethaddr. Seems like a good candidate for a > > common > > > > function in rte_ether.h > > > [Liang, Cunming] Agree with you, some of samples now use it with the same > > copy. > > > I'll rework it. Adding 'ether_format_addr' in rte_ether.h only for format the > > 48bits address output. > > > And leaving other prints for application customization. > > > > > > Sounds good. > > > > > > > > > > > +} > > > > > + > > > > > +static void > > > > > +signal_handler(int signum) > > > > > +{ > > > > > + /* When we receive a USR1 signal, print stats */ > > > > I think you mean SIGUSR2, below, SIGUSR1 tears the test down and exits the > > > > program > > > [Liang, Cunming] Thanks, it's a typo. > > > > > > > > > + if (signum == SIGUSR1) { > > > > SIGINT instead. Thats the common practice. > > > [Liang, Cunming] I understood your opinion. > > > The considerations I'm not using SIGINT instead are: > > > 1. We unset ISIG in c_lflag of term. CRTL+C won't trigger SIGINT in command > > interactive. > > > It always has to explicitly send signal. No matter SIGUSR1 or SIGINT. > > > 2. By SIGINT semantic, expect to terminate the process. > > > Here I expect to force stop this case, but still alive in command line. > > > After it stopped, it can run again or start to run other test cases. > > > So I keep SIGINT, SIGUSR1 in different behavior. > > > 3. It should be rarely used. > > > Only when exception timeout, I leave this backdoor for automation test > > control. > > > For manual test, we can easily force kill the process. > > > > > Hmm, ok, that sounds reasonable. > > > > > > > > > > > + printf("Force Stop!\n"); > > > > > + stop = 1; > > > > > + } > > > > > + if (signum == SIGUSR2) > > > > > + stats_display(0); > > > > > +} > > > > > +/* main processing loop */ > > > > > +static int > > > > > +main_loop(__rte_unused void *args) > > > > > +{ > > > > > +#define PACKET_SIZE 64 > > > > > +#define FRAME_GAP 12 > > > > > +#define MAC_PREAMBLE 8 > > > > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > > > > + unsigned lcore_id; > > > > > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > > > > > + struct lcore_conf *conf; > > > > > + uint64_t prev_tsc, cur_tsc; > > > > > + int pkt_per_port; > > > > > + uint64_t packets_per_second, total_packets; > > > > > + > > > > > + lcore_id = rte_lcore_id(); > > > > > + conf = &lcore_conf[lcore_id]; > > > > > + if (conf->status != LCORE_USED) > > > > > + return 0; > > > > > + > > > > > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > > > > > + > > > > > + int idx = 0; > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > + int num = pkt_per_port; > > > > > + portid = conf->portlist[i]; > > > > > + printf("inject %d packet to port %d\n", num, portid); > > > > > + while (num) { > > > > > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, > > > > > + &tx_burst[idx], nb_tx); > > > > > + num -= nb_tx; > > > > > + idx += nb_tx; > > > > > + } > > > > > + } > > > > > + printf("Total packets inject to prime ports = %u\n", idx); > > > > > + > > > > > + packets_per_second = (link_mbps * 1000 * 1000) / > > > > > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); > > > > > + printf("Each port will do %"PRIu64" packets per second\n", > > > > > + +packets_per_second); > > > > > + > > > > > + total_packets = RTE_TEST_DURATION * conf->nb_ports * > > > > packets_per_second; > > > > > + printf("Test will stop after at least %"PRIu64" packets received\n", > > > > > + + total_packets); > > > > > + > > > > > + prev_tsc = rte_rdtsc(); > > > > > + > > > > > + while (likely(!stop)) { > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > + portid = conf->portlist[i]; > > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > > + pkts_burst, MAX_PKT_BURST); > > > > > + if (unlikely(nb_rx == 0)) { > > > > > + idle++; > > > > > + continue; > > > > > + } > > > > > + > > > > > + count += nb_rx; > > > > Doesn't take into consideration error conditions. rte_eth_rx_burst can > > return > > > > -ENOTSUP > > > [Liang, Cunming] It returns -ENOTSUP when turning on ETHDEV_DEBUG > > CONFIG. > > > The error is used to avoid no function call register. > > > When ETHDEV_DEBUG turn off, the NULL function call cause segfault directly. > > > So I think it's a library internal error. > > No, look at FUNC_PTR_OR_ERR_RET. If the passed in function pointer is null, > > -ENOTSUPP will be returned to the application, you need to handle the error > > condition. > [Liang, Cunming] The runtime rte_eth_rx_burst is a inline function in rte_ethdev.h. > The one you're talking about is the one defined in rte_ethdev.c which is a extern non-inline function. > It works only when RTE_LIBRTE_ETHDEV_DEBUG turns on. > If we always turns such library internal checking on, it lose performance. > So I insist it's a library internal error checking, doesn't need to take care in application level. I'm really flored that you would respond this way. What you have is two versions of a function, one returns errors and one doesn't, and the version used is based on compile time configuration. You've written your application such that it can only gracefully handle one of the two versions, and you're happy to allow the other version, the one thats supposed to be in use when you are debugging applications, to create silent accounting failures in your application. And it completely ignores the fact that the non-debug version might one day return errors as well (even if it doesn't currently). Your assertion above that we can ignore it because its debug code is the most short sighted thing I've read in a long time. > > > > > In such library exceptional case, I prefer not expecting sample/application to > > condition check library functional error. > > But you're assertion about the library handling the exception is wrong. > > > > > > > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); > > > > Ditto with -ENOTSUP > > > > > > > > > + if (unlikely(nb_tx < nb_rx)) { > > > > What makes this unlikely? Seems like a perfectly reasonable condition to > > happen > > > > to me. If the network is busy, its completely likely that you will receive more > > > > frames than you send, if you elect to receive all frames. > > > [Liang, Cunming] For this case, NIC works in MAC loopback mode. > > > It firstly injects numbers of packets to NIC. Then NIC will loopback all packets to > > ingress side. > > > The code here will receive the packets, and send them back to NIC > > > Packets looping inside all come from initial injection. > > > As the total number of injected packets is much less than in-chip queue size, > > the tx egress queue shouldn't block desc. ring update. > > > So when receiving packets in line rate and nb_tx < nb_rx, it means tx cannot > > archive line rate. > > > When it happens, the cycles/packets result make no sense, as the bottle neck is > > NIC. > > > The drop counter can record it. > > Ok. > > > > > > > > > > > + drop += (nb_rx - nb_tx); > > > > > + do { > > > > > + rte_pktmbuf_free(pkts_burst[nb_tx]); > > > > Defer this, it skews your timing > > > [Liang, Cunming] Agree with you, I ever thought about it. > > > This test cases is designed to measure pure IO RX/TX routine. > > > When doing tx burst with result nb_tx < nb_rx, we either repeat re-send or > > drop it. > > > Each way introduces noise(adding additional control code), resending much > > times even cost more than free it. > > > The cycles/packets is useful when there's no packet drop, otherwise it gives the > > hint where the problem comes from (by idle or drop). > > I'm not sure what you're asserting here. Are you suggesting that you want to > > include the time it takes to free memory buffers in your testing? That seems > > dubious at best to me. If you want to measure I/O, thats great, but memory > > management of packet buffers is a separate operation from that I/O > [Liang, Cunming] Agree with you. I means it doesn't need to take care of the mbuf free cost. > As I said in previous reply, it rarely happens in line rate. > The cycle measurement doesn't make much sense if happens. > On that time the unit test just notify it happens, and keep safe free all the unsent mbuf. > > > > > > > > > > > > + } while (++nb_tx < nb_rx); > > > > > + } > > > > > + } > > > > > + if (unlikely(count >= total_packets)) > > > > > + break; > > > > Whats the reasoning here? Do you only ever expect to receive frames that > > you > > > > send? If so, seems like this should call for a big warning to get printed. > > > [Liang, Cunming] The loop exits when the pre-calculated total_packets are > > received. > > > As the nb_rx is unpredictable, the packet counter may large equal than > > total_packets the last time. > > > The reason unlikely used here is because the condition becomes true only the > > last time. > > ok > > > > > > > > > > > + } > > > > > + > > > > > + cur_tsc = rte_rdtsc(); > > > > > + > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > + portid = conf->portlist[i]; > > > > > + int nb_free = pkt_per_port; > > > > > + do { /* dry out */ > > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > > + pkts_burst, MAX_PKT_BURST); > > > > > + nb_tx = 0; > > > > > + while (nb_tx < nb_rx) > > > > > + rte_pktmbuf_free(pkts_burst[nb_tx++]); > > > > > + nb_free -= nb_rx; > > > > > + } while (nb_free != 0); > > > > > + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); > > > > > + } > > > > > + > > > > Whats the purpose of this? Are you trying to flush the device? Wouldn't it > > be > > > > enough just to stop the interface? > > > [Liang, Cunming] If we only run the cases once and exit, it doesn't matter. > > > But it's designed to run multi-times without exit, for the purpose of warming > > up or for getting average number. > > > So stopping device is not enough, we have to release the flying packets. > > > > > Ok > > > > > > > > > > > + if (count == 0) > > > > > + return -1; > > > > > + > > > > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > > > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > > > > > + > > > > Bad math here. Theres no guarantee that the tsc hasn't wrapped > > (potentially > > > > more than once) depending on your test length. you need to check the tsc > > before > > > > and after each burst and record an average of deltas instead, accounting in > > each > > > > instance for the possibility of wrap. > > > [Liang, Cunming] I'm not sure catch your point correctly. > > > I think both cur_tsc and prev_tsc are 64 bits width. > > > For 3GHz, I think it won't wrapped so quick. > > > As it's uint64_t, so even get wrapped, the delta should still be correct. > > But theres no guarantee that the tsc starts at zero when you begin your test. > > The system may have been up for a long time and near wrapping already. > > Regardless, you need to account for the possibility that cur_tsc is smaller > > than prev_tsc, or this breaks. > [Liang, Cunming] In case prev_tsc near wrapping, and cur_tsc get wrapped. > As they are unsigned, it still ok. > e.g. cur_tsc=0x2, prev_tsc=0xFFFFFFFFFFFFFFFC > Delta=cur_tsc-prev_tsc is 6, which is still correct. Ah, you're right, my fault. > And for uint64_t, we need to start computer for hundreds of years. That only assumes that the tsc has been kept at its initial state from boot, its entirely possible that the tsc is writen to a different value. IIRC, in several debug configurations, the linux kernel inits the tsc to a large value so that it rolls over shortly after boot to catch wrapping errors. Though as you note, your used of uint64 makes it irrelevant. > > > > > > > > > > > + return 0; > > > > > +} > > > > > + > > > > > +static int > > > > > +test_pmd_perf(void) > > > > > +{ > > > > > + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; > > > > > + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; > > > > > + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; > > > > > + uint16_t portid; > > > > > + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; > > > > > + int socketid = -1; > > > > > + int ret; > > > > > + > > > > > + printf("Start PMD RXTX cycles cost test.\n"); > > > > > + > > > > > + signal(SIGUSR1, signal_handler); > > > > Again SIGINT here. > > > > > > > > > + signal(SIGUSR2, signal_handler); > > > > > + > > > > > + nb_ports = rte_eth_dev_count(); > > > > > + if (nb_ports < NB_ETHPORTS_USED) { > > > > > + printf("At least %u port(s) used for perf. test\n", > > > > > + NB_ETHPORTS_USED); > > > > > + return -1; > > > > > + } > > > > > + > > > > > + if (nb_ports > RTE_MAX_ETHPORTS) > > > > > + nb_ports = RTE_MAX_ETHPORTS; > > > > > + > > > > > + nb_lcores = rte_lcore_count(); > > > > > + > > > > > + memset(lcore_conf, 0, sizeof(lcore_conf)); > > > > > + init_lcores(); > > > > > + > > > > > + init_mbufpool(NB_MBUF); > > > > > + > > > > > + reset_count(); > > > > > + num = 0; > > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > > + if (socketid == -1) { > > > > > + socketid = rte_eth_dev_socket_id(portid); > > > > > + slave_id = alloc_lcore(socketid); > > > > > + if (slave_id == (uint16_t)-1) { > > > > > + printf("No avail lcore to run test\n"); > > > > > + return -1; > > > > > + } > > > > > + printf("Performance test runs on lcore %u socket %u\n", > > > > > + slave_id, socketid); > > > > > + } > > > > > + > > > > > + if (socketid != rte_eth_dev_socket_id(portid)) { > > > > > + printf("Skip port %d\n", portid); > > > > > + continue; > > > > > + } > > > > > + > > > > > + /* port configure */ > > > > > + ret = rte_eth_dev_configure(portid, nb_rx_queue, > > > > > + nb_tx_queue, &port_conf); > > > > > + if (ret < 0) > > > > > + rte_exit(EXIT_FAILURE, > > > > > + "Cannot configure device: err=%d, port=%d\n", > > > > > + ret, portid); > > > > > + > > > > > + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); > > > > > + printf("Port %u ", portid); > > > > > + print_ethaddr("Address:", &ports_eth_addr[portid]); > > > > > + printf("\n"); > > > > > + > > > > > + /* tx queue setup */ > > > > > + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, > > > > > + socketid, &tx_conf); > > > > > + if (ret < 0) > > > > > + rte_exit(EXIT_FAILURE, > > > > > + "rte_eth_tx_queue_setup: err=%d, " > > > > > + "port=%d\n", ret, portid); > > > > > + > > > > > + /* rx queue steup */ > > > > > + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, > > > > > + socketid, &rx_conf, > > > > > + mbufpool[socketid]); > > > > > + if (ret < 0) > > > > > + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," > > > > > + "port=%d\n", ret, portid); > > > > > + > > > > > + /* Start device */ > > > > > + stop = 0; > > > > > + ret = rte_eth_dev_start(portid); > > > > > + if (ret < 0) > > > > > + rte_exit(EXIT_FAILURE, > > > > > + "rte_eth_dev_start: err=%d, port=%d\n", > > > > > + ret, portid); > > > > > + > > > > > + /* always eanble promiscuous */ > > > > > + rte_eth_promiscuous_enable(portid); > > > > > + > > > > > + lcore_conf[slave_id].portlist[num++] = portid; > > > > > + lcore_conf[slave_id].nb_ports++; > > > > > + } > > > > > + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > > > > > + > > > > > + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); > > > > > + > > > > > + rte_eal_remote_launch(main_loop, NULL, slave_id); > > > > > + if (rte_eal_wait_lcore(slave_id) < 0) > > > > > + return -1; > > > > > + > > > > > + /* port tear down */ > > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > > + if (socketid != rte_eth_dev_socket_id(portid)) > > > > > + continue; > > > > > + > > > > > + rte_eth_dev_stop(portid); > > > > > + } > > > > > + > > > > Clean up your allocated memory/lcores/etc? > > > [Liang, Cunming] It do cleanup on the beginning of case. > > > "Init_lcores","init_mbufpool","reset_count" guarantee the data clean before > > each testing. > > > And mbufpool only allocated once even if we run multiple times. > > Its a janitorial issue. Before the program exits, you need to free any > > resources that you've allocated. > [Liang, Cunming] Yes, I do but only not for the mbuf pool. > Firstly the case exit is not equal to program exit. I expect the re-enter will use the same mempool, so that can do cache warm up. > Secondly, DPDK doesn't supply really mempool release. The program exit, the memzone then free. I continue to find the fact that freeing mempools isn't supported is rediculous, but I suppose thats neither here nor there, you need to be able to clean up resources that you allocate, weather you are exiting a test case, a program, or simply a function, its just not sane to not be able to do that. Given the return code check above, all thats left to say I think is NAK to this code. If Thomas wants to ignore me, I suppose thats his perogative, but I can't see how you can ignore that case. Neil ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-22 14:03 ` Neil Horman @ 2014-10-22 14:48 ` Liang, Cunming 2014-10-22 14:53 ` Ananyev, Konstantin 1 sibling, 0 replies; 65+ messages in thread From: Liang, Cunming @ 2014-10-22 14:48 UTC (permalink / raw) To: Neil Horman; +Cc: dev > -----Original Message----- > From: Neil Horman [mailto:nhorman@tuxdriver.com] > Sent: Wednesday, October 22, 2014 10:03 PM > To: Liang, Cunming > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > cycles/packet > > On Tue, Oct 21, 2014 at 01:17:01PM +0000, Liang, Cunming wrote: > > > > > > > -----Original Message----- > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > Sent: Tuesday, October 21, 2014 6:33 PM > > > To: Liang, Cunming > > > Cc: dev@dpdk.org > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > > cycles/packet > > > > > > On Sun, Oct 12, 2014 at 11:10:39AM +0000, Liang, Cunming wrote: > > > > Hi Neil, > > > > > > > > Very appreciate your comments. > > > > I add inline reply, will send v3 asap when we get alignment. > > > > > > > > BRs, > > > > Liang Cunming > > > > > > > > > -----Original Message----- > > > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > > > Sent: Saturday, October 11, 2014 1:52 AM > > > > > To: Liang, Cunming > > > > > Cc: dev@dpdk.org > > > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > > cycles/packet > > > > > > > > > > On Fri, Oct 10, 2014 at 08:29:58PM +0800, Cunming Liang wrote: > > > > > > It provides unit test to measure cycles/packet in NIC loopback mode. > > > > > > It simply gives the average cycles of IO used per packet without test > > > equipment. > > > > > > When doing the test, make sure the link is UP. > > > > > > > > > > > > Usage Example: > > > > > > 1. Run unit test app in interactive mode > > > > > > app/test -c f -n 4 -- -i > > > > > > 2. Run and wait for the result > > > > > > pmd_perf_autotest > > > > > > > > > > > > There's option to choose rx/tx pair, default is vector. > > > > > > set_rxtx_mode [vector|scalar|full|hybrid] > > > > > > Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' > without > > > > > INC_VEC=y in config > > > > > > > > > > > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > > > > > > Acked-by: Bruce Richardson <bruce.richardson@intel.com> > > > > > > > > > > Notes inline > > > > > > > > > > > --- > > > > > > app/test/Makefile | 1 + > > > > > > app/test/commands.c | 38 +++ > > > > > > app/test/packet_burst_generator.c | 4 +- > > > > > > app/test/test.h | 4 + > > > > > > app/test/test_pmd_perf.c | 626 > > > > > +++++++++++++++++++++++++++++++++++ > > > > > > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > > > > > > 6 files changed, 677 insertions(+), 2 deletions(-) > > > > > > create mode 100644 app/test/test_pmd_perf.c > > > > > > > > > > > > diff --git a/app/test/Makefile b/app/test/Makefile > > > > > > index 6af6d76..ebfa0ba 100644 > > > > > > --- a/app/test/Makefile > > > > > > +++ b/app/test/Makefile > > > > > > @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c > > > > > > > > > > > > SRCS-y += test_ring.c > > > > > > SRCS-y += test_ring_perf.c > > > > > > +SRCS-y += test_pmd_perf.c > > > > > > > > > > > > ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) > > > > > > SRCS-y += test_table.c > > > > > > diff --git a/app/test/commands.c b/app/test/commands.c > > > > > > index a9e36b1..f1e746e 100644 > > > > > > --- a/app/test/commands.c > > > > > > +++ b/app/test/commands.c > > > > > > @@ -310,12 +310,50 @@ cmdline_parse_inst_t cmd_quit = { > > > > > > > > > > > > +#define NB_ETHPORTS_USED (1) > > > > > > +#define NB_SOCKETS (2) > > > > > > +#define MEMPOOL_CACHE_SIZE 250 > > > > > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + > > > > > RTE_PKTMBUF_HEADROOM) > > > > > Don't you want to size this in accordance with the amount of data your > > > sending > > > > > (64 Bytes as noted above)? > > > > [Liang, Cunming] The case is designed to measure small packet IO cost with > > > normal mbuf size. > > > > Even if decreasing the size, it won't gain significant cycles. > > > > > > > > That presumes a non-memory constrained system, doesn't it? I suppose in > the > > > end > > > as long as you have consistency, its not overly relevant, but it seems like > > > you'll want to add data sizing as a feature to this eventually (i.e. the ability > > > to test performance for larger frames sizes), at which point you'll need to > make > > > this non-static anyway. > > [Liang, Cunming] For a normal Ethernet packet(w/o jumbo frame), packet size is > 1518B. > > As in really network, there won't have huge number of jumbo frames. > > The mbuf size 2048 is a reasonable value to cover most of the packet size. > > It's also be chosen by lots of NIC as the default receiving buffer size in DMA > register. > > In case larger than the size, it need do scatter and gather but lose some > performance. > > The unit test won't measure size from 64 to 9600, won't plan to measure > scatter-gather rx/tx. > > It focus on 64B packet size and taking the mbuf size being used the most often. > Fine. > > > > > > > > > > +static void > > > > > > +print_ethaddr(const char *name, const struct ether_addr *eth_addr) > > > > > > +{ > > > > > > + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, > > > > > > + eth_addr->addr_bytes[0], > > > > > > + eth_addr->addr_bytes[1], > > > > > > + eth_addr->addr_bytes[2], > > > > > > + eth_addr->addr_bytes[3], > > > > > > + eth_addr->addr_bytes[4], > > > > > > + eth_addr->addr_bytes[5]); > > > > > > +} > > > > > > + > > > > > This was copieed from print_ethaddr. Seems like a good candidate for a > > > common > > > > > function in rte_ether.h > > > > [Liang, Cunming] Agree with you, some of samples now use it with the > same > > > copy. > > > > I'll rework it. Adding 'ether_format_addr' in rte_ether.h only for format the > > > 48bits address output. > > > > And leaving other prints for application customization. > > > > > > > > Sounds good. > > > > > > > > > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +signal_handler(int signum) > > > > > > +{ > > > > > > + /* When we receive a USR1 signal, print stats */ > > > > > I think you mean SIGUSR2, below, SIGUSR1 tears the test down and exits > the > > > > > program > > > > [Liang, Cunming] Thanks, it's a typo. > > > > > > > > > > > + if (signum == SIGUSR1) { > > > > > SIGINT instead. Thats the common practice. > > > > [Liang, Cunming] I understood your opinion. > > > > The considerations I'm not using SIGINT instead are: > > > > 1. We unset ISIG in c_lflag of term. CRTL+C won't trigger SIGINT in > command > > > interactive. > > > > It always has to explicitly send signal. No matter SIGUSR1 or SIGINT. > > > > 2. By SIGINT semantic, expect to terminate the process. > > > > Here I expect to force stop this case, but still alive in command line. > > > > After it stopped, it can run again or start to run other test cases. > > > > So I keep SIGINT, SIGUSR1 in different behavior. > > > > 3. It should be rarely used. > > > > Only when exception timeout, I leave this backdoor for automation test > > > control. > > > > For manual test, we can easily force kill the process. > > > > > > > Hmm, ok, that sounds reasonable. > > > > > > > > > > > > > > + printf("Force Stop!\n"); > > > > > > + stop = 1; > > > > > > + } > > > > > > + if (signum == SIGUSR2) > > > > > > + stats_display(0); > > > > > > +} > > > > > > +/* main processing loop */ > > > > > > +static int > > > > > > +main_loop(__rte_unused void *args) > > > > > > +{ > > > > > > +#define PACKET_SIZE 64 > > > > > > +#define FRAME_GAP 12 > > > > > > +#define MAC_PREAMBLE 8 > > > > > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > > > > > + unsigned lcore_id; > > > > > > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > > > > > > + struct lcore_conf *conf; > > > > > > + uint64_t prev_tsc, cur_tsc; > > > > > > + int pkt_per_port; > > > > > > + uint64_t packets_per_second, total_packets; > > > > > > + > > > > > > + lcore_id = rte_lcore_id(); > > > > > > + conf = &lcore_conf[lcore_id]; > > > > > > + if (conf->status != LCORE_USED) > > > > > > + return 0; > > > > > > + > > > > > > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > > > > > > + > > > > > > + int idx = 0; > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > + int num = pkt_per_port; > > > > > > + portid = conf->portlist[i]; > > > > > > + printf("inject %d packet to port %d\n", num, portid); > > > > > > + while (num) { > > > > > > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, > > > > > > + &tx_burst[idx], nb_tx); > > > > > > + num -= nb_tx; > > > > > > + idx += nb_tx; > > > > > > + } > > > > > > + } > > > > > > + printf("Total packets inject to prime ports = %u\n", idx); > > > > > > + > > > > > > + packets_per_second = (link_mbps * 1000 * 1000) / > > > > > > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * > CHAR_BIT); > > > > > > + printf("Each port will do %"PRIu64" packets per second\n", > > > > > > + +packets_per_second); > > > > > > + > > > > > > + total_packets = RTE_TEST_DURATION * conf->nb_ports * > > > > > packets_per_second; > > > > > > + printf("Test will stop after at least %"PRIu64" packets received\n", > > > > > > + + total_packets); > > > > > > + > > > > > > + prev_tsc = rte_rdtsc(); > > > > > > + > > > > > > + while (likely(!stop)) { > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > + portid = conf->portlist[i]; > > > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > > > + pkts_burst, MAX_PKT_BURST); > > > > > > + if (unlikely(nb_rx == 0)) { > > > > > > + idle++; > > > > > > + continue; > > > > > > + } > > > > > > + > > > > > > + count += nb_rx; > > > > > Doesn't take into consideration error conditions. rte_eth_rx_burst can > > > return > > > > > -ENOTSUP > > > > [Liang, Cunming] It returns -ENOTSUP when turning on ETHDEV_DEBUG > > > CONFIG. > > > > The error is used to avoid no function call register. > > > > When ETHDEV_DEBUG turn off, the NULL function call cause segfault > directly. > > > > So I think it's a library internal error. > > > No, look at FUNC_PTR_OR_ERR_RET. If the passed in function pointer is null, > > > -ENOTSUPP will be returned to the application, you need to handle the error > > > condition. > > [Liang, Cunming] The runtime rte_eth_rx_burst is a inline function in > rte_ethdev.h. > > The one you're talking about is the one defined in rte_ethdev.c which is a > extern non-inline function. > > It works only when RTE_LIBRTE_ETHDEV_DEBUG turns on. > > If we always turns such library internal checking on, it lose performance. > > So I insist it's a library internal error checking, doesn't need to take care in > application level. > I'm really flored that you would respond this way. > > What you have is two versions of a function, one returns errors and one doesn't, > and the version used is based on compile time configuration. You've written > your application such that it can only gracefully handle one of the two > versions, and you're happy to allow the other version, the one thats supposed to > be in use when you are debugging applications, to create silent accounting > failures in your application. And it completely ignores the fact that the > non-debug version might one day return errors as well (even if it doesn't > currently). Your assertion above that we can ignore it because its debug code > is the most short sighted thing I've read in a long time. > [Liang, Cunming] If as you said, two version of function, one returns errors and one doesn't. I'll agree with your point. But in fact in either one, will cause error. Non-debug version will just segment fault as calling NULL function pointer. Debug version will print the error root cause. If you searching the rte_eth_rx_burst calling in DPDK, it's used in every sample, no one checks it. This function call is in fast path, I don't think it's a good idea pay additional cycles on such branch condition. > > > > > > > In such library exceptional case, I prefer not expecting sample/application to > > > condition check library functional error. > > > But you're assertion about the library handling the exception is wrong. > > > > > > > > > > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); > > > > > Ditto with -ENOTSUP > > > > > > > > > > > + if (unlikely(nb_tx < nb_rx)) { > > > > > What makes this unlikely? Seems like a perfectly reasonable condition to > > > happen > > > > > to me. If the network is busy, its completely likely that you will receive > more > > > > > frames than you send, if you elect to receive all frames. > > > > [Liang, Cunming] For this case, NIC works in MAC loopback mode. > > > > It firstly injects numbers of packets to NIC. Then NIC will loopback all > packets to > > > ingress side. > > > > The code here will receive the packets, and send them back to NIC > > > > Packets looping inside all come from initial injection. > > > > As the total number of injected packets is much less than in-chip queue > size, > > > the tx egress queue shouldn't block desc. ring update. > > > > So when receiving packets in line rate and nb_tx < nb_rx, it means tx cannot > > > archive line rate. > > > > When it happens, the cycles/packets result make no sense, as the bottle > neck is > > > NIC. > > > > The drop counter can record it. > > > Ok. > > > > > > > > > > > > > > + drop += (nb_rx - nb_tx); > > > > > > + do { > > > > > > + rte_pktmbuf_free(pkts_burst[nb_tx]); > > > > > Defer this, it skews your timing > > > > [Liang, Cunming] Agree with you, I ever thought about it. > > > > This test cases is designed to measure pure IO RX/TX routine. > > > > When doing tx burst with result nb_tx < nb_rx, we either repeat re-send or > > > drop it. > > > > Each way introduces noise(adding additional control code), resending much > > > times even cost more than free it. > > > > The cycles/packets is useful when there's no packet drop, otherwise it gives > the > > > hint where the problem comes from (by idle or drop). > > > I'm not sure what you're asserting here. Are you suggesting that you want > to > > > include the time it takes to free memory buffers in your testing? That seems > > > dubious at best to me. If you want to measure I/O, thats great, but memory > > > management of packet buffers is a separate operation from that I/O > > [Liang, Cunming] Agree with you. I means it doesn't need to take care of the > mbuf free cost. > > As I said in previous reply, it rarely happens in line rate. > > The cycle measurement doesn't make much sense if happens. > > On that time the unit test just notify it happens, and keep safe free all the > unsent mbuf. > > > > > > > > > > > > > > > > + } while (++nb_tx < nb_rx); > > > > > > + } > > > > > > + } > > > > > > + if (unlikely(count >= total_packets)) > > > > > > + break; > > > > > Whats the reasoning here? Do you only ever expect to receive frames > that > > > you > > > > > send? If so, seems like this should call for a big warning to get printed. > > > > [Liang, Cunming] The loop exits when the pre-calculated total_packets are > > > received. > > > > As the nb_rx is unpredictable, the packet counter may large equal than > > > total_packets the last time. > > > > The reason unlikely used here is because the condition becomes true only > the > > > last time. > > > ok > > > > > > > > > > > > > > + } > > > > > > + > > > > > > + cur_tsc = rte_rdtsc(); > > > > > > + > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > + portid = conf->portlist[i]; > > > > > > + int nb_free = pkt_per_port; > > > > > > + do { /* dry out */ > > > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > > > + pkts_burst, MAX_PKT_BURST); > > > > > > + nb_tx = 0; > > > > > > + while (nb_tx < nb_rx) > > > > > > + rte_pktmbuf_free(pkts_burst[nb_tx++]); > > > > > > + nb_free -= nb_rx; > > > > > > + } while (nb_free != 0); > > > > > > + printf("free %d mbuf left in port %u\n", pkt_per_port, > portid); > > > > > > + } > > > > > > + > > > > > Whats the purpose of this? Are you trying to flush the device? > Wouldn't it > > > be > > > > > enough just to stop the interface? > > > > [Liang, Cunming] If we only run the cases once and exit, it doesn't matter. > > > > But it's designed to run multi-times without exit, for the purpose of > warming > > > up or for getting average number. > > > > So stopping device is not enough, we have to release the flying packets. > > > > > > > Ok > > > > > > > > > > > > > > + if (count == 0) > > > > > > + return -1; > > > > > > + > > > > > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > > > > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / > count); > > > > > > + > > > > > Bad math here. Theres no guarantee that the tsc hasn't wrapped > > > (potentially > > > > > more than once) depending on your test length. you need to check the > tsc > > > before > > > > > and after each burst and record an average of deltas instead, accounting > in > > > each > > > > > instance for the possibility of wrap. > > > > [Liang, Cunming] I'm not sure catch your point correctly. > > > > I think both cur_tsc and prev_tsc are 64 bits width. > > > > For 3GHz, I think it won't wrapped so quick. > > > > As it's uint64_t, so even get wrapped, the delta should still be correct. > > > But theres no guarantee that the tsc starts at zero when you begin your test. > > > The system may have been up for a long time and near wrapping already. > > > Regardless, you need to account for the possibility that cur_tsc is smaller > > > than prev_tsc, or this breaks. > > [Liang, Cunming] In case prev_tsc near wrapping, and cur_tsc get wrapped. > > As they are unsigned, it still ok. > > e.g. cur_tsc=0x2, prev_tsc=0xFFFFFFFFFFFFFFFC > > Delta=cur_tsc-prev_tsc is 6, which is still correct. > Ah, you're right, my fault. > > And for uint64_t, we need to start computer for hundreds of years. > That only assumes that the tsc has been kept at its initial state from boot, its > entirely possible that the tsc is writen to a different value. IIRC, in several > debug configurations, the linux kernel inits the tsc to a large value so that it > rolls over shortly after boot to catch wrapping errors. Though as you note, > your used of uint64 makes it irrelevant. [Liang, Cunming] Ok, you're right. I think we've already got agreement on delta of tsc. So we can close this one. > > > > > > > > > > > > > > > + return 0; > > > > > > +} > > > > > > + > > > > > > +static int > > > > > > +test_pmd_perf(void) > > > > > > +{ > > > > > > + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; > > > > > > + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; > > > > > > + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; > > > > > > + uint16_t portid; > > > > > > + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; > > > > > > + int socketid = -1; > > > > > > + int ret; > > > > > > + > > > > > > + printf("Start PMD RXTX cycles cost test.\n"); > > > > > > + > > > > > > + signal(SIGUSR1, signal_handler); > > > > > Again SIGINT here. > > > > > > > > > > > + signal(SIGUSR2, signal_handler); > > > > > > + > > > > > > + nb_ports = rte_eth_dev_count(); > > > > > > + if (nb_ports < NB_ETHPORTS_USED) { > > > > > > + printf("At least %u port(s) used for perf. test\n", > > > > > > + NB_ETHPORTS_USED); > > > > > > + return -1; > > > > > > + } > > > > > > + > > > > > > + if (nb_ports > RTE_MAX_ETHPORTS) > > > > > > + nb_ports = RTE_MAX_ETHPORTS; > > > > > > + > > > > > > + nb_lcores = rte_lcore_count(); > > > > > > + > > > > > > + memset(lcore_conf, 0, sizeof(lcore_conf)); > > > > > > + init_lcores(); > > > > > > + > > > > > > + init_mbufpool(NB_MBUF); > > > > > > + > > > > > > + reset_count(); > > > > > > + num = 0; > > > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > > > + if (socketid == -1) { > > > > > > + socketid = rte_eth_dev_socket_id(portid); > > > > > > + slave_id = alloc_lcore(socketid); > > > > > > + if (slave_id == (uint16_t)-1) { > > > > > > + printf("No avail lcore to run test\n"); > > > > > > + return -1; > > > > > > + } > > > > > > + printf("Performance test runs on lcore %u socket %u\n", > > > > > > + slave_id, socketid); > > > > > > + } > > > > > > + > > > > > > + if (socketid != rte_eth_dev_socket_id(portid)) { > > > > > > + printf("Skip port %d\n", portid); > > > > > > + continue; > > > > > > + } > > > > > > + > > > > > > + /* port configure */ > > > > > > + ret = rte_eth_dev_configure(portid, nb_rx_queue, > > > > > > + nb_tx_queue, &port_conf); > > > > > > + if (ret < 0) > > > > > > + rte_exit(EXIT_FAILURE, > > > > > > + "Cannot configure device: err=%d, port=%d\n", > > > > > > + ret, portid); > > > > > > + > > > > > > + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); > > > > > > + printf("Port %u ", portid); > > > > > > + print_ethaddr("Address:", &ports_eth_addr[portid]); > > > > > > + printf("\n"); > > > > > > + > > > > > > + /* tx queue setup */ > > > > > > + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, > > > > > > + socketid, &tx_conf); > > > > > > + if (ret < 0) > > > > > > + rte_exit(EXIT_FAILURE, > > > > > > + "rte_eth_tx_queue_setup: err=%d, " > > > > > > + "port=%d\n", ret, portid); > > > > > > + > > > > > > + /* rx queue steup */ > > > > > > + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, > > > > > > + socketid, &rx_conf, > > > > > > + mbufpool[socketid]); > > > > > > + if (ret < 0) > > > > > > + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: > err=%d," > > > > > > + "port=%d\n", ret, portid); > > > > > > + > > > > > > + /* Start device */ > > > > > > + stop = 0; > > > > > > + ret = rte_eth_dev_start(portid); > > > > > > + if (ret < 0) > > > > > > + rte_exit(EXIT_FAILURE, > > > > > > + "rte_eth_dev_start: err=%d, port=%d\n", > > > > > > + ret, portid); > > > > > > + > > > > > > + /* always eanble promiscuous */ > > > > > > + rte_eth_promiscuous_enable(portid); > > > > > > + > > > > > > + lcore_conf[slave_id].portlist[num++] = portid; > > > > > > + lcore_conf[slave_id].nb_ports++; > > > > > > + } > > > > > > + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > > > > > > + > > > > > > + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); > > > > > > + > > > > > > + rte_eal_remote_launch(main_loop, NULL, slave_id); > > > > > > + if (rte_eal_wait_lcore(slave_id) < 0) > > > > > > + return -1; > > > > > > + > > > > > > + /* port tear down */ > > > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > > > + if (socketid != rte_eth_dev_socket_id(portid)) > > > > > > + continue; > > > > > > + > > > > > > + rte_eth_dev_stop(portid); > > > > > > + } > > > > > > + > > > > > Clean up your allocated memory/lcores/etc? > > > > [Liang, Cunming] It do cleanup on the beginning of case. > > > > "Init_lcores","init_mbufpool","reset_count" guarantee the data clean > before > > > each testing. > > > > And mbufpool only allocated once even if we run multiple times. > > > Its a janitorial issue. Before the program exits, you need to free any > > > resources that you've allocated. > > [Liang, Cunming] Yes, I do but only not for the mbuf pool. > > Firstly the case exit is not equal to program exit. I expect the re-enter will use > the same mempool, so that can do cache warm up. > > Secondly, DPDK doesn't supply really mempool release. The program exit, the > memzone then free. > I continue to find the fact that freeing mempools isn't supported is rediculous, > but I suppose thats neither here nor there, you need to be able to clean up > resources that you allocate, weather you are exiting a test case, a program, or > simply a function, its just not sane to not be able to do that. [Liang, Cunming] Got it, agree with you on the principle. > > Given the return code check above, all thats left to say I think is NAK to this > code. If Thomas wants to ignore me, I suppose thats his perogative, but I can't > see how you can ignore that case. > > Neil ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-22 14:03 ` Neil Horman 2014-10-22 14:48 ` Liang, Cunming @ 2014-10-22 14:53 ` Ananyev, Konstantin 2014-10-22 15:09 ` Richardson, Bruce 1 sibling, 1 reply; 65+ messages in thread From: Ananyev, Konstantin @ 2014-10-22 14:53 UTC (permalink / raw) To: Neil Horman, Liang, Cunming; +Cc: dev > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > Sent: Wednesday, October 22, 2014 3:03 PM > To: Liang, Cunming > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet > > On Tue, Oct 21, 2014 at 01:17:01PM +0000, Liang, Cunming wrote: > > > > > > > -----Original Message----- > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > Sent: Tuesday, October 21, 2014 6:33 PM > > > To: Liang, Cunming > > > Cc: dev@dpdk.org > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > > cycles/packet > > > > > > On Sun, Oct 12, 2014 at 11:10:39AM +0000, Liang, Cunming wrote: > > > > Hi Neil, > > > > > > > > Very appreciate your comments. > > > > I add inline reply, will send v3 asap when we get alignment. > > > > > > > > BRs, > > > > Liang Cunming > > > > > > > > > -----Original Message----- > > > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > > > Sent: Saturday, October 11, 2014 1:52 AM > > > > > To: Liang, Cunming > > > > > Cc: dev@dpdk.org > > > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > > cycles/packet > > > > > > > > > > On Fri, Oct 10, 2014 at 08:29:58PM +0800, Cunming Liang wrote: > > > > > > It provides unit test to measure cycles/packet in NIC loopback mode. > > > > > > It simply gives the average cycles of IO used per packet without test > > > equipment. > > > > > > When doing the test, make sure the link is UP. > > > > > > > > > > > > Usage Example: > > > > > > 1. Run unit test app in interactive mode > > > > > > app/test -c f -n 4 -- -i > > > > > > 2. Run and wait for the result > > > > > > pmd_perf_autotest > > > > > > > > > > > > There's option to choose rx/tx pair, default is vector. > > > > > > set_rxtx_mode [vector|scalar|full|hybrid] > > > > > > Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without > > > > > INC_VEC=y in config > > > > > > > > > > > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > > > > > > Acked-by: Bruce Richardson <bruce.richardson@intel.com> > > > > > > > > > > Notes inline > > > > > > > > > > > --- > > > > > > app/test/Makefile | 1 + > > > > > > app/test/commands.c | 38 +++ > > > > > > app/test/packet_burst_generator.c | 4 +- > > > > > > app/test/test.h | 4 + > > > > > > app/test/test_pmd_perf.c | 626 > > > > > +++++++++++++++++++++++++++++++++++ > > > > > > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > > > > > > 6 files changed, 677 insertions(+), 2 deletions(-) > > > > > > create mode 100644 app/test/test_pmd_perf.c > > > > > > > > > > > > diff --git a/app/test/Makefile b/app/test/Makefile > > > > > > index 6af6d76..ebfa0ba 100644 > > > > > > --- a/app/test/Makefile > > > > > > +++ b/app/test/Makefile > > > > > > @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c > > > > > > > > > > > > SRCS-y += test_ring.c > > > > > > SRCS-y += test_ring_perf.c > > > > > > +SRCS-y += test_pmd_perf.c > > > > > > > > > > > > ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) > > > > > > SRCS-y += test_table.c > > > > > > diff --git a/app/test/commands.c b/app/test/commands.c > > > > > > index a9e36b1..f1e746e 100644 > > > > > > --- a/app/test/commands.c > > > > > > +++ b/app/test/commands.c > > > > > > @@ -310,12 +310,50 @@ cmdline_parse_inst_t cmd_quit = { > > > > > > > > > > > > +#define NB_ETHPORTS_USED (1) > > > > > > +#define NB_SOCKETS (2) > > > > > > +#define MEMPOOL_CACHE_SIZE 250 > > > > > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + > > > > > RTE_PKTMBUF_HEADROOM) > > > > > Don't you want to size this in accordance with the amount of data your > > > sending > > > > > (64 Bytes as noted above)? > > > > [Liang, Cunming] The case is designed to measure small packet IO cost with > > > normal mbuf size. > > > > Even if decreasing the size, it won't gain significant cycles. > > > > > > > > That presumes a non-memory constrained system, doesn't it? I suppose in the > > > end > > > as long as you have consistency, its not overly relevant, but it seems like > > > you'll want to add data sizing as a feature to this eventually (i.e. the ability > > > to test performance for larger frames sizes), at which point you'll need to make > > > this non-static anyway. > > [Liang, Cunming] For a normal Ethernet packet(w/o jumbo frame), packet size is 1518B. > > As in really network, there won't have huge number of jumbo frames. > > The mbuf size 2048 is a reasonable value to cover most of the packet size. > > It's also be chosen by lots of NIC as the default receiving buffer size in DMA register. > > In case larger than the size, it need do scatter and gather but lose some performance. > > The unit test won't measure size from 64 to 9600, won't plan to measure scatter-gather rx/tx. > > It focus on 64B packet size and taking the mbuf size being used the most often. > Fine. > > > > > > > > > > +static void > > > > > > +print_ethaddr(const char *name, const struct ether_addr *eth_addr) > > > > > > +{ > > > > > > + printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, > > > > > > + eth_addr->addr_bytes[0], > > > > > > + eth_addr->addr_bytes[1], > > > > > > + eth_addr->addr_bytes[2], > > > > > > + eth_addr->addr_bytes[3], > > > > > > + eth_addr->addr_bytes[4], > > > > > > + eth_addr->addr_bytes[5]); > > > > > > +} > > > > > > + > > > > > This was copieed from print_ethaddr. Seems like a good candidate for a > > > common > > > > > function in rte_ether.h > > > > [Liang, Cunming] Agree with you, some of samples now use it with the same > > > copy. > > > > I'll rework it. Adding 'ether_format_addr' in rte_ether.h only for format the > > > 48bits address output. > > > > And leaving other prints for application customization. > > > > > > > > Sounds good. > > > > > > > > > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +signal_handler(int signum) > > > > > > +{ > > > > > > + /* When we receive a USR1 signal, print stats */ > > > > > I think you mean SIGUSR2, below, SIGUSR1 tears the test down and exits the > > > > > program > > > > [Liang, Cunming] Thanks, it's a typo. > > > > > > > > > > > + if (signum == SIGUSR1) { > > > > > SIGINT instead. Thats the common practice. > > > > [Liang, Cunming] I understood your opinion. > > > > The considerations I'm not using SIGINT instead are: > > > > 1. We unset ISIG in c_lflag of term. CRTL+C won't trigger SIGINT in command > > > interactive. > > > > It always has to explicitly send signal. No matter SIGUSR1 or SIGINT. > > > > 2. By SIGINT semantic, expect to terminate the process. > > > > Here I expect to force stop this case, but still alive in command line. > > > > After it stopped, it can run again or start to run other test cases. > > > > So I keep SIGINT, SIGUSR1 in different behavior. > > > > 3. It should be rarely used. > > > > Only when exception timeout, I leave this backdoor for automation test > > > control. > > > > For manual test, we can easily force kill the process. > > > > > > > Hmm, ok, that sounds reasonable. > > > > > > > > > > > > > > + printf("Force Stop!\n"); > > > > > > + stop = 1; > > > > > > + } > > > > > > + if (signum == SIGUSR2) > > > > > > + stats_display(0); > > > > > > +} > > > > > > +/* main processing loop */ > > > > > > +static int > > > > > > +main_loop(__rte_unused void *args) > > > > > > +{ > > > > > > +#define PACKET_SIZE 64 > > > > > > +#define FRAME_GAP 12 > > > > > > +#define MAC_PREAMBLE 8 > > > > > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > > > > > + unsigned lcore_id; > > > > > > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > > > > > > + struct lcore_conf *conf; > > > > > > + uint64_t prev_tsc, cur_tsc; > > > > > > + int pkt_per_port; > > > > > > + uint64_t packets_per_second, total_packets; > > > > > > + > > > > > > + lcore_id = rte_lcore_id(); > > > > > > + conf = &lcore_conf[lcore_id]; > > > > > > + if (conf->status != LCORE_USED) > > > > > > + return 0; > > > > > > + > > > > > > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > > > > > > + > > > > > > + int idx = 0; > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > + int num = pkt_per_port; > > > > > > + portid = conf->portlist[i]; > > > > > > + printf("inject %d packet to port %d\n", num, portid); > > > > > > + while (num) { > > > > > > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, > > > > > > + &tx_burst[idx], nb_tx); > > > > > > + num -= nb_tx; > > > > > > + idx += nb_tx; > > > > > > + } > > > > > > + } > > > > > > + printf("Total packets inject to prime ports = %u\n", idx); > > > > > > + > > > > > > + packets_per_second = (link_mbps * 1000 * 1000) / > > > > > > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); > > > > > > + printf("Each port will do %"PRIu64" packets per second\n", > > > > > > + +packets_per_second); > > > > > > + > > > > > > + total_packets = RTE_TEST_DURATION * conf->nb_ports * > > > > > packets_per_second; > > > > > > + printf("Test will stop after at least %"PRIu64" packets received\n", > > > > > > + + total_packets); > > > > > > + > > > > > > + prev_tsc = rte_rdtsc(); > > > > > > + > > > > > > + while (likely(!stop)) { > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > + portid = conf->portlist[i]; > > > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > > > + pkts_burst, MAX_PKT_BURST); > > > > > > + if (unlikely(nb_rx == 0)) { > > > > > > + idle++; > > > > > > + continue; > > > > > > + } > > > > > > + > > > > > > + count += nb_rx; > > > > > Doesn't take into consideration error conditions. rte_eth_rx_burst can > > > return > > > > > -ENOTSUP > > > > [Liang, Cunming] It returns -ENOTSUP when turning on ETHDEV_DEBUG > > > CONFIG. > > > > The error is used to avoid no function call register. > > > > When ETHDEV_DEBUG turn off, the NULL function call cause segfault directly. > > > > So I think it's a library internal error. > > > No, look at FUNC_PTR_OR_ERR_RET. If the passed in function pointer is null, > > > -ENOTSUPP will be returned to the application, you need to handle the error > > > condition. > > [Liang, Cunming] The runtime rte_eth_rx_burst is a inline function in rte_ethdev.h. > > The one you're talking about is the one defined in rte_ethdev.c which is a extern non-inline function. > > It works only when RTE_LIBRTE_ETHDEV_DEBUG turns on. > > If we always turns such library internal checking on, it lose performance. > > So I insist it's a library internal error checking, doesn't need to take care in application level. > I'm really flored that you would respond this way. > > What you have is two versions of a function, one returns errors and one doesn't, > and the version used is based on compile time configuration. You've written > your application such that it can only gracefully handle one of the two > versions, and you're happy to allow the other version, the one thats supposed to > be in use when you are debugging applications, to create silent accounting > failures in your application. And it completely ignores the fact that the > non-debug version might one day return errors as well (even if it doesn't > currently). Your assertion above that we can ignore it because its debug code > is the most short sighted thing I've read in a long time. Actually looking at rte_eth_rx_burst(): it returns uint16_t. Which is sort of a strange if it expects to return a negative value as error code. Also reading though 'formal' comments of rte_eth_rx_burst() - it seems that it not supposed to return negative values: "... The rte_eth_rx_burst() function does not provide any error notification to avoid the corresponding overhead. As a hint, the upper-level application might check the status of the device link once being systematically returned a 0 value for a given number of tries. ... @return * The number of packets actually retrieved, which is the number * of pointers to *rte_mbuf* structures effectively supplied to the * *rx_pkts* array." Again, if you looks at rte_eth_rx_burst() implementation, when RTE_LIBRTE_ETHDEV_DEBUG is on - For some error cases it returns '0', foor other's: -ENOTSUP: FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); if (queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); return 0; } Which makes me thing that we just have errnoneous implementation of rte_eth_rx_burst() when RTE_LIBRTE_ETHDEV_DEBUG is on. Probably just a mechanical mistake, when FUNC_PTR_OR_ERR_RET() was added. I'd say we change rte_eth_rx_burst() to always return 0 (as was initially planned). Konstantin > > > > > > > > In such library exceptional case, I prefer not expecting sample/application to > > > condition check library functional error. > > > But you're assertion about the library handling the exception is wrong. > > > > > > > > > > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); > > > > > Ditto with -ENOTSUP > > > > > > > > > > > + if (unlikely(nb_tx < nb_rx)) { > > > > > What makes this unlikely? Seems like a perfectly reasonable condition to > > > happen > > > > > to me. If the network is busy, its completely likely that you will receive more > > > > > frames than you send, if you elect to receive all frames. > > > > [Liang, Cunming] For this case, NIC works in MAC loopback mode. > > > > It firstly injects numbers of packets to NIC. Then NIC will loopback all packets to > > > ingress side. > > > > The code here will receive the packets, and send them back to NIC > > > > Packets looping inside all come from initial injection. > > > > As the total number of injected packets is much less than in-chip queue size, > > > the tx egress queue shouldn't block desc. ring update. > > > > So when receiving packets in line rate and nb_tx < nb_rx, it means tx cannot > > > archive line rate. > > > > When it happens, the cycles/packets result make no sense, as the bottle neck is > > > NIC. > > > > The drop counter can record it. > > > Ok. > > > > > > > > > > > > > > + drop += (nb_rx - nb_tx); > > > > > > + do { > > > > > > + rte_pktmbuf_free(pkts_burst[nb_tx]); > > > > > Defer this, it skews your timing > > > > [Liang, Cunming] Agree with you, I ever thought about it. > > > > This test cases is designed to measure pure IO RX/TX routine. > > > > When doing tx burst with result nb_tx < nb_rx, we either repeat re-send or > > > drop it. > > > > Each way introduces noise(adding additional control code), resending much > > > times even cost more than free it. > > > > The cycles/packets is useful when there's no packet drop, otherwise it gives the > > > hint where the problem comes from (by idle or drop). > > > I'm not sure what you're asserting here. Are you suggesting that you want to > > > include the time it takes to free memory buffers in your testing? That seems > > > dubious at best to me. If you want to measure I/O, thats great, but memory > > > management of packet buffers is a separate operation from that I/O > > [Liang, Cunming] Agree with you. I means it doesn't need to take care of the mbuf free cost. > > As I said in previous reply, it rarely happens in line rate. > > The cycle measurement doesn't make much sense if happens. > > On that time the unit test just notify it happens, and keep safe free all the unsent mbuf. > > > > > > > > > > > > > > > > + } while (++nb_tx < nb_rx); > > > > > > + } > > > > > > + } > > > > > > + if (unlikely(count >= total_packets)) > > > > > > + break; > > > > > Whats the reasoning here? Do you only ever expect to receive frames that > > > you > > > > > send? If so, seems like this should call for a big warning to get printed. > > > > [Liang, Cunming] The loop exits when the pre-calculated total_packets are > > > received. > > > > As the nb_rx is unpredictable, the packet counter may large equal than > > > total_packets the last time. > > > > The reason unlikely used here is because the condition becomes true only the > > > last time. > > > ok > > > > > > > > > > > > > > + } > > > > > > + > > > > > > + cur_tsc = rte_rdtsc(); > > > > > > + > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > + portid = conf->portlist[i]; > > > > > > + int nb_free = pkt_per_port; > > > > > > + do { /* dry out */ > > > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > > > + pkts_burst, MAX_PKT_BURST); > > > > > > + nb_tx = 0; > > > > > > + while (nb_tx < nb_rx) > > > > > > + rte_pktmbuf_free(pkts_burst[nb_tx++]); > > > > > > + nb_free -= nb_rx; > > > > > > + } while (nb_free != 0); > > > > > > + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); > > > > > > + } > > > > > > + > > > > > Whats the purpose of this? Are you trying to flush the device? Wouldn't it > > > be > > > > > enough just to stop the interface? > > > > [Liang, Cunming] If we only run the cases once and exit, it doesn't matter. > > > > But it's designed to run multi-times without exit, for the purpose of warming > > > up or for getting average number. > > > > So stopping device is not enough, we have to release the flying packets. > > > > > > > Ok > > > > > > > > > > > > > > + if (count == 0) > > > > > > + return -1; > > > > > > + > > > > > > + printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); > > > > > > + printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); > > > > > > + > > > > > Bad math here. Theres no guarantee that the tsc hasn't wrapped > > > (potentially > > > > > more than once) depending on your test length. you need to check the tsc > > > before > > > > > and after each burst and record an average of deltas instead, accounting in > > > each > > > > > instance for the possibility of wrap. > > > > [Liang, Cunming] I'm not sure catch your point correctly. > > > > I think both cur_tsc and prev_tsc are 64 bits width. > > > > For 3GHz, I think it won't wrapped so quick. > > > > As it's uint64_t, so even get wrapped, the delta should still be correct. > > > But theres no guarantee that the tsc starts at zero when you begin your test. > > > The system may have been up for a long time and near wrapping already. > > > Regardless, you need to account for the possibility that cur_tsc is smaller > > > than prev_tsc, or this breaks. > > [Liang, Cunming] In case prev_tsc near wrapping, and cur_tsc get wrapped. > > As they are unsigned, it still ok. > > e.g. cur_tsc=0x2, prev_tsc=0xFFFFFFFFFFFFFFFC > > Delta=cur_tsc-prev_tsc is 6, which is still correct. > Ah, you're right, my fault. > > And for uint64_t, we need to start computer for hundreds of years. > That only assumes that the tsc has been kept at its initial state from boot, its > entirely possible that the tsc is writen to a different value. IIRC, in several > debug configurations, the linux kernel inits the tsc to a large value so that it > rolls over shortly after boot to catch wrapping errors. Though as you note, > your used of uint64 makes it irrelevant. > > > > > > > > > > > > > > > + return 0; > > > > > > +} > > > > > > + > > > > > > +static int > > > > > > +test_pmd_perf(void) > > > > > > +{ > > > > > > + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; > > > > > > + uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; > > > > > > + uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; > > > > > > + uint16_t portid; > > > > > > + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; > > > > > > + int socketid = -1; > > > > > > + int ret; > > > > > > + > > > > > > + printf("Start PMD RXTX cycles cost test.\n"); > > > > > > + > > > > > > + signal(SIGUSR1, signal_handler); > > > > > Again SIGINT here. > > > > > > > > > > > + signal(SIGUSR2, signal_handler); > > > > > > + > > > > > > + nb_ports = rte_eth_dev_count(); > > > > > > + if (nb_ports < NB_ETHPORTS_USED) { > > > > > > + printf("At least %u port(s) used for perf. test\n", > > > > > > + NB_ETHPORTS_USED); > > > > > > + return -1; > > > > > > + } > > > > > > + > > > > > > + if (nb_ports > RTE_MAX_ETHPORTS) > > > > > > + nb_ports = RTE_MAX_ETHPORTS; > > > > > > + > > > > > > + nb_lcores = rte_lcore_count(); > > > > > > + > > > > > > + memset(lcore_conf, 0, sizeof(lcore_conf)); > > > > > > + init_lcores(); > > > > > > + > > > > > > + init_mbufpool(NB_MBUF); > > > > > > + > > > > > > + reset_count(); > > > > > > + num = 0; > > > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > > > + if (socketid == -1) { > > > > > > + socketid = rte_eth_dev_socket_id(portid); > > > > > > + slave_id = alloc_lcore(socketid); > > > > > > + if (slave_id == (uint16_t)-1) { > > > > > > + printf("No avail lcore to run test\n"); > > > > > > + return -1; > > > > > > + } > > > > > > + printf("Performance test runs on lcore %u socket %u\n", > > > > > > + slave_id, socketid); > > > > > > + } > > > > > > + > > > > > > + if (socketid != rte_eth_dev_socket_id(portid)) { > > > > > > + printf("Skip port %d\n", portid); > > > > > > + continue; > > > > > > + } > > > > > > + > > > > > > + /* port configure */ > > > > > > + ret = rte_eth_dev_configure(portid, nb_rx_queue, > > > > > > + nb_tx_queue, &port_conf); > > > > > > + if (ret < 0) > > > > > > + rte_exit(EXIT_FAILURE, > > > > > > + "Cannot configure device: err=%d, port=%d\n", > > > > > > + ret, portid); > > > > > > + > > > > > > + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); > > > > > > + printf("Port %u ", portid); > > > > > > + print_ethaddr("Address:", &ports_eth_addr[portid]); > > > > > > + printf("\n"); > > > > > > + > > > > > > + /* tx queue setup */ > > > > > > + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, > > > > > > + socketid, &tx_conf); > > > > > > + if (ret < 0) > > > > > > + rte_exit(EXIT_FAILURE, > > > > > > + "rte_eth_tx_queue_setup: err=%d, " > > > > > > + "port=%d\n", ret, portid); > > > > > > + > > > > > > + /* rx queue steup */ > > > > > > + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, > > > > > > + socketid, &rx_conf, > > > > > > + mbufpool[socketid]); > > > > > > + if (ret < 0) > > > > > > + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," > > > > > > + "port=%d\n", ret, portid); > > > > > > + > > > > > > + /* Start device */ > > > > > > + stop = 0; > > > > > > + ret = rte_eth_dev_start(portid); > > > > > > + if (ret < 0) > > > > > > + rte_exit(EXIT_FAILURE, > > > > > > + "rte_eth_dev_start: err=%d, port=%d\n", > > > > > > + ret, portid); > > > > > > + > > > > > > + /* always eanble promiscuous */ > > > > > > + rte_eth_promiscuous_enable(portid); > > > > > > + > > > > > > + lcore_conf[slave_id].portlist[num++] = portid; > > > > > > + lcore_conf[slave_id].nb_ports++; > > > > > > + } > > > > > > + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > > > > > > + > > > > > > + init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); > > > > > > + > > > > > > + rte_eal_remote_launch(main_loop, NULL, slave_id); > > > > > > + if (rte_eal_wait_lcore(slave_id) < 0) > > > > > > + return -1; > > > > > > + > > > > > > + /* port tear down */ > > > > > > + for (portid = 0; portid < nb_ports; portid++) { > > > > > > + if (socketid != rte_eth_dev_socket_id(portid)) > > > > > > + continue; > > > > > > + > > > > > > + rte_eth_dev_stop(portid); > > > > > > + } > > > > > > + > > > > > Clean up your allocated memory/lcores/etc? > > > > [Liang, Cunming] It do cleanup on the beginning of case. > > > > "Init_lcores","init_mbufpool","reset_count" guarantee the data clean before > > > each testing. > > > > And mbufpool only allocated once even if we run multiple times. > > > Its a janitorial issue. Before the program exits, you need to free any > > > resources that you've allocated. > > [Liang, Cunming] Yes, I do but only not for the mbuf pool. > > Firstly the case exit is not equal to program exit. I expect the re-enter will use the same mempool, so that can do cache warm up. > > Secondly, DPDK doesn't supply really mempool release. The program exit, the memzone then free. > I continue to find the fact that freeing mempools isn't supported is rediculous, > but I suppose thats neither here nor there, you need to be able to clean up > resources that you allocate, weather you are exiting a test case, a program, or > simply a function, its just not sane to not be able to do that. > > Given the return code check above, all thats left to say I think is NAK to this > code. If Thomas wants to ignore me, I suppose thats his perogative, but I can't > see how you can ignore that case. > > Neil ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-22 14:53 ` Ananyev, Konstantin @ 2014-10-22 15:09 ` Richardson, Bruce 2014-10-24 3:06 ` Liang, Cunming 0 siblings, 1 reply; 65+ messages in thread From: Richardson, Bruce @ 2014-10-22 15:09 UTC (permalink / raw) To: Ananyev, Konstantin, Neil Horman, Liang, Cunming; +Cc: dev > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ananyev, Konstantin > Sent: Wednesday, October 22, 2014 3:53 PM > To: Neil Horman; Liang, Cunming > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > cycles/packet > > > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > > Sent: Wednesday, October 22, 2014 3:03 PM > > To: Liang, Cunming > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > cycles/packet > > > > On Tue, Oct 21, 2014 at 01:17:01PM +0000, Liang, Cunming wrote: > > > > > > > > > > -----Original Message----- > > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > > Sent: Tuesday, October 21, 2014 6:33 PM > > > > To: Liang, Cunming > > > > Cc: dev@dpdk.org > > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > > > cycles/packet > > > > > > > > On Sun, Oct 12, 2014 at 11:10:39AM +0000, Liang, Cunming wrote: > > > > > Hi Neil, > > > > > > > > > > Very appreciate your comments. > > > > > I add inline reply, will send v3 asap when we get alignment. > > > > > > > > > > BRs, > > > > > Liang Cunming > > > > > > > > > > > -----Original Message----- > > > > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > > > > Sent: Saturday, October 11, 2014 1:52 AM > > > > > > To: Liang, Cunming > > > > > > Cc: dev@dpdk.org > > > > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > > > cycles/packet > > > > > > <...snip...> > > > > > > > > > > > > > + printf("Force Stop!\n"); > > > > > > > + stop = 1; > > > > > > > + } > > > > > > > + if (signum == SIGUSR2) > > > > > > > + stats_display(0); > > > > > > > +} > > > > > > > +/* main processing loop */ > > > > > > > +static int > > > > > > > +main_loop(__rte_unused void *args) > > > > > > > +{ > > > > > > > +#define PACKET_SIZE 64 > > > > > > > +#define FRAME_GAP 12 > > > > > > > +#define MAC_PREAMBLE 8 > > > > > > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > > > > > > + unsigned lcore_id; > > > > > > > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > > > > > > > + struct lcore_conf *conf; > > > > > > > + uint64_t prev_tsc, cur_tsc; > > > > > > > + int pkt_per_port; > > > > > > > + uint64_t packets_per_second, total_packets; > > > > > > > + > > > > > > > + lcore_id = rte_lcore_id(); > > > > > > > + conf = &lcore_conf[lcore_id]; > > > > > > > + if (conf->status != LCORE_USED) > > > > > > > + return 0; > > > > > > > + > > > > > > > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > > > > > > > + > > > > > > > + int idx = 0; > > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > > + int num = pkt_per_port; > > > > > > > + portid = conf->portlist[i]; > > > > > > > + printf("inject %d packet to port %d\n", num, portid); > > > > > > > + while (num) { > > > > > > > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > > > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, > > > > > > > + &tx_burst[idx], nb_tx); > > > > > > > + num -= nb_tx; > > > > > > > + idx += nb_tx; > > > > > > > + } > > > > > > > + } > > > > > > > + printf("Total packets inject to prime ports = %u\n", idx); > > > > > > > + > > > > > > > + packets_per_second = (link_mbps * 1000 * 1000) / > > > > > > > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * > CHAR_BIT); > > > > > > > + printf("Each port will do %"PRIu64" packets per second\n", > > > > > > > + +packets_per_second); > > > > > > > + > > > > > > > + total_packets = RTE_TEST_DURATION * conf->nb_ports * > > > > > > packets_per_second; > > > > > > > + printf("Test will stop after at least %"PRIu64" packets > received\n", > > > > > > > + + total_packets); > > > > > > > + > > > > > > > + prev_tsc = rte_rdtsc(); > > > > > > > + > > > > > > > + while (likely(!stop)) { > > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > > + portid = conf->portlist[i]; > > > > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > > > > + pkts_burst, > MAX_PKT_BURST); > > > > > > > + if (unlikely(nb_rx == 0)) { > > > > > > > + idle++; > > > > > > > + continue; > > > > > > > + } > > > > > > > + > > > > > > > + count += nb_rx; > > > > > > Doesn't take into consideration error conditions. rte_eth_rx_burst can > > > > return > > > > > > -ENOTSUP > > > > > [Liang, Cunming] It returns -ENOTSUP when turning on ETHDEV_DEBUG > > > > CONFIG. > > > > > The error is used to avoid no function call register. > > > > > When ETHDEV_DEBUG turn off, the NULL function call cause segfault > directly. > > > > > So I think it's a library internal error. > > > > No, look at FUNC_PTR_OR_ERR_RET. If the passed in function pointer is > null, > > > > -ENOTSUPP will be returned to the application, you need to handle the > error > > > > condition. > > > [Liang, Cunming] The runtime rte_eth_rx_burst is a inline function in > rte_ethdev.h. > > > The one you're talking about is the one defined in rte_ethdev.c which is a > extern non-inline function. > > > It works only when RTE_LIBRTE_ETHDEV_DEBUG turns on. > > > If we always turns such library internal checking on, it lose performance. > > > So I insist it's a library internal error checking, doesn't need to take care in > application level. > > I'm really flored that you would respond this way. > > > > What you have is two versions of a function, one returns errors and one > doesn't, > > and the version used is based on compile time configuration. You've written > > your application such that it can only gracefully handle one of the two > > versions, and you're happy to allow the other version, the one thats supposed > to > > be in use when you are debugging applications, to create silent accounting > > failures in your application. And it completely ignores the fact that the > > non-debug version might one day return errors as well (even if it doesn't > > currently). Your assertion above that we can ignore it because its debug code > > is the most short sighted thing I've read in a long time. > > Actually looking at rte_eth_rx_burst(): it returns uint16_t. > Which is sort of a strange if it expects to return a negative value as error code. > Also reading though 'formal' comments of rte_eth_rx_burst() - it seems that it > not supposed to return negative values: > "... > The rte_eth_rx_burst() function does not provide any error > notification to avoid the corresponding overhead. As a hint, the > upper-level application might check the status of the device link once > being systematically returned a 0 value for a given number of tries. > ... > @return > * The number of packets actually retrieved, which is the number > * of pointers to *rte_mbuf* structures effectively supplied to the > * *rx_pkts* array." > > Again, if you looks at rte_eth_rx_burst() implementation, when > RTE_LIBRTE_ETHDEV_DEBUG is on - > For some error cases it returns '0', foor other's: -ENOTSUP: > > FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); > if (queue_id >= dev->data->nb_rx_queues) { > PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); > return 0; > } > > Which makes me thing that we just have errnoneous implementation of > rte_eth_rx_burst() > when RTE_LIBRTE_ETHDEV_DEBUG is on. > Probably just a mechanical mistake, when FUNC_PTR_OR_ERR_RET() was > added. > I'd say we change rte_eth_rx_burst() to always return 0 (as was initially > planned). > > Konstantin Agreed. For any errors other than no-packets available yet, we can also consider setting rte_errno when returning 0. /Bruce ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet 2014-10-22 15:09 ` Richardson, Bruce @ 2014-10-24 3:06 ` Liang, Cunming 0 siblings, 0 replies; 65+ messages in thread From: Liang, Cunming @ 2014-10-24 3:06 UTC (permalink / raw) To: Richardson, Bruce, Ananyev, Konstantin, Neil Horman; +Cc: dev It's reasonable to me. I'll make a patch for rte_ethdev.c. > -----Original Message----- > From: Richardson, Bruce > Sent: Wednesday, October 22, 2014 11:10 PM > To: Ananyev, Konstantin; Neil Horman; Liang, Cunming > Cc: dev@dpdk.org > Subject: RE: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > cycles/packet > > > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ananyev, Konstantin > > Sent: Wednesday, October 22, 2014 3:53 PM > > To: Neil Horman; Liang, Cunming > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > cycles/packet > > > > > > > > > -----Original Message----- > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > > > Sent: Wednesday, October 22, 2014 3:03 PM > > > To: Liang, Cunming > > > Cc: dev@dpdk.org > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > cycles/packet > > > > > > On Tue, Oct 21, 2014 at 01:17:01PM +0000, Liang, Cunming wrote: > > > > > > > > > > > > > -----Original Message----- > > > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > > > Sent: Tuesday, October 21, 2014 6:33 PM > > > > > To: Liang, Cunming > > > > > Cc: dev@dpdk.org > > > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > > > > cycles/packet > > > > > > > > > > On Sun, Oct 12, 2014 at 11:10:39AM +0000, Liang, Cunming wrote: > > > > > > Hi Neil, > > > > > > > > > > > > Very appreciate your comments. > > > > > > I add inline reply, will send v3 asap when we get alignment. > > > > > > > > > > > > BRs, > > > > > > Liang Cunming > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > > > > > Sent: Saturday, October 11, 2014 1:52 AM > > > > > > > To: Liang, Cunming > > > > > > > Cc: dev@dpdk.org > > > > > > > Subject: Re: [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx > > > > > cycles/packet > > > > > > > > <...snip...> > > > > > > > > > > > > > > > + printf("Force Stop!\n"); > > > > > > > > + stop = 1; > > > > > > > > + } > > > > > > > > + if (signum == SIGUSR2) > > > > > > > > + stats_display(0); > > > > > > > > +} > > > > > > > > +/* main processing loop */ > > > > > > > > +static int > > > > > > > > +main_loop(__rte_unused void *args) > > > > > > > > +{ > > > > > > > > +#define PACKET_SIZE 64 > > > > > > > > +#define FRAME_GAP 12 > > > > > > > > +#define MAC_PREAMBLE 8 > > > > > > > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > > > > > > > + unsigned lcore_id; > > > > > > > > + unsigned i, portid, nb_rx = 0, nb_tx = 0; > > > > > > > > + struct lcore_conf *conf; > > > > > > > > + uint64_t prev_tsc, cur_tsc; > > > > > > > > + int pkt_per_port; > > > > > > > > + uint64_t packets_per_second, total_packets; > > > > > > > > + > > > > > > > > + lcore_id = rte_lcore_id(); > > > > > > > > + conf = &lcore_conf[lcore_id]; > > > > > > > > + if (conf->status != LCORE_USED) > > > > > > > > + return 0; > > > > > > > > + > > > > > > > > + pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; > > > > > > > > + > > > > > > > > + int idx = 0; > > > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > > > + int num = pkt_per_port; > > > > > > > > + portid = conf->portlist[i]; > > > > > > > > + printf("inject %d packet to port %d\n", num, portid); > > > > > > > > + while (num) { > > > > > > > > + nb_tx = RTE_MIN(MAX_PKT_BURST, num); > > > > > > > > + nb_tx = rte_eth_tx_burst(portid, 0, > > > > > > > > + &tx_burst[idx], nb_tx); > > > > > > > > + num -= nb_tx; > > > > > > > > + idx += nb_tx; > > > > > > > > + } > > > > > > > > + } > > > > > > > > + printf("Total packets inject to prime ports = %u\n", idx); > > > > > > > > + > > > > > > > > + packets_per_second = (link_mbps * 1000 * 1000) / > > > > > > > > + +((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * > > CHAR_BIT); > > > > > > > > + printf("Each port will do %"PRIu64" packets per second\n", > > > > > > > > + +packets_per_second); > > > > > > > > + > > > > > > > > + total_packets = RTE_TEST_DURATION * conf->nb_ports * > > > > > > > packets_per_second; > > > > > > > > + printf("Test will stop after at least %"PRIu64" packets > > received\n", > > > > > > > > + + total_packets); > > > > > > > > + > > > > > > > > + prev_tsc = rte_rdtsc(); > > > > > > > > + > > > > > > > > + while (likely(!stop)) { > > > > > > > > + for (i = 0; i < conf->nb_ports; i++) { > > > > > > > > + portid = conf->portlist[i]; > > > > > > > > + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, > > > > > > > > + pkts_burst, > > MAX_PKT_BURST); > > > > > > > > + if (unlikely(nb_rx == 0)) { > > > > > > > > + idle++; > > > > > > > > + continue; > > > > > > > > + } > > > > > > > > + > > > > > > > > + count += nb_rx; > > > > > > > Doesn't take into consideration error conditions. rte_eth_rx_burst > can > > > > > return > > > > > > > -ENOTSUP > > > > > > [Liang, Cunming] It returns -ENOTSUP when turning on ETHDEV_DEBUG > > > > > CONFIG. > > > > > > The error is used to avoid no function call register. > > > > > > When ETHDEV_DEBUG turn off, the NULL function call cause segfault > > directly. > > > > > > So I think it's a library internal error. > > > > > No, look at FUNC_PTR_OR_ERR_RET. If the passed in function pointer is > > null, > > > > > -ENOTSUPP will be returned to the application, you need to handle the > > error > > > > > condition. > > > > [Liang, Cunming] The runtime rte_eth_rx_burst is a inline function in > > rte_ethdev.h. > > > > The one you're talking about is the one defined in rte_ethdev.c which is a > > extern non-inline function. > > > > It works only when RTE_LIBRTE_ETHDEV_DEBUG turns on. > > > > If we always turns such library internal checking on, it lose performance. > > > > So I insist it's a library internal error checking, doesn't need to take care in > > application level. > > > I'm really flored that you would respond this way. > > > > > > What you have is two versions of a function, one returns errors and one > > doesn't, > > > and the version used is based on compile time configuration. You've written > > > your application such that it can only gracefully handle one of the two > > > versions, and you're happy to allow the other version, the one thats > supposed > > to > > > be in use when you are debugging applications, to create silent accounting > > > failures in your application. And it completely ignores the fact that the > > > non-debug version might one day return errors as well (even if it doesn't > > > currently). Your assertion above that we can ignore it because its debug code > > > is the most short sighted thing I've read in a long time. > > > > Actually looking at rte_eth_rx_burst(): it returns uint16_t. > > Which is sort of a strange if it expects to return a negative value as error code. > > Also reading though 'formal' comments of rte_eth_rx_burst() - it seems that it > > not supposed to return negative values: > > "... > > The rte_eth_rx_burst() function does not provide any error > > notification to avoid the corresponding overhead. As a hint, the > > upper-level application might check the status of the device link once > > being systematically returned a 0 value for a given number of tries. > > ... > > @return > > * The number of packets actually retrieved, which is the number > > * of pointers to *rte_mbuf* structures effectively supplied to the > > * *rx_pkts* array." > > > > Again, if you looks at rte_eth_rx_burst() implementation, when > > RTE_LIBRTE_ETHDEV_DEBUG is on - > > For some error cases it returns '0', foor other's: -ENOTSUP: > > > > FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); > > if (queue_id >= dev->data->nb_rx_queues) { > > PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); > > return 0; > > } > > > > Which makes me thing that we just have errnoneous implementation of > > rte_eth_rx_burst() > > when RTE_LIBRTE_ETHDEV_DEBUG is on. > > Probably just a mechanical mistake, when FUNC_PTR_OR_ERR_RET() was > > added. > > I'd say we change rte_eth_rx_burst() to always return 0 (as was initially > > planned). > > > > Konstantin > > Agreed. For any errors other than no-packets available yet, we can also consider > setting rte_errno when returning 0. > > /Bruce ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v2 2/4] app/test: measure standalone rx or tx cycles/packet 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 0/4] " Cunming Liang 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet Cunming Liang @ 2014-10-10 12:29 ` Cunming Liang 2014-10-10 12:30 ` [dpdk-dev] [PATCH v2 3/4] app/test: add unit test to measure RX burst cycles Cunming Liang ` (3 subsequent siblings) 5 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-10 12:29 UTC (permalink / raw) To: dev The unit test to support measure rx or tx only. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Choose rx or tx standalone, default is both set_rxtx_anchor [rxtx|rxonly|txonly] 3. Run and wait for the result pmd_perf_autotest Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/commands.c | 38 ++++++++++ app/test/test.h | 1 + app/test/test_pmd_perf.c | 175 ++++++++++++++++++++++++++++++++++++++-------- 3 files changed, 185 insertions(+), 29 deletions(-) diff --git a/app/test/commands.c b/app/test/commands.c index f1e746e..7c5d812 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -346,6 +346,43 @@ cmdline_parse_inst_t cmd_set_rxtx = { /****************/ +/****************/ + +struct cmd_set_rxtx_anchor { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_anchor_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_anchor *res = parsed_result; + if (test_set_rxtx_anchor(res->type) < 0) + cmdline_printf(cl, "Cannot find such anchor\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, set, + "set_rxtx_anchor"); + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_anchor = { + .f = cmd_set_rxtx_anchor_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx anchor: " + "set_rxtx_anchor <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_anchor_set, + (void *)&cmd_set_rxtx_anchor_type, + NULL, + }, +}; + +/****************/ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, @@ -354,6 +391,7 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_ring, (cmdline_parse_inst_t *)&cmd_quit, (cmdline_parse_inst_t *)&cmd_set_rxtx, + (cmdline_parse_inst_t *)&cmd_set_rxtx_anchor, NULL, }; diff --git a/app/test/test.h b/app/test/test.h index 76033a8..68ae4bf 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -141,6 +141,7 @@ int test_mp_secondary(void); int test_ivshmem(void); int test_set_rxtx_conf(cmdline_fixed_string_t mode); +int test_set_rxtx_anchor(cmdline_fixed_string_t type); typedef int (test_callback)(void); TAILQ_HEAD(test_commands_list, test_command); diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c index bca4a9a..a6d6ea7 100644 --- a/app/test/test_pmd_perf.c +++ b/app/test/test_pmd_perf.c @@ -364,6 +364,125 @@ signal_handler(int signum) #define MAX_TRAFIC_BURST (4096) struct rte_mbuf *tx_burst[MAX_TRAFIC_BURST]; +uint64_t (*do_measure)(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts); + +static uint64_t +measure_rxtx(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t prev_tsc, cur_tsc; + + prev_tsc = rte_rdtsc(); + + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + cur_tsc = rte_rdtsc(); + + return cur_tsc - prev_tsc; +} + +static uint64_t +measure_rxonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + + cur_tsc = rte_rdtsc(); + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + diff_tsc += rte_rdtsc() - cur_tsc; + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +static uint64_t +measure_txonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + printf("do tx measure\n"); + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + + cur_tsc = rte_rdtsc(); + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + diff_tsc += rte_rdtsc() - cur_tsc; + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + /* main processing loop */ static int main_loop(__rte_unused void *args) @@ -375,8 +494,8 @@ main_loop(__rte_unused void *args) unsigned lcore_id; unsigned i, portid, nb_rx = 0, nb_tx = 0; struct lcore_conf *conf; - uint64_t prev_tsc, cur_tsc; int pkt_per_port; + uint64_t diff_tsc; uint64_t packets_per_second, total_packets; lcore_id = rte_lcore_id(); @@ -410,32 +529,7 @@ main_loop(__rte_unused void *args) printf("Test will stop after at least %"PRIu64" packets received\n", + total_packets); - prev_tsc = rte_rdtsc(); - - while (likely(!stop)) { - for (i = 0; i < conf->nb_ports; i++) { - portid = conf->portlist[i]; - nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, - pkts_burst, MAX_PKT_BURST); - if (unlikely(nb_rx == 0)) { - idle++; - continue; - } - - count += nb_rx; - nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); - if (unlikely(nb_tx < nb_rx)) { - drop += (nb_rx - nb_tx); - do { - rte_pktmbuf_free(pkts_burst[nb_tx]); - } while (++nb_tx < nb_rx); - } - } - if (unlikely(count >= total_packets)) - break; - } - - cur_tsc = rte_rdtsc(); + diff_tsc = do_measure(conf, pkts_burst, total_packets); for (i = 0; i < conf->nb_ports; i++) { portid = conf->portlist[i]; @@ -455,7 +549,7 @@ main_loop(__rte_unused void *args) return -1; printf("%lu packet, %lu drop, %lu idle\n", count, drop, idle); - printf("Result: %ld cycles per packet\n", (cur_tsc - prev_tsc) / count); + printf("Result: %ld cycles per packet\n", diff_tsc / count); return 0; } @@ -559,6 +653,10 @@ test_pmd_perf(void) init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); + /* do both rxtx by default */ + if (NULL == do_measure) + do_measure = measure_rxtx; + rte_eal_remote_launch(main_loop, NULL, slave_id); if (rte_eal_wait_lcore(slave_id) < 0) return -1; @@ -577,7 +675,7 @@ test_pmd_perf(void) int test_set_rxtx_conf(cmdline_fixed_string_t mode) { - printf("mode is %s\n", mode); + printf("mode switch to %s\n", mode); if (!strcmp(mode, "vector")) { /* vector rx, tx */ @@ -619,6 +717,25 @@ test_set_rxtx_conf(cmdline_fixed_string_t mode) return -1; } +int +test_set_rxtx_anchor(cmdline_fixed_string_t type) +{ + printf("type switch to %s\n", type); + + if (!strcmp(type, "rxtx")) { + do_measure = measure_rxtx; + return 0; + } else if (!strcmp(type, "rxonly")) { + do_measure = measure_rxonly; + return 0; + } else if (!strcmp(type, "txonly")) { + do_measure = measure_txonly; + return 0; + } + + return -1; +} + static struct test_command pmd_perf_cmd = { .command = "pmd_perf_autotest", .callback = test_pmd_perf, -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v2 3/4] app/test: add unit test to measure RX burst cycles 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 0/4] " Cunming Liang 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet Cunming Liang 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 2/4] app/test: measure standalone rx or " Cunming Liang @ 2014-10-10 12:30 ` Cunming Liang 2014-10-10 12:30 ` [dpdk-dev] [PATCH v2 4/4] app/test: allow to create packets in different sizes Cunming Liang ` (2 subsequent siblings) 5 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-10 12:30 UTC (permalink / raw) To: dev This patch support measure RX burst cycles. It stops only when all the injected packets are received. It won't repeat re-send recevied packets. Now it measures two situations, poll before/after xmit(w or w/o desc. cache conflict) Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] 3. Run and wait for the result pmd_perf_autotest Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/commands.c | 43 +++++++++- app/test/test.h | 1 + app/test/test_pmd_perf.c | 212 ++++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 235 insertions(+), 21 deletions(-) diff --git a/app/test/commands.c b/app/test/commands.c index 7c5d812..92a17ed 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -310,8 +310,6 @@ cmdline_parse_inst_t cmd_quit = { /****************/ -/****************/ - struct cmd_set_rxtx_result { cmdline_fixed_string_t set; cmdline_fixed_string_t mode; @@ -346,8 +344,6 @@ cmdline_parse_inst_t cmd_set_rxtx = { /****************/ -/****************/ - struct cmd_set_rxtx_anchor { cmdline_fixed_string_t set; cmdline_fixed_string_t type; @@ -384,6 +380,44 @@ cmdline_parse_inst_t cmd_set_rxtx_anchor = { /****************/ +/* for stream control */ +struct cmd_set_rxtx_sc { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_sc_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_sc *res = parsed_result; + if (test_set_rxtx_sc(res->type) < 0) + cmdline_printf(cl, "Cannot find such stream control\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_sc_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, set, + "set_rxtx_sc"); + +cmdline_parse_token_string_t cmd_set_rxtx_sc_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_sc = { + .f = cmd_set_rxtx_sc_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx stream control: " + "set_rxtx_sc <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_sc_set, + (void *)&cmd_set_rxtx_sc_type, + NULL, + }, +}; + +/****************/ + + cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, (cmdline_parse_inst_t *)&cmd_dump, @@ -392,6 +426,7 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_quit, (cmdline_parse_inst_t *)&cmd_set_rxtx, (cmdline_parse_inst_t *)&cmd_set_rxtx_anchor, + (cmdline_parse_inst_t *)&cmd_set_rxtx_sc, NULL, }; diff --git a/app/test/test.h b/app/test/test.h index 68ae4bf..bd44a7f 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -142,6 +142,7 @@ int test_mp_secondary(void); int test_ivshmem(void); int test_set_rxtx_conf(cmdline_fixed_string_t mode); int test_set_rxtx_anchor(cmdline_fixed_string_t type); +int test_set_rxtx_sc(cmdline_fixed_string_t type); typedef int (test_callback)(void); TAILQ_HEAD(test_commands_list, test_command); diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c index a6d6ea7..f3da599 100644 --- a/app/test/test_pmd_perf.c +++ b/app/test/test_pmd_perf.c @@ -39,6 +39,8 @@ #include <rte_cycles.h> #include <rte_ethdev.h> #include <rte_byteorder.h> +#include <rte_atomic.h> +#include <rte_malloc.h> #include "packet_burst_generator.h" #include "test.h" @@ -73,13 +75,15 @@ #define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */ #define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */ +#define MAX_TRAFFIC_BURST 2048 #define NB_MBUF RTE_MAX( \ - (unsigned)(nb_ports*nb_rx_queue*RTE_TEST_RX_DESC_DEFAULT + \ - nb_ports*nb_lcores*MAX_PKT_BURST + \ - nb_ports*nb_tx_queue*RTE_TEST_TX_DESC_DEFAULT + \ - nb_lcores*MEMPOOL_CACHE_SIZE), \ - (unsigned)8192) + (unsigned)(nb_ports*nb_rx_queue*nb_rxd + \ + nb_ports*nb_lcores*MAX_PKT_BURST + \ + nb_ports*nb_tx_queue*nb_txd + \ + nb_lcores*MEMPOOL_CACHE_SIZE + \ + nb_ports*MAX_TRAFFIC_BURST), \ + (unsigned)8192) static struct rte_mempool *mbufpool[NB_SOCKETS]; @@ -147,6 +151,14 @@ struct lcore_conf lcore_conf[RTE_MAX_LCORE]; static uint64_t link_mbps; +enum { + SC_CONTINUOUS = 0, + SC_BURST_POLL_FIRST, + SC_BURST_XMIT_FIRST, +}; + +static uint32_t sc_flag; + /* Check the link status of all ports in up to 3s, and print them finally */ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) @@ -361,8 +373,7 @@ signal_handler(int signum) stats_display(0); } -#define MAX_TRAFIC_BURST (4096) -struct rte_mbuf *tx_burst[MAX_TRAFIC_BURST]; +struct rte_mbuf **tx_burst; uint64_t (*do_measure)(struct lcore_conf *conf, struct rte_mbuf *pkts_burst[], @@ -503,7 +514,7 @@ main_loop(__rte_unused void *args) if (conf->status != LCORE_USED) return 0; - pkt_per_port = MAX_TRAFIC_BURST / conf->nb_ports; + pkt_per_port = MAX_TRAFFIC_BURST; int idx = 0; for (i = 0; i < conf->nb_ports; i++) { @@ -554,12 +565,135 @@ main_loop(__rte_unused void *args) return 0; } +rte_atomic64_t start; + +static inline int +poll_burst(void *args) +{ +#define MAX_IDLE (1000) + unsigned lcore_id; + struct rte_mbuf **pkts_burst; + uint64_t diff_tsc, cur_tsc; + uint16_t next[RTE_MAX_ETHPORTS]; + struct lcore_conf *conf; + uint32_t pkt_per_port = *((uint32_t *)args); + unsigned i, portid, nb_rx = 0; + uint64_t total; + uint64_t timeout = MAX_IDLE; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + total = pkt_per_port * conf->nb_ports; + printf("start to receive total expect %ld\n", total); + + pkts_burst = (struct rte_mbuf **) + rte_calloc_socket("poll_burst", + total, sizeof(void *), + CACHE_LINE_SIZE, conf->socketid); + if (!pkts_burst) + return -1; + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + next[portid] = i * pkt_per_port; + } + + while (!rte_atomic64_read(&start)) + ; + + cur_tsc = rte_rdtsc(); + while (total) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + &pkts_burst[next[portid]], + MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + timeout--; + if (unlikely(timeout == 0)) + goto timeout; + continue; + } + next[portid] += nb_rx; + total -= nb_rx; + } + } +timeout: + diff_tsc = rte_rdtsc() - cur_tsc; + + printf("%ld packets lost, IDLE %ld times\n", total, MAX_IDLE - timeout); + + /* clean up */ + total = pkt_per_port * conf->nb_ports - total; + for (i = 0; i < total; i++) + rte_pktmbuf_free(pkts_burst[i]); + + rte_free(pkts_burst); + + return diff_tsc / total; +} + +static int +exec_burst(uint32_t flags, int lcore) +{ + unsigned i, portid, nb_tx = 0; + struct lcore_conf *conf; + uint32_t pkt_per_port; + int num, idx = 0; + int diff_tsc; + + conf = &lcore_conf[lcore]; + + pkt_per_port = MAX_TRAFFIC_BURST; + num = pkt_per_port; + + rte_atomic64_init(&start); + + /* start polling thread, but not actually poll yet */ + rte_eal_remote_launch(poll_burst, + (void *)&pkt_per_port, lcore); + + /* Only when polling first */ + if (flags == SC_BURST_POLL_FIRST) + rte_atomic64_set(&start, 1); + + /* start xmit */ + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + idx += nb_tx; + } + num -= nb_tx; + } + + sleep(5); + + /* only when polling second */ + if (flags == SC_BURST_XMIT_FIRST) + rte_atomic64_set(&start, 1); + + /* wait for polling finished */ + diff_tsc = rte_eal_wait_lcore(lcore); + if (diff_tsc < 0) + return -1; + + printf("Result: %d cycles per packet\n", diff_tsc); + + return 0; +} + static int test_pmd_perf(void) { uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; - uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; - uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; + uint16_t nb_rxd = MAX_TRAFFIC_BURST; + uint16_t nb_txd = MAX_TRAFFIC_BURST; uint16_t portid; uint16_t nb_rx_queue = 1, nb_tx_queue = 1; int socketid = -1; @@ -587,6 +721,12 @@ test_pmd_perf(void) init_mbufpool(NB_MBUF); + if (sc_flag == SC_CONTINUOUS) { + nb_rxd = RTE_TEST_RX_DESC_DEFAULT; + nb_txd = RTE_TEST_TX_DESC_DEFAULT; + } + printf("CONFIG RXD=%d TXD=%d\n", nb_rxd, nb_txd); + reset_count(); num = 0; for (portid = 0; portid < nb_ports; portid++) { @@ -651,15 +791,34 @@ test_pmd_perf(void) } check_all_ports_link_status(nb_ports, RTE_PORT_ALL); - init_traffic(mbufpool[socketid], tx_burst, MAX_TRAFIC_BURST); + if (tx_burst == NULL) { + tx_burst = (struct rte_mbuf **) + rte_calloc_socket("tx_buff", + MAX_TRAFFIC_BURST * nb_ports, + sizeof(void *), + CACHE_LINE_SIZE, socketid); + if (!tx_burst) + return -1; + } - /* do both rxtx by default */ - if (NULL == do_measure) - do_measure = measure_rxtx; + init_traffic(mbufpool[socketid], + tx_burst, MAX_TRAFFIC_BURST * nb_ports); - rte_eal_remote_launch(main_loop, NULL, slave_id); - if (rte_eal_wait_lcore(slave_id) < 0) - return -1; + printf("Generate %d packets @socket %d\n", + MAX_TRAFFIC_BURST * nb_ports, socketid); + + if (sc_flag == SC_CONTINUOUS) { + /* do both rxtx by default */ + if (NULL == do_measure) + do_measure = measure_rxtx; + + rte_eal_remote_launch(main_loop, NULL, slave_id); + + if (rte_eal_wait_lcore(slave_id) < 0) + return -1; + } else if (sc_flag == SC_BURST_POLL_FIRST || + sc_flag == SC_BURST_XMIT_FIRST) + exec_burst(sc_flag, slave_id); /* port tear down */ for (portid = 0; portid < nb_ports; portid++) { @@ -736,6 +895,25 @@ test_set_rxtx_anchor(cmdline_fixed_string_t type) return -1; } +int +test_set_rxtx_sc(cmdline_fixed_string_t type) +{ + printf("stream control switch to %s\n", type); + + if (!strcmp(type, "continuous")) { + sc_flag = SC_CONTINUOUS; + return 0; + } else if (!strcmp(type, "poll_before_xmit")) { + sc_flag = SC_BURST_POLL_FIRST; + return 0; + } else if (!strcmp(type, "poll_after_xmit")) { + sc_flag = SC_BURST_XMIT_FIRST; + return 0; + } + + return -1; +} + static struct test_command pmd_perf_cmd = { .command = "pmd_perf_autotest", .callback = test_pmd_perf, -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v2 4/4] app/test: allow to create packets in different sizes 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 0/4] " Cunming Liang ` (2 preceding siblings ...) 2014-10-10 12:30 ` [dpdk-dev] [PATCH v2 3/4] app/test: add unit test to measure RX burst cycles Cunming Liang @ 2014-10-10 12:30 ` Cunming Liang 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Cunming Liang 2014-10-28 12:21 ` [dpdk-dev] [PATCH v2 0/4] app/test: unit test to measure cycles per packet Neil Horman 5 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-10 12:30 UTC (permalink / raw) To: dev adding support to allow packet burst generator to create packets in differenct sizes Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- app/test/packet_burst_generator.c | 26 ++++++++---------------- app/test/packet_burst_generator.h | 11 +++++++-- app/test/test_link_bonding.c | 39 ++++++++++++++++++++++++------------ app/test/test_pmd_perf.c | 3 +- 4 files changed, 45 insertions(+), 34 deletions(-) diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c index 9100b1d..017139b 100644 --- a/app/test/packet_burst_generator.c +++ b/app/test/packet_burst_generator.c @@ -191,20 +191,12 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, */ #define RTE_MAX_SEGS_PER_PKT 255 /**< pkt.nb_segs is a 8-bit unsigned char. */ -#define TXONLY_DEF_PACKET_LEN 60 -#define TXONLY_DEF_PACKET_LEN_128 128 - -uint16_t tx_pkt_length = TXONLY_DEF_PACKET_LEN; -uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT] = { - TXONLY_DEF_PACKET_LEN, -}; - -uint8_t tx_pkt_nb_segs = 1; - int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst) + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs) { int i, nb_pkt = 0; size_t eth_hdr_size; @@ -221,9 +213,9 @@ nomore_mbuf: break; } - pkt->data_len = tx_pkt_seg_lengths[0]; + pkt->data_len = pkt_len; pkt_seg = pkt; - for (i = 1; i < tx_pkt_nb_segs; i++) { + for (i = 1; i < nb_pkt_segs; i++) { pkt_seg->next = rte_pktmbuf_alloc(mp); if (pkt_seg->next == NULL) { pkt->nb_segs = i; @@ -231,7 +223,7 @@ nomore_mbuf: goto nomore_mbuf; } pkt_seg = pkt_seg->next; - pkt_seg->data_len = tx_pkt_seg_lengths[i]; + pkt_seg->data_len = pkt_len; } pkt_seg->next = NULL; /* Last segment of packet. */ @@ -259,8 +251,8 @@ nomore_mbuf: * Complete first mbuf of packet and append it to the * burst of packets to be transmitted. */ - pkt->nb_segs = tx_pkt_nb_segs; - pkt->pkt_len = tx_pkt_length; + pkt->nb_segs = nb_pkt_segs; + pkt->pkt_len = pkt_len; pkt->l2_len = eth_hdr_size; if (ipv4) { diff --git a/app/test/packet_burst_generator.h b/app/test/packet_burst_generator.h index 5b3cd6c..fe992ac 100644 --- a/app/test/packet_burst_generator.h +++ b/app/test/packet_burst_generator.h @@ -47,10 +47,13 @@ extern "C" { #define IPV4_ADDR(a, b, c, d)(((a & 0xff) << 24) | ((b & 0xff) << 16) | \ ((c & 0xff) << 8) | (d & 0xff)) +#define PACKET_BURST_GEN_PKT_LEN 60 +#define PACKET_BURST_GEN_PKT_LEN_128 128 void initialize_eth_header(struct ether_hdr *eth_hdr, struct ether_addr *src_mac, - struct ether_addr *dst_mac, uint8_t vlan_enabled, uint16_t van_id); + struct ether_addr *dst_mac, uint8_t vlan_enabled, + uint16_t van_id); uint16_t initialize_udp_header(struct udp_hdr *udp_hdr, uint16_t src_port, @@ -67,8 +70,10 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst); + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs); #ifdef __cplusplus } diff --git a/app/test/test_link_bonding.c b/app/test/test_link_bonding.c index db5b180..0d50203 100644 --- a/app/test/test_link_bonding.c +++ b/app/test/test_link_bonding.c @@ -1189,9 +1189,12 @@ generate_test_burst(struct rte_mbuf **pkts_burst, uint16_t burst_size, } /* Generate burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, vlan, ip_hdr, ipv4, - test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, + vlan, ip_hdr, ipv4, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN_128, 1); if (generated_burst_size != burst_size) { printf("Failed to generate packet burst"); return -1; @@ -1778,9 +1781,12 @@ test_activebackup_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; @@ -2432,8 +2438,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 1 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[0][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[0]) != burst_size[0]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[0], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[0]) return -1; initialize_eth_header(test_params->pkt_eth_hdr, @@ -2441,8 +2449,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 2 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[1][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[1]) != burst_size[1]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[1], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[1]) return -1; /* Send burst 1 on bonded port */ @@ -3259,9 +3269,12 @@ test_broadcast_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c index f3da599..b62acf1 100644 --- a/app/test/test_pmd_perf.c +++ b/app/test/test_pmd_perf.c @@ -253,7 +253,8 @@ init_traffic(struct rte_mempool *mp, return generate_packet_burst(mp, pkts_burst, &pkt_eth_hdr, 0, &pkt_ipv4_hdr, 1, - &pkt_udp_hdr, burst_size); + &pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); } static int -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 0/4] " Cunming Liang ` (3 preceding siblings ...) 2014-10-10 12:30 ` [dpdk-dev] [PATCH v2 4/4] app/test: allow to create packets in different sizes Cunming Liang @ 2014-10-20 8:13 ` Cunming Liang 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 1/2] app/test: allow to create packets in different sizes Cunming Liang ` (3 more replies) 2014-10-28 12:21 ` [dpdk-dev] [PATCH v2 0/4] app/test: unit test to measure cycles per packet Neil Horman 5 siblings, 4 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-20 8:13 UTC (permalink / raw) To: dev v3 update: # Codes refine according to the feedback. 1. add ether_format_addr to rte_ether.h 2. fix typo in code comments. 3. %lu to %PRIu64, fixing 32-bit targets compilation err # merge 2 small incremental patches to the first one. The whole unit test as a single patch in [PATCH v3 2/2] # rebase code to the latest master v2 update: Rebase code to the latest master branch. Tested-by: Zhaochen Zhan <zhaochen.zhan@intel.com> This patch has been verified on ixgbe and it is ready to be integrated to dpdk.org. For e1000 and i40e, lacking loopback supporting, this patch can't support them for now. -------------- It provides unit test to measure cycles/packet in NIC loopback mode. It simply gives the average cycles of IO used per packet without test equipment. When doing the test, make sure the link is UP. There's two stream control mode support, one is continues, another is burst. The former continues to forward the injected packets until reaching a certain amount of number. The latter one stop when all the injected packets are received. In burst stream, now measure two situations, with or without desc. cache conflict. By default, it runs in continues stream mode to measure the whole rxtx. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] 3. If choose continuous stream, there are another two options can configure 3.1 choose rx/tx pair, default is vector set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without INC_VEC=y in config 3.2 choose the area of masurement, default is rxtx set_rxtx_anchor [rxtx|rxonly|txonly] 4. Run and wait for the result pmd_perf_autotest For who simply just want to see how much cycles cost per packet. Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. Nothing else needs to configure. Using other options when you understand and what to measures more. *** BLURB HERE *** Cunming Liang (2): app/test: allow to create packets in different sizes app/test: measure the cost of rx/tx routines by cycle number app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/packet_burst_generator.c | 26 +- app/test/packet_burst_generator.h | 11 +- app/test/test.h | 6 + app/test/test_link_bonding.c | 39 +- app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ether.h | 25 + lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 9 files changed, 1114 insertions(+), 33 deletions(-) create mode 100644 app/test/test_pmd_perf.c -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v3 1/2] app/test: allow to create packets in different sizes 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Cunming Liang @ 2014-10-20 8:13 ` Cunming Liang 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 2/2] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang ` (2 subsequent siblings) 3 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-20 8:13 UTC (permalink / raw) To: dev adding support to allow packet burst generator to create packets in differenct sizes Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- app/test/packet_burst_generator.c | 26 ++++++++---------------- app/test/packet_burst_generator.h | 11 +++++++-- app/test/test_link_bonding.c | 39 ++++++++++++++++++++++++------------ 3 files changed, 43 insertions(+), 33 deletions(-) diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c index 9e747a4..017139b 100644 --- a/app/test/packet_burst_generator.c +++ b/app/test/packet_burst_generator.c @@ -191,20 +191,12 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, */ #define RTE_MAX_SEGS_PER_PKT 255 /**< pkt.nb_segs is a 8-bit unsigned char. */ -#define TXONLY_DEF_PACKET_LEN 64 -#define TXONLY_DEF_PACKET_LEN_128 128 - -uint16_t tx_pkt_length = TXONLY_DEF_PACKET_LEN; -uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT] = { - TXONLY_DEF_PACKET_LEN_128, -}; - -uint8_t tx_pkt_nb_segs = 1; - int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst) + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs) { int i, nb_pkt = 0; size_t eth_hdr_size; @@ -221,9 +213,9 @@ nomore_mbuf: break; } - pkt->data_len = tx_pkt_seg_lengths[0]; + pkt->data_len = pkt_len; pkt_seg = pkt; - for (i = 1; i < tx_pkt_nb_segs; i++) { + for (i = 1; i < nb_pkt_segs; i++) { pkt_seg->next = rte_pktmbuf_alloc(mp); if (pkt_seg->next == NULL) { pkt->nb_segs = i; @@ -231,7 +223,7 @@ nomore_mbuf: goto nomore_mbuf; } pkt_seg = pkt_seg->next; - pkt_seg->data_len = tx_pkt_seg_lengths[i]; + pkt_seg->data_len = pkt_len; } pkt_seg->next = NULL; /* Last segment of packet. */ @@ -259,8 +251,8 @@ nomore_mbuf: * Complete first mbuf of packet and append it to the * burst of packets to be transmitted. */ - pkt->nb_segs = tx_pkt_nb_segs; - pkt->pkt_len = tx_pkt_length; + pkt->nb_segs = nb_pkt_segs; + pkt->pkt_len = pkt_len; pkt->l2_len = eth_hdr_size; if (ipv4) { diff --git a/app/test/packet_burst_generator.h b/app/test/packet_burst_generator.h index 5b3cd6c..fe992ac 100644 --- a/app/test/packet_burst_generator.h +++ b/app/test/packet_burst_generator.h @@ -47,10 +47,13 @@ extern "C" { #define IPV4_ADDR(a, b, c, d)(((a & 0xff) << 24) | ((b & 0xff) << 16) | \ ((c & 0xff) << 8) | (d & 0xff)) +#define PACKET_BURST_GEN_PKT_LEN 60 +#define PACKET_BURST_GEN_PKT_LEN_128 128 void initialize_eth_header(struct ether_hdr *eth_hdr, struct ether_addr *src_mac, - struct ether_addr *dst_mac, uint8_t vlan_enabled, uint16_t van_id); + struct ether_addr *dst_mac, uint8_t vlan_enabled, + uint16_t van_id); uint16_t initialize_udp_header(struct udp_hdr *udp_hdr, uint16_t src_port, @@ -67,8 +70,10 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst); + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs); #ifdef __cplusplus } diff --git a/app/test/test_link_bonding.c b/app/test/test_link_bonding.c index db5b180..0d50203 100644 --- a/app/test/test_link_bonding.c +++ b/app/test/test_link_bonding.c @@ -1189,9 +1189,12 @@ generate_test_burst(struct rte_mbuf **pkts_burst, uint16_t burst_size, } /* Generate burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, vlan, ip_hdr, ipv4, - test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, + vlan, ip_hdr, ipv4, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN_128, 1); if (generated_burst_size != burst_size) { printf("Failed to generate packet burst"); return -1; @@ -1778,9 +1781,12 @@ test_activebackup_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; @@ -2432,8 +2438,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 1 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[0][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[0]) != burst_size[0]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[0], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[0]) return -1; initialize_eth_header(test_params->pkt_eth_hdr, @@ -2441,8 +2449,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 2 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[1][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[1]) != burst_size[1]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[1], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[1]) return -1; /* Send burst 1 on bonded port */ @@ -3259,9 +3269,12 @@ test_broadcast_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v3 2/2] app/test: measure the cost of rx/tx routines by cycle number 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Cunming Liang 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 1/2] app/test: allow to create packets in different sizes Cunming Liang @ 2014-10-20 8:13 ` Cunming Liang 2014-10-21 2:40 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Liu, Yong 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 0/3] " Cunming Liang 3 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-20 8:13 UTC (permalink / raw) To: dev The unit test can be used to measure cycles per packet in different rx/tx rouines. The NIC works in loopback mode. So it doesn't require test equipment to measure throughput. As result, the unit test shows the average cycles per packet consuming. When doing the test, make sure the link is UP. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Run and wait for the result pmd_perf_autotest There's option to choose rx/tx pair, default is vector. set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without INC_VEC=y in config It supports to measure standalone rx or tx. Usage Example: Choose rx or tx standalone, default is both set_rxtx_anchor [rxtx|rxonly|txonly] It also supports to measure standalone RX burst cycles. In this way, it won't repeat re-send recevied packets. Now it measures two situations, poll before/after xmit(w or w/o desc. cache conflict) Usage Example: Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/test.h | 6 + app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ether.h | 25 + lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 6 files changed, 1071 insertions(+), 0 deletions(-) create mode 100644 app/test/test_pmd_perf.c diff --git a/app/test/Makefile b/app/test/Makefile index 6af6d76..ebfa0ba 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c SRCS-y += test_ring.c SRCS-y += test_ring_perf.c +SRCS-y += test_pmd_perf.c ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) SRCS-y += test_table.c diff --git a/app/test/commands.c b/app/test/commands.c index a9e36b1..92a17ed 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -310,12 +310,123 @@ cmdline_parse_inst_t cmd_quit = { /****************/ +struct cmd_set_rxtx_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t mode; +}; + +static void cmd_set_rxtx_parsed(void *parsed_result, struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_result *res = parsed_result; + if (test_set_rxtx_conf(res->mode) < 0) + cmdline_printf(cl, "Cannot find such mode\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, set, + "set_rxtx_mode"); + +cmdline_parse_token_string_t cmd_set_rxtx_mode = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, mode, NULL); + +cmdline_parse_inst_t cmd_set_rxtx = { + .f = cmd_set_rxtx_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx routine: " + "set_rxtx <mode>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_set, + (void *)&cmd_set_rxtx_mode, + NULL, + }, +}; + +/****************/ + +struct cmd_set_rxtx_anchor { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_anchor_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_anchor *res = parsed_result; + if (test_set_rxtx_anchor(res->type) < 0) + cmdline_printf(cl, "Cannot find such anchor\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, set, + "set_rxtx_anchor"); + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_anchor = { + .f = cmd_set_rxtx_anchor_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx anchor: " + "set_rxtx_anchor <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_anchor_set, + (void *)&cmd_set_rxtx_anchor_type, + NULL, + }, +}; + +/****************/ + +/* for stream control */ +struct cmd_set_rxtx_sc { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_sc_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_sc *res = parsed_result; + if (test_set_rxtx_sc(res->type) < 0) + cmdline_printf(cl, "Cannot find such stream control\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_sc_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, set, + "set_rxtx_sc"); + +cmdline_parse_token_string_t cmd_set_rxtx_sc_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_sc = { + .f = cmd_set_rxtx_sc_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx stream control: " + "set_rxtx_sc <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_sc_set, + (void *)&cmd_set_rxtx_sc_type, + NULL, + }, +}; + +/****************/ + + cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, (cmdline_parse_inst_t *)&cmd_dump, (cmdline_parse_inst_t *)&cmd_dump_one, (cmdline_parse_inst_t *)&cmd_set_ring, (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_set_rxtx, + (cmdline_parse_inst_t *)&cmd_set_rxtx_anchor, + (cmdline_parse_inst_t *)&cmd_set_rxtx_sc, NULL, }; diff --git a/app/test/test.h b/app/test/test.h index 98ab804..bd44a7f 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -125,6 +125,9 @@ int unit_test_suite_runner(struct unit_test_suite *suite); #define RECURSIVE_ENV_VAR "RTE_TEST_RECURSIVE" +#include <cmdline_parse.h> +#include <cmdline_parse_string.h> + extern const char *prgname; int commands_init(void); @@ -137,6 +140,9 @@ int test_pci_run; int test_mp_secondary(void); int test_ivshmem(void); +int test_set_rxtx_conf(cmdline_fixed_string_t mode); +int test_set_rxtx_anchor(cmdline_fixed_string_t type); +int test_set_rxtx_sc(cmdline_fixed_string_t type); typedef int (test_callback)(void); TAILQ_HEAD(test_commands_list, test_command); diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c new file mode 100644 index 0000000..1c1f236 --- /dev/null +++ b/app/test/test_pmd_perf.c @@ -0,0 +1,922 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + + +#include <stdio.h> +#include <inttypes.h> +#include <signal.h> +#include <unistd.h> +#include <rte_cycles.h> +#include <rte_ethdev.h> +#include <rte_byteorder.h> +#include <rte_atomic.h> +#include <rte_malloc.h> +#include "packet_burst_generator.h" +#include "test.h" + +#define NB_ETHPORTS_USED (1) +#define NB_SOCKETS (2) +#define MEMPOOL_CACHE_SIZE 250 +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MAX_PKT_BURST (32) +#define RTE_TEST_RX_DESC_DEFAULT (128) +#define RTE_TEST_TX_DESC_DEFAULT (512) +#define RTE_PORT_ALL (~(uint8_t)0x0) + +/* how long test would take at full line rate */ +#define RTE_TEST_DURATION (2) + +/* + * RX and TX Prefetch, Host, and Write-back threshold values should be + * carefully set for optimal performance. Consult the network + * controller's datasheet and supporting DPDK documentation for guidance + * on how these parameters should be set. + */ +#define RX_PTHRESH 8 /**< Default values of RX prefetch threshold reg. */ +#define RX_HTHRESH 8 /**< Default values of RX host threshold reg. */ +#define RX_WTHRESH 0 /**< Default values of RX write-back threshold reg. */ + +/* + * These default values are optimized for use with the Intel(R) 82599 10 GbE + * Controller and the DPDK ixgbe PMD. Consider using other values for other + * network controllers and/or network drivers. + */ +#define TX_PTHRESH 32 /**< Default values of TX prefetch threshold reg. */ +#define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */ +#define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */ + +#define MAX_TRAFFIC_BURST 2048 + +#define NB_MBUF RTE_MAX( \ + (unsigned)(nb_ports*nb_rx_queue*nb_rxd + \ + nb_ports*nb_lcores*MAX_PKT_BURST + \ + nb_ports*nb_tx_queue*nb_txd + \ + nb_lcores*MEMPOOL_CACHE_SIZE + \ + nb_ports*MAX_TRAFFIC_BURST), \ + (unsigned)8192) + + +static struct rte_mempool *mbufpool[NB_SOCKETS]; +/* ethernet addresses of ports */ +static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS]; + +static struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .hw_vlan_strip = 0, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 1, /* enable loopback */ +}; + +static struct rte_eth_rxconf rx_conf = { + .rx_thresh = { + .pthresh = RX_PTHRESH, + .hthresh = RX_HTHRESH, + .wthresh = RX_WTHRESH, + }, + .rx_free_thresh = 32, +}; + +static struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = TX_PTHRESH, + .hthresh = TX_HTHRESH, + .wthresh = TX_WTHRESH, + }, + .tx_free_thresh = 32, /* Use PMD default values */ + .tx_rs_thresh = 32, /* Use PMD default values */ + .txq_flags = (ETH_TXQ_FLAGS_NOMULTSEGS | + ETH_TXQ_FLAGS_NOVLANOFFL | + ETH_TXQ_FLAGS_NOXSUMSCTP | + ETH_TXQ_FLAGS_NOXSUMUDP | + ETH_TXQ_FLAGS_NOXSUMTCP) +}; + +enum { + LCORE_INVALID = 0, + LCORE_AVAIL, + LCORE_USED, +}; + +struct lcore_conf { + uint8_t status; + uint8_t socketid; + uint16_t nb_ports; + uint8_t portlist[RTE_MAX_ETHPORTS]; +} __rte_cache_aligned; + +struct lcore_conf lcore_conf[RTE_MAX_LCORE]; + +static uint64_t link_mbps; + +enum { + SC_CONTINUOUS = 0, + SC_BURST_POLL_FIRST, + SC_BURST_XMIT_FIRST, +}; + +static uint32_t sc_flag; + +/* Check the link status of all ports in up to 3s, and print them finally */ +static void +check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +{ +#define CHECK_INTERVAL 100 /* 100ms */ +#define MAX_CHECK_TIME 30 /* 3s (30 * 100ms) in total */ + uint8_t portid, count, all_ports_up, print_flag = 0; + struct rte_eth_link link; + + printf("Checking link statuses...\n"); + fflush(stdout); + for (count = 0; count <= MAX_CHECK_TIME; count++) { + all_ports_up = 1; + for (portid = 0; portid < port_num; portid++) { + if ((port_mask & (1 << portid)) == 0) + continue; + memset(&link, 0, sizeof(link)); + rte_eth_link_get_nowait(portid, &link); + /* print link status if flag set */ + if (print_flag == 1) { + if (link.link_status) { + printf("Port %d Link Up - speed %u " + "Mbps - %s\n", (uint8_t)portid, + (unsigned)link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + if (link_mbps == 0) + link_mbps = link.link_speed; + } else + printf("Port %d Link Down\n", + (uint8_t)portid); + continue; + } + /* clear all_ports_up flag if any link down */ + if (link.link_status == 0) { + all_ports_up = 0; + break; + } + } + /* after finally printing all link status, get out */ + if (print_flag == 1) + break; + + if (all_ports_up == 0) { + fflush(stdout); + rte_delay_ms(CHECK_INTERVAL); + } + + /* set the print_flag if all ports up or timeout */ + if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) + print_flag = 1; + } +} + +static void +print_ethaddr(const char *name, const struct ether_addr *eth_addr) +{ + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); +} + +static int +init_traffic(struct rte_mempool *mp, + struct rte_mbuf **pkts_burst, uint32_t burst_size) +{ + struct ether_hdr pkt_eth_hdr; + struct ipv4_hdr pkt_ipv4_hdr; + struct udp_hdr pkt_udp_hdr; + uint32_t pktlen; + static uint8_t src_mac[] = { 0x00, 0xFF, 0xAA, 0xFF, 0xAA, 0xFF }; + static uint8_t dst_mac[] = { 0x00, 0xAA, 0xFF, 0xAA, 0xFF, 0xAA }; + + + initialize_eth_header(&pkt_eth_hdr, + (struct ether_addr *)src_mac, + (struct ether_addr *)dst_mac, 0, 0); + pkt_eth_hdr.ether_type = rte_cpu_to_be_16(ETHER_TYPE_IPv4); + + pktlen = initialize_ipv4_header(&pkt_ipv4_hdr, + IPV4_ADDR(10, 0, 0, 1), + IPV4_ADDR(10, 0, 0, 2), 26); + printf("IPv4 pktlen %u\n", pktlen); + + pktlen = initialize_udp_header(&pkt_udp_hdr, 0, 0, 18); + + printf("UDP pktlen %u\n", pktlen); + + return generate_packet_burst(mp, pkts_burst, &pkt_eth_hdr, + 0, &pkt_ipv4_hdr, 1, + &pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); +} + +static int +init_lcores(void) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + lcore_conf[lcore_id].socketid = + rte_lcore_to_socket_id(lcore_id); + if (rte_lcore_is_enabled(lcore_id) == 0) { + lcore_conf[lcore_id].status = LCORE_INVALID; + continue; + } else + lcore_conf[lcore_id].status = LCORE_AVAIL; + } + return 0; +} + +static int +init_mbufpool(unsigned nb_mbuf) +{ + int socketid; + unsigned lcore_id; + char s[64]; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + socketid = rte_lcore_to_socket_id(lcore_id); + if (socketid >= NB_SOCKETS) { + rte_exit(EXIT_FAILURE, + "Socket %d of lcore %u is out of range %d\n", + socketid, lcore_id, NB_SOCKETS); + } + if (mbufpool[socketid] == NULL) { + snprintf(s, sizeof(s), "mbuf_pool_%d", socketid); + mbufpool[socketid] = + rte_mempool_create(s, nb_mbuf, MBUF_SIZE, + MEMPOOL_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + socketid, 0); + if (mbufpool[socketid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot init mbuf pool on socket %d\n", + socketid); + else + printf("Allocated mbuf pool on socket %d\n", + socketid); + } + } + return 0; +} + +static uint16_t +alloc_lcore(uint16_t socketid) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (LCORE_AVAIL != lcore_conf[lcore_id].status || + lcore_conf[lcore_id].socketid != socketid || + lcore_id == rte_get_master_lcore()) + continue; + lcore_conf[lcore_id].status = LCORE_USED; + lcore_conf[lcore_id].nb_ports = 0; + return lcore_id; + } + + return (uint16_t)-1; +} + +volatile uint64_t stop; +uint64_t count; +uint64_t drop; +uint64_t idle; + +static void +reset_count(void) +{ + count = 0; + drop = 0; + idle = 0; +} + +static void +stats_display(uint8_t port_id) +{ + struct rte_eth_stats stats; + rte_eth_stats_get(port_id, &stats); + + printf(" RX-packets: %-10"PRIu64" RX-missed: %-10"PRIu64" RX-bytes: " + "%-"PRIu64"\n", + stats.ipackets, stats.imissed, stats.ibytes); + printf(" RX-badcrc: %-10"PRIu64" RX-badlen: %-10"PRIu64" RX-errors: " + "%-"PRIu64"\n", + stats.ibadcrc, stats.ibadlen, stats.ierrors); + printf(" RX-nombuf: %-10"PRIu64"\n", + stats.rx_nombuf); + printf(" TX-packets: %-10"PRIu64" TX-errors: %-10"PRIu64" TX-bytes: " + "%-"PRIu64"\n", + stats.opackets, stats.oerrors, stats.obytes); +} + +static void +signal_handler(int signum) +{ + /* USR1 signal, stop testing */ + if (signum == SIGUSR1) { + printf("Force Stop!\n"); + stop = 1; + } + + /* USR2 signal, print stats */ + if (signum == SIGUSR2) + stats_display(0); +} + +struct rte_mbuf **tx_burst; + +uint64_t (*do_measure)(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts); + +static uint64_t +measure_rxtx(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t prev_tsc, cur_tsc; + + prev_tsc = rte_rdtsc(); + + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + cur_tsc = rte_rdtsc(); + + return cur_tsc - prev_tsc; +} + +static uint64_t +measure_rxonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + + cur_tsc = rte_rdtsc(); + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + diff_tsc += rte_rdtsc() - cur_tsc; + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +static uint64_t +measure_txonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + printf("do tx measure\n"); + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + + cur_tsc = rte_rdtsc(); + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + diff_tsc += rte_rdtsc() - cur_tsc; + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +/* main processing loop */ +static int +main_loop(__rte_unused void *args) +{ +#define PACKET_SIZE 64 +#define FRAME_GAP 12 +#define MAC_PREAMBLE 8 + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + unsigned lcore_id; + unsigned i, portid, nb_rx = 0, nb_tx = 0; + struct lcore_conf *conf; + int pkt_per_port; + uint64_t diff_tsc; + uint64_t packets_per_second, total_packets; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + pkt_per_port = MAX_TRAFFIC_BURST; + + int idx = 0; + for (i = 0; i < conf->nb_ports; i++) { + int num = pkt_per_port; + portid = conf->portlist[i]; + printf("inject %d packet to port %d\n", num, portid); + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + nb_tx = rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + num -= nb_tx; + idx += nb_tx; + } + } + printf("Total packets inject to prime ports = %u\n", idx); + + packets_per_second = (link_mbps * 1000 * 1000) / + ((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); + printf("Each port will do %"PRIu64" packets per second\n", + packets_per_second); + + total_packets = RTE_TEST_DURATION * conf->nb_ports * packets_per_second; + printf("Test will stop after at least %"PRIu64" packets received\n", + + total_packets); + + diff_tsc = do_measure(conf, pkts_burst, total_packets); + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + int nb_free = pkt_per_port; + do { /* dry out */ + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + nb_tx = 0; + while (nb_tx < nb_rx) + rte_pktmbuf_free(pkts_burst[nb_tx++]); + nb_free -= nb_rx; + } while (nb_free != 0); + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); + } + + if (count == 0) + return -1; + + printf("%"PRIu64" packet, %"PRIu64" drop, %"PRIu64" idle\n", + count, drop, idle); + printf("Result: %"PRIu64" cycles per packet\n", diff_tsc / count); + + return 0; +} + +rte_atomic64_t start; + +static inline int +poll_burst(void *args) +{ +#define MAX_IDLE (10000) + unsigned lcore_id; + struct rte_mbuf **pkts_burst; + uint64_t diff_tsc, cur_tsc; + uint16_t next[RTE_MAX_ETHPORTS]; + struct lcore_conf *conf; + uint32_t pkt_per_port = *((uint32_t *)args); + unsigned i, portid, nb_rx = 0; + uint64_t total; + uint64_t timeout = MAX_IDLE; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + total = pkt_per_port * conf->nb_ports; + printf("start to receive total expect %"PRIu64"\n", total); + + pkts_burst = (struct rte_mbuf **) + rte_calloc_socket("poll_burst", + total, sizeof(void *), + CACHE_LINE_SIZE, conf->socketid); + if (!pkts_burst) + return -1; + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + next[portid] = i * pkt_per_port; + } + + while (!rte_atomic64_read(&start)) + ; + + cur_tsc = rte_rdtsc(); + while (total) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + &pkts_burst[next[portid]], + MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + timeout--; + if (unlikely(timeout == 0)) + goto timeout; + continue; + } + next[portid] += nb_rx; + total -= nb_rx; + } + } +timeout: + diff_tsc = rte_rdtsc() - cur_tsc; + + printf("%"PRIu64" packets lost, IDLE %"PRIu64" times\n", + total, MAX_IDLE - timeout); + + /* clean up */ + total = pkt_per_port * conf->nb_ports - total; + for (i = 0; i < total; i++) + rte_pktmbuf_free(pkts_burst[i]); + + rte_free(pkts_burst); + + return diff_tsc / total; +} + +static int +exec_burst(uint32_t flags, int lcore) +{ + unsigned i, portid, nb_tx = 0; + struct lcore_conf *conf; + uint32_t pkt_per_port; + int num, idx = 0; + int diff_tsc; + + conf = &lcore_conf[lcore]; + + pkt_per_port = MAX_TRAFFIC_BURST; + num = pkt_per_port; + + rte_atomic64_init(&start); + + /* start polling thread, but not actually poll yet */ + rte_eal_remote_launch(poll_burst, + (void *)&pkt_per_port, lcore); + + /* Only when polling first */ + if (flags == SC_BURST_POLL_FIRST) + rte_atomic64_set(&start, 1); + + /* start xmit */ + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + idx += nb_tx; + } + num -= nb_tx; + } + + sleep(5); + + /* only when polling second */ + if (flags == SC_BURST_XMIT_FIRST) + rte_atomic64_set(&start, 1); + + /* wait for polling finished */ + diff_tsc = rte_eal_wait_lcore(lcore); + if (diff_tsc < 0) + return -1; + + printf("Result: %d cycles per packet\n", diff_tsc); + + return 0; +} + +static int +test_pmd_perf(void) +{ + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; + uint16_t nb_rxd = MAX_TRAFFIC_BURST; + uint16_t nb_txd = MAX_TRAFFIC_BURST; + uint16_t portid; + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; + int socketid = -1; + int ret; + + printf("Start PMD RXTX cycles cost test.\n"); + + signal(SIGUSR1, signal_handler); + signal(SIGUSR2, signal_handler); + + nb_ports = rte_eth_dev_count(); + if (nb_ports < NB_ETHPORTS_USED) { + printf("At least %u port(s) used for perf. test\n", + NB_ETHPORTS_USED); + return -1; + } + + if (nb_ports > RTE_MAX_ETHPORTS) + nb_ports = RTE_MAX_ETHPORTS; + + nb_lcores = rte_lcore_count(); + + memset(lcore_conf, 0, sizeof(lcore_conf)); + init_lcores(); + + init_mbufpool(NB_MBUF); + + if (sc_flag == SC_CONTINUOUS) { + nb_rxd = RTE_TEST_RX_DESC_DEFAULT; + nb_txd = RTE_TEST_TX_DESC_DEFAULT; + } + printf("CONFIG RXD=%d TXD=%d\n", nb_rxd, nb_txd); + + reset_count(); + num = 0; + for (portid = 0; portid < nb_ports; portid++) { + if (socketid == -1) { + socketid = rte_eth_dev_socket_id(portid); + slave_id = alloc_lcore(socketid); + if (slave_id == (uint16_t)-1) { + printf("No avail lcore to run test\n"); + return -1; + } + printf("Performance test runs on lcore %u socket %u\n", + slave_id, socketid); + } + + if (socketid != rte_eth_dev_socket_id(portid)) { + printf("Skip port %d\n", portid); + continue; + } + + /* port configure */ + ret = rte_eth_dev_configure(portid, nb_rx_queue, + nb_tx_queue, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot configure device: err=%d, port=%d\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); + printf("Port %u ", portid); + print_ethaddr("Address:", &ports_eth_addr[portid]); + printf("\n"); + + /* tx queue setup */ + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + socketid, &tx_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup: err=%d, " + "port=%d\n", ret, portid); + + /* rx queue steup */ + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, + socketid, &rx_conf, + mbufpool[socketid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," + "port=%d\n", ret, portid); + + /* Start device */ + stop = 0; + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start: err=%d, port=%d\n", + ret, portid); + + /* always eanble promiscuous */ + rte_eth_promiscuous_enable(portid); + + lcore_conf[slave_id].portlist[num++] = portid; + lcore_conf[slave_id].nb_ports++; + } + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + + if (tx_burst == NULL) { + tx_burst = (struct rte_mbuf **) + rte_calloc_socket("tx_buff", + MAX_TRAFFIC_BURST * nb_ports, + sizeof(void *), + CACHE_LINE_SIZE, socketid); + if (!tx_burst) + return -1; + } + + init_traffic(mbufpool[socketid], + tx_burst, MAX_TRAFFIC_BURST * nb_ports); + + printf("Generate %d packets @socket %d\n", + MAX_TRAFFIC_BURST * nb_ports, socketid); + + if (sc_flag == SC_CONTINUOUS) { + /* do both rxtx by default */ + if (NULL == do_measure) + do_measure = measure_rxtx; + + rte_eal_remote_launch(main_loop, NULL, slave_id); + + if (rte_eal_wait_lcore(slave_id) < 0) + return -1; + } else if (sc_flag == SC_BURST_POLL_FIRST || + sc_flag == SC_BURST_XMIT_FIRST) + exec_burst(sc_flag, slave_id); + + /* port tear down */ + for (portid = 0; portid < nb_ports; portid++) { + if (socketid != rte_eth_dev_socket_id(portid)) + continue; + + rte_eth_dev_stop(portid); + } + + return 0; +} + +int +test_set_rxtx_conf(cmdline_fixed_string_t mode) +{ + printf("mode switch to %s\n", mode); + + if (!strcmp(mode, "vector")) { + /* vector rx, tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "scalar")) { + /* bulk alloc rx, simple tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 128; + tx_conf.tx_free_thresh = 128; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "hybrid")) { + /* bulk alloc rx, vector tx + * when vec macro not define, + * using the same rx/tx as scalar + */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "full")) { + /* full feature rx,tx pair */ + tx_conf.txq_flags = 0x0; /* must condition */ + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 1; /* must condition */ + return 0; + } + + return -1; +} + +int +test_set_rxtx_anchor(cmdline_fixed_string_t type) +{ + printf("type switch to %s\n", type); + + if (!strcmp(type, "rxtx")) { + do_measure = measure_rxtx; + return 0; + } else if (!strcmp(type, "rxonly")) { + do_measure = measure_rxonly; + return 0; + } else if (!strcmp(type, "txonly")) { + do_measure = measure_txonly; + return 0; + } + + return -1; +} + +int +test_set_rxtx_sc(cmdline_fixed_string_t type) +{ + printf("stream control switch to %s\n", type); + + if (!strcmp(type, "continuous")) { + sc_flag = SC_CONTINUOUS; + return 0; + } else if (!strcmp(type, "poll_before_xmit")) { + sc_flag = SC_BURST_POLL_FIRST; + return 0; + } else if (!strcmp(type, "poll_after_xmit")) { + sc_flag = SC_BURST_XMIT_FIRST; + return 0; + } + + return -1; +} + +static struct test_command pmd_perf_cmd = { + .command = "pmd_perf_autotest", + .callback = test_pmd_perf, +}; +REGISTER_TEST_COMMAND(pmd_perf_cmd); diff --git a/lib/librte_ether/rte_ether.h b/lib/librte_ether/rte_ether.h index 2e08f23..b18be7b 100644 --- a/lib/librte_ether/rte_ether.h +++ b/lib/librte_ether/rte_ether.h @@ -45,6 +45,7 @@ extern "C" { #endif #include <stdint.h> +#include <stdio.h> #include <rte_memcpy.h> #include <rte_random.h> @@ -266,6 +267,30 @@ static inline void ether_addr_copy(const struct ether_addr *ea_from, #endif } +#define ETHER_ADDR_FMT_SIZE 18 +/** + * Format 48bits Ethernet address in pattern xx:xx:xx:xx:xx:xx. + * + * @param buf + * A pointer to buffer contains the formatted MAC address. + * @param size + * The format buffer size. + * @param ea_to + * A pointer to a ether_addr structure. + */ +static inline void +ether_format_addr(char *buf, uint16_t size, + const struct ether_addr *eth_addr) +{ + snprintf(buf, size, "%02X:%02X:%02X:%02X:%02X:%02X", + eth_addr->addr_bytes[0], + eth_addr->addr_bytes[1], + eth_addr->addr_bytes[2], + eth_addr->addr_bytes[3], + eth_addr->addr_bytes[4], + eth_addr->addr_bytes[5]); +} + /** * Ethernet header: Contains the destination address, source address * and frame type. diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c index c5e4b71..e85d827 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c +++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c @@ -1600,6 +1600,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev) ixgbe_dev_clear_queues(dev); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + /* Clear recorded link status */ memset(&link, 0, sizeof(link)); rte_ixgbe_dev_atomic_write_link_status(dev, &link); @@ -2888,6 +2891,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) */ ixgbevf_set_vfta_all(dev,0); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + ixgbe_dev_clear_queues(dev); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Cunming Liang 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 1/2] app/test: allow to create packets in different sizes Cunming Liang 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 2/2] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang @ 2014-10-21 2:40 ` Liu, Yong 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 0/3] " Cunming Liang 3 siblings, 0 replies; 65+ messages in thread From: Liu, Yong @ 2014-10-21 2:40 UTC (permalink / raw) To: dev Patch name: PMD performance unit test Brief description: unit test to measure cycles per packet Test Flag: Tested-by Tester name: yong.liu@intel.com Test environment: OS: Fedora20 3.11.10-301.fc20.x86_64 GCC: gcc version 4.8.3 20140911 CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] Commit ID: 455d09e54b92a4626e178b020fe9c23e43ede3f7 Detailed Testing information DPDK SW Configuration: Default x86_64-native-linuxapp-gcc configuration Test Result Summary: Total 2 cases, 2 passed, 0 failed Test Case - name: Continuous Mode Performance Test Case - Description: Measure continuous mode cycles/packet in NIC loopback mode Test Case -command / instruction: Set stream control mode to continuous RTE>>set_rxtx_sc continuous Choose rx/tx pair between vector|scalar|full|hybrid RTE>>set_rxtx_mode vector Start pmd performance measurement RTE>>pmd_perf_autotest Test Case - expected test result: Test result is OK and output cycle number for each packet. Test Case - name: Burst Mode Performance Test Case - Description: Measure burst mode cycles/packet in NIC loopback mode Test Case -command / instruction: Start unit test sample Set stream control mode to poll_before_xmit or poll_after_xmit RTE>>set_rxtx_sc poll_before_xmit Start pmd performance measurement RTE>>pmd_perf_autotest Start pmd performance measurement RTE>>pmd_perf_autotest Test Case - expected test result: Test result is OK and output cycle number for each packet. > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Cunming Liang > Sent: Monday, October 20, 2014 4:14 PM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per > packet > > v3 update: > # Codes refine according to the feedback. > 1. add ether_format_addr to rte_ether.h > 2. fix typo in code comments. > 3. %lu to %PRIu64, fixing 32-bit targets compilation err > # merge 2 small incremental patches to the first one. > The whole unit test as a single patch in [PATCH v3 2/2] > # rebase code to the latest master > > v2 update: > Rebase code to the latest master branch. > > Tested-by: Zhaochen Zhan <zhaochen.zhan@intel.com> > This patch has been verified on ixgbe and it is ready to be integrated to > dpdk.org. > For e1000 and i40e, lacking loopback supporting, this patch can't support > them for now. > > -------------- > > It provides unit test to measure cycles/packet in NIC loopback mode. > It simply gives the average cycles of IO used per packet without test > equipment. > When doing the test, make sure the link is UP. > > There's two stream control mode support, one is continues, another is burst. > The former continues to forward the injected packets until reaching a certain > amount of number. > The latter one stop when all the injected packets are received. > In burst stream, now measure two situations, with or without desc. cache > conflict. > By default, it runs in continues stream mode to measure the whole rxtx. > > Usage Example: > 1. Run unit test app in interactive mode > app/test -c f -n 4 -- -i > 2. Set stream control mode, by default is continuous > set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] > 3. If choose continuous stream, there are another two options can configure > 3.1 choose rx/tx pair, default is vector > set_rxtx_mode [vector|scalar|full|hybrid] > Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without > INC_VEC=y in config > 3.2 choose the area of masurement, default is rxtx > set_rxtx_anchor [rxtx|rxonly|txonly] > 4. Run and wait for the result > pmd_perf_autotest > > For who simply just want to see how much cycles cost per packet. > Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. > Nothing else needs to configure. > Using other options when you understand and what to measures more. > > *** BLURB HERE *** > > Cunming Liang (2): > app/test: allow to create packets in different sizes > app/test: measure the cost of rx/tx routines by cycle number > > app/test/Makefile | 1 + > app/test/commands.c | 111 +++++ > app/test/packet_burst_generator.c | 26 +- > app/test/packet_burst_generator.h | 11 +- > app/test/test.h | 6 + > app/test/test_link_bonding.c | 39 +- > app/test/test_pmd_perf.c | 922 > +++++++++++++++++++++++++++++++++++ > lib/librte_ether/rte_ether.h | 25 + > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > 9 files changed, 1114 insertions(+), 33 deletions(-) > create mode 100644 app/test/test_pmd_perf.c > > -- > 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v4 0/3] app/test: unit test to measure cycles per packet 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Cunming Liang ` (2 preceding siblings ...) 2014-10-21 2:40 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Liu, Yong @ 2014-10-24 5:39 ` Cunming Liang 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 1/3] app/test: allow to create packets in different sizes Cunming Liang ` (5 more replies) 3 siblings, 6 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-24 5:39 UTC (permalink / raw) To: dev BTW, [1/3] is the same patch as below one. http://dpdk.org/dev/patchwork/patch/817 v4 update: # fix the confusing of retval in some API of rte_ethdev v3 update: # Codes refine according to the feedback. 1. add ether_format_addr to rte_ether.h 2. fix typo in code comments. 3. %lu to %PRIu64, fixing 32-bit targets compilation err # merge 2 small incremental patches to the first one. The whole unit test as a single patch in [PATCH v3 2/2] # rebase code to the latest master v2 update: Rebase code to the latest master branch. It provides unit test to measure cycles/packet in NIC loopback mode. It simply gives the average cycles of IO used per packet without test equipment. When doing the test, make sure the link is UP. There's two stream control mode support, one is continues, another is burst. The former continues to forward the injected packets until reaching a certain amount of number. The latter one stop when all the injected packets are received. In burst stream, now measure two situations, with or without desc. cache conflict. By default, it runs in continues stream mode to measure the whole rxtx. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] 3. If choose continuous stream, there are another two options can configure 3.1 choose rx/tx pair, default is vector set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without INC_VEC=y in config 3.2 choose the area of masurement, default is rxtx set_rxtx_anchor [rxtx|rxonly|txonly] 4. Run and wait for the result pmd_perf_autotest For who simply just want to see how much cycles cost per packet. Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. Nothing else needs to configure. Using other options when you understand and what to measures more. *** BLURB HERE *** Cunming Liang (3): app/test: allow to create packets in different sizes app/test: measure the cost of rx/tx routines by cycle number ethdev: fix wrong error return refer to API definition app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/packet_burst_generator.c | 26 +- app/test/packet_burst_generator.h | 11 +- app/test/test.h | 6 + app/test/test_link_bonding.c | 39 +- app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.c | 6 +- lib/librte_ether/rte_ether.h | 25 + lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 10 files changed, 1117 insertions(+), 36 deletions(-) create mode 100644 app/test/test_pmd_perf.c -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v4 1/3] app/test: allow to create packets in different sizes 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 0/3] " Cunming Liang @ 2014-10-24 5:39 ` Cunming Liang 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang ` (4 subsequent siblings) 5 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-24 5:39 UTC (permalink / raw) To: dev adding support to allow packet burst generator to create packets in differenct sizes Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- app/test/packet_burst_generator.c | 26 ++++++++---------------- app/test/packet_burst_generator.h | 11 +++++++-- app/test/test_link_bonding.c | 39 ++++++++++++++++++++++++------------ 3 files changed, 43 insertions(+), 33 deletions(-) diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c index 9e747a4..017139b 100644 --- a/app/test/packet_burst_generator.c +++ b/app/test/packet_burst_generator.c @@ -191,20 +191,12 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, */ #define RTE_MAX_SEGS_PER_PKT 255 /**< pkt.nb_segs is a 8-bit unsigned char. */ -#define TXONLY_DEF_PACKET_LEN 64 -#define TXONLY_DEF_PACKET_LEN_128 128 - -uint16_t tx_pkt_length = TXONLY_DEF_PACKET_LEN; -uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT] = { - TXONLY_DEF_PACKET_LEN_128, -}; - -uint8_t tx_pkt_nb_segs = 1; - int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst) + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs) { int i, nb_pkt = 0; size_t eth_hdr_size; @@ -221,9 +213,9 @@ nomore_mbuf: break; } - pkt->data_len = tx_pkt_seg_lengths[0]; + pkt->data_len = pkt_len; pkt_seg = pkt; - for (i = 1; i < tx_pkt_nb_segs; i++) { + for (i = 1; i < nb_pkt_segs; i++) { pkt_seg->next = rte_pktmbuf_alloc(mp); if (pkt_seg->next == NULL) { pkt->nb_segs = i; @@ -231,7 +223,7 @@ nomore_mbuf: goto nomore_mbuf; } pkt_seg = pkt_seg->next; - pkt_seg->data_len = tx_pkt_seg_lengths[i]; + pkt_seg->data_len = pkt_len; } pkt_seg->next = NULL; /* Last segment of packet. */ @@ -259,8 +251,8 @@ nomore_mbuf: * Complete first mbuf of packet and append it to the * burst of packets to be transmitted. */ - pkt->nb_segs = tx_pkt_nb_segs; - pkt->pkt_len = tx_pkt_length; + pkt->nb_segs = nb_pkt_segs; + pkt->pkt_len = pkt_len; pkt->l2_len = eth_hdr_size; if (ipv4) { diff --git a/app/test/packet_burst_generator.h b/app/test/packet_burst_generator.h index 5b3cd6c..fe992ac 100644 --- a/app/test/packet_burst_generator.h +++ b/app/test/packet_burst_generator.h @@ -47,10 +47,13 @@ extern "C" { #define IPV4_ADDR(a, b, c, d)(((a & 0xff) << 24) | ((b & 0xff) << 16) | \ ((c & 0xff) << 8) | (d & 0xff)) +#define PACKET_BURST_GEN_PKT_LEN 60 +#define PACKET_BURST_GEN_PKT_LEN_128 128 void initialize_eth_header(struct ether_hdr *eth_hdr, struct ether_addr *src_mac, - struct ether_addr *dst_mac, uint8_t vlan_enabled, uint16_t van_id); + struct ether_addr *dst_mac, uint8_t vlan_enabled, + uint16_t van_id); uint16_t initialize_udp_header(struct udp_hdr *udp_hdr, uint16_t src_port, @@ -67,8 +70,10 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst); + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs); #ifdef __cplusplus } diff --git a/app/test/test_link_bonding.c b/app/test/test_link_bonding.c index 214d2a2..d407e4f 100644 --- a/app/test/test_link_bonding.c +++ b/app/test/test_link_bonding.c @@ -1192,9 +1192,12 @@ generate_test_burst(struct rte_mbuf **pkts_burst, uint16_t burst_size, } /* Generate burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, vlan, ip_hdr, ipv4, - test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, + vlan, ip_hdr, ipv4, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN_128, 1); if (generated_burst_size != burst_size) { printf("Failed to generate packet burst"); return -1; @@ -1781,9 +1784,12 @@ test_activebackup_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; @@ -2435,8 +2441,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 1 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[0][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[0]) != burst_size[0]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[0], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[0]) return -1; initialize_eth_header(test_params->pkt_eth_hdr, @@ -2444,8 +2452,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 2 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[1][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[1]) != burst_size[1]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[1], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[1]) return -1; /* Send burst 1 on bonded port */ @@ -3263,9 +3273,12 @@ test_broadcast_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v4 2/3] app/test: measure the cost of rx/tx routines by cycle number 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 0/3] " Cunming Liang 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 1/3] app/test: allow to create packets in different sizes Cunming Liang @ 2014-10-24 5:39 ` Cunming Liang 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 3/3] ethdev: fix wrong error return refer to API definition Cunming Liang ` (3 subsequent siblings) 5 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-24 5:39 UTC (permalink / raw) To: dev The unit test can be used to measure cycles per packet in different rx/tx rouines. The NIC works in loopback mode. So it doesn't require test equipment to measure throughput. As result, the unit test shows the average cycles per packet consuming. When doing the test, make sure the link is UP. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Run and wait for the result pmd_perf_autotest There's option to choose rx/tx pair, default is vector. set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without INC_VEC=y in config It supports to measure standalone rx or tx. Usage Example: Choose rx or tx standalone, default is both set_rxtx_anchor [rxtx|rxonly|txonly] It also supports to measure standalone RX burst cycles. In this way, it won't repeat re-send recevied packets. Now it measures two situations, poll before/after xmit(w or w/o desc. cache conflict) Usage Example: Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/test.h | 6 + app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ether.h | 25 + lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 6 files changed, 1071 insertions(+), 0 deletions(-) create mode 100644 app/test/test_pmd_perf.c diff --git a/app/test/Makefile b/app/test/Makefile index 6af6d76..ebfa0ba 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c SRCS-y += test_ring.c SRCS-y += test_ring_perf.c +SRCS-y += test_pmd_perf.c ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) SRCS-y += test_table.c diff --git a/app/test/commands.c b/app/test/commands.c index a9e36b1..92a17ed 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -310,12 +310,123 @@ cmdline_parse_inst_t cmd_quit = { /****************/ +struct cmd_set_rxtx_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t mode; +}; + +static void cmd_set_rxtx_parsed(void *parsed_result, struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_result *res = parsed_result; + if (test_set_rxtx_conf(res->mode) < 0) + cmdline_printf(cl, "Cannot find such mode\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, set, + "set_rxtx_mode"); + +cmdline_parse_token_string_t cmd_set_rxtx_mode = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, mode, NULL); + +cmdline_parse_inst_t cmd_set_rxtx = { + .f = cmd_set_rxtx_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx routine: " + "set_rxtx <mode>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_set, + (void *)&cmd_set_rxtx_mode, + NULL, + }, +}; + +/****************/ + +struct cmd_set_rxtx_anchor { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_anchor_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_anchor *res = parsed_result; + if (test_set_rxtx_anchor(res->type) < 0) + cmdline_printf(cl, "Cannot find such anchor\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, set, + "set_rxtx_anchor"); + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_anchor = { + .f = cmd_set_rxtx_anchor_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx anchor: " + "set_rxtx_anchor <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_anchor_set, + (void *)&cmd_set_rxtx_anchor_type, + NULL, + }, +}; + +/****************/ + +/* for stream control */ +struct cmd_set_rxtx_sc { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_sc_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_sc *res = parsed_result; + if (test_set_rxtx_sc(res->type) < 0) + cmdline_printf(cl, "Cannot find such stream control\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_sc_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, set, + "set_rxtx_sc"); + +cmdline_parse_token_string_t cmd_set_rxtx_sc_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_sc = { + .f = cmd_set_rxtx_sc_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx stream control: " + "set_rxtx_sc <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_sc_set, + (void *)&cmd_set_rxtx_sc_type, + NULL, + }, +}; + +/****************/ + + cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, (cmdline_parse_inst_t *)&cmd_dump, (cmdline_parse_inst_t *)&cmd_dump_one, (cmdline_parse_inst_t *)&cmd_set_ring, (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_set_rxtx, + (cmdline_parse_inst_t *)&cmd_set_rxtx_anchor, + (cmdline_parse_inst_t *)&cmd_set_rxtx_sc, NULL, }; diff --git a/app/test/test.h b/app/test/test.h index 98ab804..bd44a7f 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -125,6 +125,9 @@ int unit_test_suite_runner(struct unit_test_suite *suite); #define RECURSIVE_ENV_VAR "RTE_TEST_RECURSIVE" +#include <cmdline_parse.h> +#include <cmdline_parse_string.h> + extern const char *prgname; int commands_init(void); @@ -137,6 +140,9 @@ int test_pci_run; int test_mp_secondary(void); int test_ivshmem(void); +int test_set_rxtx_conf(cmdline_fixed_string_t mode); +int test_set_rxtx_anchor(cmdline_fixed_string_t type); +int test_set_rxtx_sc(cmdline_fixed_string_t type); typedef int (test_callback)(void); TAILQ_HEAD(test_commands_list, test_command); diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c new file mode 100644 index 0000000..1c1f236 --- /dev/null +++ b/app/test/test_pmd_perf.c @@ -0,0 +1,922 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + + +#include <stdio.h> +#include <inttypes.h> +#include <signal.h> +#include <unistd.h> +#include <rte_cycles.h> +#include <rte_ethdev.h> +#include <rte_byteorder.h> +#include <rte_atomic.h> +#include <rte_malloc.h> +#include "packet_burst_generator.h" +#include "test.h" + +#define NB_ETHPORTS_USED (1) +#define NB_SOCKETS (2) +#define MEMPOOL_CACHE_SIZE 250 +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MAX_PKT_BURST (32) +#define RTE_TEST_RX_DESC_DEFAULT (128) +#define RTE_TEST_TX_DESC_DEFAULT (512) +#define RTE_PORT_ALL (~(uint8_t)0x0) + +/* how long test would take at full line rate */ +#define RTE_TEST_DURATION (2) + +/* + * RX and TX Prefetch, Host, and Write-back threshold values should be + * carefully set for optimal performance. Consult the network + * controller's datasheet and supporting DPDK documentation for guidance + * on how these parameters should be set. + */ +#define RX_PTHRESH 8 /**< Default values of RX prefetch threshold reg. */ +#define RX_HTHRESH 8 /**< Default values of RX host threshold reg. */ +#define RX_WTHRESH 0 /**< Default values of RX write-back threshold reg. */ + +/* + * These default values are optimized for use with the Intel(R) 82599 10 GbE + * Controller and the DPDK ixgbe PMD. Consider using other values for other + * network controllers and/or network drivers. + */ +#define TX_PTHRESH 32 /**< Default values of TX prefetch threshold reg. */ +#define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */ +#define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */ + +#define MAX_TRAFFIC_BURST 2048 + +#define NB_MBUF RTE_MAX( \ + (unsigned)(nb_ports*nb_rx_queue*nb_rxd + \ + nb_ports*nb_lcores*MAX_PKT_BURST + \ + nb_ports*nb_tx_queue*nb_txd + \ + nb_lcores*MEMPOOL_CACHE_SIZE + \ + nb_ports*MAX_TRAFFIC_BURST), \ + (unsigned)8192) + + +static struct rte_mempool *mbufpool[NB_SOCKETS]; +/* ethernet addresses of ports */ +static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS]; + +static struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .hw_vlan_strip = 0, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 1, /* enable loopback */ +}; + +static struct rte_eth_rxconf rx_conf = { + .rx_thresh = { + .pthresh = RX_PTHRESH, + .hthresh = RX_HTHRESH, + .wthresh = RX_WTHRESH, + }, + .rx_free_thresh = 32, +}; + +static struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = TX_PTHRESH, + .hthresh = TX_HTHRESH, + .wthresh = TX_WTHRESH, + }, + .tx_free_thresh = 32, /* Use PMD default values */ + .tx_rs_thresh = 32, /* Use PMD default values */ + .txq_flags = (ETH_TXQ_FLAGS_NOMULTSEGS | + ETH_TXQ_FLAGS_NOVLANOFFL | + ETH_TXQ_FLAGS_NOXSUMSCTP | + ETH_TXQ_FLAGS_NOXSUMUDP | + ETH_TXQ_FLAGS_NOXSUMTCP) +}; + +enum { + LCORE_INVALID = 0, + LCORE_AVAIL, + LCORE_USED, +}; + +struct lcore_conf { + uint8_t status; + uint8_t socketid; + uint16_t nb_ports; + uint8_t portlist[RTE_MAX_ETHPORTS]; +} __rte_cache_aligned; + +struct lcore_conf lcore_conf[RTE_MAX_LCORE]; + +static uint64_t link_mbps; + +enum { + SC_CONTINUOUS = 0, + SC_BURST_POLL_FIRST, + SC_BURST_XMIT_FIRST, +}; + +static uint32_t sc_flag; + +/* Check the link status of all ports in up to 3s, and print them finally */ +static void +check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +{ +#define CHECK_INTERVAL 100 /* 100ms */ +#define MAX_CHECK_TIME 30 /* 3s (30 * 100ms) in total */ + uint8_t portid, count, all_ports_up, print_flag = 0; + struct rte_eth_link link; + + printf("Checking link statuses...\n"); + fflush(stdout); + for (count = 0; count <= MAX_CHECK_TIME; count++) { + all_ports_up = 1; + for (portid = 0; portid < port_num; portid++) { + if ((port_mask & (1 << portid)) == 0) + continue; + memset(&link, 0, sizeof(link)); + rte_eth_link_get_nowait(portid, &link); + /* print link status if flag set */ + if (print_flag == 1) { + if (link.link_status) { + printf("Port %d Link Up - speed %u " + "Mbps - %s\n", (uint8_t)portid, + (unsigned)link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + if (link_mbps == 0) + link_mbps = link.link_speed; + } else + printf("Port %d Link Down\n", + (uint8_t)portid); + continue; + } + /* clear all_ports_up flag if any link down */ + if (link.link_status == 0) { + all_ports_up = 0; + break; + } + } + /* after finally printing all link status, get out */ + if (print_flag == 1) + break; + + if (all_ports_up == 0) { + fflush(stdout); + rte_delay_ms(CHECK_INTERVAL); + } + + /* set the print_flag if all ports up or timeout */ + if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) + print_flag = 1; + } +} + +static void +print_ethaddr(const char *name, const struct ether_addr *eth_addr) +{ + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); +} + +static int +init_traffic(struct rte_mempool *mp, + struct rte_mbuf **pkts_burst, uint32_t burst_size) +{ + struct ether_hdr pkt_eth_hdr; + struct ipv4_hdr pkt_ipv4_hdr; + struct udp_hdr pkt_udp_hdr; + uint32_t pktlen; + static uint8_t src_mac[] = { 0x00, 0xFF, 0xAA, 0xFF, 0xAA, 0xFF }; + static uint8_t dst_mac[] = { 0x00, 0xAA, 0xFF, 0xAA, 0xFF, 0xAA }; + + + initialize_eth_header(&pkt_eth_hdr, + (struct ether_addr *)src_mac, + (struct ether_addr *)dst_mac, 0, 0); + pkt_eth_hdr.ether_type = rte_cpu_to_be_16(ETHER_TYPE_IPv4); + + pktlen = initialize_ipv4_header(&pkt_ipv4_hdr, + IPV4_ADDR(10, 0, 0, 1), + IPV4_ADDR(10, 0, 0, 2), 26); + printf("IPv4 pktlen %u\n", pktlen); + + pktlen = initialize_udp_header(&pkt_udp_hdr, 0, 0, 18); + + printf("UDP pktlen %u\n", pktlen); + + return generate_packet_burst(mp, pkts_burst, &pkt_eth_hdr, + 0, &pkt_ipv4_hdr, 1, + &pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); +} + +static int +init_lcores(void) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + lcore_conf[lcore_id].socketid = + rte_lcore_to_socket_id(lcore_id); + if (rte_lcore_is_enabled(lcore_id) == 0) { + lcore_conf[lcore_id].status = LCORE_INVALID; + continue; + } else + lcore_conf[lcore_id].status = LCORE_AVAIL; + } + return 0; +} + +static int +init_mbufpool(unsigned nb_mbuf) +{ + int socketid; + unsigned lcore_id; + char s[64]; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + socketid = rte_lcore_to_socket_id(lcore_id); + if (socketid >= NB_SOCKETS) { + rte_exit(EXIT_FAILURE, + "Socket %d of lcore %u is out of range %d\n", + socketid, lcore_id, NB_SOCKETS); + } + if (mbufpool[socketid] == NULL) { + snprintf(s, sizeof(s), "mbuf_pool_%d", socketid); + mbufpool[socketid] = + rte_mempool_create(s, nb_mbuf, MBUF_SIZE, + MEMPOOL_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + socketid, 0); + if (mbufpool[socketid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot init mbuf pool on socket %d\n", + socketid); + else + printf("Allocated mbuf pool on socket %d\n", + socketid); + } + } + return 0; +} + +static uint16_t +alloc_lcore(uint16_t socketid) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (LCORE_AVAIL != lcore_conf[lcore_id].status || + lcore_conf[lcore_id].socketid != socketid || + lcore_id == rte_get_master_lcore()) + continue; + lcore_conf[lcore_id].status = LCORE_USED; + lcore_conf[lcore_id].nb_ports = 0; + return lcore_id; + } + + return (uint16_t)-1; +} + +volatile uint64_t stop; +uint64_t count; +uint64_t drop; +uint64_t idle; + +static void +reset_count(void) +{ + count = 0; + drop = 0; + idle = 0; +} + +static void +stats_display(uint8_t port_id) +{ + struct rte_eth_stats stats; + rte_eth_stats_get(port_id, &stats); + + printf(" RX-packets: %-10"PRIu64" RX-missed: %-10"PRIu64" RX-bytes: " + "%-"PRIu64"\n", + stats.ipackets, stats.imissed, stats.ibytes); + printf(" RX-badcrc: %-10"PRIu64" RX-badlen: %-10"PRIu64" RX-errors: " + "%-"PRIu64"\n", + stats.ibadcrc, stats.ibadlen, stats.ierrors); + printf(" RX-nombuf: %-10"PRIu64"\n", + stats.rx_nombuf); + printf(" TX-packets: %-10"PRIu64" TX-errors: %-10"PRIu64" TX-bytes: " + "%-"PRIu64"\n", + stats.opackets, stats.oerrors, stats.obytes); +} + +static void +signal_handler(int signum) +{ + /* USR1 signal, stop testing */ + if (signum == SIGUSR1) { + printf("Force Stop!\n"); + stop = 1; + } + + /* USR2 signal, print stats */ + if (signum == SIGUSR2) + stats_display(0); +} + +struct rte_mbuf **tx_burst; + +uint64_t (*do_measure)(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts); + +static uint64_t +measure_rxtx(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t prev_tsc, cur_tsc; + + prev_tsc = rte_rdtsc(); + + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + cur_tsc = rte_rdtsc(); + + return cur_tsc - prev_tsc; +} + +static uint64_t +measure_rxonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + + cur_tsc = rte_rdtsc(); + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + diff_tsc += rte_rdtsc() - cur_tsc; + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +static uint64_t +measure_txonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + printf("do tx measure\n"); + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + + cur_tsc = rte_rdtsc(); + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + diff_tsc += rte_rdtsc() - cur_tsc; + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +/* main processing loop */ +static int +main_loop(__rte_unused void *args) +{ +#define PACKET_SIZE 64 +#define FRAME_GAP 12 +#define MAC_PREAMBLE 8 + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + unsigned lcore_id; + unsigned i, portid, nb_rx = 0, nb_tx = 0; + struct lcore_conf *conf; + int pkt_per_port; + uint64_t diff_tsc; + uint64_t packets_per_second, total_packets; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + pkt_per_port = MAX_TRAFFIC_BURST; + + int idx = 0; + for (i = 0; i < conf->nb_ports; i++) { + int num = pkt_per_port; + portid = conf->portlist[i]; + printf("inject %d packet to port %d\n", num, portid); + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + nb_tx = rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + num -= nb_tx; + idx += nb_tx; + } + } + printf("Total packets inject to prime ports = %u\n", idx); + + packets_per_second = (link_mbps * 1000 * 1000) / + ((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); + printf("Each port will do %"PRIu64" packets per second\n", + packets_per_second); + + total_packets = RTE_TEST_DURATION * conf->nb_ports * packets_per_second; + printf("Test will stop after at least %"PRIu64" packets received\n", + + total_packets); + + diff_tsc = do_measure(conf, pkts_burst, total_packets); + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + int nb_free = pkt_per_port; + do { /* dry out */ + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + nb_tx = 0; + while (nb_tx < nb_rx) + rte_pktmbuf_free(pkts_burst[nb_tx++]); + nb_free -= nb_rx; + } while (nb_free != 0); + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); + } + + if (count == 0) + return -1; + + printf("%"PRIu64" packet, %"PRIu64" drop, %"PRIu64" idle\n", + count, drop, idle); + printf("Result: %"PRIu64" cycles per packet\n", diff_tsc / count); + + return 0; +} + +rte_atomic64_t start; + +static inline int +poll_burst(void *args) +{ +#define MAX_IDLE (10000) + unsigned lcore_id; + struct rte_mbuf **pkts_burst; + uint64_t diff_tsc, cur_tsc; + uint16_t next[RTE_MAX_ETHPORTS]; + struct lcore_conf *conf; + uint32_t pkt_per_port = *((uint32_t *)args); + unsigned i, portid, nb_rx = 0; + uint64_t total; + uint64_t timeout = MAX_IDLE; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + total = pkt_per_port * conf->nb_ports; + printf("start to receive total expect %"PRIu64"\n", total); + + pkts_burst = (struct rte_mbuf **) + rte_calloc_socket("poll_burst", + total, sizeof(void *), + CACHE_LINE_SIZE, conf->socketid); + if (!pkts_burst) + return -1; + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + next[portid] = i * pkt_per_port; + } + + while (!rte_atomic64_read(&start)) + ; + + cur_tsc = rte_rdtsc(); + while (total) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + &pkts_burst[next[portid]], + MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + timeout--; + if (unlikely(timeout == 0)) + goto timeout; + continue; + } + next[portid] += nb_rx; + total -= nb_rx; + } + } +timeout: + diff_tsc = rte_rdtsc() - cur_tsc; + + printf("%"PRIu64" packets lost, IDLE %"PRIu64" times\n", + total, MAX_IDLE - timeout); + + /* clean up */ + total = pkt_per_port * conf->nb_ports - total; + for (i = 0; i < total; i++) + rte_pktmbuf_free(pkts_burst[i]); + + rte_free(pkts_burst); + + return diff_tsc / total; +} + +static int +exec_burst(uint32_t flags, int lcore) +{ + unsigned i, portid, nb_tx = 0; + struct lcore_conf *conf; + uint32_t pkt_per_port; + int num, idx = 0; + int diff_tsc; + + conf = &lcore_conf[lcore]; + + pkt_per_port = MAX_TRAFFIC_BURST; + num = pkt_per_port; + + rte_atomic64_init(&start); + + /* start polling thread, but not actually poll yet */ + rte_eal_remote_launch(poll_burst, + (void *)&pkt_per_port, lcore); + + /* Only when polling first */ + if (flags == SC_BURST_POLL_FIRST) + rte_atomic64_set(&start, 1); + + /* start xmit */ + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + idx += nb_tx; + } + num -= nb_tx; + } + + sleep(5); + + /* only when polling second */ + if (flags == SC_BURST_XMIT_FIRST) + rte_atomic64_set(&start, 1); + + /* wait for polling finished */ + diff_tsc = rte_eal_wait_lcore(lcore); + if (diff_tsc < 0) + return -1; + + printf("Result: %d cycles per packet\n", diff_tsc); + + return 0; +} + +static int +test_pmd_perf(void) +{ + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; + uint16_t nb_rxd = MAX_TRAFFIC_BURST; + uint16_t nb_txd = MAX_TRAFFIC_BURST; + uint16_t portid; + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; + int socketid = -1; + int ret; + + printf("Start PMD RXTX cycles cost test.\n"); + + signal(SIGUSR1, signal_handler); + signal(SIGUSR2, signal_handler); + + nb_ports = rte_eth_dev_count(); + if (nb_ports < NB_ETHPORTS_USED) { + printf("At least %u port(s) used for perf. test\n", + NB_ETHPORTS_USED); + return -1; + } + + if (nb_ports > RTE_MAX_ETHPORTS) + nb_ports = RTE_MAX_ETHPORTS; + + nb_lcores = rte_lcore_count(); + + memset(lcore_conf, 0, sizeof(lcore_conf)); + init_lcores(); + + init_mbufpool(NB_MBUF); + + if (sc_flag == SC_CONTINUOUS) { + nb_rxd = RTE_TEST_RX_DESC_DEFAULT; + nb_txd = RTE_TEST_TX_DESC_DEFAULT; + } + printf("CONFIG RXD=%d TXD=%d\n", nb_rxd, nb_txd); + + reset_count(); + num = 0; + for (portid = 0; portid < nb_ports; portid++) { + if (socketid == -1) { + socketid = rte_eth_dev_socket_id(portid); + slave_id = alloc_lcore(socketid); + if (slave_id == (uint16_t)-1) { + printf("No avail lcore to run test\n"); + return -1; + } + printf("Performance test runs on lcore %u socket %u\n", + slave_id, socketid); + } + + if (socketid != rte_eth_dev_socket_id(portid)) { + printf("Skip port %d\n", portid); + continue; + } + + /* port configure */ + ret = rte_eth_dev_configure(portid, nb_rx_queue, + nb_tx_queue, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot configure device: err=%d, port=%d\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); + printf("Port %u ", portid); + print_ethaddr("Address:", &ports_eth_addr[portid]); + printf("\n"); + + /* tx queue setup */ + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + socketid, &tx_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup: err=%d, " + "port=%d\n", ret, portid); + + /* rx queue steup */ + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, + socketid, &rx_conf, + mbufpool[socketid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," + "port=%d\n", ret, portid); + + /* Start device */ + stop = 0; + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start: err=%d, port=%d\n", + ret, portid); + + /* always eanble promiscuous */ + rte_eth_promiscuous_enable(portid); + + lcore_conf[slave_id].portlist[num++] = portid; + lcore_conf[slave_id].nb_ports++; + } + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + + if (tx_burst == NULL) { + tx_burst = (struct rte_mbuf **) + rte_calloc_socket("tx_buff", + MAX_TRAFFIC_BURST * nb_ports, + sizeof(void *), + CACHE_LINE_SIZE, socketid); + if (!tx_burst) + return -1; + } + + init_traffic(mbufpool[socketid], + tx_burst, MAX_TRAFFIC_BURST * nb_ports); + + printf("Generate %d packets @socket %d\n", + MAX_TRAFFIC_BURST * nb_ports, socketid); + + if (sc_flag == SC_CONTINUOUS) { + /* do both rxtx by default */ + if (NULL == do_measure) + do_measure = measure_rxtx; + + rte_eal_remote_launch(main_loop, NULL, slave_id); + + if (rte_eal_wait_lcore(slave_id) < 0) + return -1; + } else if (sc_flag == SC_BURST_POLL_FIRST || + sc_flag == SC_BURST_XMIT_FIRST) + exec_burst(sc_flag, slave_id); + + /* port tear down */ + for (portid = 0; portid < nb_ports; portid++) { + if (socketid != rte_eth_dev_socket_id(portid)) + continue; + + rte_eth_dev_stop(portid); + } + + return 0; +} + +int +test_set_rxtx_conf(cmdline_fixed_string_t mode) +{ + printf("mode switch to %s\n", mode); + + if (!strcmp(mode, "vector")) { + /* vector rx, tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "scalar")) { + /* bulk alloc rx, simple tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 128; + tx_conf.tx_free_thresh = 128; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "hybrid")) { + /* bulk alloc rx, vector tx + * when vec macro not define, + * using the same rx/tx as scalar + */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "full")) { + /* full feature rx,tx pair */ + tx_conf.txq_flags = 0x0; /* must condition */ + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 1; /* must condition */ + return 0; + } + + return -1; +} + +int +test_set_rxtx_anchor(cmdline_fixed_string_t type) +{ + printf("type switch to %s\n", type); + + if (!strcmp(type, "rxtx")) { + do_measure = measure_rxtx; + return 0; + } else if (!strcmp(type, "rxonly")) { + do_measure = measure_rxonly; + return 0; + } else if (!strcmp(type, "txonly")) { + do_measure = measure_txonly; + return 0; + } + + return -1; +} + +int +test_set_rxtx_sc(cmdline_fixed_string_t type) +{ + printf("stream control switch to %s\n", type); + + if (!strcmp(type, "continuous")) { + sc_flag = SC_CONTINUOUS; + return 0; + } else if (!strcmp(type, "poll_before_xmit")) { + sc_flag = SC_BURST_POLL_FIRST; + return 0; + } else if (!strcmp(type, "poll_after_xmit")) { + sc_flag = SC_BURST_XMIT_FIRST; + return 0; + } + + return -1; +} + +static struct test_command pmd_perf_cmd = { + .command = "pmd_perf_autotest", + .callback = test_pmd_perf, +}; +REGISTER_TEST_COMMAND(pmd_perf_cmd); diff --git a/lib/librte_ether/rte_ether.h b/lib/librte_ether/rte_ether.h index 2e08f23..b18be7b 100644 --- a/lib/librte_ether/rte_ether.h +++ b/lib/librte_ether/rte_ether.h @@ -45,6 +45,7 @@ extern "C" { #endif #include <stdint.h> +#include <stdio.h> #include <rte_memcpy.h> #include <rte_random.h> @@ -266,6 +267,30 @@ static inline void ether_addr_copy(const struct ether_addr *ea_from, #endif } +#define ETHER_ADDR_FMT_SIZE 18 +/** + * Format 48bits Ethernet address in pattern xx:xx:xx:xx:xx:xx. + * + * @param buf + * A pointer to buffer contains the formatted MAC address. + * @param size + * The format buffer size. + * @param ea_to + * A pointer to a ether_addr structure. + */ +static inline void +ether_format_addr(char *buf, uint16_t size, + const struct ether_addr *eth_addr) +{ + snprintf(buf, size, "%02X:%02X:%02X:%02X:%02X:%02X", + eth_addr->addr_bytes[0], + eth_addr->addr_bytes[1], + eth_addr->addr_bytes[2], + eth_addr->addr_bytes[3], + eth_addr->addr_bytes[4], + eth_addr->addr_bytes[5]); +} + /** * Ethernet header: Contains the destination address, source address * and frame type. diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c index c5e4b71..e85d827 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c +++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c @@ -1600,6 +1600,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev) ixgbe_dev_clear_queues(dev); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + /* Clear recorded link status */ memset(&link, 0, sizeof(link)); rte_ixgbe_dev_atomic_write_link_status(dev, &link); @@ -2888,6 +2891,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) */ ixgbevf_set_vfta_all(dev,0); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + ixgbe_dev_clear_queues(dev); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v4 3/3] ethdev: fix wrong error return refer to API definition 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 0/3] " Cunming Liang 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 1/3] app/test: allow to create packets in different sizes Cunming Liang 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang @ 2014-10-24 5:39 ` Cunming Liang 2014-10-24 5:57 ` [dpdk-dev] [PATCH v5 0/3] app/test: unit test to measure cycles per packet Cunming Liang ` (2 subsequent siblings) 5 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-24 5:39 UTC (permalink / raw) To: dev Per definition, rte_eth_rx_burst/rte_eth_tx_burst/rte_eth_rx_queue_count returns the packet number. When RTE_LIBRTE_ETHDEV_DEBUG turns on, retval of FUNC_PTR_OR_ERR_RTE was set to -ENOTSUP. It makes confusing. The patch always return 0 no matter no packet or there's error. Meanwhile set errno in such kind of checking. Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- lib/librte_ether/rte_ethdev.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 50f10d9..922a0c6 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -2530,7 +2530,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, return 0; } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); return 0; @@ -2551,7 +2551,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); if (queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", queue_id); return 0; @@ -2570,7 +2570,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) return 0; } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v5 0/3] app/test: unit test to measure cycles per packet 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 0/3] " Cunming Liang ` (2 preceding siblings ...) 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 3/3] ethdev: fix wrong error return refer to API definition Cunming Liang @ 2014-10-24 5:57 ` Cunming Liang 2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 1/3] app/test: allow to create packets in different sizes Cunming Liang ` (3 more replies) 2014-10-24 5:59 ` [dpdk-dev] [PATCH v4 0/3] " Liang, Cunming [not found] ` <1414130090-17910-1-git-send-email-y> 5 siblings, 4 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-24 5:57 UTC (permalink / raw) To: dev BTW, [1/3] is the same patch as below one. http://dpdk.org/dev/patchwork/patch/817 v5 update: # fix the confusing of retval in some API of rte_ethdev v4 ignore v3 update: # Codes refine according to the feedback. 1. add ether_format_addr to rte_ether.h 2. fix typo in code comments. 3. %lu to %PRIu64, fixing 32-bit targets compilation err # merge 2 small incremental patches to the first one. The whole unit test as a single patch in [PATCH v3 2/2] # rebase code to the latest master v2 update: Rebase code to the latest master branch. It provides unit test to measure cycles/packet in NIC loopback mode. It simply gives the average cycles of IO used per packet without test equipment. When doing the test, make sure the link is UP. There's two stream control mode support, one is continues, another is burst. The former continues to forward the injected packets until reaching a certain amount of number. The latter one stop when all the injected packets are received. In burst stream, now measure two situations, with or without desc. cache conflict. By default, it runs in continues stream mode to measure the whole rxtx. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] 3. If choose continuous stream, there are another two options can configure 3.1 choose rx/tx pair, default is vector set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without INC_VEC=y in config 3.2 choose the area of masurement, default is rxtx set_rxtx_anchor [rxtx|rxonly|txonly] 4. Run and wait for the result pmd_perf_autotest For who simply just want to see how much cycles cost per packet. Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. Nothing else needs to configure. Using other options when you understand and what to measures more. *** BLURB HERE *** Cunming Liang (3): app/test: allow to create packets in different sizes app/test: measure the cost of rx/tx routines by cycle number ethdev: fix wrong error return refere to API definition app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/packet_burst_generator.c | 26 +- app/test/packet_burst_generator.h | 11 +- app/test/test.h | 6 + app/test/test_link_bonding.c | 39 +- app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.c | 10 +- lib/librte_ether/rte_ether.h | 25 + lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 10 files changed, 1121 insertions(+), 36 deletions(-) create mode 100644 app/test/test_pmd_perf.c -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v5 1/3] app/test: allow to create packets in different sizes 2014-10-24 5:57 ` [dpdk-dev] [PATCH v5 0/3] app/test: unit test to measure cycles per packet Cunming Liang @ 2014-10-24 5:58 ` Cunming Liang 2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang ` (2 subsequent siblings) 3 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-24 5:58 UTC (permalink / raw) To: dev adding support to allow packet burst generator to create packets in differenct sizes Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- app/test/packet_burst_generator.c | 26 ++++++++---------------- app/test/packet_burst_generator.h | 11 +++++++-- app/test/test_link_bonding.c | 39 ++++++++++++++++++++++++------------ 3 files changed, 43 insertions(+), 33 deletions(-) diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c index 9e747a4..017139b 100644 --- a/app/test/packet_burst_generator.c +++ b/app/test/packet_burst_generator.c @@ -191,20 +191,12 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, */ #define RTE_MAX_SEGS_PER_PKT 255 /**< pkt.nb_segs is a 8-bit unsigned char. */ -#define TXONLY_DEF_PACKET_LEN 64 -#define TXONLY_DEF_PACKET_LEN_128 128 - -uint16_t tx_pkt_length = TXONLY_DEF_PACKET_LEN; -uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT] = { - TXONLY_DEF_PACKET_LEN_128, -}; - -uint8_t tx_pkt_nb_segs = 1; - int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst) + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs) { int i, nb_pkt = 0; size_t eth_hdr_size; @@ -221,9 +213,9 @@ nomore_mbuf: break; } - pkt->data_len = tx_pkt_seg_lengths[0]; + pkt->data_len = pkt_len; pkt_seg = pkt; - for (i = 1; i < tx_pkt_nb_segs; i++) { + for (i = 1; i < nb_pkt_segs; i++) { pkt_seg->next = rte_pktmbuf_alloc(mp); if (pkt_seg->next == NULL) { pkt->nb_segs = i; @@ -231,7 +223,7 @@ nomore_mbuf: goto nomore_mbuf; } pkt_seg = pkt_seg->next; - pkt_seg->data_len = tx_pkt_seg_lengths[i]; + pkt_seg->data_len = pkt_len; } pkt_seg->next = NULL; /* Last segment of packet. */ @@ -259,8 +251,8 @@ nomore_mbuf: * Complete first mbuf of packet and append it to the * burst of packets to be transmitted. */ - pkt->nb_segs = tx_pkt_nb_segs; - pkt->pkt_len = tx_pkt_length; + pkt->nb_segs = nb_pkt_segs; + pkt->pkt_len = pkt_len; pkt->l2_len = eth_hdr_size; if (ipv4) { diff --git a/app/test/packet_burst_generator.h b/app/test/packet_burst_generator.h index 5b3cd6c..fe992ac 100644 --- a/app/test/packet_burst_generator.h +++ b/app/test/packet_burst_generator.h @@ -47,10 +47,13 @@ extern "C" { #define IPV4_ADDR(a, b, c, d)(((a & 0xff) << 24) | ((b & 0xff) << 16) | \ ((c & 0xff) << 8) | (d & 0xff)) +#define PACKET_BURST_GEN_PKT_LEN 60 +#define PACKET_BURST_GEN_PKT_LEN_128 128 void initialize_eth_header(struct ether_hdr *eth_hdr, struct ether_addr *src_mac, - struct ether_addr *dst_mac, uint8_t vlan_enabled, uint16_t van_id); + struct ether_addr *dst_mac, uint8_t vlan_enabled, + uint16_t van_id); uint16_t initialize_udp_header(struct udp_hdr *udp_hdr, uint16_t src_port, @@ -67,8 +70,10 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst); + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs); #ifdef __cplusplus } diff --git a/app/test/test_link_bonding.c b/app/test/test_link_bonding.c index 214d2a2..d407e4f 100644 --- a/app/test/test_link_bonding.c +++ b/app/test/test_link_bonding.c @@ -1192,9 +1192,12 @@ generate_test_burst(struct rte_mbuf **pkts_burst, uint16_t burst_size, } /* Generate burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, vlan, ip_hdr, ipv4, - test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, + vlan, ip_hdr, ipv4, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN_128, 1); if (generated_burst_size != burst_size) { printf("Failed to generate packet burst"); return -1; @@ -1781,9 +1784,12 @@ test_activebackup_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; @@ -2435,8 +2441,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 1 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[0][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[0]) != burst_size[0]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[0], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[0]) return -1; initialize_eth_header(test_params->pkt_eth_hdr, @@ -2444,8 +2452,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 2 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[1][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[1]) != burst_size[1]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[1], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[1]) return -1; /* Send burst 1 on bonded port */ @@ -3263,9 +3273,12 @@ test_broadcast_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v5 2/3] app/test: measure the cost of rx/tx routines by cycle number 2014-10-24 5:57 ` [dpdk-dev] [PATCH v5 0/3] app/test: unit test to measure cycles per packet Cunming Liang 2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 1/3] app/test: allow to create packets in different sizes Cunming Liang @ 2014-10-24 5:58 ` Cunming Liang 2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition Cunming Liang 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Cunming Liang 3 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-24 5:58 UTC (permalink / raw) To: dev The unit test can be used to measure cycles per packet in different rx/tx rouines. The NIC works in loopback mode. So it doesn't require test equipment to measure throughput. As result, the unit test shows the average cycles per packet consuming. When doing the test, make sure the link is UP. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Run and wait for the result pmd_perf_autotest There's option to choose rx/tx pair, default is vector. set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without INC_VEC=y in config It supports to measure standalone rx or tx. Usage Example: Choose rx or tx standalone, default is both set_rxtx_anchor [rxtx|rxonly|txonly] It also supports to measure standalone RX burst cycles. In this way, it won't repeat re-send recevied packets. Now it measures two situations, poll before/after xmit(w or w/o desc. cache conflict) Usage Example: Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/test.h | 6 + app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ether.h | 25 + lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 6 files changed, 1071 insertions(+), 0 deletions(-) create mode 100644 app/test/test_pmd_perf.c diff --git a/app/test/Makefile b/app/test/Makefile index 6af6d76..ebfa0ba 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c SRCS-y += test_ring.c SRCS-y += test_ring_perf.c +SRCS-y += test_pmd_perf.c ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) SRCS-y += test_table.c diff --git a/app/test/commands.c b/app/test/commands.c index a9e36b1..92a17ed 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -310,12 +310,123 @@ cmdline_parse_inst_t cmd_quit = { /****************/ +struct cmd_set_rxtx_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t mode; +}; + +static void cmd_set_rxtx_parsed(void *parsed_result, struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_result *res = parsed_result; + if (test_set_rxtx_conf(res->mode) < 0) + cmdline_printf(cl, "Cannot find such mode\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, set, + "set_rxtx_mode"); + +cmdline_parse_token_string_t cmd_set_rxtx_mode = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, mode, NULL); + +cmdline_parse_inst_t cmd_set_rxtx = { + .f = cmd_set_rxtx_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx routine: " + "set_rxtx <mode>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_set, + (void *)&cmd_set_rxtx_mode, + NULL, + }, +}; + +/****************/ + +struct cmd_set_rxtx_anchor { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_anchor_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_anchor *res = parsed_result; + if (test_set_rxtx_anchor(res->type) < 0) + cmdline_printf(cl, "Cannot find such anchor\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, set, + "set_rxtx_anchor"); + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_anchor = { + .f = cmd_set_rxtx_anchor_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx anchor: " + "set_rxtx_anchor <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_anchor_set, + (void *)&cmd_set_rxtx_anchor_type, + NULL, + }, +}; + +/****************/ + +/* for stream control */ +struct cmd_set_rxtx_sc { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_sc_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_sc *res = parsed_result; + if (test_set_rxtx_sc(res->type) < 0) + cmdline_printf(cl, "Cannot find such stream control\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_sc_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, set, + "set_rxtx_sc"); + +cmdline_parse_token_string_t cmd_set_rxtx_sc_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_sc = { + .f = cmd_set_rxtx_sc_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx stream control: " + "set_rxtx_sc <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_sc_set, + (void *)&cmd_set_rxtx_sc_type, + NULL, + }, +}; + +/****************/ + + cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, (cmdline_parse_inst_t *)&cmd_dump, (cmdline_parse_inst_t *)&cmd_dump_one, (cmdline_parse_inst_t *)&cmd_set_ring, (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_set_rxtx, + (cmdline_parse_inst_t *)&cmd_set_rxtx_anchor, + (cmdline_parse_inst_t *)&cmd_set_rxtx_sc, NULL, }; diff --git a/app/test/test.h b/app/test/test.h index 98ab804..bd44a7f 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -125,6 +125,9 @@ int unit_test_suite_runner(struct unit_test_suite *suite); #define RECURSIVE_ENV_VAR "RTE_TEST_RECURSIVE" +#include <cmdline_parse.h> +#include <cmdline_parse_string.h> + extern const char *prgname; int commands_init(void); @@ -137,6 +140,9 @@ int test_pci_run; int test_mp_secondary(void); int test_ivshmem(void); +int test_set_rxtx_conf(cmdline_fixed_string_t mode); +int test_set_rxtx_anchor(cmdline_fixed_string_t type); +int test_set_rxtx_sc(cmdline_fixed_string_t type); typedef int (test_callback)(void); TAILQ_HEAD(test_commands_list, test_command); diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c new file mode 100644 index 0000000..1c1f236 --- /dev/null +++ b/app/test/test_pmd_perf.c @@ -0,0 +1,922 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + + +#include <stdio.h> +#include <inttypes.h> +#include <signal.h> +#include <unistd.h> +#include <rte_cycles.h> +#include <rte_ethdev.h> +#include <rte_byteorder.h> +#include <rte_atomic.h> +#include <rte_malloc.h> +#include "packet_burst_generator.h" +#include "test.h" + +#define NB_ETHPORTS_USED (1) +#define NB_SOCKETS (2) +#define MEMPOOL_CACHE_SIZE 250 +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MAX_PKT_BURST (32) +#define RTE_TEST_RX_DESC_DEFAULT (128) +#define RTE_TEST_TX_DESC_DEFAULT (512) +#define RTE_PORT_ALL (~(uint8_t)0x0) + +/* how long test would take at full line rate */ +#define RTE_TEST_DURATION (2) + +/* + * RX and TX Prefetch, Host, and Write-back threshold values should be + * carefully set for optimal performance. Consult the network + * controller's datasheet and supporting DPDK documentation for guidance + * on how these parameters should be set. + */ +#define RX_PTHRESH 8 /**< Default values of RX prefetch threshold reg. */ +#define RX_HTHRESH 8 /**< Default values of RX host threshold reg. */ +#define RX_WTHRESH 0 /**< Default values of RX write-back threshold reg. */ + +/* + * These default values are optimized for use with the Intel(R) 82599 10 GbE + * Controller and the DPDK ixgbe PMD. Consider using other values for other + * network controllers and/or network drivers. + */ +#define TX_PTHRESH 32 /**< Default values of TX prefetch threshold reg. */ +#define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */ +#define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */ + +#define MAX_TRAFFIC_BURST 2048 + +#define NB_MBUF RTE_MAX( \ + (unsigned)(nb_ports*nb_rx_queue*nb_rxd + \ + nb_ports*nb_lcores*MAX_PKT_BURST + \ + nb_ports*nb_tx_queue*nb_txd + \ + nb_lcores*MEMPOOL_CACHE_SIZE + \ + nb_ports*MAX_TRAFFIC_BURST), \ + (unsigned)8192) + + +static struct rte_mempool *mbufpool[NB_SOCKETS]; +/* ethernet addresses of ports */ +static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS]; + +static struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .hw_vlan_strip = 0, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 1, /* enable loopback */ +}; + +static struct rte_eth_rxconf rx_conf = { + .rx_thresh = { + .pthresh = RX_PTHRESH, + .hthresh = RX_HTHRESH, + .wthresh = RX_WTHRESH, + }, + .rx_free_thresh = 32, +}; + +static struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = TX_PTHRESH, + .hthresh = TX_HTHRESH, + .wthresh = TX_WTHRESH, + }, + .tx_free_thresh = 32, /* Use PMD default values */ + .tx_rs_thresh = 32, /* Use PMD default values */ + .txq_flags = (ETH_TXQ_FLAGS_NOMULTSEGS | + ETH_TXQ_FLAGS_NOVLANOFFL | + ETH_TXQ_FLAGS_NOXSUMSCTP | + ETH_TXQ_FLAGS_NOXSUMUDP | + ETH_TXQ_FLAGS_NOXSUMTCP) +}; + +enum { + LCORE_INVALID = 0, + LCORE_AVAIL, + LCORE_USED, +}; + +struct lcore_conf { + uint8_t status; + uint8_t socketid; + uint16_t nb_ports; + uint8_t portlist[RTE_MAX_ETHPORTS]; +} __rte_cache_aligned; + +struct lcore_conf lcore_conf[RTE_MAX_LCORE]; + +static uint64_t link_mbps; + +enum { + SC_CONTINUOUS = 0, + SC_BURST_POLL_FIRST, + SC_BURST_XMIT_FIRST, +}; + +static uint32_t sc_flag; + +/* Check the link status of all ports in up to 3s, and print them finally */ +static void +check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +{ +#define CHECK_INTERVAL 100 /* 100ms */ +#define MAX_CHECK_TIME 30 /* 3s (30 * 100ms) in total */ + uint8_t portid, count, all_ports_up, print_flag = 0; + struct rte_eth_link link; + + printf("Checking link statuses...\n"); + fflush(stdout); + for (count = 0; count <= MAX_CHECK_TIME; count++) { + all_ports_up = 1; + for (portid = 0; portid < port_num; portid++) { + if ((port_mask & (1 << portid)) == 0) + continue; + memset(&link, 0, sizeof(link)); + rte_eth_link_get_nowait(portid, &link); + /* print link status if flag set */ + if (print_flag == 1) { + if (link.link_status) { + printf("Port %d Link Up - speed %u " + "Mbps - %s\n", (uint8_t)portid, + (unsigned)link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + if (link_mbps == 0) + link_mbps = link.link_speed; + } else + printf("Port %d Link Down\n", + (uint8_t)portid); + continue; + } + /* clear all_ports_up flag if any link down */ + if (link.link_status == 0) { + all_ports_up = 0; + break; + } + } + /* after finally printing all link status, get out */ + if (print_flag == 1) + break; + + if (all_ports_up == 0) { + fflush(stdout); + rte_delay_ms(CHECK_INTERVAL); + } + + /* set the print_flag if all ports up or timeout */ + if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) + print_flag = 1; + } +} + +static void +print_ethaddr(const char *name, const struct ether_addr *eth_addr) +{ + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); +} + +static int +init_traffic(struct rte_mempool *mp, + struct rte_mbuf **pkts_burst, uint32_t burst_size) +{ + struct ether_hdr pkt_eth_hdr; + struct ipv4_hdr pkt_ipv4_hdr; + struct udp_hdr pkt_udp_hdr; + uint32_t pktlen; + static uint8_t src_mac[] = { 0x00, 0xFF, 0xAA, 0xFF, 0xAA, 0xFF }; + static uint8_t dst_mac[] = { 0x00, 0xAA, 0xFF, 0xAA, 0xFF, 0xAA }; + + + initialize_eth_header(&pkt_eth_hdr, + (struct ether_addr *)src_mac, + (struct ether_addr *)dst_mac, 0, 0); + pkt_eth_hdr.ether_type = rte_cpu_to_be_16(ETHER_TYPE_IPv4); + + pktlen = initialize_ipv4_header(&pkt_ipv4_hdr, + IPV4_ADDR(10, 0, 0, 1), + IPV4_ADDR(10, 0, 0, 2), 26); + printf("IPv4 pktlen %u\n", pktlen); + + pktlen = initialize_udp_header(&pkt_udp_hdr, 0, 0, 18); + + printf("UDP pktlen %u\n", pktlen); + + return generate_packet_burst(mp, pkts_burst, &pkt_eth_hdr, + 0, &pkt_ipv4_hdr, 1, + &pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); +} + +static int +init_lcores(void) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + lcore_conf[lcore_id].socketid = + rte_lcore_to_socket_id(lcore_id); + if (rte_lcore_is_enabled(lcore_id) == 0) { + lcore_conf[lcore_id].status = LCORE_INVALID; + continue; + } else + lcore_conf[lcore_id].status = LCORE_AVAIL; + } + return 0; +} + +static int +init_mbufpool(unsigned nb_mbuf) +{ + int socketid; + unsigned lcore_id; + char s[64]; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + socketid = rte_lcore_to_socket_id(lcore_id); + if (socketid >= NB_SOCKETS) { + rte_exit(EXIT_FAILURE, + "Socket %d of lcore %u is out of range %d\n", + socketid, lcore_id, NB_SOCKETS); + } + if (mbufpool[socketid] == NULL) { + snprintf(s, sizeof(s), "mbuf_pool_%d", socketid); + mbufpool[socketid] = + rte_mempool_create(s, nb_mbuf, MBUF_SIZE, + MEMPOOL_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + socketid, 0); + if (mbufpool[socketid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot init mbuf pool on socket %d\n", + socketid); + else + printf("Allocated mbuf pool on socket %d\n", + socketid); + } + } + return 0; +} + +static uint16_t +alloc_lcore(uint16_t socketid) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (LCORE_AVAIL != lcore_conf[lcore_id].status || + lcore_conf[lcore_id].socketid != socketid || + lcore_id == rte_get_master_lcore()) + continue; + lcore_conf[lcore_id].status = LCORE_USED; + lcore_conf[lcore_id].nb_ports = 0; + return lcore_id; + } + + return (uint16_t)-1; +} + +volatile uint64_t stop; +uint64_t count; +uint64_t drop; +uint64_t idle; + +static void +reset_count(void) +{ + count = 0; + drop = 0; + idle = 0; +} + +static void +stats_display(uint8_t port_id) +{ + struct rte_eth_stats stats; + rte_eth_stats_get(port_id, &stats); + + printf(" RX-packets: %-10"PRIu64" RX-missed: %-10"PRIu64" RX-bytes: " + "%-"PRIu64"\n", + stats.ipackets, stats.imissed, stats.ibytes); + printf(" RX-badcrc: %-10"PRIu64" RX-badlen: %-10"PRIu64" RX-errors: " + "%-"PRIu64"\n", + stats.ibadcrc, stats.ibadlen, stats.ierrors); + printf(" RX-nombuf: %-10"PRIu64"\n", + stats.rx_nombuf); + printf(" TX-packets: %-10"PRIu64" TX-errors: %-10"PRIu64" TX-bytes: " + "%-"PRIu64"\n", + stats.opackets, stats.oerrors, stats.obytes); +} + +static void +signal_handler(int signum) +{ + /* USR1 signal, stop testing */ + if (signum == SIGUSR1) { + printf("Force Stop!\n"); + stop = 1; + } + + /* USR2 signal, print stats */ + if (signum == SIGUSR2) + stats_display(0); +} + +struct rte_mbuf **tx_burst; + +uint64_t (*do_measure)(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts); + +static uint64_t +measure_rxtx(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t prev_tsc, cur_tsc; + + prev_tsc = rte_rdtsc(); + + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + cur_tsc = rte_rdtsc(); + + return cur_tsc - prev_tsc; +} + +static uint64_t +measure_rxonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + + cur_tsc = rte_rdtsc(); + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + diff_tsc += rte_rdtsc() - cur_tsc; + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +static uint64_t +measure_txonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + printf("do tx measure\n"); + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + + cur_tsc = rte_rdtsc(); + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + diff_tsc += rte_rdtsc() - cur_tsc; + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +/* main processing loop */ +static int +main_loop(__rte_unused void *args) +{ +#define PACKET_SIZE 64 +#define FRAME_GAP 12 +#define MAC_PREAMBLE 8 + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + unsigned lcore_id; + unsigned i, portid, nb_rx = 0, nb_tx = 0; + struct lcore_conf *conf; + int pkt_per_port; + uint64_t diff_tsc; + uint64_t packets_per_second, total_packets; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + pkt_per_port = MAX_TRAFFIC_BURST; + + int idx = 0; + for (i = 0; i < conf->nb_ports; i++) { + int num = pkt_per_port; + portid = conf->portlist[i]; + printf("inject %d packet to port %d\n", num, portid); + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + nb_tx = rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + num -= nb_tx; + idx += nb_tx; + } + } + printf("Total packets inject to prime ports = %u\n", idx); + + packets_per_second = (link_mbps * 1000 * 1000) / + ((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); + printf("Each port will do %"PRIu64" packets per second\n", + packets_per_second); + + total_packets = RTE_TEST_DURATION * conf->nb_ports * packets_per_second; + printf("Test will stop after at least %"PRIu64" packets received\n", + + total_packets); + + diff_tsc = do_measure(conf, pkts_burst, total_packets); + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + int nb_free = pkt_per_port; + do { /* dry out */ + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + nb_tx = 0; + while (nb_tx < nb_rx) + rte_pktmbuf_free(pkts_burst[nb_tx++]); + nb_free -= nb_rx; + } while (nb_free != 0); + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); + } + + if (count == 0) + return -1; + + printf("%"PRIu64" packet, %"PRIu64" drop, %"PRIu64" idle\n", + count, drop, idle); + printf("Result: %"PRIu64" cycles per packet\n", diff_tsc / count); + + return 0; +} + +rte_atomic64_t start; + +static inline int +poll_burst(void *args) +{ +#define MAX_IDLE (10000) + unsigned lcore_id; + struct rte_mbuf **pkts_burst; + uint64_t diff_tsc, cur_tsc; + uint16_t next[RTE_MAX_ETHPORTS]; + struct lcore_conf *conf; + uint32_t pkt_per_port = *((uint32_t *)args); + unsigned i, portid, nb_rx = 0; + uint64_t total; + uint64_t timeout = MAX_IDLE; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + total = pkt_per_port * conf->nb_ports; + printf("start to receive total expect %"PRIu64"\n", total); + + pkts_burst = (struct rte_mbuf **) + rte_calloc_socket("poll_burst", + total, sizeof(void *), + CACHE_LINE_SIZE, conf->socketid); + if (!pkts_burst) + return -1; + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + next[portid] = i * pkt_per_port; + } + + while (!rte_atomic64_read(&start)) + ; + + cur_tsc = rte_rdtsc(); + while (total) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + &pkts_burst[next[portid]], + MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + timeout--; + if (unlikely(timeout == 0)) + goto timeout; + continue; + } + next[portid] += nb_rx; + total -= nb_rx; + } + } +timeout: + diff_tsc = rte_rdtsc() - cur_tsc; + + printf("%"PRIu64" packets lost, IDLE %"PRIu64" times\n", + total, MAX_IDLE - timeout); + + /* clean up */ + total = pkt_per_port * conf->nb_ports - total; + for (i = 0; i < total; i++) + rte_pktmbuf_free(pkts_burst[i]); + + rte_free(pkts_burst); + + return diff_tsc / total; +} + +static int +exec_burst(uint32_t flags, int lcore) +{ + unsigned i, portid, nb_tx = 0; + struct lcore_conf *conf; + uint32_t pkt_per_port; + int num, idx = 0; + int diff_tsc; + + conf = &lcore_conf[lcore]; + + pkt_per_port = MAX_TRAFFIC_BURST; + num = pkt_per_port; + + rte_atomic64_init(&start); + + /* start polling thread, but not actually poll yet */ + rte_eal_remote_launch(poll_burst, + (void *)&pkt_per_port, lcore); + + /* Only when polling first */ + if (flags == SC_BURST_POLL_FIRST) + rte_atomic64_set(&start, 1); + + /* start xmit */ + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + idx += nb_tx; + } + num -= nb_tx; + } + + sleep(5); + + /* only when polling second */ + if (flags == SC_BURST_XMIT_FIRST) + rte_atomic64_set(&start, 1); + + /* wait for polling finished */ + diff_tsc = rte_eal_wait_lcore(lcore); + if (diff_tsc < 0) + return -1; + + printf("Result: %d cycles per packet\n", diff_tsc); + + return 0; +} + +static int +test_pmd_perf(void) +{ + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; + uint16_t nb_rxd = MAX_TRAFFIC_BURST; + uint16_t nb_txd = MAX_TRAFFIC_BURST; + uint16_t portid; + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; + int socketid = -1; + int ret; + + printf("Start PMD RXTX cycles cost test.\n"); + + signal(SIGUSR1, signal_handler); + signal(SIGUSR2, signal_handler); + + nb_ports = rte_eth_dev_count(); + if (nb_ports < NB_ETHPORTS_USED) { + printf("At least %u port(s) used for perf. test\n", + NB_ETHPORTS_USED); + return -1; + } + + if (nb_ports > RTE_MAX_ETHPORTS) + nb_ports = RTE_MAX_ETHPORTS; + + nb_lcores = rte_lcore_count(); + + memset(lcore_conf, 0, sizeof(lcore_conf)); + init_lcores(); + + init_mbufpool(NB_MBUF); + + if (sc_flag == SC_CONTINUOUS) { + nb_rxd = RTE_TEST_RX_DESC_DEFAULT; + nb_txd = RTE_TEST_TX_DESC_DEFAULT; + } + printf("CONFIG RXD=%d TXD=%d\n", nb_rxd, nb_txd); + + reset_count(); + num = 0; + for (portid = 0; portid < nb_ports; portid++) { + if (socketid == -1) { + socketid = rte_eth_dev_socket_id(portid); + slave_id = alloc_lcore(socketid); + if (slave_id == (uint16_t)-1) { + printf("No avail lcore to run test\n"); + return -1; + } + printf("Performance test runs on lcore %u socket %u\n", + slave_id, socketid); + } + + if (socketid != rte_eth_dev_socket_id(portid)) { + printf("Skip port %d\n", portid); + continue; + } + + /* port configure */ + ret = rte_eth_dev_configure(portid, nb_rx_queue, + nb_tx_queue, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot configure device: err=%d, port=%d\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); + printf("Port %u ", portid); + print_ethaddr("Address:", &ports_eth_addr[portid]); + printf("\n"); + + /* tx queue setup */ + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + socketid, &tx_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup: err=%d, " + "port=%d\n", ret, portid); + + /* rx queue steup */ + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, + socketid, &rx_conf, + mbufpool[socketid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," + "port=%d\n", ret, portid); + + /* Start device */ + stop = 0; + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start: err=%d, port=%d\n", + ret, portid); + + /* always eanble promiscuous */ + rte_eth_promiscuous_enable(portid); + + lcore_conf[slave_id].portlist[num++] = portid; + lcore_conf[slave_id].nb_ports++; + } + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + + if (tx_burst == NULL) { + tx_burst = (struct rte_mbuf **) + rte_calloc_socket("tx_buff", + MAX_TRAFFIC_BURST * nb_ports, + sizeof(void *), + CACHE_LINE_SIZE, socketid); + if (!tx_burst) + return -1; + } + + init_traffic(mbufpool[socketid], + tx_burst, MAX_TRAFFIC_BURST * nb_ports); + + printf("Generate %d packets @socket %d\n", + MAX_TRAFFIC_BURST * nb_ports, socketid); + + if (sc_flag == SC_CONTINUOUS) { + /* do both rxtx by default */ + if (NULL == do_measure) + do_measure = measure_rxtx; + + rte_eal_remote_launch(main_loop, NULL, slave_id); + + if (rte_eal_wait_lcore(slave_id) < 0) + return -1; + } else if (sc_flag == SC_BURST_POLL_FIRST || + sc_flag == SC_BURST_XMIT_FIRST) + exec_burst(sc_flag, slave_id); + + /* port tear down */ + for (portid = 0; portid < nb_ports; portid++) { + if (socketid != rte_eth_dev_socket_id(portid)) + continue; + + rte_eth_dev_stop(portid); + } + + return 0; +} + +int +test_set_rxtx_conf(cmdline_fixed_string_t mode) +{ + printf("mode switch to %s\n", mode); + + if (!strcmp(mode, "vector")) { + /* vector rx, tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "scalar")) { + /* bulk alloc rx, simple tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 128; + tx_conf.tx_free_thresh = 128; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "hybrid")) { + /* bulk alloc rx, vector tx + * when vec macro not define, + * using the same rx/tx as scalar + */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "full")) { + /* full feature rx,tx pair */ + tx_conf.txq_flags = 0x0; /* must condition */ + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 1; /* must condition */ + return 0; + } + + return -1; +} + +int +test_set_rxtx_anchor(cmdline_fixed_string_t type) +{ + printf("type switch to %s\n", type); + + if (!strcmp(type, "rxtx")) { + do_measure = measure_rxtx; + return 0; + } else if (!strcmp(type, "rxonly")) { + do_measure = measure_rxonly; + return 0; + } else if (!strcmp(type, "txonly")) { + do_measure = measure_txonly; + return 0; + } + + return -1; +} + +int +test_set_rxtx_sc(cmdline_fixed_string_t type) +{ + printf("stream control switch to %s\n", type); + + if (!strcmp(type, "continuous")) { + sc_flag = SC_CONTINUOUS; + return 0; + } else if (!strcmp(type, "poll_before_xmit")) { + sc_flag = SC_BURST_POLL_FIRST; + return 0; + } else if (!strcmp(type, "poll_after_xmit")) { + sc_flag = SC_BURST_XMIT_FIRST; + return 0; + } + + return -1; +} + +static struct test_command pmd_perf_cmd = { + .command = "pmd_perf_autotest", + .callback = test_pmd_perf, +}; +REGISTER_TEST_COMMAND(pmd_perf_cmd); diff --git a/lib/librte_ether/rte_ether.h b/lib/librte_ether/rte_ether.h index 2e08f23..b18be7b 100644 --- a/lib/librte_ether/rte_ether.h +++ b/lib/librte_ether/rte_ether.h @@ -45,6 +45,7 @@ extern "C" { #endif #include <stdint.h> +#include <stdio.h> #include <rte_memcpy.h> #include <rte_random.h> @@ -266,6 +267,30 @@ static inline void ether_addr_copy(const struct ether_addr *ea_from, #endif } +#define ETHER_ADDR_FMT_SIZE 18 +/** + * Format 48bits Ethernet address in pattern xx:xx:xx:xx:xx:xx. + * + * @param buf + * A pointer to buffer contains the formatted MAC address. + * @param size + * The format buffer size. + * @param ea_to + * A pointer to a ether_addr structure. + */ +static inline void +ether_format_addr(char *buf, uint16_t size, + const struct ether_addr *eth_addr) +{ + snprintf(buf, size, "%02X:%02X:%02X:%02X:%02X:%02X", + eth_addr->addr_bytes[0], + eth_addr->addr_bytes[1], + eth_addr->addr_bytes[2], + eth_addr->addr_bytes[3], + eth_addr->addr_bytes[4], + eth_addr->addr_bytes[5]); +} + /** * Ethernet header: Contains the destination address, source address * and frame type. diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c index c5e4b71..e85d827 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c +++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c @@ -1600,6 +1600,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev) ixgbe_dev_clear_queues(dev); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + /* Clear recorded link status */ memset(&link, 0, sizeof(link)); rte_ixgbe_dev_atomic_write_link_status(dev, &link); @@ -2888,6 +2891,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) */ ixgbevf_set_vfta_all(dev,0); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + ixgbe_dev_clear_queues(dev); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition 2014-10-24 5:57 ` [dpdk-dev] [PATCH v5 0/3] app/test: unit test to measure cycles per packet Cunming Liang 2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 1/3] app/test: allow to create packets in different sizes Cunming Liang 2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang @ 2014-10-24 5:58 ` Cunming Liang 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Cunming Liang 3 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-24 5:58 UTC (permalink / raw) To: dev Per definition, rte_eth_rx_burst/rte_eth_tx_burst/rte_eth_rx_queue_count returns the packet number. When RTE_LIBRTE_ETHDEV_DEBUG turns on, retval of FUNC_PTR_OR_ERR_RTE was set to -ENOTSUP. It makes confusing. The patch always return 0 no matter no packet or there's error. Meanwhile set errno in such kind of checking. Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- lib/librte_ether/rte_ethdev.c | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 50f10d9..6675f28 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -81,12 +81,14 @@ /* Macros for checking for restricting functions to primary instance only */ #define PROC_PRIMARY_OR_ERR_RET(retval) do { \ if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \ + rte_errno = -E_RTE_SECONDARY; \ PMD_DEBUG_TRACE("Cannot run in secondary processes\n"); \ return (retval); \ } \ } while(0) #define PROC_PRIMARY_OR_RET() do { \ if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \ + rte_errno = -E_RTE_SECONDARY; \ PMD_DEBUG_TRACE("Cannot run in secondary processes\n"); \ return; \ } \ @@ -95,12 +97,14 @@ /* Macros to check for invlaid function pointers in dev_ops structure */ #define FUNC_PTR_OR_ERR_RET(func, retval) do { \ if ((func) == NULL) { \ + rte_errno = -ENOTSUP; \ PMD_DEBUG_TRACE("Function not supported\n"); \ return (retval); \ } \ } while(0) #define FUNC_PTR_OR_RET(func) do { \ if ((func) == NULL) { \ + rte_errno = -ENOTSUP; \ PMD_DEBUG_TRACE("Function not supported\n"); \ return; \ } \ @@ -2530,7 +2534,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, return 0; } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); return 0; @@ -2551,7 +2555,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); if (queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", queue_id); return 0; @@ -2570,7 +2574,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) return 0; } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet 2014-10-24 5:57 ` [dpdk-dev] [PATCH v5 0/3] app/test: unit test to measure cycles per packet Cunming Liang ` (2 preceding siblings ...) 2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition Cunming Liang @ 2014-10-27 1:20 ` Cunming Liang 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 1/3] app/test: allow to create packets in different sizes Cunming Liang ` (5 more replies) 3 siblings, 6 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-27 1:20 UTC (permalink / raw) To: dev v6 update: # leave FUNC_PTR_OR_*_RET unmodified v5 update: # fix the confusing of retval in some API of rte_ethdev v4 ignore v3 update: # Codes refine according to the feedback. 1. add ether_format_addr to rte_ether.h 2. fix typo in code comments. 3. %lu to %PRIu64, fixing 32-bit targets compilation err # merge 2 small incremental patches to the first one. The whole unit test as a single patch in [PATCH v3 2/2] # rebase code to the latest master v2 update: Rebase code to the latest master branch. It provides unit test to measure cycles/packet in NIC loopback mode. It simply gives the average cycles of IO used per packet without test equipment. When doing the test, make sure the link is UP. There's two stream control mode support, one is continues, another is burst. The former continues to forward the injected packets until reaching a certain amount of number. The latter one stop when all the injected packets are received. In burst stream, now measure two situations, with or without desc. cache conflict. By default, it runs in continues stream mode to measure the whole rxtx. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] 3. If choose continuous stream, there are another two options can configure 3.1 choose rx/tx pair, default is vector set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without INC_VEC=y in config 3.2 choose the area of masurement, default is rxtx set_rxtx_anchor [rxtx|rxonly|txonly] 4. Run and wait for the result pmd_perf_autotest For who simply just want to see how much cycles cost per packet. Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. Nothing else needs to configure. Using other options when you understand and what to measures more. BTW, [1/3] is the same patch as below one. http://dpdk.org/dev/patchwork/patch/817 *** BLURB HERE *** Cunming Liang (3): app/test: allow to create packets in different sizes app/test: measure the cost of rx/tx routines by cycle number ethdev: fix wrong error return refere to API definition app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/packet_burst_generator.c | 26 +- app/test/packet_burst_generator.h | 11 +- app/test/test.h | 6 + app/test/test_link_bonding.c | 39 +- app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.c | 6 +- lib/librte_ether/rte_ether.h | 25 + lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 10 files changed, 1117 insertions(+), 36 deletions(-) create mode 100644 app/test/test_pmd_perf.c -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v6 1/3] app/test: allow to create packets in different sizes 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Cunming Liang @ 2014-10-27 1:20 ` Cunming Liang 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang ` (4 subsequent siblings) 5 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-10-27 1:20 UTC (permalink / raw) To: dev adding support to allow packet burst generator to create packets in differenct sizes Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- app/test/packet_burst_generator.c | 26 ++++++++---------------- app/test/packet_burst_generator.h | 11 +++++++-- app/test/test_link_bonding.c | 39 ++++++++++++++++++++++++------------ 3 files changed, 43 insertions(+), 33 deletions(-) diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c index 9e747a4..017139b 100644 --- a/app/test/packet_burst_generator.c +++ b/app/test/packet_burst_generator.c @@ -191,20 +191,12 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, */ #define RTE_MAX_SEGS_PER_PKT 255 /**< pkt.nb_segs is a 8-bit unsigned char. */ -#define TXONLY_DEF_PACKET_LEN 64 -#define TXONLY_DEF_PACKET_LEN_128 128 - -uint16_t tx_pkt_length = TXONLY_DEF_PACKET_LEN; -uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT] = { - TXONLY_DEF_PACKET_LEN_128, -}; - -uint8_t tx_pkt_nb_segs = 1; - int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst) + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs) { int i, nb_pkt = 0; size_t eth_hdr_size; @@ -221,9 +213,9 @@ nomore_mbuf: break; } - pkt->data_len = tx_pkt_seg_lengths[0]; + pkt->data_len = pkt_len; pkt_seg = pkt; - for (i = 1; i < tx_pkt_nb_segs; i++) { + for (i = 1; i < nb_pkt_segs; i++) { pkt_seg->next = rte_pktmbuf_alloc(mp); if (pkt_seg->next == NULL) { pkt->nb_segs = i; @@ -231,7 +223,7 @@ nomore_mbuf: goto nomore_mbuf; } pkt_seg = pkt_seg->next; - pkt_seg->data_len = tx_pkt_seg_lengths[i]; + pkt_seg->data_len = pkt_len; } pkt_seg->next = NULL; /* Last segment of packet. */ @@ -259,8 +251,8 @@ nomore_mbuf: * Complete first mbuf of packet and append it to the * burst of packets to be transmitted. */ - pkt->nb_segs = tx_pkt_nb_segs; - pkt->pkt_len = tx_pkt_length; + pkt->nb_segs = nb_pkt_segs; + pkt->pkt_len = pkt_len; pkt->l2_len = eth_hdr_size; if (ipv4) { diff --git a/app/test/packet_burst_generator.h b/app/test/packet_burst_generator.h index 5b3cd6c..fe992ac 100644 --- a/app/test/packet_burst_generator.h +++ b/app/test/packet_burst_generator.h @@ -47,10 +47,13 @@ extern "C" { #define IPV4_ADDR(a, b, c, d)(((a & 0xff) << 24) | ((b & 0xff) << 16) | \ ((c & 0xff) << 8) | (d & 0xff)) +#define PACKET_BURST_GEN_PKT_LEN 60 +#define PACKET_BURST_GEN_PKT_LEN_128 128 void initialize_eth_header(struct ether_hdr *eth_hdr, struct ether_addr *src_mac, - struct ether_addr *dst_mac, uint8_t vlan_enabled, uint16_t van_id); + struct ether_addr *dst_mac, uint8_t vlan_enabled, + uint16_t van_id); uint16_t initialize_udp_header(struct udp_hdr *udp_hdr, uint16_t src_port, @@ -67,8 +70,10 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst); + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs); #ifdef __cplusplus } diff --git a/app/test/test_link_bonding.c b/app/test/test_link_bonding.c index 214d2a2..d407e4f 100644 --- a/app/test/test_link_bonding.c +++ b/app/test/test_link_bonding.c @@ -1192,9 +1192,12 @@ generate_test_burst(struct rte_mbuf **pkts_burst, uint16_t burst_size, } /* Generate burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, vlan, ip_hdr, ipv4, - test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, + vlan, ip_hdr, ipv4, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN_128, 1); if (generated_burst_size != burst_size) { printf("Failed to generate packet burst"); return -1; @@ -1781,9 +1784,12 @@ test_activebackup_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; @@ -2435,8 +2441,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 1 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[0][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[0]) != burst_size[0]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[0], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[0]) return -1; initialize_eth_header(test_params->pkt_eth_hdr, @@ -2444,8 +2452,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 2 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[1][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[1]) != burst_size[1]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[1], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[1]) return -1; /* Send burst 1 on bonded port */ @@ -3263,9 +3273,12 @@ test_broadcast_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v6 2/3] app/test: measure the cost of rx/tx routines by cycle number 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Cunming Liang 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 1/3] app/test: allow to create packets in different sizes Cunming Liang @ 2014-10-27 1:20 ` Cunming Liang 2014-11-11 23:28 ` Thomas Monjalon 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 3/3] ethdev: fix wrong error return refere to API definition Cunming Liang ` (3 subsequent siblings) 5 siblings, 1 reply; 65+ messages in thread From: Cunming Liang @ 2014-10-27 1:20 UTC (permalink / raw) To: dev The unit test can be used to measure cycles per packet in different rx/tx rouines. The NIC works in loopback mode. So it doesn't require test equipment to measure throughput. As result, the unit test shows the average cycles per packet consuming. When doing the test, make sure the link is UP. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Run and wait for the result pmd_perf_autotest There's option to choose rx/tx pair, default is vector. set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without INC_VEC=y in config It supports to measure standalone rx or tx. Usage Example: Choose rx or tx standalone, default is both set_rxtx_anchor [rxtx|rxonly|txonly] It also supports to measure standalone RX burst cycles. In this way, it won't repeat re-send recevied packets. Now it measures two situations, poll before/after xmit(w or w/o desc. cache conflict) Usage Example: Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] Signed-off-by: Cunming Liang <cunming.liang@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/test.h | 6 + app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ether.h | 25 + lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 6 files changed, 1071 insertions(+), 0 deletions(-) create mode 100644 app/test/test_pmd_perf.c diff --git a/app/test/Makefile b/app/test/Makefile index 6af6d76..ebfa0ba 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c SRCS-y += test_ring.c SRCS-y += test_ring_perf.c +SRCS-y += test_pmd_perf.c ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) SRCS-y += test_table.c diff --git a/app/test/commands.c b/app/test/commands.c index a9e36b1..92a17ed 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -310,12 +310,123 @@ cmdline_parse_inst_t cmd_quit = { /****************/ +struct cmd_set_rxtx_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t mode; +}; + +static void cmd_set_rxtx_parsed(void *parsed_result, struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_result *res = parsed_result; + if (test_set_rxtx_conf(res->mode) < 0) + cmdline_printf(cl, "Cannot find such mode\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, set, + "set_rxtx_mode"); + +cmdline_parse_token_string_t cmd_set_rxtx_mode = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, mode, NULL); + +cmdline_parse_inst_t cmd_set_rxtx = { + .f = cmd_set_rxtx_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx routine: " + "set_rxtx <mode>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_set, + (void *)&cmd_set_rxtx_mode, + NULL, + }, +}; + +/****************/ + +struct cmd_set_rxtx_anchor { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_anchor_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_anchor *res = parsed_result; + if (test_set_rxtx_anchor(res->type) < 0) + cmdline_printf(cl, "Cannot find such anchor\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, set, + "set_rxtx_anchor"); + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_anchor = { + .f = cmd_set_rxtx_anchor_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx anchor: " + "set_rxtx_anchor <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_anchor_set, + (void *)&cmd_set_rxtx_anchor_type, + NULL, + }, +}; + +/****************/ + +/* for stream control */ +struct cmd_set_rxtx_sc { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_sc_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_sc *res = parsed_result; + if (test_set_rxtx_sc(res->type) < 0) + cmdline_printf(cl, "Cannot find such stream control\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_sc_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, set, + "set_rxtx_sc"); + +cmdline_parse_token_string_t cmd_set_rxtx_sc_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_sc = { + .f = cmd_set_rxtx_sc_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx stream control: " + "set_rxtx_sc <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_sc_set, + (void *)&cmd_set_rxtx_sc_type, + NULL, + }, +}; + +/****************/ + + cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, (cmdline_parse_inst_t *)&cmd_dump, (cmdline_parse_inst_t *)&cmd_dump_one, (cmdline_parse_inst_t *)&cmd_set_ring, (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_set_rxtx, + (cmdline_parse_inst_t *)&cmd_set_rxtx_anchor, + (cmdline_parse_inst_t *)&cmd_set_rxtx_sc, NULL, }; diff --git a/app/test/test.h b/app/test/test.h index 98ab804..bd44a7f 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -125,6 +125,9 @@ int unit_test_suite_runner(struct unit_test_suite *suite); #define RECURSIVE_ENV_VAR "RTE_TEST_RECURSIVE" +#include <cmdline_parse.h> +#include <cmdline_parse_string.h> + extern const char *prgname; int commands_init(void); @@ -137,6 +140,9 @@ int test_pci_run; int test_mp_secondary(void); int test_ivshmem(void); +int test_set_rxtx_conf(cmdline_fixed_string_t mode); +int test_set_rxtx_anchor(cmdline_fixed_string_t type); +int test_set_rxtx_sc(cmdline_fixed_string_t type); typedef int (test_callback)(void); TAILQ_HEAD(test_commands_list, test_command); diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c new file mode 100644 index 0000000..1c1f236 --- /dev/null +++ b/app/test/test_pmd_perf.c @@ -0,0 +1,922 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + + +#include <stdio.h> +#include <inttypes.h> +#include <signal.h> +#include <unistd.h> +#include <rte_cycles.h> +#include <rte_ethdev.h> +#include <rte_byteorder.h> +#include <rte_atomic.h> +#include <rte_malloc.h> +#include "packet_burst_generator.h" +#include "test.h" + +#define NB_ETHPORTS_USED (1) +#define NB_SOCKETS (2) +#define MEMPOOL_CACHE_SIZE 250 +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MAX_PKT_BURST (32) +#define RTE_TEST_RX_DESC_DEFAULT (128) +#define RTE_TEST_TX_DESC_DEFAULT (512) +#define RTE_PORT_ALL (~(uint8_t)0x0) + +/* how long test would take at full line rate */ +#define RTE_TEST_DURATION (2) + +/* + * RX and TX Prefetch, Host, and Write-back threshold values should be + * carefully set for optimal performance. Consult the network + * controller's datasheet and supporting DPDK documentation for guidance + * on how these parameters should be set. + */ +#define RX_PTHRESH 8 /**< Default values of RX prefetch threshold reg. */ +#define RX_HTHRESH 8 /**< Default values of RX host threshold reg. */ +#define RX_WTHRESH 0 /**< Default values of RX write-back threshold reg. */ + +/* + * These default values are optimized for use with the Intel(R) 82599 10 GbE + * Controller and the DPDK ixgbe PMD. Consider using other values for other + * network controllers and/or network drivers. + */ +#define TX_PTHRESH 32 /**< Default values of TX prefetch threshold reg. */ +#define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */ +#define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */ + +#define MAX_TRAFFIC_BURST 2048 + +#define NB_MBUF RTE_MAX( \ + (unsigned)(nb_ports*nb_rx_queue*nb_rxd + \ + nb_ports*nb_lcores*MAX_PKT_BURST + \ + nb_ports*nb_tx_queue*nb_txd + \ + nb_lcores*MEMPOOL_CACHE_SIZE + \ + nb_ports*MAX_TRAFFIC_BURST), \ + (unsigned)8192) + + +static struct rte_mempool *mbufpool[NB_SOCKETS]; +/* ethernet addresses of ports */ +static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS]; + +static struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .hw_vlan_strip = 0, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 1, /* enable loopback */ +}; + +static struct rte_eth_rxconf rx_conf = { + .rx_thresh = { + .pthresh = RX_PTHRESH, + .hthresh = RX_HTHRESH, + .wthresh = RX_WTHRESH, + }, + .rx_free_thresh = 32, +}; + +static struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = TX_PTHRESH, + .hthresh = TX_HTHRESH, + .wthresh = TX_WTHRESH, + }, + .tx_free_thresh = 32, /* Use PMD default values */ + .tx_rs_thresh = 32, /* Use PMD default values */ + .txq_flags = (ETH_TXQ_FLAGS_NOMULTSEGS | + ETH_TXQ_FLAGS_NOVLANOFFL | + ETH_TXQ_FLAGS_NOXSUMSCTP | + ETH_TXQ_FLAGS_NOXSUMUDP | + ETH_TXQ_FLAGS_NOXSUMTCP) +}; + +enum { + LCORE_INVALID = 0, + LCORE_AVAIL, + LCORE_USED, +}; + +struct lcore_conf { + uint8_t status; + uint8_t socketid; + uint16_t nb_ports; + uint8_t portlist[RTE_MAX_ETHPORTS]; +} __rte_cache_aligned; + +struct lcore_conf lcore_conf[RTE_MAX_LCORE]; + +static uint64_t link_mbps; + +enum { + SC_CONTINUOUS = 0, + SC_BURST_POLL_FIRST, + SC_BURST_XMIT_FIRST, +}; + +static uint32_t sc_flag; + +/* Check the link status of all ports in up to 3s, and print them finally */ +static void +check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +{ +#define CHECK_INTERVAL 100 /* 100ms */ +#define MAX_CHECK_TIME 30 /* 3s (30 * 100ms) in total */ + uint8_t portid, count, all_ports_up, print_flag = 0; + struct rte_eth_link link; + + printf("Checking link statuses...\n"); + fflush(stdout); + for (count = 0; count <= MAX_CHECK_TIME; count++) { + all_ports_up = 1; + for (portid = 0; portid < port_num; portid++) { + if ((port_mask & (1 << portid)) == 0) + continue; + memset(&link, 0, sizeof(link)); + rte_eth_link_get_nowait(portid, &link); + /* print link status if flag set */ + if (print_flag == 1) { + if (link.link_status) { + printf("Port %d Link Up - speed %u " + "Mbps - %s\n", (uint8_t)portid, + (unsigned)link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + if (link_mbps == 0) + link_mbps = link.link_speed; + } else + printf("Port %d Link Down\n", + (uint8_t)portid); + continue; + } + /* clear all_ports_up flag if any link down */ + if (link.link_status == 0) { + all_ports_up = 0; + break; + } + } + /* after finally printing all link status, get out */ + if (print_flag == 1) + break; + + if (all_ports_up == 0) { + fflush(stdout); + rte_delay_ms(CHECK_INTERVAL); + } + + /* set the print_flag if all ports up or timeout */ + if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) + print_flag = 1; + } +} + +static void +print_ethaddr(const char *name, const struct ether_addr *eth_addr) +{ + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); +} + +static int +init_traffic(struct rte_mempool *mp, + struct rte_mbuf **pkts_burst, uint32_t burst_size) +{ + struct ether_hdr pkt_eth_hdr; + struct ipv4_hdr pkt_ipv4_hdr; + struct udp_hdr pkt_udp_hdr; + uint32_t pktlen; + static uint8_t src_mac[] = { 0x00, 0xFF, 0xAA, 0xFF, 0xAA, 0xFF }; + static uint8_t dst_mac[] = { 0x00, 0xAA, 0xFF, 0xAA, 0xFF, 0xAA }; + + + initialize_eth_header(&pkt_eth_hdr, + (struct ether_addr *)src_mac, + (struct ether_addr *)dst_mac, 0, 0); + pkt_eth_hdr.ether_type = rte_cpu_to_be_16(ETHER_TYPE_IPv4); + + pktlen = initialize_ipv4_header(&pkt_ipv4_hdr, + IPV4_ADDR(10, 0, 0, 1), + IPV4_ADDR(10, 0, 0, 2), 26); + printf("IPv4 pktlen %u\n", pktlen); + + pktlen = initialize_udp_header(&pkt_udp_hdr, 0, 0, 18); + + printf("UDP pktlen %u\n", pktlen); + + return generate_packet_burst(mp, pkts_burst, &pkt_eth_hdr, + 0, &pkt_ipv4_hdr, 1, + &pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); +} + +static int +init_lcores(void) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + lcore_conf[lcore_id].socketid = + rte_lcore_to_socket_id(lcore_id); + if (rte_lcore_is_enabled(lcore_id) == 0) { + lcore_conf[lcore_id].status = LCORE_INVALID; + continue; + } else + lcore_conf[lcore_id].status = LCORE_AVAIL; + } + return 0; +} + +static int +init_mbufpool(unsigned nb_mbuf) +{ + int socketid; + unsigned lcore_id; + char s[64]; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + socketid = rte_lcore_to_socket_id(lcore_id); + if (socketid >= NB_SOCKETS) { + rte_exit(EXIT_FAILURE, + "Socket %d of lcore %u is out of range %d\n", + socketid, lcore_id, NB_SOCKETS); + } + if (mbufpool[socketid] == NULL) { + snprintf(s, sizeof(s), "mbuf_pool_%d", socketid); + mbufpool[socketid] = + rte_mempool_create(s, nb_mbuf, MBUF_SIZE, + MEMPOOL_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + socketid, 0); + if (mbufpool[socketid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot init mbuf pool on socket %d\n", + socketid); + else + printf("Allocated mbuf pool on socket %d\n", + socketid); + } + } + return 0; +} + +static uint16_t +alloc_lcore(uint16_t socketid) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (LCORE_AVAIL != lcore_conf[lcore_id].status || + lcore_conf[lcore_id].socketid != socketid || + lcore_id == rte_get_master_lcore()) + continue; + lcore_conf[lcore_id].status = LCORE_USED; + lcore_conf[lcore_id].nb_ports = 0; + return lcore_id; + } + + return (uint16_t)-1; +} + +volatile uint64_t stop; +uint64_t count; +uint64_t drop; +uint64_t idle; + +static void +reset_count(void) +{ + count = 0; + drop = 0; + idle = 0; +} + +static void +stats_display(uint8_t port_id) +{ + struct rte_eth_stats stats; + rte_eth_stats_get(port_id, &stats); + + printf(" RX-packets: %-10"PRIu64" RX-missed: %-10"PRIu64" RX-bytes: " + "%-"PRIu64"\n", + stats.ipackets, stats.imissed, stats.ibytes); + printf(" RX-badcrc: %-10"PRIu64" RX-badlen: %-10"PRIu64" RX-errors: " + "%-"PRIu64"\n", + stats.ibadcrc, stats.ibadlen, stats.ierrors); + printf(" RX-nombuf: %-10"PRIu64"\n", + stats.rx_nombuf); + printf(" TX-packets: %-10"PRIu64" TX-errors: %-10"PRIu64" TX-bytes: " + "%-"PRIu64"\n", + stats.opackets, stats.oerrors, stats.obytes); +} + +static void +signal_handler(int signum) +{ + /* USR1 signal, stop testing */ + if (signum == SIGUSR1) { + printf("Force Stop!\n"); + stop = 1; + } + + /* USR2 signal, print stats */ + if (signum == SIGUSR2) + stats_display(0); +} + +struct rte_mbuf **tx_burst; + +uint64_t (*do_measure)(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts); + +static uint64_t +measure_rxtx(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t prev_tsc, cur_tsc; + + prev_tsc = rte_rdtsc(); + + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + cur_tsc = rte_rdtsc(); + + return cur_tsc - prev_tsc; +} + +static uint64_t +measure_rxonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + + cur_tsc = rte_rdtsc(); + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + diff_tsc += rte_rdtsc() - cur_tsc; + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +static uint64_t +measure_txonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + printf("do tx measure\n"); + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + + cur_tsc = rte_rdtsc(); + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + diff_tsc += rte_rdtsc() - cur_tsc; + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +/* main processing loop */ +static int +main_loop(__rte_unused void *args) +{ +#define PACKET_SIZE 64 +#define FRAME_GAP 12 +#define MAC_PREAMBLE 8 + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + unsigned lcore_id; + unsigned i, portid, nb_rx = 0, nb_tx = 0; + struct lcore_conf *conf; + int pkt_per_port; + uint64_t diff_tsc; + uint64_t packets_per_second, total_packets; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + pkt_per_port = MAX_TRAFFIC_BURST; + + int idx = 0; + for (i = 0; i < conf->nb_ports; i++) { + int num = pkt_per_port; + portid = conf->portlist[i]; + printf("inject %d packet to port %d\n", num, portid); + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + nb_tx = rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + num -= nb_tx; + idx += nb_tx; + } + } + printf("Total packets inject to prime ports = %u\n", idx); + + packets_per_second = (link_mbps * 1000 * 1000) / + ((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); + printf("Each port will do %"PRIu64" packets per second\n", + packets_per_second); + + total_packets = RTE_TEST_DURATION * conf->nb_ports * packets_per_second; + printf("Test will stop after at least %"PRIu64" packets received\n", + + total_packets); + + diff_tsc = do_measure(conf, pkts_burst, total_packets); + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + int nb_free = pkt_per_port; + do { /* dry out */ + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + nb_tx = 0; + while (nb_tx < nb_rx) + rte_pktmbuf_free(pkts_burst[nb_tx++]); + nb_free -= nb_rx; + } while (nb_free != 0); + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); + } + + if (count == 0) + return -1; + + printf("%"PRIu64" packet, %"PRIu64" drop, %"PRIu64" idle\n", + count, drop, idle); + printf("Result: %"PRIu64" cycles per packet\n", diff_tsc / count); + + return 0; +} + +rte_atomic64_t start; + +static inline int +poll_burst(void *args) +{ +#define MAX_IDLE (10000) + unsigned lcore_id; + struct rte_mbuf **pkts_burst; + uint64_t diff_tsc, cur_tsc; + uint16_t next[RTE_MAX_ETHPORTS]; + struct lcore_conf *conf; + uint32_t pkt_per_port = *((uint32_t *)args); + unsigned i, portid, nb_rx = 0; + uint64_t total; + uint64_t timeout = MAX_IDLE; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + total = pkt_per_port * conf->nb_ports; + printf("start to receive total expect %"PRIu64"\n", total); + + pkts_burst = (struct rte_mbuf **) + rte_calloc_socket("poll_burst", + total, sizeof(void *), + CACHE_LINE_SIZE, conf->socketid); + if (!pkts_burst) + return -1; + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + next[portid] = i * pkt_per_port; + } + + while (!rte_atomic64_read(&start)) + ; + + cur_tsc = rte_rdtsc(); + while (total) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + &pkts_burst[next[portid]], + MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + timeout--; + if (unlikely(timeout == 0)) + goto timeout; + continue; + } + next[portid] += nb_rx; + total -= nb_rx; + } + } +timeout: + diff_tsc = rte_rdtsc() - cur_tsc; + + printf("%"PRIu64" packets lost, IDLE %"PRIu64" times\n", + total, MAX_IDLE - timeout); + + /* clean up */ + total = pkt_per_port * conf->nb_ports - total; + for (i = 0; i < total; i++) + rte_pktmbuf_free(pkts_burst[i]); + + rte_free(pkts_burst); + + return diff_tsc / total; +} + +static int +exec_burst(uint32_t flags, int lcore) +{ + unsigned i, portid, nb_tx = 0; + struct lcore_conf *conf; + uint32_t pkt_per_port; + int num, idx = 0; + int diff_tsc; + + conf = &lcore_conf[lcore]; + + pkt_per_port = MAX_TRAFFIC_BURST; + num = pkt_per_port; + + rte_atomic64_init(&start); + + /* start polling thread, but not actually poll yet */ + rte_eal_remote_launch(poll_burst, + (void *)&pkt_per_port, lcore); + + /* Only when polling first */ + if (flags == SC_BURST_POLL_FIRST) + rte_atomic64_set(&start, 1); + + /* start xmit */ + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + idx += nb_tx; + } + num -= nb_tx; + } + + sleep(5); + + /* only when polling second */ + if (flags == SC_BURST_XMIT_FIRST) + rte_atomic64_set(&start, 1); + + /* wait for polling finished */ + diff_tsc = rte_eal_wait_lcore(lcore); + if (diff_tsc < 0) + return -1; + + printf("Result: %d cycles per packet\n", diff_tsc); + + return 0; +} + +static int +test_pmd_perf(void) +{ + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; + uint16_t nb_rxd = MAX_TRAFFIC_BURST; + uint16_t nb_txd = MAX_TRAFFIC_BURST; + uint16_t portid; + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; + int socketid = -1; + int ret; + + printf("Start PMD RXTX cycles cost test.\n"); + + signal(SIGUSR1, signal_handler); + signal(SIGUSR2, signal_handler); + + nb_ports = rte_eth_dev_count(); + if (nb_ports < NB_ETHPORTS_USED) { + printf("At least %u port(s) used for perf. test\n", + NB_ETHPORTS_USED); + return -1; + } + + if (nb_ports > RTE_MAX_ETHPORTS) + nb_ports = RTE_MAX_ETHPORTS; + + nb_lcores = rte_lcore_count(); + + memset(lcore_conf, 0, sizeof(lcore_conf)); + init_lcores(); + + init_mbufpool(NB_MBUF); + + if (sc_flag == SC_CONTINUOUS) { + nb_rxd = RTE_TEST_RX_DESC_DEFAULT; + nb_txd = RTE_TEST_TX_DESC_DEFAULT; + } + printf("CONFIG RXD=%d TXD=%d\n", nb_rxd, nb_txd); + + reset_count(); + num = 0; + for (portid = 0; portid < nb_ports; portid++) { + if (socketid == -1) { + socketid = rte_eth_dev_socket_id(portid); + slave_id = alloc_lcore(socketid); + if (slave_id == (uint16_t)-1) { + printf("No avail lcore to run test\n"); + return -1; + } + printf("Performance test runs on lcore %u socket %u\n", + slave_id, socketid); + } + + if (socketid != rte_eth_dev_socket_id(portid)) { + printf("Skip port %d\n", portid); + continue; + } + + /* port configure */ + ret = rte_eth_dev_configure(portid, nb_rx_queue, + nb_tx_queue, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot configure device: err=%d, port=%d\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); + printf("Port %u ", portid); + print_ethaddr("Address:", &ports_eth_addr[portid]); + printf("\n"); + + /* tx queue setup */ + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + socketid, &tx_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup: err=%d, " + "port=%d\n", ret, portid); + + /* rx queue steup */ + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, + socketid, &rx_conf, + mbufpool[socketid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," + "port=%d\n", ret, portid); + + /* Start device */ + stop = 0; + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start: err=%d, port=%d\n", + ret, portid); + + /* always eanble promiscuous */ + rte_eth_promiscuous_enable(portid); + + lcore_conf[slave_id].portlist[num++] = portid; + lcore_conf[slave_id].nb_ports++; + } + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + + if (tx_burst == NULL) { + tx_burst = (struct rte_mbuf **) + rte_calloc_socket("tx_buff", + MAX_TRAFFIC_BURST * nb_ports, + sizeof(void *), + CACHE_LINE_SIZE, socketid); + if (!tx_burst) + return -1; + } + + init_traffic(mbufpool[socketid], + tx_burst, MAX_TRAFFIC_BURST * nb_ports); + + printf("Generate %d packets @socket %d\n", + MAX_TRAFFIC_BURST * nb_ports, socketid); + + if (sc_flag == SC_CONTINUOUS) { + /* do both rxtx by default */ + if (NULL == do_measure) + do_measure = measure_rxtx; + + rte_eal_remote_launch(main_loop, NULL, slave_id); + + if (rte_eal_wait_lcore(slave_id) < 0) + return -1; + } else if (sc_flag == SC_BURST_POLL_FIRST || + sc_flag == SC_BURST_XMIT_FIRST) + exec_burst(sc_flag, slave_id); + + /* port tear down */ + for (portid = 0; portid < nb_ports; portid++) { + if (socketid != rte_eth_dev_socket_id(portid)) + continue; + + rte_eth_dev_stop(portid); + } + + return 0; +} + +int +test_set_rxtx_conf(cmdline_fixed_string_t mode) +{ + printf("mode switch to %s\n", mode); + + if (!strcmp(mode, "vector")) { + /* vector rx, tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "scalar")) { + /* bulk alloc rx, simple tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 128; + tx_conf.tx_free_thresh = 128; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "hybrid")) { + /* bulk alloc rx, vector tx + * when vec macro not define, + * using the same rx/tx as scalar + */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "full")) { + /* full feature rx,tx pair */ + tx_conf.txq_flags = 0x0; /* must condition */ + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 1; /* must condition */ + return 0; + } + + return -1; +} + +int +test_set_rxtx_anchor(cmdline_fixed_string_t type) +{ + printf("type switch to %s\n", type); + + if (!strcmp(type, "rxtx")) { + do_measure = measure_rxtx; + return 0; + } else if (!strcmp(type, "rxonly")) { + do_measure = measure_rxonly; + return 0; + } else if (!strcmp(type, "txonly")) { + do_measure = measure_txonly; + return 0; + } + + return -1; +} + +int +test_set_rxtx_sc(cmdline_fixed_string_t type) +{ + printf("stream control switch to %s\n", type); + + if (!strcmp(type, "continuous")) { + sc_flag = SC_CONTINUOUS; + return 0; + } else if (!strcmp(type, "poll_before_xmit")) { + sc_flag = SC_BURST_POLL_FIRST; + return 0; + } else if (!strcmp(type, "poll_after_xmit")) { + sc_flag = SC_BURST_XMIT_FIRST; + return 0; + } + + return -1; +} + +static struct test_command pmd_perf_cmd = { + .command = "pmd_perf_autotest", + .callback = test_pmd_perf, +}; +REGISTER_TEST_COMMAND(pmd_perf_cmd); diff --git a/lib/librte_ether/rte_ether.h b/lib/librte_ether/rte_ether.h index 2e08f23..b18be7b 100644 --- a/lib/librte_ether/rte_ether.h +++ b/lib/librte_ether/rte_ether.h @@ -45,6 +45,7 @@ extern "C" { #endif #include <stdint.h> +#include <stdio.h> #include <rte_memcpy.h> #include <rte_random.h> @@ -266,6 +267,30 @@ static inline void ether_addr_copy(const struct ether_addr *ea_from, #endif } +#define ETHER_ADDR_FMT_SIZE 18 +/** + * Format 48bits Ethernet address in pattern xx:xx:xx:xx:xx:xx. + * + * @param buf + * A pointer to buffer contains the formatted MAC address. + * @param size + * The format buffer size. + * @param ea_to + * A pointer to a ether_addr structure. + */ +static inline void +ether_format_addr(char *buf, uint16_t size, + const struct ether_addr *eth_addr) +{ + snprintf(buf, size, "%02X:%02X:%02X:%02X:%02X:%02X", + eth_addr->addr_bytes[0], + eth_addr->addr_bytes[1], + eth_addr->addr_bytes[2], + eth_addr->addr_bytes[3], + eth_addr->addr_bytes[4], + eth_addr->addr_bytes[5]); +} + /** * Ethernet header: Contains the destination address, source address * and frame type. diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c index c5e4b71..e85d827 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c +++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c @@ -1600,6 +1600,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev) ixgbe_dev_clear_queues(dev); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + /* Clear recorded link status */ memset(&link, 0, sizeof(link)); rte_ixgbe_dev_atomic_write_link_status(dev, &link); @@ -2888,6 +2891,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) */ ixgbevf_set_vfta_all(dev,0); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + ixgbe_dev_clear_queues(dev); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v6 2/3] app/test: measure the cost of rx/tx routines by cycle number 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang @ 2014-11-11 23:28 ` Thomas Monjalon 2014-11-12 6:32 ` Liang, Cunming 0 siblings, 1 reply; 65+ messages in thread From: Thomas Monjalon @ 2014-11-11 23:28 UTC (permalink / raw) To: Cunming Liang; +Cc: dev Hi Cunming, 2014-10-27 09:20, Cunming Liang: > --- a/lib/librte_ether/rte_ether.h > +++ b/lib/librte_ether/rte_ether.h > @@ -45,6 +45,7 @@ extern "C" { > #endif > > #include <stdint.h> > +#include <stdio.h> > > #include <rte_memcpy.h> > #include <rte_random.h> > @@ -266,6 +267,30 @@ static inline void ether_addr_copy(const struct ether_addr *ea_from, > #endif > } > > +#define ETHER_ADDR_FMT_SIZE 18 > +/** > + * Format 48bits Ethernet address in pattern xx:xx:xx:xx:xx:xx. > + * > + * @param buf > + * A pointer to buffer contains the formatted MAC address. > + * @param size > + * The format buffer size. > + * @param ea_to > + * A pointer to a ether_addr structure. > + */ > +static inline void > +ether_format_addr(char *buf, uint16_t size, > + const struct ether_addr *eth_addr) > +{ > + snprintf(buf, size, "%02X:%02X:%02X:%02X:%02X:%02X", > + eth_addr->addr_bytes[0], > + eth_addr->addr_bytes[1], > + eth_addr->addr_bytes[2], > + eth_addr->addr_bytes[3], > + eth_addr->addr_bytes[4], > + eth_addr->addr_bytes[5]); > +} Please, could you do a separate patch for this new API? Could it be used in some apps or PMDs? It would be a nice cleanup. > --- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c > +++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c > @@ -1600,6 +1600,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev) > > ixgbe_dev_clear_queues(dev); > > + /* Clear stored conf */ > + dev->data->scattered_rx = 0; > + > /* Clear recorded link status */ > memset(&link, 0, sizeof(link)); > rte_ixgbe_dev_atomic_write_link_status(dev, &link); > @@ -2888,6 +2891,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) > */ > ixgbevf_set_vfta_all(dev,0); > > + /* Clear stored conf */ > + dev->data->scattered_rx = 0; > + > ixgbe_dev_clear_queues(dev); > } Please, this patch needs a separate patch with a clear explanation in the log. Thanks -- Thomas ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v6 2/3] app/test: measure the cost of rx/tx routines by cycle number 2014-11-11 23:28 ` Thomas Monjalon @ 2014-11-12 6:32 ` Liang, Cunming 0 siblings, 0 replies; 65+ messages in thread From: Liang, Cunming @ 2014-11-12 6:32 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev Hi Thomas, I've split the patch in v7 and also do cleanup by the new API. Thanks. -Liang Cunming > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, November 12, 2014 7:29 AM > To: Liang, Cunming > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v6 2/3] app/test: measure the cost of rx/tx > routines by cycle number > > Hi Cunming, > > 2014-10-27 09:20, Cunming Liang: > > --- a/lib/librte_ether/rte_ether.h > > +++ b/lib/librte_ether/rte_ether.h > > @@ -45,6 +45,7 @@ extern "C" { > > #endif > > > > #include <stdint.h> > > +#include <stdio.h> > > > > #include <rte_memcpy.h> > > #include <rte_random.h> > > @@ -266,6 +267,30 @@ static inline void ether_addr_copy(const struct > ether_addr *ea_from, > > #endif > > } > > > > +#define ETHER_ADDR_FMT_SIZE 18 > > +/** > > + * Format 48bits Ethernet address in pattern xx:xx:xx:xx:xx:xx. > > + * > > + * @param buf > > + * A pointer to buffer contains the formatted MAC address. > > + * @param size > > + * The format buffer size. > > + * @param ea_to > > + * A pointer to a ether_addr structure. > > + */ > > +static inline void > > +ether_format_addr(char *buf, uint16_t size, > > + const struct ether_addr *eth_addr) > > +{ > > + snprintf(buf, size, "%02X:%02X:%02X:%02X:%02X:%02X", > > + eth_addr->addr_bytes[0], > > + eth_addr->addr_bytes[1], > > + eth_addr->addr_bytes[2], > > + eth_addr->addr_bytes[3], > > + eth_addr->addr_bytes[4], > > + eth_addr->addr_bytes[5]); > > +} > > Please, could you do a separate patch for this new API? > Could it be used in some apps or PMDs? It would be a nice cleanup. > > > --- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c > > +++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c > > @@ -1600,6 +1600,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev) > > > > ixgbe_dev_clear_queues(dev); > > > > + /* Clear stored conf */ > > + dev->data->scattered_rx = 0; > > + > > /* Clear recorded link status */ > > memset(&link, 0, sizeof(link)); > > rte_ixgbe_dev_atomic_write_link_status(dev, &link); > > @@ -2888,6 +2891,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) > > */ > > ixgbevf_set_vfta_all(dev,0); > > > > + /* Clear stored conf */ > > + dev->data->scattered_rx = 0; > > + > > ixgbe_dev_clear_queues(dev); > > } > > Please, this patch needs a separate patch with a clear explanation in the log. > > Thanks > -- > Thomas ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v6 3/3] ethdev: fix wrong error return refere to API definition 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Cunming Liang 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 1/3] app/test: allow to create packets in different sizes Cunming Liang 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang @ 2014-10-27 1:20 ` Cunming Liang 2014-10-27 16:03 ` Ananyev, Konstantin 2014-10-27 1:45 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Liu, Yong ` (2 subsequent siblings) 5 siblings, 1 reply; 65+ messages in thread From: Cunming Liang @ 2014-10-27 1:20 UTC (permalink / raw) To: dev Per definition, rte_eth_rx_burst/rte_eth_tx_burst/rte_eth_rx_queue_count returns the packet number When RTE_LIBRTE_ETHDEV_DEBUG turns on, retval of FUNC_PTR_OR_ERR_RTE was set to -ENOTSUP. It makes confusing. The patch always return 0 no matter no packet or there's error. Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- lib/librte_ether/rte_ethdev.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 50f10d9..922a0c6 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -2530,7 +2530,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, return 0; } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); return 0; @@ -2551,7 +2551,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); if (queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", queue_id); return 0; @@ -2570,7 +2570,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) return 0; } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v6 3/3] ethdev: fix wrong error return refere to API definition 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 3/3] ethdev: fix wrong error return refere to API definition Cunming Liang @ 2014-10-27 16:03 ` Ananyev, Konstantin 0 siblings, 0 replies; 65+ messages in thread From: Ananyev, Konstantin @ 2014-10-27 16:03 UTC (permalink / raw) To: Liang, Cunming, dev > From: Liang, Cunming > Sent: Monday, October 27, 2014 1:20 AM > To: dev@dpdk.org > Cc: nhorman@tuxdriver.com; Ananyev, Konstantin; Richardson, Bruce; De Lara Guarch, Pablo; Liang, Cunming > Subject: [PATCH v6 3/3] ethdev: fix wrong error return refere to API definition > > Per definition, rte_eth_rx_burst/rte_eth_tx_burst/rte_eth_rx_queue_count returns the packet number > When RTE_LIBRTE_ETHDEV_DEBUG turns on, retval of FUNC_PTR_OR_ERR_RTE was set to -ENOTSUP. > It makes confusing. > The patch always return 0 no matter no packet or there's error. > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > --- > lib/librte_ether/rte_ethdev.c | 6 +++--- > 1 files changed, 3 insertions(+), 3 deletions(-) > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c > index 50f10d9..922a0c6 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -2530,7 +2530,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, > return 0; > } > dev = &rte_eth_devices[port_id]; > - FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); > + FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); > if (queue_id >= dev->data->nb_rx_queues) { > PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); > return 0; > @@ -2551,7 +2551,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, > } > dev = &rte_eth_devices[port_id]; > > - FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, -ENOTSUP); > + FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); > if (queue_id >= dev->data->nb_tx_queues) { > PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", queue_id); > return 0; > @@ -2570,7 +2570,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) > return 0; > } > dev = &rte_eth_devices[port_id]; > - FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, -ENOTSUP); > + FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); > return (*dev->dev_ops->rx_queue_count)(dev, queue_id); > } > > -- > 1.7.4.1 Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com> ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Cunming Liang ` (2 preceding siblings ...) 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 3/3] ethdev: fix wrong error return refere to API definition Cunming Liang @ 2014-10-27 1:45 ` Liu, Yong 2014-10-29 5:06 ` Liang, Cunming 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang 5 siblings, 0 replies; 65+ messages in thread From: Liu, Yong @ 2014-10-27 1:45 UTC (permalink / raw) To: Liang, Cunming, dev Tested-by: Yong Liu <yong.liu@intel.com> - Tested Commit: 1ab07743b21b785a71fa334641ab58e779532600 - OS: Fedora20 3.15.8-200.fc20.x86_64 - GCC: gcc version 4.8.3 20140624 - CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz - NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] - Default x86_64-native-linuxapp-gcc configuration - Total 2 cases, 2 passed, 0 failed - Case: Continuous Mode Performance Description: Measure continous mode cycles/packet in NIC loopback mode Command / instruction: Start sample test application. ./app/test/test -n 1 -c ffff Set stream control mode to continuous RTE>>set_rxtx_sc continuous Choose rx/tx pair between vector|scalar|full|hybrid RTE>>set_rxtx_mode vector Choose the area of measurement RTE>>set_rxtx_anchor rxtx Start pmd performance measurement RTE>>pmd_perf_autotest Expected test result: Test result is OK and output cycle number for each packet. - Case: Burst Mode Performance Description: Measure burst mode cycles/packet in NIC loopback mode Command / instruction: Start sample test application. ./app/test/test -n 1 -c ffff Set stream control mode to poll_before_xmit or poll_after_xmit. RTE>>set_rxtx_sc poll_before_xmit Start pmd performance measurement RTE>>pmd_perf_autotest Expected test result: Test result is OK and output cycle number for each packet. > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Cunming Liang > Sent: Monday, October 27, 2014 9:20 AM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per > packet > > v6 update: > # leave FUNC_PTR_OR_*_RET unmodified > > v5 update: > # fix the confusing of retval in some API of rte_ethdev > > v4 ignore > > v3 update: > # Codes refine according to the feedback. > 1. add ether_format_addr to rte_ether.h > 2. fix typo in code comments. > 3. %lu to %PRIu64, fixing 32-bit targets compilation err > # merge 2 small incremental patches to the first one. > The whole unit test as a single patch in [PATCH v3 2/2] > # rebase code to the latest master > > v2 update: > Rebase code to the latest master branch. > > It provides unit test to measure cycles/packet in NIC loopback mode. > It simply gives the average cycles of IO used per packet without test > equipment. > When doing the test, make sure the link is UP. > > There's two stream control mode support, one is continues, another is burst. > The former continues to forward the injected packets until reaching a certain > amount of number. > The latter one stop when all the injected packets are received. > In burst stream, now measure two situations, with or without desc. cache > conflict. > By default, it runs in continues stream mode to measure the whole rxtx. > > Usage Example: > 1. Run unit test app in interactive mode > app/test -c f -n 4 -- -i > 2. Set stream control mode, by default is continuous > set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] > 3. If choose continuous stream, there are another two options can configure > 3.1 choose rx/tx pair, default is vector > set_rxtx_mode [vector|scalar|full|hybrid] > Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without > INC_VEC=y in config > 3.2 choose the area of masurement, default is rxtx > set_rxtx_anchor [rxtx|rxonly|txonly] > 4. Run and wait for the result > pmd_perf_autotest > > For who simply just want to see how much cycles cost per packet. > Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. > Nothing else needs to configure. > Using other options when you understand and what to measures more. > > > BTW, [1/3] is the same patch as below one. > http://dpdk.org/dev/patchwork/patch/817 > > *** BLURB HERE *** > > Cunming Liang (3): > app/test: allow to create packets in different sizes > app/test: measure the cost of rx/tx routines by cycle number > ethdev: fix wrong error return refere to API definition > > app/test/Makefile | 1 + > app/test/commands.c | 111 +++++ > app/test/packet_burst_generator.c | 26 +- > app/test/packet_burst_generator.h | 11 +- > app/test/test.h | 6 + > app/test/test_link_bonding.c | 39 +- > app/test/test_pmd_perf.c | 922 > +++++++++++++++++++++++++++++++++++ > lib/librte_ether/rte_ethdev.c | 6 +- > lib/librte_ether/rte_ether.h | 25 + > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > 10 files changed, 1117 insertions(+), 36 deletions(-) > create mode 100644 app/test/test_pmd_perf.c > > -- > 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Cunming Liang ` (3 preceding siblings ...) 2014-10-27 1:45 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Liu, Yong @ 2014-10-29 5:06 ` Liang, Cunming 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang 5 siblings, 0 replies; 65+ messages in thread From: Liang, Cunming @ 2014-10-29 5:06 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev Hi Thomas, All the open issues from the former patches are closed. Could you please have a look and get it applied ? -Liang Cunming > -----Original Message----- > From: Liang, Cunming > Sent: Monday, October 27, 2014 9:20 AM > To: dev@dpdk.org > Cc: nhorman@tuxdriver.com; Ananyev, Konstantin; Richardson, Bruce; De Lara > Guarch, Pablo; Liang, Cunming > Subject: [PATCH v6 0/3] app/test: unit test to measure cycles per packet > > v6 update: > # leave FUNC_PTR_OR_*_RET unmodified > > v5 update: > # fix the confusing of retval in some API of rte_ethdev > > v4 ignore > > v3 update: > # Codes refine according to the feedback. > 1. add ether_format_addr to rte_ether.h > 2. fix typo in code comments. > 3. %lu to %PRIu64, fixing 32-bit targets compilation err > # merge 2 small incremental patches to the first one. > The whole unit test as a single patch in [PATCH v3 2/2] > # rebase code to the latest master > > v2 update: > Rebase code to the latest master branch. > > It provides unit test to measure cycles/packet in NIC loopback mode. > It simply gives the average cycles of IO used per packet without test equipment. > When doing the test, make sure the link is UP. > > There's two stream control mode support, one is continues, another is burst. > The former continues to forward the injected packets until reaching a certain > amount of number. > The latter one stop when all the injected packets are received. > In burst stream, now measure two situations, with or without desc. cache conflict. > By default, it runs in continues stream mode to measure the whole rxtx. > > Usage Example: > 1. Run unit test app in interactive mode > app/test -c f -n 4 -- -i > 2. Set stream control mode, by default is continuous > set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] > 3. If choose continuous stream, there are another two options can configure > 3.1 choose rx/tx pair, default is vector > set_rxtx_mode [vector|scalar|full|hybrid] > Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without > INC_VEC=y in config > 3.2 choose the area of masurement, default is rxtx > set_rxtx_anchor [rxtx|rxonly|txonly] > 4. Run and wait for the result > pmd_perf_autotest > > For who simply just want to see how much cycles cost per packet. > Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. > Nothing else needs to configure. > Using other options when you understand and what to measures more. > > > BTW, [1/3] is the same patch as below one. > http://dpdk.org/dev/patchwork/patch/817 > > *** BLURB HERE *** > > Cunming Liang (3): > app/test: allow to create packets in different sizes > app/test: measure the cost of rx/tx routines by cycle number > ethdev: fix wrong error return refer to API definition > > app/test/Makefile | 1 + > app/test/commands.c | 111 +++++ > app/test/packet_burst_generator.c | 26 +- > app/test/packet_burst_generator.h | 11 +- > app/test/test.h | 6 + > app/test/test_link_bonding.c | 39 +- > app/test/test_pmd_perf.c | 922 > +++++++++++++++++++++++++++++++++++ > lib/librte_ether/rte_ethdev.c | 6 +- > lib/librte_ether/rte_ether.h | 25 + > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > 10 files changed, 1117 insertions(+), 36 deletions(-) > create mode 100644 app/test/test_pmd_perf.c > > -- > 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v7 0/7] app/test: unit test to measure cycles per packet 2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Cunming Liang ` (4 preceding siblings ...) 2014-10-29 5:06 ` Liang, Cunming @ 2014-11-12 6:24 ` Cunming Liang 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 1/7] app/test: allow to create packets in different sizes Cunming Liang ` (7 more replies) 5 siblings, 8 replies; 65+ messages in thread From: Cunming Liang @ 2014-11-12 6:24 UTC (permalink / raw) To: dev v7 update: # patch split and re-orginize v6 update: # leave FUNC_PTR_OR_*_RET unmodified v5 update: # fix the confusing of retval in some API of rte_ethdev v4 ignore v3 update: # Codes refine according to the feedback. 1. add ether_format_addr to rte_ether.h 2. fix typo in code comments. 3. %lu to %PRIu64, fixing 32-bit targets compilation err # merge 2 small incremental patches to the first one. The whole unit test as a single patch in [PATCH v3 2/2] # rebase code to the latest master v2 update: Rebase code to the latest master branch. It provides unit test to measure cycles/packet in NIC loopback mode. It simply gives the average cycles of IO used per packet without test equipment. When doing the test, make sure the link is UP. There's two stream control mode support, one is continues, another is burst. The former continues to forward the injected packets until reaching a certain amount of number. The latter one stop when all the injected packets are received. In burst stream, now measure two situations, with or without desc. cache conflict. By default, it runs in continues stream mode to measure the whole rxtx. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] 3. If choose continuous stream, there are another two options can configure 3.1 choose rx/tx pair, default is vector set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without INC_VEC=y in config 3.2 choose the area of masurement, default is rxtx set_rxtx_anchor [rxtx|rxonly|txonly] 4. Run and wait for the result pmd_perf_autotest For who simply just want to see how much cycles cost per packet. Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. Nothing else needs to configure. Using other options when you understand and what to measures more. BTW, [1/3] is the same patch as below one. http://dpdk.org/dev/patchwork/patch/817 *** BLURB HERE *** Cunming Liang (7): app/test: allow to create packets in different sizes ixgbe:clean scattered_rx configure in dev_stop ether: new API to format eth_addr in string app/testpmd: cleanup eth_addr print examples: cleanup eth_addr print app/test: measure the cost of rx/tx routines by cycle number ethdev: fix wrong error return refere to API definition app/test-pmd/config.c | 10 +- app/test-pmd/icmpecho.c | 16 +- app/test-pmd/rxonly.c | 11 +- app/test/Makefile | 1 + app/test/commands.c | 111 +++++ app/test/packet_burst_generator.c | 26 +- app/test/packet_burst_generator.h | 11 +- app/test/test.h | 6 + app/test/test_link_bonding.c | 39 +- app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ examples/dpdk_qat/main.c | 10 +- examples/ip_fragmentation/main.c | 10 +- examples/ip_reassembly/main.c | 10 +- examples/ipv4_multicast/main.c | 10 +- examples/l3fwd-acl/main.c | 10 +- examples/l3fwd-power/main.c | 10 +- examples/l3fwd-vf/main.c | 10 +- examples/l3fwd/main.c | 10 +- lib/librte_ether/rte_ethdev.c | 6 +- lib/librte_ether/rte_ether.h | 25 + lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + 21 files changed, 1149 insertions(+), 121 deletions(-) create mode 100644 app/test/test_pmd_perf.c -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v7 1/7] app/test: allow to create packets in different sizes 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang @ 2014-11-12 6:24 ` Cunming Liang 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop Cunming Liang ` (6 subsequent siblings) 7 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-11-12 6:24 UTC (permalink / raw) To: dev adding support to allow packet burst generator to create packets in differenct sizes Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- app/test/packet_burst_generator.c | 26 ++++++++---------------- app/test/packet_burst_generator.h | 11 +++++++-- app/test/test_link_bonding.c | 39 ++++++++++++++++++++++++------------ 3 files changed, 43 insertions(+), 33 deletions(-) diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c index 9e747a4..017139b 100644 --- a/app/test/packet_burst_generator.c +++ b/app/test/packet_burst_generator.c @@ -191,20 +191,12 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, */ #define RTE_MAX_SEGS_PER_PKT 255 /**< pkt.nb_segs is a 8-bit unsigned char. */ -#define TXONLY_DEF_PACKET_LEN 64 -#define TXONLY_DEF_PACKET_LEN_128 128 - -uint16_t tx_pkt_length = TXONLY_DEF_PACKET_LEN; -uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT] = { - TXONLY_DEF_PACKET_LEN_128, -}; - -uint8_t tx_pkt_nb_segs = 1; - int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst) + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs) { int i, nb_pkt = 0; size_t eth_hdr_size; @@ -221,9 +213,9 @@ nomore_mbuf: break; } - pkt->data_len = tx_pkt_seg_lengths[0]; + pkt->data_len = pkt_len; pkt_seg = pkt; - for (i = 1; i < tx_pkt_nb_segs; i++) { + for (i = 1; i < nb_pkt_segs; i++) { pkt_seg->next = rte_pktmbuf_alloc(mp); if (pkt_seg->next == NULL) { pkt->nb_segs = i; @@ -231,7 +223,7 @@ nomore_mbuf: goto nomore_mbuf; } pkt_seg = pkt_seg->next; - pkt_seg->data_len = tx_pkt_seg_lengths[i]; + pkt_seg->data_len = pkt_len; } pkt_seg->next = NULL; /* Last segment of packet. */ @@ -259,8 +251,8 @@ nomore_mbuf: * Complete first mbuf of packet and append it to the * burst of packets to be transmitted. */ - pkt->nb_segs = tx_pkt_nb_segs; - pkt->pkt_len = tx_pkt_length; + pkt->nb_segs = nb_pkt_segs; + pkt->pkt_len = pkt_len; pkt->l2_len = eth_hdr_size; if (ipv4) { diff --git a/app/test/packet_burst_generator.h b/app/test/packet_burst_generator.h index 5b3cd6c..fe992ac 100644 --- a/app/test/packet_burst_generator.h +++ b/app/test/packet_burst_generator.h @@ -47,10 +47,13 @@ extern "C" { #define IPV4_ADDR(a, b, c, d)(((a & 0xff) << 24) | ((b & 0xff) << 16) | \ ((c & 0xff) << 8) | (d & 0xff)) +#define PACKET_BURST_GEN_PKT_LEN 60 +#define PACKET_BURST_GEN_PKT_LEN_128 128 void initialize_eth_header(struct ether_hdr *eth_hdr, struct ether_addr *src_mac, - struct ether_addr *dst_mac, uint8_t vlan_enabled, uint16_t van_id); + struct ether_addr *dst_mac, uint8_t vlan_enabled, + uint16_t van_id); uint16_t initialize_udp_header(struct udp_hdr *udp_hdr, uint16_t src_port, @@ -67,8 +70,10 @@ initialize_ipv4_header(struct ipv4_hdr *ip_hdr, uint32_t src_addr, int generate_packet_burst(struct rte_mempool *mp, struct rte_mbuf **pkts_burst, - struct ether_hdr *eth_hdr, uint8_t vlan_enabled, void *ip_hdr, - uint8_t ipv4, struct udp_hdr *udp_hdr, int nb_pkt_per_burst); + struct ether_hdr *eth_hdr, uint8_t vlan_enabled, + void *ip_hdr, uint8_t ipv4, struct udp_hdr *udp_hdr, + int nb_pkt_per_burst, uint8_t pkt_len, + uint8_t nb_pkt_segs); #ifdef __cplusplus } diff --git a/app/test/test_link_bonding.c b/app/test/test_link_bonding.c index 214d2a2..d407e4f 100644 --- a/app/test/test_link_bonding.c +++ b/app/test/test_link_bonding.c @@ -1192,9 +1192,12 @@ generate_test_burst(struct rte_mbuf **pkts_burst, uint16_t burst_size, } /* Generate burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, vlan, ip_hdr, ipv4, - test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, + vlan, ip_hdr, ipv4, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN_128, 1); if (generated_burst_size != burst_size) { printf("Failed to generate packet burst"); return -1; @@ -1781,9 +1784,12 @@ test_activebackup_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; @@ -2435,8 +2441,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 1 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[0][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[0]) != burst_size[0]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[0], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[0]) return -1; initialize_eth_header(test_params->pkt_eth_hdr, @@ -2444,8 +2452,10 @@ test_balance_l2_tx_burst(void) /* Generate a burst 2 of packets to transmit */ if (generate_packet_burst(test_params->mbuf_pool, &pkts_burst[1][0], - test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, 1, - test_params->pkt_udp_hdr, burst_size[1]) != burst_size[1]) + test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size[1], + PACKET_BURST_GEN_PKT_LEN, 1) != burst_size[1]) return -1; /* Send burst 1 on bonded port */ @@ -3263,9 +3273,12 @@ test_broadcast_tx_burst(void) } /* Generate a burst of packets to transmit */ - generated_burst_size = generate_packet_burst(test_params->mbuf_pool, - pkts_burst, test_params->pkt_eth_hdr, 0, test_params->pkt_ipv4_hdr, - 1, test_params->pkt_udp_hdr, burst_size); + generated_burst_size = + generate_packet_burst(test_params->mbuf_pool, + pkts_burst, test_params->pkt_eth_hdr, 0, + test_params->pkt_ipv4_hdr, 1, + test_params->pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); if (generated_burst_size != burst_size) return -1; -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 1/7] app/test: allow to create packets in different sizes Cunming Liang @ 2014-11-12 6:24 ` Cunming Liang 2014-11-12 7:53 ` Thomas Monjalon 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 3/7] ether: new API to format eth_addr in string Cunming Liang ` (5 subsequent siblings) 7 siblings, 1 reply; 65+ messages in thread From: Cunming Liang @ 2014-11-12 6:24 UTC (permalink / raw) To: dev Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c index 9c73a30..05490dc 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c +++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c @@ -1600,6 +1600,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev) ixgbe_dev_clear_queues(dev); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + /* Clear recorded link status */ memset(&link, 0, sizeof(link)); rte_ixgbe_dev_atomic_write_link_status(dev, &link); @@ -2889,6 +2892,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev) */ ixgbevf_set_vfta_all(dev,0); + /* Clear stored conf */ + dev->data->scattered_rx = 0; + ixgbe_dev_clear_queues(dev); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop Cunming Liang @ 2014-11-12 7:53 ` Thomas Monjalon 2014-11-12 8:21 ` Liang, Cunming 0 siblings, 1 reply; 65+ messages in thread From: Thomas Monjalon @ 2014-11-12 7:53 UTC (permalink / raw) To: Cunming Liang; +Cc: dev Hi Cunming, Please, could you provide an explanation for the commit log? It should answer to the question "what was the issue?" If it's a fix, the title should start with "fix". Maybe that the same kind of fix is needed for em, igb and i40e? Thanks -- Thomas ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop 2014-11-12 7:53 ` Thomas Monjalon @ 2014-11-12 8:21 ` Liang, Cunming 2014-11-12 9:24 ` Thomas Monjalon 0 siblings, 1 reply; 65+ messages in thread From: Liang, Cunming @ 2014-11-12 8:21 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev The scattered_rx is update in dev_start. In this unit test, we will re-configure and change the scatter mode. When we stop, re-configure and then re-start, it expect using the new configure. But during re-configure, the stored data may still old. The patch clean the configure anyway in dev_stop. For em, igb and i40e, we haven't provide so much rx/tx pair for switching. > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, November 12, 2014 3:53 PM > To: Liang, Cunming > Cc: dev@dpdk.org; nhorman@tuxdriver.com; Ananyev, Konstantin; Richardson, > Bruce; De Lara Guarch, Pablo > Subject: Re: [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop > > Hi Cunming, > > Please, could you provide an explanation for the commit log? > It should answer to the question "what was the issue?" > If it's a fix, the title should start with "fix". > > Maybe that the same kind of fix is needed for em, igb and i40e? > > Thanks > -- > Thomas ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop 2014-11-12 8:21 ` Liang, Cunming @ 2014-11-12 9:24 ` Thomas Monjalon 2014-11-12 10:29 ` Liang, Cunming 0 siblings, 1 reply; 65+ messages in thread From: Thomas Monjalon @ 2014-11-12 9:24 UTC (permalink / raw) To: Liang, Cunming; +Cc: dev 2014-11-12 08:21, Liang, Cunming: > For em, igb and i40e, we haven't provide so much rx/tx pair for switching. Sorry, I don't understand. Which pair are you telling about? em, igb and i40e have scattered Rx functions. -- Thomas ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop 2014-11-12 9:24 ` Thomas Monjalon @ 2014-11-12 10:29 ` Liang, Cunming 2014-11-12 10:32 ` Thomas Monjalon 0 siblings, 1 reply; 65+ messages in thread From: Liang, Cunming @ 2014-11-12 10:29 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev Maybe pair is not accurate, I means the different rx/tx register function, like: Ixgbe_recv_bulk_alloc/ixgbe_recv_(scattered_)pkts_vec/ixgbe_recv_scattered_pkts ixgbe_xmit_pkts_simple/Ixgbe_xmit_pkts_vec/ixgbe_xmit_pkts -Liang Cunming > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, November 12, 2014 5:25 PM > To: Liang, Cunming > Cc: dev@dpdk.org; nhorman@tuxdriver.com; Ananyev, Konstantin; Richardson, > Bruce; De Lara Guarch, Pablo > Subject: Re: [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop > > 2014-11-12 08:21, Liang, Cunming: > > For em, igb and i40e, we haven't provide so much rx/tx pair for switching. > > Sorry, I don't understand. Which pair are you telling about? > em, igb and i40e have scattered Rx functions. > > -- > Thomas ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop 2014-11-12 10:29 ` Liang, Cunming @ 2014-11-12 10:32 ` Thomas Monjalon 2014-11-12 10:42 ` Liang, Cunming 0 siblings, 1 reply; 65+ messages in thread From: Thomas Monjalon @ 2014-11-12 10:32 UTC (permalink / raw) To: Liang, Cunming; +Cc: dev 2014-11-12 10:29, Liang, Cunming: > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > 2014-11-12 08:21, Liang, Cunming: > > > For em, igb and i40e, we haven't provide so much rx/tx pair for switching. > > > > Sorry, I don't understand. Which pair are you telling about? > > em, igb and i40e have scattered Rx functions. > > Maybe pair is not accurate, I means the different rx/tx register function, like: > Ixgbe_recv_bulk_alloc/ixgbe_recv_(scattered_)pkts_vec/ixgbe_recv_scattered_pkts > ixgbe_xmit_pkts_simple/Ixgbe_xmit_pkts_vec/ixgbe_xmit_pkts OK that's what I understood. However, you should check the scatter/non-scatter Rx functions of other drivers. I think they need the same fix. Thanks -- Thomas ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop 2014-11-12 10:32 ` Thomas Monjalon @ 2014-11-12 10:42 ` Liang, Cunming 0 siblings, 0 replies; 65+ messages in thread From: Liang, Cunming @ 2014-11-12 10:42 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev scatter/non-scatter always be checked during dev_start. For others, it's only have the two. Won't do additional check during rx/tx_queue_setup(before dev_start). So they won't have problem, I think. But for ixgbe, it will check it meets vector condition or not, then choose the best performance function. As the happens before dev_start, so the old value will impact the condition check. -Liang Cunming > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, November 12, 2014 6:33 PM > To: Liang, Cunming > Cc: dev@dpdk.org; nhorman@tuxdriver.com; Ananyev, Konstantin; Richardson, > Bruce; De Lara Guarch, Pablo > Subject: Re: [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop > > 2014-11-12 10:29, Liang, Cunming: > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > > 2014-11-12 08:21, Liang, Cunming: > > > > For em, igb and i40e, we haven't provide so much rx/tx pair for switching. > > > > > > Sorry, I don't understand. Which pair are you telling about? > > > em, igb and i40e have scattered Rx functions. > > > > Maybe pair is not accurate, I means the different rx/tx register function, like: > > > Ixgbe_recv_bulk_alloc/ixgbe_recv_(scattered_)pkts_vec/ixgbe_recv_scattered_p > kts > > ixgbe_xmit_pkts_simple/Ixgbe_xmit_pkts_vec/ixgbe_xmit_pkts > > OK that's what I understood. > However, you should check the scatter/non-scatter Rx functions of other drivers. > I think they need the same fix. > > Thanks > -- > Thomas ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v7 3/7] ether: new API to format eth_addr in string 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 1/7] app/test: allow to create packets in different sizes Cunming Liang 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop Cunming Liang @ 2014-11-12 6:24 ` Cunming Liang 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 4/7] app/testpmd: cleanup eth_addr print Cunming Liang ` (4 subsequent siblings) 7 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-11-12 6:24 UTC (permalink / raw) To: dev Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- lib/librte_ether/rte_ether.h | 25 +++++++++++++++++++++++++ 1 files changed, 25 insertions(+), 0 deletions(-) diff --git a/lib/librte_ether/rte_ether.h b/lib/librte_ether/rte_ether.h index f173509..187608d 100644 --- a/lib/librte_ether/rte_ether.h +++ b/lib/librte_ether/rte_ether.h @@ -45,6 +45,7 @@ extern "C" { #endif #include <stdint.h> +#include <stdio.h> #include <rte_memcpy.h> #include <rte_random.h> @@ -266,6 +267,30 @@ static inline void ether_addr_copy(const struct ether_addr *ea_from, #endif } +#define ETHER_ADDR_FMT_SIZE 18 +/** + * Format 48bits Ethernet address in pattern xx:xx:xx:xx:xx:xx. + * + * @param buf + * A pointer to buffer contains the formatted MAC address. + * @param size + * The format buffer size. + * @param ea_to + * A pointer to a ether_addr structure. + */ +static inline void +ether_format_addr(char *buf, uint16_t size, + const struct ether_addr *eth_addr) +{ + snprintf(buf, size, "%02X:%02X:%02X:%02X:%02X:%02X", + eth_addr->addr_bytes[0], + eth_addr->addr_bytes[1], + eth_addr->addr_bytes[2], + eth_addr->addr_bytes[3], + eth_addr->addr_bytes[4], + eth_addr->addr_bytes[5]); +} + /** * Ethernet header: Contains the destination address, source address * and frame type. -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v7 4/7] app/testpmd: cleanup eth_addr print 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang ` (2 preceding siblings ...) 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 3/7] ether: new API to format eth_addr in string Cunming Liang @ 2014-11-12 6:24 ` Cunming Liang 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 5/7] examples: " Cunming Liang ` (3 subsequent siblings) 7 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-11-12 6:24 UTC (permalink / raw) To: dev Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- app/test-pmd/config.c | 10 +++------- app/test-pmd/icmpecho.c | 16 ++-------------- app/test-pmd/rxonly.c | 11 +++-------- 3 files changed, 8 insertions(+), 29 deletions(-) diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index 9bc08f4..b102b72 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -99,13 +99,9 @@ static void print_ethaddr(const char *name, struct ether_addr *eth_addr) { - printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, - (unsigned int)eth_addr->addr_bytes[0], - (unsigned int)eth_addr->addr_bytes[1], - (unsigned int)eth_addr->addr_bytes[2], - (unsigned int)eth_addr->addr_bytes[3], - (unsigned int)eth_addr->addr_bytes[4], - (unsigned int)eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); } void diff --git a/app/test-pmd/icmpecho.c b/app/test-pmd/icmpecho.c index 7fd4b6d..774924e 100644 --- a/app/test-pmd/icmpecho.c +++ b/app/test-pmd/icmpecho.c @@ -236,18 +236,6 @@ ip_proto_name(uint8_t ip_proto) } static void -ether_addr_to_hexa(const struct ether_addr *ea, char *buf) -{ - sprintf(buf, "%02x:%02x:%02x:%02x:%02x:%02x", - ea->addr_bytes[0], - ea->addr_bytes[1], - ea->addr_bytes[2], - ea->addr_bytes[3], - ea->addr_bytes[4], - ea->addr_bytes[5]); -} - -static void ipv4_addr_to_dot(uint32_t be_ipv4_addr, char *buf) { uint32_t ipv4_addr; @@ -261,9 +249,9 @@ ipv4_addr_to_dot(uint32_t be_ipv4_addr, char *buf) static void ether_addr_dump(const char *what, const struct ether_addr *ea) { - char buf[18]; + char buf[ETHER_ADDR_FMT_SIZE]; - ether_addr_to_hexa(ea, buf); + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, ea); if (what) printf("%s", what); printf("%s", buf); diff --git a/app/test-pmd/rxonly.c b/app/test-pmd/rxonly.c index 4410c3d..9ad1df6 100644 --- a/app/test-pmd/rxonly.c +++ b/app/test-pmd/rxonly.c @@ -94,14 +94,9 @@ static const char *pkt_rx_flag_names[MAX_PKT_RX_FLAGS] = { static inline void print_ether_addr(const char *what, struct ether_addr *eth_addr) { - printf("%s%02X:%02X:%02X:%02X:%02X:%02X", - what, - eth_addr->addr_bytes[0], - eth_addr->addr_bytes[1], - eth_addr->addr_bytes[2], - eth_addr->addr_bytes[3], - eth_addr->addr_bytes[4], - eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", what, buf); } /* -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v7 5/7] examples: cleanup eth_addr print 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang ` (3 preceding siblings ...) 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 4/7] app/testpmd: cleanup eth_addr print Cunming Liang @ 2014-11-12 6:24 ` Cunming Liang 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 6/7] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang ` (2 subsequent siblings) 7 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-11-12 6:24 UTC (permalink / raw) To: dev Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- examples/dpdk_qat/main.c | 10 +++------- examples/ip_fragmentation/main.c | 10 +++------- examples/ip_reassembly/main.c | 10 +++------- examples/ipv4_multicast/main.c | 10 +++------- examples/l3fwd-acl/main.c | 10 +++------- examples/l3fwd-power/main.c | 10 +++------- examples/l3fwd-vf/main.c | 10 +++------- examples/l3fwd/main.c | 10 +++------- 8 files changed, 24 insertions(+), 56 deletions(-) diff --git a/examples/dpdk_qat/main.c b/examples/dpdk_qat/main.c index 59a97bc..127ee40 100644 --- a/examples/dpdk_qat/main.c +++ b/examples/dpdk_qat/main.c @@ -592,13 +592,9 @@ parse_args(int argc, char **argv) static void print_ethaddr(const char *name, const struct ether_addr *eth_addr) { - printf ("%s%02X:%02X:%02X:%02X:%02X:%02X", name, - eth_addr->addr_bytes[0], - eth_addr->addr_bytes[1], - eth_addr->addr_bytes[2], - eth_addr->addr_bytes[3], - eth_addr->addr_bytes[4], - eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); } static int diff --git a/examples/ip_fragmentation/main.c b/examples/ip_fragmentation/main.c index 65c4cf6..0ad8fca 100644 --- a/examples/ip_fragmentation/main.c +++ b/examples/ip_fragmentation/main.c @@ -600,13 +600,9 @@ parse_args(int argc, char **argv) static void print_ethaddr(const char *name, struct ether_addr *eth_addr) { - printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, - eth_addr->addr_bytes[0], - eth_addr->addr_bytes[1], - eth_addr->addr_bytes[2], - eth_addr->addr_bytes[3], - eth_addr->addr_bytes[4], - eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); } /* Check the link status of all ports in up to 9s, and print them finally */ diff --git a/examples/ip_reassembly/main.c b/examples/ip_reassembly/main.c index 0d92b3c..39d60ec 100644 --- a/examples/ip_reassembly/main.c +++ b/examples/ip_reassembly/main.c @@ -729,13 +729,9 @@ parse_args(int argc, char **argv) static void print_ethaddr(const char *name, const struct ether_addr *eth_addr) { - printf ("%s%02X:%02X:%02X:%02X:%02X:%02X", name, - eth_addr->addr_bytes[0], - eth_addr->addr_bytes[1], - eth_addr->addr_bytes[2], - eth_addr->addr_bytes[3], - eth_addr->addr_bytes[4], - eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); } /* Check the link status of all ports in up to 9s, and print them finally */ diff --git a/examples/ipv4_multicast/main.c b/examples/ipv4_multicast/main.c index de5e6be..590d11a 100644 --- a/examples/ipv4_multicast/main.c +++ b/examples/ipv4_multicast/main.c @@ -586,13 +586,9 @@ parse_args(int argc, char **argv) static void print_ethaddr(const char *name, struct ether_addr *eth_addr) { - printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, - eth_addr->addr_bytes[0], - eth_addr->addr_bytes[1], - eth_addr->addr_bytes[2], - eth_addr->addr_bytes[3], - eth_addr->addr_bytes[4], - eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); } static int diff --git a/examples/l3fwd-acl/main.c b/examples/l3fwd-acl/main.c index e31a7d7..f12e5bb 100644 --- a/examples/l3fwd-acl/main.c +++ b/examples/l3fwd-acl/main.c @@ -1820,13 +1820,9 @@ parse_args(int argc, char **argv) static void print_ethaddr(const char *name, const struct ether_addr *eth_addr) { - printf("%s%02X:%02X:%02X:%02X:%02X:%02X", name, - eth_addr->addr_bytes[0], - eth_addr->addr_bytes[1], - eth_addr->addr_bytes[2], - eth_addr->addr_bytes[3], - eth_addr->addr_bytes[4], - eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); } static int diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index 12e4fd6..76a512b 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -1240,13 +1240,9 @@ parse_args(int argc, char **argv) static void print_ethaddr(const char *name, const struct ether_addr *eth_addr) { - printf ("%s%02X:%02X:%02X:%02X:%02X:%02X", name, - eth_addr->addr_bytes[0], - eth_addr->addr_bytes[1], - eth_addr->addr_bytes[2], - eth_addr->addr_bytes[3], - eth_addr->addr_bytes[4], - eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); } #if (APP_LOOKUP_METHOD == APP_LOOKUP_EXACT_MATCH) diff --git a/examples/l3fwd-vf/main.c b/examples/l3fwd-vf/main.c index 0288d99..b560b7d 100644 --- a/examples/l3fwd-vf/main.c +++ b/examples/l3fwd-vf/main.c @@ -826,13 +826,9 @@ parse_args(int argc, char **argv) static void print_ethaddr(const char *name, const struct ether_addr *eth_addr) { - printf ("%s%02X:%02X:%02X:%02X:%02X:%02X", name, - eth_addr->addr_bytes[0], - eth_addr->addr_bytes[1], - eth_addr->addr_bytes[2], - eth_addr->addr_bytes[3], - eth_addr->addr_bytes[4], - eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); } #if (APP_LOOKUP_METHOD == APP_LOOKUP_EXACT_MATCH) diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c index 181d682..46ac68e 100644 --- a/examples/l3fwd/main.c +++ b/examples/l3fwd/main.c @@ -1976,13 +1976,9 @@ parse_args(int argc, char **argv) static void print_ethaddr(const char *name, const struct ether_addr *eth_addr) { - printf ("%s%02X:%02X:%02X:%02X:%02X:%02X", name, - eth_addr->addr_bytes[0], - eth_addr->addr_bytes[1], - eth_addr->addr_bytes[2], - eth_addr->addr_bytes[3], - eth_addr->addr_bytes[4], - eth_addr->addr_bytes[5]); + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); } #if (APP_LOOKUP_METHOD == APP_LOOKUP_EXACT_MATCH) -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v7 6/7] app/test: measure the cost of rx/tx routines by cycle number 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang ` (4 preceding siblings ...) 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 5/7] examples: " Cunming Liang @ 2014-11-12 6:24 ` Cunming Liang 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 7/7] ethdev: fix wrong error return refere to API definition Cunming Liang 2014-11-12 23:50 ` [dpdk-dev] [PATCH v7 0/7] app/test: unit test to measure cycles per packet Thomas Monjalon 7 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-11-12 6:24 UTC (permalink / raw) To: dev The unit test can be used to measure cycles per packet in different rx/tx rouines. The NIC works in loopback mode. So it doesn't require test equipment to measure throughput. As result, the unit test shows the average cycles per packet consuming. When doing the test, make sure the link is UP. Usage Example: 1. Run unit test app in interactive mode app/test -c f -n 4 -- -i 2. Run and wait for the result pmd_perf_autotest There's option to choose rx/tx pair, default is vector. set_rxtx_mode [vector|scalar|full|hybrid] Note: To get acurate scalar fast, please choose 'vector' or 'hybrid' without INC_VEC=y in config It supports to measure standalone rx or tx. Usage Example: Choose rx or tx standalone, default is both set_rxtx_anchor [rxtx|rxonly|txonly] It also supports to measure standalone RX burst cycles. In this way, it won't repeat re-send recevied packets. Now it measures two situations, poll before/after xmit(w or w/o desc. cache conflict) Usage Example: Set stream control mode, by default is continuous set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- app/test/Makefile | 1 + app/test/commands.c | 111 ++++++ app/test/test.h | 6 + app/test/test_pmd_perf.c | 922 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 1040 insertions(+), 0 deletions(-) create mode 100644 app/test/test_pmd_perf.c diff --git a/app/test/Makefile b/app/test/Makefile index 6af6d76..ebfa0ba 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -56,6 +56,7 @@ SRCS-y += test_memzone.c SRCS-y += test_ring.c SRCS-y += test_ring_perf.c +SRCS-y += test_pmd_perf.c ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) SRCS-y += test_table.c diff --git a/app/test/commands.c b/app/test/commands.c index a9e36b1..92a17ed 100644 --- a/app/test/commands.c +++ b/app/test/commands.c @@ -310,12 +310,123 @@ cmdline_parse_inst_t cmd_quit = { /****************/ +struct cmd_set_rxtx_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t mode; +}; + +static void cmd_set_rxtx_parsed(void *parsed_result, struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_result *res = parsed_result; + if (test_set_rxtx_conf(res->mode) < 0) + cmdline_printf(cl, "Cannot find such mode\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, set, + "set_rxtx_mode"); + +cmdline_parse_token_string_t cmd_set_rxtx_mode = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_result, mode, NULL); + +cmdline_parse_inst_t cmd_set_rxtx = { + .f = cmd_set_rxtx_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx routine: " + "set_rxtx <mode>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_set, + (void *)&cmd_set_rxtx_mode, + NULL, + }, +}; + +/****************/ + +struct cmd_set_rxtx_anchor { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_anchor_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_anchor *res = parsed_result; + if (test_set_rxtx_anchor(res->type) < 0) + cmdline_printf(cl, "Cannot find such anchor\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, set, + "set_rxtx_anchor"); + +cmdline_parse_token_string_t cmd_set_rxtx_anchor_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_anchor, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_anchor = { + .f = cmd_set_rxtx_anchor_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx anchor: " + "set_rxtx_anchor <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_anchor_set, + (void *)&cmd_set_rxtx_anchor_type, + NULL, + }, +}; + +/****************/ + +/* for stream control */ +struct cmd_set_rxtx_sc { + cmdline_fixed_string_t set; + cmdline_fixed_string_t type; +}; + +static void +cmd_set_rxtx_sc_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_set_rxtx_sc *res = parsed_result; + if (test_set_rxtx_sc(res->type) < 0) + cmdline_printf(cl, "Cannot find such stream control\n"); +} + +cmdline_parse_token_string_t cmd_set_rxtx_sc_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, set, + "set_rxtx_sc"); + +cmdline_parse_token_string_t cmd_set_rxtx_sc_type = + TOKEN_STRING_INITIALIZER(struct cmd_set_rxtx_sc, type, NULL); + +cmdline_parse_inst_t cmd_set_rxtx_sc = { + .f = cmd_set_rxtx_sc_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "set rxtx stream control: " + "set_rxtx_sc <type>", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_set_rxtx_sc_set, + (void *)&cmd_set_rxtx_sc_type, + NULL, + }, +}; + +/****************/ + + cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_autotest, (cmdline_parse_inst_t *)&cmd_dump, (cmdline_parse_inst_t *)&cmd_dump_one, (cmdline_parse_inst_t *)&cmd_set_ring, (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_set_rxtx, + (cmdline_parse_inst_t *)&cmd_set_rxtx_anchor, + (cmdline_parse_inst_t *)&cmd_set_rxtx_sc, NULL, }; diff --git a/app/test/test.h b/app/test/test.h index 98ab804..bd44a7f 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -125,6 +125,9 @@ int unit_test_suite_runner(struct unit_test_suite *suite); #define RECURSIVE_ENV_VAR "RTE_TEST_RECURSIVE" +#include <cmdline_parse.h> +#include <cmdline_parse_string.h> + extern const char *prgname; int commands_init(void); @@ -137,6 +140,9 @@ int test_pci_run; int test_mp_secondary(void); int test_ivshmem(void); +int test_set_rxtx_conf(cmdline_fixed_string_t mode); +int test_set_rxtx_anchor(cmdline_fixed_string_t type); +int test_set_rxtx_sc(cmdline_fixed_string_t type); typedef int (test_callback)(void); TAILQ_HEAD(test_commands_list, test_command); diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c new file mode 100644 index 0000000..1c1f236 --- /dev/null +++ b/app/test/test_pmd_perf.c @@ -0,0 +1,922 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + + +#include <stdio.h> +#include <inttypes.h> +#include <signal.h> +#include <unistd.h> +#include <rte_cycles.h> +#include <rte_ethdev.h> +#include <rte_byteorder.h> +#include <rte_atomic.h> +#include <rte_malloc.h> +#include "packet_burst_generator.h" +#include "test.h" + +#define NB_ETHPORTS_USED (1) +#define NB_SOCKETS (2) +#define MEMPOOL_CACHE_SIZE 250 +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MAX_PKT_BURST (32) +#define RTE_TEST_RX_DESC_DEFAULT (128) +#define RTE_TEST_TX_DESC_DEFAULT (512) +#define RTE_PORT_ALL (~(uint8_t)0x0) + +/* how long test would take at full line rate */ +#define RTE_TEST_DURATION (2) + +/* + * RX and TX Prefetch, Host, and Write-back threshold values should be + * carefully set for optimal performance. Consult the network + * controller's datasheet and supporting DPDK documentation for guidance + * on how these parameters should be set. + */ +#define RX_PTHRESH 8 /**< Default values of RX prefetch threshold reg. */ +#define RX_HTHRESH 8 /**< Default values of RX host threshold reg. */ +#define RX_WTHRESH 0 /**< Default values of RX write-back threshold reg. */ + +/* + * These default values are optimized for use with the Intel(R) 82599 10 GbE + * Controller and the DPDK ixgbe PMD. Consider using other values for other + * network controllers and/or network drivers. + */ +#define TX_PTHRESH 32 /**< Default values of TX prefetch threshold reg. */ +#define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */ +#define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */ + +#define MAX_TRAFFIC_BURST 2048 + +#define NB_MBUF RTE_MAX( \ + (unsigned)(nb_ports*nb_rx_queue*nb_rxd + \ + nb_ports*nb_lcores*MAX_PKT_BURST + \ + nb_ports*nb_tx_queue*nb_txd + \ + nb_lcores*MEMPOOL_CACHE_SIZE + \ + nb_ports*MAX_TRAFFIC_BURST), \ + (unsigned)8192) + + +static struct rte_mempool *mbufpool[NB_SOCKETS]; +/* ethernet addresses of ports */ +static struct ether_addr ports_eth_addr[RTE_MAX_ETHPORTS]; + +static struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .hw_vlan_strip = 0, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 1, /* enable loopback */ +}; + +static struct rte_eth_rxconf rx_conf = { + .rx_thresh = { + .pthresh = RX_PTHRESH, + .hthresh = RX_HTHRESH, + .wthresh = RX_WTHRESH, + }, + .rx_free_thresh = 32, +}; + +static struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = TX_PTHRESH, + .hthresh = TX_HTHRESH, + .wthresh = TX_WTHRESH, + }, + .tx_free_thresh = 32, /* Use PMD default values */ + .tx_rs_thresh = 32, /* Use PMD default values */ + .txq_flags = (ETH_TXQ_FLAGS_NOMULTSEGS | + ETH_TXQ_FLAGS_NOVLANOFFL | + ETH_TXQ_FLAGS_NOXSUMSCTP | + ETH_TXQ_FLAGS_NOXSUMUDP | + ETH_TXQ_FLAGS_NOXSUMTCP) +}; + +enum { + LCORE_INVALID = 0, + LCORE_AVAIL, + LCORE_USED, +}; + +struct lcore_conf { + uint8_t status; + uint8_t socketid; + uint16_t nb_ports; + uint8_t portlist[RTE_MAX_ETHPORTS]; +} __rte_cache_aligned; + +struct lcore_conf lcore_conf[RTE_MAX_LCORE]; + +static uint64_t link_mbps; + +enum { + SC_CONTINUOUS = 0, + SC_BURST_POLL_FIRST, + SC_BURST_XMIT_FIRST, +}; + +static uint32_t sc_flag; + +/* Check the link status of all ports in up to 3s, and print them finally */ +static void +check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +{ +#define CHECK_INTERVAL 100 /* 100ms */ +#define MAX_CHECK_TIME 30 /* 3s (30 * 100ms) in total */ + uint8_t portid, count, all_ports_up, print_flag = 0; + struct rte_eth_link link; + + printf("Checking link statuses...\n"); + fflush(stdout); + for (count = 0; count <= MAX_CHECK_TIME; count++) { + all_ports_up = 1; + for (portid = 0; portid < port_num; portid++) { + if ((port_mask & (1 << portid)) == 0) + continue; + memset(&link, 0, sizeof(link)); + rte_eth_link_get_nowait(portid, &link); + /* print link status if flag set */ + if (print_flag == 1) { + if (link.link_status) { + printf("Port %d Link Up - speed %u " + "Mbps - %s\n", (uint8_t)portid, + (unsigned)link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex\n")); + if (link_mbps == 0) + link_mbps = link.link_speed; + } else + printf("Port %d Link Down\n", + (uint8_t)portid); + continue; + } + /* clear all_ports_up flag if any link down */ + if (link.link_status == 0) { + all_ports_up = 0; + break; + } + } + /* after finally printing all link status, get out */ + if (print_flag == 1) + break; + + if (all_ports_up == 0) { + fflush(stdout); + rte_delay_ms(CHECK_INTERVAL); + } + + /* set the print_flag if all ports up or timeout */ + if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) + print_flag = 1; + } +} + +static void +print_ethaddr(const char *name, const struct ether_addr *eth_addr) +{ + char buf[ETHER_ADDR_FMT_SIZE]; + ether_format_addr(buf, ETHER_ADDR_FMT_SIZE, eth_addr); + printf("%s%s", name, buf); +} + +static int +init_traffic(struct rte_mempool *mp, + struct rte_mbuf **pkts_burst, uint32_t burst_size) +{ + struct ether_hdr pkt_eth_hdr; + struct ipv4_hdr pkt_ipv4_hdr; + struct udp_hdr pkt_udp_hdr; + uint32_t pktlen; + static uint8_t src_mac[] = { 0x00, 0xFF, 0xAA, 0xFF, 0xAA, 0xFF }; + static uint8_t dst_mac[] = { 0x00, 0xAA, 0xFF, 0xAA, 0xFF, 0xAA }; + + + initialize_eth_header(&pkt_eth_hdr, + (struct ether_addr *)src_mac, + (struct ether_addr *)dst_mac, 0, 0); + pkt_eth_hdr.ether_type = rte_cpu_to_be_16(ETHER_TYPE_IPv4); + + pktlen = initialize_ipv4_header(&pkt_ipv4_hdr, + IPV4_ADDR(10, 0, 0, 1), + IPV4_ADDR(10, 0, 0, 2), 26); + printf("IPv4 pktlen %u\n", pktlen); + + pktlen = initialize_udp_header(&pkt_udp_hdr, 0, 0, 18); + + printf("UDP pktlen %u\n", pktlen); + + return generate_packet_burst(mp, pkts_burst, &pkt_eth_hdr, + 0, &pkt_ipv4_hdr, 1, + &pkt_udp_hdr, burst_size, + PACKET_BURST_GEN_PKT_LEN, 1); +} + +static int +init_lcores(void) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + lcore_conf[lcore_id].socketid = + rte_lcore_to_socket_id(lcore_id); + if (rte_lcore_is_enabled(lcore_id) == 0) { + lcore_conf[lcore_id].status = LCORE_INVALID; + continue; + } else + lcore_conf[lcore_id].status = LCORE_AVAIL; + } + return 0; +} + +static int +init_mbufpool(unsigned nb_mbuf) +{ + int socketid; + unsigned lcore_id; + char s[64]; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + socketid = rte_lcore_to_socket_id(lcore_id); + if (socketid >= NB_SOCKETS) { + rte_exit(EXIT_FAILURE, + "Socket %d of lcore %u is out of range %d\n", + socketid, lcore_id, NB_SOCKETS); + } + if (mbufpool[socketid] == NULL) { + snprintf(s, sizeof(s), "mbuf_pool_%d", socketid); + mbufpool[socketid] = + rte_mempool_create(s, nb_mbuf, MBUF_SIZE, + MEMPOOL_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + socketid, 0); + if (mbufpool[socketid] == NULL) + rte_exit(EXIT_FAILURE, + "Cannot init mbuf pool on socket %d\n", + socketid); + else + printf("Allocated mbuf pool on socket %d\n", + socketid); + } + } + return 0; +} + +static uint16_t +alloc_lcore(uint16_t socketid) +{ + unsigned lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (LCORE_AVAIL != lcore_conf[lcore_id].status || + lcore_conf[lcore_id].socketid != socketid || + lcore_id == rte_get_master_lcore()) + continue; + lcore_conf[lcore_id].status = LCORE_USED; + lcore_conf[lcore_id].nb_ports = 0; + return lcore_id; + } + + return (uint16_t)-1; +} + +volatile uint64_t stop; +uint64_t count; +uint64_t drop; +uint64_t idle; + +static void +reset_count(void) +{ + count = 0; + drop = 0; + idle = 0; +} + +static void +stats_display(uint8_t port_id) +{ + struct rte_eth_stats stats; + rte_eth_stats_get(port_id, &stats); + + printf(" RX-packets: %-10"PRIu64" RX-missed: %-10"PRIu64" RX-bytes: " + "%-"PRIu64"\n", + stats.ipackets, stats.imissed, stats.ibytes); + printf(" RX-badcrc: %-10"PRIu64" RX-badlen: %-10"PRIu64" RX-errors: " + "%-"PRIu64"\n", + stats.ibadcrc, stats.ibadlen, stats.ierrors); + printf(" RX-nombuf: %-10"PRIu64"\n", + stats.rx_nombuf); + printf(" TX-packets: %-10"PRIu64" TX-errors: %-10"PRIu64" TX-bytes: " + "%-"PRIu64"\n", + stats.opackets, stats.oerrors, stats.obytes); +} + +static void +signal_handler(int signum) +{ + /* USR1 signal, stop testing */ + if (signum == SIGUSR1) { + printf("Force Stop!\n"); + stop = 1; + } + + /* USR2 signal, print stats */ + if (signum == SIGUSR2) + stats_display(0); +} + +struct rte_mbuf **tx_burst; + +uint64_t (*do_measure)(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts); + +static uint64_t +measure_rxtx(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t prev_tsc, cur_tsc; + + prev_tsc = rte_rdtsc(); + + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + cur_tsc = rte_rdtsc(); + + return cur_tsc - prev_tsc; +} + +static uint64_t +measure_rxonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + + cur_tsc = rte_rdtsc(); + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + diff_tsc += rte_rdtsc() - cur_tsc; + + count += nb_rx; + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +static uint64_t +measure_txonly(struct lcore_conf *conf, + struct rte_mbuf *pkts_burst[], + uint64_t total_pkts) +{ + unsigned i, portid, nb_rx, nb_tx; + uint64_t diff_tsc, cur_tsc; + + printf("do tx measure\n"); + diff_tsc = 0; + while (likely(!stop)) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + idle++; + continue; + } + + count += nb_rx; + + cur_tsc = rte_rdtsc(); + nb_tx = rte_eth_tx_burst(portid, 0, pkts_burst, nb_rx); + if (unlikely(nb_tx < nb_rx)) { + drop += (nb_rx - nb_tx); + do { + rte_pktmbuf_free(pkts_burst[nb_tx]); + } while (++nb_tx < nb_rx); + } + diff_tsc += rte_rdtsc() - cur_tsc; + } + if (unlikely(count >= total_pkts)) + break; + } + + return diff_tsc; +} + +/* main processing loop */ +static int +main_loop(__rte_unused void *args) +{ +#define PACKET_SIZE 64 +#define FRAME_GAP 12 +#define MAC_PREAMBLE 8 + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + unsigned lcore_id; + unsigned i, portid, nb_rx = 0, nb_tx = 0; + struct lcore_conf *conf; + int pkt_per_port; + uint64_t diff_tsc; + uint64_t packets_per_second, total_packets; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + pkt_per_port = MAX_TRAFFIC_BURST; + + int idx = 0; + for (i = 0; i < conf->nb_ports; i++) { + int num = pkt_per_port; + portid = conf->portlist[i]; + printf("inject %d packet to port %d\n", num, portid); + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + nb_tx = rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + num -= nb_tx; + idx += nb_tx; + } + } + printf("Total packets inject to prime ports = %u\n", idx); + + packets_per_second = (link_mbps * 1000 * 1000) / + ((PACKET_SIZE + FRAME_GAP + MAC_PREAMBLE) * CHAR_BIT); + printf("Each port will do %"PRIu64" packets per second\n", + packets_per_second); + + total_packets = RTE_TEST_DURATION * conf->nb_ports * packets_per_second; + printf("Test will stop after at least %"PRIu64" packets received\n", + + total_packets); + + diff_tsc = do_measure(conf, pkts_burst, total_packets); + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + int nb_free = pkt_per_port; + do { /* dry out */ + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + pkts_burst, MAX_PKT_BURST); + nb_tx = 0; + while (nb_tx < nb_rx) + rte_pktmbuf_free(pkts_burst[nb_tx++]); + nb_free -= nb_rx; + } while (nb_free != 0); + printf("free %d mbuf left in port %u\n", pkt_per_port, portid); + } + + if (count == 0) + return -1; + + printf("%"PRIu64" packet, %"PRIu64" drop, %"PRIu64" idle\n", + count, drop, idle); + printf("Result: %"PRIu64" cycles per packet\n", diff_tsc / count); + + return 0; +} + +rte_atomic64_t start; + +static inline int +poll_burst(void *args) +{ +#define MAX_IDLE (10000) + unsigned lcore_id; + struct rte_mbuf **pkts_burst; + uint64_t diff_tsc, cur_tsc; + uint16_t next[RTE_MAX_ETHPORTS]; + struct lcore_conf *conf; + uint32_t pkt_per_port = *((uint32_t *)args); + unsigned i, portid, nb_rx = 0; + uint64_t total; + uint64_t timeout = MAX_IDLE; + + lcore_id = rte_lcore_id(); + conf = &lcore_conf[lcore_id]; + if (conf->status != LCORE_USED) + return 0; + + total = pkt_per_port * conf->nb_ports; + printf("start to receive total expect %"PRIu64"\n", total); + + pkts_burst = (struct rte_mbuf **) + rte_calloc_socket("poll_burst", + total, sizeof(void *), + CACHE_LINE_SIZE, conf->socketid); + if (!pkts_burst) + return -1; + + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + next[portid] = i * pkt_per_port; + } + + while (!rte_atomic64_read(&start)) + ; + + cur_tsc = rte_rdtsc(); + while (total) { + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, + &pkts_burst[next[portid]], + MAX_PKT_BURST); + if (unlikely(nb_rx == 0)) { + timeout--; + if (unlikely(timeout == 0)) + goto timeout; + continue; + } + next[portid] += nb_rx; + total -= nb_rx; + } + } +timeout: + diff_tsc = rte_rdtsc() - cur_tsc; + + printf("%"PRIu64" packets lost, IDLE %"PRIu64" times\n", + total, MAX_IDLE - timeout); + + /* clean up */ + total = pkt_per_port * conf->nb_ports - total; + for (i = 0; i < total; i++) + rte_pktmbuf_free(pkts_burst[i]); + + rte_free(pkts_burst); + + return diff_tsc / total; +} + +static int +exec_burst(uint32_t flags, int lcore) +{ + unsigned i, portid, nb_tx = 0; + struct lcore_conf *conf; + uint32_t pkt_per_port; + int num, idx = 0; + int diff_tsc; + + conf = &lcore_conf[lcore]; + + pkt_per_port = MAX_TRAFFIC_BURST; + num = pkt_per_port; + + rte_atomic64_init(&start); + + /* start polling thread, but not actually poll yet */ + rte_eal_remote_launch(poll_burst, + (void *)&pkt_per_port, lcore); + + /* Only when polling first */ + if (flags == SC_BURST_POLL_FIRST) + rte_atomic64_set(&start, 1); + + /* start xmit */ + while (num) { + nb_tx = RTE_MIN(MAX_PKT_BURST, num); + for (i = 0; i < conf->nb_ports; i++) { + portid = conf->portlist[i]; + rte_eth_tx_burst(portid, 0, + &tx_burst[idx], nb_tx); + idx += nb_tx; + } + num -= nb_tx; + } + + sleep(5); + + /* only when polling second */ + if (flags == SC_BURST_XMIT_FIRST) + rte_atomic64_set(&start, 1); + + /* wait for polling finished */ + diff_tsc = rte_eal_wait_lcore(lcore); + if (diff_tsc < 0) + return -1; + + printf("Result: %d cycles per packet\n", diff_tsc); + + return 0; +} + +static int +test_pmd_perf(void) +{ + uint16_t nb_ports, num, nb_lcores, slave_id = (uint16_t)-1; + uint16_t nb_rxd = MAX_TRAFFIC_BURST; + uint16_t nb_txd = MAX_TRAFFIC_BURST; + uint16_t portid; + uint16_t nb_rx_queue = 1, nb_tx_queue = 1; + int socketid = -1; + int ret; + + printf("Start PMD RXTX cycles cost test.\n"); + + signal(SIGUSR1, signal_handler); + signal(SIGUSR2, signal_handler); + + nb_ports = rte_eth_dev_count(); + if (nb_ports < NB_ETHPORTS_USED) { + printf("At least %u port(s) used for perf. test\n", + NB_ETHPORTS_USED); + return -1; + } + + if (nb_ports > RTE_MAX_ETHPORTS) + nb_ports = RTE_MAX_ETHPORTS; + + nb_lcores = rte_lcore_count(); + + memset(lcore_conf, 0, sizeof(lcore_conf)); + init_lcores(); + + init_mbufpool(NB_MBUF); + + if (sc_flag == SC_CONTINUOUS) { + nb_rxd = RTE_TEST_RX_DESC_DEFAULT; + nb_txd = RTE_TEST_TX_DESC_DEFAULT; + } + printf("CONFIG RXD=%d TXD=%d\n", nb_rxd, nb_txd); + + reset_count(); + num = 0; + for (portid = 0; portid < nb_ports; portid++) { + if (socketid == -1) { + socketid = rte_eth_dev_socket_id(portid); + slave_id = alloc_lcore(socketid); + if (slave_id == (uint16_t)-1) { + printf("No avail lcore to run test\n"); + return -1; + } + printf("Performance test runs on lcore %u socket %u\n", + slave_id, socketid); + } + + if (socketid != rte_eth_dev_socket_id(portid)) { + printf("Skip port %d\n", portid); + continue; + } + + /* port configure */ + ret = rte_eth_dev_configure(portid, nb_rx_queue, + nb_tx_queue, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Cannot configure device: err=%d, port=%d\n", + ret, portid); + + rte_eth_macaddr_get(portid, &ports_eth_addr[portid]); + printf("Port %u ", portid); + print_ethaddr("Address:", &ports_eth_addr[portid]); + printf("\n"); + + /* tx queue setup */ + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + socketid, &tx_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_tx_queue_setup: err=%d, " + "port=%d\n", ret, portid); + + /* rx queue steup */ + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, + socketid, &rx_conf, + mbufpool[socketid]); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d," + "port=%d\n", ret, portid); + + /* Start device */ + stop = 0; + ret = rte_eth_dev_start(portid); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start: err=%d, port=%d\n", + ret, portid); + + /* always eanble promiscuous */ + rte_eth_promiscuous_enable(portid); + + lcore_conf[slave_id].portlist[num++] = portid; + lcore_conf[slave_id].nb_ports++; + } + check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + + if (tx_burst == NULL) { + tx_burst = (struct rte_mbuf **) + rte_calloc_socket("tx_buff", + MAX_TRAFFIC_BURST * nb_ports, + sizeof(void *), + CACHE_LINE_SIZE, socketid); + if (!tx_burst) + return -1; + } + + init_traffic(mbufpool[socketid], + tx_burst, MAX_TRAFFIC_BURST * nb_ports); + + printf("Generate %d packets @socket %d\n", + MAX_TRAFFIC_BURST * nb_ports, socketid); + + if (sc_flag == SC_CONTINUOUS) { + /* do both rxtx by default */ + if (NULL == do_measure) + do_measure = measure_rxtx; + + rte_eal_remote_launch(main_loop, NULL, slave_id); + + if (rte_eal_wait_lcore(slave_id) < 0) + return -1; + } else if (sc_flag == SC_BURST_POLL_FIRST || + sc_flag == SC_BURST_XMIT_FIRST) + exec_burst(sc_flag, slave_id); + + /* port tear down */ + for (portid = 0; portid < nb_ports; portid++) { + if (socketid != rte_eth_dev_socket_id(portid)) + continue; + + rte_eth_dev_stop(portid); + } + + return 0; +} + +int +test_set_rxtx_conf(cmdline_fixed_string_t mode) +{ + printf("mode switch to %s\n", mode); + + if (!strcmp(mode, "vector")) { + /* vector rx, tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "scalar")) { + /* bulk alloc rx, simple tx */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 128; + tx_conf.tx_free_thresh = 128; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "hybrid")) { + /* bulk alloc rx, vector tx + * when vec macro not define, + * using the same rx/tx as scalar + */ + tx_conf.txq_flags = 0xf01; + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 1; + port_conf.rxmode.enable_scatter = 0; + return 0; + } else if (!strcmp(mode, "full")) { + /* full feature rx,tx pair */ + tx_conf.txq_flags = 0x0; /* must condition */ + tx_conf.tx_rs_thresh = 32; + tx_conf.tx_free_thresh = 32; + port_conf.rxmode.hw_ip_checksum = 0; + port_conf.rxmode.enable_scatter = 1; /* must condition */ + return 0; + } + + return -1; +} + +int +test_set_rxtx_anchor(cmdline_fixed_string_t type) +{ + printf("type switch to %s\n", type); + + if (!strcmp(type, "rxtx")) { + do_measure = measure_rxtx; + return 0; + } else if (!strcmp(type, "rxonly")) { + do_measure = measure_rxonly; + return 0; + } else if (!strcmp(type, "txonly")) { + do_measure = measure_txonly; + return 0; + } + + return -1; +} + +int +test_set_rxtx_sc(cmdline_fixed_string_t type) +{ + printf("stream control switch to %s\n", type); + + if (!strcmp(type, "continuous")) { + sc_flag = SC_CONTINUOUS; + return 0; + } else if (!strcmp(type, "poll_before_xmit")) { + sc_flag = SC_BURST_POLL_FIRST; + return 0; + } else if (!strcmp(type, "poll_after_xmit")) { + sc_flag = SC_BURST_XMIT_FIRST; + return 0; + } + + return -1; +} + +static struct test_command pmd_perf_cmd = { + .command = "pmd_perf_autotest", + .callback = test_pmd_perf, +}; +REGISTER_TEST_COMMAND(pmd_perf_cmd); -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [dpdk-dev] [PATCH v7 7/7] ethdev: fix wrong error return refere to API definition 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang ` (5 preceding siblings ...) 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 6/7] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang @ 2014-11-12 6:24 ` Cunming Liang 2014-11-12 23:50 ` [dpdk-dev] [PATCH v7 0/7] app/test: unit test to measure cycles per packet Thomas Monjalon 7 siblings, 0 replies; 65+ messages in thread From: Cunming Liang @ 2014-11-12 6:24 UTC (permalink / raw) To: dev Per definition, rte_eth_rx_burst/rte_eth_tx_burst/rte_eth_rx_queue_count returns the packet number When RTE_LIBRTE_ETHDEV_DEBUG turns on, retval of FUNC_PTR_OR_ERR_RTE was set to -ENOTSUP. It makes confusing. The patch always return 0 no matter no packet or there's error. Signed-off-by: Cunming Liang <cunming.liang@intel.com> --- lib/librte_ether/rte_ethdev.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 5e9d576..8c65d72 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -2586,7 +2586,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, return 0; } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); return 0; @@ -2607,7 +2607,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); if (queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", queue_id); return 0; @@ -2626,7 +2626,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) return 0; } dev = &rte_eth_devices[port_id]; - FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); } -- 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v7 0/7] app/test: unit test to measure cycles per packet 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang ` (6 preceding siblings ...) 2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 7/7] ethdev: fix wrong error return refere to API definition Cunming Liang @ 2014-11-12 23:50 ` Thomas Monjalon 7 siblings, 0 replies; 65+ messages in thread From: Thomas Monjalon @ 2014-11-12 23:50 UTC (permalink / raw) To: Cunming Liang; +Cc: dev 2014-11-12 14:24, Cunming Liang: > v7 update: > # patch split and re-orginize > > v6 update: > # leave FUNC_PTR_OR_*_RET unmodified > > v5 update: > # fix the confusing of retval in some API of rte_ethdev > > v4 ignore > > v3 update: > # Codes refine according to the feedback. > 1. add ether_format_addr to rte_ether.h > 2. fix typo in code comments. > 3. %lu to %PRIu64, fixing 32-bit targets compilation err > # merge 2 small incremental patches to the first one. > The whole unit test as a single patch in [PATCH v3 2/2] > # rebase code to the latest master > > v2 update: > Rebase code to the latest master branch. > > It provides unit test to measure cycles/packet in NIC loopback mode. > It simply gives the average cycles of IO used per packet without test equipment. > When doing the test, make sure the link is UP. > > There's two stream control mode support, one is continues, another is burst. > The former continues to forward the injected packets until reaching a certain amount of number. > The latter one stop when all the injected packets are received. > In burst stream, now measure two situations, with or without desc. cache conflict. > By default, it runs in continues stream mode to measure the whole rxtx. > > Usage Example: > 1. Run unit test app in interactive mode > app/test -c f -n 4 -- -i > 2. Set stream control mode, by default is continuous > set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] > 3. If choose continuous stream, there are another two options can configure > 3.1 choose rx/tx pair, default is vector > set_rxtx_mode [vector|scalar|full|hybrid] > Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without INC_VEC=y in config > 3.2 choose the area of masurement, default is rxtx > set_rxtx_anchor [rxtx|rxonly|txonly] > 4. Run and wait for the result > pmd_perf_autotest > > For who simply just want to see how much cycles cost per packet. > Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. > Nothing else needs to configure. > Using other options when you understand and what to measures more. Applied Thanks -- Thomas ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v4 0/3] app/test: unit test to measure cycles per packet 2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 0/3] " Cunming Liang ` (3 preceding siblings ...) 2014-10-24 5:57 ` [dpdk-dev] [PATCH v5 0/3] app/test: unit test to measure cycles per packet Cunming Liang @ 2014-10-24 5:59 ` Liang, Cunming [not found] ` <1414130090-17910-1-git-send-email-y> 5 siblings, 0 replies; 65+ messages in thread From: Liang, Cunming @ 2014-10-24 5:59 UTC (permalink / raw) To: dev Sorry, just ignore this version. > -----Original Message----- > From: Liang, Cunming > Sent: Friday, October 24, 2014 1:40 PM > To: dev@dpdk.org > Cc: nhorman@tuxdriver.com; Richardson, Bruce; Ananyev, Konstantin; De Lara > Guarch, Pablo; Liang, Cunming > Subject: [PATCH v4 0/3] app/test: unit test to measure cycles per packet > Importance: High > > BTW, [1/3] is the same patch as below one. > http://dpdk.org/dev/patchwork/patch/817 > > v4 update: > # fix the confusing of retval in some API of rte_ethdev > > v3 update: > # Codes refine according to the feedback. > 1. add ether_format_addr to rte_ether.h > 2. fix typo in code comments. > 3. %lu to %PRIu64, fixing 32-bit targets compilation err > # merge 2 small incremental patches to the first one. > The whole unit test as a single patch in [PATCH v3 2/2] > # rebase code to the latest master > > v2 update: > Rebase code to the latest master branch. > > It provides unit test to measure cycles/packet in NIC loopback mode. > It simply gives the average cycles of IO used per packet without test equipment. > When doing the test, make sure the link is UP. > > There's two stream control mode support, one is continues, another is burst. > The former continues to forward the injected packets until reaching a certain > amount of number. > The latter one stop when all the injected packets are received. > In burst stream, now measure two situations, with or without desc. cache conflict. > By default, it runs in continues stream mode to measure the whole rxtx. > > Usage Example: > 1. Run unit test app in interactive mode > app/test -c f -n 4 -- -i > 2. Set stream control mode, by default is continuous > set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] > 3. If choose continuous stream, there are another two options can configure > 3.1 choose rx/tx pair, default is vector > set_rxtx_mode [vector|scalar|full|hybrid] > Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without > INC_VEC=y in config > 3.2 choose the area of masurement, default is rxtx > set_rxtx_anchor [rxtx|rxonly|txonly] > 4. Run and wait for the result > pmd_perf_autotest > > For who simply just want to see how much cycles cost per packet. > Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. > Nothing else needs to configure. > Using other options when you understand and what to measures more. > > > *** BLURB HERE *** > > Cunming Liang (3): > app/test: allow to create packets in different sizes > app/test: measure the cost of rx/tx routines by cycle number > ethdev: fix wrong error return refer to API definition > > app/test/Makefile | 1 + > app/test/commands.c | 111 +++++ > app/test/packet_burst_generator.c | 26 +- > app/test/packet_burst_generator.h | 11 +- > app/test/test.h | 6 + > app/test/test_link_bonding.c | 39 +- > app/test/test_pmd_perf.c | 922 > +++++++++++++++++++++++++++++++++++ > lib/librte_ether/rte_ethdev.c | 6 +- > lib/librte_ether/rte_ether.h | 25 + > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > 10 files changed, 1117 insertions(+), 36 deletions(-) > create mode 100644 app/test/test_pmd_perf.c > > -- > 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
[parent not found: <1414130090-17910-1-git-send-email-y>]
[parent not found: <1414130090-17910-4-git-send-email-y>]
* Re: [dpdk-dev] [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition [not found] ` <1414130090-17910-4-git-send-email-y> @ 2014-10-24 11:04 ` Ananyev, Konstantin 2014-10-27 0:58 ` Liang, Cunming 0 siblings, 1 reply; 65+ messages in thread From: Ananyev, Konstantin @ 2014-10-24 11:04 UTC (permalink / raw) To: dev > -----Original Message----- > From: y@ecsmtp.sh.intel.com [mailto:y@ecsmtp.sh.intel.com] > Sent: Friday, October 24, 2014 6:55 AM > To: dev@dpdk.org > Cc: nhorman@tuxdriver.com; Richardson, Bruce; Ananyev, Konstantin; De Lara Guarch, Pablo; Liang, Cunming > Subject: [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition > > From: Cunming Liang <cunming.liang@intel.com> > > Per definition, rte_eth_rx_burst/rte_eth_tx_burst/rte_eth_rx_queue_count returns the packet number. > When RTE_LIBRTE_ETHDEV_DEBUG turns on, retval of FUNC_PTR_OR_ERR_RTE was set to -ENOTSUP. > It makes confusing. > The patch always return 0 no matter no packet or there's error. > Meanwhile set errno in such kind of checking. > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > --- > lib/librte_ether/rte_ethdev.c | 10 +++++++--- > 1 files changed, 7 insertions(+), 3 deletions(-) > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c > index 50f10d9..6675f28 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -81,12 +81,14 @@ > /* Macros for checking for restricting functions to primary instance only */ > #define PROC_PRIMARY_OR_ERR_RET(retval) do { \ > if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \ > + rte_errno = -E_RTE_SECONDARY; \ > PMD_DEBUG_TRACE("Cannot run in secondary processes\n"); \ > return (retval); \ > } \ > } while(0) > #define PROC_PRIMARY_OR_RET() do { \ > if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \ > + rte_errno = -E_RTE_SECONDARY; \ > PMD_DEBUG_TRACE("Cannot run in secondary processes\n"); \ > return; \ > } \ > @@ -95,12 +97,14 @@ > /* Macros to check for invlaid function pointers in dev_ops structure */ > #define FUNC_PTR_OR_ERR_RET(func, retval) do { \ > if ((func) == NULL) { \ > + rte_errno = -ENOTSUP; \ > PMD_DEBUG_TRACE("Function not supported\n"); \ > return (retval); \ > } \ > } while(0) > #define FUNC_PTR_OR_RET(func) do { \ > if ((func) == NULL) { \ > + rte_errno = -ENOTSUP; \ > PMD_DEBUG_TRACE("Function not supported\n"); \ > return; \ > } \ > @@ -2530,7 +2534,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, > return 0; > } > dev = &rte_eth_devices[port_id]; > - FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); > + FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); > if (queue_id >= dev->data->nb_rx_queues) { > PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); > return 0; > @@ -2551,7 +2555,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, > } > dev = &rte_eth_devices[port_id]; > > - FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, -ENOTSUP); > + FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); > if (queue_id >= dev->data->nb_tx_queues) { > PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", queue_id); > return 0; > @@ -2570,7 +2574,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) > return 0; > } > dev = &rte_eth_devices[port_id]; > - FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, -ENOTSUP); > + FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); > return (*dev->dev_ops->rx_queue_count)(dev, queue_id); > } There are few things that worry me with that approach: 1. Different behaviour of rte_eth_rx_burst/rte_eth_tx_burst for RTE_LIBRTE_ETHDEV_DEBUG switched on/off. So application might need to differentiate its code depending on RTE_LIBRTE_ETHDEV_DEBUG value. 2. Even for RTE_LIBRTE_ETHDEV_DEBUG is on the behaviour of rte_eth_rx_burst/ rte_eth_tx_burst will be inconsistent: It sets rte_errno if dev->rx_pkt_burst == NULL, but doesn't do the same for other error conditions: When port_id or queue_id is invalid. 3. Modifying FUNC_PTR_OR_ERR_RET() to set rte_errno, we make behaviour of other rte_ethdev functions inconsistent too: Now for some error conditions they do set rte_errno, for others they don't. So if it would be me, I'll just: - leave FUNC_PTR_OR_*_RET unmodified. - changes rte_eth_rx_burst/tx_burst for RTE_LIBRTE_ETHDEV_DEBUG something like: - FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); + FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); I think, that just error logging is enough here. Konstantin > > -- > 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition 2014-10-24 11:04 ` [dpdk-dev] [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition Ananyev, Konstantin @ 2014-10-27 0:58 ` Liang, Cunming 0 siblings, 0 replies; 65+ messages in thread From: Liang, Cunming @ 2014-10-27 0:58 UTC (permalink / raw) To: Ananyev, Konstantin, dev Ok, I'll roll back to v4. > -----Original Message----- > From: Ananyev, Konstantin > Sent: Friday, October 24, 2014 7:05 PM > To: dev@dpdk.org > Cc: nhorman@tuxdriver.com; Richardson, Bruce; De Lara Guarch, Pablo; Liang, > Cunming > Subject: RE: [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition > > > > > -----Original Message----- > > From: y@ecsmtp.sh.intel.com [mailto:y@ecsmtp.sh.intel.com] > > Sent: Friday, October 24, 2014 6:55 AM > > To: dev@dpdk.org > > Cc: nhorman@tuxdriver.com; Richardson, Bruce; Ananyev, Konstantin; De Lara > Guarch, Pablo; Liang, Cunming > > Subject: [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition > > > > From: Cunming Liang <cunming.liang@intel.com> > > > > Per definition, rte_eth_rx_burst/rte_eth_tx_burst/rte_eth_rx_queue_count > returns the packet number. > > When RTE_LIBRTE_ETHDEV_DEBUG turns on, retval of FUNC_PTR_OR_ERR_RTE > was set to -ENOTSUP. > > It makes confusing. > > The patch always return 0 no matter no packet or there's error. > > Meanwhile set errno in such kind of checking. > > > > Signed-off-by: Cunming Liang <cunming.liang@intel.com> > > --- > > lib/librte_ether/rte_ethdev.c | 10 +++++++--- > > 1 files changed, 7 insertions(+), 3 deletions(-) > > > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c > > index 50f10d9..6675f28 100644 > > --- a/lib/librte_ether/rte_ethdev.c > > +++ b/lib/librte_ether/rte_ethdev.c > > @@ -81,12 +81,14 @@ > > /* Macros for checking for restricting functions to primary instance only */ > > #define PROC_PRIMARY_OR_ERR_RET(retval) do { \ > > if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \ > > + rte_errno = -E_RTE_SECONDARY; \ > > PMD_DEBUG_TRACE("Cannot run in secondary processes\n"); \ > > return (retval); \ > > } \ > > } while(0) > > #define PROC_PRIMARY_OR_RET() do { \ > > if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \ > > + rte_errno = -E_RTE_SECONDARY; \ > > PMD_DEBUG_TRACE("Cannot run in secondary processes\n"); \ > > return; \ > > } \ > > @@ -95,12 +97,14 @@ > > /* Macros to check for invlaid function pointers in dev_ops structure */ > > #define FUNC_PTR_OR_ERR_RET(func, retval) do { \ > > if ((func) == NULL) { \ > > + rte_errno = -ENOTSUP; \ > > PMD_DEBUG_TRACE("Function not supported\n"); \ > > return (retval); \ > > } \ > > } while(0) > > #define FUNC_PTR_OR_RET(func) do { \ > > if ((func) == NULL) { \ > > + rte_errno = -ENOTSUP; \ > > PMD_DEBUG_TRACE("Function not supported\n"); \ > > return; \ > > } \ > > @@ -2530,7 +2534,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t > queue_id, > > return 0; > > } > > dev = &rte_eth_devices[port_id]; > > - FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); > > + FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); > > if (queue_id >= dev->data->nb_rx_queues) { > > PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", queue_id); > > return 0; > > @@ -2551,7 +2555,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t > queue_id, > > } > > dev = &rte_eth_devices[port_id]; > > > > - FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, -ENOTSUP); > > + FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); > > if (queue_id >= dev->data->nb_tx_queues) { > > PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", queue_id); > > return 0; > > @@ -2570,7 +2574,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t > queue_id) > > return 0; > > } > > dev = &rte_eth_devices[port_id]; > > - FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, -ENOTSUP); > > + FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); > > return (*dev->dev_ops->rx_queue_count)(dev, queue_id); > > } > > There are few things that worry me with that approach: > > 1. Different behaviour of rte_eth_rx_burst/rte_eth_tx_burst for > RTE_LIBRTE_ETHDEV_DEBUG switched on/off. > So application might need to differentiate its code depending on > RTE_LIBRTE_ETHDEV_DEBUG value. > > 2. Even for RTE_LIBRTE_ETHDEV_DEBUG is on the behaviour of rte_eth_rx_burst/ > rte_eth_tx_burst will be inconsistent: > It sets rte_errno if dev->rx_pkt_burst == NULL, but doesn't do the same for other > error conditions: > When port_id or queue_id is invalid. > > 3. Modifying FUNC_PTR_OR_ERR_RET() to set rte_errno, we make behaviour of > other rte_ethdev functions inconsistent too: > Now for some error conditions they do set rte_errno, for others they don't. > > So if it would be me, I'll just: > - leave FUNC_PTR_OR_*_RET unmodified. > - changes rte_eth_rx_burst/tx_burst for RTE_LIBRTE_ETHDEV_DEBUG something > like: > > - FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, -ENOTSUP); > + FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); > > I think, that just error logging is enough here. > > Konstantin > > > > > -- > > 1.7.4.1 ^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/4] app/test: unit test to measure cycles per packet 2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 0/4] " Cunming Liang ` (4 preceding siblings ...) 2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Cunming Liang @ 2014-10-28 12:21 ` Neil Horman 5 siblings, 0 replies; 65+ messages in thread From: Neil Horman @ 2014-10-28 12:21 UTC (permalink / raw) To: Cunming Liang; +Cc: dev On Fri, Oct 10, 2014 at 08:29:57PM +0800, Cunming Liang wrote: > v2 update: > Rebase code to the latest master branch. > > -------------- > > Tested-by: Zhaochen Zhan <zhaochen.zhan@intel.com> > This patch has been verified on ixgbe and it is ready to be integrated to dpdk.org. > For e1000 and i40e, lacking loopback supporting, this patch can't support them for now. > > It provides unit test to measure cycles/packet in NIC loopback mode. > It simply gives the average cycles of IO used per packet without test equipment. > When doing the test, make sure the link is UP. > > There's two stream control mode support, one is continues, another is burst. > The former continues to forward the injected packets until reaching a certain amount of number. > The latter one stop when all the injected packets are received. > In burst stream, now measure two situations, with or without desc. cache conflict. > By default, it runs in continues stream mode to measure the whole rxtx. > > Usage Example: > 1. Run unit test app in interactive mode > app/test -c f -n 4 -- -i > 2. Set stream control mode, by default is continuous > set_rxtx_sc [continuous|poll_before_xmit|poll_after_xmit] > 3. If choose continuous stream, there are another two options can configure > 3.1 choose rx/tx pair, default is vector > set_rxtx_mode [vector|scalar|full|hybrid] > Note: To get acurate scalar fast, plz choose 'vector' or 'hybrid' without INC_VEC=y in config > 3.2 choose the area of masurement, default is rxtx > set_rxtx_anchor [rxtx|rxonly|txonly] > 4. Run and wait for the result > pmd_perf_autotest > > For who simply just want to see how much cycles cost per packet. > Compile DPDK, Run 'app/test', and type 'pmd_perf_autotest', that's it. > Nothing else needs to configure. > Using other options when you understand and what to measures more. > > *** BLURB HERE *** > > Cunming Liang (4): > app/test: unit test for rx and tx cycles/packet > app/test: measure standalone rx or tx cycles/packet > app/test: add unit test to measure RX burst cycles > app/test: allow to create packets in different sizes > > app/test/Makefile | 1 + > app/test/commands.c | 111 +++++ > app/test/packet_burst_generator.c | 26 +- > app/test/packet_burst_generator.h | 11 +- > app/test/test.h | 6 + > app/test/test_link_bonding.c | 39 +- > app/test/test_pmd_perf.c | 922 +++++++++++++++++++++++++++++++++++ > lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 6 + > 8 files changed, 1089 insertions(+), 33 deletions(-) > create mode 100644 app/test/test_pmd_perf.c > > -- > 1.7.4.1 > > Given that a subsequent patch was submitted to bring the debug and non-debug versions of rte_eth_rx_burst behavior in line Acked-by: Neil Horman <nhorman@tuxdriver.com> ^ permalink raw reply [flat|nested] 65+ messages in thread
end of thread, other threads:[~2014-11-12 23:41 UTC | newest]
Thread overview: 65+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-08-25 6:12 [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Cunming Liang
2014-08-25 6:12 ` [dpdk-dev] [PATCH 1/5] app/test: unit test for rx and tx cycles/packet Cunming Liang
2014-08-25 6:12 ` [dpdk-dev] [PATCH 2/5] app/test: measure standalone rx or " Cunming Liang
2014-08-25 6:12 ` [dpdk-dev] [PATCH 3/5] ixgbe/vpmd: add fix to store unaligned mbuf point array Cunming Liang
2014-08-25 6:12 ` [dpdk-dev] [PATCH 4/5] app/test: add unit test to measure RX burst cycles Cunming Liang
2014-08-25 6:12 ` [dpdk-dev] [PATCH 5/5] app/test: allow to create packets in different sizes Cunming Liang
2014-09-03 1:46 ` [dpdk-dev] [PATCH 0/5] app/test: unit test to measure cycles per packet Zhan, Zhaochen
2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 0/4] " Cunming Liang
2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 1/4] app/test: unit test for rx and tx cycles/packet Cunming Liang
2014-10-10 17:52 ` Neil Horman
2014-10-12 11:10 ` Liang, Cunming
[not found] ` <E115CCD9D858EF4F90C690B0DCB4D8972262B720@IRSMSX108.ger.corp.intel.com>
2014-10-14 0:54 ` Liang, Cunming
2014-10-21 10:33 ` Neil Horman
2014-10-21 10:43 ` Richardson, Bruce
2014-10-21 13:37 ` Neil Horman
2014-10-21 22:43 ` Ananyev, Konstantin
2014-10-21 13:17 ` Liang, Cunming
2014-10-22 14:03 ` Neil Horman
2014-10-22 14:48 ` Liang, Cunming
2014-10-22 14:53 ` Ananyev, Konstantin
2014-10-22 15:09 ` Richardson, Bruce
2014-10-24 3:06 ` Liang, Cunming
2014-10-10 12:29 ` [dpdk-dev] [PATCH v2 2/4] app/test: measure standalone rx or " Cunming Liang
2014-10-10 12:30 ` [dpdk-dev] [PATCH v2 3/4] app/test: add unit test to measure RX burst cycles Cunming Liang
2014-10-10 12:30 ` [dpdk-dev] [PATCH v2 4/4] app/test: allow to create packets in different sizes Cunming Liang
2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Cunming Liang
2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 1/2] app/test: allow to create packets in different sizes Cunming Liang
2014-10-20 8:13 ` [dpdk-dev] [PATCH v3 2/2] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang
2014-10-21 2:40 ` [dpdk-dev] [PATCH v3 0/2] app/test: unit test to measure cycles per packet Liu, Yong
2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 0/3] " Cunming Liang
2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 1/3] app/test: allow to create packets in different sizes Cunming Liang
2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang
2014-10-24 5:39 ` [dpdk-dev] [PATCH v4 3/3] ethdev: fix wrong error return refer to API definition Cunming Liang
2014-10-24 5:57 ` [dpdk-dev] [PATCH v5 0/3] app/test: unit test to measure cycles per packet Cunming Liang
2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 1/3] app/test: allow to create packets in different sizes Cunming Liang
2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang
2014-10-24 5:58 ` [dpdk-dev] [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition Cunming Liang
2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Cunming Liang
2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 1/3] app/test: allow to create packets in different sizes Cunming Liang
2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 2/3] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang
2014-11-11 23:28 ` Thomas Monjalon
2014-11-12 6:32 ` Liang, Cunming
2014-10-27 1:20 ` [dpdk-dev] [PATCH v6 3/3] ethdev: fix wrong error return refere to API definition Cunming Liang
2014-10-27 16:03 ` Ananyev, Konstantin
2014-10-27 1:45 ` [dpdk-dev] [PATCH v6 0/3] app/test: unit test to measure cycles per packet Liu, Yong
2014-10-29 5:06 ` Liang, Cunming
2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 0/7] " Cunming Liang
2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 1/7] app/test: allow to create packets in different sizes Cunming Liang
2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 2/7] ixgbe:clean scattered_rx configure in dev_stop Cunming Liang
2014-11-12 7:53 ` Thomas Monjalon
2014-11-12 8:21 ` Liang, Cunming
2014-11-12 9:24 ` Thomas Monjalon
2014-11-12 10:29 ` Liang, Cunming
2014-11-12 10:32 ` Thomas Monjalon
2014-11-12 10:42 ` Liang, Cunming
2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 3/7] ether: new API to format eth_addr in string Cunming Liang
2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 4/7] app/testpmd: cleanup eth_addr print Cunming Liang
2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 5/7] examples: " Cunming Liang
2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 6/7] app/test: measure the cost of rx/tx routines by cycle number Cunming Liang
2014-11-12 6:24 ` [dpdk-dev] [PATCH v7 7/7] ethdev: fix wrong error return refere to API definition Cunming Liang
2014-11-12 23:50 ` [dpdk-dev] [PATCH v7 0/7] app/test: unit test to measure cycles per packet Thomas Monjalon
2014-10-24 5:59 ` [dpdk-dev] [PATCH v4 0/3] " Liang, Cunming
[not found] ` <1414130090-17910-1-git-send-email-y>
[not found] ` <1414130090-17910-4-git-send-email-y>
2014-10-24 11:04 ` [dpdk-dev] [PATCH v5 3/3] ethdev: fix wrong error return refere to API definition Ananyev, Konstantin
2014-10-27 0:58 ` Liang, Cunming
2014-10-28 12:21 ` [dpdk-dev] [PATCH v2 0/4] app/test: unit test to measure cycles per packet Neil Horman
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).