From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id D06AAAD7C for ; Wed, 8 Jun 2016 02:25:12 +0200 (CEST) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga102.fm.intel.com with ESMTP; 07 Jun 2016 17:25:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.26,436,1459839600"; d="scan'208";a="823708567" Received: from irsmsx108.ger.corp.intel.com ([163.33.3.3]) by orsmga003.jf.intel.com with ESMTP; 07 Jun 2016 17:25:10 -0700 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.51]) by IRSMSX108.ger.corp.intel.com ([169.254.11.183]) with mapi id 14.03.0248.002; Wed, 8 Jun 2016 01:25:09 +0100 From: "Ananyev, Konstantin" To: "Lu, Wenzhuo" , "dev@dpdk.org" CC: "Lu, Wenzhuo" Thread-Topic: [dpdk-dev] [PATCH] examples: add a new example for link reset Thread-Index: AQHRv7crPDbbDxXhKEe6mCFtk4nOaJ/et9/w Date: Wed, 8 Jun 2016 00:25:08 +0000 Message-ID: <2601191342CEEE43887BDE71AB97725836B6CBA3@irsmsx105.ger.corp.intel.com> References: <1465192093-28651-1-git-send-email-wenzhuo.lu@intel.com> <1465192093-28651-2-git-send-email-wenzhuo.lu@intel.com> In-Reply-To: <1465192093-28651-2-git-send-email-wenzhuo.lu@intel.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] examples: add a new example for link reset X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jun 2016 00:25:13 -0000 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu > Sent: Monday, June 06, 2016 6:48 AM > To: dev@dpdk.org > Cc: Lu, Wenzhuo > Subject: [dpdk-dev] [PATCH] examples: add a new example for link reset >=20 > Add a new example to show when the PF is down and up, > VF port can be reset and recover. Do we really need a totally new example for it? Can't we put it in one of already existing ones? Let say we have l3fwd-vf... wouldn't that suit your needs? Konstantin >=20 > Signed-off-by: Wenzhuo Lu > --- > MAINTAINERS | 4 + > doc/guides/sample_app_ug/link_reset.rst | 177 ++++++++ > examples/link_reset/Makefile | 50 +++ > examples/link_reset/main.c | 769 ++++++++++++++++++++++++++= ++++++ > 4 files changed, 1000 insertions(+) > create mode 100644 doc/guides/sample_app_ug/link_reset.rst > create mode 100644 examples/link_reset/Makefile > create mode 100644 examples/link_reset/main.c >=20 > diff --git a/MAINTAINERS b/MAINTAINERS > index 3e8558f..76879c3 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -650,3 +650,7 @@ F: examples/tep_termination/ > F: examples/vmdq/ > F: examples/vmdq_dcb/ > F: doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst > + > +M: Wenzhuo Lu > +F: examples/link_reset/ > +F: doc/guides/sample_app_ug/link_reset.rst > diff --git a/doc/guides/sample_app_ug/link_reset.rst b/doc/guides/sample_= app_ug/link_reset.rst > new file mode 100644 > index 0000000..fecae6d > --- /dev/null > +++ b/doc/guides/sample_app_ug/link_reset.rst > @@ -0,0 +1,177 @@ > +.. BSD LICENSE > + Copyright(c) 2010-2016 Intel Corporation. All rights reserved. > + All rights reserved. > + > + Redistribution and use in source and binary forms, with or without > + modification, are permitted provided that the following conditions > + are met: > + > + * Redistributions of source code must retain the above copyright > + notice, this list of conditions and the following disclaimer. > + * Redistributions in binary form must reproduce the above copyright > + notice, this list of conditions and the following disclaimer in > + the documentation and/or other materials provided with the > + distribution. > + * Neither the name of Intel Corporation nor the names of its > + contributors may be used to endorse or promote products derived > + from this software without specific prior written permission. > + > + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FO= R > + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL= , > + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE= , > + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON AN= Y > + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE US= E > + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +Link Reset Sample Application (in Virtualized Environments) > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +The Link Reset sample application is a simple example of VF traffic reco= very > +using the Data Plane Development Kit (DPDK) which also takes advantage o= f Single > +Root I/O Virtualization (SR-IOV) features in a virtualized environment. > + > +Overview > +-------- > + > +The Link Reset sample application, which should operate in virtualized > +environments, performs L2 forwarding for each packet that is received on= an > +RX_PORT. > +This example is extended from the L2 forwarding example. Please referenc= e the > +example of L2 forwarding in virtualized environments for more details an= d > +explanation about the behavior of forwarding and how to setup the test. > +The purpose of this example is to show when the PF port is down and up, = the VF > +port can recover and the traffic can recover too. > + > +Virtual Function Setup Instructions > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +This application can use the virtual function available in the system an= d > +therefore can be used in a virtual machine without passing through > +the whole Network Device into a guest machine in a virtualized scenario. > +The virtual functions can be enabled in the host machine or the hypervis= or > +with the respective physical function driver. > + > +For example, in a Linux* host machine, it is possible to enable a virtua= l > +function using the following command: > + > +.. code-block:: console > + > + modprobe ixgbe max_vfs=3D2,2 > + > +This command enables two Virtual Functions on each of Physical Function = of the > +NIC, with two physical ports in the PCI configuration space. > +It is important to note that enabled Virtual Function 0 and 2 would belo= ng to > +Physical Function 0 and Virtual Function 1 and 3 would belong to Physica= l > +Function 1, in this case enabling a total of four Virtual Functions. > + > +Compiling the Application > +------------------------- > + > +#. Go to the example directory: > + > + .. code-block:: console > + > + export RTE_SDK=3D/path/to/rte_sdk > + cd ${RTE_SDK}/examples/link_reset > + > +#. Set the target (a default target is used if not specified). For exam= ple: > + > + .. code-block:: console > + > + export RTE_TARGET=3Dx86_64-native-linuxapp-gcc > + > + *See the DPDK Getting Started Guide* for possible RTE_TARGET values. > + > +#. Build the application: > + > + .. code-block:: console > + > + make > + > +Running the Application > +----------------------- > + > +The application requires a number of command line options: > + > +.. code-block:: console > + > + ./build/link_reset [EAL options] -- -p PORTMASK [-q NQ] > + > +where, > + > +* p PORTMASK: A hexadecimal bitmask of the ports to configure > + > +* q NQ: A number of queues (=3Dports) per lcore (default is 1) > + > +To run the application in linuxapp environment with 4 lcores, 16 ports a= nd 8 RX > +queues per lcore, issue the command: > + > +.. code-block:: console > + > + $ ./build/link_reset -c f -n 4 -- -q 8 -p ffff > + > +Refer to the *DPDK Getting Started Guide* for general information on run= ning applications > +and the Environment Abstraction Layer (EAL) options. > + > +Explanation > +----------- > + > +Handle VF link reset event > +~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +In main function, when initialising each port, register a callback for r= eset > +event. > + > +.. code-block:: c > + > + /* Initialise each port */ > + for (portid =3D 0; portid < nb_ports; portid++) { > + ...... > + /* register reset interrupt callback */ > + rte_eth_dev_callback_register(portid, > + RTE_ETH_EVENT_INTR_RESET, reset_event_callback, = NULL); > + ...... > + } > + > +The callback function *reset_event_callback* will be executed in the > +interruption thread. But we want the event to be handled in the manageme= nt > +thread. So in the callback function only a reset flag *stop_forwarding* = is set. > + > +.. code-block:: c > + > + static void > + reset_event_callback(uint8_t port_id, enum rte_eth_event_type ty= pe, void *param) > + { > + RTE_SET_USED(param); > + > + printf("\n\nIn registered callback...\n"); > + printf("Event type: %s on port %d\n", > + type =3D=3D RTE_ETH_EVENT_INTR_RESET ? "RESET in= terrupt" : > + "unknown event", port_id); > + reset_port =3D port_id; > + rte_compiler_barrier(); /* prevent compiler reordering */ > + stop_forwarding =3D 1; > + } > + > +The management thread keeps checking the reset flag *stop_forwarding* to= see > +if VF port reset is needed. If so, it should call the API *rte_eth_dev_r= eset* > +to reset VF port. After that, the traffic will recover. > + > +.. code-block:: c > + > + while (1) { > + rte_delay_ms(1000); > + printf(".."); > + if (stop_forwarding =3D=3D 1) { > + printf("\nreset\n"); > + rte_eth_dev_reset(reset_port); > + stop_forwarding =3D 0; > + } > + if (force_quit) > + break; > + } > diff --git a/examples/link_reset/Makefile b/examples/link_reset/Makefile > new file mode 100644 > index 0000000..6a63256 > --- /dev/null > +++ b/examples/link_reset/Makefile > @@ -0,0 +1,50 @@ > +# BSD LICENSE > +# > +# Copyright(c) 2010-2016 Intel Corporation. All rights reserved. > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without > +# modification, are permitted provided that the following conditions > +# are met: > +# > +# * Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyrigh= t > +# notice, this list of conditions and the following disclaimer in > +# the documentation and/or other materials provided with the > +# distribution. > +# * Neither the name of Intel Corporation nor the names of its > +# contributors may be used to endorse or promote products derived > +# from this software without specific prior written permission. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FO= R > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL= , > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE= , > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON AN= Y > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE US= E > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +ifeq ($(RTE_SDK),) > +$(error "Please define RTE_SDK environment variable") > +endif > + > +# Default target, can be overridden by command line or environment > +RTE_TARGET ?=3D x86_64-native-linuxapp-gcc > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +# binary name > +APP =3D link_reset > + > +# all source are stored in SRCS-y > +SRCS-y :=3D main.c > + > +CFLAGS +=3D -O3 > +CFLAGS +=3D $(WERROR_FLAGS) > + > +include $(RTE_SDK)/mk/rte.extapp.mk > diff --git a/examples/link_reset/main.c b/examples/link_reset/main.c > new file mode 100644 > index 0000000..f151a83 > --- /dev/null > +++ b/examples/link_reset/main.c > @@ -0,0 +1,769 @@ > +/*- > + * BSD LICENSE > + * > + * Copyright(c) 2010-2016 Intel Corporation. All rights reserved. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * > + * * Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * * Redistributions in binary form must reproduce the above copyrig= ht > + * notice, this list of conditions and the following disclaimer in > + * the documentation and/or other materials provided with the > + * distribution. > + * * Neither the name of Intel Corporation nor the names of its > + * contributors may be used to endorse or promote products derived > + * from this software without specific prior written permission. > + * > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS F= OR > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGH= T > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTA= L, > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON A= NY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE U= SE > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE= . > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +static volatile bool force_quit; > + > +#define RTE_LOGTYPE_L2FWD RTE_LOGTYPE_USER1 > + > +#define NB_MBUF 8192 > + > +#define MAX_PKT_BURST 32 > +#define BURST_TX_DRAIN_US 100 /* TX drain every ~100us */ > + > +/* > + * Configurable number of RX/TX ring descriptors > + */ > +#define RTE_TEST_RX_DESC_DEFAULT 128 > +#define RTE_TEST_TX_DESC_DEFAULT 512 > +static uint16_t nb_rxd =3D RTE_TEST_RX_DESC_DEFAULT; > +static uint16_t nb_txd =3D RTE_TEST_TX_DESC_DEFAULT; > + > +/* ethernet addresses of ports */ > +static struct ether_addr l2fwd_ports_eth_addr[RTE_MAX_ETHPORTS]; > + > +/* mask of enabled ports */ > +static uint32_t l2fwd_enabled_port_mask; > + > +/* list of enabled ports */ > +static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS]; > + > +static unsigned int l2fwd_rx_queue_per_lcore =3D 1; > + > +#define MAX_RX_QUEUE_PER_LCORE 16 > +#define MAX_TX_QUEUE_PER_PORT 16 > +struct lcore_queue_conf { > + unsigned n_rx_port; > + unsigned rx_port_list[MAX_RX_QUEUE_PER_LCORE]; > +} __rte_cache_aligned; > +struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE]; > + > +static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS]; > + > +static const struct rte_eth_conf port_conf =3D { > + .rxmode =3D { > + .split_hdr_size =3D 0, > + .header_split =3D 0, /**< Header Split disabled */ > + .hw_ip_checksum =3D 0, /**< IP checksum offload disabled */ > + .hw_vlan_filter =3D 0, /**< VLAN filtering disabled */ > + .jumbo_frame =3D 0, /**< Jumbo Frame Support disabled */ > + .hw_strip_crc =3D 0, /**< CRC stripped by hardware */ > + .lock_mode =3D 1, /**< Use lock mode in RX */ > + }, > + .txmode =3D { > + .mq_mode =3D ETH_MQ_TX_NONE, > + .lock_mode =3D 1, /**< Use lock mode in TX */ > + }, > +}; > + > +struct rte_mempool *l2fwd_pktmbuf_pool; > + > +/* Per-port statistics struct */ > +struct l2fwd_port_statistics { > + uint64_t tx; > + uint64_t rx; > + uint64_t dropped; > +} __rte_cache_aligned; > +struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS]; > + > +/* A tsc-based timer responsible for triggering statistics printout */ > +#define TIMER_MILLISECOND 2000000ULL /* around 1ms at 2 Ghz */ > +#define MAX_TIMER_PERIOD 86400 /* 1 day max */ > +/* default period is 10 seconds */ > +static int64_t timer_period =3D 10 * TIMER_MILLISECOND * 1000; > + > +static uint32_t stop_forwarding; > +static uint8_t reset_port; > + > +/* Print out statistics on packets dropped */ > +static void > +print_stats(void) > +{ > + uint64_t total_packets_dropped, total_packets_tx, total_packets_rx; > + unsigned portid; > + > + total_packets_dropped =3D 0; > + total_packets_tx =3D 0; > + total_packets_rx =3D 0; > + > + const char clr[] =3D { 27, '[', '2', 'J', '\0' }; > + const char topLeft[] =3D { 27, '[', '1', ';', '1', 'H', '\0' }; > + > + /* Clear screen and move to top left */ > + printf("%s%s", clr, topLeft); > + > + printf("\nPort statistics =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D"); > + > + for (portid =3D 0; portid < RTE_MAX_ETHPORTS; portid++) { > + /* skip disabled ports */ > + if ((l2fwd_enabled_port_mask & (1 << portid)) =3D=3D 0) > + continue; > + printf("\nStatistics for port %u ------------------------------" > + "\nPackets sent: %24"PRIu64 > + "\nPackets received: %20"PRIu64 > + "\nPackets dropped: %21"PRIu64, > + portid, > + port_statistics[portid].tx, > + port_statistics[portid].rx, > + port_statistics[portid].dropped); > + > + total_packets_dropped +=3D port_statistics[portid].dropped; > + total_packets_tx +=3D port_statistics[portid].tx; > + total_packets_rx +=3D port_statistics[portid].rx; > + } > + printf("\nAggregate statistics =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D" > + "\nTotal packets sent: %18"PRIu64 > + "\nTotal packets received: %14"PRIu64 > + "\nTotal packets dropped: %15"PRIu64, > + total_packets_tx, > + total_packets_rx, > + total_packets_dropped); > + printf("\n=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D\n"); > +} > + > +static void > +l2fwd_simple_forward(struct rte_mbuf *m, unsigned portid) > +{ > + struct ether_hdr *eth; > + void *tmp; > + unsigned dst_port; > + int sent; > + struct rte_eth_dev_tx_buffer *buffer; > + > + dst_port =3D l2fwd_dst_ports[portid]; > + eth =3D rte_pktmbuf_mtod(m, struct ether_hdr *); > + > + /* 02:00:00:00:00:xx */ > + tmp =3D ð->d_addr.addr_bytes[0]; > + *((uint64_t *)tmp) =3D 0x000000000002 + ((uint64_t)dst_port << 40); > + > + /* src addr */ > + ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], ð->s_addr); > + > + buffer =3D tx_buffer[dst_port]; > + sent =3D rte_eth_tx_buffer(dst_port, 0, buffer, m); > + if (sent) > + port_statistics[dst_port].tx +=3D sent; > +} > + > +/* main processing loop */ > +static void > +l2fwd_main_loop(void) > +{ > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > + struct rte_mbuf *m; > + int sent; > + unsigned lcore_id; > + uint64_t prev_tsc, diff_tsc, cur_tsc, timer_tsc; > + unsigned i, j, portid, nb_rx; > + struct lcore_queue_conf *qconf; > + const uint64_t drain_tsc =3D (rte_get_tsc_hz() + US_PER_S - 1) / > + US_PER_S * BURST_TX_DRAIN_US; > + struct rte_eth_dev_tx_buffer *buffer; > + > + prev_tsc =3D 0; > + timer_tsc =3D 0; > + > + lcore_id =3D rte_lcore_id(); > + qconf =3D &lcore_queue_conf[lcore_id]; > + > + if (qconf->n_rx_port =3D=3D 0) { > + RTE_LOG(INFO, L2FWD, "lcore %u has nothing to do\n", lcore_id); > + return; > + } > + > + RTE_LOG(INFO, L2FWD, "entering main loop on lcore %u\n", lcore_id); > + > + for (i =3D 0; i < qconf->n_rx_port; i++) { > + > + portid =3D qconf->rx_port_list[i]; > + RTE_LOG(INFO, L2FWD, " -- lcoreid=3D%u portid=3D%u\n", lcore_id, > + portid); > + > + } > + > + while (!force_quit) { > + > + cur_tsc =3D rte_rdtsc(); > + > + /* > + * TX burst queue drain > + */ > + diff_tsc =3D cur_tsc - prev_tsc; > + if (unlikely(diff_tsc > drain_tsc)) { > + > + for (i =3D 0; i < qconf->n_rx_port; i++) { > + portid =3D > + l2fwd_dst_ports[qconf->rx_port_list[i]]; > + buffer =3D tx_buffer[portid]; > + > + sent =3D rte_eth_tx_buffer_flush(portid, > + 0, > + buffer); > + if (sent) > + port_statistics[portid].tx +=3D sent; > + } > + > + /* if timer is enabled */ > + if (timer_period > 0) { > + > + /* advance the timer */ > + timer_tsc +=3D diff_tsc; > + > + /* if timer has reached its timeout */ > + if (unlikely(timer_tsc >=3D > + (uint64_t) timer_period)) { > + /* do this only on master core */ > + if (lcore_id =3D=3D > + rte_get_master_lcore()) { > + print_stats(); > + /* reset the timer */ > + timer_tsc =3D 0; > + } > + } > + } > + > + prev_tsc =3D cur_tsc; > + } > + > + /* > + * Read packet from RX queues > + */ > + for (i =3D 0; i < qconf->n_rx_port; i++) { > + > + portid =3D qconf->rx_port_list[i]; > + nb_rx =3D rte_eth_rx_burst((uint8_t) portid, 0, > + pkts_burst, MAX_PKT_BURST); > + > + port_statistics[portid].rx +=3D nb_rx; > + > + for (j =3D 0; j < nb_rx; j++) { > + m =3D pkts_burst[j]; > + rte_prefetch0(rte_pktmbuf_mtod(m, void *)); > + l2fwd_simple_forward(m, portid); > + } > + } > + } > +} > + > +static int > +l2fwd_launch_one_lcore(__attribute__((unused)) void *dummy) > +{ > + l2fwd_main_loop(); > + return 0; > +} > + > +/* display usage */ > +static void > +l2fwd_usage(const char *prgname) > +{ > + printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n" > + " -p PORTMASK: hexadecimal bitmask of ports to configure\n" > + " -q NQ: number of queue (=3Dports) per lcore (default is 1)\n" > + " -T PERIOD: statistics will be refreshed each PERIOD seconds (0 t= o disable, 10 default, 86400 maximum)\n", > + prgname); > +} > + > +static int > +l2fwd_parse_portmask(const char *portmask) > +{ > + char *end =3D NULL; > + unsigned long pm; > + > + /* parse hexadecimal string */ > + pm =3D strtoul(portmask, &end, 16); > + if ((portmask[0] =3D=3D '\0') || (end =3D=3D NULL) || (*end !=3D '\0')) > + return -1; > + > + if (pm =3D=3D 0) > + return -1; > + > + return pm; > +} > + > +static unsigned int > +l2fwd_parse_nqueue(const char *q_arg) > +{ > + char *end =3D NULL; > + unsigned long n; > + > + /* parse hexadecimal string */ > + n =3D strtoul(q_arg, &end, 10); > + if ((q_arg[0] =3D=3D '\0') || (end =3D=3D NULL) || (*end !=3D '\0')) > + return 0; > + if (n =3D=3D 0) > + return 0; > + if (n >=3D MAX_RX_QUEUE_PER_LCORE) > + return 0; > + > + return n; > +} > + > +static int > +l2fwd_parse_timer_period(const char *q_arg) > +{ > + char *end =3D NULL; > + int n; > + > + /* parse number string */ > + n =3D strtol(q_arg, &end, 10); > + if ((q_arg[0] =3D=3D '\0') || (end =3D=3D NULL) || (*end !=3D '\0')) > + return -1; > + if (n >=3D MAX_TIMER_PERIOD) > + return -1; > + > + return n; > +} > + > +/* Parse the argument given in the command line of the application */ > +static int > +l2fwd_parse_args(int argc, char **argv) > +{ > + int opt, ret; > + char **argvopt; > + int option_index; > + char *prgname =3D argv[0]; > + static struct option lgopts[] =3D { > + {NULL, 0, 0, 0} > + }; > + > + argvopt =3D argv; > + > + while ((opt =3D getopt_long(argc, argvopt, "p:q:T:", > + lgopts, &option_index)) !=3D EOF) { > + > + switch (opt) { > + /* portmask */ > + case 'p': > + l2fwd_enabled_port_mask =3D l2fwd_parse_portmask(optarg); > + if (l2fwd_enabled_port_mask =3D=3D 0) { > + printf("invalid portmask\n"); > + l2fwd_usage(prgname); > + return -1; > + } > + break; > + > + /* nqueue */ > + case 'q': > + l2fwd_rx_queue_per_lcore =3D l2fwd_parse_nqueue(optarg); > + if (l2fwd_rx_queue_per_lcore =3D=3D 0) { > + printf("invalid queue number\n"); > + l2fwd_usage(prgname); > + return -1; > + } > + break; > + > + /* timer period */ > + case 'T': > + timer_period =3D l2fwd_parse_timer_period(optarg) * > + 1000 * TIMER_MILLISECOND; > + if (timer_period < 0) { > + printf("invalid timer period\n"); > + l2fwd_usage(prgname); > + return -1; > + } > + break; > + > + /* long options */ > + case 0: > + l2fwd_usage(prgname); > + return -1; > + > + default: > + l2fwd_usage(prgname); > + return -1; > + } > + } > + > + if (optind >=3D 0) > + argv[optind-1] =3D prgname; > + > + ret =3D optind-1; > + optind =3D 0; /* reset getopt lib */ > + return ret; > +} > + > +/* Check the link status of all ports in up to 9s, and print them finall= y */ > +static void > +check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) > +{ > +#define CHECK_INTERVAL 100 /* 100ms */ > +#define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ > + uint8_t portid, count, all_ports_up, print_flag =3D 0; > + struct rte_eth_link link; > + > + printf("\nChecking link status"); > + fflush(stdout); > + for (count =3D 0; count <=3D MAX_CHECK_TIME; count++) { > + if (force_quit) > + return; > + all_ports_up =3D 1; > + for (portid =3D 0; portid < port_num; portid++) { > + if (force_quit) > + return; > + if ((port_mask & (1 << portid)) =3D=3D 0) > + continue; > + memset(&link, 0, sizeof(link)); > + rte_eth_link_get_nowait(portid, &link); > + /* print link status if flag set */ > + if (print_flag =3D=3D 1) { > + if (link.link_status) > + printf("Port %d Link Up - speed %u " > + "Mbps - %s\n", (uint8_t)portid, > + (unsigned)link.link_speed, > + (link.link_duplex =3D=3D ETH_LINK_FULL_DUPLEX) ? > + ("full-duplex") : ("half-duplex\n")); > + else > + printf("Port %d Link Down\n", > + (uint8_t)portid); > + continue; > + } > + /* clear all_ports_up flag if any link down */ > + if (link.link_status =3D=3D ETH_LINK_DOWN) { > + all_ports_up =3D 0; > + break; > + } > + } > + /* after finally printing all link status, get out */ > + if (print_flag =3D=3D 1) > + break; > + > + if (all_ports_up =3D=3D 0) { > + printf("."); > + fflush(stdout); > + rte_delay_ms(CHECK_INTERVAL); > + } > + > + /* set the print_flag if all ports up or timeout */ > + if (all_ports_up =3D=3D 1 || count =3D=3D (MAX_CHECK_TIME - 1)) { > + print_flag =3D 1; > + printf("done\n"); > + } > + } > +} > + > +static void > +signal_handler(int signum) > +{ > + if (signum =3D=3D SIGINT || signum =3D=3D SIGTERM) { > + printf("\n\nSignal %d received, preparing to exit...\n", > + signum); > + force_quit =3D true; > + } > +} > + > +static void > +reset_event_callback(uint8_t port_id, enum rte_eth_event_type type, void= *param) > +{ > + RTE_SET_USED(param); > + > + printf("\n\nIn registered callback...\n"); > + printf("Event type: %s on port %d\n", > + type =3D=3D RTE_ETH_EVENT_INTR_RESET ? "RESET interrupt" : > + "unknown event", port_id); > + reset_port =3D port_id; > + rte_compiler_barrier(); /* prevent compiler reordering */ > + stop_forwarding =3D 1; > +} > + > +int > +main(int argc, char **argv) > +{ > + struct lcore_queue_conf *qconf; > + struct rte_eth_dev_info dev_info; > + int ret; > + uint8_t nb_ports; > + uint8_t nb_ports_available; > + uint8_t portid, last_port; > + unsigned lcore_id, rx_lcore_id; > + unsigned nb_ports_in_mask =3D 0; > + > + /* init EAL */ > + ret =3D rte_eal_init(argc, argv); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); > + argc -=3D ret; > + argv +=3D ret; > + > + force_quit =3D false; > + signal(SIGINT, signal_handler); > + signal(SIGTERM, signal_handler); > + > + /* parse application arguments (after the EAL ones) */ > + ret =3D l2fwd_parse_args(argc, argv); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, "Invalid L2FWD arguments\n"); > + > + /* create the mbuf pool */ > + l2fwd_pktmbuf_pool =3D rte_pktmbuf_pool_create("mbuf_pool", NB_MBUF, 32= , > + 0, RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id()); > + if (l2fwd_pktmbuf_pool =3D=3D NULL) > + rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n"); > + > + nb_ports =3D rte_eth_dev_count(); > + if (nb_ports =3D=3D 0) > + rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n"); > + > + if (nb_ports > RTE_MAX_ETHPORTS) > + nb_ports =3D RTE_MAX_ETHPORTS; > + > + /* reset l2fwd_dst_ports */ > + for (portid =3D 0; portid < RTE_MAX_ETHPORTS; portid++) > + l2fwd_dst_ports[portid] =3D 0; > + last_port =3D 0; > + > + /* > + * Each logical core is assigned a dedicated TX queue on each port. > + */ > + for (portid =3D 0; portid < nb_ports; portid++) { > + /* skip ports that are not enabled */ > + if ((l2fwd_enabled_port_mask & (1 << portid)) =3D=3D 0) > + continue; > + > + if (nb_ports_in_mask % 2) { > + l2fwd_dst_ports[portid] =3D last_port; > + l2fwd_dst_ports[last_port] =3D portid; > + } else > + last_port =3D portid; > + > + nb_ports_in_mask++; > + > + rte_eth_dev_info_get(portid, &dev_info); > + } > + if (nb_ports_in_mask % 2) { > + printf("Notice: odd number of ports in portmask.\n"); > + l2fwd_dst_ports[last_port] =3D last_port; > + } > + > + rx_lcore_id =3D 1; > + qconf =3D NULL; > + > + /* Initialize the port/queue configuration of each logical core */ > + for (portid =3D 0; portid < nb_ports; portid++) { > + /* skip ports that are not enabled */ > + if ((l2fwd_enabled_port_mask & (1 << portid)) =3D=3D 0) > + continue; > + > + /* get the lcore_id for this port */ > + while (rte_lcore_is_enabled(rx_lcore_id) =3D=3D 0 || > + lcore_queue_conf[rx_lcore_id].n_rx_port =3D=3D > + l2fwd_rx_queue_per_lcore) { > + rx_lcore_id++; > + if (rx_lcore_id >=3D RTE_MAX_LCORE) > + rte_exit(EXIT_FAILURE, "Not enough cores\n"); > + } > + > + if (qconf !=3D &lcore_queue_conf[rx_lcore_id]) > + /* Assigned a new logical core in the loop above. */ > + qconf =3D &lcore_queue_conf[rx_lcore_id]; > + > + qconf->rx_port_list[qconf->n_rx_port] =3D portid; > + qconf->n_rx_port++; > + printf("Lcore %u: RX port %u\n", > + rx_lcore_id, > + (unsigned) portid); > + } > + > + nb_ports_available =3D nb_ports; > + > + /* Initialise each port */ > + for (portid =3D 0; portid < nb_ports; portid++) { > + /* skip ports that are not enabled */ > + if ((l2fwd_enabled_port_mask & (1 << portid)) =3D=3D 0) { > + printf("Skipping disabled port %u\n", > + (unsigned) portid); > + nb_ports_available--; > + continue; > + } > + /* init port */ > + printf("Initializing port %u... ", (unsigned) portid); > + fflush(stdout); > + ret =3D rte_eth_dev_configure(portid, 1, 1, &port_conf); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, > + "Cannot configure device: err=3D%d, port=3D%u\n", > + ret, (unsigned) portid); > + > + /* register reset interrupt callback */ > + rte_eth_dev_callback_register(portid, > + RTE_ETH_EVENT_INTR_RESET, reset_event_callback, NULL); > + > + rte_eth_macaddr_get(portid, &l2fwd_ports_eth_addr[portid]); > + > + /* init one RX queue */ > + fflush(stdout); > + ret =3D rte_eth_rx_queue_setup(portid, 0, nb_rxd, > + rte_eth_dev_socket_id(portid), > + NULL, > + l2fwd_pktmbuf_pool); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, > + "rte_eth_rx_queue_setup:err=3D%d, port=3D%u\n", > + ret, (unsigned) portid); > + > + /* init one TX queue on each port */ > + fflush(stdout); > + ret =3D rte_eth_tx_queue_setup(portid, 0, nb_txd, > + rte_eth_dev_socket_id(portid), > + NULL); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, > + "rte_eth_tx_queue_setup:err=3D%d, port=3D%u\n", > + ret, (unsigned) portid); > + > + /* Initialize TX buffers */ > + tx_buffer[portid] =3D rte_zmalloc_socket("tx_buffer", > + RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0, > + rte_eth_dev_socket_id(portid)); > + if (tx_buffer[portid] =3D=3D NULL) > + rte_exit(EXIT_FAILURE, > + "Cannot allocate buffer for tx on port %u\n", > + (unsigned) portid); > + > + rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST); > + > + ret =3D rte_eth_tx_buffer_set_err_callback(tx_buffer[portid], > + rte_eth_tx_buffer_count_callback, > + &port_statistics[portid].dropped); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, "Cannot set error callback for " > + "tx buffer on port %u\n", (unsigned) portid); > + > + /* Start device */ > + ret =3D rte_eth_dev_start(portid); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, > + "rte_eth_dev_start:err=3D%d, port=3D%u\n", > + ret, (unsigned) portid); > + > + printf("done:\n"); > + > + rte_eth_promiscuous_enable(portid); > + > + printf("Port %u, MAC address: " > + "%02X:%02X:%02X:%02X:%02X:%02X\n\n", > + (unsigned) portid, > + l2fwd_ports_eth_addr[portid].addr_bytes[0], > + l2fwd_ports_eth_addr[portid].addr_bytes[1], > + l2fwd_ports_eth_addr[portid].addr_bytes[2], > + l2fwd_ports_eth_addr[portid].addr_bytes[3], > + l2fwd_ports_eth_addr[portid].addr_bytes[4], > + l2fwd_ports_eth_addr[portid].addr_bytes[5]); > + > + /* initialize port stats */ > + memset(&port_statistics, 0, sizeof(port_statistics)); > + } > + > + if (!nb_ports_available) { > + rte_exit(EXIT_FAILURE, > + "All available ports are disabled. Please set portmask.\n"); > + } > + > + check_all_ports_link_status(nb_ports, l2fwd_enabled_port_mask); > + > + ret =3D 0; > + /* launch per-lcore init on every lcore */ > + rte_eal_mp_remote_launch(l2fwd_launch_one_lcore, NULL, SKIP_MASTER); > + > + printf("\nwaiting.."); > + while (1) { > + rte_delay_ms(1000); > + printf(".."); > + if (stop_forwarding =3D=3D 1) { > + printf("\nreset\n"); > + rte_eth_dev_reset(reset_port); > + stop_forwarding =3D 0; > + } > + if (force_quit) > + break; > + } > + > + RTE_LCORE_FOREACH_SLAVE(lcore_id) { > + if (rte_eal_wait_lcore(lcore_id) < 0) { > + ret =3D -1; > + break; > + } > + } > + > + for (portid =3D 0; portid < nb_ports; portid++) { > + if ((l2fwd_enabled_port_mask & (1 << portid)) =3D=3D 0) > + continue; > + printf("Closing port %d...", portid); > + rte_eth_dev_stop(portid); > + rte_eth_dev_close(portid); > + printf(" Done\n"); > + } > + printf("Bye...\n"); > + > + return ret; > +} > -- > 1.9.3