From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2D682A3160 for ; Fri, 11 Oct 2019 16:11:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 91BF91EAF1; Fri, 11 Oct 2019 16:11:37 +0200 (CEST) Received: from mail-io1-f67.google.com (mail-io1-f67.google.com [209.85.166.67]) by dpdk.org (Postfix) with ESMTP id 0C78D1EAED for ; Fri, 11 Oct 2019 16:11:36 +0200 (CEST) Received: by mail-io1-f67.google.com with SMTP id z19so21883828ior.0 for ; Fri, 11 Oct 2019 07:11:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qSzUCt8q7vGEriZO3/I2jzQd7JdFfJWEAS1G6Qmbyq4=; b=uNR322EgVdlUpxW1N2G+6Aa7btJIP4AejOJRNGEmLJ5CWeKpFQb7aPKoJie76wWuS1 LgGczKXhYGSrkW7SgTKRNjiwS432BFu5S2xSPNRZI+p1yrDrETdK0HyYaMd7I5ZWCKfC h80fUZz7YTvr99Uhfpzwbtb4SPTGbT/zHW9fsrE5S+SnAbS0Y5cYJqj/GbBUS+a136R6 V2iN09/1F3NyOccxCOxxxLcXYu94TxTdM2UYgWrEKhgR2dEbFqEgN3KkIAsuyOjHjO3s ZjmxWtpGgTfN7AwHxVe+MNLjVjvvlYAmSYxXw+Zz2k+vUiOV2z4SklDTCbZu1m4HRBlc 2oZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qSzUCt8q7vGEriZO3/I2jzQd7JdFfJWEAS1G6Qmbyq4=; b=C1zVPkL+nMS16989LBZXPYHKwS6nT7TtROxw1s5/ickqIYkBVBwPjliIYpWn36SZOF VB5QkbmAnvkharpEVT8kn1pbXWYt/iDehThB+YHR54cwsF1hZTb6OU2baMW884mgCwVZ /8j5M6X5cthEMmXByuIC02hPuO1ZTSGQORkJLjTVZ+VqwEM9l+xsfxFMig0k8N4wem6t dNeMLJU/QzDlP0Da5DatNY9aBsEWd4eHrZpvNQhjV7J9TUBL29Q+xk2gIYxpw5I4BCYq S9tslLBqJhRTHg5xLGHg5+EPvp8WvWuqrU8qoHIP2khdVO8zTLlpQ/CBRIG2H+craDna OZOA== X-Gm-Message-State: APjAAAW684GkQ70Sl5V1b8Lm9Af3Cu4FHuCAdnSoEpZh3yFA/GEwTKYr qkFeva8A7Pv6zhT/JI8Wsd3cP6yj2kAsBCvfi/g= X-Google-Smtp-Source: APXvYqyG5zlHJLJkppolnXik6D25N4oKpzHBiK5ZSZmE3sxm6hJ6Qq7ZWLCYW5KHvvbaheIXtQdJJ7jtCH3e77iJCp4= X-Received: by 2002:a02:ca06:: with SMTP id i6mr17740903jak.112.1570803094727; Fri, 11 Oct 2019 07:11:34 -0700 (PDT) MIME-Version: 1.0 References: <20190924094209.3827-1-pbhagavatula@marvell.com> <20191002205754.11746-1-pbhagavatula@marvell.com> <20191002205754.11746-11-pbhagavatula@marvell.com> In-Reply-To: <20191002205754.11746-11-pbhagavatula@marvell.com> From: Jerin Jacob Date: Fri, 11 Oct 2019 19:41:23 +0530 Message-ID: To: Pavan Nikhilesh Cc: Jerin Jacob , "Richardson, Bruce" , Akhil Goyal , Thomas Monjalon , John McNamara , Marko Kovacevic , Ori Kam , Radu Nicolau , Tomasz Kantecki , Sunil Kumar Kori , dpdk-dev Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v5 10/10] doc: add application usage guide for l2fwd-event X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, Oct 3, 2019 at 2:29 AM wrote: > > From: Sunil Kumar Kori > > Add documentation for l2fwd-event example. > Update release notes. > > Signed-off-by: Sunil Kumar Kori # Please fix the typos through with the following command aspell --lang=en_US --check doc/guides/sample_app_ug/l2_forward_event.rst # Remove the following warning. $ make doc-guides-html doc/guides/sample_app_ug/l2_forward_real_virtual.rst:39: WARNING: duplicate label figure_l2_fwd_benchmark_setup, > --- > MAINTAINERS | 1 + > doc/guides/rel_notes/release_19_11.rst | 6 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/intro.rst | 5 + > doc/guides/sample_app_ug/l2_forward_event.rst | 755 ++++++++++++++++++ > 5 files changed, 768 insertions(+) > create mode 100644 doc/guides/sample_app_ug/l2_forward_event.rst > > diff --git a/MAINTAINERS b/MAINTAINERS > index 292ac10c3..94a49b812 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -1461,6 +1461,7 @@ F: examples/l2fwd-cat/ > M: Sunil Kumar Kori > M: Pavan Nikhilesh > F: examples/l2fwd-event/ > +F: doc/guides/sample_app_ug/l2_forward_event.rst > T: git://dpdk.org/next/dpdk-next-eventdev > > F: examples/l3fwd/ > diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst > index 27cfbd9e3..071593e4d 100644 > --- a/doc/guides/rel_notes/release_19_11.rst > +++ b/doc/guides/rel_notes/release_19_11.rst > @@ -56,6 +56,12 @@ New Features > Also, make sure to start the actual text at the margin. > ========================================================= > > +* **Added new example l2fwd-event** Added new example l2fwd-event application > + > + Added an example application `l2fwd-event` that adds event device support to > + traditional l2fwd example. The default poll mode is also preserved for > + readability. Please change the last sentence to the following or something similar "It demonstrates usage of poll and event mode IO mechanism under a single application" > + > > Removed Items > ------------- > diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst > index f23f8f59e..41388231a 100644 > --- a/doc/guides/sample_app_ug/index.rst > +++ b/doc/guides/sample_app_ug/index.rst > @@ -26,6 +26,7 @@ Sample Applications User Guides > l2_forward_crypto > l2_forward_job_stats > l2_forward_real_virtual > + l2_forward_event > l2_forward_cat > l3_forward > l3_forward_power_man > diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst > index 90704194a..84591c0a1 100644 > --- a/doc/guides/sample_app_ug/intro.rst > +++ b/doc/guides/sample_app_ug/intro.rst > @@ -87,6 +87,11 @@ examples are highlighted below. > forwarding, or ``l2fwd`` application does forwarding based on Ethernet MAC > addresses like a simple switch. > > +* :doc:`Network Layer 2 forwarding`: The Network Layer 2 > + forwarding, or ``l2fwd-event`` application does forwarding based on Ethernet MAC > + addresses like a simple switch. It demonstrate usage of poll and event mode Rx/Tx > + mechanism. Please change the last sentence to the following or something similar "It demonstrates usage of poll and event mode IO mechanism under a single application" > + > * :doc:`Network Layer 3 forwarding`: The Network Layer3 > forwarding, or ``l3fwd`` application does forwarding based on Internet > Protocol, IPv4 or IPv6 like a simple router. > diff --git a/doc/guides/sample_app_ug/l2_forward_event.rst b/doc/guides/sample_app_ug/l2_forward_event.rst > new file mode 100644 > index 000000000..250d16887 > --- /dev/null > +++ b/doc/guides/sample_app_ug/l2_forward_event.rst > @@ -0,0 +1,755 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2010-2014 Intel Corporation. > + > +.. _l2_fwd_event_app: > + > +L2 Forwarding Eventdev Sample Application > +========================================= > + > +The L2 Forwarding eventdev sample application is a simple example of packet > +processing using the Data Plane Development Kit (DPDK) to demonstrate usage of > +poll and event mode packet I/O mechanism. > + > +Overview > +-------- > + > +The L2 Forwarding eventdev sample application, performs L2 forwarding for each > +packet that is received on an RX_PORT. The destination port is the adjacent port > +from the enabled portmask, that is, if the first four ports are enabled (portmask=0x0f), > +ports 1 and 2 forward into each other, and ports 3 and 4 forward into each other. > +Also, if MAC addresses updating is enabled, the MAC addresses are affected as follows: > + > +* The source MAC address is replaced by the TX_PORT MAC address > + > +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID > + > +Appliation receives packets from RX_PORT using below mentioned methods: > + > +* Poll mode > + > +* Eventdev mode (default) > + > +This application can be used to benchmark performance using a traffic-generator, > +as shown in the :numref:`figure_l2_fwd_benchmark_setup`. > + > +.. _figure_l2_fwd_benchmark_setup: > + > +.. figure:: img/l2_fwd_benchmark_setup.* Looks like there is a typo in the original l2fwd application image. The diagram shows NUT instead of DUT. Please send a separate patch. > + > + Performance Benchmark Setup (Basic Environment) > + > +Compiling the Application > +------------------------- > + > +To compile the sample application see :doc:`compiling`. > + > +The application is located in the ``l2fwd-event`` sub-directory. > + > +Running the Application > +----------------------- > + > +The application requires a number of command line options: > + > +.. code-block:: console > + > + ./build/l2fwd-event [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating --mode=MODE --eventq-sync=SYNC_MODE > + > +where, > + > +* p PORTMASK: A hexadecimal bitmask of the ports to configure > + > +* q NQ: A number of queues (=ports) per lcore (default is 1) > + > +* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). > + > +* --mode=MODE: Packet transfer mode for I/O, poll or eventdev. Eventdev by default. > + > +* --eventq-sync=SYNC_MODE: Event queue synchronization method, Ordered or Atomic. Atomic by default. > + > +Sample usage commands are given below to run the application into different mode: > + > +Poll mode on linux environment with 4 lcores, 16 ports and 8 RX queues per lcore Nothing specific to Linux here, so please remove Linux environment > +Driver Initialization > +~~~~~~~~~~~~~~~~~~~~~ > + > +The main part of the code in the main() function relates to the initialization > +of the driver. To fully understand this code, it is recommended to study the > +chapters that related to the Poll Mode and Event mode Driver in the > +*DPDK Programmer's Guide* - Rel 1.4 EAR and the *DPDK API Reference*. > + > +.. code-block:: c > + > + if (rte_pci_probe() < 0) > + rte_exit(EXIT_FAILURE, "Cannot probe PCI\n"); > + > + /* reset l2fwd_dst_ports */ > + > + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) > + l2fwd_dst_ports[portid] = 0; > + > + last_port = 0; > + > + /* > + * Each logical core is assigned a dedicated TX queue on each port. > + */ > + > + RTE_ETH_FOREACH_DEV(portid) { > + /* skip ports that are not enabled */ > + > + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0) > + continue; > + > + if (nb_ports_in_mask % 2) { > + l2fwd_dst_ports[portid] = last_port; > + l2fwd_dst_ports[last_port] = portid; > + } > + else > + last_port = portid; > + > + nb_ports_in_mask++; > + > + rte_eth_dev_info_get((uint8_t) portid, &dev_info); > + } > + > +Observe that: > + > +* rte_igb_pmd_init_all() simultaneously registers the driver as a PCI driver Looks like rte_igb_pmd_init_all() from legacy l2fwd doc, Please fix in both docs. The original l2fwd doc fix send it as a separate patch. > +.. code-block:: c > + > + /* Start event device service */ > + ret = rte_event_dev_service_id_get(eventdev_rsrc.event_d_id, > + &service_id); > + if (ret != -ESRCH && ret != 0) > + rte_exit(EXIT_FAILURE, "Error in starting eventdev"); > + > + rte_service_runstate_set(service_id, 1); > + rte_service_set_runstate_mapped_check(service_id, 0); > + eventdev_rsrc.service_id = service_id; eventdev_rsrc removed recently in code review, Please sync code with sync in all code sections. > +.. note:: > + > + In the following code, one line for getting the output port requires some > + explanation. Do we need this note: ? > + > +During the initialization process, a static array of destination ports > +(l2fwd_dst_ports[]) is filled such that for each source port, a destination port > +is assigned that is either the next or previous enabled port from the portmask. > +If number of ports are odd in portmask then packet from last port will be > +forwarded to first port i.e. if portmask=0x07, then forwarding will take place > +like p0--->p1, p1--->p2, p2--->p0. > + > +Also to optimize enqueue opeartion, l2fwd_simple_forward() stores incoming mbus > +upto MAX_PKT_BURST. Once it reaches upto limit, all packets are transmitted to > +destination ports. > + > +.. code-block:: c > + > + static void > + l2fwd_simple_forward(struct rte_mbuf *m, uint32_t portid) > + { > + uint32_t dst_port; > + int32_t sent; > + struct rte_eth_dev_tx_buffer *buffer; > + > + dst_port = l2fwd_dst_ports[portid]; > + > + if (mac_updating) > + l2fwd_mac_updating(m, dst_port); > + > + buffer = tx_buffer[dst_port]; > + sent = rte_eth_tx_buffer(dst_port, 0, buffer, m); > + if (sent) > + port_statistics[dst_port].tx += sent; > + } > + > +For this test application, the processing is exactly the same for all packets > +arriving on the same RX port. Therefore, it would have been possible to call > +the rte_eth_tx_buffer() function directly from the main loop to send all the > +received packets on the same TX port, using the burst-oriented send function, > +which is more efficient. > + > +However, in real-life applications (such as, L3 routing), > +packet N is not necessarily forwarded on the same port as packet N-1. > +The application is implemented to illustrate that, so the same approach can be > +reused in a more complex application. > + > +To ensure that no packets remain in the tables, each lcore does a draining of TX > +queue in its main loop. This technique introduces some latency when there are > +not many packets to send, however it improves performance: > + > +.. code-block:: c > + > + cur_tsc = rte_rdtsc(); > + > + /* > + * TX burst queue drain > + */ > + diff_tsc = cur_tsc - prev_tsc; > + if (unlikely(diff_tsc > drain_tsc)) { > + for (i = 0; i < qconf->n_rx_port; i++) { > + portid = l2fwd_dst_ports[qconf->rx_port_list[i]]; > + buffer = tx_buffer[portid]; > + sent = rte_eth_tx_buffer_flush(portid, 0, > + buffer); > + if (sent) > + port_statistics[portid].tx += sent; > + } > + > + /* if timer is enabled */ > + if (timer_period > 0) { > + /* advance the timer */ > + timer_tsc += diff_tsc; > + > + /* if timer has reached its timeout */ > + if (unlikely(timer_tsc >= timer_period)) { > + /* do this only on master core */ > + if (lcore_id == rte_get_master_lcore()) { > + print_stats(); > + /* reset the timer */ > + timer_tsc = 0; > + } > + } > + } > + > + prev_tsc = cur_tsc; > + } > + > +In the **l2fwd_main_loop_eventdev()** function, the main task is to read ingress > +packets from the event ports. This is done using the following code: > + > +.. code-block:: c > + > + /* Read packet from eventdev */ > + nb_rx = rte_event_dequeue_burst(event_d_id, event_p_id, > + events, deq_len, 0); > + if (nb_rx == 0) { > + rte_pause(); > + continue; > + } > + > + for (i = 0; i < nb_rx; i++) { > + mbuf[i] = events[i].mbuf; > + rte_prefetch0(rte_pktmbuf_mtod(mbuf[i], void *)); > + } > + > + > +Before reading packets, deq_len is fetched to ensure correct allowed deq length > +by the eventdev. > +The rte_event_dequeue_burst() function writes the mbuf pointers in a local table > +and returns the number of available mbufs in the table. > + > +Then, each mbuf in the table is processed by the l2fwd_eventdev_forward() > +function. The processing is very simple: process the TX port from the RX port, > +then replace the source and destination MAC addresses if MAC addresses updating > +is enabled. > + > +.. note:: > + > + In the following code, one line for getting the output port requires some > + explanation. Do we need this note: ? > + > +During the initialization process, a static array of destination ports > +(l2fwd_dst_ports[]) is filled such that for each source port, a destination port > +is assigned that is either the next or previous enabled port from the portmask. > +If number of ports are odd in portmask then packet from last port will be > +forwarded to first port i.e. if portmask=0x07, then forwarding will take place > +like p0--->p1, p1--->p2, p2--->p0. > + > +l2fwd_eventdev_forward() does not stores incoming mbufs. Packet will forwarded > +be to destination ports via Tx adapter or generic event dev enqueue API > +depending H/W or S/W scheduler is used. > + > +.. code-block:: c > + > + static inline void > + l2fwd_eventdev_forward(struct rte_mbuf *m[], uint32_t portid, > + uint16_t nb_rx, uint16_t event_p_id) > + { > + uint32_t dst_port, i; > + > + dst_port = l2fwd_dst_ports[portid]; > + > + for (i = 0; i < nb_rx; i++) { > + if (mac_updating) > + l2fwd_mac_updating(m[i], dst_port); > + > + m[i]->port = dst_port; > + } > + > + if (timer_period > 0) { > + rte_spinlock_lock(&port_stats_lock); > + port_statistics[dst_port].tx += nb_rx; > + rte_spinlock_unlock(&port_stats_lock); > + } > + /* Registered callback is invoked for Tx */ > + eventdev_rsrc.send_burst_eventdev(m, nb_rx, event_p_id); > + } > -- > 2.17.1 >