DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH] examples/ioat: create sample app on ioat driver usage
@ 2019-09-09  8:29 Marcin Baran
  2019-09-09 13:12 ` Aaron Conole
                   ` (2 more replies)
  0 siblings, 3 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-09  8:29 UTC (permalink / raw)
  To: dev; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The patch includes sample application
as well as it's guide.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 doc/guides/sample_app_ug/index.rst |    1 +
 doc/guides/sample_app_ug/intro.rst |    4 +
 doc/guides/sample_app_ug/ioat.rst  |  691 +++++++++++++++++++
 examples/Makefile                  |    3 +
 examples/ioat/Makefile             |   54 ++
 examples/ioat/ioatfwd.c            | 1010 ++++++++++++++++++++++++++++
 examples/ioat/meson.build          |   13 +
 examples/meson.build               |    1 +
 8 files changed, 1777 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index f23f8f59e..a6a1d9e7a 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -23,6 +23,7 @@ Sample Applications User Guides
     ip_reassembly
     kernel_nic_interface
     keep_alive
+    ioat
     l2_forward_crypto
     l2_forward_job_stats
     l2_forward_real_virtual
diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
index 90704194a..74462312f 100644
--- a/doc/guides/sample_app_ug/intro.rst
+++ b/doc/guides/sample_app_ug/intro.rst
@@ -91,6 +91,10 @@ examples are highlighted below.
   forwarding, or ``l3fwd`` application does forwarding based on Internet
   Protocol, IPv4 or IPv6 like a simple router.
 
+* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
+  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
+  copying packets between two threads.
+
 * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
   demonstrates how to distribute packets arriving on an Rx port to different
   cores for processing and transmission.
diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
new file mode 100644
index 000000000..378d70b81
--- /dev/null
+++ b/doc/guides/sample_app_ug/ioat.rst
@@ -0,0 +1,691 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Intel Corporation.
+
+Sample Application of packet copying using Intel\|reg| QuickData Technology
+============================================================================
+
+Overview
+--------
+
+This sample is intended as a demonstration of the basic components of a DPDK
+forwarding application and example of how to use IOAT driver API to make
+packets copies.
+
+Also while forwarding, the MAC addresses are affected as follows:
+
+*   The source MAC address is replaced by the TX port MAC address
+
+*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
+
+This application can be used to compare performance of using software packet
+copy with copy done using a DMA device for different sizes of packets.
+The example will print out statistics each second. The stats shows
+received/send packets and packets dropped or failed to copy.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``ioat`` sub-directory.
+
+
+Running the Application
+-----------------------
+
+In order to run the hardware copy application, the copying device
+needs to be bound to user-space IO driver.
+
+Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology*
+guide for information on using the driver.
+
+The application requires a number of command line options:
+
+.. code-block:: console
+
+    ./build/ioatfwd [EAL options] -- -p MASK [-C CT] [--[no-]mac-updating]
+
+where,
+
+*   p MASK: A hexadecimal bitmask of the ports to configure
+
+*   c CT: Performed packet copy type: software (sw) or hardware using
+    DMA (rawdev)
+
+*   s RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for
+    software copy mode
+
+*   --[no-]mac-updating: Whether MAC address of packets should be changed
+    or not
+
+The application can be launched in 2 different configurations:
+
+*   Performing software packet copying
+
+*   Performing hardware packet copying
+
+Each port needs 2 lcores: one of them receives incoming traffic and makes
+a copy of each packet. The second lcore then updates MAC address and sends
+the copy. For each configuration an additional lcore is needed since
+master lcore in use which is responsible for configuration, statistics
+printing and safe deinitialization of all ports and devices.
+
+The application can use a maximum of 8 ports.
+
+To run the application in a Linux environment with 3 lcores (one of them
+is master lcore), 1 port (port 0), software copying and MAC updating issue
+the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
+
+To run the application in a Linux environment with 5 lcores (one of them
+is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC
+updating issue the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-4 -n 1 -- -p 0x3 --no-mac-updating -c rawdev
+
+Refer to the *DPDK Getting Started Guide* for general information on
+running applications and the Environment Abstraction Layer (EAL) options.
+
+Explanation
+-----------
+
+The following sections provide an explanation of the main components of the
+code.
+
+All DPDK library functions used in the sample code are prefixed with
+``rte_`` and are explained in detail in the *DPDK API Documentation*.
+
+
+The Main Function
+~~~~~~~~~~~~~~~~~
+
+The ``main()`` function performs the initialization and calls the execution
+threads for each lcore.
+
+The first task is to initialize the Environment Abstraction Layer (EAL).
+The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
+function. The value returned is the number of parsed arguments:
+
+.. code-block:: c
+
+    /* init EAL */
+    ret = rte_eal_init(argc, argv);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+
+
+The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
+used by the application:
+
+.. code-block:: c
+
+    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
+        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+        MIN_POOL_SIZE);
+
+    /* Create the mbuf pool */
+    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+        rte_socket_id());
+    if (ioat_pktmbuf_pool == NULL)
+        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+Mbufs are the packet buffer structure used by DPDK. They are explained in
+detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
+
+The ``main()`` function also initializes the ports:
+
+.. code-block:: c
+
+    /* Initialise each port */
+    RTE_ETH_FOREACH_DEV(portid) {
+        port_init(portid, ioat_pktmbuf_pool);
+    }
+
+Each port is configured using ``port_init()``:
+
+.. code-block:: c
+
+    static inline void
+    port_init(uint16_t portid, struct rte_mempool *mbuf_pool)
+    {
+        struct rte_eth_rxconf rxq_conf;
+        struct rte_eth_txconf txq_conf;
+        struct rte_eth_conf local_port_conf = port_conf;
+        struct rte_eth_dev_info dev_info;
+        int ret;
+
+        /* Skip ports that are not enabled */
+        if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+            printf("Skipping disabled port %u\n", portid);
+            return;
+        }
+
+        /* Init port */
+        printf("Initializing port %u... ", portid);
+        fflush(stdout);
+        rte_eth_dev_info_get(portid, &dev_info);
+        if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+            local_port_conf.txmode.offloads |=
+                DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+        ret = rte_eth_dev_configure(portid, 1, 1, &local_port_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE, "Cannot configure device: err=%d, port=%u\n",
+                    ret, portid);
+
+        ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+                            &nb_txd);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                    "Cannot adjust number of descriptors: err=%d, port=%u\n",
+                    ret, portid);
+
+        rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+        /* Init one RX queue */
+        fflush(stdout);
+        rxq_conf = dev_info.default_rxconf;
+        rxq_conf.offloads = local_port_conf.rxmode.offloads;
+        ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd,
+                        rte_eth_dev_socket_id(portid),
+                        &rxq_conf,
+                        mbuf_pool);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup:err=%d, port=%u\n",
+                    ret, portid);
+
+        /* Init one TX queue on each port */
+        fflush(stdout);
+        txq_conf = dev_info.default_txconf;
+        txq_conf.offloads = local_port_conf.txmode.offloads;
+        ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+                rte_eth_dev_socket_id(portid),
+                &txq_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d, port=%u\n",
+                ret, portid);
+
+        /* Initialize TX buffers */
+        tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+                RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+                rte_eth_dev_socket_id(portid));
+        if (tx_buffer[portid] == NULL)
+            rte_exit(EXIT_FAILURE, "Cannot allocate buffer for tx "
+                    "on port %u\n", portid);
+
+        rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+        ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+                rte_eth_tx_buffer_count_callback,
+                &port_statistics[portid].tx_dropped);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+            "Cannot set error callback for tx buffer on port %u\n",
+                    portid);
+
+        /* Start device */
+        ret = rte_eth_dev_start(portid);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
+                    ret, portid);
+
+        rte_eth_promiscuous_enable(portid);
+
+        printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+                portid,
+                ioat_ports_eth_addr[portid].addr_bytes[0],
+                ioat_ports_eth_addr[portid].addr_bytes[1],
+                ioat_ports_eth_addr[portid].addr_bytes[2],
+                ioat_ports_eth_addr[portid].addr_bytes[3],
+                ioat_ports_eth_addr[portid].addr_bytes[4],
+                ioat_ports_eth_addr[portid].addr_bytes[5]);
+    }
+
+The Ethernet ports are configured with local settings using the
+``rte_eth_dev_configure()`` function and the ``port_conf`` struct:
+
+.. code-block:: c
+
+    static struct rte_eth_conf port_conf = {
+        .rxmode = {
+            .max_rx_pkt_len = RTE_ETHER_MAX_LEN,
+        },
+    };
+
+For this example the ports are set up with 1 RX and 1 TX queue using the
+``rte_eth_rx_queue_setup()`` and ``rte_eth_tx_queue_setup()`` functions.
+
+The Ethernet port is then started:
+
+.. code-block:: c
+
+    ret = rte_eth_dev_start(portid);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
+            ret, portid);
+
+
+Finally the RX port is set in promiscuous mode:
+
+.. code-block:: c
+
+    rte_eth_promiscuous_enable(portid);
+
+
+After that each port application assigns resources needed.
+
+.. code-block:: c
+
+    check_link_status(ioat_enabled_port_mask);
+
+    if (!cfg.nb_ports) {
+        rte_exit(EXIT_FAILURE,
+            "All available ports are disabled. Please set portmask.\n");
+    }
+
+    /* Check if there is enough lcores for all ports. */
+    cfg.nb_lcores = rte_lcore_count() - 1;
+    if (cfg.nb_lcores < 1)
+        rte_exit(EXIT_FAILURE,
+            "There should be at least one slave lcore.\n");
+
+    ret = 0;
+
+    if (copy_mode == COPY_MODE_IOAT_NUM) {
+        assign_rawdevs();
+    } else /* copy_mode == COPY_MODE_SW_NUM */ {
+        assign_rings();
+    }
+
+A link status is checked of each port enabled by port mask
+using ``check_link_status()`` function.
+
+.. code-block:: c
+
+    /* Check the link status of all ports in up to 9s, and print them finally */
+    static void
+    check_link_status(uint32_t port_mask)
+    {
+
+        uint16_t portid;
+        struct rte_eth_link link;
+
+        cfg.nb_ports = 0;
+
+        printf("\nChecking link status\n");
+        fflush(stdout);
+        RTE_ETH_FOREACH_DEV(portid) {
+            if (force_quit)
+                return;
+            if ((port_mask & (1 << portid)) == 0)
+                continue;
+
+            store_port_nb(portid);
+
+            memset(&link, 0, sizeof(link));
+            rte_eth_link_get(portid, &link);
+
+            /* Print link status */
+            if (link.link_status) {
+                printf(
+                    "Port %d Link Up. Speed %u Mbps - %s\n",
+                    portid, link.link_speed,
+                    (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+                    ("full-duplex") : ("half-duplex\n"));
+            }
+            else
+                printf("Port %d Link Down\n", portid);
+        }
+    }
+
+Depending on mode set (whether copy should be done by software or by hardware)
+special structures are assigned to each port. If software copy was chosen,
+application have to assign ring structures for packet exchanging between lcores
+assigned to ports.
+
+.. code-block:: c
+
+    static void
+    assign_rings(void)
+    {
+        uint32_t i;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            char ring_name[20];
+
+            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
+            /* Create ring for inter core communication */
+            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+                    ring_name, ring_size,
+                    rte_socket_id(), RING_F_SP_ENQ);
+
+            if (cfg.ports[i].rx_to_tx_ring == NULL)
+                rte_exit(EXIT_FAILURE, "%s\n",
+                        rte_strerror(rte_errno));
+        }
+    }
+
+
+When using hardware copy each port is assigned an IOAT device
+(``assign_rawdevs()``) using IOAT Rawdev Driver API functions:
+
+.. code-block:: c
+
+    static void
+    assign_rawdevs(void)
+    {
+        uint16_t nb_rawdev = 0;
+        uint32_t i;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            struct rte_rawdev_info rdev_info = {0};
+            rte_rawdev_info_get(0, &rdev_info);
+
+            if (strcmp(rdev_info.driver_name, "rawdev_ioat") == 0) {
+                configure_rawdev_queue(i);
+                cfg.ports[i].dev_id = i;
+                ++nb_rawdev;
+            }
+        }
+
+        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+
+        if (nb_rawdev < cfg.nb_ports)
+            rte_exit(EXIT_FAILURE, "Not enough IOAT rawdevs (%u) for ports (%u).\n",
+                    nb_rawdev, cfg.nb_ports);
+    }
+
+
+The initialization of hardware device is done by ``rte_rawdev_configure()``
+function and ``rte_rawdev_info`` struct. After configuration the device is
+started using ``rte_rawdev_start()`` function. Each of the above operations
+is done in ``configure_rawdev_queue()``.
+
+.. code-block:: c
+
+    static void
+    configure_rawdev_queue(uint32_t dev_id)
+    {
+        struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+        /* Configure hardware copy device */
+        dev_config.ring_size = ring_size;
+
+        if (rte_rawdev_configure(dev_id, &info) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_configure()\n");
+        }
+        rte_rawdev_info_get(dev_id, &info);
+        if (dev_config.ring_size != ring_size) {
+            rte_exit(EXIT_FAILURE,
+                "Error, ring size is not %d (%d)\n",
+                ring_size, (int)dev_config.ring_size);
+        }
+        if (rte_rawdev_start(dev_id) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_start()\n");
+        }
+    }
+
+If initialization is successful memory for hardware device
+statistics is allocated.
+
+Finally ``main()`` functions starts all processing lcores and starts
+printing stats in a loop on master lcore. The application can be
+interrupted and closed using ``Ctrl-C``. The master lcore waits for
+all slave processes to finish, deallocates resources and exits.
+
+The processing lcores launching function are described below.
+
+The Lcores Launching Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As described above ``main()`` function invokes ``run_transmission()``
+function in order to start processing for each lcore:
+
+.. code-block:: c
+
+    static void run_transmission(void)
+    {
+        uint32_t lcore_id = rte_lcore_id();
+
+        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+                __func__, rte_lcore_id());
+
+        if (cfg.nb_lcores == 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop, NULL, lcore_id);
+        } else if (cfg.nb_lcores > 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rx_main_loop, NULL, lcore_id);
+
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL, lcore_id);
+        }
+    }
+
+The function launches rx/tx processing functions on configured lcores
+for each port using ``rte_eal_remote_launch()``. The configured ports,
+their number and number of assigned lcores are stored in user-defined
+``rxtx_transmission_config`` struct that is initialized before launching
+tasks:
+
+.. code-block:: c
+
+    struct rxtx_transmission_config {
+        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+        uint16_t nb_ports;
+        uint16_t nb_lcores;
+    };
+
+The Lcores Processing Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For receiving packets on each port an ``ioat_rx_port()`` function is used.
+Depending on mode the user chose, it will enqueue packets to IOAT rawdev
+and then invoke copy process (hardware copy), or perform software copy
+of each packet using ``pktmbuf_sw_copy()`` function and enqueue them to
+rte_ring:
+
+.. code-block:: c
+
+    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+    static void
+    ioat_rx_port(struct rxtx_port_config *rx_config)
+    {
+        uint32_t nb_rx, nb_enq, i;
+        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+
+        nb_rx = rte_eth_rx_burst(rx_config->rx_portId, 0,
+            pkts_burst, MAX_PKT_BURST);
+
+        if (nb_rx == 0)
+            return;
+
+        port_statistics[rx_config->rx_portId].rx += nb_rx;
+
+        if (copy_mode == COPY_MODE_IOAT_NUM) {
+            /* Perform packet hardware copy */
+            nb_enq = ioat_enqueue_packets(rx_config,
+                pkts_burst, nb_rx);
+
+            if (nb_enq > 0)
+                rte_ioat_do_copies(rx_config->dev_id);
+        } else {
+            /* Perform packet software copy, free source packets */
+            int ret;
+            struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+            ret = rte_pktmbuf_alloc_bulk(ioat_pktmbuf_pool,
+                    pkts_burst_copy, nb_rx);
+
+            if (unlikely(ret < 0))
+                rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+            for (i = 0; i < nb_rx; i++) {
+                pktmbuf_sw_copy(pkts_burst[i], pkts_burst_copy[i]);
+                rte_pktmbuf_free(pkts_burst[i]);
+            }
+
+            nb_enq = rte_ring_enqueue_burst(rx_config->rx_to_tx_ring,
+                (void *)pkts_burst_copy, nb_rx, NULL);
+
+            /* Free any not enqueued packets. */
+            for (i = nb_enq; i < nb_rx; i++)
+                rte_pktmbuf_free(pkts_burst_copy[i]);
+        }
+
+        port_statistics[rx_config->rx_portId].copy_dropped
+            += (nb_rx - nb_enq);
+    }
+
+The packets are received in burst mode using ``rte_eth_rx_burst()``
+function. When using hardware copy mode the packets are enqueued in
+copying device's buffer using ``ioat_enqueue_packets()`` which calls
+``rte_ioat_enqueue_copy()``. When all received packets are in the
+buffer the copies are invoked by calling ``rte_ioat_do_copies()``.
+Function ``rte_ioat_enqueue_copy()`` operates on physical address of
+the packet. Structure ``rte_mbuf`` contains only physical address to
+start of the data buffer (``buf_iova``). Thus the address is shifted
+by ``addr_offset`` value in order to get pointer to ``rearm_data``
+member of ``rte_mbuf``. That way the packet is copied all at once
+(with data and metadata).
+
+.. code-block:: c
+
+    static uint32_t
+    ioat_enqueue_packets(struct rxtx_port_config *rx_config,
+        struct rte_mbuf **pkts, uint32_t nb_rx)
+    {
+        int ret;
+        uint32_t i;
+        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+            &pkts[0]->rearm_data);
+
+        ret = rte_pktmbuf_alloc_bulk(ioat_pktmbuf_pool, pkts_copy, nb_rx);
+
+        if (unlikely(ret < 0))
+            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+        for (i = 0; i < nb_rx; i++) {
+            /* Perform data copy */
+            ret = rte_ioat_enqueue_copy(rx_config->dev_id,
+                pkts[i]->buf_iova
+                    - addr_offset,
+                pkts_copy[i]->buf_iova
+                    - addr_offset,
+                rte_pktmbuf_data_len(pkts[i])
+                    + addr_offset,
+                (uintptr_t)pkts[i],
+                (uintptr_t)pkts_copy[i],
+                0 /* nofence */);
+
+            if (ret != 1)
+                break;
+        }
+
+        ret = i;
+        /* Free any not enqueued packets. */
+        for (; i < nb_rx; i++) {
+            rte_pktmbuf_free(pkts[i]);
+            rte_pktmbuf_free(pkts_copy[i]);
+        }
+
+        return ret;
+    }
+
+
+All done copies are processed by ``ioat_tx_port()`` function. When using
+hardware copy mode the function invokes ``rte_ioat_completed_copies()``
+to gather copied packets. If software copy mode is used the function
+dequeues copied packets from rte_ring. Then each packet MAC address
+is changed if it was enabled. After that copies are sent in burst mode
+using `` rte_eth_tx_burst()``.
+
+
+.. code-block:: c
+
+    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
+    static void
+    ioat_tx_port(struct rxtx_port_config *tx_config)
+    {
+        uint32_t i, nb_dq;
+        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
+        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+        if (copy_mode == COPY_MODE_IOAT_NUM) {
+            /* Deque the mbufs from IOAT device. */
+            nb_dq = rte_ioat_completed_copies(tx_config->dev_id,
+                MAX_PKT_BURST, (void *)mbufs_src, (void *)mbufs_dst);
+        } else {
+            /* Deque the mbufs from rx_to_tx_ring. */
+            nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+                (void *)mbufs_dst, MAX_PKT_BURST, NULL);
+        }
+
+        if (nb_dq == 0)
+            return;
+
+        /* Free source packets */
+        if (copy_mode == COPY_MODE_IOAT_NUM) {
+            for (i = 0; i < nb_dq; i++)
+                rte_pktmbuf_free(mbufs_src[i]);
+        }
+
+        /* Update macs if enabled */
+        if (mac_updating) {
+            for (i = 0; i < nb_dq; i++)
+                update_mac_addrs(mbufs_dst[i],
+                    tx_config->tx_portId);
+        }
+
+        const uint16_t nb_tx = rte_eth_tx_burst(tx_config->tx_portId,
+            0, (void *)mbufs_dst, nb_dq);
+
+        port_statistics[tx_config->tx_portId].tx += nb_tx;
+
+        /* Free any unsent packets. */
+        if (unlikely(nb_tx < nb_dq)) {
+            for (i = nb_tx; i < nb_dq; i++)
+                rte_pktmbuf_free(mbufs_dst[i]);
+        }
+    }
+
+The Packet Copying Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to perform packet copy there is a user-defined function
+``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
+metadata from source packet to new mbuf, and then copying a data
+chunk of source packet. Both memory copies are done using
+``rte_memcpy()``:
+
+.. code-block:: c
+
+    static inline void
+    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+    {
+        /* Copy packet metadata */
+        rte_memcpy(&dst->rearm_data,
+            &src->rearm_data,
+            offsetof(struct rte_mbuf, cacheline1)
+                - offsetof(struct rte_mbuf, rearm_data));
+
+        /* Copy packet data */
+        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+            rte_pktmbuf_mtod(src, char *), src->data_len);
+    }
+
+The metadata in this example is copied from ``rearm_data`` member of
+``rte_mbuf`` struct up to ``cacheline1``.
+
+In order to understand why software packet copying is done as shown
+above please refer to the "Mbuf Library" section of the
+*DPDK Programmer's Guide*.
\ No newline at end of file
diff --git a/examples/Makefile b/examples/Makefile
index de11dd487..3cb313d7d 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation
 DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify
 DIRS-y += flow_filtering
 DIRS-y += helloworld
+ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y)
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat
+endif
 DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline
 ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
 DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
diff --git a/examples/ioat/Makefile b/examples/ioat/Makefile
new file mode 100644
index 000000000..2a4d1da2d
--- /dev/null
+++ b/examples/ioat/Makefile
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# binary name
+APP = ioatfwd
+
+# all source are stored in SRCS-y
+SRCS-y := ioatfwd.c
+
+# Build using pkg-config variables if possible
+ifeq ($(shell pkg-config --exists libdpdk && echo 0),0)
+
+all: shared
+.PHONY: shared static
+shared: build/$(APP)-shared
+	ln -sf $(APP)-shared build/$(APP)
+static: build/$(APP)-static
+	ln -sf $(APP)-static build/$(APP)
+
+PC_FILE := $(shell pkg-config --path libdpdk)
+CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
+LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
+
+build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
+
+build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
+
+build:
+	@mkdir -p $@
+
+.PHONY: clean
+clean:
+	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
+	test -d build && rmdir -p build || true
+
+else # Build using legacy build system
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, detect a build directory, by looking for a path with a .config
+RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config)))))
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+endif
diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
new file mode 100644
index 000000000..8463d82f3
--- /dev/null
+++ b/examples/ioat/ioatfwd.c
@@ -0,0 +1,1010 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#include <stdint.h>
+#include <getopt.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+#include <rte_ethdev.h>
+#include <rte_rawdev.h>
+#include <rte_ioat_rawdev.h>
+
+/* size of ring used for software copying between rx and tx. */
+#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
+#define MAX_PKT_BURST 32
+#define MEMPOOL_CACHE_SIZE 512
+#define MIN_POOL_SIZE 65536U
+#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
+#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
+#define CMD_LINE_OPT_PORTMASK "portmask"
+#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
+#define CMD_LINE_OPT_COPY_TYPE "copy-type"
+#define CMD_LINE_OPT_RING_SIZE "ring-size"
+
+/* configurable number of RX/TX ring descriptors */
+#define RX_DEFAULT_RINGSIZE 1024
+#define TX_DEFAULT_RINGSIZE 1024
+
+/* max number of RX queues per port */
+#define MAX_RX_QUEUES_COUNT 8
+
+struct rxtx_port_config {
+	/* common config */
+	uint16_t rxtx_port;
+	uint16_t nb_queues;
+	/* for software copy mode */
+	struct rte_ring *rx_to_tx_ring;
+	/* for IOAT rawdev copy mode */
+	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT];
+};
+
+struct rxtx_transmission_config {
+	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+	uint16_t nb_ports;
+	uint16_t nb_lcores;
+};
+
+/* per-port statistics struct */
+struct ioat_port_statistics {
+	uint64_t rx[RTE_MAX_ETHPORTS];
+	uint64_t tx[RTE_MAX_ETHPORTS];
+	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
+	uint64_t copy_dropped[RTE_MAX_ETHPORTS];
+};
+struct ioat_port_statistics port_statistics;
+
+struct total_statistics {
+	uint64_t total_packets_dropped;
+	uint64_t total_packets_tx;
+	uint64_t total_packets_rx;
+	uint64_t total_successful_enqueues;
+	uint64_t total_failed_enqueues;
+};
+
+typedef enum copy_mode_t {
+#define COPY_MODE_SW "sw"
+	COPY_MODE_SW_NUM,
+#define COPY_MODE_IOAT "rawdev"
+	COPY_MODE_IOAT_NUM,
+	COPY_MODE_INVALID_NUM,
+	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
+} copy_mode_t;
+
+/* mask of enabled ports */
+static uint32_t ioat_enabled_port_mask;
+
+/* number of RX queues per port */
+static uint16_t nb_queues = 1;
+
+/* MAC updating enabled by default. */
+static int mac_updating = 1;
+
+/* hardare copy mode enabled by default. */
+static copy_mode_t copy_mode = COPY_MODE_IOAT_NUM;
+
+/* size of IOAT rawdev ring for hardware copy mode or
+ * rte_ring for software copy mode
+ */
+static unsigned short ring_size = 2048;
+
+/* global transmission config */
+struct rxtx_transmission_config cfg;
+
+/* configurable number of RX/TX ring descriptors */
+static uint16_t nb_rxd = RX_DEFAULT_RINGSIZE;
+static uint16_t nb_txd = TX_DEFAULT_RINGSIZE;
+
+static volatile bool force_quit;
+
+/* ethernet addresses of ports */
+static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
+struct rte_mempool *ioat_pktmbuf_pool;
+
+/* Print out statistics for one port. */
+static void
+print_port_stats(uint16_t port_id)
+{
+	printf("\nStatistics for port %u ------------------------------"
+		"\nPackets sent: %34"PRIu64
+		"\nPackets received: %30"PRIu64
+		"\nPackets dropped on tx: %25"PRIu64
+		"\nPackets dropped on copy: %23"PRIu64,
+		port_id,
+		port_statistics.tx[port_id],
+		port_statistics.rx[port_id],
+		port_statistics.tx_dropped[port_id],
+		port_statistics.copy_dropped[port_id]);
+}
+
+/* Print out statistics for one IOAT rawdev device. */
+static void
+print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
+	uint16_t nb_xstats, struct rte_rawdev_xstats_name *names_xstats)
+{
+	uint16_t i;
+
+	printf("\nIOAT channel %u", dev_id);
+	for (i = 0; i < nb_xstats; i++)
+		if (strstr(names_xstats[i].name, "enqueues"))
+			printf("\n\t %s: %*"PRIu64,
+				names_xstats[i].name,
+				(int)(37 - strlen(names_xstats[i].name)),
+				xstats[i]);
+}
+
+static void
+print_total_stats(struct total_statistics *ts)
+{
+	printf("\nAggregate statistics ==============================="
+		"\nTotal packets sent: %28"PRIu64
+		"\nTotal packets received: %24"PRIu64
+		"\nTotal packets dropped: %25"PRIu64,
+		ts->total_packets_tx,
+		ts->total_packets_rx,
+		ts->total_packets_dropped);
+
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		printf("\nTotal IOAT successful enqueues: %16"PRIu64
+			"\nTotal IOAT failed enqueues: %20"PRIu64,
+			ts->total_successful_enqueues,
+			ts->total_failed_enqueues);
+	}
+
+	printf("\n====================================================\n");
+}
+
+/* Print out statistics on packets dropped. */
+static void
+print_stats(char *prgname)
+{
+	struct total_statistics ts;
+	uint32_t i, port_id, dev_id;
+	struct rte_rawdev_xstats_name *names_xstats;
+	uint64_t *xstats;
+	unsigned int *ids_xstats;
+	unsigned int nb_xstats, id_fail_enq, id_succ_enq;
+	char status_string[120]; /* to print at the top of the output */
+	int status_strlen;
+
+
+	const char clr[] = { 27, '[', '2', 'J', '\0' };
+	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+	status_strlen = snprintf(status_string, sizeof(status_string),
+		"%s, ", prgname);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Worker Threads = %d, ",
+		rte_lcore_count() > 2 ? 2 : 1);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
+		COPY_MODE_SW : COPY_MODE_IOAT);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Updating MAC = %s, ", mac_updating ?
+		"enabled" : "disabled");
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Rx Queues = %d, ", nb_queues);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Ring Size = %d\n", ring_size);
+
+	/* Allocate memory for xstats names and values */
+	nb_xstats = rte_rawdev_xstats_names_get(
+			cfg.ports[0].ioat_ids[0], NULL, 0);
+
+	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
+	if (names_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat names memory\n");
+	}
+	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
+			names_xstats, nb_xstats);
+
+	ids_xstats = malloc(sizeof(*ids_xstats) * nb_xstats);
+	if (ids_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat ids_xstats memory\n");
+	}
+
+	for (i = 0; i < nb_xstats; i++)
+		ids_xstats[i] = i;
+
+	xstats = malloc(sizeof(*xstats) * nb_xstats);
+	if (xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat memory\n");
+	}
+
+	/* Get failed/successful enqueues stats index */
+	id_fail_enq = id_succ_enq = nb_xstats;
+	for (i = 0; i < nb_xstats; i++) {
+		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
+			id_fail_enq = i;
+		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
+			id_succ_enq = i;
+		if (id_fail_enq < nb_xstats && id_succ_enq < nb_xstats)
+			break;
+	}
+	if (id_fail_enq == nb_xstats || id_succ_enq == nb_xstats) {
+		rte_exit(EXIT_FAILURE,
+			"Error getting failed/successful enqueues stats index\n");
+	}
+
+	while (!force_quit) {
+		/* Sleep for 1 second each round - init sleep allows reading
+		 * messages from app startup.
+		 */
+		sleep(1);
+
+		/* Clear screen and move to top left */
+		printf("%s%s", clr, topLeft);
+
+		memset(&ts, 0, sizeof(struct total_statistics));
+
+		printf("%s", status_string);
+
+		for (i = 0; i < cfg.nb_ports; i++) {
+			port_id = cfg.ports[i].rxtx_port;
+			print_port_stats(port_id);
+
+			ts.total_packets_dropped +=
+				port_statistics.tx_dropped[port_id]
+				+ port_statistics.copy_dropped[port_id];
+			ts.total_packets_tx += port_statistics.tx[port_id];
+			ts.total_packets_rx += port_statistics.rx[port_id];
+
+			if (copy_mode == COPY_MODE_IOAT_NUM) {
+				uint32_t j;
+
+				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+					dev_id = cfg.ports[i].ioat_ids[j];
+					rte_rawdev_xstats_get(dev_id,
+						ids_xstats, xstats, nb_xstats);
+
+					print_rawdev_stats(dev_id, xstats,
+						nb_xstats, names_xstats);
+
+					ts.total_successful_enqueues +=
+						xstats[id_succ_enq];
+					ts.total_failed_enqueues +=
+						xstats[id_fail_enq];
+				}
+			}
+		}
+		printf("\n");
+
+		print_total_stats(&ts);
+	}
+
+	free(names_xstats);
+	free(xstats);
+	free(ids_xstats);
+}
+
+static void
+update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
+{
+	struct rte_ether_hdr *eth;
+	void *tmp;
+
+	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
+
+	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
+	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
+	 */
+	tmp = &eth->d_addr.addr_bytes[0];
+	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
+
+	/* src addr */
+	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], &eth->s_addr);
+}
+
+static inline void
+pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+{
+	/* Copy packet metadata */
+	rte_memcpy(&dst->rearm_data,
+		&src->rearm_data,
+		offsetof(struct rte_mbuf, cacheline1)
+			- offsetof(struct rte_mbuf, rearm_data));
+
+	/* Copy packet data */
+	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+		rte_pktmbuf_mtod(src, char *), src->data_len);
+}
+
+static uint32_t
+ioat_enqueue_packets(struct rte_mbuf **pkts,
+	uint32_t nb_rx, uint16_t dev_id)
+{
+	int ret;
+	uint32_t i;
+	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+		&pkts[0]->rearm_data);
+
+	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+			(void *)pkts_copy, nb_rx);
+
+	if (unlikely(ret < 0))
+		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+	for (i = 0; i < nb_rx; i++) {
+		/* Perform data copy */
+		ret = rte_ioat_enqueue_copy(dev_id,
+			pkts[i]->buf_iova
+				- addr_offset,
+			pkts_copy[i]->buf_iova
+				- addr_offset,
+			rte_pktmbuf_data_len(pkts[i])
+				+ addr_offset,
+			(uintptr_t)pkts[i],
+			(uintptr_t)pkts_copy[i],
+			0 /* nofence */);
+
+		if (ret != 1)
+			break;
+	}
+
+	ret = i;
+	/* Free any not enqueued packets. */
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+		nb_rx - i);
+
+
+	return ret;
+}
+
+/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+static void
+ioat_rx_port(struct rxtx_port_config *rx_config)
+{
+	uint32_t nb_rx, nb_enq, i, j;
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+
+	for (i = 0; i < rx_config->nb_queues; i++) {
+
+		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+			pkts_burst, MAX_PKT_BURST);
+
+		if (nb_rx == 0)
+			continue;
+
+		port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			/* Perform packet hardware copy */
+			nb_enq = ioat_enqueue_packets(pkts_burst,
+				nb_rx, rx_config->ioat_ids[i]);
+			if (nb_enq > 0)
+				rte_ioat_do_copies(rx_config->ioat_ids[i]);
+		} else {
+			/* Perform packet software copy, free source packets */
+			int ret;
+			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst_copy, nb_rx);
+
+			if (unlikely(ret < 0))
+				rte_exit(EXIT_FAILURE,
+					"Unable to allocate memory.\n");
+
+			for (j = 0; j < nb_rx; j++)
+				pktmbuf_sw_copy(pkts_burst[j],
+					pkts_burst_copy[j]);
+
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst, nb_rx);
+
+			nb_enq = rte_ring_enqueue_burst(
+				rx_config->rx_to_tx_ring,
+				(void *)pkts_burst_copy, nb_rx, NULL);
+
+			/* Free any not enqueued packets. */
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&pkts_burst_copy[nb_enq],
+				nb_rx - nb_enq);
+		}
+
+		port_statistics.copy_dropped[rx_config->rxtx_port] +=
+			(nb_rx - nb_enq);
+	}
+}
+
+/* Transmit packets from IOAT rawdev/rte_ring for one port. */
+static void
+ioat_tx_port(struct rxtx_port_config *tx_config)
+{
+	uint32_t i, j, nb_dq = 0;
+	struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
+	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		/* Deque the mbufs from IOAT device. */
+		for (i = 0; i < tx_config->nb_queues; i++) {
+			nb_dq = rte_ioat_completed_copies(
+				tx_config->ioat_ids[i], MAX_PKT_BURST,
+				(void *)mbufs_src, (void *)mbufs_dst);
+
+			if (nb_dq == 0)
+				break;
+
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)mbufs_src, nb_dq);
+
+			/* Update macs if enabled */
+			if (mac_updating) {
+				for (j = 0; j < nb_dq; j++)
+					update_mac_addrs(mbufs_dst[j],
+						tx_config->rxtx_port);
+			}
+
+			const uint16_t nb_tx = rte_eth_tx_burst(
+				tx_config->rxtx_port, 0,
+				(void *)mbufs_dst, nb_dq);
+
+			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_dq))
+				rte_mempool_put_bulk(ioat_pktmbuf_pool,
+					(void *)&mbufs_dst[nb_tx],
+					nb_dq - nb_tx);
+		}
+	} else {
+		/* Deque the mbufs from rx_to_tx_ring. */
+		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+		if (nb_dq == 0)
+			return;
+
+		/* Update macs if enabled */
+		if (mac_updating) {
+			for (i = 0; i < nb_dq; i++)
+				update_mac_addrs(mbufs_dst[i],
+					tx_config->rxtx_port);
+		}
+
+		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+			0, (void *)mbufs_dst, nb_dq);
+
+		port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+		/* Free any unsent packets. */
+		if (unlikely(nb_tx < nb_dq))
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&mbufs_dst[nb_tx],
+				nb_dq - nb_tx);
+	}
+}
+
+/* Main rx processing loop for IOAT rawdev. */
+static void
+rx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_rx_port(&cfg.ports[i]);
+}
+
+/* Main tx processing loop for hardware copy. */
+static void
+tx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main tx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_tx_port(&cfg.ports[i]);
+}
+
+/* Main rx and tx loop if only one slave lcore available */
+static void
+rxtx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
+		" lcore %u\n", rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++) {
+			ioat_rx_port(&cfg.ports[i]);
+			ioat_tx_port(&cfg.ports[i]);
+		}
+}
+
+static void start_forwarding_cores(void)
+{
+	uint32_t lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+			__func__, rte_lcore_id());
+
+	if (cfg.nb_lcores == 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+			NULL, lcore_id);
+	} else if (cfg.nb_lcores > 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+			NULL, lcore_id);
+
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+			lcore_id);
+	}
+}
+
+/* Display usage */
+static void
+ioat_usage(const char *prgname)
+{
+	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
+		"  -p --portmask: hexadecimal bitmask of ports to configure\n"
+		"  -q NQ: number of RX queues per port (default is 1)\n"
+		"  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
+		"      When enabled:\n"
+		"       - The source MAC address is replaced by the TX port MAC address\n"
+		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
+		"  -c --copy-type CT: type of copy: sw|rawdev\n"
+		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
+	       prgname);
+}
+
+static int
+ioat_parse_portmask(const char *portmask)
+{
+	char *end = NULL;
+	unsigned long pm;
+
+	/* Parse hexadecimal string */
+	pm = strtoul(portmask, &end, 16);
+	if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+
+	return pm;
+}
+
+static copy_mode_t
+ioat_parse_copy_mode(const char *copy_mode)
+{
+	if (strcmp(copy_mode, COPY_MODE_SW) == 0)
+		return COPY_MODE_SW_NUM;
+	else if (strcmp(copy_mode, COPY_MODE_IOAT) == 0)
+		return COPY_MODE_IOAT_NUM;
+
+	return COPY_MODE_INVALID_NUM;
+}
+
+/* Parse the argument given in the command line of the application */
+static int
+ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
+{
+	static const char short_options[] =
+		"p:"  /* portmask */
+		"q:"  /* number of RX queues per port */
+		"c:"  /* copy type (sw|rawdev) */
+		"s:"  /* ring size */
+		;
+
+	static const struct option lgopts[] = {
+		{CMD_LINE_OPT_MAC_UPDATING, no_argument, &mac_updating, 1},
+		{CMD_LINE_OPT_NO_MAC_UPDATING, no_argument, &mac_updating, 0},
+		{CMD_LINE_OPT_PORTMASK, required_argument, NULL, 'p'},
+		{CMD_LINE_OPT_NB_QUEUE, required_argument, NULL, 'q'},
+		{CMD_LINE_OPT_COPY_TYPE, required_argument, NULL, 'c'},
+		{CMD_LINE_OPT_RING_SIZE, required_argument, NULL, 's'},
+		{NULL, 0, 0, 0}
+	};
+
+	const unsigned int default_port_mask = (1 << nb_ports) - 1;
+	int opt, ret;
+	char **argvopt;
+	int option_index;
+	char *prgname = argv[0];
+
+	ioat_enabled_port_mask = default_port_mask;
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, short_options,
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		/* portmask */
+		case 'p':
+			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
+			if (ioat_enabled_port_mask & ~default_port_mask ||
+					ioat_enabled_port_mask <= 0) {
+				printf("Invalid portmask, %s, suggest 0x%x\n",
+						optarg, default_port_mask);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'q':
+			nb_queues = atoi(optarg);
+			if (nb_queues == 0 || nb_queues > MAX_RX_QUEUES_COUNT) {
+				printf("Invalid RX queues number %s. Max %u\n",
+					optarg, MAX_RX_QUEUES_COUNT);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'c':
+			copy_mode = ioat_parse_copy_mode(optarg);
+			if (copy_mode == COPY_MODE_INVALID_NUM) {
+				printf("Invalid copy type. Use: sw, rawdev\n");
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 's':
+			ring_size = atoi(optarg);
+			if (ring_size == 0) {
+				printf("Invalid ring size, %s.\n", optarg);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		/* long options */
+		case 0:
+			break;
+
+		default:
+			ioat_usage(prgname);
+			return -1;
+		}
+	}
+
+	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
+	if (optind >= 0)
+		argv[optind-1] = prgname;
+
+	ret = optind-1;
+	optind = 1; /* reset getopt lib */
+	return ret;
+}
+
+/* check link status, return true if at least one port is up */
+static int
+check_link_status(uint32_t port_mask)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	int retval = 0;
+
+	printf("\nChecking link status\n");
+	RTE_ETH_FOREACH_DEV(portid) {
+		if ((port_mask & (1 << portid)) == 0)
+			continue;
+
+		memset(&link, 0, sizeof(link));
+		rte_eth_link_get(portid, &link);
+
+		/* Print link status */
+		if (link.link_status) {
+			printf(
+				"Port %d Link Up. Speed %u Mbps - %s\n",
+				portid, link.link_speed,
+				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+				("full-duplex") : ("half-duplex\n"));
+			retval = 1;
+		} else
+			printf("Port %d Link Down\n", portid);
+	}
+	return retval;
+}
+
+static void
+configure_rawdev_queue(uint32_t dev_id)
+{
+	struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
+	struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+	if (rte_rawdev_configure(dev_id, &info) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_configure()\n");
+	}
+	if (rte_rawdev_start(dev_id) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_start()\n");
+	}
+}
+
+static void
+assign_rawdevs(void)
+{
+	uint16_t nb_rawdev = 0, rdev_id = 0;
+	uint32_t i, j;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+			struct rte_rawdev_info rdev_info = { 0 };
+
+			do {
+				if (rdev_id == rte_rawdev_count())
+					goto end;
+				rte_rawdev_info_get(rdev_id++, &rdev_info);
+			} while (strcmp(rdev_info.driver_name,
+				IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+			cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+			configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+			++nb_rawdev;
+		}
+	}
+end:
+	if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+		rte_exit(EXIT_FAILURE,
+			"Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+			nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+	RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+}
+
+static void
+assign_rings(void)
+{
+	uint32_t i;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		char ring_name[RTE_RING_NAMESIZE];
+
+		snprintf(ring_name, sizeof(ring_name), "rx_to_tx_ring_%u", i);
+		/* Create ring for inter core communication */
+		cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+				ring_name, ring_size,
+				rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
+
+		if (cfg.ports[i].rx_to_tx_ring == NULL)
+			rte_exit(EXIT_FAILURE, "Ring create failed: %s\n",
+					rte_strerror(rte_errno));
+	}
+}
+
+/*
+ * Initializes a given port using global settings and with the RX buffers
+ * coming from the mbuf_pool passed as a parameter.
+ */
+static inline void
+port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+{
+	/* configuring port to use RSS for multiple RX queues */
+	static const struct rte_eth_conf port_conf = {
+		.rxmode = {
+			.mq_mode        = ETH_MQ_RX_RSS,
+			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
+		},
+		.rx_adv_conf = {
+			.rss_conf = {
+				.rss_key = NULL,
+				.rss_hf = ETH_RSS_PROTO_MASK,
+			}
+		}
+	};
+
+	struct rte_eth_rxconf rxq_conf;
+	struct rte_eth_txconf txq_conf;
+	struct rte_eth_conf local_port_conf = port_conf;
+	struct rte_eth_dev_info dev_info;
+	int ret, i;
+
+	/* Skip ports that are not enabled */
+	if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+		printf("Skipping disabled port %u\n", portid);
+		return;
+	}
+
+	/* Init port */
+	printf("Initializing port %u... ", portid);
+	fflush(stdout);
+	rte_eth_dev_info_get(portid, &dev_info);
+	local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+		dev_info.flow_type_rss_offloads;
+	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+		local_port_conf.txmode.offloads |=
+			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Cannot configure device:"
+			" err=%d, port=%u\n", ret, portid);
+
+	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+						&nb_txd);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot adjust number of descriptors: err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+	/* Init RX queues */
+	rxq_conf = dev_info.default_rxconf;
+	rxq_conf.offloads = local_port_conf.rxmode.offloads;
+	for (i = 0; i < nb_queues; i++) {
+		ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+			rte_eth_dev_socket_id(portid), &rxq_conf,
+			mbuf_pool);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE,
+				"rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+				ret, portid, i);
+	}
+
+	/* Init one TX queue on each port */
+	txq_conf = dev_info.default_txconf;
+	txq_conf.offloads = local_port_conf.txmode.offloads;
+	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+			rte_eth_dev_socket_id(portid),
+			&txq_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
+			ret, portid);
+
+	/* Initialize TX buffers */
+	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+			rte_eth_dev_socket_id(portid));
+	if (tx_buffer[portid] == NULL)
+		rte_exit(EXIT_FAILURE,
+			"Cannot allocate buffer for tx on port %u\n",
+			portid);
+
+	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+	ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+			rte_eth_tx_buffer_count_callback,
+			&port_statistics.tx_dropped[portid]);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot set error callback for tx buffer on port %u\n",
+			portid);
+
+	/* Start device */
+	ret = rte_eth_dev_start(portid);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_dev_start:err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_promiscuous_enable(portid);
+
+	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+			portid,
+			ioat_ports_eth_addr[portid].addr_bytes[0],
+			ioat_ports_eth_addr[portid].addr_bytes[1],
+			ioat_ports_eth_addr[portid].addr_bytes[2],
+			ioat_ports_eth_addr[portid].addr_bytes[3],
+			ioat_ports_eth_addr[portid].addr_bytes[4],
+			ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+	cfg.ports[cfg.nb_ports].rxtx_port = portid;
+	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\n\nSignal %d received, preparing to exit...\n",
+			signum);
+		force_quit = true;
+	}
+}
+
+int
+main(int argc, char **argv)
+{
+	int ret;
+	uint16_t nb_ports, portid;
+	uint32_t i;
+	unsigned int nb_mbufs;
+
+	/* Init EAL */
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+	argc -= ret;
+	argv += ret;
+
+	force_quit = false;
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	nb_ports = rte_eth_dev_count_avail();
+	if (nb_ports == 0)
+		rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
+
+	/* Parse application arguments (after the EAL ones) */
+	ret = ioat_parse_args(argc, argv, nb_ports);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid IOAT arguments\n");
+
+	nb_mbufs = RTE_MAX(nb_ports * (nb_queues * (nb_rxd + nb_txd +
+		4 * MAX_PKT_BURST) + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+		MIN_POOL_SIZE);
+
+	/* Create the mbuf pool */
+	ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+		MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+		rte_socket_id());
+	if (ioat_pktmbuf_pool == NULL)
+		rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+	/* Initialise each port */
+	cfg.nb_ports = 0;
+	RTE_ETH_FOREACH_DEV(portid)
+		port_init(portid, ioat_pktmbuf_pool, nb_queues);
+
+	/* Initialize port xstats */
+	memset(&port_statistics, 0, sizeof(port_statistics));
+
+	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
+		sleep(1);
+
+	/* Check if there is enough lcores for all ports. */
+	cfg.nb_lcores = rte_lcore_count() - 1;
+	if (cfg.nb_lcores < 1)
+		rte_exit(EXIT_FAILURE,
+			"There should be at least one slave lcore.\n");
+
+	if (copy_mode == COPY_MODE_IOAT_NUM)
+		assign_rawdevs();
+	else /* copy_mode == COPY_MODE_SW_NUM */
+		assign_rings();
+
+	start_forwarding_cores();
+	/* master core prints stats while other cores forward */
+	print_stats(argv[0]);
+
+	/* force_quit is true when we get here */
+	rte_eal_mp_wait_lcore();
+
+	uint32_t j;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
+		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
+		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+				printf("Stopping rawdev %d\n",
+					cfg.ports[i].ioat_ids[j]);
+				rte_rawdev_stop(cfg.ports[i].ioat_ids[j]);
+			}
+		} else /* copy_mode == COPY_MODE_SW_NUM */
+			rte_ring_free(cfg.ports[i].rx_to_tx_ring);
+	}
+
+	printf("Bye...\n");
+	return 0;
+}
diff --git a/examples/ioat/meson.build b/examples/ioat/meson.build
new file mode 100644
index 000000000..ff56dc99c
--- /dev/null
+++ b/examples/ioat/meson.build
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+deps += ['pmd_ioat']
+
+sources = files(
+	'ioatfwd.c'
+)
diff --git a/examples/meson.build b/examples/meson.build
index a046b74ad..c2e18b59b 100644
--- a/examples/meson.build
+++ b/examples/meson.build
@@ -16,6 +16,7 @@ all_examples = [
 	'eventdev_pipeline', 'exception_path',
 	'fips_validation', 'flow_classify',
 	'flow_filtering', 'helloworld',
+	'ioat',
 	'ip_fragmentation', 'ip_pipeline',
 	'ip_reassembly', 'ipsec-secgw',
 	'ipv4_multicast', 'kni',
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH] examples/ioat: create sample app on ioat driver usage
  2019-09-09  8:29 [dpdk-dev] [PATCH] examples/ioat: create sample app on ioat driver usage Marcin Baran
@ 2019-09-09 13:12 ` Aaron Conole
  2019-09-09 13:58   ` Bruce Richardson
  2019-09-12  9:52 ` Bruce Richardson
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
  2 siblings, 1 reply; 76+ messages in thread
From: Aaron Conole @ 2019-09-09 13:12 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, Pawel Modrak

Marcin Baran <marcinx.baran@intel.com> writes:

> From: Pawel Modrak <pawelx.modrak@intel.com>
>
> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware
> copy, changes packets' MAC addresses (if enabled) and
> forwards them. The patch includes sample application
> as well as it's guide.
>
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  doc/guides/sample_app_ug/index.rst |    1 +
>  doc/guides/sample_app_ug/intro.rst |    4 +
>  doc/guides/sample_app_ug/ioat.rst  |  691 +++++++++++++++++++
>  examples/Makefile                  |    3 +
>  examples/ioat/Makefile             |   54 ++
>  examples/ioat/ioatfwd.c            | 1010 ++++++++++++++++++++++++++++
>  examples/ioat/meson.build          |   13 +
>  examples/meson.build               |    1 +
>  8 files changed, 1777 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
>  create mode 100644 examples/ioat/Makefile
>  create mode 100644 examples/ioat/ioatfwd.c
>  create mode 100644 examples/ioat/meson.build
>
> diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
> index f23f8f59e..a6a1d9e7a 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -23,6 +23,7 @@ Sample Applications User Guides
>      ip_reassembly
>      kernel_nic_interface
>      keep_alive
> +    ioat
>      l2_forward_crypto
>      l2_forward_job_stats
>      l2_forward_real_virtual
> diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
> index 90704194a..74462312f 100644
> --- a/doc/guides/sample_app_ug/intro.rst
> +++ b/doc/guides/sample_app_ug/intro.rst
> @@ -91,6 +91,10 @@ examples are highlighted below.
>    forwarding, or ``l3fwd`` application does forwarding based on Internet
>    Protocol, IPv4 or IPv6 like a simple router.
>  
> +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
> +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
> +  copying packets between two threads.
> +
>  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
>    demonstrates how to distribute packets arriving on an Rx port to different
>    cores for processing and transmission.
> diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
> new file mode 100644
> index 000000000..378d70b81
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/ioat.rst
> @@ -0,0 +1,691 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Sample Application of packet copying using Intel\|reg| QuickData Technology
> +============================================================================
> +
> +Overview
> +--------
> +
> +This sample is intended as a demonstration of the basic components of a DPDK
> +forwarding application and example of how to use IOAT driver API to make
> +packets copies.
> +
> +Also while forwarding, the MAC addresses are affected as follows:
> +
> +*   The source MAC address is replaced by the TX port MAC address
> +
> +*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
> +
> +This application can be used to compare performance of using software packet
> +copy with copy done using a DMA device for different sizes of packets.
> +The example will print out statistics each second. The stats shows
> +received/send packets and packets dropped or failed to copy.
> +
> +Compiling the Application
> +-------------------------
> +
> +To compile the sample application see :doc:`compiling`.
> +
> +The application is located in the ``ioat`` sub-directory.
> +
> +
> +Running the Application
> +-----------------------
> +
> +In order to run the hardware copy application, the copying device
> +needs to be bound to user-space IO driver.
> +
> +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology*
> +guide for information on using the driver.
> +
> +The application requires a number of command line options:
> +
> +.. code-block:: console
> +
> +    ./build/ioatfwd [EAL options] -- -p MASK [-C CT] [--[no-]mac-updating]
> +
> +where,
> +
> +*   p MASK: A hexadecimal bitmask of the ports to configure
> +
> +*   c CT: Performed packet copy type: software (sw) or hardware using
> +    DMA (rawdev)
> +
> +*   s RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for
> +    software copy mode
> +
> +*   --[no-]mac-updating: Whether MAC address of packets should be changed
> +    or not
> +
> +The application can be launched in 2 different configurations:
> +
> +*   Performing software packet copying
> +
> +*   Performing hardware packet copying
> +
> +Each port needs 2 lcores: one of them receives incoming traffic and makes
> +a copy of each packet. The second lcore then updates MAC address and sends
> +the copy. For each configuration an additional lcore is needed since
> +master lcore in use which is responsible for configuration, statistics
> +printing and safe deinitialization of all ports and devices.
> +
> +The application can use a maximum of 8 ports.
> +
> +To run the application in a Linux environment with 3 lcores (one of them
> +is master lcore), 1 port (port 0), software copying and MAC updating issue
> +the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
> +
> +To run the application in a Linux environment with 5 lcores (one of them
> +is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC
> +updating issue the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-4 -n 1 -- -p 0x3 --no-mac-updating -c rawdev
> +
> +Refer to the *DPDK Getting Started Guide* for general information on
> +running applications and the Environment Abstraction Layer (EAL) options.
> +
> +Explanation
> +-----------
> +
> +The following sections provide an explanation of the main components of the
> +code.
> +
> +All DPDK library functions used in the sample code are prefixed with
> +``rte_`` and are explained in detail in the *DPDK API Documentation*.
> +
> +
> +The Main Function
> +~~~~~~~~~~~~~~~~~
> +
> +The ``main()`` function performs the initialization and calls the execution
> +threads for each lcore.
> +
> +The first task is to initialize the Environment Abstraction Layer (EAL).
> +The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
> +function. The value returned is the number of parsed arguments:
> +
> +.. code-block:: c
> +
> +    /* init EAL */
> +    ret = rte_eal_init(argc, argv);
> +    if (ret < 0)
> +        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
> +
> +
> +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
> +used by the application:
> +
> +.. code-block:: c
> +
> +    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
> +        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
> +        MIN_POOL_SIZE);
> +
> +    /* Create the mbuf pool */
> +    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
> +        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
> +        rte_socket_id());
> +    if (ioat_pktmbuf_pool == NULL)
> +        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
> +
> +Mbufs are the packet buffer structure used by DPDK. They are explained in
> +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
> +
> +The ``main()`` function also initializes the ports:
> +
> +.. code-block:: c
> +
> +    /* Initialise each port */
> +    RTE_ETH_FOREACH_DEV(portid) {
> +        port_init(portid, ioat_pktmbuf_pool);
> +    }
> +
> +Each port is configured using ``port_init()``:
> +
> +.. code-block:: c
> +
> +    static inline void
> +    port_init(uint16_t portid, struct rte_mempool *mbuf_pool)
> +    {
> +        struct rte_eth_rxconf rxq_conf;
> +        struct rte_eth_txconf txq_conf;
> +        struct rte_eth_conf local_port_conf = port_conf;
> +        struct rte_eth_dev_info dev_info;
> +        int ret;
> +
> +        /* Skip ports that are not enabled */
> +        if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
> +            printf("Skipping disabled port %u\n", portid);
> +            return;
> +        }
> +
> +        /* Init port */
> +        printf("Initializing port %u... ", portid);
> +        fflush(stdout);
> +        rte_eth_dev_info_get(portid, &dev_info);
> +        if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
> +            local_port_conf.txmode.offloads |=
> +                DEV_TX_OFFLOAD_MBUF_FAST_FREE;
> +        ret = rte_eth_dev_configure(portid, 1, 1, &local_port_conf);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE, "Cannot configure device: err=%d, port=%u\n",
> +                    ret, portid);
> +
> +        ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
> +                            &nb_txd);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                    "Cannot adjust number of descriptors: err=%d, port=%u\n",
> +                    ret, portid);
> +
> +        rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
> +
> +        /* Init one RX queue */
> +        fflush(stdout);
> +        rxq_conf = dev_info.default_rxconf;
> +        rxq_conf.offloads = local_port_conf.rxmode.offloads;
> +        ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd,
> +                        rte_eth_dev_socket_id(portid),
> +                        &rxq_conf,
> +                        mbuf_pool);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup:err=%d, port=%u\n",
> +                    ret, portid);
> +
> +        /* Init one TX queue on each port */
> +        fflush(stdout);
> +        txq_conf = dev_info.default_txconf;
> +        txq_conf.offloads = local_port_conf.txmode.offloads;
> +        ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
> +                rte_eth_dev_socket_id(portid),
> +                &txq_conf);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d, port=%u\n",
> +                ret, portid);
> +
> +        /* Initialize TX buffers */
> +        tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
> +                RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
> +                rte_eth_dev_socket_id(portid));
> +        if (tx_buffer[portid] == NULL)
> +            rte_exit(EXIT_FAILURE, "Cannot allocate buffer for tx "
> +                    "on port %u\n", portid);
> +
> +        rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
> +
> +        ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
> +                rte_eth_tx_buffer_count_callback,
> +                &port_statistics[portid].tx_dropped);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +            "Cannot set error callback for tx buffer on port %u\n",
> +                    portid);
> +
> +        /* Start device */
> +        ret = rte_eth_dev_start(portid);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
> +                    ret, portid);
> +
> +        rte_eth_promiscuous_enable(portid);
> +
> +        printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
> +                portid,
> +                ioat_ports_eth_addr[portid].addr_bytes[0],
> +                ioat_ports_eth_addr[portid].addr_bytes[1],
> +                ioat_ports_eth_addr[portid].addr_bytes[2],
> +                ioat_ports_eth_addr[portid].addr_bytes[3],
> +                ioat_ports_eth_addr[portid].addr_bytes[4],
> +                ioat_ports_eth_addr[portid].addr_bytes[5]);
> +    }
> +
> +The Ethernet ports are configured with local settings using the
> +``rte_eth_dev_configure()`` function and the ``port_conf`` struct:
> +
> +.. code-block:: c
> +
> +    static struct rte_eth_conf port_conf = {
> +        .rxmode = {
> +            .max_rx_pkt_len = RTE_ETHER_MAX_LEN,
> +        },
> +    };
> +
> +For this example the ports are set up with 1 RX and 1 TX queue using the
> +``rte_eth_rx_queue_setup()`` and ``rte_eth_tx_queue_setup()`` functions.
> +
> +The Ethernet port is then started:
> +
> +.. code-block:: c
> +
> +    ret = rte_eth_dev_start(portid);
> +    if (ret < 0)
> +        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
> +            ret, portid);
> +
> +
> +Finally the RX port is set in promiscuous mode:
> +
> +.. code-block:: c
> +
> +    rte_eth_promiscuous_enable(portid);
> +
> +
> +After that each port application assigns resources needed.
> +
> +.. code-block:: c
> +
> +    check_link_status(ioat_enabled_port_mask);
> +
> +    if (!cfg.nb_ports) {
> +        rte_exit(EXIT_FAILURE,
> +            "All available ports are disabled. Please set portmask.\n");
> +    }
> +
> +    /* Check if there is enough lcores for all ports. */
> +    cfg.nb_lcores = rte_lcore_count() - 1;
> +    if (cfg.nb_lcores < 1)
> +        rte_exit(EXIT_FAILURE,
> +            "There should be at least one slave lcore.\n");
> +
> +    ret = 0;
> +
> +    if (copy_mode == COPY_MODE_IOAT_NUM) {
> +        assign_rawdevs();
> +    } else /* copy_mode == COPY_MODE_SW_NUM */ {
> +        assign_rings();
> +    }
> +
> +A link status is checked of each port enabled by port mask
> +using ``check_link_status()`` function.
> +
> +.. code-block:: c
> +
> +    /* Check the link status of all ports in up to 9s, and print them finally */
> +    static void
> +    check_link_status(uint32_t port_mask)
> +    {
> +
> +        uint16_t portid;
> +        struct rte_eth_link link;
> +
> +        cfg.nb_ports = 0;
> +
> +        printf("\nChecking link status\n");
> +        fflush(stdout);
> +        RTE_ETH_FOREACH_DEV(portid) {
> +            if (force_quit)
> +                return;
> +            if ((port_mask & (1 << portid)) == 0)
> +                continue;
> +
> +            store_port_nb(portid);
> +
> +            memset(&link, 0, sizeof(link));
> +            rte_eth_link_get(portid, &link);
> +
> +            /* Print link status */
> +            if (link.link_status) {
> +                printf(
> +                    "Port %d Link Up. Speed %u Mbps - %s\n",
> +                    portid, link.link_speed,
> +                    (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
> +                    ("full-duplex") : ("half-duplex\n"));
> +            }
> +            else
> +                printf("Port %d Link Down\n", portid);
> +        }
> +    }
> +
> +Depending on mode set (whether copy should be done by software or by hardware)
> +special structures are assigned to each port. If software copy was chosen,
> +application have to assign ring structures for packet exchanging between lcores
> +assigned to ports.
> +
> +.. code-block:: c
> +
> +    static void
> +    assign_rings(void)
> +    {
> +        uint32_t i;
> +
> +        for (i = 0; i < cfg.nb_ports; i++) {
> +            char ring_name[20];
> +
> +            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
> +            /* Create ring for inter core communication */
> +            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
> +                    ring_name, ring_size,
> +                    rte_socket_id(), RING_F_SP_ENQ);
> +
> +            if (cfg.ports[i].rx_to_tx_ring == NULL)
> +                rte_exit(EXIT_FAILURE, "%s\n",
> +                        rte_strerror(rte_errno));
> +        }
> +    }
> +
> +
> +When using hardware copy each port is assigned an IOAT device
> +(``assign_rawdevs()``) using IOAT Rawdev Driver API functions:
> +
> +.. code-block:: c
> +
> +    static void
> +    assign_rawdevs(void)
> +    {
> +        uint16_t nb_rawdev = 0;
> +        uint32_t i;
> +
> +        for (i = 0; i < cfg.nb_ports; i++) {
> +            struct rte_rawdev_info rdev_info = {0};
> +            rte_rawdev_info_get(0, &rdev_info);
> +
> +            if (strcmp(rdev_info.driver_name, "rawdev_ioat") == 0) {
> +                configure_rawdev_queue(i);
> +                cfg.ports[i].dev_id = i;
> +                ++nb_rawdev;
> +            }
> +        }
> +
> +        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
> +
> +        if (nb_rawdev < cfg.nb_ports)
> +            rte_exit(EXIT_FAILURE, "Not enough IOAT rawdevs (%u) for ports (%u).\n",
> +                    nb_rawdev, cfg.nb_ports);
> +    }
> +
> +
> +The initialization of hardware device is done by ``rte_rawdev_configure()``
> +function and ``rte_rawdev_info`` struct. After configuration the device is
> +started using ``rte_rawdev_start()`` function. Each of the above operations
> +is done in ``configure_rawdev_queue()``.
> +
> +.. code-block:: c
> +
> +    static void
> +    configure_rawdev_queue(uint32_t dev_id)
> +    {
> +        struct rte_rawdev_info info = { .dev_private = &dev_config };
> +
> +        /* Configure hardware copy device */
> +        dev_config.ring_size = ring_size;
> +
> +        if (rte_rawdev_configure(dev_id, &info) != 0) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error with rte_rawdev_configure()\n");
> +        }
> +        rte_rawdev_info_get(dev_id, &info);
> +        if (dev_config.ring_size != ring_size) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error, ring size is not %d (%d)\n",
> +                ring_size, (int)dev_config.ring_size);
> +        }
> +        if (rte_rawdev_start(dev_id) != 0) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error with rte_rawdev_start()\n");
> +        }
> +    }
> +
> +If initialization is successful memory for hardware device
> +statistics is allocated.
> +
> +Finally ``main()`` functions starts all processing lcores and starts
> +printing stats in a loop on master lcore. The application can be
> +interrupted and closed using ``Ctrl-C``. The master lcore waits for
> +all slave processes to finish, deallocates resources and exits.
> +
> +The processing lcores launching function are described below.
> +
> +The Lcores Launching Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +As described above ``main()`` function invokes ``run_transmission()``
> +function in order to start processing for each lcore:
> +
> +.. code-block:: c
> +
> +    static void run_transmission(void)
> +    {
> +        uint32_t lcore_id = rte_lcore_id();
> +
> +        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
> +                __func__, rte_lcore_id());
> +
> +        if (cfg.nb_lcores == 1) {
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop, NULL, lcore_id);
> +        } else if (cfg.nb_lcores > 1) {
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)rx_main_loop, NULL, lcore_id);
> +
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL, lcore_id);
> +        }
> +    }
> +
> +The function launches rx/tx processing functions on configured lcores
> +for each port using ``rte_eal_remote_launch()``. The configured ports,
> +their number and number of assigned lcores are stored in user-defined
> +``rxtx_transmission_config`` struct that is initialized before launching
> +tasks:
> +
> +.. code-block:: c
> +
> +    struct rxtx_transmission_config {
> +        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
> +        uint16_t nb_ports;
> +        uint16_t nb_lcores;
> +    };
> +
> +The Lcores Processing Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +For receiving packets on each port an ``ioat_rx_port()`` function is used.
> +Depending on mode the user chose, it will enqueue packets to IOAT rawdev
> +and then invoke copy process (hardware copy), or perform software copy
> +of each packet using ``pktmbuf_sw_copy()`` function and enqueue them to
> +rte_ring:
> +
> +.. code-block:: c
> +
> +    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
> +    static void
> +    ioat_rx_port(struct rxtx_port_config *rx_config)
> +    {
> +        uint32_t nb_rx, nb_enq, i;
> +        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
> +
> +        nb_rx = rte_eth_rx_burst(rx_config->rx_portId, 0,
> +            pkts_burst, MAX_PKT_BURST);
> +
> +        if (nb_rx == 0)
> +            return;
> +
> +        port_statistics[rx_config->rx_portId].rx += nb_rx;
> +
> +        if (copy_mode == COPY_MODE_IOAT_NUM) {
> +            /* Perform packet hardware copy */
> +            nb_enq = ioat_enqueue_packets(rx_config,
> +                pkts_burst, nb_rx);
> +
> +            if (nb_enq > 0)
> +                rte_ioat_do_copies(rx_config->dev_id);
> +        } else {
> +            /* Perform packet software copy, free source packets */
> +            int ret;
> +            struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> +
> +            ret = rte_pktmbuf_alloc_bulk(ioat_pktmbuf_pool,
> +                    pkts_burst_copy, nb_rx);
> +
> +            if (unlikely(ret < 0))
> +                rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
> +
> +            for (i = 0; i < nb_rx; i++) {
> +                pktmbuf_sw_copy(pkts_burst[i], pkts_burst_copy[i]);
> +                rte_pktmbuf_free(pkts_burst[i]);
> +            }
> +
> +            nb_enq = rte_ring_enqueue_burst(rx_config->rx_to_tx_ring,
> +                (void *)pkts_burst_copy, nb_rx, NULL);
> +
> +            /* Free any not enqueued packets. */
> +            for (i = nb_enq; i < nb_rx; i++)
> +                rte_pktmbuf_free(pkts_burst_copy[i]);
> +        }
> +
> +        port_statistics[rx_config->rx_portId].copy_dropped
> +            += (nb_rx - nb_enq);
> +    }
> +
> +The packets are received in burst mode using ``rte_eth_rx_burst()``
> +function. When using hardware copy mode the packets are enqueued in
> +copying device's buffer using ``ioat_enqueue_packets()`` which calls
> +``rte_ioat_enqueue_copy()``. When all received packets are in the
> +buffer the copies are invoked by calling ``rte_ioat_do_copies()``.
> +Function ``rte_ioat_enqueue_copy()`` operates on physical address of
> +the packet. Structure ``rte_mbuf`` contains only physical address to
> +start of the data buffer (``buf_iova``). Thus the address is shifted
> +by ``addr_offset`` value in order to get pointer to ``rearm_data``
> +member of ``rte_mbuf``. That way the packet is copied all at once
> +(with data and metadata).
> +
> +.. code-block:: c
> +
> +    static uint32_t
> +    ioat_enqueue_packets(struct rxtx_port_config *rx_config,
> +        struct rte_mbuf **pkts, uint32_t nb_rx)
> +    {
> +        int ret;
> +        uint32_t i;
> +        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
> +
> +        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
> +            &pkts[0]->rearm_data);
> +
> +        ret = rte_pktmbuf_alloc_bulk(ioat_pktmbuf_pool, pkts_copy, nb_rx);
> +
> +        if (unlikely(ret < 0))
> +            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
> +
> +        for (i = 0; i < nb_rx; i++) {
> +            /* Perform data copy */
> +            ret = rte_ioat_enqueue_copy(rx_config->dev_id,
> +                pkts[i]->buf_iova
> +                    - addr_offset,
> +                pkts_copy[i]->buf_iova
> +                    - addr_offset,
> +                rte_pktmbuf_data_len(pkts[i])
> +                    + addr_offset,
> +                (uintptr_t)pkts[i],
> +                (uintptr_t)pkts_copy[i],
> +                0 /* nofence */);
> +
> +            if (ret != 1)
> +                break;
> +        }
> +
> +        ret = i;
> +        /* Free any not enqueued packets. */
> +        for (; i < nb_rx; i++) {
> +            rte_pktmbuf_free(pkts[i]);
> +            rte_pktmbuf_free(pkts_copy[i]);
> +        }
> +
> +        return ret;
> +    }
> +
> +
> +All done copies are processed by ``ioat_tx_port()`` function. When using
> +hardware copy mode the function invokes ``rte_ioat_completed_copies()``
> +to gather copied packets. If software copy mode is used the function
> +dequeues copied packets from rte_ring. Then each packet MAC address
> +is changed if it was enabled. After that copies are sent in burst mode
> +using `` rte_eth_tx_burst()``.
> +
> +
> +.. code-block:: c
> +
> +    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
> +    static void
> +    ioat_tx_port(struct rxtx_port_config *tx_config)
> +    {
> +        uint32_t i, nb_dq;
> +        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
> +        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
> +
> +        if (copy_mode == COPY_MODE_IOAT_NUM) {
> +            /* Deque the mbufs from IOAT device. */
> +            nb_dq = rte_ioat_completed_copies(tx_config->dev_id,
> +                MAX_PKT_BURST, (void *)mbufs_src, (void *)mbufs_dst);
> +        } else {
> +            /* Deque the mbufs from rx_to_tx_ring. */
> +            nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
> +                (void *)mbufs_dst, MAX_PKT_BURST, NULL);
> +        }
> +
> +        if (nb_dq == 0)
> +            return;
> +
> +        /* Free source packets */
> +        if (copy_mode == COPY_MODE_IOAT_NUM) {
> +            for (i = 0; i < nb_dq; i++)
> +                rte_pktmbuf_free(mbufs_src[i]);
> +        }
> +
> +        /* Update macs if enabled */
> +        if (mac_updating) {
> +            for (i = 0; i < nb_dq; i++)
> +                update_mac_addrs(mbufs_dst[i],
> +                    tx_config->tx_portId);
> +        }
> +
> +        const uint16_t nb_tx = rte_eth_tx_burst(tx_config->tx_portId,
> +            0, (void *)mbufs_dst, nb_dq);
> +
> +        port_statistics[tx_config->tx_portId].tx += nb_tx;
> +
> +        /* Free any unsent packets. */
> +        if (unlikely(nb_tx < nb_dq)) {
> +            for (i = nb_tx; i < nb_dq; i++)
> +                rte_pktmbuf_free(mbufs_dst[i]);
> +        }
> +    }
> +
> +The Packet Copying Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In order to perform packet copy there is a user-defined function
> +``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
> +metadata from source packet to new mbuf, and then copying a data
> +chunk of source packet. Both memory copies are done using
> +``rte_memcpy()``:
> +
> +.. code-block:: c
> +
> +    static inline void
> +    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
> +    {
> +        /* Copy packet metadata */
> +        rte_memcpy(&dst->rearm_data,
> +            &src->rearm_data,
> +            offsetof(struct rte_mbuf, cacheline1)
> +                - offsetof(struct rte_mbuf, rearm_data));
> +
> +        /* Copy packet data */
> +        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
> +            rte_pktmbuf_mtod(src, char *), src->data_len);
> +    }
> +
> +The metadata in this example is copied from ``rearm_data`` member of
> +``rte_mbuf`` struct up to ``cacheline1``.
> +
> +In order to understand why software packet copying is done as shown
> +above please refer to the "Mbuf Library" section of the
> +*DPDK Programmer's Guide*.
> \ No newline at end of file
> diff --git a/examples/Makefile b/examples/Makefile
> index de11dd487..3cb313d7d 100644
> --- a/examples/Makefile
> +++ b/examples/Makefile
> @@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation
>  DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify
>  DIRS-y += flow_filtering
>  DIRS-y += helloworld
> +ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y)
> +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat
> +endif
>  DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline
>  ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
>  DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
> diff --git a/examples/ioat/Makefile b/examples/ioat/Makefile
> new file mode 100644
> index 000000000..2a4d1da2d
> --- /dev/null
> +++ b/examples/ioat/Makefile
> @@ -0,0 +1,54 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2019 Intel Corporation
> +
> +# binary name
> +APP = ioatfwd
> +
> +# all source are stored in SRCS-y
> +SRCS-y := ioatfwd.c
> +
> +# Build using pkg-config variables if possible
> +ifeq ($(shell pkg-config --exists libdpdk && echo 0),0)
> +
> +all: shared
> +.PHONY: shared static
> +shared: build/$(APP)-shared
> +	ln -sf $(APP)-shared build/$(APP)
> +static: build/$(APP)-static
> +	ln -sf $(APP)-static build/$(APP)
> +
> +PC_FILE := $(shell pkg-config --path libdpdk)
> +CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
> +LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
> +LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
> +
> +build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
> +	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
> +
> +build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
> +	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
> +
> +build:
> +	@mkdir -p $@
> +
> +.PHONY: clean
> +clean:
> +	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
> +	test -d build && rmdir -p build || true
> +
> +else # Build using legacy build system
> +ifeq ($(RTE_SDK),)
> +$(error "Please define RTE_SDK environment variable")
> +endif
> +
> +# Default target, detect a build directory, by looking for a path with a .config
> +RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config)))))
> +
> +include $(RTE_SDK)/mk/rte.vars.mk
> +
> +
> +CFLAGS += -O3
> +CFLAGS += $(WERROR_FLAGS)
> +
> +include $(RTE_SDK)/mk/rte.extapp.mk
> +endif
> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
> new file mode 100644
> index 000000000..8463d82f3
> --- /dev/null
> +++ b/examples/ioat/ioatfwd.c
> @@ -0,0 +1,1010 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2019 Intel Corporation
> + */
> +
> +#include <stdint.h>
> +#include <getopt.h>
> +#include <signal.h>
> +#include <stdbool.h>
> +#include <unistd.h>
> +
> +#include <rte_malloc.h>
> +#include <rte_ethdev.h>
> +#include <rte_rawdev.h>
> +#include <rte_ioat_rawdev.h>
> +
> +/* size of ring used for software copying between rx and tx. */
> +#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
> +#define MAX_PKT_BURST 32
> +#define MEMPOOL_CACHE_SIZE 512
> +#define MIN_POOL_SIZE 65536U
> +#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
> +#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
> +#define CMD_LINE_OPT_PORTMASK "portmask"
> +#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
> +#define CMD_LINE_OPT_COPY_TYPE "copy-type"
> +#define CMD_LINE_OPT_RING_SIZE "ring-size"
> +
> +/* configurable number of RX/TX ring descriptors */
> +#define RX_DEFAULT_RINGSIZE 1024
> +#define TX_DEFAULT_RINGSIZE 1024
> +
> +/* max number of RX queues per port */
> +#define MAX_RX_QUEUES_COUNT 8
> +
> +struct rxtx_port_config {
> +	/* common config */
> +	uint16_t rxtx_port;
> +	uint16_t nb_queues;
> +	/* for software copy mode */
> +	struct rte_ring *rx_to_tx_ring;
> +	/* for IOAT rawdev copy mode */
> +	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT];
> +};
> +
> +struct rxtx_transmission_config {
> +	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
> +	uint16_t nb_ports;
> +	uint16_t nb_lcores;
> +};
> +
> +/* per-port statistics struct */
> +struct ioat_port_statistics {
> +	uint64_t rx[RTE_MAX_ETHPORTS];
> +	uint64_t tx[RTE_MAX_ETHPORTS];
> +	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
> +	uint64_t copy_dropped[RTE_MAX_ETHPORTS];
> +};
> +struct ioat_port_statistics port_statistics;
> +
> +struct total_statistics {
> +	uint64_t total_packets_dropped;
> +	uint64_t total_packets_tx;
> +	uint64_t total_packets_rx;
> +	uint64_t total_successful_enqueues;
> +	uint64_t total_failed_enqueues;
> +};
> +
> +typedef enum copy_mode_t {
> +#define COPY_MODE_SW "sw"
> +	COPY_MODE_SW_NUM,
> +#define COPY_MODE_IOAT "rawdev"
> +	COPY_MODE_IOAT_NUM,
> +	COPY_MODE_INVALID_NUM,
> +	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
> +} copy_mode_t;
> +
> +/* mask of enabled ports */
> +static uint32_t ioat_enabled_port_mask;
> +
> +/* number of RX queues per port */
> +static uint16_t nb_queues = 1;
> +
> +/* MAC updating enabled by default. */
> +static int mac_updating = 1;
> +
> +/* hardare copy mode enabled by default. */
> +static copy_mode_t copy_mode = COPY_MODE_IOAT_NUM;
> +
> +/* size of IOAT rawdev ring for hardware copy mode or
> + * rte_ring for software copy mode
> + */
> +static unsigned short ring_size = 2048;
> +
> +/* global transmission config */
> +struct rxtx_transmission_config cfg;
> +
> +/* configurable number of RX/TX ring descriptors */
> +static uint16_t nb_rxd = RX_DEFAULT_RINGSIZE;
> +static uint16_t nb_txd = TX_DEFAULT_RINGSIZE;
> +
> +static volatile bool force_quit;
> +
> +/* ethernet addresses of ports */
> +static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
> +
> +static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
> +struct rte_mempool *ioat_pktmbuf_pool;
> +
> +/* Print out statistics for one port. */
> +static void
> +print_port_stats(uint16_t port_id)
> +{
> +	printf("\nStatistics for port %u ------------------------------"
> +		"\nPackets sent: %34"PRIu64
> +		"\nPackets received: %30"PRIu64
> +		"\nPackets dropped on tx: %25"PRIu64
> +		"\nPackets dropped on copy: %23"PRIu64,
> +		port_id,
> +		port_statistics.tx[port_id],
> +		port_statistics.rx[port_id],
> +		port_statistics.tx_dropped[port_id],
> +		port_statistics.copy_dropped[port_id]);
> +}
> +
> +/* Print out statistics for one IOAT rawdev device. */
> +static void
> +print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
> +	uint16_t nb_xstats, struct rte_rawdev_xstats_name *names_xstats)
> +{
> +	uint16_t i;
> +
> +	printf("\nIOAT channel %u", dev_id);
> +	for (i = 0; i < nb_xstats; i++)
> +		if (strstr(names_xstats[i].name, "enqueues"))
> +			printf("\n\t %s: %*"PRIu64,
> +				names_xstats[i].name,
> +				(int)(37 - strlen(names_xstats[i].name)),
> +				xstats[i]);
> +}
> +
> +static void
> +print_total_stats(struct total_statistics *ts)
> +{
> +	printf("\nAggregate statistics ==============================="
> +		"\nTotal packets sent: %28"PRIu64
> +		"\nTotal packets received: %24"PRIu64
> +		"\nTotal packets dropped: %25"PRIu64,
> +		ts->total_packets_tx,
> +		ts->total_packets_rx,
> +		ts->total_packets_dropped);
> +
> +	if (copy_mode == COPY_MODE_IOAT_NUM) {
> +		printf("\nTotal IOAT successful enqueues: %16"PRIu64
> +			"\nTotal IOAT failed enqueues: %20"PRIu64,
> +			ts->total_successful_enqueues,
> +			ts->total_failed_enqueues);
> +	}
> +
> +	printf("\n====================================================\n");
> +}
> +
> +/* Print out statistics on packets dropped. */
> +static void
> +print_stats(char *prgname)
> +{
> +	struct total_statistics ts;
> +	uint32_t i, port_id, dev_id;
> +	struct rte_rawdev_xstats_name *names_xstats;
> +	uint64_t *xstats;
> +	unsigned int *ids_xstats;
> +	unsigned int nb_xstats, id_fail_enq, id_succ_enq;
> +	char status_string[120]; /* to print at the top of the output */
> +	int status_strlen;
> +
> +
> +	const char clr[] = { 27, '[', '2', 'J', '\0' };
> +	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
> +
> +	status_strlen = snprintf(status_string, sizeof(status_string),
> +		"%s, ", prgname);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Worker Threads = %d, ",
> +		rte_lcore_count() > 2 ? 2 : 1);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
> +		COPY_MODE_SW : COPY_MODE_IOAT);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Updating MAC = %s, ", mac_updating ?
> +		"enabled" : "disabled");
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Rx Queues = %d, ", nb_queues);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Ring Size = %d\n", ring_size);
> +
> +	/* Allocate memory for xstats names and values */
> +	nb_xstats = rte_rawdev_xstats_names_get(
> +			cfg.ports[0].ioat_ids[0], NULL, 0);
> +
> +	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
> +	if (names_xstats == NULL) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error allocating xstat names memory\n");
> +	}
> +	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
> +			names_xstats, nb_xstats);
> +
> +	ids_xstats = malloc(sizeof(*ids_xstats) * nb_xstats);
> +	if (ids_xstats == NULL) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error allocating xstat ids_xstats memory\n");
> +	}
> +
> +	for (i = 0; i < nb_xstats; i++)
> +		ids_xstats[i] = i;
> +
> +	xstats = malloc(sizeof(*xstats) * nb_xstats);
> +	if (xstats == NULL) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error allocating xstat memory\n");
> +	}
> +
> +	/* Get failed/successful enqueues stats index */
> +	id_fail_enq = id_succ_enq = nb_xstats;
> +	for (i = 0; i < nb_xstats; i++) {
> +		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
> +			id_fail_enq = i;
> +		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
> +			id_succ_enq = i;
> +		if (id_fail_enq < nb_xstats && id_succ_enq < nb_xstats)
> +			break;
> +	}
> +	if (id_fail_enq == nb_xstats || id_succ_enq == nb_xstats) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error getting failed/successful enqueues stats index\n");
> +	}
> +
> +	while (!force_quit) {
> +		/* Sleep for 1 second each round - init sleep allows reading
> +		 * messages from app startup.
> +		 */
> +		sleep(1);
> +
> +		/* Clear screen and move to top left */
> +		printf("%s%s", clr, topLeft);
> +
> +		memset(&ts, 0, sizeof(struct total_statistics));
> +
> +		printf("%s", status_string);
> +
> +		for (i = 0; i < cfg.nb_ports; i++) {
> +			port_id = cfg.ports[i].rxtx_port;
> +			print_port_stats(port_id);
> +
> +			ts.total_packets_dropped +=
> +				port_statistics.tx_dropped[port_id]
> +				+ port_statistics.copy_dropped[port_id];
> +			ts.total_packets_tx += port_statistics.tx[port_id];
> +			ts.total_packets_rx += port_statistics.rx[port_id];
> +
> +			if (copy_mode == COPY_MODE_IOAT_NUM) {
> +				uint32_t j;
> +
> +				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
> +					dev_id = cfg.ports[i].ioat_ids[j];
> +					rte_rawdev_xstats_get(dev_id,
> +						ids_xstats, xstats, nb_xstats);
> +
> +					print_rawdev_stats(dev_id, xstats,
> +						nb_xstats, names_xstats);
> +
> +					ts.total_successful_enqueues +=
> +						xstats[id_succ_enq];
> +					ts.total_failed_enqueues +=
> +						xstats[id_fail_enq];
> +				}
> +			}
> +		}
> +		printf("\n");
> +
> +		print_total_stats(&ts);
> +	}
> +
> +	free(names_xstats);
> +	free(xstats);
> +	free(ids_xstats);
> +}
> +
> +static void
> +update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
> +{
> +	struct rte_ether_hdr *eth;
> +	void *tmp;
> +
> +	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
> +
> +	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
> +	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
> +	 */
> +	tmp = &eth->d_addr.addr_bytes[0];
> +	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
> +
> +	/* src addr */
> +	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], &eth->s_addr);
> +}
> +
> +static inline void
> +pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
> +{
> +	/* Copy packet metadata */
> +	rte_memcpy(&dst->rearm_data,
> +		&src->rearm_data,
> +		offsetof(struct rte_mbuf, cacheline1)
> +			- offsetof(struct rte_mbuf, rearm_data));
> +
> +	/* Copy packet data */
> +	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
> +		rte_pktmbuf_mtod(src, char *), src->data_len);
> +}
> +
> +static uint32_t
> +ioat_enqueue_packets(struct rte_mbuf **pkts,
> +	uint32_t nb_rx, uint16_t dev_id)
> +{
> +	int ret;
> +	uint32_t i;
> +	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
> +
> +	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
> +		&pkts[0]->rearm_data);
> +
> +	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +			(void *)pkts_copy, nb_rx);
> +
> +	if (unlikely(ret < 0))
> +		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
> +
> +	for (i = 0; i < nb_rx; i++) {
> +		/* Perform data copy */
> +		ret = rte_ioat_enqueue_copy(dev_id,
> +			pkts[i]->buf_iova
> +				- addr_offset,
> +			pkts_copy[i]->buf_iova
> +				- addr_offset,
> +			rte_pktmbuf_data_len(pkts[i])
> +				+ addr_offset,
> +			(uintptr_t)pkts[i],
> +			(uintptr_t)pkts_copy[i],
> +			0 /* nofence */);
> +
> +		if (ret != 1)
> +			break;
> +	}
> +
> +	ret = i;
> +	/* Free any not enqueued packets. */
> +	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
> +	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
> +		nb_rx - i);
> +
> +
> +	return ret;
> +}
> +
> +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
> +static void
> +ioat_rx_port(struct rxtx_port_config *rx_config)
> +{
> +	uint32_t nb_rx, nb_enq, i, j;
> +	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
> +
> +	for (i = 0; i < rx_config->nb_queues; i++) {
> +
> +		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
> +			pkts_burst, MAX_PKT_BURST);
> +
> +		if (nb_rx == 0)
> +			continue;
> +
> +		port_statistics.rx[rx_config->rxtx_port] += nb_rx;
> +
> +		if (copy_mode == COPY_MODE_IOAT_NUM) {
> +			/* Perform packet hardware copy */
> +			nb_enq = ioat_enqueue_packets(pkts_burst,
> +				nb_rx, rx_config->ioat_ids[i]);
> +			if (nb_enq > 0)
> +				rte_ioat_do_copies(rx_config->ioat_ids[i]);
> +		} else {
> +			/* Perform packet software copy, free source packets */
> +			int ret;
> +			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> +
> +			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +				(void *)pkts_burst_copy, nb_rx);
> +
> +			if (unlikely(ret < 0))
> +				rte_exit(EXIT_FAILURE,
> +					"Unable to allocate memory.\n");
> +
> +			for (j = 0; j < nb_rx; j++)
> +				pktmbuf_sw_copy(pkts_burst[j],
> +					pkts_burst_copy[j]);
> +
> +			rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +				(void *)pkts_burst, nb_rx);
> +
> +			nb_enq = rte_ring_enqueue_burst(
> +				rx_config->rx_to_tx_ring,
> +				(void *)pkts_burst_copy, nb_rx, NULL);
> +
> +			/* Free any not enqueued packets. */
> +			rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +				(void *)&pkts_burst_copy[nb_enq],
> +				nb_rx - nb_enq);
> +		}
> +
> +		port_statistics.copy_dropped[rx_config->rxtx_port] +=
> +			(nb_rx - nb_enq);
> +	}
> +}
> +
> +/* Transmit packets from IOAT rawdev/rte_ring for one port. */
> +static void
> +ioat_tx_port(struct rxtx_port_config *tx_config)
> +{
> +	uint32_t i, j, nb_dq = 0;
> +	struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
> +	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
> +
> +	if (copy_mode == COPY_MODE_IOAT_NUM) {
> +		/* Deque the mbufs from IOAT device. */
> +		for (i = 0; i < tx_config->nb_queues; i++) {
> +			nb_dq = rte_ioat_completed_copies(
> +				tx_config->ioat_ids[i], MAX_PKT_BURST,
> +				(void *)mbufs_src, (void *)mbufs_dst);
> +
> +			if (nb_dq == 0)
> +				break;
> +
> +			rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +				(void *)mbufs_src, nb_dq);
> +
> +			/* Update macs if enabled */
> +			if (mac_updating) {
> +				for (j = 0; j < nb_dq; j++)
> +					update_mac_addrs(mbufs_dst[j],
> +						tx_config->rxtx_port);
> +			}
> +
> +			const uint16_t nb_tx = rte_eth_tx_burst(
> +				tx_config->rxtx_port, 0,
> +				(void *)mbufs_dst, nb_dq);
> +
> +			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
> +
> +			/* Free any unsent packets. */
> +			if (unlikely(nb_tx < nb_dq))
> +				rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +					(void *)&mbufs_dst[nb_tx],
> +					nb_dq - nb_tx);
> +		}
> +	} else {
> +		/* Deque the mbufs from rx_to_tx_ring. */
> +		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
> +			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
> +
> +		if (nb_dq == 0)
> +			return;
> +
> +		/* Update macs if enabled */
> +		if (mac_updating) {
> +			for (i = 0; i < nb_dq; i++)
> +				update_mac_addrs(mbufs_dst[i],
> +					tx_config->rxtx_port);
> +		}
> +
> +		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
> +			0, (void *)mbufs_dst, nb_dq);
> +
> +		port_statistics.tx[tx_config->rxtx_port] += nb_tx;
> +
> +		/* Free any unsent packets. */
> +		if (unlikely(nb_tx < nb_dq))
> +			rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +				(void *)&mbufs_dst[nb_tx],
> +				nb_dq - nb_tx);
> +	}
> +}
> +
> +/* Main rx processing loop for IOAT rawdev. */
> +static void
> +rx_main_loop(void)
> +{
> +	uint16_t i;
> +	uint16_t nb_ports = cfg.nb_ports;
> +
> +	RTE_LOG(INFO, IOAT, "Entering main rx loop for copy on lcore %u\n",
> +		rte_lcore_id());
> +
> +	while (!force_quit)
> +		for (i = 0; i < nb_ports; i++)
> +			ioat_rx_port(&cfg.ports[i]);
> +}
> +
> +/* Main tx processing loop for hardware copy. */
> +static void
> +tx_main_loop(void)
> +{
> +	uint16_t i;
> +	uint16_t nb_ports = cfg.nb_ports;
> +
> +	RTE_LOG(INFO, IOAT, "Entering main tx loop for copy on lcore %u\n",
> +		rte_lcore_id());
> +
> +	while (!force_quit)
> +		for (i = 0; i < nb_ports; i++)
> +			ioat_tx_port(&cfg.ports[i]);
> +}
> +
> +/* Main rx and tx loop if only one slave lcore available */
> +static void
> +rxtx_main_loop(void)
> +{
> +	uint16_t i;
> +	uint16_t nb_ports = cfg.nb_ports;
> +
> +	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
> +		" lcore %u\n", rte_lcore_id());
> +
> +	while (!force_quit)
> +		for (i = 0; i < nb_ports; i++) {
> +			ioat_rx_port(&cfg.ports[i]);
> +			ioat_tx_port(&cfg.ports[i]);
> +		}
> +}
> +
> +static void start_forwarding_cores(void)
> +{
> +	uint32_t lcore_id = rte_lcore_id();
> +
> +	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
> +			__func__, rte_lcore_id());
> +
> +	if (cfg.nb_lcores == 1) {
> +		lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +		rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
> +			NULL, lcore_id);
> +	} else if (cfg.nb_lcores > 1) {
> +		lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +		rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
> +			NULL, lcore_id);
> +
> +		lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +		rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
> +			lcore_id);
> +	}
> +}
> +
> +/* Display usage */
> +static void
> +ioat_usage(const char *prgname)
> +{
> +	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
> +		"  -p --portmask: hexadecimal bitmask of ports to configure\n"
> +		"  -q NQ: number of RX queues per port (default is 1)\n"
> +		"  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
> +		"      When enabled:\n"
> +		"       - The source MAC address is replaced by the TX port MAC address\n"
> +		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
> +		"  -c --copy-type CT: type of copy: sw|rawdev\n"
> +		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
> +	       prgname);
> +}
> +
> +static int
> +ioat_parse_portmask(const char *portmask)
> +{
> +	char *end = NULL;
> +	unsigned long pm;
> +
> +	/* Parse hexadecimal string */
> +	pm = strtoul(portmask, &end, 16);
> +	if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
> +		return -1;
> +
> +	return pm;
> +}
> +
> +static copy_mode_t
> +ioat_parse_copy_mode(const char *copy_mode)
> +{
> +	if (strcmp(copy_mode, COPY_MODE_SW) == 0)
> +		return COPY_MODE_SW_NUM;
> +	else if (strcmp(copy_mode, COPY_MODE_IOAT) == 0)
> +		return COPY_MODE_IOAT_NUM;
> +
> +	return COPY_MODE_INVALID_NUM;
> +}
> +
> +/* Parse the argument given in the command line of the application */
> +static int
> +ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
> +{
> +	static const char short_options[] =
> +		"p:"  /* portmask */
> +		"q:"  /* number of RX queues per port */
> +		"c:"  /* copy type (sw|rawdev) */
> +		"s:"  /* ring size */
> +		;
> +
> +	static const struct option lgopts[] = {
> +		{CMD_LINE_OPT_MAC_UPDATING, no_argument, &mac_updating, 1},
> +		{CMD_LINE_OPT_NO_MAC_UPDATING, no_argument, &mac_updating, 0},
> +		{CMD_LINE_OPT_PORTMASK, required_argument, NULL, 'p'},
> +		{CMD_LINE_OPT_NB_QUEUE, required_argument, NULL, 'q'},
> +		{CMD_LINE_OPT_COPY_TYPE, required_argument, NULL, 'c'},
> +		{CMD_LINE_OPT_RING_SIZE, required_argument, NULL, 's'},
> +		{NULL, 0, 0, 0}
> +	};
> +
> +	const unsigned int default_port_mask = (1 << nb_ports) - 1;
> +	int opt, ret;
> +	char **argvopt;
> +	int option_index;
> +	char *prgname = argv[0];
> +
> +	ioat_enabled_port_mask = default_port_mask;
> +	argvopt = argv;
> +
> +	while ((opt = getopt_long(argc, argvopt, short_options,
> +				  lgopts, &option_index)) != EOF) {
> +
> +		switch (opt) {
> +		/* portmask */
> +		case 'p':
> +			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
> +			if (ioat_enabled_port_mask & ~default_port_mask ||
> +					ioat_enabled_port_mask <= 0) {
> +				printf("Invalid portmask, %s, suggest 0x%x\n",
> +						optarg, default_port_mask);
> +				ioat_usage(prgname);
> +				return -1;
> +			}
> +			break;
> +
> +		case 'q':
> +			nb_queues = atoi(optarg);
> +			if (nb_queues == 0 || nb_queues > MAX_RX_QUEUES_COUNT) {
> +				printf("Invalid RX queues number %s. Max %u\n",
> +					optarg, MAX_RX_QUEUES_COUNT);
> +				ioat_usage(prgname);
> +				return -1;
> +			}
> +			break;
> +
> +		case 'c':
> +			copy_mode = ioat_parse_copy_mode(optarg);
> +			if (copy_mode == COPY_MODE_INVALID_NUM) {
> +				printf("Invalid copy type. Use: sw, rawdev\n");
> +				ioat_usage(prgname);
> +				return -1;
> +			}
> +			break;
> +
> +		case 's':
> +			ring_size = atoi(optarg);
> +			if (ring_size == 0) {
> +				printf("Invalid ring size, %s.\n", optarg);
> +				ioat_usage(prgname);
> +				return -1;
> +			}
> +			break;
> +
> +		/* long options */
> +		case 0:
> +			break;
> +
> +		default:
> +			ioat_usage(prgname);
> +			return -1;
> +		}
> +	}
> +
> +	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
> +	if (optind >= 0)
> +		argv[optind-1] = prgname;
> +
> +	ret = optind-1;
> +	optind = 1; /* reset getopt lib */
> +	return ret;
> +}
> +
> +/* check link status, return true if at least one port is up */
> +static int
> +check_link_status(uint32_t port_mask)
> +{
> +	uint16_t portid;
> +	struct rte_eth_link link;
> +	int retval = 0;
> +
> +	printf("\nChecking link status\n");
> +	RTE_ETH_FOREACH_DEV(portid) {
> +		if ((port_mask & (1 << portid)) == 0)
> +			continue;
> +
> +		memset(&link, 0, sizeof(link));
> +		rte_eth_link_get(portid, &link);
> +
> +		/* Print link status */
> +		if (link.link_status) {
> +			printf(
> +				"Port %d Link Up. Speed %u Mbps - %s\n",
> +				portid, link.link_speed,
> +				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
> +				("full-duplex") : ("half-duplex\n"));
> +			retval = 1;
> +		} else
> +			printf("Port %d Link Down\n", portid);
> +	}
> +	return retval;
> +}
> +
> +static void
> +configure_rawdev_queue(uint32_t dev_id)
> +{
> +	struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
> +	struct rte_rawdev_info info = { .dev_private = &dev_config };
> +
> +	if (rte_rawdev_configure(dev_id, &info) != 0) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error with rte_rawdev_configure()\n");
> +	}
> +	if (rte_rawdev_start(dev_id) != 0) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error with rte_rawdev_start()\n");
> +	}
> +}
> +
> +static void
> +assign_rawdevs(void)
> +{
> +	uint16_t nb_rawdev = 0, rdev_id = 0;
> +	uint32_t i, j;
> +
> +	for (i = 0; i < cfg.nb_ports; i++) {
> +		for (j = 0; j < cfg.ports[i].nb_queues; j++) {
> +			struct rte_rawdev_info rdev_info = { 0 };
> +
> +			do {
> +				if (rdev_id == rte_rawdev_count())
> +					goto end;
> +				rte_rawdev_info_get(rdev_id++, &rdev_info);
> +			} while (strcmp(rdev_info.driver_name,
> +				IOAT_PMD_RAWDEV_NAME_STR) != 0);
> +
> +			cfg.ports[i].ioat_ids[j] = rdev_id - 1;
> +			configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
> +			++nb_rawdev;
> +		}
> +	}
> +end:
> +	if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
> +		rte_exit(EXIT_FAILURE,
> +			"Not enough IOAT rawdevs (%u) for all queues (%u).\n",
> +			nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
> +	RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
> +}
> +
> +static void
> +assign_rings(void)
> +{
> +	uint32_t i;
> +
> +	for (i = 0; i < cfg.nb_ports; i++) {
> +		char ring_name[RTE_RING_NAMESIZE];
> +
> +		snprintf(ring_name, sizeof(ring_name), "rx_to_tx_ring_%u", i);
> +		/* Create ring for inter core communication */
> +		cfg.ports[i].rx_to_tx_ring = rte_ring_create(
> +				ring_name, ring_size,
> +				rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
> +
> +		if (cfg.ports[i].rx_to_tx_ring == NULL)
> +			rte_exit(EXIT_FAILURE, "Ring create failed: %s\n",
> +					rte_strerror(rte_errno));
> +	}
> +}
> +
> +/*
> + * Initializes a given port using global settings and with the RX buffers
> + * coming from the mbuf_pool passed as a parameter.
> + */
> +static inline void
> +port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
> +{
> +	/* configuring port to use RSS for multiple RX queues */
> +	static const struct rte_eth_conf port_conf = {
> +		.rxmode = {
> +			.mq_mode        = ETH_MQ_RX_RSS,
> +			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
> +		},
> +		.rx_adv_conf = {
> +			.rss_conf = {
> +				.rss_key = NULL,
> +				.rss_hf = ETH_RSS_PROTO_MASK,
> +			}
> +		}
> +	};
> +
> +	struct rte_eth_rxconf rxq_conf;
> +	struct rte_eth_txconf txq_conf;
> +	struct rte_eth_conf local_port_conf = port_conf;
> +	struct rte_eth_dev_info dev_info;
> +	int ret, i;
> +
> +	/* Skip ports that are not enabled */
> +	if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
> +		printf("Skipping disabled port %u\n", portid);
> +		return;
> +	}
> +
> +	/* Init port */
> +	printf("Initializing port %u... ", portid);
> +	fflush(stdout);
> +	rte_eth_dev_info_get(portid, &dev_info);
> +	local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
> +		dev_info.flow_type_rss_offloads;
> +	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
> +		local_port_conf.txmode.offloads |=
> +			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
> +	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
> +	if (ret < 0)
> +		rte_exit(EXIT_FAILURE, "Cannot configure device:"
> +			" err=%d, port=%u\n", ret, portid);
> +
> +	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
> +						&nb_txd);
> +	if (ret < 0)
> +		rte_exit(EXIT_FAILURE,
> +			"Cannot adjust number of descriptors: err=%d, port=%u\n",
> +			ret, portid);
> +
> +	rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
> +
> +	/* Init RX queues */
> +	rxq_conf = dev_info.default_rxconf;
> +	rxq_conf.offloads = local_port_conf.rxmode.offloads;
> +	for (i = 0; i < nb_queues; i++) {
> +		ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
> +			rte_eth_dev_socket_id(portid), &rxq_conf,
> +			mbuf_pool);
> +		if (ret < 0)
> +			rte_exit(EXIT_FAILURE,
> +				"rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
> +				ret, portid, i);
> +	}
> +
> +	/* Init one TX queue on each port */
> +	txq_conf = dev_info.default_txconf;
> +	txq_conf.offloads = local_port_conf.txmode.offloads;
> +	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
> +			rte_eth_dev_socket_id(portid),
> +			&txq_conf);
> +	if (ret < 0)
> +		rte_exit(EXIT_FAILURE,
> +			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
> +			ret, portid);
> +
> +	/* Initialize TX buffers */
> +	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
> +			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
> +			rte_eth_dev_socket_id(portid));
> +	if (tx_buffer[portid] == NULL)
> +		rte_exit(EXIT_FAILURE,
> +			"Cannot allocate buffer for tx on port %u\n",
> +			portid);
> +
> +	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
> +
> +	ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
> +			rte_eth_tx_buffer_count_callback,
> +			&port_statistics.tx_dropped[portid]);
> +	if (ret < 0)
> +		rte_exit(EXIT_FAILURE,
> +			"Cannot set error callback for tx buffer on port %u\n",
> +			portid);
> +
> +	/* Start device */
> +	ret = rte_eth_dev_start(portid);
> +	if (ret < 0)
> +		rte_exit(EXIT_FAILURE,
> +			"rte_eth_dev_start:err=%d, port=%u\n",
> +			ret, portid);
> +
> +	rte_eth_promiscuous_enable(portid);
> +
> +	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
> +			portid,
> +			ioat_ports_eth_addr[portid].addr_bytes[0],
> +			ioat_ports_eth_addr[portid].addr_bytes[1],
> +			ioat_ports_eth_addr[portid].addr_bytes[2],
> +			ioat_ports_eth_addr[portid].addr_bytes[3],
> +			ioat_ports_eth_addr[portid].addr_bytes[4],
> +			ioat_ports_eth_addr[portid].addr_bytes[5]);
> +
> +	cfg.ports[cfg.nb_ports].rxtx_port = portid;
> +	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
> +}
> +
> +static void
> +signal_handler(int signum)
> +{
> +	if (signum == SIGINT || signum == SIGTERM) {
> +		printf("\n\nSignal %d received, preparing to exit...\n",
> +			signum);
> +		force_quit = true;
> +	}
> +}
> +
> +int
> +main(int argc, char **argv)
> +{
> +	int ret;
> +	uint16_t nb_ports, portid;
> +	uint32_t i;
> +	unsigned int nb_mbufs;
> +
> +	/* Init EAL */
> +	ret = rte_eal_init(argc, argv);
> +	if (ret < 0)
> +		rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
> +	argc -= ret;
> +	argv += ret;
> +
> +	force_quit = false;
> +	signal(SIGINT, signal_handler);
> +	signal(SIGTERM, signal_handler);
> +
> +	nb_ports = rte_eth_dev_count_avail();
> +	if (nb_ports == 0)
> +		rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
> +
> +	/* Parse application arguments (after the EAL ones) */
> +	ret = ioat_parse_args(argc, argv, nb_ports);
> +	if (ret < 0)
> +		rte_exit(EXIT_FAILURE, "Invalid IOAT arguments\n");
> +
> +	nb_mbufs = RTE_MAX(nb_ports * (nb_queues * (nb_rxd + nb_txd +
> +		4 * MAX_PKT_BURST) + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
> +		MIN_POOL_SIZE);
> +
> +	/* Create the mbuf pool */
> +	ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
> +		MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
> +		rte_socket_id());
> +	if (ioat_pktmbuf_pool == NULL)
> +		rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
> +
> +	/* Initialise each port */
> +	cfg.nb_ports = 0;
> +	RTE_ETH_FOREACH_DEV(portid)
> +		port_init(portid, ioat_pktmbuf_pool, nb_queues);
> +
> +	/* Initialize port xstats */
> +	memset(&port_statistics, 0, sizeof(port_statistics));
> +
> +	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
> +		sleep(1);
> +
> +	/* Check if there is enough lcores for all ports. */
> +	cfg.nb_lcores = rte_lcore_count() - 1;
> +	if (cfg.nb_lcores < 1)
> +		rte_exit(EXIT_FAILURE,
> +			"There should be at least one slave lcore.\n");
> +
> +	if (copy_mode == COPY_MODE_IOAT_NUM)
> +		assign_rawdevs();
> +	else /* copy_mode == COPY_MODE_SW_NUM */
> +		assign_rings();
> +
> +	start_forwarding_cores();
> +	/* master core prints stats while other cores forward */
> +	print_stats(argv[0]);
> +
> +	/* force_quit is true when we get here */
> +	rte_eal_mp_wait_lcore();
> +
> +	uint32_t j;
> +
> +	for (i = 0; i < cfg.nb_ports; i++) {
> +		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
> +		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
> +		rte_eth_dev_close(cfg.ports[i].rxtx_port);
> +		if (copy_mode == COPY_MODE_IOAT_NUM) {
> +			for (j = 0; j < cfg.ports[i].nb_queues; j++) {
> +				printf("Stopping rawdev %d\n",
> +					cfg.ports[i].ioat_ids[j]);
> +				rte_rawdev_stop(cfg.ports[i].ioat_ids[j]);
> +			}
> +		} else /* copy_mode == COPY_MODE_SW_NUM */
> +			rte_ring_free(cfg.ports[i].rx_to_tx_ring);
> +	}
> +
> +	printf("Bye...\n");
> +	return 0;
> +}
> diff --git a/examples/ioat/meson.build b/examples/ioat/meson.build
> new file mode 100644
> index 000000000..ff56dc99c
> --- /dev/null
> +++ b/examples/ioat/meson.build
> @@ -0,0 +1,13 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2019 Intel Corporation
> +
> +# meson file, for building this example as part of a main DPDK build.
> +#
> +# To build this example as a standalone application with an already-installed
> +# DPDK instance, use 'make'
> +
> +deps += ['pmd_ioat']
> +
> +sources = files(
> +	'ioatfwd.c'
> +)
> diff --git a/examples/meson.build b/examples/meson.build
> index a046b74ad..c2e18b59b 100644
> --- a/examples/meson.build
> +++ b/examples/meson.build

You'll need to add the pmd_ioat dependency to this example it seems:

   examples/meson.build:89:4: ERROR: Problem encountered: Missing dependency "pmd_ioat" for example "ioat"

> @@ -16,6 +16,7 @@ all_examples = [
>  	'eventdev_pipeline', 'exception_path',
>  	'fips_validation', 'flow_classify',
>  	'flow_filtering', 'helloworld',
> +	'ioat',
>  	'ip_fragmentation', 'ip_pipeline',
>  	'ip_reassembly', 'ipsec-secgw',
>  	'ipv4_multicast', 'kni',

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH] examples/ioat: create sample app on ioat driver usage
  2019-09-09 13:12 ` Aaron Conole
@ 2019-09-09 13:58   ` Bruce Richardson
  0 siblings, 0 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-09-09 13:58 UTC (permalink / raw)
  To: Aaron Conole; +Cc: Marcin Baran, dev, Pawel Modrak

On Mon, Sep 09, 2019 at 09:12:32AM -0400, Aaron Conole wrote:
> Marcin Baran <marcinx.baran@intel.com> writes:
> 
> > From: Pawel Modrak <pawelx.modrak@intel.com>
> >
> > A new sample app demonstrating use of driver for CBDMA.
> > The app receives packets, performs software or hardware
> > copy, changes packets' MAC addresses (if enabled) and
> > forwards them. The patch includes sample application
> > as well as it's guide.
> >
> > Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> > Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> > ---
<snip>
> > diff --git a/examples/ioat/meson.build b/examples/ioat/meson.build
> > new file mode 100644
> > index 000000000..ff56dc99c
> > --- /dev/null
> > +++ b/examples/ioat/meson.build
> > @@ -0,0 +1,13 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2019 Intel Corporation
> > +
> > +# meson file, for building this example as part of a main DPDK build.
> > +#
> > +# To build this example as a standalone application with an already-installed
> > +# DPDK instance, use 'make'
> > +
> > +deps += ['pmd_ioat']
> > +
> > +sources = files(
> > +	'ioatfwd.c'
> > +)
> > diff --git a/examples/meson.build b/examples/meson.build
> > index a046b74ad..c2e18b59b 100644
> > --- a/examples/meson.build
> > +++ b/examples/meson.build
> 
> You'll need to add the pmd_ioat dependency to this example it seems:
> 
>    examples/meson.build:89:4: ERROR: Problem encountered: Missing dependency "pmd_ioat" for example "ioat"
> 
Actually, its added. What I think is missing is a check to disable the app
if the dependency is not available.

/Bruce

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH] examples/ioat: create sample app on ioat driver usage
  2019-09-09  8:29 [dpdk-dev] [PATCH] examples/ioat: create sample app on ioat driver usage Marcin Baran
  2019-09-09 13:12 ` Aaron Conole
@ 2019-09-12  9:52 ` Bruce Richardson
  2019-09-12 12:18   ` Baran, MarcinX
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
  2 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-12  9:52 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, Pawel Modrak

On Mon, Sep 09, 2019 at 10:29:38AM +0200, Marcin Baran wrote:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> 
> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware
> copy, changes packets' MAC addresses (if enabled) and
> forwards them. The patch includes sample application
> as well as it's guide.
> 
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---

Thanks, Pawel and Marcin. Some comments on doc and code inline below.

>  doc/guides/sample_app_ug/index.rst |    1 +
>  doc/guides/sample_app_ug/intro.rst |    4 +
>  doc/guides/sample_app_ug/ioat.rst  |  691 +++++++++++++++++++
>  examples/Makefile                  |    3 +
>  examples/ioat/Makefile             |   54 ++
>  examples/ioat/ioatfwd.c            | 1010 ++++++++++++++++++++++++++++
>  examples/ioat/meson.build          |   13 +
>  examples/meson.build               |    1 +
>  8 files changed, 1777 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
>  create mode 100644 examples/ioat/Makefile
>  create mode 100644 examples/ioat/ioatfwd.c
>  create mode 100644 examples/ioat/meson.build
> 
> diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
> index f23f8f59e..a6a1d9e7a 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -23,6 +23,7 @@ Sample Applications User Guides
>      ip_reassembly
>      kernel_nic_interface
>      keep_alive
> +    ioat
>      l2_forward_crypto
>      l2_forward_job_stats
>      l2_forward_real_virtual
> diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
> index 90704194a..74462312f 100644
> --- a/doc/guides/sample_app_ug/intro.rst
> +++ b/doc/guides/sample_app_ug/intro.rst
> @@ -91,6 +91,10 @@ examples are highlighted below.
>    forwarding, or ``l3fwd`` application does forwarding based on Internet
>    Protocol, IPv4 or IPv6 like a simple router.
>  
> +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
> +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
> +  copying packets between two threads.
> +
>  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
>    demonstrates how to distribute packets arriving on an Rx port to different
>    cores for processing and transmission.
> diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
> new file mode 100644
> index 000000000..378d70b81
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/ioat.rst
> @@ -0,0 +1,691 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Sample Application of packet copying using Intel\|reg| QuickData Technology
> +============================================================================

You need a space before the |reg| bit otherwise the reg doesn't get the
symbol replaced. It should be "Intel\ |reg|".

> +
> +Overview
> +--------
> +
> +This sample is intended as a demonstration of the basic components of a DPDK
> +forwarding application and example of how to use IOAT driver API to make
> +packets copies.
> +
> +Also while forwarding, the MAC addresses are affected as follows:
> +
> +*   The source MAC address is replaced by the TX port MAC address
> +
> +*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
> +
> +This application can be used to compare performance of using software packet
> +copy with copy done using a DMA device for different sizes of packets.
> +The example will print out statistics each second. The stats shows
> +received/send packets and packets dropped or failed to copy.
> +
> +Compiling the Application
> +-------------------------
> +
> +To compile the sample application see :doc:`compiling`.
> +
> +The application is located in the ``ioat`` sub-directory.
> +
> +
> +Running the Application
> +-----------------------
> +
> +In order to run the hardware copy application, the copying device
> +needs to be bound to user-space IO driver.
> +
> +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology*
> +guide for information on using the driver.
> +
> +The application requires a number of command line options:
> +
> +.. code-block:: console
> +
> +    ./build/ioatfwd [EAL options] -- -p MASK [-C CT] [--[no-]mac-updating]

I think the app uses lower case "c" rather than upper case, as called out
below. Since the "CT" value can only be one of two possibilities, I think
you should explicitly include them, e.g. "[-c <sw|rawdev>]". "rawdev" is
also a rather long name for this parameter, why not just call them sw and
hw?

> +
> +where,
> +
> +*   p MASK: A hexadecimal bitmask of the ports to configure
> +
> +*   c CT: Performed packet copy type: software (sw) or hardware using
> +    DMA (rawdev)
> +
> +*   s RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for
> +    software copy mode
> +

This parameter is missing from the summary above.

> +*   --[no-]mac-updating: Whether MAC address of packets should be changed
> +    or not
> +
> +The application can be launched in 2 different configurations:
> +
> +*   Performing software packet copying
> +
> +*   Performing hardware packet copying

Two thoughts here:
a) is this not obvious from the parameter list
b) is not more that two configurations, given that you can have:
  * sw copy with mac updating
  * sw copy without mac updating
  * etc.
not including the possibly port-mask, ring size and single-core vs two core
configurations.

> +
> +Each port needs 2 lcores: one of them receives incoming traffic and makes
> +a copy of each packet. The second lcore then updates MAC address and sends
> +the copy. For each configuration an additional lcore is needed since
> +master lcore in use which is responsible for configuration, statistics
> +printing and safe deinitialization of all ports and devices.
> +

I believe the app also supports running with 1 or 2 cores total, right?

> +The application can use a maximum of 8 ports.

Why this limitation?

> +
> +To run the application in a Linux environment with 3 lcores (one of them
> +is master lcore), 1 port (port 0), software copying and MAC updating issue
> +the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
> +
> +To run the application in a Linux environment with 5 lcores (one of them
> +is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC
> +updating issue the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-4 -n 1 -- -p 0x3 --no-mac-updating -c rawdev
> +
> +Refer to the *DPDK Getting Started Guide* for general information on
> +running applications and the Environment Abstraction Layer (EAL) options.
> +

<snip>

> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
> new file mode 100644
> index 000000000..8463d82f3
> --- /dev/null
> +++ b/examples/ioat/ioatfwd.c
> @@ -0,0 +1,1010 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2019 Intel Corporation
> + */
> +
> +#include <stdint.h>
> +#include <getopt.h>
> +#include <signal.h>
> +#include <stdbool.h>
> +#include <unistd.h>
> +
> +#include <rte_malloc.h>
> +#include <rte_ethdev.h>
> +#include <rte_rawdev.h>
> +#include <rte_ioat_rawdev.h>
> +
> +/* size of ring used for software copying between rx and tx. */
> +#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
> +#define MAX_PKT_BURST 32
Seems a low max, assume this is actually the default burst size?

> +#define MEMPOOL_CACHE_SIZE 512
> +#define MIN_POOL_SIZE 65536U
> +#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
> +#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
> +#define CMD_LINE_OPT_PORTMASK "portmask"
> +#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
> +#define CMD_LINE_OPT_COPY_TYPE "copy-type"
> +#define CMD_LINE_OPT_RING_SIZE "ring-size"
> +
> +/* configurable number of RX/TX ring descriptors */
> +#define RX_DEFAULT_RINGSIZE 1024
> +#define TX_DEFAULT_RINGSIZE 1024
> +
> +/* max number of RX queues per port */
> +#define MAX_RX_QUEUES_COUNT 8
> +
> +struct rxtx_port_config {
> +	/* common config */
> +	uint16_t rxtx_port;
> +	uint16_t nb_queues;
> +	/* for software copy mode */
> +	struct rte_ring *rx_to_tx_ring;
> +	/* for IOAT rawdev copy mode */
> +	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT];
> +};
> +
> +struct rxtx_transmission_config {
> +	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
> +	uint16_t nb_ports;
> +	uint16_t nb_lcores;
> +};
> +
> +/* per-port statistics struct */
> +struct ioat_port_statistics {
> +	uint64_t rx[RTE_MAX_ETHPORTS];
> +	uint64_t tx[RTE_MAX_ETHPORTS];
> +	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
> +	uint64_t copy_dropped[RTE_MAX_ETHPORTS];
> +};
> +struct ioat_port_statistics port_statistics;
> +
> +struct total_statistics {
> +	uint64_t total_packets_dropped;
> +	uint64_t total_packets_tx;
> +	uint64_t total_packets_rx;
> +	uint64_t total_successful_enqueues;
> +	uint64_t total_failed_enqueues;
> +};
> +
> +typedef enum copy_mode_t {
> +#define COPY_MODE_SW "sw"
> +	COPY_MODE_SW_NUM,
> +#define COPY_MODE_IOAT "rawdev"
> +	COPY_MODE_IOAT_NUM,
> +	COPY_MODE_INVALID_NUM,
> +	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
> +} copy_mode_t;
> +
> +/* mask of enabled ports */
> +static uint32_t ioat_enabled_port_mask;
> +
> +/* number of RX queues per port */
> +static uint16_t nb_queues = 1;
> +
> +/* MAC updating enabled by default. */
> +static int mac_updating = 1;
> +
> +/* hardare copy mode enabled by default. */
> +static copy_mode_t copy_mode = COPY_MODE_IOAT_NUM;
> +
> +/* size of IOAT rawdev ring for hardware copy mode or
> + * rte_ring for software copy mode
> + */
> +static unsigned short ring_size = 2048;
> +
> +/* global transmission config */
> +struct rxtx_transmission_config cfg;
> +
> +/* configurable number of RX/TX ring descriptors */
> +static uint16_t nb_rxd = RX_DEFAULT_RINGSIZE;
> +static uint16_t nb_txd = TX_DEFAULT_RINGSIZE;
> +
> +static volatile bool force_quit;
> +
> +/* ethernet addresses of ports */
> +static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
> +
> +static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
> +struct rte_mempool *ioat_pktmbuf_pool;
> +
> +/* Print out statistics for one port. */
> +static void
> +print_port_stats(uint16_t port_id)
> +{
> +	printf("\nStatistics for port %u ------------------------------"
> +		"\nPackets sent: %34"PRIu64
> +		"\nPackets received: %30"PRIu64
> +		"\nPackets dropped on tx: %25"PRIu64
> +		"\nPackets dropped on copy: %23"PRIu64,
> +		port_id,
> +		port_statistics.tx[port_id],
> +		port_statistics.rx[port_id],
> +		port_statistics.tx_dropped[port_id],
> +		port_statistics.copy_dropped[port_id]);
> +}
> +
> +/* Print out statistics for one IOAT rawdev device. */
> +static void
> +print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
> +	uint16_t nb_xstats, struct rte_rawdev_xstats_name *names_xstats)
> +{
> +	uint16_t i;
> +
> +	printf("\nIOAT channel %u", dev_id);
> +	for (i = 0; i < nb_xstats; i++)
> +		if (strstr(names_xstats[i].name, "enqueues"))
> +			printf("\n\t %s: %*"PRIu64,
> +				names_xstats[i].name,
> +				(int)(37 - strlen(names_xstats[i].name)),
> +				xstats[i]);
> +}
> +
> +static void
> +print_total_stats(struct total_statistics *ts)
> +{
> +	printf("\nAggregate statistics ==============================="
> +		"\nTotal packets sent: %28"PRIu64
> +		"\nTotal packets received: %24"PRIu64
> +		"\nTotal packets dropped: %25"PRIu64,
> +		ts->total_packets_tx,
> +		ts->total_packets_rx,
> +		ts->total_packets_dropped);
> +
> +	if (copy_mode == COPY_MODE_IOAT_NUM) {
> +		printf("\nTotal IOAT successful enqueues: %16"PRIu64
> +			"\nTotal IOAT failed enqueues: %20"PRIu64,
> +			ts->total_successful_enqueues,
> +			ts->total_failed_enqueues);
> +	}
> +
> +	printf("\n====================================================\n");
> +}
> +

For these stats, it would be nice to have deltas i.e. pps, rather than (or
as well as) the raw packet count numbers. Since your main stats loop below
has a "sleep(1)" at the start, just computing the deltas should give a good
enough PPS value.

> +/* Print out statistics on packets dropped. */
> +static void
> +print_stats(char *prgname)
> +{
> +	struct total_statistics ts;
> +	uint32_t i, port_id, dev_id;
> +	struct rte_rawdev_xstats_name *names_xstats;
> +	uint64_t *xstats;
> +	unsigned int *ids_xstats;
> +	unsigned int nb_xstats, id_fail_enq, id_succ_enq;
> +	char status_string[120]; /* to print at the top of the output */
> +	int status_strlen;
> +
> +
> +	const char clr[] = { 27, '[', '2', 'J', '\0' };
> +	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
> +
> +	status_strlen = snprintf(status_string, sizeof(status_string),
> +		"%s, ", prgname);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Worker Threads = %d, ",
> +		rte_lcore_count() > 2 ? 2 : 1);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
> +		COPY_MODE_SW : COPY_MODE_IOAT);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Updating MAC = %s, ", mac_updating ?
> +		"enabled" : "disabled");
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Rx Queues = %d, ", nb_queues);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Ring Size = %d\n", ring_size);
> +
> +	/* Allocate memory for xstats names and values */
> +	nb_xstats = rte_rawdev_xstats_names_get(
> +			cfg.ports[0].ioat_ids[0], NULL, 0);
> +
> +	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
> +	if (names_xstats == NULL) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error allocating xstat names memory\n");
> +	}
> +	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
> +			names_xstats, nb_xstats);
> +
> +	ids_xstats = malloc(sizeof(*ids_xstats) * nb_xstats);
> +	if (ids_xstats == NULL) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error allocating xstat ids_xstats memory\n");
> +	}
> +
> +	for (i = 0; i < nb_xstats; i++)
> +		ids_xstats[i] = i;
> +
> +	xstats = malloc(sizeof(*xstats) * nb_xstats);
> +	if (xstats == NULL) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error allocating xstat memory\n");
> +	}
> +
> +	/* Get failed/successful enqueues stats index */
> +	id_fail_enq = id_succ_enq = nb_xstats;
> +	for (i = 0; i < nb_xstats; i++) {
> +		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
> +			id_fail_enq = i;
> +		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
> +			id_succ_enq = i;
> +		if (id_fail_enq < nb_xstats && id_succ_enq < nb_xstats)
> +			break;
> +	}
> +	if (id_fail_enq == nb_xstats || id_succ_enq == nb_xstats) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error getting failed/successful enqueues stats index\n");
> +	}
> +
> +	while (!force_quit) {
> +		/* Sleep for 1 second each round - init sleep allows reading
> +		 * messages from app startup.
> +		 */
> +		sleep(1);
> +
> +		/* Clear screen and move to top left */
> +		printf("%s%s", clr, topLeft);
> +
> +		memset(&ts, 0, sizeof(struct total_statistics));
> +
> +		printf("%s", status_string);
> +
> +		for (i = 0; i < cfg.nb_ports; i++) {
> +			port_id = cfg.ports[i].rxtx_port;
> +			print_port_stats(port_id);
> +
> +			ts.total_packets_dropped +=
> +				port_statistics.tx_dropped[port_id]
> +				+ port_statistics.copy_dropped[port_id];
> +			ts.total_packets_tx += port_statistics.tx[port_id];
> +			ts.total_packets_rx += port_statistics.rx[port_id];
> +
> +			if (copy_mode == COPY_MODE_IOAT_NUM) {
> +				uint32_t j;
> +
> +				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
> +					dev_id = cfg.ports[i].ioat_ids[j];
> +					rte_rawdev_xstats_get(dev_id,
> +						ids_xstats, xstats, nb_xstats);
> +
> +					print_rawdev_stats(dev_id, xstats,
> +						nb_xstats, names_xstats);
> +
> +					ts.total_successful_enqueues +=
> +						xstats[id_succ_enq];
> +					ts.total_failed_enqueues +=
> +						xstats[id_fail_enq];
> +				}
> +			}
> +		}
> +		printf("\n");
> +
> +		print_total_stats(&ts);
> +	}
> +
> +	free(names_xstats);
> +	free(xstats);
> +	free(ids_xstats);
> +}
<snip>

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH] examples/ioat: create sample app on ioat driver usage
  2019-09-12  9:52 ` Bruce Richardson
@ 2019-09-12 12:18   ` Baran, MarcinX
  0 siblings, 0 replies; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-12 12:18 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev, Modrak, PawelX

On Mon, Sep 09, 2019 at 10:29:38AM +0200, Marcin Baran wrote:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> 
> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware copy, changes 
> packets' MAC addresses (if enabled) and forwards them. The patch 
> includes sample application as well as it's guide.
> 
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---

Thanks, Pawel and Marcin. Some comments on doc and code inline below.

>  doc/guides/sample_app_ug/index.rst |    1 +
>  doc/guides/sample_app_ug/intro.rst |    4 +
>  doc/guides/sample_app_ug/ioat.rst  |  691 +++++++++++++++++++
>  examples/Makefile                  |    3 +
>  examples/ioat/Makefile             |   54 ++
>  examples/ioat/ioatfwd.c            | 1010 ++++++++++++++++++++++++++++
>  examples/ioat/meson.build          |   13 +
>  examples/meson.build               |    1 +
>  8 files changed, 1777 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst  create mode 
> 100644 examples/ioat/Makefile  create mode 100644 
> examples/ioat/ioatfwd.c  create mode 100644 examples/ioat/meson.build
> 
> diff --git a/doc/guides/sample_app_ug/index.rst 
> b/doc/guides/sample_app_ug/index.rst
> index f23f8f59e..a6a1d9e7a 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -23,6 +23,7 @@ Sample Applications User Guides
>      ip_reassembly
>      kernel_nic_interface
>      keep_alive
> +    ioat
>      l2_forward_crypto
>      l2_forward_job_stats
>      l2_forward_real_virtual
> diff --git a/doc/guides/sample_app_ug/intro.rst 
> b/doc/guides/sample_app_ug/intro.rst
> index 90704194a..74462312f 100644
> --- a/doc/guides/sample_app_ug/intro.rst
> +++ b/doc/guides/sample_app_ug/intro.rst
> @@ -91,6 +91,10 @@ examples are highlighted below.
>    forwarding, or ``l3fwd`` application does forwarding based on Internet
>    Protocol, IPv4 or IPv6 like a simple router.
>  
> +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
> +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev 
> +driver for
> +  copying packets between two threads.
> +
>  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
>    demonstrates how to distribute packets arriving on an Rx port to different
>    cores for processing and transmission.
> diff --git a/doc/guides/sample_app_ug/ioat.rst 
> b/doc/guides/sample_app_ug/ioat.rst
> new file mode 100644
> index 000000000..378d70b81
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/ioat.rst
> @@ -0,0 +1,691 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Sample Application of packet copying using Intel\|reg| QuickData 
> +Technology 
> +=====================================================================
> +=======

You need a space before the |reg| bit otherwise the reg doesn't get the symbol replaced. It should be "Intel\ |reg|".
[Marcin] Will fix this in v2

> +
> +Overview
> +--------
> +
> +This sample is intended as a demonstration of the basic components of 
> +a DPDK forwarding application and example of how to use IOAT driver 
> +API to make packets copies.
> +
> +Also while forwarding, the MAC addresses are affected as follows:
> +
> +*   The source MAC address is replaced by the TX port MAC address
> +
> +*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
> +
> +This application can be used to compare performance of using software 
> +packet copy with copy done using a DMA device for different sizes of packets.
> +The example will print out statistics each second. The stats shows 
> +received/send packets and packets dropped or failed to copy.
> +
> +Compiling the Application
> +-------------------------
> +
> +To compile the sample application see :doc:`compiling`.
> +
> +The application is located in the ``ioat`` sub-directory.
> +
> +
> +Running the Application
> +-----------------------
> +
> +In order to run the hardware copy application, the copying device 
> +needs to be bound to user-space IO driver.
> +
> +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData 
> +Technology* guide for information on using the driver.
> +
> +The application requires a number of command line options:
> +
> +.. code-block:: console
> +
> +    ./build/ioatfwd [EAL options] -- -p MASK [-C CT] 
> + [--[no-]mac-updating]

I think the app uses lower case "c" rather than upper case, as called out below. Since the "CT" value can only be one of two possibilities, I think you should explicitly include them, e.g. "[-c <sw|rawdev>]". "rawdev" is also a rather long name for this parameter, why not just call them sw and hw?
[Marcin] Yes, it should be lower case. It is my mistake, the doc was not updated before sending patch. Proper guide is already prepared for v2. As for the "rawdev" parameter name, I will correct it.

> +
> +where,
> +
> +*   p MASK: A hexadecimal bitmask of the ports to configure
> +
> +*   c CT: Performed packet copy type: software (sw) or hardware using
> +    DMA (rawdev)
> +
> +*   s RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for
> +    software copy mode
> +

This parameter is missing from the summary above.
[Marcin] Didn't update doc along with code. It is fixed for v2.

> +*   --[no-]mac-updating: Whether MAC address of packets should be changed
> +    or not
> +
> +The application can be launched in 2 different configurations:
> +
> +*   Performing software packet copying
> +
> +*   Performing hardware packet copying

Two thoughts here:
a) is this not obvious from the parameter list
b) is not more that two configurations, given that you can have:
  * sw copy with mac updating
  * sw copy without mac updating
  * etc.
not including the possibly port-mask, ring size and single-core vs two core configurations.
[Marcin] Good point. I will rephrase that.

> +
> +Each port needs 2 lcores: one of them receives incoming traffic and 
> +makes a copy of each packet. The second lcore then updates MAC 
> +address and sends the copy. For each configuration an additional 
> +lcore is needed since master lcore in use which is responsible for 
> +configuration, statistics printing and safe deinitialization of all ports and devices.
> +

I believe the app also supports running with 1 or 2 cores total, right?
[Marcin]Yes, that's right. Didn't update doc along with code. It is fixed for v2.

> +The application can use a maximum of 8 ports.

Why this limitation?
[Marcin] Seemed reasonable since on testing machine one IOAT device had 8 channels.

> +
> +To run the application in a Linux environment with 3 lcores (one of 
> +them is master lcore), 1 port (port 0), software copying and MAC 
> +updating issue the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
> +
> +To run the application in a Linux environment with 5 lcores (one of 
> +them is master lcore), 2 ports (ports 0 and 1), hardware copying and 
> +no MAC updating issue the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-4 -n 1 -- -p 0x3 --no-mac-updating -c 
> + rawdev
> +
> +Refer to the *DPDK Getting Started Guide* for general information on 
> +running applications and the Environment Abstraction Layer (EAL) options.
> +

<snip>

> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c new 
> file mode 100644 index 000000000..8463d82f3
> --- /dev/null
> +++ b/examples/ioat/ioatfwd.c
> @@ -0,0 +1,1010 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2019 Intel Corporation  */
> +
> +#include <stdint.h>
> +#include <getopt.h>
> +#include <signal.h>
> +#include <stdbool.h>
> +#include <unistd.h>
> +
> +#include <rte_malloc.h>
> +#include <rte_ethdev.h>
> +#include <rte_rawdev.h>
> +#include <rte_ioat_rawdev.h>
> +
> +/* size of ring used for software copying between rx and tx. */ 
> +#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1 #define MAX_PKT_BURST 32
Seems a low max, assume this is actually the default burst size?

> +#define MEMPOOL_CACHE_SIZE 512
> +#define MIN_POOL_SIZE 65536U
> +#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
> +#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
> +#define CMD_LINE_OPT_PORTMASK "portmask"
> +#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
> +#define CMD_LINE_OPT_COPY_TYPE "copy-type"
> +#define CMD_LINE_OPT_RING_SIZE "ring-size"
> +
> +/* configurable number of RX/TX ring descriptors */ #define 
> +RX_DEFAULT_RINGSIZE 1024 #define TX_DEFAULT_RINGSIZE 1024
> +
> +/* max number of RX queues per port */ #define MAX_RX_QUEUES_COUNT 8
> +
> +struct rxtx_port_config {
> +	/* common config */
> +	uint16_t rxtx_port;
> +	uint16_t nb_queues;
> +	/* for software copy mode */
> +	struct rte_ring *rx_to_tx_ring;
> +	/* for IOAT rawdev copy mode */
> +	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT]; };
> +
> +struct rxtx_transmission_config {
> +	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
> +	uint16_t nb_ports;
> +	uint16_t nb_lcores;
> +};
> +
> +/* per-port statistics struct */
> +struct ioat_port_statistics {
> +	uint64_t rx[RTE_MAX_ETHPORTS];
> +	uint64_t tx[RTE_MAX_ETHPORTS];
> +	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
> +	uint64_t copy_dropped[RTE_MAX_ETHPORTS]; }; struct 
> +ioat_port_statistics port_statistics;
> +
> +struct total_statistics {
> +	uint64_t total_packets_dropped;
> +	uint64_t total_packets_tx;
> +	uint64_t total_packets_rx;
> +	uint64_t total_successful_enqueues;
> +	uint64_t total_failed_enqueues;
> +};
> +
> +typedef enum copy_mode_t {
> +#define COPY_MODE_SW "sw"
> +	COPY_MODE_SW_NUM,
> +#define COPY_MODE_IOAT "rawdev"
> +	COPY_MODE_IOAT_NUM,
> +	COPY_MODE_INVALID_NUM,
> +	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM } copy_mode_t;
> +
> +/* mask of enabled ports */
> +static uint32_t ioat_enabled_port_mask;
> +
> +/* number of RX queues per port */
> +static uint16_t nb_queues = 1;
> +
> +/* MAC updating enabled by default. */ static int mac_updating = 1;
> +
> +/* hardare copy mode enabled by default. */ static copy_mode_t 
> +copy_mode = COPY_MODE_IOAT_NUM;
> +
> +/* size of IOAT rawdev ring for hardware copy mode or
> + * rte_ring for software copy mode
> + */
> +static unsigned short ring_size = 2048;
> +
> +/* global transmission config */
> +struct rxtx_transmission_config cfg;
> +
> +/* configurable number of RX/TX ring descriptors */ static uint16_t 
> +nb_rxd = RX_DEFAULT_RINGSIZE; static uint16_t nb_txd = 
> +TX_DEFAULT_RINGSIZE;
> +
> +static volatile bool force_quit;
> +
> +/* ethernet addresses of ports */
> +static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
> +
> +static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS]; 
> +struct rte_mempool *ioat_pktmbuf_pool;
> +
> +/* Print out statistics for one port. */ static void 
> +print_port_stats(uint16_t port_id) {
> +	printf("\nStatistics for port %u ------------------------------"
> +		"\nPackets sent: %34"PRIu64
> +		"\nPackets received: %30"PRIu64
> +		"\nPackets dropped on tx: %25"PRIu64
> +		"\nPackets dropped on copy: %23"PRIu64,
> +		port_id,
> +		port_statistics.tx[port_id],
> +		port_statistics.rx[port_id],
> +		port_statistics.tx_dropped[port_id],
> +		port_statistics.copy_dropped[port_id]);
> +}
> +
> +/* Print out statistics for one IOAT rawdev device. */ static void 
> +print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
> +	uint16_t nb_xstats, struct rte_rawdev_xstats_name *names_xstats) {
> +	uint16_t i;
> +
> +	printf("\nIOAT channel %u", dev_id);
> +	for (i = 0; i < nb_xstats; i++)
> +		if (strstr(names_xstats[i].name, "enqueues"))
> +			printf("\n\t %s: %*"PRIu64,
> +				names_xstats[i].name,
> +				(int)(37 - strlen(names_xstats[i].name)),
> +				xstats[i]);
> +}
> +
> +static void
> +print_total_stats(struct total_statistics *ts) {
> +	printf("\nAggregate statistics ==============================="
> +		"\nTotal packets sent: %28"PRIu64
> +		"\nTotal packets received: %24"PRIu64
> +		"\nTotal packets dropped: %25"PRIu64,
> +		ts->total_packets_tx,
> +		ts->total_packets_rx,
> +		ts->total_packets_dropped);
> +
> +	if (copy_mode == COPY_MODE_IOAT_NUM) {
> +		printf("\nTotal IOAT successful enqueues: %16"PRIu64
> +			"\nTotal IOAT failed enqueues: %20"PRIu64,
> +			ts->total_successful_enqueues,
> +			ts->total_failed_enqueues);
> +	}
> +
> +	printf("\n====================================================\n");
> +}
> +

For these stats, it would be nice to have deltas i.e. pps, rather than (or as well as) the raw packet count numbers. Since your main stats loop below has a "sleep(1)" at the start, just computing the deltas should give a good enough PPS value.
[Marcin] Ok, will be prepared for v2 as PPS value.

> +/* Print out statistics on packets dropped. */ static void 
> +print_stats(char *prgname) {
> +	struct total_statistics ts;
> +	uint32_t i, port_id, dev_id;
> +	struct rte_rawdev_xstats_name *names_xstats;
> +	uint64_t *xstats;
> +	unsigned int *ids_xstats;
> +	unsigned int nb_xstats, id_fail_enq, id_succ_enq;
> +	char status_string[120]; /* to print at the top of the output */
> +	int status_strlen;
> +
> +
> +	const char clr[] = { 27, '[', '2', 'J', '\0' };
> +	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
> +
> +	status_strlen = snprintf(status_string, sizeof(status_string),
> +		"%s, ", prgname);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Worker Threads = %d, ",
> +		rte_lcore_count() > 2 ? 2 : 1);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
> +		COPY_MODE_SW : COPY_MODE_IOAT);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Updating MAC = %s, ", mac_updating ?
> +		"enabled" : "disabled");
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Rx Queues = %d, ", nb_queues);
> +	status_strlen += snprintf(status_string + status_strlen,
> +		sizeof(status_string) - status_strlen,
> +		"Ring Size = %d\n", ring_size);
> +
> +	/* Allocate memory for xstats names and values */
> +	nb_xstats = rte_rawdev_xstats_names_get(
> +			cfg.ports[0].ioat_ids[0], NULL, 0);
> +
> +	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
> +	if (names_xstats == NULL) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error allocating xstat names memory\n");
> +	}
> +	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
> +			names_xstats, nb_xstats);
> +
> +	ids_xstats = malloc(sizeof(*ids_xstats) * nb_xstats);
> +	if (ids_xstats == NULL) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error allocating xstat ids_xstats memory\n");
> +	}
> +
> +	for (i = 0; i < nb_xstats; i++)
> +		ids_xstats[i] = i;
> +
> +	xstats = malloc(sizeof(*xstats) * nb_xstats);
> +	if (xstats == NULL) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error allocating xstat memory\n");
> +	}
> +
> +	/* Get failed/successful enqueues stats index */
> +	id_fail_enq = id_succ_enq = nb_xstats;
> +	for (i = 0; i < nb_xstats; i++) {
> +		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
> +			id_fail_enq = i;
> +		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
> +			id_succ_enq = i;
> +		if (id_fail_enq < nb_xstats && id_succ_enq < nb_xstats)
> +			break;
> +	}
> +	if (id_fail_enq == nb_xstats || id_succ_enq == nb_xstats) {
> +		rte_exit(EXIT_FAILURE,
> +			"Error getting failed/successful enqueues stats index\n");
> +	}
> +
> +	while (!force_quit) {
> +		/* Sleep for 1 second each round - init sleep allows reading
> +		 * messages from app startup.
> +		 */
> +		sleep(1);
> +
> +		/* Clear screen and move to top left */
> +		printf("%s%s", clr, topLeft);
> +
> +		memset(&ts, 0, sizeof(struct total_statistics));
> +
> +		printf("%s", status_string);
> +
> +		for (i = 0; i < cfg.nb_ports; i++) {
> +			port_id = cfg.ports[i].rxtx_port;
> +			print_port_stats(port_id);
> +
> +			ts.total_packets_dropped +=
> +				port_statistics.tx_dropped[port_id]
> +				+ port_statistics.copy_dropped[port_id];
> +			ts.total_packets_tx += port_statistics.tx[port_id];
> +			ts.total_packets_rx += port_statistics.rx[port_id];
> +
> +			if (copy_mode == COPY_MODE_IOAT_NUM) {
> +				uint32_t j;
> +
> +				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
> +					dev_id = cfg.ports[i].ioat_ids[j];
> +					rte_rawdev_xstats_get(dev_id,
> +						ids_xstats, xstats, nb_xstats);
> +
> +					print_rawdev_stats(dev_id, xstats,
> +						nb_xstats, names_xstats);
> +
> +					ts.total_successful_enqueues +=
> +						xstats[id_succ_enq];
> +					ts.total_failed_enqueues +=
> +						xstats[id_fail_enq];
> +				}
> +			}
> +		}
> +		printf("\n");
> +
> +		print_total_stats(&ts);
> +	}
> +
> +	free(names_xstats);
> +	free(xstats);
> +	free(ids_xstats);
> +}
<snip>

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-09  8:29 [dpdk-dev] [PATCH] examples/ioat: create sample app on ioat driver usage Marcin Baran
  2019-09-09 13:12 ` Aaron Conole
  2019-09-12  9:52 ` Bruce Richardson
@ 2019-09-13 14:39 ` Marcin Baran
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 1/6] examples/ioat: create " Marcin Baran
                     ` (7 more replies)
  2 siblings, 8 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-13 14:39 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The patch includes sample application as
well as it's guide.

v2:
 - change printing total stats to be deltas
 - update documentation
 - fix 1 thread/sw copy/multiple Rx queues packet dropping
 - divide patch into several presenting functional change

Marcin Baran (4):
  examples/ioat: add software copy support
  examples/ioat: add rawdev copy mode support
  examples/ioat: add stats printing for each port
  doc/guides/: provide IOAT sample app guide

Pawel Modrak (2):
  examples/ioat: create sample app on ioat driver usage
  examples/ioat: add two threads configuration

 doc/guides/sample_app_ug/index.rst |    1 +
 doc/guides/sample_app_ug/intro.rst |    4 +
 doc/guides/sample_app_ug/ioat.rst  |  764 +++++++++++++++++++++
 examples/Makefile                  |    3 +
 examples/ioat/Makefile             |   54 ++
 examples/ioat/ioatfwd.c            | 1025 ++++++++++++++++++++++++++++
 examples/ioat/meson.build          |   13 +
 examples/meson.build               |    1 +
 8 files changed, 1865 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v2 1/6] examples/ioat: create sample app on ioat driver usage
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
@ 2019-09-13 14:39   ` Marcin Baran
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 2/6] examples/ioat: add software copy support Marcin Baran
                     ` (6 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-13 14:39 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The change covers ports initialization,
closing connection and argument parsing.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/Makefile         |   3 +
 examples/ioat/Makefile    |  54 +++++
 examples/ioat/ioatfwd.c   | 439 ++++++++++++++++++++++++++++++++++++++
 examples/ioat/meson.build |  13 ++
 examples/meson.build      |   1 +
 5 files changed, 510 insertions(+)
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

diff --git a/examples/Makefile b/examples/Makefile
index de11dd4875..3cb313d7d8 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation
 DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify
 DIRS-y += flow_filtering
 DIRS-y += helloworld
+ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y)
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat
+endif
 DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline
 ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
 DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
diff --git a/examples/ioat/Makefile b/examples/ioat/Makefile
new file mode 100644
index 0000000000..2a4d1da2db
--- /dev/null
+++ b/examples/ioat/Makefile
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# binary name
+APP = ioatfwd
+
+# all source are stored in SRCS-y
+SRCS-y := ioatfwd.c
+
+# Build using pkg-config variables if possible
+ifeq ($(shell pkg-config --exists libdpdk && echo 0),0)
+
+all: shared
+.PHONY: shared static
+shared: build/$(APP)-shared
+	ln -sf $(APP)-shared build/$(APP)
+static: build/$(APP)-static
+	ln -sf $(APP)-static build/$(APP)
+
+PC_FILE := $(shell pkg-config --path libdpdk)
+CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
+LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
+
+build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
+
+build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
+
+build:
+	@mkdir -p $@
+
+.PHONY: clean
+clean:
+	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
+	test -d build && rmdir -p build || true
+
+else # Build using legacy build system
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, detect a build directory, by looking for a path with a .config
+RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config)))))
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+endif
diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
new file mode 100644
index 0000000000..977ea6a61c
--- /dev/null
+++ b/examples/ioat/ioatfwd.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#include <stdint.h>
+#include <getopt.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+#include <rte_ethdev.h>
+#include <rte_rawdev.h>
+#include <rte_ioat_rawdev.h>
+
+/* size of ring used for software copying between rx and tx. */
+#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
+#define MAX_PKT_BURST 32
+#define MEMPOOL_CACHE_SIZE 512
+#define MIN_POOL_SIZE 65536U
+#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
+#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
+#define CMD_LINE_OPT_PORTMASK "portmask"
+#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
+#define CMD_LINE_OPT_COPY_TYPE "copy-type"
+#define CMD_LINE_OPT_RING_SIZE "ring-size"
+
+/* configurable number of RX/TX ring descriptors */
+#define RX_DEFAULT_RINGSIZE 1024
+#define TX_DEFAULT_RINGSIZE 1024
+
+/* max number of RX queues per port */
+#define MAX_RX_QUEUES_COUNT 8
+
+struct rxtx_port_config {
+	/* common config */
+	uint16_t rxtx_port;
+	uint16_t nb_queues;
+	/* for software copy mode */
+	struct rte_ring *rx_to_tx_ring;
+	/* for IOAT rawdev copy mode */
+	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT];
+};
+
+struct rxtx_transmission_config {
+	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+	uint16_t nb_ports;
+	uint16_t nb_lcores;
+};
+
+typedef enum copy_mode_t {
+#define COPY_MODE_SW "sw"
+	COPY_MODE_SW_NUM,
+#define COPY_MODE_IOAT "rawdev"
+	COPY_MODE_IOAT_NUM,
+	COPY_MODE_INVALID_NUM,
+	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
+} copy_mode_t;
+
+/* mask of enabled ports */
+static uint32_t ioat_enabled_port_mask;
+
+/* number of RX queues per port */
+static uint16_t nb_queues = 1;
+
+/* MAC updating enabled by default. */
+static int mac_updating = 1;
+
+/* hardare copy mode enabled by default. */
+static copy_mode_t copy_mode = COPY_MODE_IOAT_NUM;
+
+/* size of IOAT rawdev ring for hardware copy mode or
+ * rte_ring for software copy mode
+ */
+static unsigned short ring_size = 2048;
+
+/* global transmission config */
+struct rxtx_transmission_config cfg;
+
+/* configurable number of RX/TX ring descriptors */
+static uint16_t nb_rxd = RX_DEFAULT_RINGSIZE;
+static uint16_t nb_txd = TX_DEFAULT_RINGSIZE;
+
+static volatile bool force_quit;
+
+/* ethernet addresses of ports */
+static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
+struct rte_mempool *ioat_pktmbuf_pool;
+
+/* Display usage */
+static void
+ioat_usage(const char *prgname)
+{
+	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
+		"  -p --portmask: hexadecimal bitmask of ports to configure\n"
+		"  -q NQ: number of RX queues per port (default is 1)\n"
+		"  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
+		"      When enabled:\n"
+		"       - The source MAC address is replaced by the TX port MAC address\n"
+		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
+		"  -c --copy-type CT: type of copy: sw|rawdev\n"
+		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
+	       prgname);
+}
+
+static int
+ioat_parse_portmask(const char *portmask)
+{
+	char *end = NULL;
+	unsigned long pm;
+
+	/* Parse hexadecimal string */
+	pm = strtoul(portmask, &end, 16);
+	if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+
+	return pm;
+}
+
+static copy_mode_t
+ioat_parse_copy_mode(const char *copy_mode)
+{
+	if (strcmp(copy_mode, COPY_MODE_SW) == 0)
+		return COPY_MODE_SW_NUM;
+	else if (strcmp(copy_mode, COPY_MODE_IOAT) == 0)
+		return COPY_MODE_IOAT_NUM;
+
+	return COPY_MODE_INVALID_NUM;
+}
+
+/* Parse the argument given in the command line of the application */
+static int
+ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
+{
+	static const char short_options[] =
+		"p:"  /* portmask */
+		"q:"  /* number of RX queues per port */
+		"c:"  /* copy type (sw|rawdev) */
+		"s:"  /* ring size */
+		;
+
+	static const struct option lgopts[] = {
+		{CMD_LINE_OPT_MAC_UPDATING, no_argument, &mac_updating, 1},
+		{CMD_LINE_OPT_NO_MAC_UPDATING, no_argument, &mac_updating, 0},
+		{CMD_LINE_OPT_PORTMASK, required_argument, NULL, 'p'},
+		{CMD_LINE_OPT_NB_QUEUE, required_argument, NULL, 'q'},
+		{CMD_LINE_OPT_COPY_TYPE, required_argument, NULL, 'c'},
+		{CMD_LINE_OPT_RING_SIZE, required_argument, NULL, 's'},
+		{NULL, 0, 0, 0}
+	};
+
+	const unsigned int default_port_mask = (1 << nb_ports) - 1;
+	int opt, ret;
+	char **argvopt;
+	int option_index;
+	char *prgname = argv[0];
+
+	ioat_enabled_port_mask = default_port_mask;
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, short_options,
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		/* portmask */
+		case 'p':
+			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
+			if (ioat_enabled_port_mask & ~default_port_mask ||
+					ioat_enabled_port_mask <= 0) {
+				printf("Invalid portmask, %s, suggest 0x%x\n",
+						optarg, default_port_mask);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'q':
+			nb_queues = atoi(optarg);
+			if (nb_queues == 0 || nb_queues > MAX_RX_QUEUES_COUNT) {
+				printf("Invalid RX queues number %s. Max %u\n",
+					optarg, MAX_RX_QUEUES_COUNT);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'c':
+			copy_mode = ioat_parse_copy_mode(optarg);
+			if (copy_mode == COPY_MODE_INVALID_NUM) {
+				printf("Invalid copy type. Use: sw, rawdev\n");
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 's':
+			ring_size = atoi(optarg);
+			if (ring_size == 0) {
+				printf("Invalid ring size, %s.\n", optarg);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		/* long options */
+		case 0:
+			break;
+
+		default:
+			ioat_usage(prgname);
+			return -1;
+		}
+	}
+
+	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
+	if (optind >= 0)
+		argv[optind-1] = prgname;
+
+	ret = optind-1;
+	optind = 1; /* reset getopt lib */
+	return ret;
+}
+
+/* check link status, return true if at least one port is up */
+static int
+check_link_status(uint32_t port_mask)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	int retval = 0;
+
+	printf("\nChecking link status\n");
+	RTE_ETH_FOREACH_DEV(portid) {
+		if ((port_mask & (1 << portid)) == 0)
+			continue;
+
+		memset(&link, 0, sizeof(link));
+		rte_eth_link_get(portid, &link);
+
+		/* Print link status */
+		if (link.link_status) {
+			printf(
+				"Port %d Link Up. Speed %u Mbps - %s\n",
+				portid, link.link_speed,
+				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+				("full-duplex") : ("half-duplex\n"));
+			retval = 1;
+		} else
+			printf("Port %d Link Down\n", portid);
+	}
+	return retval;
+}
+
+/*
+ * Initializes a given port using global settings and with the RX buffers
+ * coming from the mbuf_pool passed as a parameter.
+ */
+static inline void
+port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+{
+	/* configuring port to use RSS for multiple RX queues */
+	static const struct rte_eth_conf port_conf = {
+		.rxmode = {
+			.mq_mode        = ETH_MQ_RX_RSS,
+			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
+		},
+		.rx_adv_conf = {
+			.rss_conf = {
+				.rss_key = NULL,
+				.rss_hf = ETH_RSS_PROTO_MASK,
+			}
+		}
+	};
+
+	struct rte_eth_rxconf rxq_conf;
+	struct rte_eth_txconf txq_conf;
+	struct rte_eth_conf local_port_conf = port_conf;
+	struct rte_eth_dev_info dev_info;
+	int ret, i;
+
+	/* Skip ports that are not enabled */
+	if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+		printf("Skipping disabled port %u\n", portid);
+		return;
+	}
+
+	/* Init port */
+	printf("Initializing port %u... ", portid);
+	fflush(stdout);
+	rte_eth_dev_info_get(portid, &dev_info);
+	local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+		dev_info.flow_type_rss_offloads;
+	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+		local_port_conf.txmode.offloads |=
+			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Cannot configure device:"
+			" err=%d, port=%u\n", ret, portid);
+
+	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+						&nb_txd);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot adjust number of descriptors: err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+	/* Init RX queues */
+	rxq_conf = dev_info.default_rxconf;
+	rxq_conf.offloads = local_port_conf.rxmode.offloads;
+	for (i = 0; i < nb_queues; i++) {
+		ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+			rte_eth_dev_socket_id(portid), &rxq_conf,
+			mbuf_pool);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE,
+				"rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+				ret, portid, i);
+	}
+
+	/* Init one TX queue on each port */
+	txq_conf = dev_info.default_txconf;
+	txq_conf.offloads = local_port_conf.txmode.offloads;
+	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+			rte_eth_dev_socket_id(portid),
+			&txq_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
+			ret, portid);
+
+	/* Initialize TX buffers */
+	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+			rte_eth_dev_socket_id(portid));
+	if (tx_buffer[portid] == NULL)
+		rte_exit(EXIT_FAILURE,
+			"Cannot allocate buffer for tx on port %u\n",
+			portid);
+
+	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+	/* Start device */
+	ret = rte_eth_dev_start(portid);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_dev_start:err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_promiscuous_enable(portid);
+
+	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+			portid,
+			ioat_ports_eth_addr[portid].addr_bytes[0],
+			ioat_ports_eth_addr[portid].addr_bytes[1],
+			ioat_ports_eth_addr[portid].addr_bytes[2],
+			ioat_ports_eth_addr[portid].addr_bytes[3],
+			ioat_ports_eth_addr[portid].addr_bytes[4],
+			ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+	cfg.ports[cfg.nb_ports].rxtx_port = portid;
+	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\n\nSignal %d received, preparing to exit...\n",
+			signum);
+		force_quit = true;
+	}
+}
+
+int
+main(int argc, char **argv)
+{
+	int ret;
+	uint16_t nb_ports, portid;
+	uint32_t i;
+	unsigned int nb_mbufs;
+
+	/* Init EAL */
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+	argc -= ret;
+	argv += ret;
+
+	force_quit = false;
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	nb_ports = rte_eth_dev_count_avail();
+	if (nb_ports == 0)
+		rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
+
+	/* Parse application arguments (after the EAL ones) */
+	ret = ioat_parse_args(argc, argv, nb_ports);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid IOAT arguments\n");
+
+	nb_mbufs = RTE_MAX(nb_ports * (nb_queues * (nb_rxd + nb_txd +
+		4 * MAX_PKT_BURST) + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+		MIN_POOL_SIZE);
+
+	/* Create the mbuf pool */
+	ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+		MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+		rte_socket_id());
+	if (ioat_pktmbuf_pool == NULL)
+		rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+	/* Initialise each port */
+	cfg.nb_ports = 0;
+	RTE_ETH_FOREACH_DEV(portid)
+		port_init(portid, ioat_pktmbuf_pool, nb_queues);
+
+	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
+		sleep(1);
+
+	/* Check if there is enough lcores for all ports. */
+	cfg.nb_lcores = rte_lcore_count() - 1;
+	if (cfg.nb_lcores < 1)
+		rte_exit(EXIT_FAILURE,
+			"There should be at least one slave lcore.\n");
+	for (i = 0; i < cfg.nb_ports; i++) {
+		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
+		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
+		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+	}
+
+	printf("Bye...\n");
+	return 0;
+}
diff --git a/examples/ioat/meson.build b/examples/ioat/meson.build
new file mode 100644
index 0000000000..ff56dc99c4
--- /dev/null
+++ b/examples/ioat/meson.build
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+deps += ['pmd_ioat']
+
+sources = files(
+	'ioatfwd.c'
+)
diff --git a/examples/meson.build b/examples/meson.build
index a046b74ade..c2e18b59b6 100644
--- a/examples/meson.build
+++ b/examples/meson.build
@@ -16,6 +16,7 @@ all_examples = [
 	'eventdev_pipeline', 'exception_path',
 	'fips_validation', 'flow_classify',
 	'flow_filtering', 'helloworld',
+	'ioat',
 	'ip_fragmentation', 'ip_pipeline',
 	'ip_reassembly', 'ipsec-secgw',
 	'ipv4_multicast', 'kni',
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v2 2/6] examples/ioat: add software copy support
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 1/6] examples/ioat: create " Marcin Baran
@ 2019-09-13 14:39   ` Marcin Baran
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
                     ` (5 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-13 14:39 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 188 insertions(+), 23 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 977ea6a61c..3a092c6cfb 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -13,7 +13,7 @@
 #include <rte_rawdev.h>
 #include <rte_ioat_rawdev.h>
 
-/* size of ring used for software copying between rx and tx. */
+ /* size of ring used for software copying between rx and tx. */
 #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
 #define MAX_PKT_BURST 32
 #define MEMPOOL_CACHE_SIZE 512
@@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+static void
+update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
+{
+	struct rte_ether_hdr *eth;
+	void *tmp;
+
+	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
+
+	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
+	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
+	 */
+	tmp = &eth->d_addr.addr_bytes[0];
+	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
+
+	/* src addr */
+	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], &eth->s_addr);
+}
+
+static inline void
+pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+{
+	/* Copy packet metadata */
+	rte_memcpy(&dst->rearm_data,
+		&src->rearm_data,
+		offsetof(struct rte_mbuf, cacheline1)
+		- offsetof(struct rte_mbuf, rearm_data));
+
+	/* Copy packet data */
+	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+		rte_pktmbuf_mtod(src, char *), src->data_len);
+}
+
+/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+static void
+ioat_rx_port(struct rxtx_port_config *rx_config)
+{
+	uint32_t nb_rx, nb_enq, i, j;
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+
+	for (i = 0; i < rx_config->nb_queues; i++) {
+
+		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+			pkts_burst, MAX_PKT_BURST);
+
+		if (nb_rx == 0)
+			continue;
+
+		/* Perform packet software copy, free source packets */
+		int ret;
+		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+			(void *)pkts_burst_copy, nb_rx);
+
+		if (unlikely(ret < 0))
+			rte_exit(EXIT_FAILURE,
+				"Unable to allocate memory.\n");
+
+		for (j = 0; j < nb_rx; j++)
+			pktmbuf_sw_copy(pkts_burst[j],
+				pkts_burst_copy[j]);
+
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)pkts_burst, nb_rx);
+
+		nb_enq = rte_ring_enqueue_burst(
+			rx_config->rx_to_tx_ring,
+			(void *)pkts_burst_copy, nb_rx, NULL);
+
+		/* Free any not enqueued packets. */
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&pkts_burst_copy[nb_enq],
+			nb_rx - nb_enq);
+	}
+}
+
+/* Transmit packets from IOAT rawdev/rte_ring for one port. */
+static void
+ioat_tx_port(struct rxtx_port_config *tx_config)
+{
+	uint32_t i, nb_dq = 0;
+	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+	/* Deque the mbufs from rx_to_tx_ring. */
+	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+	if (nb_dq == 0)
+		return;
+
+	/* Update macs if enabled */
+	if (mac_updating) {
+		for (i = 0; i < nb_dq; i++)
+			update_mac_addrs(mbufs_dst[i],
+				tx_config->rxtx_port);
+	}
+
+	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+		0, (void *)mbufs_dst, nb_dq);
+
+	/* Free any unsent packets. */
+	if (unlikely(nb_tx < nb_dq))
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+		(void *)&mbufs_dst[nb_tx],
+			nb_dq - nb_tx);
+}
+
+/* Main rx and tx loop if only one slave lcore available */
+static void
+rxtx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
+		" lcore %u\n", rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++) {
+			ioat_rx_port(&cfg.ports[i]);
+			ioat_tx_port(&cfg.ports[i]);
+		}
+}
+
+static void start_forwarding_cores(void)
+{
+	uint32_t lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+		__func__, rte_lcore_id());
+
+	lcore_id = rte_get_next_lcore(lcore_id, true, true);
+	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+		NULL, lcore_id);
+}
+
 /* Display usage */
 static void
 ioat_usage(const char *prgname)
@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)
 		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
 		"  -c --copy-type CT: type of copy: sw|rawdev\n"
 		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
-	       prgname);
+		prgname);
 }
 
 static int
@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 	argvopt = argv;
 
 	while ((opt = getopt_long(argc, argvopt, short_options,
-				  lgopts, &option_index)) != EOF) {
+		lgopts, &option_index)) != EOF) {
 
 		switch (opt) {
-		/* portmask */
+			/* portmask */
 		case 'p':
 			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
 			if (ioat_enabled_port_mask & ~default_port_mask ||
-					ioat_enabled_port_mask <= 0) {
+				ioat_enabled_port_mask <= 0) {
 				printf("Invalid portmask, %s, suggest 0x%x\n",
-						optarg, default_port_mask);
+					optarg, default_port_mask);
 				ioat_usage(prgname);
 				return -1;
 			}
@@ -204,7 +340,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 			}
 			break;
 
-		/* long options */
+			/* long options */
 		case 0:
 			break;
 
@@ -216,9 +352,9 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 
 	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
 	if (optind >= 0)
-		argv[optind-1] = prgname;
+		argv[optind - 1] = prgname;
 
-	ret = optind-1;
+	ret = optind - 1;
 	optind = 1; /* reset getopt lib */
 	return ret;
 }
@@ -253,6 +389,26 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+assign_rings(void)
+{
+	uint32_t i;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		char ring_name[RTE_RING_NAMESIZE];
+
+		snprintf(ring_name, sizeof(ring_name), "rx_to_tx_ring_%u", i);
+		/* Create ring for inter core communication */
+		cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+			ring_name, ring_size,
+			rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
+
+		if (cfg.ports[i].rx_to_tx_ring == NULL)
+			rte_exit(EXIT_FAILURE, "Ring create failed: %s\n",
+				rte_strerror(rte_errno));
+	}
+}
+
 /*
  * Initializes a given port using global settings and with the RX buffers
  * coming from the mbuf_pool passed as a parameter.
@@ -263,7 +419,7 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	/* configuring port to use RSS for multiple RX queues */
 	static const struct rte_eth_conf port_conf = {
 		.rxmode = {
-			.mq_mode        = ETH_MQ_RX_RSS,
+			.mq_mode = ETH_MQ_RX_RSS,
 			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
 		},
 		.rx_adv_conf = {
@@ -294,14 +450,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 		dev_info.flow_type_rss_offloads;
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
 		local_port_conf.txmode.offloads |=
-			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+		DEV_TX_OFFLOAD_MBUF_FAST_FREE;
 	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE, "Cannot configure device:"
 			" err=%d, port=%u\n", ret, portid);
 
 	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
-						&nb_txd);
+		&nb_txd);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE,
 			"Cannot adjust number of descriptors: err=%d, port=%u\n",
@@ -326,8 +482,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	txq_conf = dev_info.default_txconf;
 	txq_conf.offloads = local_port_conf.txmode.offloads;
 	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
-			rte_eth_dev_socket_id(portid),
-			&txq_conf);
+		rte_eth_dev_socket_id(portid),
+		&txq_conf);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE,
 			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
@@ -335,8 +491,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	/* Initialize TX buffers */
 	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
-			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
-			rte_eth_dev_socket_id(portid));
+		RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+		rte_eth_dev_socket_id(portid));
 	if (tx_buffer[portid] == NULL)
 		rte_exit(EXIT_FAILURE,
 			"Cannot allocate buffer for tx on port %u\n",
@@ -354,13 +510,13 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	rte_eth_promiscuous_enable(portid);
 
 	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
-			portid,
-			ioat_ports_eth_addr[portid].addr_bytes[0],
-			ioat_ports_eth_addr[portid].addr_bytes[1],
-			ioat_ports_eth_addr[portid].addr_bytes[2],
-			ioat_ports_eth_addr[portid].addr_bytes[3],
-			ioat_ports_eth_addr[portid].addr_bytes[4],
-			ioat_ports_eth_addr[portid].addr_bytes[5]);
+		portid,
+		ioat_ports_eth_addr[portid].addr_bytes[0],
+		ioat_ports_eth_addr[portid].addr_bytes[1],
+		ioat_ports_eth_addr[portid].addr_bytes[2],
+		ioat_ports_eth_addr[portid].addr_bytes[3],
+		ioat_ports_eth_addr[portid].addr_bytes[4],
+		ioat_ports_eth_addr[portid].addr_bytes[5]);
 
 	cfg.ports[cfg.nb_ports].rxtx_port = portid;
 	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
@@ -428,10 +584,19 @@ main(int argc, char **argv)
 	if (cfg.nb_lcores < 1)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
+
+	assign_rings();
+
+	start_forwarding_cores();
+
+	/* force_quit is true when we get here */
+	rte_eal_mp_wait_lcore();
+
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v2 3/6] examples/ioat: add rawdev copy mode support
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 1/6] examples/ioat: create " Marcin Baran
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 2/6] examples/ioat: add software copy support Marcin Baran
@ 2019-09-13 14:39   ` Marcin Baran
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 4/6] examples/ioat: add two threads configuration Marcin Baran
                     ` (4 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-13 14:39 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++--------
 1 file changed, 189 insertions(+), 47 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 3a092c6cfb..c66ce7e49e 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
 		rte_pktmbuf_mtod(src, char *), src->data_len);
 }
 
+static uint32_t
+ioat_enqueue_packets(struct rte_mbuf **pkts,
+	uint32_t nb_rx, uint16_t dev_id)
+{
+	int ret;
+	uint32_t i;
+	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+		&pkts[0]->rearm_data);
+
+	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+		(void *)pkts_copy, nb_rx);
+
+	if (unlikely(ret < 0))
+		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+	for (i = 0; i < nb_rx; i++) {
+		/* Perform data copy */
+		ret = rte_ioat_enqueue_copy(dev_id,
+			pkts[i]->buf_iova
+			- addr_offset,
+			pkts_copy[i]->buf_iova
+			- addr_offset,
+			rte_pktmbuf_data_len(pkts[i])
+			+ addr_offset,
+			(uintptr_t)pkts[i],
+			(uintptr_t)pkts_copy[i],
+			0 /* nofence */);
+
+		if (ret != 1)
+			break;
+	}
+
+	ret = i;
+	/* Free any not enqueued packets. */
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+		nb_rx - i);
+
+
+	return ret;
+}
+
 /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
 static void
 ioat_rx_port(struct rxtx_port_config *rx_config)
@@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
-		/* Perform packet software copy, free source packets */
-		int ret;
-		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
-
-		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
-			(void *)pkts_burst_copy, nb_rx);
-
-		if (unlikely(ret < 0))
-			rte_exit(EXIT_FAILURE,
-				"Unable to allocate memory.\n");
-
-		for (j = 0; j < nb_rx; j++)
-			pktmbuf_sw_copy(pkts_burst[j],
-				pkts_burst_copy[j]);
-
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)pkts_burst, nb_rx);
-
-		nb_enq = rte_ring_enqueue_burst(
-			rx_config->rx_to_tx_ring,
-			(void *)pkts_burst_copy, nb_rx, NULL);
-
-		/* Free any not enqueued packets. */
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)&pkts_burst_copy[nb_enq],
-			nb_rx - nb_enq);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			/* Perform packet hardware copy */
+			nb_enq = ioat_enqueue_packets(pkts_burst,
+				nb_rx, rx_config->ioat_ids[i]);
+			if (nb_enq > 0)
+				rte_ioat_do_copies(rx_config->ioat_ids[i]);
+		} else {
+			/* Perform packet software copy, free source packets */
+			int ret;
+			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst_copy, nb_rx);
+
+			if (unlikely(ret < 0))
+				rte_exit(EXIT_FAILURE,
+					"Unable to allocate memory.\n");
+
+			for (j = 0; j < nb_rx; j++)
+				pktmbuf_sw_copy(pkts_burst[j],
+					pkts_burst_copy[j]);
+
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst, nb_rx);
+
+			nb_enq = rte_ring_enqueue_burst(
+				rx_config->rx_to_tx_ring,
+				(void *)pkts_burst_copy, nb_rx, NULL);
+
+			/* Free any not enqueued packets. */
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&pkts_burst_copy[nb_enq],
+				nb_rx - nb_enq);
+		}
 	}
 }
 
@@ -169,31 +221,64 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 static void
 ioat_tx_port(struct rxtx_port_config *tx_config)
 {
-	uint32_t i, nb_dq = 0;
+	uint32_t i, j, nb_dq = 0;
+	struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
-	/* Deque the mbufs from rx_to_tx_ring. */
-	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
-		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		/* Deque the mbufs from IOAT device. */
+		for (i = 0; i < tx_config->nb_queues; i++) {
+			nb_dq = rte_ioat_completed_copies(
+				tx_config->ioat_ids[i], MAX_PKT_BURST,
+				(void *)mbufs_src, (void *)mbufs_dst);
 
-	if (nb_dq == 0)
-		return;
+			if (nb_dq == 0)
+				break;
 
-	/* Update macs if enabled */
-	if (mac_updating) {
-		for (i = 0; i < nb_dq; i++)
-			update_mac_addrs(mbufs_dst[i],
-				tx_config->rxtx_port);
-	}
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)mbufs_src, nb_dq);
 
-	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
-		0, (void *)mbufs_dst, nb_dq);
+			/* Update macs if enabled */
+			if (mac_updating) {
+				for (j = 0; j < nb_dq; j++)
+					update_mac_addrs(mbufs_dst[j],
+						tx_config->rxtx_port);
+			}
+
+			const uint16_t nb_tx = rte_eth_tx_burst(
+				tx_config->rxtx_port, 0,
+				(void *)mbufs_dst, nb_dq);
 
-	/* Free any unsent packets. */
-	if (unlikely(nb_tx < nb_dq))
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-		(void *)&mbufs_dst[nb_tx],
-			nb_dq - nb_tx);
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_dq))
+				rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&mbufs_dst[nb_tx],
+					nb_dq - nb_tx);
+		}
+	} else {
+		/* Deque the mbufs from rx_to_tx_ring. */
+		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+		if (nb_dq == 0)
+			return;
+
+		/* Update macs if enabled */
+		if (mac_updating) {
+			for (i = 0; i < nb_dq; i++)
+				update_mac_addrs(mbufs_dst[i],
+					tx_config->rxtx_port);
+		}
+
+		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+			0, (void *)mbufs_dst, nb_dq);
+
+		/* Free any unsent packets. */
+		if (unlikely(nb_tx < nb_dq))
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&mbufs_dst[nb_tx],
+				nb_dq - nb_tx);
+	}
 }
 
 /* Main rx and tx loop if only one slave lcore available */
@@ -389,6 +474,52 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+configure_rawdev_queue(uint32_t dev_id)
+{
+	struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
+	struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+	if (rte_rawdev_configure(dev_id, &info) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_configure()\n");
+	}
+	if (rte_rawdev_start(dev_id) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_start()\n");
+	}
+}
+
+static void
+assign_rawdevs(void)
+{
+	uint16_t nb_rawdev = 0, rdev_id = 0;
+	uint32_t i, j;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+			struct rte_rawdev_info rdev_info = { 0 };
+
+			do {
+				if (rdev_id == rte_rawdev_count())
+					goto end;
+				rte_rawdev_info_get(rdev_id++, &rdev_info);
+			} while (strcmp(rdev_info.driver_name,
+				IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+			cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+			configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+			++nb_rawdev;
+		}
+	}
+end:
+	if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+		rte_exit(EXIT_FAILURE,
+			"Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+			nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+	RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+}
+
 static void
 assign_rings(void)
 {
@@ -585,18 +716,29 @@ main(int argc, char **argv)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
 
-	assign_rings();
+	if (copy_mode == COPY_MODE_IOAT_NUM)
+		assign_rawdevs();
+	else /* copy_mode == COPY_MODE_SW_NUM */
+		assign_rings();
 
 	start_forwarding_cores();
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
 
+	uint32_t j;
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
-		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+				printf("Stopping rawdev %d\n",
+					cfg.ports[i].ioat_ids[j]);
+				rte_rawdev_stop(cfg.ports[i].ioat_ids[j]);
+			}
+		} else /* copy_mode == COPY_MODE_SW_NUM */
+			rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v2 4/6] examples/ioat: add two threads configuration
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
                     ` (2 preceding siblings ...)
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
@ 2019-09-13 14:39   ` Marcin Baran
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 5/6] examples/ioat: add stats printing for each port Marcin Baran
                     ` (3 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-13 14:39 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++-----------
 1 file changed, 67 insertions(+), 25 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index c66ce7e49e..4c51db6bda 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,
 	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
 		nb_rx - i);
 
-
 	return ret;
 }
 
@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
 	if (copy_mode == COPY_MODE_IOAT_NUM) {
-		/* Deque the mbufs from IOAT device. */
 		for (i = 0; i < tx_config->nb_queues; i++) {
+			/* Deque the mbufs from IOAT device. */
 			nb_dq = rte_ioat_completed_copies(
 				tx_config->ioat_ids[i], MAX_PKT_BURST,
 				(void *)mbufs_src, (void *)mbufs_dst);
@@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 					nb_dq - nb_tx);
 		}
 	} else {
-		/* Deque the mbufs from rx_to_tx_ring. */
-		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
-			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
-
-		if (nb_dq == 0)
-			return;
-
-		/* Update macs if enabled */
-		if (mac_updating) {
-			for (i = 0; i < nb_dq; i++)
-				update_mac_addrs(mbufs_dst[i],
-					tx_config->rxtx_port);
-		}
+		for (i = 0; i < tx_config->nb_queues; i++) {
+			/* Deque the mbufs from IOAT device. */
+			nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+				(void *)mbufs_dst, MAX_PKT_BURST, NULL);
 
-		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
-			0, (void *)mbufs_dst, nb_dq);
+			if (nb_dq == 0)
+				return;
 
-		/* Free any unsent packets. */
-		if (unlikely(nb_tx < nb_dq))
-			rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)&mbufs_dst[nb_tx],
-				nb_dq - nb_tx);
+			/* Update macs if enabled */
+			if (mac_updating) {
+				for (j = 0; j < nb_dq; j++)
+					update_mac_addrs(mbufs_dst[j],
+						tx_config->rxtx_port);
+			}
+
+			const uint16_t nb_tx =
+				rte_eth_tx_burst(tx_config->rxtx_port, 0,
+					(void *)mbufs_dst, nb_dq);
+
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_dq))
+				rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&mbufs_dst[nb_tx],
+					nb_dq - nb_tx);
+		}
 	}
 }
 
+/* Main rx processing loop for IOAT rawdev. */
+static void
+rx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_rx_port(&cfg.ports[i]);
+}
+
+/* Main tx processing loop for hardware copy. */
+static void
+tx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main tx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_tx_port(&cfg.ports[i]);
+}
+
 /* Main rx and tx loop if only one slave lcore available */
 static void
 rxtx_main_loop(void)
@@ -305,9 +337,19 @@ static void start_forwarding_cores(void)
 	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
 		__func__, rte_lcore_id());
 
-	lcore_id = rte_get_next_lcore(lcore_id, true, true);
-	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
-		NULL, lcore_id);
+	if (cfg.nb_lcores == 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+			NULL, lcore_id);
+	} else if (cfg.nb_lcores > 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+			NULL, lcore_id);
+
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+			lcore_id);
+	}
 }
 
 /* Display usage */
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v2 5/6] examples/ioat: add stats printing for each port
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
                     ` (3 preceding siblings ...)
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 4/6] examples/ioat: add two threads configuration Marcin Baran
@ 2019-09-13 14:39   ` Marcin Baran
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
                     ` (2 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-13 14:39 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added printing stats of ports each second.
The stats printing is done using master core.
The information provided informs about packets
received, dropped and send as well as statistics
of rawdev devices.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 241 insertions(+), 4 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 4c51db6bda..c6b9948323 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -48,10 +48,27 @@ struct rxtx_transmission_config {
 	uint16_t nb_lcores;
 };
 
+/* per-port statistics struct */
+struct ioat_port_statistics {
+	uint64_t rx[RTE_MAX_ETHPORTS];
+	uint64_t tx[RTE_MAX_ETHPORTS];
+	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
+	uint64_t copy_dropped[RTE_MAX_ETHPORTS];
+};
+struct ioat_port_statistics port_statistics;
+
+struct total_statistics {
+	uint64_t total_packets_dropped;
+	uint64_t total_packets_tx;
+	uint64_t total_packets_rx;
+	uint64_t total_successful_enqueues;
+	uint64_t total_failed_enqueues;
+};
+
 typedef enum copy_mode_t {
 #define COPY_MODE_SW "sw"
 	COPY_MODE_SW_NUM,
-#define COPY_MODE_IOAT "rawdev"
+#define COPY_MODE_IOAT "hw"
 	COPY_MODE_IOAT_NUM,
 	COPY_MODE_INVALID_NUM,
 	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+/* Print out statistics for one port. */
+static void
+print_port_stats(uint16_t port_id)
+{
+	printf("\nStatistics for port %u ------------------------------"
+		"\nPackets sent: %34"PRIu64
+		"\nPackets received: %30"PRIu64
+		"\nPackets dropped on tx: %25"PRIu64
+		"\nPackets dropped on copy: %23"PRIu64,
+		port_id,
+		port_statistics.tx[port_id],
+		port_statistics.rx[port_id],
+		port_statistics.tx_dropped[port_id],
+		port_statistics.copy_dropped[port_id]);
+}
+
+/* Print out statistics for one IOAT rawdev device. */
+static void
+print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
+	unsigned int *ids_xstats, uint16_t nb_xstats,
+	struct rte_rawdev_xstats_name *names_xstats)
+{
+	uint16_t i;
+
+	printf("\nIOAT channel %u", dev_id);
+	for (i = 0; i < nb_xstats; i++)
+		printf("\n\t %s: %*"PRIu64,
+			names_xstats[ids_xstats[i]].name,
+			(int)(37 - strlen(names_xstats[ids_xstats[i]].name)),
+			xstats[i]);
+}
+
+static void
+print_total_stats(struct total_statistics *ts)
+{
+	printf("\nAggregate statistics ==============================="
+		"\nTotal packets Tx: %24"PRIu64" [pps]"
+		"\nTotal packets Rx: %24"PRIu64" [pps]"
+		"\nTotal packets dropped: %19"PRIu64" [pps]",
+		ts->total_packets_tx,
+		ts->total_packets_rx,
+		ts->total_packets_dropped);
+
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		printf("\nTotal IOAT successful enqueues: %8"PRIu64" [enq/s]"
+			"\nTotal IOAT failed enqueues: %12"PRIu64" [enq/s]",
+			ts->total_successful_enqueues,
+			ts->total_failed_enqueues);
+	}
+
+	printf("\n====================================================\n");
+}
+
+/* Print out statistics on packets dropped. */
+static void
+print_stats(char *prgname)
+{
+	struct total_statistics ts, delta_ts;
+	uint32_t i, port_id, dev_id;
+	struct rte_rawdev_xstats_name *names_xstats;
+	uint64_t *xstats;
+	unsigned int *ids_xstats, nb_xstats;
+	char status_string[120]; /* to print at the top of the output */
+	int status_strlen;
+
+
+	const char clr[] = { 27, '[', '2', 'J', '\0' };
+	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+	status_strlen = snprintf(status_string, sizeof(status_string),
+		"%s, ", prgname);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Worker Threads = %d, ",
+		rte_lcore_count() > 2 ? 2 : 1);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
+		COPY_MODE_SW : COPY_MODE_IOAT);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Updating MAC = %s, ", mac_updating ?
+		"enabled" : "disabled");
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Rx Queues = %d, ", nb_queues);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Ring Size = %d\n", ring_size);
+
+	/* Allocate memory for xstats names and values */
+	nb_xstats = rte_rawdev_xstats_names_get(
+		cfg.ports[0].ioat_ids[0], NULL, 0);
+
+	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
+	if (names_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat names memory\n");
+	}
+	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
+		names_xstats, nb_xstats);
+
+	ids_xstats = malloc(sizeof(*ids_xstats) * 2);
+	if (ids_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat ids_xstats memory\n");
+	}
+
+	xstats = malloc(sizeof(*xstats) * 2);
+	if (xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat memory\n");
+	}
+
+	/* Get failed/successful enqueues stats index */
+	ids_xstats[0] = ids_xstats[1] = nb_xstats;
+	for (i = 0; i < nb_xstats; i++) {
+		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
+			ids_xstats[0] = i;
+		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
+			ids_xstats[1] = i;
+		if (ids_xstats[0] < nb_xstats && ids_xstats[1] < nb_xstats)
+			break;
+	}
+	if (ids_xstats[0] == nb_xstats || ids_xstats[1] == nb_xstats) {
+		rte_exit(EXIT_FAILURE,
+			"Error getting failed/successful enqueues stats index\n");
+	}
+
+	memset(&ts, 0, sizeof(struct total_statistics));
+
+	while (!force_quit) {
+		/* Sleep for 1 second each round - init sleep allows reading
+		 * messages from app startup.
+		 */
+		sleep(1);
+
+		/* Clear screen and move to top left */
+		printf("%s%s", clr, topLeft);
+
+		memset(&delta_ts, 0, sizeof(struct total_statistics));
+
+		printf("%s", status_string);
+
+		for (i = 0; i < cfg.nb_ports; i++) {
+			port_id = cfg.ports[i].rxtx_port;
+			print_port_stats(port_id);
+
+			delta_ts.total_packets_dropped +=
+				port_statistics.tx_dropped[port_id]
+				+ port_statistics.copy_dropped[port_id];
+			delta_ts.total_packets_tx +=
+				port_statistics.tx[port_id];
+			delta_ts.total_packets_rx +=
+				port_statistics.rx[port_id];
+
+			if (copy_mode == COPY_MODE_IOAT_NUM) {
+				uint32_t j;
+
+				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+					dev_id = cfg.ports[i].ioat_ids[j];
+					rte_rawdev_xstats_get(dev_id,
+						ids_xstats, xstats, 2);
+
+					print_rawdev_stats(dev_id, xstats,
+						ids_xstats, 2, names_xstats);
+
+					delta_ts.total_failed_enqueues +=
+						xstats[ids_xstats[0]];
+					delta_ts.total_successful_enqueues +=
+						xstats[ids_xstats[1]];
+				}
+			}
+		}
+
+		delta_ts.total_packets_tx -= ts.total_packets_tx;
+		delta_ts.total_packets_rx -= ts.total_packets_rx;
+		delta_ts.total_packets_dropped -= ts.total_packets_dropped;
+		delta_ts.total_failed_enqueues -= ts.total_failed_enqueues;
+		delta_ts.total_successful_enqueues -=
+			ts.total_successful_enqueues;
+
+		printf("\n");
+		print_total_stats(&delta_ts);
+
+		ts.total_packets_tx += delta_ts.total_packets_tx;
+		ts.total_packets_rx += delta_ts.total_packets_rx;
+		ts.total_packets_dropped += delta_ts.total_packets_dropped;
+		ts.total_failed_enqueues += delta_ts.total_failed_enqueues;
+		ts.total_successful_enqueues +=
+			delta_ts.total_successful_enqueues;
+	}
+
+	free(names_xstats);
+	free(xstats);
+	free(ids_xstats);
+}
+
 static void
 update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
 {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
+		port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Perform packet hardware copy */
 			nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 				(void *)&pkts_burst_copy[nb_enq],
 				nb_rx - nb_enq);
 		}
+
+		port_statistics.copy_dropped[rx_config->rxtx_port] +=
+			(nb_rx - nb_enq);
 	}
 }
 
@@ -248,6 +468,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 				tx_config->rxtx_port, 0,
 				(void *)mbufs_dst, nb_dq);
 
+			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 			/* Free any unsent packets. */
 			if (unlikely(nb_tx < nb_dq))
 				rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -274,6 +496,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 				rte_eth_tx_burst(tx_config->rxtx_port, 0,
 					(void *)mbufs_dst, nb_dq);
 
+			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 			/* Free any unsent packets. */
 			if (unlikely(nb_tx < nb_dq))
 				rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)
 		"      When enabled:\n"
 		"       - The source MAC address is replaced by the TX port MAC address\n"
 		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
-		"  -c --copy-type CT: type of copy: sw|rawdev\n"
+		"  -c --copy-type CT: type of copy: sw|hw\n"
 		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
 		prgname);
 }
@@ -400,7 +624,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 	static const char short_options[] =
 		"p:"  /* portmask */
 		"q:"  /* number of RX queues per port */
-		"c:"  /* copy type (sw|rawdev) */
+		"c:"  /* copy type (sw|hw) */
 		"s:"  /* ring size */
 		;
 
@@ -452,7 +676,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 		case 'c':
 			copy_mode = ioat_parse_copy_mode(optarg);
 			if (copy_mode == COPY_MODE_INVALID_NUM) {
-				printf("Invalid copy type. Use: sw, rawdev\n");
+				printf("Invalid copy type. Use: sw, hw\n");
 				ioat_usage(prgname);
 				return -1;
 			}
@@ -673,6 +897,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
 
+	ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+		rte_eth_tx_buffer_count_callback,
+		&port_statistics.tx_dropped[portid]);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot set error callback for tx buffer on port %u\n",
+			portid);
+
 	/* Start device */
 	ret = rte_eth_dev_start(portid);
 	if (ret < 0)
@@ -749,6 +981,9 @@ main(int argc, char **argv)
 	RTE_ETH_FOREACH_DEV(portid)
 		port_init(portid, ioat_pktmbuf_pool, nb_queues);
 
+	/* Initialize port xstats */
+	memset(&port_statistics, 0, sizeof(port_statistics));
+
 	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
 		sleep(1);
 
@@ -764,6 +999,8 @@ main(int argc, char **argv)
 		assign_rings();
 
 	start_forwarding_cores();
+	/* master core prints stats while other cores forward */
+	print_stats(argv[0]);
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v2 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
                     ` (4 preceding siblings ...)
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 5/6] examples/ioat: add stats printing for each port Marcin Baran
@ 2019-09-13 14:39   ` Marcin Baran
  2019-09-13 18:45   ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage Aaron Conole
  2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-13 14:39 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran

Added guide for IOAT sample app usage and
code description.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 doc/guides/sample_app_ug/index.rst |   1 +
 doc/guides/sample_app_ug/intro.rst |   4 +
 doc/guides/sample_app_ug/ioat.rst  | 764 +++++++++++++++++++++++++++++
 3 files changed, 769 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst

diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index f23f8f59e9..a6a1d9e7a1 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -23,6 +23,7 @@ Sample Applications User Guides
     ip_reassembly
     kernel_nic_interface
     keep_alive
+    ioat
     l2_forward_crypto
     l2_forward_job_stats
     l2_forward_real_virtual
diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
index 90704194a1..74462312ff 100644
--- a/doc/guides/sample_app_ug/intro.rst
+++ b/doc/guides/sample_app_ug/intro.rst
@@ -91,6 +91,10 @@ examples are highlighted below.
   forwarding, or ``l3fwd`` application does forwarding based on Internet
   Protocol, IPv4 or IPv6 like a simple router.
 
+* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
+  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
+  copying packets between two threads.
+
 * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
   demonstrates how to distribute packets arriving on an Rx port to different
   cores for processing and transmission.
diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
new file mode 100644
index 0000000000..69621673bb
--- /dev/null
+++ b/doc/guides/sample_app_ug/ioat.rst
@@ -0,0 +1,764 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Intel Corporation.
+
+Sample Application of packet copying using Intel\ |reg| QuickData Technology
+============================================================================
+
+Overview
+--------
+
+This sample is intended as a demonstration of the basic components of a DPDK
+forwarding application and example of how to use IOAT driver API to make
+packets copies.
+
+Also while forwarding, the MAC addresses are affected as follows:
+
+*   The source MAC address is replaced by the TX port MAC address
+
+*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
+
+This application can be used to compare performance of using software packet
+copy with copy done using a DMA device for different sizes of packets.
+The example will print out statistics each second. The stats shows
+received/send packets and packets dropped or failed to copy.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``ioat`` sub-directory.
+
+
+Running the Application
+-----------------------
+
+In order to run the hardware copy application, the copying device
+needs to be bound to user-space IO driver.
+
+Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology*
+guide for information on using the driver.
+
+The application requires a number of command line options:
+
+.. code-block:: console
+
+    ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>]
+        [--[no-]mac-updating]
+
+where,
+
+*   p MASK: A hexadecimal bitmask of the ports to configure
+
+*   q NQ: Number of Rx queues used per port equivalent to CBDMA channels
+    per port
+
+*   c CT: Performed packet copy type: software (sw) or hardware using
+    DMA (hw)
+
+*   s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for
+    software copy mode
+
+*   --[no-]mac-updating: Whether MAC address of packets should be changed
+    or not
+
+The application can be launched in various configurations depending on
+provided parameters. Each port can use up to 2 lcores: one of them receives
+incoming traffic and makes a copy of each packet. The second lcore then
+updates MAC address and sends the copy. If one lcore per port is used,
+both operations are done sequentially. For each configuration an additional
+lcore is needed since master lcore in use which is responsible for
+configuration, statistics printing and safe deinitialization of all ports
+and devices.
+
+The application can use a maximum of 8 ports.
+
+To run the application in a Linux environment with 3 lcores (one of them
+is master lcore), 1 port (port 0), software copying and MAC updating issue
+the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
+
+To run the application in a Linux environment with 2 lcores (one of them
+is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC
+updating issue the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw
+
+Refer to the *DPDK Getting Started Guide* for general information on
+running applications and the Environment Abstraction Layer (EAL) options.
+
+Explanation
+-----------
+
+The following sections provide an explanation of the main components of the
+code.
+
+All DPDK library functions used in the sample code are prefixed with
+``rte_`` and are explained in detail in the *DPDK API Documentation*.
+
+
+The Main Function
+~~~~~~~~~~~~~~~~~
+
+The ``main()`` function performs the initialization and calls the execution
+threads for each lcore.
+
+The first task is to initialize the Environment Abstraction Layer (EAL).
+The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
+function. The value returned is the number of parsed arguments:
+
+.. code-block:: c
+
+    /* init EAL */
+    ret = rte_eal_init(argc, argv);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+
+
+The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
+used by the application:
+
+.. code-block:: c
+
+    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
+        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+        MIN_POOL_SIZE);
+
+    /* Create the mbuf pool */
+    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+        rte_socket_id());
+    if (ioat_pktmbuf_pool == NULL)
+        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+Mbufs are the packet buffer structure used by DPDK. They are explained in
+detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
+
+The ``main()`` function also initializes the ports:
+
+.. code-block:: c
+
+    /* Initialise each port */
+    RTE_ETH_FOREACH_DEV(portid) {
+        port_init(portid, ioat_pktmbuf_pool);
+    }
+
+Each port is configured using ``port_init()``:
+
+.. code-block:: c
+
+     /*
+     * Initializes a given port using global settings and with the RX buffers
+     * coming from the mbuf_pool passed as a parameter.
+     */
+    static inline void
+    port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+    {
+        /* configuring port to use RSS for multiple RX queues */
+        static const struct rte_eth_conf port_conf = {
+            .rxmode = {
+                .mq_mode        = ETH_MQ_RX_RSS,
+                .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+            },
+            .rx_adv_conf = {
+                .rss_conf = {
+                    .rss_key = NULL,
+                    .rss_hf = ETH_RSS_PROTO_MASK,
+                }
+            }
+        };
+
+        struct rte_eth_rxconf rxq_conf;
+        struct rte_eth_txconf txq_conf;
+        struct rte_eth_conf local_port_conf = port_conf;
+        struct rte_eth_dev_info dev_info;
+        int ret, i;
+
+        /* Skip ports that are not enabled */
+        if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+            printf("Skipping disabled port %u\n", portid);
+            return;
+        }
+
+        /* Init port */
+        printf("Initializing port %u... ", portid);
+        fflush(stdout);
+        rte_eth_dev_info_get(portid, &dev_info);
+        local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+            dev_info.flow_type_rss_offloads;
+        if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+            local_port_conf.txmode.offloads |=
+                DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+        ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE, "Cannot configure device:"
+                " err=%d, port=%u\n", ret, portid);
+
+        ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+                            &nb_txd);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "Cannot adjust number of descriptors: err=%d, port=%u\n",
+                ret, portid);
+
+        rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+        /* Init Rx queues */
+        rxq_conf = dev_info.default_rxconf;
+        rxq_conf.offloads = local_port_conf.rxmode.offloads;
+        for (i = 0; i < nb_queues; i++) {
+            ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+                rte_eth_dev_socket_id(portid), &rxq_conf,
+                mbuf_pool);
+            if (ret < 0)
+                rte_exit(EXIT_FAILURE,
+                    "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+                    ret, portid, i);
+        }
+
+        /* Init one TX queue on each port */
+        txq_conf = dev_info.default_txconf;
+        txq_conf.offloads = local_port_conf.txmode.offloads;
+        ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+                rte_eth_dev_socket_id(portid),
+                &txq_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "rte_eth_tx_queue_setup:err=%d,port=%u\n",
+                ret, portid);
+
+        /* Initialize TX buffers */
+        tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+                RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+                rte_eth_dev_socket_id(portid));
+        if (tx_buffer[portid] == NULL)
+            rte_exit(EXIT_FAILURE,
+                "Cannot allocate buffer for tx on port %u\n",
+                portid);
+
+        rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+        ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+                rte_eth_tx_buffer_count_callback,
+                &port_statistics.tx_dropped[portid]);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "Cannot set error callback for tx buffer on port %u\n",
+                portid);
+
+        /* Start device */
+        ret = rte_eth_dev_start(portid);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "rte_eth_dev_start:err=%d, port=%u\n",
+                ret, portid);
+
+        rte_eth_promiscuous_enable(portid);
+
+        printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+                portid,
+                ioat_ports_eth_addr[portid].addr_bytes[0],
+                ioat_ports_eth_addr[portid].addr_bytes[1],
+                ioat_ports_eth_addr[portid].addr_bytes[2],
+                ioat_ports_eth_addr[portid].addr_bytes[3],
+                ioat_ports_eth_addr[portid].addr_bytes[4],
+                ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+        cfg.ports[cfg.nb_ports].rxtx_port = portid;
+        cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+    }
+
+The Ethernet ports are configured with local settings using the
+``rte_eth_dev_configure()`` function and the ``port_conf`` struct.
+The RSS is enabled so that multiple Rx queues could be used for
+packet receiving and copying by multiple CBDMA channels per port:
+
+.. code-block:: c
+
+    /* configuring port to use RSS for multiple RX queues */
+    static const struct rte_eth_conf port_conf = {
+        .rxmode = {
+            .mq_mode        = ETH_MQ_RX_RSS,
+            .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+        },
+        .rx_adv_conf = {
+            .rss_conf = {
+                .rss_key = NULL,
+                .rss_hf = ETH_RSS_PROTO_MASK,
+            }
+        }
+    };
+
+For this example the ports are set up with the number of Rx queues provided
+with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()``
+and ``rte_eth_tx_queue_setup()`` functions.
+
+The Ethernet port is then started:
+
+.. code-block:: c
+
+    ret = rte_eth_dev_start(portid);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
+            ret, portid);
+
+
+Finally the Rx port is set in promiscuous mode:
+
+.. code-block:: c
+
+    rte_eth_promiscuous_enable(portid);
+
+
+After that each port application assigns resources needed.
+
+.. code-block:: c
+
+    check_link_status(ioat_enabled_port_mask);
+
+    if (!cfg.nb_ports) {
+        rte_exit(EXIT_FAILURE,
+            "All available ports are disabled. Please set portmask.\n");
+    }
+
+    /* Check if there is enough lcores for all ports. */
+    cfg.nb_lcores = rte_lcore_count() - 1;
+    if (cfg.nb_lcores < 1)
+        rte_exit(EXIT_FAILURE,
+            "There should be at least one slave lcore.\n");
+
+    ret = 0;
+
+    if (copy_mode == COPY_MODE_IOAT_NUM) {
+        assign_rawdevs();
+    } else /* copy_mode == COPY_MODE_SW_NUM */ {
+        assign_rings();
+    }
+
+A link status is checked of each port enabled by port mask
+using ``check_link_status()`` function.
+
+.. code-block:: c
+
+    /* check link status, return true if at least one port is up */
+    static int
+    check_link_status(uint32_t port_mask)
+    {
+        uint16_t portid;
+        struct rte_eth_link link;
+        int retval = 0;
+
+        printf("\nChecking link status\n");
+        RTE_ETH_FOREACH_DEV(portid) {
+            if ((port_mask & (1 << portid)) == 0)
+                continue;
+
+            memset(&link, 0, sizeof(link));
+            rte_eth_link_get(portid, &link);
+
+            /* Print link status */
+            if (link.link_status) {
+                printf(
+                    "Port %d Link Up. Speed %u Mbps - %s\n",
+                    portid, link.link_speed,
+                    (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+                    ("full-duplex") : ("half-duplex\n"));
+                retval = 1;
+            } else
+                printf("Port %d Link Down\n", portid);
+        }
+        return retval;
+    }
+
+Depending on mode set (whether copy should be done by software or by hardware)
+special structures are assigned to each port. If software copy was chosen,
+application have to assign ring structures for packet exchanging between lcores
+assigned to ports.
+
+.. code-block:: c
+
+    static void
+    assign_rings(void)
+    {
+        uint32_t i;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            char ring_name[20];
+
+            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
+            /* Create ring for inter core communication */
+            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+                    ring_name, ring_size,
+                    rte_socket_id(), RING_F_SP_ENQ);
+
+            if (cfg.ports[i].rx_to_tx_ring == NULL)
+                rte_exit(EXIT_FAILURE, "%s\n",
+                        rte_strerror(rte_errno));
+        }
+    }
+
+
+When using hardware copy each Rx queue of the port is assigned an
+IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API
+functions:
+
+.. code-block:: c
+
+    static void
+    assign_rawdevs(void)
+    {
+        uint16_t nb_rawdev = 0, rdev_id = 0;
+        uint32_t i, j;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+                struct rte_rawdev_info rdev_info = { 0 };
+
+                do {
+                    if (rdev_id == rte_rawdev_count())
+                        goto end;
+                    rte_rawdev_info_get(rdev_id++, &rdev_info);
+                } while (strcmp(rdev_info.driver_name,
+                    IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+                cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+                configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+                ++nb_rawdev;
+            }
+        }
+    end:
+        if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+            rte_exit(EXIT_FAILURE,
+                "Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+                nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+    }
+
+
+The initialization of hardware device is done by ``rte_rawdev_configure()``
+function and ``rte_rawdev_info`` struct. After configuration the device is
+started using ``rte_rawdev_start()`` function. Each of the above operations
+is done in ``configure_rawdev_queue()``.
+
+.. code-block:: c
+
+    static void
+    configure_rawdev_queue(uint32_t dev_id)
+    {
+        struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+        /* Configure hardware copy device */
+        dev_config.ring_size = ring_size;
+
+        if (rte_rawdev_configure(dev_id, &info) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_configure()\n");
+        }
+        rte_rawdev_info_get(dev_id, &info);
+        if (dev_config.ring_size != ring_size) {
+            rte_exit(EXIT_FAILURE,
+                "Error, ring size is not %d (%d)\n",
+                ring_size, (int)dev_config.ring_size);
+        }
+        if (rte_rawdev_start(dev_id) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_start()\n");
+        }
+    }
+
+If initialization is successful memory for hardware device
+statistics is allocated.
+
+Finally ``main()`` functions starts all processing lcores and starts
+printing stats in a loop on master lcore. The application can be
+interrupted and closed using ``Ctrl-C``. The master lcore waits for
+all slave processes to finish, deallocates resources and exits.
+
+The processing lcores launching function are described below.
+
+The Lcores Launching Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As described above ``main()`` function invokes ``start_forwarding_cores()``
+function in order to start processing for each lcore:
+
+.. code-block:: c
+
+    static void start_forwarding_cores(void)
+    {
+        uint32_t lcore_id = rte_lcore_id();
+
+        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+                __func__, rte_lcore_id());
+
+        if (cfg.nb_lcores == 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+                NULL, lcore_id);
+        } else if (cfg.nb_lcores > 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+                NULL, lcore_id);
+
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+                lcore_id);
+        }
+    }
+
+The function launches Rx/Tx processing functions on configured lcores
+for each port using ``rte_eal_remote_launch()``. The configured ports,
+their number and number of assigned lcores are stored in user-defined
+``rxtx_transmission_config`` struct that is initialized before launching
+tasks:
+
+.. code-block:: c
+
+    struct rxtx_transmission_config {
+        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+        uint16_t nb_ports;
+        uint16_t nb_lcores;
+    };
+
+The Lcores Processing Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For receiving packets on each port an ``ioat_rx_port()`` function is used.
+The function receives packets on each configured Rx queue. Depending on mode
+the user chose, it will enqueue packets to IOAT rawdev channels and then invoke
+copy process (hardware copy), or perform software copy of each packet using
+``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring:
+
+.. code-block:: c
+
+    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+    static void
+    ioat_rx_port(struct rxtx_port_config *rx_config)
+    {
+        uint32_t nb_rx, nb_enq, i, j;
+        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+        for (i = 0; i < rx_config->nb_queues; i++) {
+
+            nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+                pkts_burst, MAX_PKT_BURST);
+
+            if (nb_rx == 0)
+                continue;
+
+            port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
+            if (copy_mode == COPY_MODE_IOAT_NUM) {
+                /* Perform packet hardware copy */
+                nb_enq = ioat_enqueue_packets(pkts_burst,
+                    nb_rx, rx_config->ioat_ids[i]);
+                if (nb_enq > 0)
+                    rte_ioat_do_copies(rx_config->ioat_ids[i]);
+            } else {
+                /* Perform packet software copy, free source packets */
+                int ret;
+                struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+                ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst_copy, nb_rx);
+
+                if (unlikely(ret < 0))
+                    rte_exit(EXIT_FAILURE,
+                        "Unable to allocate memory.\n");
+
+                for (j = 0; j < nb_rx; j++)
+                    pktmbuf_sw_copy(pkts_burst[j],
+                        pkts_burst_copy[j]);
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst, nb_rx);
+
+                nb_enq = rte_ring_enqueue_burst(
+                    rx_config->rx_to_tx_ring,
+                    (void *)pkts_burst_copy, nb_rx, NULL);
+
+                /* Free any not enqueued packets. */
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&pkts_burst_copy[nb_enq],
+                    nb_rx - nb_enq);
+            }
+
+            port_statistics.copy_dropped[rx_config->rxtx_port] +=
+                (nb_rx - nb_enq);
+        }
+    }
+
+The packets are received in burst mode using ``rte_eth_rx_burst()``
+function. When using hardware copy mode the packets are enqueued in
+copying device's buffer using ``ioat_enqueue_packets()`` which calls
+``rte_ioat_enqueue_copy()``. When all received packets are in the
+buffer the copies are invoked by calling ``rte_ioat_do_copies()``.
+Function ``rte_ioat_enqueue_copy()`` operates on physical address of
+the packet. Structure ``rte_mbuf`` contains only physical address to
+start of the data buffer (``buf_iova``). Thus the address is shifted
+by ``addr_offset`` value in order to get pointer to ``rearm_data``
+member of ``rte_mbuf``. That way the packet is copied all at once
+(with data and metadata).
+
+.. code-block:: c
+
+    static uint32_t
+    ioat_enqueue_packets(struct rte_mbuf **pkts,
+        uint32_t nb_rx, uint16_t dev_id)
+    {
+        int ret;
+        uint32_t i;
+        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+            &pkts[0]->rearm_data);
+
+        ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                (void *)pkts_copy, nb_rx);
+
+        if (unlikely(ret < 0))
+            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+        for (i = 0; i < nb_rx; i++) {
+            /* Perform data copy */
+            ret = rte_ioat_enqueue_copy(dev_id,
+                pkts[i]->buf_iova
+                    - addr_offset,
+                pkts_copy[i]->buf_iova
+                    - addr_offset,
+                rte_pktmbuf_data_len(pkts[i])
+                    + addr_offset,
+                (uintptr_t)pkts[i],
+                (uintptr_t)pkts_copy[i],
+                0 /* nofence */);
+
+            if (ret != 1)
+                break;
+        }
+
+        ret = i;
+        /* Free any not enqueued packets. */
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+            nb_rx - i);
+
+        return ret;
+    }
+
+
+All done copies are processed by ``ioat_tx_port()`` function. When using
+hardware copy mode the function invokes ``rte_ioat_completed_copies()``
+on each assigned IOAT channel to gather copied packets. If software copy
+mode is used the function dequeues copied packets from the rte_ring. Then each
+packet MAC address is changed if it was enabled. After that copies are sent
+in burst mode using `` rte_eth_tx_burst()``.
+
+
+.. code-block:: c
+
+    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
+    static void
+    ioat_tx_port(struct rxtx_port_config *tx_config)
+    {
+        uint32_t i, j, nb_dq = 0;
+        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
+        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+        if (copy_mode == COPY_MODE_IOAT_NUM) {
+            for (i = 0; i < tx_config->nb_queues; i++) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ioat_completed_copies(
+                    tx_config->ioat_ids[i], MAX_PKT_BURST,
+                    (void *)mbufs_src, (void *)mbufs_dst);
+
+                if (nb_dq == 0)
+                    break;
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)mbufs_src, nb_dq);
+
+                /* Update macs if enabled */
+                if (mac_updating) {
+                    for (j = 0; j < nb_dq; j++)
+                        update_mac_addrs(mbufs_dst[j],
+                            tx_config->rxtx_port);
+                }
+
+                const uint16_t nb_tx = rte_eth_tx_burst(
+                    tx_config->rxtx_port, 0,
+                    (void *)mbufs_dst, nb_dq);
+
+                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+                /* Free any unsent packets. */
+                if (unlikely(nb_tx < nb_dq))
+                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&mbufs_dst[nb_tx],
+                        nb_dq - nb_tx);
+            }
+        }
+        else {
+            for (i = 0; i < tx_config->nb_queues; i++) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+                    (void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+                if (nb_dq == 0)
+                    return;
+
+                /* Update macs if enabled */
+                if (mac_updating) {
+                    for (j = 0; j < nb_dq; j++)
+                        update_mac_addrs(mbufs_dst[j],
+                            tx_config->rxtx_port);
+                }
+
+                const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+                    0, (void *)mbufs_dst, nb_dq);
+
+                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+                /* Free any unsent packets. */
+                if (unlikely(nb_tx < nb_dq))
+                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&mbufs_dst[nb_tx],
+                        nb_dq - nb_tx);
+            }
+        }
+    }
+
+The Packet Copying Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to perform packet copy there is a user-defined function
+``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
+metadata from source packet to new mbuf, and then copying a data
+chunk of source packet. Both memory copies are done using
+``rte_memcpy()``:
+
+.. code-block:: c
+
+    static inline void
+    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+    {
+        /* Copy packet metadata */
+        rte_memcpy(&dst->rearm_data,
+            &src->rearm_data,
+            offsetof(struct rte_mbuf, cacheline1)
+                - offsetof(struct rte_mbuf, rearm_data));
+
+        /* Copy packet data */
+        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+            rte_pktmbuf_mtod(src, char *), src->data_len);
+    }
+
+The metadata in this example is copied from ``rearm_data`` member of
+``rte_mbuf`` struct up to ``cacheline1``.
+
+In order to understand why software packet copying is done as shown
+above please refer to the "Mbuf Library" section of the
+*DPDK Programmer's Guide*.
\ No newline at end of file
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
                     ` (5 preceding siblings ...)
  2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
@ 2019-09-13 18:45   ` Aaron Conole
  2019-09-16  9:42     ` Baran, MarcinX
  2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
  7 siblings, 1 reply; 76+ messages in thread
From: Aaron Conole @ 2019-09-13 18:45 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, bruce.richardson

Marcin Baran <marcinx.baran@intel.com> writes:

> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware
> copy, changes packets' MAC addresses (if enabled) and
> forwards them. The patch includes sample application as
> well as it's guide.

Please fix the dependencies:

https://travis-ci.com/ovsrobot/dpdk/jobs/234891633

examples/meson.build:89:4: ERROR: Problem encountered: Missing
dependency "pmd_ioat" for example "ioat"


> v2:
>  - change printing total stats to be deltas
>  - update documentation
>  - fix 1 thread/sw copy/multiple Rx queues packet dropping
>  - divide patch into several presenting functional change
>
> Marcin Baran (4):
>   examples/ioat: add software copy support
>   examples/ioat: add rawdev copy mode support
>   examples/ioat: add stats printing for each port
>   doc/guides/: provide IOAT sample app guide
>
> Pawel Modrak (2):
>   examples/ioat: create sample app on ioat driver usage
>   examples/ioat: add two threads configuration
>
>  doc/guides/sample_app_ug/index.rst |    1 +
>  doc/guides/sample_app_ug/intro.rst |    4 +
>  doc/guides/sample_app_ug/ioat.rst  |  764 +++++++++++++++++++++
>  examples/Makefile                  |    3 +
>  examples/ioat/Makefile             |   54 ++
>  examples/ioat/ioatfwd.c            | 1025 ++++++++++++++++++++++++++++
>  examples/ioat/meson.build          |   13 +
>  examples/meson.build               |    1 +
>  8 files changed, 1865 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
>  create mode 100644 examples/ioat/Makefile
>  create mode 100644 examples/ioat/ioatfwd.c
>  create mode 100644 examples/ioat/meson.build

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-13 18:45   ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage Aaron Conole
@ 2019-09-16  9:42     ` Baran, MarcinX
  2019-09-19  9:19       ` Aaron Conole
  0 siblings, 1 reply; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-16  9:42 UTC (permalink / raw)
  To: Aaron Conole; +Cc: dev, Richardson, Bruce



-----Original Message-----
From: Aaron Conole <aconole@redhat.com> 
Sent: Friday, September 13, 2019 8:45 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage

Marcin Baran <marcinx.baran@intel.com> writes:

> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware copy, changes 
> packets' MAC addresses (if enabled) and forwards them. The patch 
> includes sample application as well as it's guide.

Please fix the dependencies:

https://travis-ci.com/ovsrobot/dpdk/jobs/234891633

examples/meson.build:89:4: ERROR: Problem encountered: Missing dependency "pmd_ioat" for example "ioat"

[Marcin]: I will add check for dependency in example's  meson.build for v3:

	if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
		build = false
		subdir_done()
	endif

> v2:
>  - change printing total stats to be deltas
>  - update documentation
>  - fix 1 thread/sw copy/multiple Rx queues packet dropping
>  - divide patch into several presenting functional change
>
> Marcin Baran (4):
>   examples/ioat: add software copy support
>   examples/ioat: add rawdev copy mode support
>   examples/ioat: add stats printing for each port
>   doc/guides/: provide IOAT sample app guide
>
> Pawel Modrak (2):
>   examples/ioat: create sample app on ioat driver usage
>   examples/ioat: add two threads configuration
>
>  doc/guides/sample_app_ug/index.rst |    1 +
>  doc/guides/sample_app_ug/intro.rst |    4 +
>  doc/guides/sample_app_ug/ioat.rst  |  764 +++++++++++++++++++++
>  examples/Makefile                  |    3 +
>  examples/ioat/Makefile             |   54 ++
>  examples/ioat/ioatfwd.c            | 1025 ++++++++++++++++++++++++++++
>  examples/ioat/meson.build          |   13 +
>  examples/meson.build               |    1 +
>  8 files changed, 1865 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst  create mode 
> 100644 examples/ioat/Makefile  create mode 100644 
> examples/ioat/ioatfwd.c  create mode 100644 examples/ioat/meson.build

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v3 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
                     ` (6 preceding siblings ...)
  2019-09-13 18:45   ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage Aaron Conole
@ 2019-09-18  9:11   ` Marcin Baran
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 1/6] examples/ioat: create " Marcin Baran
                       ` (6 more replies)
  7 siblings, 7 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-18  9:11 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The patch includes sample application as
well as it's guide.

v3:
 - add check for meson build whether IOAT is supported

v2:
 - change printing total stats to be deltas
 - update documentation
 - fix 1 thread/sw copy/multiple Rx queues packet dropping
 - divide patch into several presenting functional change

Marcin Baran (4):
  examples/ioat: add software copy support
  examples/ioat: add rawdev copy mode support
  examples/ioat: add stats printing for each port
  doc/guides/: provide IOAT sample app guide

Pawel Modrak (2):
  examples/ioat: create sample app on ioat driver usage
  examples/ioat: add two threads configuration

 doc/guides/sample_app_ug/index.rst |    1 +
 doc/guides/sample_app_ug/intro.rst |    4 +
 doc/guides/sample_app_ug/ioat.rst  |  764 +++++++++++++++++++++
 examples/Makefile                  |    3 +
 examples/ioat/Makefile             |   54 ++
 examples/ioat/ioatfwd.c            | 1025 ++++++++++++++++++++++++++++
 examples/ioat/meson.build          |   18 +
 examples/meson.build               |    1 +
 8 files changed, 1870 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v3 1/6] examples/ioat: create sample app on ioat driver usage
  2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
@ 2019-09-18  9:11     ` Marcin Baran
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 2/6] examples/ioat: add software copy support Marcin Baran
                       ` (5 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-18  9:11 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The change covers ports initialization,
closing connection and argument parsing.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/Makefile         |   3 +
 examples/ioat/Makefile    |  54 +++++
 examples/ioat/ioatfwd.c   | 439 ++++++++++++++++++++++++++++++++++++++
 examples/ioat/meson.build |  18 ++
 examples/meson.build      |   1 +
 5 files changed, 515 insertions(+)
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

diff --git a/examples/Makefile b/examples/Makefile
index de11dd487..3cb313d7d 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation
 DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify
 DIRS-y += flow_filtering
 DIRS-y += helloworld
+ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y)
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat
+endif
 DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline
 ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
 DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
diff --git a/examples/ioat/Makefile b/examples/ioat/Makefile
new file mode 100644
index 000000000..2a4d1da2d
--- /dev/null
+++ b/examples/ioat/Makefile
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# binary name
+APP = ioatfwd
+
+# all source are stored in SRCS-y
+SRCS-y := ioatfwd.c
+
+# Build using pkg-config variables if possible
+ifeq ($(shell pkg-config --exists libdpdk && echo 0),0)
+
+all: shared
+.PHONY: shared static
+shared: build/$(APP)-shared
+	ln -sf $(APP)-shared build/$(APP)
+static: build/$(APP)-static
+	ln -sf $(APP)-static build/$(APP)
+
+PC_FILE := $(shell pkg-config --path libdpdk)
+CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
+LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
+
+build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
+
+build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
+
+build:
+	@mkdir -p $@
+
+.PHONY: clean
+clean:
+	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
+	test -d build && rmdir -p build || true
+
+else # Build using legacy build system
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, detect a build directory, by looking for a path with a .config
+RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config)))))
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+endif
diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
new file mode 100644
index 000000000..977ea6a61
--- /dev/null
+++ b/examples/ioat/ioatfwd.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#include <stdint.h>
+#include <getopt.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+#include <rte_ethdev.h>
+#include <rte_rawdev.h>
+#include <rte_ioat_rawdev.h>
+
+/* size of ring used for software copying between rx and tx. */
+#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
+#define MAX_PKT_BURST 32
+#define MEMPOOL_CACHE_SIZE 512
+#define MIN_POOL_SIZE 65536U
+#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
+#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
+#define CMD_LINE_OPT_PORTMASK "portmask"
+#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
+#define CMD_LINE_OPT_COPY_TYPE "copy-type"
+#define CMD_LINE_OPT_RING_SIZE "ring-size"
+
+/* configurable number of RX/TX ring descriptors */
+#define RX_DEFAULT_RINGSIZE 1024
+#define TX_DEFAULT_RINGSIZE 1024
+
+/* max number of RX queues per port */
+#define MAX_RX_QUEUES_COUNT 8
+
+struct rxtx_port_config {
+	/* common config */
+	uint16_t rxtx_port;
+	uint16_t nb_queues;
+	/* for software copy mode */
+	struct rte_ring *rx_to_tx_ring;
+	/* for IOAT rawdev copy mode */
+	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT];
+};
+
+struct rxtx_transmission_config {
+	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+	uint16_t nb_ports;
+	uint16_t nb_lcores;
+};
+
+typedef enum copy_mode_t {
+#define COPY_MODE_SW "sw"
+	COPY_MODE_SW_NUM,
+#define COPY_MODE_IOAT "rawdev"
+	COPY_MODE_IOAT_NUM,
+	COPY_MODE_INVALID_NUM,
+	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
+} copy_mode_t;
+
+/* mask of enabled ports */
+static uint32_t ioat_enabled_port_mask;
+
+/* number of RX queues per port */
+static uint16_t nb_queues = 1;
+
+/* MAC updating enabled by default. */
+static int mac_updating = 1;
+
+/* hardare copy mode enabled by default. */
+static copy_mode_t copy_mode = COPY_MODE_IOAT_NUM;
+
+/* size of IOAT rawdev ring for hardware copy mode or
+ * rte_ring for software copy mode
+ */
+static unsigned short ring_size = 2048;
+
+/* global transmission config */
+struct rxtx_transmission_config cfg;
+
+/* configurable number of RX/TX ring descriptors */
+static uint16_t nb_rxd = RX_DEFAULT_RINGSIZE;
+static uint16_t nb_txd = TX_DEFAULT_RINGSIZE;
+
+static volatile bool force_quit;
+
+/* ethernet addresses of ports */
+static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
+struct rte_mempool *ioat_pktmbuf_pool;
+
+/* Display usage */
+static void
+ioat_usage(const char *prgname)
+{
+	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
+		"  -p --portmask: hexadecimal bitmask of ports to configure\n"
+		"  -q NQ: number of RX queues per port (default is 1)\n"
+		"  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
+		"      When enabled:\n"
+		"       - The source MAC address is replaced by the TX port MAC address\n"
+		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
+		"  -c --copy-type CT: type of copy: sw|rawdev\n"
+		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
+	       prgname);
+}
+
+static int
+ioat_parse_portmask(const char *portmask)
+{
+	char *end = NULL;
+	unsigned long pm;
+
+	/* Parse hexadecimal string */
+	pm = strtoul(portmask, &end, 16);
+	if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+
+	return pm;
+}
+
+static copy_mode_t
+ioat_parse_copy_mode(const char *copy_mode)
+{
+	if (strcmp(copy_mode, COPY_MODE_SW) == 0)
+		return COPY_MODE_SW_NUM;
+	else if (strcmp(copy_mode, COPY_MODE_IOAT) == 0)
+		return COPY_MODE_IOAT_NUM;
+
+	return COPY_MODE_INVALID_NUM;
+}
+
+/* Parse the argument given in the command line of the application */
+static int
+ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
+{
+	static const char short_options[] =
+		"p:"  /* portmask */
+		"q:"  /* number of RX queues per port */
+		"c:"  /* copy type (sw|rawdev) */
+		"s:"  /* ring size */
+		;
+
+	static const struct option lgopts[] = {
+		{CMD_LINE_OPT_MAC_UPDATING, no_argument, &mac_updating, 1},
+		{CMD_LINE_OPT_NO_MAC_UPDATING, no_argument, &mac_updating, 0},
+		{CMD_LINE_OPT_PORTMASK, required_argument, NULL, 'p'},
+		{CMD_LINE_OPT_NB_QUEUE, required_argument, NULL, 'q'},
+		{CMD_LINE_OPT_COPY_TYPE, required_argument, NULL, 'c'},
+		{CMD_LINE_OPT_RING_SIZE, required_argument, NULL, 's'},
+		{NULL, 0, 0, 0}
+	};
+
+	const unsigned int default_port_mask = (1 << nb_ports) - 1;
+	int opt, ret;
+	char **argvopt;
+	int option_index;
+	char *prgname = argv[0];
+
+	ioat_enabled_port_mask = default_port_mask;
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, short_options,
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		/* portmask */
+		case 'p':
+			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
+			if (ioat_enabled_port_mask & ~default_port_mask ||
+					ioat_enabled_port_mask <= 0) {
+				printf("Invalid portmask, %s, suggest 0x%x\n",
+						optarg, default_port_mask);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'q':
+			nb_queues = atoi(optarg);
+			if (nb_queues == 0 || nb_queues > MAX_RX_QUEUES_COUNT) {
+				printf("Invalid RX queues number %s. Max %u\n",
+					optarg, MAX_RX_QUEUES_COUNT);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'c':
+			copy_mode = ioat_parse_copy_mode(optarg);
+			if (copy_mode == COPY_MODE_INVALID_NUM) {
+				printf("Invalid copy type. Use: sw, rawdev\n");
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 's':
+			ring_size = atoi(optarg);
+			if (ring_size == 0) {
+				printf("Invalid ring size, %s.\n", optarg);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		/* long options */
+		case 0:
+			break;
+
+		default:
+			ioat_usage(prgname);
+			return -1;
+		}
+	}
+
+	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
+	if (optind >= 0)
+		argv[optind-1] = prgname;
+
+	ret = optind-1;
+	optind = 1; /* reset getopt lib */
+	return ret;
+}
+
+/* check link status, return true if at least one port is up */
+static int
+check_link_status(uint32_t port_mask)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	int retval = 0;
+
+	printf("\nChecking link status\n");
+	RTE_ETH_FOREACH_DEV(portid) {
+		if ((port_mask & (1 << portid)) == 0)
+			continue;
+
+		memset(&link, 0, sizeof(link));
+		rte_eth_link_get(portid, &link);
+
+		/* Print link status */
+		if (link.link_status) {
+			printf(
+				"Port %d Link Up. Speed %u Mbps - %s\n",
+				portid, link.link_speed,
+				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+				("full-duplex") : ("half-duplex\n"));
+			retval = 1;
+		} else
+			printf("Port %d Link Down\n", portid);
+	}
+	return retval;
+}
+
+/*
+ * Initializes a given port using global settings and with the RX buffers
+ * coming from the mbuf_pool passed as a parameter.
+ */
+static inline void
+port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+{
+	/* configuring port to use RSS for multiple RX queues */
+	static const struct rte_eth_conf port_conf = {
+		.rxmode = {
+			.mq_mode        = ETH_MQ_RX_RSS,
+			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
+		},
+		.rx_adv_conf = {
+			.rss_conf = {
+				.rss_key = NULL,
+				.rss_hf = ETH_RSS_PROTO_MASK,
+			}
+		}
+	};
+
+	struct rte_eth_rxconf rxq_conf;
+	struct rte_eth_txconf txq_conf;
+	struct rte_eth_conf local_port_conf = port_conf;
+	struct rte_eth_dev_info dev_info;
+	int ret, i;
+
+	/* Skip ports that are not enabled */
+	if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+		printf("Skipping disabled port %u\n", portid);
+		return;
+	}
+
+	/* Init port */
+	printf("Initializing port %u... ", portid);
+	fflush(stdout);
+	rte_eth_dev_info_get(portid, &dev_info);
+	local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+		dev_info.flow_type_rss_offloads;
+	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+		local_port_conf.txmode.offloads |=
+			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Cannot configure device:"
+			" err=%d, port=%u\n", ret, portid);
+
+	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+						&nb_txd);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot adjust number of descriptors: err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+	/* Init RX queues */
+	rxq_conf = dev_info.default_rxconf;
+	rxq_conf.offloads = local_port_conf.rxmode.offloads;
+	for (i = 0; i < nb_queues; i++) {
+		ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+			rte_eth_dev_socket_id(portid), &rxq_conf,
+			mbuf_pool);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE,
+				"rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+				ret, portid, i);
+	}
+
+	/* Init one TX queue on each port */
+	txq_conf = dev_info.default_txconf;
+	txq_conf.offloads = local_port_conf.txmode.offloads;
+	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+			rte_eth_dev_socket_id(portid),
+			&txq_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
+			ret, portid);
+
+	/* Initialize TX buffers */
+	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+			rte_eth_dev_socket_id(portid));
+	if (tx_buffer[portid] == NULL)
+		rte_exit(EXIT_FAILURE,
+			"Cannot allocate buffer for tx on port %u\n",
+			portid);
+
+	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+	/* Start device */
+	ret = rte_eth_dev_start(portid);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_dev_start:err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_promiscuous_enable(portid);
+
+	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+			portid,
+			ioat_ports_eth_addr[portid].addr_bytes[0],
+			ioat_ports_eth_addr[portid].addr_bytes[1],
+			ioat_ports_eth_addr[portid].addr_bytes[2],
+			ioat_ports_eth_addr[portid].addr_bytes[3],
+			ioat_ports_eth_addr[portid].addr_bytes[4],
+			ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+	cfg.ports[cfg.nb_ports].rxtx_port = portid;
+	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\n\nSignal %d received, preparing to exit...\n",
+			signum);
+		force_quit = true;
+	}
+}
+
+int
+main(int argc, char **argv)
+{
+	int ret;
+	uint16_t nb_ports, portid;
+	uint32_t i;
+	unsigned int nb_mbufs;
+
+	/* Init EAL */
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+	argc -= ret;
+	argv += ret;
+
+	force_quit = false;
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	nb_ports = rte_eth_dev_count_avail();
+	if (nb_ports == 0)
+		rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
+
+	/* Parse application arguments (after the EAL ones) */
+	ret = ioat_parse_args(argc, argv, nb_ports);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid IOAT arguments\n");
+
+	nb_mbufs = RTE_MAX(nb_ports * (nb_queues * (nb_rxd + nb_txd +
+		4 * MAX_PKT_BURST) + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+		MIN_POOL_SIZE);
+
+	/* Create the mbuf pool */
+	ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+		MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+		rte_socket_id());
+	if (ioat_pktmbuf_pool == NULL)
+		rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+	/* Initialise each port */
+	cfg.nb_ports = 0;
+	RTE_ETH_FOREACH_DEV(portid)
+		port_init(portid, ioat_pktmbuf_pool, nb_queues);
+
+	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
+		sleep(1);
+
+	/* Check if there is enough lcores for all ports. */
+	cfg.nb_lcores = rte_lcore_count() - 1;
+	if (cfg.nb_lcores < 1)
+		rte_exit(EXIT_FAILURE,
+			"There should be at least one slave lcore.\n");
+	for (i = 0; i < cfg.nb_ports; i++) {
+		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
+		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
+		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+	}
+
+	printf("Bye...\n");
+	return 0;
+}
diff --git a/examples/ioat/meson.build b/examples/ioat/meson.build
new file mode 100644
index 000000000..d55353ee3
--- /dev/null
+++ b/examples/ioat/meson.build
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
+	build = false
+	subdir_done()
+endif
+
+deps += ['pmd_ioat']
+
+sources = files(
+	'ioatfwd.c'
+)
diff --git a/examples/meson.build b/examples/meson.build
index a046b74ad..c2e18b59b 100644
--- a/examples/meson.build
+++ b/examples/meson.build
@@ -16,6 +16,7 @@ all_examples = [
 	'eventdev_pipeline', 'exception_path',
 	'fips_validation', 'flow_classify',
 	'flow_filtering', 'helloworld',
+	'ioat',
 	'ip_fragmentation', 'ip_pipeline',
 	'ip_reassembly', 'ipsec-secgw',
 	'ipv4_multicast', 'kni',
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v3 2/6] examples/ioat: add software copy support
  2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 1/6] examples/ioat: create " Marcin Baran
@ 2019-09-18  9:11     ` Marcin Baran
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
                       ` (4 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-18  9:11 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 188 insertions(+), 23 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 977ea6a61..3a092c6cf 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -13,7 +13,7 @@
 #include <rte_rawdev.h>
 #include <rte_ioat_rawdev.h>
 
-/* size of ring used for software copying between rx and tx. */
+ /* size of ring used for software copying between rx and tx. */
 #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
 #define MAX_PKT_BURST 32
 #define MEMPOOL_CACHE_SIZE 512
@@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+static void
+update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
+{
+	struct rte_ether_hdr *eth;
+	void *tmp;
+
+	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
+
+	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
+	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
+	 */
+	tmp = &eth->d_addr.addr_bytes[0];
+	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
+
+	/* src addr */
+	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], &eth->s_addr);
+}
+
+static inline void
+pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+{
+	/* Copy packet metadata */
+	rte_memcpy(&dst->rearm_data,
+		&src->rearm_data,
+		offsetof(struct rte_mbuf, cacheline1)
+		- offsetof(struct rte_mbuf, rearm_data));
+
+	/* Copy packet data */
+	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+		rte_pktmbuf_mtod(src, char *), src->data_len);
+}
+
+/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+static void
+ioat_rx_port(struct rxtx_port_config *rx_config)
+{
+	uint32_t nb_rx, nb_enq, i, j;
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+
+	for (i = 0; i < rx_config->nb_queues; i++) {
+
+		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+			pkts_burst, MAX_PKT_BURST);
+
+		if (nb_rx == 0)
+			continue;
+
+		/* Perform packet software copy, free source packets */
+		int ret;
+		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+			(void *)pkts_burst_copy, nb_rx);
+
+		if (unlikely(ret < 0))
+			rte_exit(EXIT_FAILURE,
+				"Unable to allocate memory.\n");
+
+		for (j = 0; j < nb_rx; j++)
+			pktmbuf_sw_copy(pkts_burst[j],
+				pkts_burst_copy[j]);
+
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)pkts_burst, nb_rx);
+
+		nb_enq = rte_ring_enqueue_burst(
+			rx_config->rx_to_tx_ring,
+			(void *)pkts_burst_copy, nb_rx, NULL);
+
+		/* Free any not enqueued packets. */
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&pkts_burst_copy[nb_enq],
+			nb_rx - nb_enq);
+	}
+}
+
+/* Transmit packets from IOAT rawdev/rte_ring for one port. */
+static void
+ioat_tx_port(struct rxtx_port_config *tx_config)
+{
+	uint32_t i, nb_dq = 0;
+	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+	/* Deque the mbufs from rx_to_tx_ring. */
+	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+	if (nb_dq == 0)
+		return;
+
+	/* Update macs if enabled */
+	if (mac_updating) {
+		for (i = 0; i < nb_dq; i++)
+			update_mac_addrs(mbufs_dst[i],
+				tx_config->rxtx_port);
+	}
+
+	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+		0, (void *)mbufs_dst, nb_dq);
+
+	/* Free any unsent packets. */
+	if (unlikely(nb_tx < nb_dq))
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+		(void *)&mbufs_dst[nb_tx],
+			nb_dq - nb_tx);
+}
+
+/* Main rx and tx loop if only one slave lcore available */
+static void
+rxtx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
+		" lcore %u\n", rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++) {
+			ioat_rx_port(&cfg.ports[i]);
+			ioat_tx_port(&cfg.ports[i]);
+		}
+}
+
+static void start_forwarding_cores(void)
+{
+	uint32_t lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+		__func__, rte_lcore_id());
+
+	lcore_id = rte_get_next_lcore(lcore_id, true, true);
+	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+		NULL, lcore_id);
+}
+
 /* Display usage */
 static void
 ioat_usage(const char *prgname)
@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)
 		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
 		"  -c --copy-type CT: type of copy: sw|rawdev\n"
 		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
-	       prgname);
+		prgname);
 }
 
 static int
@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 	argvopt = argv;
 
 	while ((opt = getopt_long(argc, argvopt, short_options,
-				  lgopts, &option_index)) != EOF) {
+		lgopts, &option_index)) != EOF) {
 
 		switch (opt) {
-		/* portmask */
+			/* portmask */
 		case 'p':
 			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
 			if (ioat_enabled_port_mask & ~default_port_mask ||
-					ioat_enabled_port_mask <= 0) {
+				ioat_enabled_port_mask <= 0) {
 				printf("Invalid portmask, %s, suggest 0x%x\n",
-						optarg, default_port_mask);
+					optarg, default_port_mask);
 				ioat_usage(prgname);
 				return -1;
 			}
@@ -204,7 +340,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 			}
 			break;
 
-		/* long options */
+			/* long options */
 		case 0:
 			break;
 
@@ -216,9 +352,9 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 
 	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
 	if (optind >= 0)
-		argv[optind-1] = prgname;
+		argv[optind - 1] = prgname;
 
-	ret = optind-1;
+	ret = optind - 1;
 	optind = 1; /* reset getopt lib */
 	return ret;
 }
@@ -253,6 +389,26 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+assign_rings(void)
+{
+	uint32_t i;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		char ring_name[RTE_RING_NAMESIZE];
+
+		snprintf(ring_name, sizeof(ring_name), "rx_to_tx_ring_%u", i);
+		/* Create ring for inter core communication */
+		cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+			ring_name, ring_size,
+			rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
+
+		if (cfg.ports[i].rx_to_tx_ring == NULL)
+			rte_exit(EXIT_FAILURE, "Ring create failed: %s\n",
+				rte_strerror(rte_errno));
+	}
+}
+
 /*
  * Initializes a given port using global settings and with the RX buffers
  * coming from the mbuf_pool passed as a parameter.
@@ -263,7 +419,7 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	/* configuring port to use RSS for multiple RX queues */
 	static const struct rte_eth_conf port_conf = {
 		.rxmode = {
-			.mq_mode        = ETH_MQ_RX_RSS,
+			.mq_mode = ETH_MQ_RX_RSS,
 			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
 		},
 		.rx_adv_conf = {
@@ -294,14 +450,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 		dev_info.flow_type_rss_offloads;
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
 		local_port_conf.txmode.offloads |=
-			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+		DEV_TX_OFFLOAD_MBUF_FAST_FREE;
 	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE, "Cannot configure device:"
 			" err=%d, port=%u\n", ret, portid);
 
 	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
-						&nb_txd);
+		&nb_txd);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE,
 			"Cannot adjust number of descriptors: err=%d, port=%u\n",
@@ -326,8 +482,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	txq_conf = dev_info.default_txconf;
 	txq_conf.offloads = local_port_conf.txmode.offloads;
 	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
-			rte_eth_dev_socket_id(portid),
-			&txq_conf);
+		rte_eth_dev_socket_id(portid),
+		&txq_conf);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE,
 			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
@@ -335,8 +491,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	/* Initialize TX buffers */
 	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
-			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
-			rte_eth_dev_socket_id(portid));
+		RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+		rte_eth_dev_socket_id(portid));
 	if (tx_buffer[portid] == NULL)
 		rte_exit(EXIT_FAILURE,
 			"Cannot allocate buffer for tx on port %u\n",
@@ -354,13 +510,13 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	rte_eth_promiscuous_enable(portid);
 
 	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
-			portid,
-			ioat_ports_eth_addr[portid].addr_bytes[0],
-			ioat_ports_eth_addr[portid].addr_bytes[1],
-			ioat_ports_eth_addr[portid].addr_bytes[2],
-			ioat_ports_eth_addr[portid].addr_bytes[3],
-			ioat_ports_eth_addr[portid].addr_bytes[4],
-			ioat_ports_eth_addr[portid].addr_bytes[5]);
+		portid,
+		ioat_ports_eth_addr[portid].addr_bytes[0],
+		ioat_ports_eth_addr[portid].addr_bytes[1],
+		ioat_ports_eth_addr[portid].addr_bytes[2],
+		ioat_ports_eth_addr[portid].addr_bytes[3],
+		ioat_ports_eth_addr[portid].addr_bytes[4],
+		ioat_ports_eth_addr[portid].addr_bytes[5]);
 
 	cfg.ports[cfg.nb_ports].rxtx_port = portid;
 	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
@@ -428,10 +584,19 @@ main(int argc, char **argv)
 	if (cfg.nb_lcores < 1)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
+
+	assign_rings();
+
+	start_forwarding_cores();
+
+	/* force_quit is true when we get here */
+	rte_eal_mp_wait_lcore();
+
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v3 3/6] examples/ioat: add rawdev copy mode support
  2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 1/6] examples/ioat: create " Marcin Baran
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 2/6] examples/ioat: add software copy support Marcin Baran
@ 2019-09-18  9:11     ` Marcin Baran
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 4/6] examples/ioat: add two threads configuration Marcin Baran
                       ` (3 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-18  9:11 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++--------
 1 file changed, 189 insertions(+), 47 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 3a092c6cf..c66ce7e49 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
 		rte_pktmbuf_mtod(src, char *), src->data_len);
 }
 
+static uint32_t
+ioat_enqueue_packets(struct rte_mbuf **pkts,
+	uint32_t nb_rx, uint16_t dev_id)
+{
+	int ret;
+	uint32_t i;
+	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+		&pkts[0]->rearm_data);
+
+	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+		(void *)pkts_copy, nb_rx);
+
+	if (unlikely(ret < 0))
+		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+	for (i = 0; i < nb_rx; i++) {
+		/* Perform data copy */
+		ret = rte_ioat_enqueue_copy(dev_id,
+			pkts[i]->buf_iova
+			- addr_offset,
+			pkts_copy[i]->buf_iova
+			- addr_offset,
+			rte_pktmbuf_data_len(pkts[i])
+			+ addr_offset,
+			(uintptr_t)pkts[i],
+			(uintptr_t)pkts_copy[i],
+			0 /* nofence */);
+
+		if (ret != 1)
+			break;
+	}
+
+	ret = i;
+	/* Free any not enqueued packets. */
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+		nb_rx - i);
+
+
+	return ret;
+}
+
 /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
 static void
 ioat_rx_port(struct rxtx_port_config *rx_config)
@@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
-		/* Perform packet software copy, free source packets */
-		int ret;
-		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
-
-		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
-			(void *)pkts_burst_copy, nb_rx);
-
-		if (unlikely(ret < 0))
-			rte_exit(EXIT_FAILURE,
-				"Unable to allocate memory.\n");
-
-		for (j = 0; j < nb_rx; j++)
-			pktmbuf_sw_copy(pkts_burst[j],
-				pkts_burst_copy[j]);
-
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)pkts_burst, nb_rx);
-
-		nb_enq = rte_ring_enqueue_burst(
-			rx_config->rx_to_tx_ring,
-			(void *)pkts_burst_copy, nb_rx, NULL);
-
-		/* Free any not enqueued packets. */
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)&pkts_burst_copy[nb_enq],
-			nb_rx - nb_enq);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			/* Perform packet hardware copy */
+			nb_enq = ioat_enqueue_packets(pkts_burst,
+				nb_rx, rx_config->ioat_ids[i]);
+			if (nb_enq > 0)
+				rte_ioat_do_copies(rx_config->ioat_ids[i]);
+		} else {
+			/* Perform packet software copy, free source packets */
+			int ret;
+			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst_copy, nb_rx);
+
+			if (unlikely(ret < 0))
+				rte_exit(EXIT_FAILURE,
+					"Unable to allocate memory.\n");
+
+			for (j = 0; j < nb_rx; j++)
+				pktmbuf_sw_copy(pkts_burst[j],
+					pkts_burst_copy[j]);
+
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst, nb_rx);
+
+			nb_enq = rte_ring_enqueue_burst(
+				rx_config->rx_to_tx_ring,
+				(void *)pkts_burst_copy, nb_rx, NULL);
+
+			/* Free any not enqueued packets. */
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&pkts_burst_copy[nb_enq],
+				nb_rx - nb_enq);
+		}
 	}
 }
 
@@ -169,31 +221,64 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 static void
 ioat_tx_port(struct rxtx_port_config *tx_config)
 {
-	uint32_t i, nb_dq = 0;
+	uint32_t i, j, nb_dq = 0;
+	struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
-	/* Deque the mbufs from rx_to_tx_ring. */
-	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
-		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		/* Deque the mbufs from IOAT device. */
+		for (i = 0; i < tx_config->nb_queues; i++) {
+			nb_dq = rte_ioat_completed_copies(
+				tx_config->ioat_ids[i], MAX_PKT_BURST,
+				(void *)mbufs_src, (void *)mbufs_dst);
 
-	if (nb_dq == 0)
-		return;
+			if (nb_dq == 0)
+				break;
 
-	/* Update macs if enabled */
-	if (mac_updating) {
-		for (i = 0; i < nb_dq; i++)
-			update_mac_addrs(mbufs_dst[i],
-				tx_config->rxtx_port);
-	}
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)mbufs_src, nb_dq);
 
-	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
-		0, (void *)mbufs_dst, nb_dq);
+			/* Update macs if enabled */
+			if (mac_updating) {
+				for (j = 0; j < nb_dq; j++)
+					update_mac_addrs(mbufs_dst[j],
+						tx_config->rxtx_port);
+			}
+
+			const uint16_t nb_tx = rte_eth_tx_burst(
+				tx_config->rxtx_port, 0,
+				(void *)mbufs_dst, nb_dq);
 
-	/* Free any unsent packets. */
-	if (unlikely(nb_tx < nb_dq))
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-		(void *)&mbufs_dst[nb_tx],
-			nb_dq - nb_tx);
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_dq))
+				rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&mbufs_dst[nb_tx],
+					nb_dq - nb_tx);
+		}
+	} else {
+		/* Deque the mbufs from rx_to_tx_ring. */
+		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+		if (nb_dq == 0)
+			return;
+
+		/* Update macs if enabled */
+		if (mac_updating) {
+			for (i = 0; i < nb_dq; i++)
+				update_mac_addrs(mbufs_dst[i],
+					tx_config->rxtx_port);
+		}
+
+		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+			0, (void *)mbufs_dst, nb_dq);
+
+		/* Free any unsent packets. */
+		if (unlikely(nb_tx < nb_dq))
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&mbufs_dst[nb_tx],
+				nb_dq - nb_tx);
+	}
 }
 
 /* Main rx and tx loop if only one slave lcore available */
@@ -389,6 +474,52 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+configure_rawdev_queue(uint32_t dev_id)
+{
+	struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
+	struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+	if (rte_rawdev_configure(dev_id, &info) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_configure()\n");
+	}
+	if (rte_rawdev_start(dev_id) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_start()\n");
+	}
+}
+
+static void
+assign_rawdevs(void)
+{
+	uint16_t nb_rawdev = 0, rdev_id = 0;
+	uint32_t i, j;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+			struct rte_rawdev_info rdev_info = { 0 };
+
+			do {
+				if (rdev_id == rte_rawdev_count())
+					goto end;
+				rte_rawdev_info_get(rdev_id++, &rdev_info);
+			} while (strcmp(rdev_info.driver_name,
+				IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+			cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+			configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+			++nb_rawdev;
+		}
+	}
+end:
+	if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+		rte_exit(EXIT_FAILURE,
+			"Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+			nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+	RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+}
+
 static void
 assign_rings(void)
 {
@@ -585,18 +716,29 @@ main(int argc, char **argv)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
 
-	assign_rings();
+	if (copy_mode == COPY_MODE_IOAT_NUM)
+		assign_rawdevs();
+	else /* copy_mode == COPY_MODE_SW_NUM */
+		assign_rings();
 
 	start_forwarding_cores();
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
 
+	uint32_t j;
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
-		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+				printf("Stopping rawdev %d\n",
+					cfg.ports[i].ioat_ids[j]);
+				rte_rawdev_stop(cfg.ports[i].ioat_ids[j]);
+			}
+		} else /* copy_mode == COPY_MODE_SW_NUM */
+			rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v3 4/6] examples/ioat: add two threads configuration
  2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
                       ` (2 preceding siblings ...)
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
@ 2019-09-18  9:11     ` Marcin Baran
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 5/6] examples/ioat: add stats printing for each port Marcin Baran
                       ` (2 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-18  9:11 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++-----------
 1 file changed, 67 insertions(+), 25 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index c66ce7e49..4c51db6bd 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,
 	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
 		nb_rx - i);
 
-
 	return ret;
 }
 
@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
 	if (copy_mode == COPY_MODE_IOAT_NUM) {
-		/* Deque the mbufs from IOAT device. */
 		for (i = 0; i < tx_config->nb_queues; i++) {
+			/* Deque the mbufs from IOAT device. */
 			nb_dq = rte_ioat_completed_copies(
 				tx_config->ioat_ids[i], MAX_PKT_BURST,
 				(void *)mbufs_src, (void *)mbufs_dst);
@@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 					nb_dq - nb_tx);
 		}
 	} else {
-		/* Deque the mbufs from rx_to_tx_ring. */
-		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
-			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
-
-		if (nb_dq == 0)
-			return;
-
-		/* Update macs if enabled */
-		if (mac_updating) {
-			for (i = 0; i < nb_dq; i++)
-				update_mac_addrs(mbufs_dst[i],
-					tx_config->rxtx_port);
-		}
+		for (i = 0; i < tx_config->nb_queues; i++) {
+			/* Deque the mbufs from IOAT device. */
+			nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+				(void *)mbufs_dst, MAX_PKT_BURST, NULL);
 
-		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
-			0, (void *)mbufs_dst, nb_dq);
+			if (nb_dq == 0)
+				return;
 
-		/* Free any unsent packets. */
-		if (unlikely(nb_tx < nb_dq))
-			rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)&mbufs_dst[nb_tx],
-				nb_dq - nb_tx);
+			/* Update macs if enabled */
+			if (mac_updating) {
+				for (j = 0; j < nb_dq; j++)
+					update_mac_addrs(mbufs_dst[j],
+						tx_config->rxtx_port);
+			}
+
+			const uint16_t nb_tx =
+				rte_eth_tx_burst(tx_config->rxtx_port, 0,
+					(void *)mbufs_dst, nb_dq);
+
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_dq))
+				rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&mbufs_dst[nb_tx],
+					nb_dq - nb_tx);
+		}
 	}
 }
 
+/* Main rx processing loop for IOAT rawdev. */
+static void
+rx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_rx_port(&cfg.ports[i]);
+}
+
+/* Main tx processing loop for hardware copy. */
+static void
+tx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main tx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_tx_port(&cfg.ports[i]);
+}
+
 /* Main rx and tx loop if only one slave lcore available */
 static void
 rxtx_main_loop(void)
@@ -305,9 +337,19 @@ static void start_forwarding_cores(void)
 	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
 		__func__, rte_lcore_id());
 
-	lcore_id = rte_get_next_lcore(lcore_id, true, true);
-	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
-		NULL, lcore_id);
+	if (cfg.nb_lcores == 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+			NULL, lcore_id);
+	} else if (cfg.nb_lcores > 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+			NULL, lcore_id);
+
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+			lcore_id);
+	}
 }
 
 /* Display usage */
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v3 5/6] examples/ioat: add stats printing for each port
  2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
                       ` (3 preceding siblings ...)
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 4/6] examples/ioat: add two threads configuration Marcin Baran
@ 2019-09-18  9:11     ` Marcin Baran
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
  2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-18  9:11 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added printing stats of ports each second.
The stats printing is done using master core.
The information provided informs about packets
received, dropped and send as well as statistics
of rawdev devices.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 241 insertions(+), 4 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 4c51db6bd..c6b994832 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -48,10 +48,27 @@ struct rxtx_transmission_config {
 	uint16_t nb_lcores;
 };
 
+/* per-port statistics struct */
+struct ioat_port_statistics {
+	uint64_t rx[RTE_MAX_ETHPORTS];
+	uint64_t tx[RTE_MAX_ETHPORTS];
+	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
+	uint64_t copy_dropped[RTE_MAX_ETHPORTS];
+};
+struct ioat_port_statistics port_statistics;
+
+struct total_statistics {
+	uint64_t total_packets_dropped;
+	uint64_t total_packets_tx;
+	uint64_t total_packets_rx;
+	uint64_t total_successful_enqueues;
+	uint64_t total_failed_enqueues;
+};
+
 typedef enum copy_mode_t {
 #define COPY_MODE_SW "sw"
 	COPY_MODE_SW_NUM,
-#define COPY_MODE_IOAT "rawdev"
+#define COPY_MODE_IOAT "hw"
 	COPY_MODE_IOAT_NUM,
 	COPY_MODE_INVALID_NUM,
 	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+/* Print out statistics for one port. */
+static void
+print_port_stats(uint16_t port_id)
+{
+	printf("\nStatistics for port %u ------------------------------"
+		"\nPackets sent: %34"PRIu64
+		"\nPackets received: %30"PRIu64
+		"\nPackets dropped on tx: %25"PRIu64
+		"\nPackets dropped on copy: %23"PRIu64,
+		port_id,
+		port_statistics.tx[port_id],
+		port_statistics.rx[port_id],
+		port_statistics.tx_dropped[port_id],
+		port_statistics.copy_dropped[port_id]);
+}
+
+/* Print out statistics for one IOAT rawdev device. */
+static void
+print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
+	unsigned int *ids_xstats, uint16_t nb_xstats,
+	struct rte_rawdev_xstats_name *names_xstats)
+{
+	uint16_t i;
+
+	printf("\nIOAT channel %u", dev_id);
+	for (i = 0; i < nb_xstats; i++)
+		printf("\n\t %s: %*"PRIu64,
+			names_xstats[ids_xstats[i]].name,
+			(int)(37 - strlen(names_xstats[ids_xstats[i]].name)),
+			xstats[i]);
+}
+
+static void
+print_total_stats(struct total_statistics *ts)
+{
+	printf("\nAggregate statistics ==============================="
+		"\nTotal packets Tx: %24"PRIu64" [pps]"
+		"\nTotal packets Rx: %24"PRIu64" [pps]"
+		"\nTotal packets dropped: %19"PRIu64" [pps]",
+		ts->total_packets_tx,
+		ts->total_packets_rx,
+		ts->total_packets_dropped);
+
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		printf("\nTotal IOAT successful enqueues: %8"PRIu64" [enq/s]"
+			"\nTotal IOAT failed enqueues: %12"PRIu64" [enq/s]",
+			ts->total_successful_enqueues,
+			ts->total_failed_enqueues);
+	}
+
+	printf("\n====================================================\n");
+}
+
+/* Print out statistics on packets dropped. */
+static void
+print_stats(char *prgname)
+{
+	struct total_statistics ts, delta_ts;
+	uint32_t i, port_id, dev_id;
+	struct rte_rawdev_xstats_name *names_xstats;
+	uint64_t *xstats;
+	unsigned int *ids_xstats, nb_xstats;
+	char status_string[120]; /* to print at the top of the output */
+	int status_strlen;
+
+
+	const char clr[] = { 27, '[', '2', 'J', '\0' };
+	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+	status_strlen = snprintf(status_string, sizeof(status_string),
+		"%s, ", prgname);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Worker Threads = %d, ",
+		rte_lcore_count() > 2 ? 2 : 1);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
+		COPY_MODE_SW : COPY_MODE_IOAT);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Updating MAC = %s, ", mac_updating ?
+		"enabled" : "disabled");
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Rx Queues = %d, ", nb_queues);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Ring Size = %d\n", ring_size);
+
+	/* Allocate memory for xstats names and values */
+	nb_xstats = rte_rawdev_xstats_names_get(
+		cfg.ports[0].ioat_ids[0], NULL, 0);
+
+	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
+	if (names_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat names memory\n");
+	}
+	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
+		names_xstats, nb_xstats);
+
+	ids_xstats = malloc(sizeof(*ids_xstats) * 2);
+	if (ids_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat ids_xstats memory\n");
+	}
+
+	xstats = malloc(sizeof(*xstats) * 2);
+	if (xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat memory\n");
+	}
+
+	/* Get failed/successful enqueues stats index */
+	ids_xstats[0] = ids_xstats[1] = nb_xstats;
+	for (i = 0; i < nb_xstats; i++) {
+		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
+			ids_xstats[0] = i;
+		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
+			ids_xstats[1] = i;
+		if (ids_xstats[0] < nb_xstats && ids_xstats[1] < nb_xstats)
+			break;
+	}
+	if (ids_xstats[0] == nb_xstats || ids_xstats[1] == nb_xstats) {
+		rte_exit(EXIT_FAILURE,
+			"Error getting failed/successful enqueues stats index\n");
+	}
+
+	memset(&ts, 0, sizeof(struct total_statistics));
+
+	while (!force_quit) {
+		/* Sleep for 1 second each round - init sleep allows reading
+		 * messages from app startup.
+		 */
+		sleep(1);
+
+		/* Clear screen and move to top left */
+		printf("%s%s", clr, topLeft);
+
+		memset(&delta_ts, 0, sizeof(struct total_statistics));
+
+		printf("%s", status_string);
+
+		for (i = 0; i < cfg.nb_ports; i++) {
+			port_id = cfg.ports[i].rxtx_port;
+			print_port_stats(port_id);
+
+			delta_ts.total_packets_dropped +=
+				port_statistics.tx_dropped[port_id]
+				+ port_statistics.copy_dropped[port_id];
+			delta_ts.total_packets_tx +=
+				port_statistics.tx[port_id];
+			delta_ts.total_packets_rx +=
+				port_statistics.rx[port_id];
+
+			if (copy_mode == COPY_MODE_IOAT_NUM) {
+				uint32_t j;
+
+				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+					dev_id = cfg.ports[i].ioat_ids[j];
+					rte_rawdev_xstats_get(dev_id,
+						ids_xstats, xstats, 2);
+
+					print_rawdev_stats(dev_id, xstats,
+						ids_xstats, 2, names_xstats);
+
+					delta_ts.total_failed_enqueues +=
+						xstats[ids_xstats[0]];
+					delta_ts.total_successful_enqueues +=
+						xstats[ids_xstats[1]];
+				}
+			}
+		}
+
+		delta_ts.total_packets_tx -= ts.total_packets_tx;
+		delta_ts.total_packets_rx -= ts.total_packets_rx;
+		delta_ts.total_packets_dropped -= ts.total_packets_dropped;
+		delta_ts.total_failed_enqueues -= ts.total_failed_enqueues;
+		delta_ts.total_successful_enqueues -=
+			ts.total_successful_enqueues;
+
+		printf("\n");
+		print_total_stats(&delta_ts);
+
+		ts.total_packets_tx += delta_ts.total_packets_tx;
+		ts.total_packets_rx += delta_ts.total_packets_rx;
+		ts.total_packets_dropped += delta_ts.total_packets_dropped;
+		ts.total_failed_enqueues += delta_ts.total_failed_enqueues;
+		ts.total_successful_enqueues +=
+			delta_ts.total_successful_enqueues;
+	}
+
+	free(names_xstats);
+	free(xstats);
+	free(ids_xstats);
+}
+
 static void
 update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
 {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
+		port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Perform packet hardware copy */
 			nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 				(void *)&pkts_burst_copy[nb_enq],
 				nb_rx - nb_enq);
 		}
+
+		port_statistics.copy_dropped[rx_config->rxtx_port] +=
+			(nb_rx - nb_enq);
 	}
 }
 
@@ -248,6 +468,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 				tx_config->rxtx_port, 0,
 				(void *)mbufs_dst, nb_dq);
 
+			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 			/* Free any unsent packets. */
 			if (unlikely(nb_tx < nb_dq))
 				rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -274,6 +496,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 				rte_eth_tx_burst(tx_config->rxtx_port, 0,
 					(void *)mbufs_dst, nb_dq);
 
+			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 			/* Free any unsent packets. */
 			if (unlikely(nb_tx < nb_dq))
 				rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)
 		"      When enabled:\n"
 		"       - The source MAC address is replaced by the TX port MAC address\n"
 		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
-		"  -c --copy-type CT: type of copy: sw|rawdev\n"
+		"  -c --copy-type CT: type of copy: sw|hw\n"
 		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
 		prgname);
 }
@@ -400,7 +624,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 	static const char short_options[] =
 		"p:"  /* portmask */
 		"q:"  /* number of RX queues per port */
-		"c:"  /* copy type (sw|rawdev) */
+		"c:"  /* copy type (sw|hw) */
 		"s:"  /* ring size */
 		;
 
@@ -452,7 +676,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 		case 'c':
 			copy_mode = ioat_parse_copy_mode(optarg);
 			if (copy_mode == COPY_MODE_INVALID_NUM) {
-				printf("Invalid copy type. Use: sw, rawdev\n");
+				printf("Invalid copy type. Use: sw, hw\n");
 				ioat_usage(prgname);
 				return -1;
 			}
@@ -673,6 +897,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
 
+	ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+		rte_eth_tx_buffer_count_callback,
+		&port_statistics.tx_dropped[portid]);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot set error callback for tx buffer on port %u\n",
+			portid);
+
 	/* Start device */
 	ret = rte_eth_dev_start(portid);
 	if (ret < 0)
@@ -749,6 +981,9 @@ main(int argc, char **argv)
 	RTE_ETH_FOREACH_DEV(portid)
 		port_init(portid, ioat_pktmbuf_pool, nb_queues);
 
+	/* Initialize port xstats */
+	memset(&port_statistics, 0, sizeof(port_statistics));
+
 	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
 		sleep(1);
 
@@ -764,6 +999,8 @@ main(int argc, char **argv)
 		assign_rings();
 
 	start_forwarding_cores();
+	/* master core prints stats while other cores forward */
+	print_stats(argv[0]);
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v3 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
                       ` (4 preceding siblings ...)
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 5/6] examples/ioat: add stats printing for each port Marcin Baran
@ 2019-09-18  9:11     ` Marcin Baran
  2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-18  9:11 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran

Added guide for IOAT sample app usage and
code description.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 doc/guides/sample_app_ug/index.rst |   1 +
 doc/guides/sample_app_ug/intro.rst |   4 +
 doc/guides/sample_app_ug/ioat.rst  | 764 +++++++++++++++++++++++++++++
 3 files changed, 769 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst

diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index f23f8f59e..a6a1d9e7a 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -23,6 +23,7 @@ Sample Applications User Guides
     ip_reassembly
     kernel_nic_interface
     keep_alive
+    ioat
     l2_forward_crypto
     l2_forward_job_stats
     l2_forward_real_virtual
diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
index 90704194a..74462312f 100644
--- a/doc/guides/sample_app_ug/intro.rst
+++ b/doc/guides/sample_app_ug/intro.rst
@@ -91,6 +91,10 @@ examples are highlighted below.
   forwarding, or ``l3fwd`` application does forwarding based on Internet
   Protocol, IPv4 or IPv6 like a simple router.
 
+* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
+  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
+  copying packets between two threads.
+
 * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
   demonstrates how to distribute packets arriving on an Rx port to different
   cores for processing and transmission.
diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
new file mode 100644
index 000000000..69621673b
--- /dev/null
+++ b/doc/guides/sample_app_ug/ioat.rst
@@ -0,0 +1,764 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Intel Corporation.
+
+Sample Application of packet copying using Intel\ |reg| QuickData Technology
+============================================================================
+
+Overview
+--------
+
+This sample is intended as a demonstration of the basic components of a DPDK
+forwarding application and example of how to use IOAT driver API to make
+packets copies.
+
+Also while forwarding, the MAC addresses are affected as follows:
+
+*   The source MAC address is replaced by the TX port MAC address
+
+*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
+
+This application can be used to compare performance of using software packet
+copy with copy done using a DMA device for different sizes of packets.
+The example will print out statistics each second. The stats shows
+received/send packets and packets dropped or failed to copy.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``ioat`` sub-directory.
+
+
+Running the Application
+-----------------------
+
+In order to run the hardware copy application, the copying device
+needs to be bound to user-space IO driver.
+
+Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology*
+guide for information on using the driver.
+
+The application requires a number of command line options:
+
+.. code-block:: console
+
+    ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>]
+        [--[no-]mac-updating]
+
+where,
+
+*   p MASK: A hexadecimal bitmask of the ports to configure
+
+*   q NQ: Number of Rx queues used per port equivalent to CBDMA channels
+    per port
+
+*   c CT: Performed packet copy type: software (sw) or hardware using
+    DMA (hw)
+
+*   s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for
+    software copy mode
+
+*   --[no-]mac-updating: Whether MAC address of packets should be changed
+    or not
+
+The application can be launched in various configurations depending on
+provided parameters. Each port can use up to 2 lcores: one of them receives
+incoming traffic and makes a copy of each packet. The second lcore then
+updates MAC address and sends the copy. If one lcore per port is used,
+both operations are done sequentially. For each configuration an additional
+lcore is needed since master lcore in use which is responsible for
+configuration, statistics printing and safe deinitialization of all ports
+and devices.
+
+The application can use a maximum of 8 ports.
+
+To run the application in a Linux environment with 3 lcores (one of them
+is master lcore), 1 port (port 0), software copying and MAC updating issue
+the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
+
+To run the application in a Linux environment with 2 lcores (one of them
+is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC
+updating issue the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw
+
+Refer to the *DPDK Getting Started Guide* for general information on
+running applications and the Environment Abstraction Layer (EAL) options.
+
+Explanation
+-----------
+
+The following sections provide an explanation of the main components of the
+code.
+
+All DPDK library functions used in the sample code are prefixed with
+``rte_`` and are explained in detail in the *DPDK API Documentation*.
+
+
+The Main Function
+~~~~~~~~~~~~~~~~~
+
+The ``main()`` function performs the initialization and calls the execution
+threads for each lcore.
+
+The first task is to initialize the Environment Abstraction Layer (EAL).
+The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
+function. The value returned is the number of parsed arguments:
+
+.. code-block:: c
+
+    /* init EAL */
+    ret = rte_eal_init(argc, argv);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+
+
+The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
+used by the application:
+
+.. code-block:: c
+
+    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
+        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+        MIN_POOL_SIZE);
+
+    /* Create the mbuf pool */
+    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+        rte_socket_id());
+    if (ioat_pktmbuf_pool == NULL)
+        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+Mbufs are the packet buffer structure used by DPDK. They are explained in
+detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
+
+The ``main()`` function also initializes the ports:
+
+.. code-block:: c
+
+    /* Initialise each port */
+    RTE_ETH_FOREACH_DEV(portid) {
+        port_init(portid, ioat_pktmbuf_pool);
+    }
+
+Each port is configured using ``port_init()``:
+
+.. code-block:: c
+
+     /*
+     * Initializes a given port using global settings and with the RX buffers
+     * coming from the mbuf_pool passed as a parameter.
+     */
+    static inline void
+    port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+    {
+        /* configuring port to use RSS for multiple RX queues */
+        static const struct rte_eth_conf port_conf = {
+            .rxmode = {
+                .mq_mode        = ETH_MQ_RX_RSS,
+                .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+            },
+            .rx_adv_conf = {
+                .rss_conf = {
+                    .rss_key = NULL,
+                    .rss_hf = ETH_RSS_PROTO_MASK,
+                }
+            }
+        };
+
+        struct rte_eth_rxconf rxq_conf;
+        struct rte_eth_txconf txq_conf;
+        struct rte_eth_conf local_port_conf = port_conf;
+        struct rte_eth_dev_info dev_info;
+        int ret, i;
+
+        /* Skip ports that are not enabled */
+        if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+            printf("Skipping disabled port %u\n", portid);
+            return;
+        }
+
+        /* Init port */
+        printf("Initializing port %u... ", portid);
+        fflush(stdout);
+        rte_eth_dev_info_get(portid, &dev_info);
+        local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+            dev_info.flow_type_rss_offloads;
+        if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+            local_port_conf.txmode.offloads |=
+                DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+        ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE, "Cannot configure device:"
+                " err=%d, port=%u\n", ret, portid);
+
+        ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+                            &nb_txd);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "Cannot adjust number of descriptors: err=%d, port=%u\n",
+                ret, portid);
+
+        rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+        /* Init Rx queues */
+        rxq_conf = dev_info.default_rxconf;
+        rxq_conf.offloads = local_port_conf.rxmode.offloads;
+        for (i = 0; i < nb_queues; i++) {
+            ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+                rte_eth_dev_socket_id(portid), &rxq_conf,
+                mbuf_pool);
+            if (ret < 0)
+                rte_exit(EXIT_FAILURE,
+                    "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+                    ret, portid, i);
+        }
+
+        /* Init one TX queue on each port */
+        txq_conf = dev_info.default_txconf;
+        txq_conf.offloads = local_port_conf.txmode.offloads;
+        ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+                rte_eth_dev_socket_id(portid),
+                &txq_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "rte_eth_tx_queue_setup:err=%d,port=%u\n",
+                ret, portid);
+
+        /* Initialize TX buffers */
+        tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+                RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+                rte_eth_dev_socket_id(portid));
+        if (tx_buffer[portid] == NULL)
+            rte_exit(EXIT_FAILURE,
+                "Cannot allocate buffer for tx on port %u\n",
+                portid);
+
+        rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+        ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+                rte_eth_tx_buffer_count_callback,
+                &port_statistics.tx_dropped[portid]);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "Cannot set error callback for tx buffer on port %u\n",
+                portid);
+
+        /* Start device */
+        ret = rte_eth_dev_start(portid);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "rte_eth_dev_start:err=%d, port=%u\n",
+                ret, portid);
+
+        rte_eth_promiscuous_enable(portid);
+
+        printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+                portid,
+                ioat_ports_eth_addr[portid].addr_bytes[0],
+                ioat_ports_eth_addr[portid].addr_bytes[1],
+                ioat_ports_eth_addr[portid].addr_bytes[2],
+                ioat_ports_eth_addr[portid].addr_bytes[3],
+                ioat_ports_eth_addr[portid].addr_bytes[4],
+                ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+        cfg.ports[cfg.nb_ports].rxtx_port = portid;
+        cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+    }
+
+The Ethernet ports are configured with local settings using the
+``rte_eth_dev_configure()`` function and the ``port_conf`` struct.
+The RSS is enabled so that multiple Rx queues could be used for
+packet receiving and copying by multiple CBDMA channels per port:
+
+.. code-block:: c
+
+    /* configuring port to use RSS for multiple RX queues */
+    static const struct rte_eth_conf port_conf = {
+        .rxmode = {
+            .mq_mode        = ETH_MQ_RX_RSS,
+            .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+        },
+        .rx_adv_conf = {
+            .rss_conf = {
+                .rss_key = NULL,
+                .rss_hf = ETH_RSS_PROTO_MASK,
+            }
+        }
+    };
+
+For this example the ports are set up with the number of Rx queues provided
+with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()``
+and ``rte_eth_tx_queue_setup()`` functions.
+
+The Ethernet port is then started:
+
+.. code-block:: c
+
+    ret = rte_eth_dev_start(portid);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
+            ret, portid);
+
+
+Finally the Rx port is set in promiscuous mode:
+
+.. code-block:: c
+
+    rte_eth_promiscuous_enable(portid);
+
+
+After that each port application assigns resources needed.
+
+.. code-block:: c
+
+    check_link_status(ioat_enabled_port_mask);
+
+    if (!cfg.nb_ports) {
+        rte_exit(EXIT_FAILURE,
+            "All available ports are disabled. Please set portmask.\n");
+    }
+
+    /* Check if there is enough lcores for all ports. */
+    cfg.nb_lcores = rte_lcore_count() - 1;
+    if (cfg.nb_lcores < 1)
+        rte_exit(EXIT_FAILURE,
+            "There should be at least one slave lcore.\n");
+
+    ret = 0;
+
+    if (copy_mode == COPY_MODE_IOAT_NUM) {
+        assign_rawdevs();
+    } else /* copy_mode == COPY_MODE_SW_NUM */ {
+        assign_rings();
+    }
+
+A link status is checked of each port enabled by port mask
+using ``check_link_status()`` function.
+
+.. code-block:: c
+
+    /* check link status, return true if at least one port is up */
+    static int
+    check_link_status(uint32_t port_mask)
+    {
+        uint16_t portid;
+        struct rte_eth_link link;
+        int retval = 0;
+
+        printf("\nChecking link status\n");
+        RTE_ETH_FOREACH_DEV(portid) {
+            if ((port_mask & (1 << portid)) == 0)
+                continue;
+
+            memset(&link, 0, sizeof(link));
+            rte_eth_link_get(portid, &link);
+
+            /* Print link status */
+            if (link.link_status) {
+                printf(
+                    "Port %d Link Up. Speed %u Mbps - %s\n",
+                    portid, link.link_speed,
+                    (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+                    ("full-duplex") : ("half-duplex\n"));
+                retval = 1;
+            } else
+                printf("Port %d Link Down\n", portid);
+        }
+        return retval;
+    }
+
+Depending on mode set (whether copy should be done by software or by hardware)
+special structures are assigned to each port. If software copy was chosen,
+application have to assign ring structures for packet exchanging between lcores
+assigned to ports.
+
+.. code-block:: c
+
+    static void
+    assign_rings(void)
+    {
+        uint32_t i;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            char ring_name[20];
+
+            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
+            /* Create ring for inter core communication */
+            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+                    ring_name, ring_size,
+                    rte_socket_id(), RING_F_SP_ENQ);
+
+            if (cfg.ports[i].rx_to_tx_ring == NULL)
+                rte_exit(EXIT_FAILURE, "%s\n",
+                        rte_strerror(rte_errno));
+        }
+    }
+
+
+When using hardware copy each Rx queue of the port is assigned an
+IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API
+functions:
+
+.. code-block:: c
+
+    static void
+    assign_rawdevs(void)
+    {
+        uint16_t nb_rawdev = 0, rdev_id = 0;
+        uint32_t i, j;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+                struct rte_rawdev_info rdev_info = { 0 };
+
+                do {
+                    if (rdev_id == rte_rawdev_count())
+                        goto end;
+                    rte_rawdev_info_get(rdev_id++, &rdev_info);
+                } while (strcmp(rdev_info.driver_name,
+                    IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+                cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+                configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+                ++nb_rawdev;
+            }
+        }
+    end:
+        if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+            rte_exit(EXIT_FAILURE,
+                "Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+                nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+    }
+
+
+The initialization of hardware device is done by ``rte_rawdev_configure()``
+function and ``rte_rawdev_info`` struct. After configuration the device is
+started using ``rte_rawdev_start()`` function. Each of the above operations
+is done in ``configure_rawdev_queue()``.
+
+.. code-block:: c
+
+    static void
+    configure_rawdev_queue(uint32_t dev_id)
+    {
+        struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+        /* Configure hardware copy device */
+        dev_config.ring_size = ring_size;
+
+        if (rte_rawdev_configure(dev_id, &info) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_configure()\n");
+        }
+        rte_rawdev_info_get(dev_id, &info);
+        if (dev_config.ring_size != ring_size) {
+            rte_exit(EXIT_FAILURE,
+                "Error, ring size is not %d (%d)\n",
+                ring_size, (int)dev_config.ring_size);
+        }
+        if (rte_rawdev_start(dev_id) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_start()\n");
+        }
+    }
+
+If initialization is successful memory for hardware device
+statistics is allocated.
+
+Finally ``main()`` functions starts all processing lcores and starts
+printing stats in a loop on master lcore. The application can be
+interrupted and closed using ``Ctrl-C``. The master lcore waits for
+all slave processes to finish, deallocates resources and exits.
+
+The processing lcores launching function are described below.
+
+The Lcores Launching Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As described above ``main()`` function invokes ``start_forwarding_cores()``
+function in order to start processing for each lcore:
+
+.. code-block:: c
+
+    static void start_forwarding_cores(void)
+    {
+        uint32_t lcore_id = rte_lcore_id();
+
+        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+                __func__, rte_lcore_id());
+
+        if (cfg.nb_lcores == 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+                NULL, lcore_id);
+        } else if (cfg.nb_lcores > 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+                NULL, lcore_id);
+
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+                lcore_id);
+        }
+    }
+
+The function launches Rx/Tx processing functions on configured lcores
+for each port using ``rte_eal_remote_launch()``. The configured ports,
+their number and number of assigned lcores are stored in user-defined
+``rxtx_transmission_config`` struct that is initialized before launching
+tasks:
+
+.. code-block:: c
+
+    struct rxtx_transmission_config {
+        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+        uint16_t nb_ports;
+        uint16_t nb_lcores;
+    };
+
+The Lcores Processing Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For receiving packets on each port an ``ioat_rx_port()`` function is used.
+The function receives packets on each configured Rx queue. Depending on mode
+the user chose, it will enqueue packets to IOAT rawdev channels and then invoke
+copy process (hardware copy), or perform software copy of each packet using
+``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring:
+
+.. code-block:: c
+
+    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+    static void
+    ioat_rx_port(struct rxtx_port_config *rx_config)
+    {
+        uint32_t nb_rx, nb_enq, i, j;
+        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+        for (i = 0; i < rx_config->nb_queues; i++) {
+
+            nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+                pkts_burst, MAX_PKT_BURST);
+
+            if (nb_rx == 0)
+                continue;
+
+            port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
+            if (copy_mode == COPY_MODE_IOAT_NUM) {
+                /* Perform packet hardware copy */
+                nb_enq = ioat_enqueue_packets(pkts_burst,
+                    nb_rx, rx_config->ioat_ids[i]);
+                if (nb_enq > 0)
+                    rte_ioat_do_copies(rx_config->ioat_ids[i]);
+            } else {
+                /* Perform packet software copy, free source packets */
+                int ret;
+                struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+                ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst_copy, nb_rx);
+
+                if (unlikely(ret < 0))
+                    rte_exit(EXIT_FAILURE,
+                        "Unable to allocate memory.\n");
+
+                for (j = 0; j < nb_rx; j++)
+                    pktmbuf_sw_copy(pkts_burst[j],
+                        pkts_burst_copy[j]);
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst, nb_rx);
+
+                nb_enq = rte_ring_enqueue_burst(
+                    rx_config->rx_to_tx_ring,
+                    (void *)pkts_burst_copy, nb_rx, NULL);
+
+                /* Free any not enqueued packets. */
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&pkts_burst_copy[nb_enq],
+                    nb_rx - nb_enq);
+            }
+
+            port_statistics.copy_dropped[rx_config->rxtx_port] +=
+                (nb_rx - nb_enq);
+        }
+    }
+
+The packets are received in burst mode using ``rte_eth_rx_burst()``
+function. When using hardware copy mode the packets are enqueued in
+copying device's buffer using ``ioat_enqueue_packets()`` which calls
+``rte_ioat_enqueue_copy()``. When all received packets are in the
+buffer the copies are invoked by calling ``rte_ioat_do_copies()``.
+Function ``rte_ioat_enqueue_copy()`` operates on physical address of
+the packet. Structure ``rte_mbuf`` contains only physical address to
+start of the data buffer (``buf_iova``). Thus the address is shifted
+by ``addr_offset`` value in order to get pointer to ``rearm_data``
+member of ``rte_mbuf``. That way the packet is copied all at once
+(with data and metadata).
+
+.. code-block:: c
+
+    static uint32_t
+    ioat_enqueue_packets(struct rte_mbuf **pkts,
+        uint32_t nb_rx, uint16_t dev_id)
+    {
+        int ret;
+        uint32_t i;
+        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+            &pkts[0]->rearm_data);
+
+        ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                (void *)pkts_copy, nb_rx);
+
+        if (unlikely(ret < 0))
+            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+        for (i = 0; i < nb_rx; i++) {
+            /* Perform data copy */
+            ret = rte_ioat_enqueue_copy(dev_id,
+                pkts[i]->buf_iova
+                    - addr_offset,
+                pkts_copy[i]->buf_iova
+                    - addr_offset,
+                rte_pktmbuf_data_len(pkts[i])
+                    + addr_offset,
+                (uintptr_t)pkts[i],
+                (uintptr_t)pkts_copy[i],
+                0 /* nofence */);
+
+            if (ret != 1)
+                break;
+        }
+
+        ret = i;
+        /* Free any not enqueued packets. */
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+            nb_rx - i);
+
+        return ret;
+    }
+
+
+All done copies are processed by ``ioat_tx_port()`` function. When using
+hardware copy mode the function invokes ``rte_ioat_completed_copies()``
+on each assigned IOAT channel to gather copied packets. If software copy
+mode is used the function dequeues copied packets from the rte_ring. Then each
+packet MAC address is changed if it was enabled. After that copies are sent
+in burst mode using `` rte_eth_tx_burst()``.
+
+
+.. code-block:: c
+
+    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
+    static void
+    ioat_tx_port(struct rxtx_port_config *tx_config)
+    {
+        uint32_t i, j, nb_dq = 0;
+        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
+        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+        if (copy_mode == COPY_MODE_IOAT_NUM) {
+            for (i = 0; i < tx_config->nb_queues; i++) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ioat_completed_copies(
+                    tx_config->ioat_ids[i], MAX_PKT_BURST,
+                    (void *)mbufs_src, (void *)mbufs_dst);
+
+                if (nb_dq == 0)
+                    break;
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)mbufs_src, nb_dq);
+
+                /* Update macs if enabled */
+                if (mac_updating) {
+                    for (j = 0; j < nb_dq; j++)
+                        update_mac_addrs(mbufs_dst[j],
+                            tx_config->rxtx_port);
+                }
+
+                const uint16_t nb_tx = rte_eth_tx_burst(
+                    tx_config->rxtx_port, 0,
+                    (void *)mbufs_dst, nb_dq);
+
+                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+                /* Free any unsent packets. */
+                if (unlikely(nb_tx < nb_dq))
+                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&mbufs_dst[nb_tx],
+                        nb_dq - nb_tx);
+            }
+        }
+        else {
+            for (i = 0; i < tx_config->nb_queues; i++) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+                    (void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+                if (nb_dq == 0)
+                    return;
+
+                /* Update macs if enabled */
+                if (mac_updating) {
+                    for (j = 0; j < nb_dq; j++)
+                        update_mac_addrs(mbufs_dst[j],
+                            tx_config->rxtx_port);
+                }
+
+                const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+                    0, (void *)mbufs_dst, nb_dq);
+
+                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+                /* Free any unsent packets. */
+                if (unlikely(nb_tx < nb_dq))
+                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&mbufs_dst[nb_tx],
+                        nb_dq - nb_tx);
+            }
+        }
+    }
+
+The Packet Copying Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to perform packet copy there is a user-defined function
+``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
+metadata from source packet to new mbuf, and then copying a data
+chunk of source packet. Both memory copies are done using
+``rte_memcpy()``:
+
+.. code-block:: c
+
+    static inline void
+    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+    {
+        /* Copy packet metadata */
+        rte_memcpy(&dst->rearm_data,
+            &src->rearm_data,
+            offsetof(struct rte_mbuf, cacheline1)
+                - offsetof(struct rte_mbuf, rearm_data));
+
+        /* Copy packet data */
+        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+            rte_pktmbuf_mtod(src, char *), src->data_len);
+    }
+
+The metadata in this example is copied from ``rearm_data`` member of
+``rte_mbuf`` struct up to ``cacheline1``.
+
+In order to understand why software packet copying is done as shown
+above please refer to the "Mbuf Library" section of the
+*DPDK Programmer's Guide*.
\ No newline at end of file
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-16  9:42     ` Baran, MarcinX
@ 2019-09-19  9:19       ` Aaron Conole
  0 siblings, 0 replies; 76+ messages in thread
From: Aaron Conole @ 2019-09-19  9:19 UTC (permalink / raw)
  To: Baran, MarcinX; +Cc: dev, Richardson, Bruce

"Baran, MarcinX" <marcinx.baran@intel.com> writes:

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com> 
> Sent: Friday, September 13, 2019 8:45 PM
> To: Baran, MarcinX <marcinx.baran@intel.com>
> Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage
>
> Marcin Baran <marcinx.baran@intel.com> writes:
>
>> A new sample app demonstrating use of driver for CBDMA.
>> The app receives packets, performs software or hardware copy, changes 
>> packets' MAC addresses (if enabled) and forwards them. The patch 
>> includes sample application as well as it's guide.
>
> Please fix the dependencies:
>
> https://travis-ci.com/ovsrobot/dpdk/jobs/234891633
>
> examples/meson.build:89:4: ERROR: Problem encountered: Missing dependency "pmd_ioat" for example "ioat"
>
> [Marcin]: I will add check for dependency in example's  meson.build for v3:
>
> 	if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
> 		build = false
> 		subdir_done()
> 	endif

Seems it didn't work in v3:

https://travis-ci.com/ovsrobot/dpdk/builds/128147569

-Aaron

>> v2:
>>  - change printing total stats to be deltas
>>  - update documentation
>>  - fix 1 thread/sw copy/multiple Rx queues packet dropping
>>  - divide patch into several presenting functional change
>>
>> Marcin Baran (4):
>>   examples/ioat: add software copy support
>>   examples/ioat: add rawdev copy mode support
>>   examples/ioat: add stats printing for each port
>>   doc/guides/: provide IOAT sample app guide
>>
>> Pawel Modrak (2):
>>   examples/ioat: create sample app on ioat driver usage
>>   examples/ioat: add two threads configuration
>>
>>  doc/guides/sample_app_ug/index.rst |    1 +
>>  doc/guides/sample_app_ug/intro.rst |    4 +
>>  doc/guides/sample_app_ug/ioat.rst  |  764 +++++++++++++++++++++
>>  examples/Makefile                  |    3 +
>>  examples/ioat/Makefile             |   54 ++
>>  examples/ioat/ioatfwd.c            | 1025 ++++++++++++++++++++++++++++
>>  examples/ioat/meson.build          |   13 +
>>  examples/meson.build               |    1 +
>>  8 files changed, 1865 insertions(+)
>>  create mode 100644 doc/guides/sample_app_ug/ioat.rst  create mode 
>> 100644 examples/ioat/Makefile  create mode 100644 
>> examples/ioat/ioatfwd.c  create mode 100644 examples/ioat/meson.build

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
                       ` (5 preceding siblings ...)
  2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
@ 2019-09-19  9:38     ` Marcin Baran
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 1/6] examples/ioat: create " Marcin Baran
                         ` (6 more replies)
  6 siblings, 7 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-19  9:38 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The patch includes sample application as
well as it's guide.

v4:
 - fix meson build support check

v3:
 - add check for meson build whether IOAT is supported

v2:
 - change printing total stats to be deltas
 - update documentation
 - fix 1 thread/sw copy/multiple Rx queues packet dropping
 - divide patch into several presenting functional change


Marcin Baran (4):
  examples/ioat: add software copy support
  examples/ioat: add rawdev copy mode support
  examples/ioat: add stats printing for each port
  doc/guides/: provide IOAT sample app guide

Pawel Modrak (2):
  examples/ioat: create sample app on ioat driver usage
  examples/ioat: add two threads configuration

 doc/guides/sample_app_ug/index.rst |    1 +
 doc/guides/sample_app_ug/intro.rst |    4 +
 doc/guides/sample_app_ug/ioat.rst  |  764 +++++++++++++++++++++
 examples/Makefile                  |    3 +
 examples/ioat/Makefile             |   54 ++
 examples/ioat/ioatfwd.c            | 1025 ++++++++++++++++++++++++++++
 examples/ioat/meson.build          |   23 +
 examples/meson.build               |    1 +
 8 files changed, 1875 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v4 1/6] examples/ioat: create sample app on ioat driver usage
  2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
@ 2019-09-19  9:38       ` Marcin Baran
  2019-09-19 14:44         ` Bruce Richardson
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 2/6] examples/ioat: add software copy support Marcin Baran
                         ` (5 subsequent siblings)
  6 siblings, 1 reply; 76+ messages in thread
From: Marcin Baran @ 2019-09-19  9:38 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The change covers ports initialization,
closing connection and argument parsing.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/Makefile         |   3 +
 examples/ioat/Makefile    |  54 +++++
 examples/ioat/ioatfwd.c   | 439 ++++++++++++++++++++++++++++++++++++++
 examples/ioat/meson.build |  23 ++
 examples/meson.build      |   1 +
 5 files changed, 520 insertions(+)
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

diff --git a/examples/Makefile b/examples/Makefile
index de11dd487..3cb313d7d 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation
 DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify
 DIRS-y += flow_filtering
 DIRS-y += helloworld
+ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y)
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat
+endif
 DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline
 ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
 DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
diff --git a/examples/ioat/Makefile b/examples/ioat/Makefile
new file mode 100644
index 000000000..2a4d1da2d
--- /dev/null
+++ b/examples/ioat/Makefile
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# binary name
+APP = ioatfwd
+
+# all source are stored in SRCS-y
+SRCS-y := ioatfwd.c
+
+# Build using pkg-config variables if possible
+ifeq ($(shell pkg-config --exists libdpdk && echo 0),0)
+
+all: shared
+.PHONY: shared static
+shared: build/$(APP)-shared
+	ln -sf $(APP)-shared build/$(APP)
+static: build/$(APP)-static
+	ln -sf $(APP)-static build/$(APP)
+
+PC_FILE := $(shell pkg-config --path libdpdk)
+CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
+LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
+
+build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
+
+build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
+
+build:
+	@mkdir -p $@
+
+.PHONY: clean
+clean:
+	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
+	test -d build && rmdir -p build || true
+
+else # Build using legacy build system
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, detect a build directory, by looking for a path with a .config
+RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config)))))
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+endif
diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
new file mode 100644
index 000000000..977ea6a61
--- /dev/null
+++ b/examples/ioat/ioatfwd.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#include <stdint.h>
+#include <getopt.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+#include <rte_ethdev.h>
+#include <rte_rawdev.h>
+#include <rte_ioat_rawdev.h>
+
+/* size of ring used for software copying between rx and tx. */
+#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
+#define MAX_PKT_BURST 32
+#define MEMPOOL_CACHE_SIZE 512
+#define MIN_POOL_SIZE 65536U
+#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
+#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
+#define CMD_LINE_OPT_PORTMASK "portmask"
+#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
+#define CMD_LINE_OPT_COPY_TYPE "copy-type"
+#define CMD_LINE_OPT_RING_SIZE "ring-size"
+
+/* configurable number of RX/TX ring descriptors */
+#define RX_DEFAULT_RINGSIZE 1024
+#define TX_DEFAULT_RINGSIZE 1024
+
+/* max number of RX queues per port */
+#define MAX_RX_QUEUES_COUNT 8
+
+struct rxtx_port_config {
+	/* common config */
+	uint16_t rxtx_port;
+	uint16_t nb_queues;
+	/* for software copy mode */
+	struct rte_ring *rx_to_tx_ring;
+	/* for IOAT rawdev copy mode */
+	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT];
+};
+
+struct rxtx_transmission_config {
+	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+	uint16_t nb_ports;
+	uint16_t nb_lcores;
+};
+
+typedef enum copy_mode_t {
+#define COPY_MODE_SW "sw"
+	COPY_MODE_SW_NUM,
+#define COPY_MODE_IOAT "rawdev"
+	COPY_MODE_IOAT_NUM,
+	COPY_MODE_INVALID_NUM,
+	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
+} copy_mode_t;
+
+/* mask of enabled ports */
+static uint32_t ioat_enabled_port_mask;
+
+/* number of RX queues per port */
+static uint16_t nb_queues = 1;
+
+/* MAC updating enabled by default. */
+static int mac_updating = 1;
+
+/* hardare copy mode enabled by default. */
+static copy_mode_t copy_mode = COPY_MODE_IOAT_NUM;
+
+/* size of IOAT rawdev ring for hardware copy mode or
+ * rte_ring for software copy mode
+ */
+static unsigned short ring_size = 2048;
+
+/* global transmission config */
+struct rxtx_transmission_config cfg;
+
+/* configurable number of RX/TX ring descriptors */
+static uint16_t nb_rxd = RX_DEFAULT_RINGSIZE;
+static uint16_t nb_txd = TX_DEFAULT_RINGSIZE;
+
+static volatile bool force_quit;
+
+/* ethernet addresses of ports */
+static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
+struct rte_mempool *ioat_pktmbuf_pool;
+
+/* Display usage */
+static void
+ioat_usage(const char *prgname)
+{
+	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
+		"  -p --portmask: hexadecimal bitmask of ports to configure\n"
+		"  -q NQ: number of RX queues per port (default is 1)\n"
+		"  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
+		"      When enabled:\n"
+		"       - The source MAC address is replaced by the TX port MAC address\n"
+		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
+		"  -c --copy-type CT: type of copy: sw|rawdev\n"
+		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
+	       prgname);
+}
+
+static int
+ioat_parse_portmask(const char *portmask)
+{
+	char *end = NULL;
+	unsigned long pm;
+
+	/* Parse hexadecimal string */
+	pm = strtoul(portmask, &end, 16);
+	if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+
+	return pm;
+}
+
+static copy_mode_t
+ioat_parse_copy_mode(const char *copy_mode)
+{
+	if (strcmp(copy_mode, COPY_MODE_SW) == 0)
+		return COPY_MODE_SW_NUM;
+	else if (strcmp(copy_mode, COPY_MODE_IOAT) == 0)
+		return COPY_MODE_IOAT_NUM;
+
+	return COPY_MODE_INVALID_NUM;
+}
+
+/* Parse the argument given in the command line of the application */
+static int
+ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
+{
+	static const char short_options[] =
+		"p:"  /* portmask */
+		"q:"  /* number of RX queues per port */
+		"c:"  /* copy type (sw|rawdev) */
+		"s:"  /* ring size */
+		;
+
+	static const struct option lgopts[] = {
+		{CMD_LINE_OPT_MAC_UPDATING, no_argument, &mac_updating, 1},
+		{CMD_LINE_OPT_NO_MAC_UPDATING, no_argument, &mac_updating, 0},
+		{CMD_LINE_OPT_PORTMASK, required_argument, NULL, 'p'},
+		{CMD_LINE_OPT_NB_QUEUE, required_argument, NULL, 'q'},
+		{CMD_LINE_OPT_COPY_TYPE, required_argument, NULL, 'c'},
+		{CMD_LINE_OPT_RING_SIZE, required_argument, NULL, 's'},
+		{NULL, 0, 0, 0}
+	};
+
+	const unsigned int default_port_mask = (1 << nb_ports) - 1;
+	int opt, ret;
+	char **argvopt;
+	int option_index;
+	char *prgname = argv[0];
+
+	ioat_enabled_port_mask = default_port_mask;
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, short_options,
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		/* portmask */
+		case 'p':
+			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
+			if (ioat_enabled_port_mask & ~default_port_mask ||
+					ioat_enabled_port_mask <= 0) {
+				printf("Invalid portmask, %s, suggest 0x%x\n",
+						optarg, default_port_mask);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'q':
+			nb_queues = atoi(optarg);
+			if (nb_queues == 0 || nb_queues > MAX_RX_QUEUES_COUNT) {
+				printf("Invalid RX queues number %s. Max %u\n",
+					optarg, MAX_RX_QUEUES_COUNT);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'c':
+			copy_mode = ioat_parse_copy_mode(optarg);
+			if (copy_mode == COPY_MODE_INVALID_NUM) {
+				printf("Invalid copy type. Use: sw, rawdev\n");
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 's':
+			ring_size = atoi(optarg);
+			if (ring_size == 0) {
+				printf("Invalid ring size, %s.\n", optarg);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		/* long options */
+		case 0:
+			break;
+
+		default:
+			ioat_usage(prgname);
+			return -1;
+		}
+	}
+
+	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
+	if (optind >= 0)
+		argv[optind-1] = prgname;
+
+	ret = optind-1;
+	optind = 1; /* reset getopt lib */
+	return ret;
+}
+
+/* check link status, return true if at least one port is up */
+static int
+check_link_status(uint32_t port_mask)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	int retval = 0;
+
+	printf("\nChecking link status\n");
+	RTE_ETH_FOREACH_DEV(portid) {
+		if ((port_mask & (1 << portid)) == 0)
+			continue;
+
+		memset(&link, 0, sizeof(link));
+		rte_eth_link_get(portid, &link);
+
+		/* Print link status */
+		if (link.link_status) {
+			printf(
+				"Port %d Link Up. Speed %u Mbps - %s\n",
+				portid, link.link_speed,
+				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+				("full-duplex") : ("half-duplex\n"));
+			retval = 1;
+		} else
+			printf("Port %d Link Down\n", portid);
+	}
+	return retval;
+}
+
+/*
+ * Initializes a given port using global settings and with the RX buffers
+ * coming from the mbuf_pool passed as a parameter.
+ */
+static inline void
+port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+{
+	/* configuring port to use RSS for multiple RX queues */
+	static const struct rte_eth_conf port_conf = {
+		.rxmode = {
+			.mq_mode        = ETH_MQ_RX_RSS,
+			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
+		},
+		.rx_adv_conf = {
+			.rss_conf = {
+				.rss_key = NULL,
+				.rss_hf = ETH_RSS_PROTO_MASK,
+			}
+		}
+	};
+
+	struct rte_eth_rxconf rxq_conf;
+	struct rte_eth_txconf txq_conf;
+	struct rte_eth_conf local_port_conf = port_conf;
+	struct rte_eth_dev_info dev_info;
+	int ret, i;
+
+	/* Skip ports that are not enabled */
+	if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+		printf("Skipping disabled port %u\n", portid);
+		return;
+	}
+
+	/* Init port */
+	printf("Initializing port %u... ", portid);
+	fflush(stdout);
+	rte_eth_dev_info_get(portid, &dev_info);
+	local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+		dev_info.flow_type_rss_offloads;
+	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+		local_port_conf.txmode.offloads |=
+			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Cannot configure device:"
+			" err=%d, port=%u\n", ret, portid);
+
+	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+						&nb_txd);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot adjust number of descriptors: err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+	/* Init RX queues */
+	rxq_conf = dev_info.default_rxconf;
+	rxq_conf.offloads = local_port_conf.rxmode.offloads;
+	for (i = 0; i < nb_queues; i++) {
+		ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+			rte_eth_dev_socket_id(portid), &rxq_conf,
+			mbuf_pool);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE,
+				"rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+				ret, portid, i);
+	}
+
+	/* Init one TX queue on each port */
+	txq_conf = dev_info.default_txconf;
+	txq_conf.offloads = local_port_conf.txmode.offloads;
+	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+			rte_eth_dev_socket_id(portid),
+			&txq_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
+			ret, portid);
+
+	/* Initialize TX buffers */
+	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+			rte_eth_dev_socket_id(portid));
+	if (tx_buffer[portid] == NULL)
+		rte_exit(EXIT_FAILURE,
+			"Cannot allocate buffer for tx on port %u\n",
+			portid);
+
+	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+	/* Start device */
+	ret = rte_eth_dev_start(portid);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_dev_start:err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_promiscuous_enable(portid);
+
+	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+			portid,
+			ioat_ports_eth_addr[portid].addr_bytes[0],
+			ioat_ports_eth_addr[portid].addr_bytes[1],
+			ioat_ports_eth_addr[portid].addr_bytes[2],
+			ioat_ports_eth_addr[portid].addr_bytes[3],
+			ioat_ports_eth_addr[portid].addr_bytes[4],
+			ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+	cfg.ports[cfg.nb_ports].rxtx_port = portid;
+	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\n\nSignal %d received, preparing to exit...\n",
+			signum);
+		force_quit = true;
+	}
+}
+
+int
+main(int argc, char **argv)
+{
+	int ret;
+	uint16_t nb_ports, portid;
+	uint32_t i;
+	unsigned int nb_mbufs;
+
+	/* Init EAL */
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+	argc -= ret;
+	argv += ret;
+
+	force_quit = false;
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	nb_ports = rte_eth_dev_count_avail();
+	if (nb_ports == 0)
+		rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
+
+	/* Parse application arguments (after the EAL ones) */
+	ret = ioat_parse_args(argc, argv, nb_ports);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid IOAT arguments\n");
+
+	nb_mbufs = RTE_MAX(nb_ports * (nb_queues * (nb_rxd + nb_txd +
+		4 * MAX_PKT_BURST) + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+		MIN_POOL_SIZE);
+
+	/* Create the mbuf pool */
+	ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+		MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+		rte_socket_id());
+	if (ioat_pktmbuf_pool == NULL)
+		rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+	/* Initialise each port */
+	cfg.nb_ports = 0;
+	RTE_ETH_FOREACH_DEV(portid)
+		port_init(portid, ioat_pktmbuf_pool, nb_queues);
+
+	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
+		sleep(1);
+
+	/* Check if there is enough lcores for all ports. */
+	cfg.nb_lcores = rte_lcore_count() - 1;
+	if (cfg.nb_lcores < 1)
+		rte_exit(EXIT_FAILURE,
+			"There should be at least one slave lcore.\n");
+	for (i = 0; i < cfg.nb_ports; i++) {
+		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
+		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
+		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+	}
+
+	printf("Bye...\n");
+	return 0;
+}
diff --git a/examples/ioat/meson.build b/examples/ioat/meson.build
new file mode 100644
index 000000000..ca5cdc9ee
--- /dev/null
+++ b/examples/ioat/meson.build
@@ -0,0 +1,23 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
+	build = false
+	subdir_done()
+endif
+
+if dpdk_conf.get('RTE_LIBRTE_PMD_IOAT_RAWDEV', 0) != 1
+	build = false
+	subdir_done()
+endif
+
+deps += ['pmd_ioat']
+
+sources = files(
+	'ioatfwd.c'
+)
diff --git a/examples/meson.build b/examples/meson.build
index a046b74ad..c2e18b59b 100644
--- a/examples/meson.build
+++ b/examples/meson.build
@@ -16,6 +16,7 @@ all_examples = [
 	'eventdev_pipeline', 'exception_path',
 	'fips_validation', 'flow_classify',
 	'flow_filtering', 'helloworld',
+	'ioat',
 	'ip_fragmentation', 'ip_pipeline',
 	'ip_reassembly', 'ipsec-secgw',
 	'ipv4_multicast', 'kni',
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v4 2/6] examples/ioat: add software copy support
  2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 1/6] examples/ioat: create " Marcin Baran
@ 2019-09-19  9:38       ` Marcin Baran
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
                         ` (4 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-19  9:38 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 188 insertions(+), 23 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 977ea6a61..3a092c6cf 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -13,7 +13,7 @@
 #include <rte_rawdev.h>
 #include <rte_ioat_rawdev.h>
 
-/* size of ring used for software copying between rx and tx. */
+ /* size of ring used for software copying between rx and tx. */
 #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
 #define MAX_PKT_BURST 32
 #define MEMPOOL_CACHE_SIZE 512
@@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+static void
+update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
+{
+	struct rte_ether_hdr *eth;
+	void *tmp;
+
+	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
+
+	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
+	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
+	 */
+	tmp = &eth->d_addr.addr_bytes[0];
+	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
+
+	/* src addr */
+	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], &eth->s_addr);
+}
+
+static inline void
+pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+{
+	/* Copy packet metadata */
+	rte_memcpy(&dst->rearm_data,
+		&src->rearm_data,
+		offsetof(struct rte_mbuf, cacheline1)
+		- offsetof(struct rte_mbuf, rearm_data));
+
+	/* Copy packet data */
+	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+		rte_pktmbuf_mtod(src, char *), src->data_len);
+}
+
+/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+static void
+ioat_rx_port(struct rxtx_port_config *rx_config)
+{
+	uint32_t nb_rx, nb_enq, i, j;
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+
+	for (i = 0; i < rx_config->nb_queues; i++) {
+
+		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+			pkts_burst, MAX_PKT_BURST);
+
+		if (nb_rx == 0)
+			continue;
+
+		/* Perform packet software copy, free source packets */
+		int ret;
+		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+			(void *)pkts_burst_copy, nb_rx);
+
+		if (unlikely(ret < 0))
+			rte_exit(EXIT_FAILURE,
+				"Unable to allocate memory.\n");
+
+		for (j = 0; j < nb_rx; j++)
+			pktmbuf_sw_copy(pkts_burst[j],
+				pkts_burst_copy[j]);
+
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)pkts_burst, nb_rx);
+
+		nb_enq = rte_ring_enqueue_burst(
+			rx_config->rx_to_tx_ring,
+			(void *)pkts_burst_copy, nb_rx, NULL);
+
+		/* Free any not enqueued packets. */
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&pkts_burst_copy[nb_enq],
+			nb_rx - nb_enq);
+	}
+}
+
+/* Transmit packets from IOAT rawdev/rte_ring for one port. */
+static void
+ioat_tx_port(struct rxtx_port_config *tx_config)
+{
+	uint32_t i, nb_dq = 0;
+	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+	/* Deque the mbufs from rx_to_tx_ring. */
+	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+	if (nb_dq == 0)
+		return;
+
+	/* Update macs if enabled */
+	if (mac_updating) {
+		for (i = 0; i < nb_dq; i++)
+			update_mac_addrs(mbufs_dst[i],
+				tx_config->rxtx_port);
+	}
+
+	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+		0, (void *)mbufs_dst, nb_dq);
+
+	/* Free any unsent packets. */
+	if (unlikely(nb_tx < nb_dq))
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+		(void *)&mbufs_dst[nb_tx],
+			nb_dq - nb_tx);
+}
+
+/* Main rx and tx loop if only one slave lcore available */
+static void
+rxtx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
+		" lcore %u\n", rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++) {
+			ioat_rx_port(&cfg.ports[i]);
+			ioat_tx_port(&cfg.ports[i]);
+		}
+}
+
+static void start_forwarding_cores(void)
+{
+	uint32_t lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+		__func__, rte_lcore_id());
+
+	lcore_id = rte_get_next_lcore(lcore_id, true, true);
+	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+		NULL, lcore_id);
+}
+
 /* Display usage */
 static void
 ioat_usage(const char *prgname)
@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)
 		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
 		"  -c --copy-type CT: type of copy: sw|rawdev\n"
 		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
-	       prgname);
+		prgname);
 }
 
 static int
@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 	argvopt = argv;
 
 	while ((opt = getopt_long(argc, argvopt, short_options,
-				  lgopts, &option_index)) != EOF) {
+		lgopts, &option_index)) != EOF) {
 
 		switch (opt) {
-		/* portmask */
+			/* portmask */
 		case 'p':
 			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
 			if (ioat_enabled_port_mask & ~default_port_mask ||
-					ioat_enabled_port_mask <= 0) {
+				ioat_enabled_port_mask <= 0) {
 				printf("Invalid portmask, %s, suggest 0x%x\n",
-						optarg, default_port_mask);
+					optarg, default_port_mask);
 				ioat_usage(prgname);
 				return -1;
 			}
@@ -204,7 +340,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 			}
 			break;
 
-		/* long options */
+			/* long options */
 		case 0:
 			break;
 
@@ -216,9 +352,9 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 
 	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
 	if (optind >= 0)
-		argv[optind-1] = prgname;
+		argv[optind - 1] = prgname;
 
-	ret = optind-1;
+	ret = optind - 1;
 	optind = 1; /* reset getopt lib */
 	return ret;
 }
@@ -253,6 +389,26 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+assign_rings(void)
+{
+	uint32_t i;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		char ring_name[RTE_RING_NAMESIZE];
+
+		snprintf(ring_name, sizeof(ring_name), "rx_to_tx_ring_%u", i);
+		/* Create ring for inter core communication */
+		cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+			ring_name, ring_size,
+			rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
+
+		if (cfg.ports[i].rx_to_tx_ring == NULL)
+			rte_exit(EXIT_FAILURE, "Ring create failed: %s\n",
+				rte_strerror(rte_errno));
+	}
+}
+
 /*
  * Initializes a given port using global settings and with the RX buffers
  * coming from the mbuf_pool passed as a parameter.
@@ -263,7 +419,7 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	/* configuring port to use RSS for multiple RX queues */
 	static const struct rte_eth_conf port_conf = {
 		.rxmode = {
-			.mq_mode        = ETH_MQ_RX_RSS,
+			.mq_mode = ETH_MQ_RX_RSS,
 			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
 		},
 		.rx_adv_conf = {
@@ -294,14 +450,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 		dev_info.flow_type_rss_offloads;
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
 		local_port_conf.txmode.offloads |=
-			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+		DEV_TX_OFFLOAD_MBUF_FAST_FREE;
 	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE, "Cannot configure device:"
 			" err=%d, port=%u\n", ret, portid);
 
 	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
-						&nb_txd);
+		&nb_txd);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE,
 			"Cannot adjust number of descriptors: err=%d, port=%u\n",
@@ -326,8 +482,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	txq_conf = dev_info.default_txconf;
 	txq_conf.offloads = local_port_conf.txmode.offloads;
 	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
-			rte_eth_dev_socket_id(portid),
-			&txq_conf);
+		rte_eth_dev_socket_id(portid),
+		&txq_conf);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE,
 			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
@@ -335,8 +491,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	/* Initialize TX buffers */
 	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
-			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
-			rte_eth_dev_socket_id(portid));
+		RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+		rte_eth_dev_socket_id(portid));
 	if (tx_buffer[portid] == NULL)
 		rte_exit(EXIT_FAILURE,
 			"Cannot allocate buffer for tx on port %u\n",
@@ -354,13 +510,13 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	rte_eth_promiscuous_enable(portid);
 
 	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
-			portid,
-			ioat_ports_eth_addr[portid].addr_bytes[0],
-			ioat_ports_eth_addr[portid].addr_bytes[1],
-			ioat_ports_eth_addr[portid].addr_bytes[2],
-			ioat_ports_eth_addr[portid].addr_bytes[3],
-			ioat_ports_eth_addr[portid].addr_bytes[4],
-			ioat_ports_eth_addr[portid].addr_bytes[5]);
+		portid,
+		ioat_ports_eth_addr[portid].addr_bytes[0],
+		ioat_ports_eth_addr[portid].addr_bytes[1],
+		ioat_ports_eth_addr[portid].addr_bytes[2],
+		ioat_ports_eth_addr[portid].addr_bytes[3],
+		ioat_ports_eth_addr[portid].addr_bytes[4],
+		ioat_ports_eth_addr[portid].addr_bytes[5]);
 
 	cfg.ports[cfg.nb_ports].rxtx_port = portid;
 	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
@@ -428,10 +584,19 @@ main(int argc, char **argv)
 	if (cfg.nb_lcores < 1)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
+
+	assign_rings();
+
+	start_forwarding_cores();
+
+	/* force_quit is true when we get here */
+	rte_eal_mp_wait_lcore();
+
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v4 3/6] examples/ioat: add rawdev copy mode support
  2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 1/6] examples/ioat: create " Marcin Baran
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 2/6] examples/ioat: add software copy support Marcin Baran
@ 2019-09-19  9:38       ` Marcin Baran
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 4/6] examples/ioat: add two threads configuration Marcin Baran
                         ` (3 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-19  9:38 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++--------
 1 file changed, 189 insertions(+), 47 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 3a092c6cf..c66ce7e49 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
 		rte_pktmbuf_mtod(src, char *), src->data_len);
 }
 
+static uint32_t
+ioat_enqueue_packets(struct rte_mbuf **pkts,
+	uint32_t nb_rx, uint16_t dev_id)
+{
+	int ret;
+	uint32_t i;
+	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+		&pkts[0]->rearm_data);
+
+	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+		(void *)pkts_copy, nb_rx);
+
+	if (unlikely(ret < 0))
+		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+	for (i = 0; i < nb_rx; i++) {
+		/* Perform data copy */
+		ret = rte_ioat_enqueue_copy(dev_id,
+			pkts[i]->buf_iova
+			- addr_offset,
+			pkts_copy[i]->buf_iova
+			- addr_offset,
+			rte_pktmbuf_data_len(pkts[i])
+			+ addr_offset,
+			(uintptr_t)pkts[i],
+			(uintptr_t)pkts_copy[i],
+			0 /* nofence */);
+
+		if (ret != 1)
+			break;
+	}
+
+	ret = i;
+	/* Free any not enqueued packets. */
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+		nb_rx - i);
+
+
+	return ret;
+}
+
 /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
 static void
 ioat_rx_port(struct rxtx_port_config *rx_config)
@@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
-		/* Perform packet software copy, free source packets */
-		int ret;
-		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
-
-		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
-			(void *)pkts_burst_copy, nb_rx);
-
-		if (unlikely(ret < 0))
-			rte_exit(EXIT_FAILURE,
-				"Unable to allocate memory.\n");
-
-		for (j = 0; j < nb_rx; j++)
-			pktmbuf_sw_copy(pkts_burst[j],
-				pkts_burst_copy[j]);
-
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)pkts_burst, nb_rx);
-
-		nb_enq = rte_ring_enqueue_burst(
-			rx_config->rx_to_tx_ring,
-			(void *)pkts_burst_copy, nb_rx, NULL);
-
-		/* Free any not enqueued packets. */
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)&pkts_burst_copy[nb_enq],
-			nb_rx - nb_enq);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			/* Perform packet hardware copy */
+			nb_enq = ioat_enqueue_packets(pkts_burst,
+				nb_rx, rx_config->ioat_ids[i]);
+			if (nb_enq > 0)
+				rte_ioat_do_copies(rx_config->ioat_ids[i]);
+		} else {
+			/* Perform packet software copy, free source packets */
+			int ret;
+			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst_copy, nb_rx);
+
+			if (unlikely(ret < 0))
+				rte_exit(EXIT_FAILURE,
+					"Unable to allocate memory.\n");
+
+			for (j = 0; j < nb_rx; j++)
+				pktmbuf_sw_copy(pkts_burst[j],
+					pkts_burst_copy[j]);
+
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst, nb_rx);
+
+			nb_enq = rte_ring_enqueue_burst(
+				rx_config->rx_to_tx_ring,
+				(void *)pkts_burst_copy, nb_rx, NULL);
+
+			/* Free any not enqueued packets. */
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&pkts_burst_copy[nb_enq],
+				nb_rx - nb_enq);
+		}
 	}
 }
 
@@ -169,31 +221,64 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 static void
 ioat_tx_port(struct rxtx_port_config *tx_config)
 {
-	uint32_t i, nb_dq = 0;
+	uint32_t i, j, nb_dq = 0;
+	struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
-	/* Deque the mbufs from rx_to_tx_ring. */
-	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
-		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		/* Deque the mbufs from IOAT device. */
+		for (i = 0; i < tx_config->nb_queues; i++) {
+			nb_dq = rte_ioat_completed_copies(
+				tx_config->ioat_ids[i], MAX_PKT_BURST,
+				(void *)mbufs_src, (void *)mbufs_dst);
 
-	if (nb_dq == 0)
-		return;
+			if (nb_dq == 0)
+				break;
 
-	/* Update macs if enabled */
-	if (mac_updating) {
-		for (i = 0; i < nb_dq; i++)
-			update_mac_addrs(mbufs_dst[i],
-				tx_config->rxtx_port);
-	}
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)mbufs_src, nb_dq);
 
-	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
-		0, (void *)mbufs_dst, nb_dq);
+			/* Update macs if enabled */
+			if (mac_updating) {
+				for (j = 0; j < nb_dq; j++)
+					update_mac_addrs(mbufs_dst[j],
+						tx_config->rxtx_port);
+			}
+
+			const uint16_t nb_tx = rte_eth_tx_burst(
+				tx_config->rxtx_port, 0,
+				(void *)mbufs_dst, nb_dq);
 
-	/* Free any unsent packets. */
-	if (unlikely(nb_tx < nb_dq))
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-		(void *)&mbufs_dst[nb_tx],
-			nb_dq - nb_tx);
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_dq))
+				rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&mbufs_dst[nb_tx],
+					nb_dq - nb_tx);
+		}
+	} else {
+		/* Deque the mbufs from rx_to_tx_ring. */
+		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+		if (nb_dq == 0)
+			return;
+
+		/* Update macs if enabled */
+		if (mac_updating) {
+			for (i = 0; i < nb_dq; i++)
+				update_mac_addrs(mbufs_dst[i],
+					tx_config->rxtx_port);
+		}
+
+		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+			0, (void *)mbufs_dst, nb_dq);
+
+		/* Free any unsent packets. */
+		if (unlikely(nb_tx < nb_dq))
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&mbufs_dst[nb_tx],
+				nb_dq - nb_tx);
+	}
 }
 
 /* Main rx and tx loop if only one slave lcore available */
@@ -389,6 +474,52 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+configure_rawdev_queue(uint32_t dev_id)
+{
+	struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
+	struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+	if (rte_rawdev_configure(dev_id, &info) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_configure()\n");
+	}
+	if (rte_rawdev_start(dev_id) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_start()\n");
+	}
+}
+
+static void
+assign_rawdevs(void)
+{
+	uint16_t nb_rawdev = 0, rdev_id = 0;
+	uint32_t i, j;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+			struct rte_rawdev_info rdev_info = { 0 };
+
+			do {
+				if (rdev_id == rte_rawdev_count())
+					goto end;
+				rte_rawdev_info_get(rdev_id++, &rdev_info);
+			} while (strcmp(rdev_info.driver_name,
+				IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+			cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+			configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+			++nb_rawdev;
+		}
+	}
+end:
+	if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+		rte_exit(EXIT_FAILURE,
+			"Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+			nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+	RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+}
+
 static void
 assign_rings(void)
 {
@@ -585,18 +716,29 @@ main(int argc, char **argv)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
 
-	assign_rings();
+	if (copy_mode == COPY_MODE_IOAT_NUM)
+		assign_rawdevs();
+	else /* copy_mode == COPY_MODE_SW_NUM */
+		assign_rings();
 
 	start_forwarding_cores();
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
 
+	uint32_t j;
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
-		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+				printf("Stopping rawdev %d\n",
+					cfg.ports[i].ioat_ids[j]);
+				rte_rawdev_stop(cfg.ports[i].ioat_ids[j]);
+			}
+		} else /* copy_mode == COPY_MODE_SW_NUM */
+			rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v4 4/6] examples/ioat: add two threads configuration
  2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                         ` (2 preceding siblings ...)
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
@ 2019-09-19  9:38       ` Marcin Baran
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 5/6] examples/ioat: add stats printing for each port Marcin Baran
                         ` (2 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-19  9:38 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++-----------
 1 file changed, 67 insertions(+), 25 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index c66ce7e49..4c51db6bd 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,
 	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
 		nb_rx - i);
 
-
 	return ret;
 }
 
@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
 	if (copy_mode == COPY_MODE_IOAT_NUM) {
-		/* Deque the mbufs from IOAT device. */
 		for (i = 0; i < tx_config->nb_queues; i++) {
+			/* Deque the mbufs from IOAT device. */
 			nb_dq = rte_ioat_completed_copies(
 				tx_config->ioat_ids[i], MAX_PKT_BURST,
 				(void *)mbufs_src, (void *)mbufs_dst);
@@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 					nb_dq - nb_tx);
 		}
 	} else {
-		/* Deque the mbufs from rx_to_tx_ring. */
-		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
-			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
-
-		if (nb_dq == 0)
-			return;
-
-		/* Update macs if enabled */
-		if (mac_updating) {
-			for (i = 0; i < nb_dq; i++)
-				update_mac_addrs(mbufs_dst[i],
-					tx_config->rxtx_port);
-		}
+		for (i = 0; i < tx_config->nb_queues; i++) {
+			/* Deque the mbufs from IOAT device. */
+			nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+				(void *)mbufs_dst, MAX_PKT_BURST, NULL);
 
-		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
-			0, (void *)mbufs_dst, nb_dq);
+			if (nb_dq == 0)
+				return;
 
-		/* Free any unsent packets. */
-		if (unlikely(nb_tx < nb_dq))
-			rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)&mbufs_dst[nb_tx],
-				nb_dq - nb_tx);
+			/* Update macs if enabled */
+			if (mac_updating) {
+				for (j = 0; j < nb_dq; j++)
+					update_mac_addrs(mbufs_dst[j],
+						tx_config->rxtx_port);
+			}
+
+			const uint16_t nb_tx =
+				rte_eth_tx_burst(tx_config->rxtx_port, 0,
+					(void *)mbufs_dst, nb_dq);
+
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_dq))
+				rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&mbufs_dst[nb_tx],
+					nb_dq - nb_tx);
+		}
 	}
 }
 
+/* Main rx processing loop for IOAT rawdev. */
+static void
+rx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_rx_port(&cfg.ports[i]);
+}
+
+/* Main tx processing loop for hardware copy. */
+static void
+tx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main tx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_tx_port(&cfg.ports[i]);
+}
+
 /* Main rx and tx loop if only one slave lcore available */
 static void
 rxtx_main_loop(void)
@@ -305,9 +337,19 @@ static void start_forwarding_cores(void)
 	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
 		__func__, rte_lcore_id());
 
-	lcore_id = rte_get_next_lcore(lcore_id, true, true);
-	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
-		NULL, lcore_id);
+	if (cfg.nb_lcores == 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+			NULL, lcore_id);
+	} else if (cfg.nb_lcores > 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+			NULL, lcore_id);
+
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+			lcore_id);
+	}
 }
 
 /* Display usage */
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v4 5/6] examples/ioat: add stats printing for each port
  2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                         ` (3 preceding siblings ...)
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 4/6] examples/ioat: add two threads configuration Marcin Baran
@ 2019-09-19  9:38       ` Marcin Baran
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-19  9:38 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added printing stats of ports each second.
The stats printing is done using master core.
The information provided informs about packets
received, dropped and send as well as statistics
of rawdev devices.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 241 insertions(+), 4 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 4c51db6bd..c6b994832 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -48,10 +48,27 @@ struct rxtx_transmission_config {
 	uint16_t nb_lcores;
 };
 
+/* per-port statistics struct */
+struct ioat_port_statistics {
+	uint64_t rx[RTE_MAX_ETHPORTS];
+	uint64_t tx[RTE_MAX_ETHPORTS];
+	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
+	uint64_t copy_dropped[RTE_MAX_ETHPORTS];
+};
+struct ioat_port_statistics port_statistics;
+
+struct total_statistics {
+	uint64_t total_packets_dropped;
+	uint64_t total_packets_tx;
+	uint64_t total_packets_rx;
+	uint64_t total_successful_enqueues;
+	uint64_t total_failed_enqueues;
+};
+
 typedef enum copy_mode_t {
 #define COPY_MODE_SW "sw"
 	COPY_MODE_SW_NUM,
-#define COPY_MODE_IOAT "rawdev"
+#define COPY_MODE_IOAT "hw"
 	COPY_MODE_IOAT_NUM,
 	COPY_MODE_INVALID_NUM,
 	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+/* Print out statistics for one port. */
+static void
+print_port_stats(uint16_t port_id)
+{
+	printf("\nStatistics for port %u ------------------------------"
+		"\nPackets sent: %34"PRIu64
+		"\nPackets received: %30"PRIu64
+		"\nPackets dropped on tx: %25"PRIu64
+		"\nPackets dropped on copy: %23"PRIu64,
+		port_id,
+		port_statistics.tx[port_id],
+		port_statistics.rx[port_id],
+		port_statistics.tx_dropped[port_id],
+		port_statistics.copy_dropped[port_id]);
+}
+
+/* Print out statistics for one IOAT rawdev device. */
+static void
+print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
+	unsigned int *ids_xstats, uint16_t nb_xstats,
+	struct rte_rawdev_xstats_name *names_xstats)
+{
+	uint16_t i;
+
+	printf("\nIOAT channel %u", dev_id);
+	for (i = 0; i < nb_xstats; i++)
+		printf("\n\t %s: %*"PRIu64,
+			names_xstats[ids_xstats[i]].name,
+			(int)(37 - strlen(names_xstats[ids_xstats[i]].name)),
+			xstats[i]);
+}
+
+static void
+print_total_stats(struct total_statistics *ts)
+{
+	printf("\nAggregate statistics ==============================="
+		"\nTotal packets Tx: %24"PRIu64" [pps]"
+		"\nTotal packets Rx: %24"PRIu64" [pps]"
+		"\nTotal packets dropped: %19"PRIu64" [pps]",
+		ts->total_packets_tx,
+		ts->total_packets_rx,
+		ts->total_packets_dropped);
+
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		printf("\nTotal IOAT successful enqueues: %8"PRIu64" [enq/s]"
+			"\nTotal IOAT failed enqueues: %12"PRIu64" [enq/s]",
+			ts->total_successful_enqueues,
+			ts->total_failed_enqueues);
+	}
+
+	printf("\n====================================================\n");
+}
+
+/* Print out statistics on packets dropped. */
+static void
+print_stats(char *prgname)
+{
+	struct total_statistics ts, delta_ts;
+	uint32_t i, port_id, dev_id;
+	struct rte_rawdev_xstats_name *names_xstats;
+	uint64_t *xstats;
+	unsigned int *ids_xstats, nb_xstats;
+	char status_string[120]; /* to print at the top of the output */
+	int status_strlen;
+
+
+	const char clr[] = { 27, '[', '2', 'J', '\0' };
+	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+	status_strlen = snprintf(status_string, sizeof(status_string),
+		"%s, ", prgname);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Worker Threads = %d, ",
+		rte_lcore_count() > 2 ? 2 : 1);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
+		COPY_MODE_SW : COPY_MODE_IOAT);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Updating MAC = %s, ", mac_updating ?
+		"enabled" : "disabled");
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Rx Queues = %d, ", nb_queues);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Ring Size = %d\n", ring_size);
+
+	/* Allocate memory for xstats names and values */
+	nb_xstats = rte_rawdev_xstats_names_get(
+		cfg.ports[0].ioat_ids[0], NULL, 0);
+
+	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
+	if (names_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat names memory\n");
+	}
+	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
+		names_xstats, nb_xstats);
+
+	ids_xstats = malloc(sizeof(*ids_xstats) * 2);
+	if (ids_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat ids_xstats memory\n");
+	}
+
+	xstats = malloc(sizeof(*xstats) * 2);
+	if (xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat memory\n");
+	}
+
+	/* Get failed/successful enqueues stats index */
+	ids_xstats[0] = ids_xstats[1] = nb_xstats;
+	for (i = 0; i < nb_xstats; i++) {
+		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
+			ids_xstats[0] = i;
+		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
+			ids_xstats[1] = i;
+		if (ids_xstats[0] < nb_xstats && ids_xstats[1] < nb_xstats)
+			break;
+	}
+	if (ids_xstats[0] == nb_xstats || ids_xstats[1] == nb_xstats) {
+		rte_exit(EXIT_FAILURE,
+			"Error getting failed/successful enqueues stats index\n");
+	}
+
+	memset(&ts, 0, sizeof(struct total_statistics));
+
+	while (!force_quit) {
+		/* Sleep for 1 second each round - init sleep allows reading
+		 * messages from app startup.
+		 */
+		sleep(1);
+
+		/* Clear screen and move to top left */
+		printf("%s%s", clr, topLeft);
+
+		memset(&delta_ts, 0, sizeof(struct total_statistics));
+
+		printf("%s", status_string);
+
+		for (i = 0; i < cfg.nb_ports; i++) {
+			port_id = cfg.ports[i].rxtx_port;
+			print_port_stats(port_id);
+
+			delta_ts.total_packets_dropped +=
+				port_statistics.tx_dropped[port_id]
+				+ port_statistics.copy_dropped[port_id];
+			delta_ts.total_packets_tx +=
+				port_statistics.tx[port_id];
+			delta_ts.total_packets_rx +=
+				port_statistics.rx[port_id];
+
+			if (copy_mode == COPY_MODE_IOAT_NUM) {
+				uint32_t j;
+
+				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+					dev_id = cfg.ports[i].ioat_ids[j];
+					rte_rawdev_xstats_get(dev_id,
+						ids_xstats, xstats, 2);
+
+					print_rawdev_stats(dev_id, xstats,
+						ids_xstats, 2, names_xstats);
+
+					delta_ts.total_failed_enqueues +=
+						xstats[ids_xstats[0]];
+					delta_ts.total_successful_enqueues +=
+						xstats[ids_xstats[1]];
+				}
+			}
+		}
+
+		delta_ts.total_packets_tx -= ts.total_packets_tx;
+		delta_ts.total_packets_rx -= ts.total_packets_rx;
+		delta_ts.total_packets_dropped -= ts.total_packets_dropped;
+		delta_ts.total_failed_enqueues -= ts.total_failed_enqueues;
+		delta_ts.total_successful_enqueues -=
+			ts.total_successful_enqueues;
+
+		printf("\n");
+		print_total_stats(&delta_ts);
+
+		ts.total_packets_tx += delta_ts.total_packets_tx;
+		ts.total_packets_rx += delta_ts.total_packets_rx;
+		ts.total_packets_dropped += delta_ts.total_packets_dropped;
+		ts.total_failed_enqueues += delta_ts.total_failed_enqueues;
+		ts.total_successful_enqueues +=
+			delta_ts.total_successful_enqueues;
+	}
+
+	free(names_xstats);
+	free(xstats);
+	free(ids_xstats);
+}
+
 static void
 update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
 {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
+		port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Perform packet hardware copy */
 			nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 				(void *)&pkts_burst_copy[nb_enq],
 				nb_rx - nb_enq);
 		}
+
+		port_statistics.copy_dropped[rx_config->rxtx_port] +=
+			(nb_rx - nb_enq);
 	}
 }
 
@@ -248,6 +468,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 				tx_config->rxtx_port, 0,
 				(void *)mbufs_dst, nb_dq);
 
+			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 			/* Free any unsent packets. */
 			if (unlikely(nb_tx < nb_dq))
 				rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -274,6 +496,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 				rte_eth_tx_burst(tx_config->rxtx_port, 0,
 					(void *)mbufs_dst, nb_dq);
 
+			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 			/* Free any unsent packets. */
 			if (unlikely(nb_tx < nb_dq))
 				rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)
 		"      When enabled:\n"
 		"       - The source MAC address is replaced by the TX port MAC address\n"
 		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
-		"  -c --copy-type CT: type of copy: sw|rawdev\n"
+		"  -c --copy-type CT: type of copy: sw|hw\n"
 		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
 		prgname);
 }
@@ -400,7 +624,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 	static const char short_options[] =
 		"p:"  /* portmask */
 		"q:"  /* number of RX queues per port */
-		"c:"  /* copy type (sw|rawdev) */
+		"c:"  /* copy type (sw|hw) */
 		"s:"  /* ring size */
 		;
 
@@ -452,7 +676,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 		case 'c':
 			copy_mode = ioat_parse_copy_mode(optarg);
 			if (copy_mode == COPY_MODE_INVALID_NUM) {
-				printf("Invalid copy type. Use: sw, rawdev\n");
+				printf("Invalid copy type. Use: sw, hw\n");
 				ioat_usage(prgname);
 				return -1;
 			}
@@ -673,6 +897,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
 
+	ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+		rte_eth_tx_buffer_count_callback,
+		&port_statistics.tx_dropped[portid]);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot set error callback for tx buffer on port %u\n",
+			portid);
+
 	/* Start device */
 	ret = rte_eth_dev_start(portid);
 	if (ret < 0)
@@ -749,6 +981,9 @@ main(int argc, char **argv)
 	RTE_ETH_FOREACH_DEV(portid)
 		port_init(portid, ioat_pktmbuf_pool, nb_queues);
 
+	/* Initialize port xstats */
+	memset(&port_statistics, 0, sizeof(port_statistics));
+
 	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
 		sleep(1);
 
@@ -764,6 +999,8 @@ main(int argc, char **argv)
 		assign_rings();
 
 	start_forwarding_cores();
+	/* master core prints stats while other cores forward */
+	print_stats(argv[0]);
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v4 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                         ` (4 preceding siblings ...)
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 5/6] examples/ioat: add stats printing for each port Marcin Baran
@ 2019-09-19  9:38       ` Marcin Baran
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  6 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-19  9:38 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran

Added guide for IOAT sample app usage and
code description.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 doc/guides/sample_app_ug/index.rst |   1 +
 doc/guides/sample_app_ug/intro.rst |   4 +
 doc/guides/sample_app_ug/ioat.rst  | 764 +++++++++++++++++++++++++++++
 3 files changed, 769 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst

diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index f23f8f59e..a6a1d9e7a 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -23,6 +23,7 @@ Sample Applications User Guides
     ip_reassembly
     kernel_nic_interface
     keep_alive
+    ioat
     l2_forward_crypto
     l2_forward_job_stats
     l2_forward_real_virtual
diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
index 90704194a..74462312f 100644
--- a/doc/guides/sample_app_ug/intro.rst
+++ b/doc/guides/sample_app_ug/intro.rst
@@ -91,6 +91,10 @@ examples are highlighted below.
   forwarding, or ``l3fwd`` application does forwarding based on Internet
   Protocol, IPv4 or IPv6 like a simple router.
 
+* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
+  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
+  copying packets between two threads.
+
 * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
   demonstrates how to distribute packets arriving on an Rx port to different
   cores for processing and transmission.
diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
new file mode 100644
index 000000000..69621673b
--- /dev/null
+++ b/doc/guides/sample_app_ug/ioat.rst
@@ -0,0 +1,764 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Intel Corporation.
+
+Sample Application of packet copying using Intel\ |reg| QuickData Technology
+============================================================================
+
+Overview
+--------
+
+This sample is intended as a demonstration of the basic components of a DPDK
+forwarding application and example of how to use IOAT driver API to make
+packets copies.
+
+Also while forwarding, the MAC addresses are affected as follows:
+
+*   The source MAC address is replaced by the TX port MAC address
+
+*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
+
+This application can be used to compare performance of using software packet
+copy with copy done using a DMA device for different sizes of packets.
+The example will print out statistics each second. The stats shows
+received/send packets and packets dropped or failed to copy.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``ioat`` sub-directory.
+
+
+Running the Application
+-----------------------
+
+In order to run the hardware copy application, the copying device
+needs to be bound to user-space IO driver.
+
+Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology*
+guide for information on using the driver.
+
+The application requires a number of command line options:
+
+.. code-block:: console
+
+    ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>]
+        [--[no-]mac-updating]
+
+where,
+
+*   p MASK: A hexadecimal bitmask of the ports to configure
+
+*   q NQ: Number of Rx queues used per port equivalent to CBDMA channels
+    per port
+
+*   c CT: Performed packet copy type: software (sw) or hardware using
+    DMA (hw)
+
+*   s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for
+    software copy mode
+
+*   --[no-]mac-updating: Whether MAC address of packets should be changed
+    or not
+
+The application can be launched in various configurations depending on
+provided parameters. Each port can use up to 2 lcores: one of them receives
+incoming traffic and makes a copy of each packet. The second lcore then
+updates MAC address and sends the copy. If one lcore per port is used,
+both operations are done sequentially. For each configuration an additional
+lcore is needed since master lcore in use which is responsible for
+configuration, statistics printing and safe deinitialization of all ports
+and devices.
+
+The application can use a maximum of 8 ports.
+
+To run the application in a Linux environment with 3 lcores (one of them
+is master lcore), 1 port (port 0), software copying and MAC updating issue
+the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
+
+To run the application in a Linux environment with 2 lcores (one of them
+is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC
+updating issue the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw
+
+Refer to the *DPDK Getting Started Guide* for general information on
+running applications and the Environment Abstraction Layer (EAL) options.
+
+Explanation
+-----------
+
+The following sections provide an explanation of the main components of the
+code.
+
+All DPDK library functions used in the sample code are prefixed with
+``rte_`` and are explained in detail in the *DPDK API Documentation*.
+
+
+The Main Function
+~~~~~~~~~~~~~~~~~
+
+The ``main()`` function performs the initialization and calls the execution
+threads for each lcore.
+
+The first task is to initialize the Environment Abstraction Layer (EAL).
+The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
+function. The value returned is the number of parsed arguments:
+
+.. code-block:: c
+
+    /* init EAL */
+    ret = rte_eal_init(argc, argv);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+
+
+The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
+used by the application:
+
+.. code-block:: c
+
+    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
+        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+        MIN_POOL_SIZE);
+
+    /* Create the mbuf pool */
+    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+        rte_socket_id());
+    if (ioat_pktmbuf_pool == NULL)
+        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+Mbufs are the packet buffer structure used by DPDK. They are explained in
+detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
+
+The ``main()`` function also initializes the ports:
+
+.. code-block:: c
+
+    /* Initialise each port */
+    RTE_ETH_FOREACH_DEV(portid) {
+        port_init(portid, ioat_pktmbuf_pool);
+    }
+
+Each port is configured using ``port_init()``:
+
+.. code-block:: c
+
+     /*
+     * Initializes a given port using global settings and with the RX buffers
+     * coming from the mbuf_pool passed as a parameter.
+     */
+    static inline void
+    port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+    {
+        /* configuring port to use RSS for multiple RX queues */
+        static const struct rte_eth_conf port_conf = {
+            .rxmode = {
+                .mq_mode        = ETH_MQ_RX_RSS,
+                .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+            },
+            .rx_adv_conf = {
+                .rss_conf = {
+                    .rss_key = NULL,
+                    .rss_hf = ETH_RSS_PROTO_MASK,
+                }
+            }
+        };
+
+        struct rte_eth_rxconf rxq_conf;
+        struct rte_eth_txconf txq_conf;
+        struct rte_eth_conf local_port_conf = port_conf;
+        struct rte_eth_dev_info dev_info;
+        int ret, i;
+
+        /* Skip ports that are not enabled */
+        if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+            printf("Skipping disabled port %u\n", portid);
+            return;
+        }
+
+        /* Init port */
+        printf("Initializing port %u... ", portid);
+        fflush(stdout);
+        rte_eth_dev_info_get(portid, &dev_info);
+        local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+            dev_info.flow_type_rss_offloads;
+        if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+            local_port_conf.txmode.offloads |=
+                DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+        ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE, "Cannot configure device:"
+                " err=%d, port=%u\n", ret, portid);
+
+        ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+                            &nb_txd);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "Cannot adjust number of descriptors: err=%d, port=%u\n",
+                ret, portid);
+
+        rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+        /* Init Rx queues */
+        rxq_conf = dev_info.default_rxconf;
+        rxq_conf.offloads = local_port_conf.rxmode.offloads;
+        for (i = 0; i < nb_queues; i++) {
+            ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+                rte_eth_dev_socket_id(portid), &rxq_conf,
+                mbuf_pool);
+            if (ret < 0)
+                rte_exit(EXIT_FAILURE,
+                    "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+                    ret, portid, i);
+        }
+
+        /* Init one TX queue on each port */
+        txq_conf = dev_info.default_txconf;
+        txq_conf.offloads = local_port_conf.txmode.offloads;
+        ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+                rte_eth_dev_socket_id(portid),
+                &txq_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "rte_eth_tx_queue_setup:err=%d,port=%u\n",
+                ret, portid);
+
+        /* Initialize TX buffers */
+        tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+                RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+                rte_eth_dev_socket_id(portid));
+        if (tx_buffer[portid] == NULL)
+            rte_exit(EXIT_FAILURE,
+                "Cannot allocate buffer for tx on port %u\n",
+                portid);
+
+        rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+        ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+                rte_eth_tx_buffer_count_callback,
+                &port_statistics.tx_dropped[portid]);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "Cannot set error callback for tx buffer on port %u\n",
+                portid);
+
+        /* Start device */
+        ret = rte_eth_dev_start(portid);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "rte_eth_dev_start:err=%d, port=%u\n",
+                ret, portid);
+
+        rte_eth_promiscuous_enable(portid);
+
+        printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+                portid,
+                ioat_ports_eth_addr[portid].addr_bytes[0],
+                ioat_ports_eth_addr[portid].addr_bytes[1],
+                ioat_ports_eth_addr[portid].addr_bytes[2],
+                ioat_ports_eth_addr[portid].addr_bytes[3],
+                ioat_ports_eth_addr[portid].addr_bytes[4],
+                ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+        cfg.ports[cfg.nb_ports].rxtx_port = portid;
+        cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+    }
+
+The Ethernet ports are configured with local settings using the
+``rte_eth_dev_configure()`` function and the ``port_conf`` struct.
+The RSS is enabled so that multiple Rx queues could be used for
+packet receiving and copying by multiple CBDMA channels per port:
+
+.. code-block:: c
+
+    /* configuring port to use RSS for multiple RX queues */
+    static const struct rte_eth_conf port_conf = {
+        .rxmode = {
+            .mq_mode        = ETH_MQ_RX_RSS,
+            .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+        },
+        .rx_adv_conf = {
+            .rss_conf = {
+                .rss_key = NULL,
+                .rss_hf = ETH_RSS_PROTO_MASK,
+            }
+        }
+    };
+
+For this example the ports are set up with the number of Rx queues provided
+with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()``
+and ``rte_eth_tx_queue_setup()`` functions.
+
+The Ethernet port is then started:
+
+.. code-block:: c
+
+    ret = rte_eth_dev_start(portid);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
+            ret, portid);
+
+
+Finally the Rx port is set in promiscuous mode:
+
+.. code-block:: c
+
+    rte_eth_promiscuous_enable(portid);
+
+
+After that each port application assigns resources needed.
+
+.. code-block:: c
+
+    check_link_status(ioat_enabled_port_mask);
+
+    if (!cfg.nb_ports) {
+        rte_exit(EXIT_FAILURE,
+            "All available ports are disabled. Please set portmask.\n");
+    }
+
+    /* Check if there is enough lcores for all ports. */
+    cfg.nb_lcores = rte_lcore_count() - 1;
+    if (cfg.nb_lcores < 1)
+        rte_exit(EXIT_FAILURE,
+            "There should be at least one slave lcore.\n");
+
+    ret = 0;
+
+    if (copy_mode == COPY_MODE_IOAT_NUM) {
+        assign_rawdevs();
+    } else /* copy_mode == COPY_MODE_SW_NUM */ {
+        assign_rings();
+    }
+
+A link status is checked of each port enabled by port mask
+using ``check_link_status()`` function.
+
+.. code-block:: c
+
+    /* check link status, return true if at least one port is up */
+    static int
+    check_link_status(uint32_t port_mask)
+    {
+        uint16_t portid;
+        struct rte_eth_link link;
+        int retval = 0;
+
+        printf("\nChecking link status\n");
+        RTE_ETH_FOREACH_DEV(portid) {
+            if ((port_mask & (1 << portid)) == 0)
+                continue;
+
+            memset(&link, 0, sizeof(link));
+            rte_eth_link_get(portid, &link);
+
+            /* Print link status */
+            if (link.link_status) {
+                printf(
+                    "Port %d Link Up. Speed %u Mbps - %s\n",
+                    portid, link.link_speed,
+                    (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+                    ("full-duplex") : ("half-duplex\n"));
+                retval = 1;
+            } else
+                printf("Port %d Link Down\n", portid);
+        }
+        return retval;
+    }
+
+Depending on mode set (whether copy should be done by software or by hardware)
+special structures are assigned to each port. If software copy was chosen,
+application have to assign ring structures for packet exchanging between lcores
+assigned to ports.
+
+.. code-block:: c
+
+    static void
+    assign_rings(void)
+    {
+        uint32_t i;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            char ring_name[20];
+
+            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
+            /* Create ring for inter core communication */
+            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+                    ring_name, ring_size,
+                    rte_socket_id(), RING_F_SP_ENQ);
+
+            if (cfg.ports[i].rx_to_tx_ring == NULL)
+                rte_exit(EXIT_FAILURE, "%s\n",
+                        rte_strerror(rte_errno));
+        }
+    }
+
+
+When using hardware copy each Rx queue of the port is assigned an
+IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API
+functions:
+
+.. code-block:: c
+
+    static void
+    assign_rawdevs(void)
+    {
+        uint16_t nb_rawdev = 0, rdev_id = 0;
+        uint32_t i, j;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+                struct rte_rawdev_info rdev_info = { 0 };
+
+                do {
+                    if (rdev_id == rte_rawdev_count())
+                        goto end;
+                    rte_rawdev_info_get(rdev_id++, &rdev_info);
+                } while (strcmp(rdev_info.driver_name,
+                    IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+                cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+                configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+                ++nb_rawdev;
+            }
+        }
+    end:
+        if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+            rte_exit(EXIT_FAILURE,
+                "Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+                nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+    }
+
+
+The initialization of hardware device is done by ``rte_rawdev_configure()``
+function and ``rte_rawdev_info`` struct. After configuration the device is
+started using ``rte_rawdev_start()`` function. Each of the above operations
+is done in ``configure_rawdev_queue()``.
+
+.. code-block:: c
+
+    static void
+    configure_rawdev_queue(uint32_t dev_id)
+    {
+        struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+        /* Configure hardware copy device */
+        dev_config.ring_size = ring_size;
+
+        if (rte_rawdev_configure(dev_id, &info) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_configure()\n");
+        }
+        rte_rawdev_info_get(dev_id, &info);
+        if (dev_config.ring_size != ring_size) {
+            rte_exit(EXIT_FAILURE,
+                "Error, ring size is not %d (%d)\n",
+                ring_size, (int)dev_config.ring_size);
+        }
+        if (rte_rawdev_start(dev_id) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_start()\n");
+        }
+    }
+
+If initialization is successful memory for hardware device
+statistics is allocated.
+
+Finally ``main()`` functions starts all processing lcores and starts
+printing stats in a loop on master lcore. The application can be
+interrupted and closed using ``Ctrl-C``. The master lcore waits for
+all slave processes to finish, deallocates resources and exits.
+
+The processing lcores launching function are described below.
+
+The Lcores Launching Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As described above ``main()`` function invokes ``start_forwarding_cores()``
+function in order to start processing for each lcore:
+
+.. code-block:: c
+
+    static void start_forwarding_cores(void)
+    {
+        uint32_t lcore_id = rte_lcore_id();
+
+        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+                __func__, rte_lcore_id());
+
+        if (cfg.nb_lcores == 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+                NULL, lcore_id);
+        } else if (cfg.nb_lcores > 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+                NULL, lcore_id);
+
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+                lcore_id);
+        }
+    }
+
+The function launches Rx/Tx processing functions on configured lcores
+for each port using ``rte_eal_remote_launch()``. The configured ports,
+their number and number of assigned lcores are stored in user-defined
+``rxtx_transmission_config`` struct that is initialized before launching
+tasks:
+
+.. code-block:: c
+
+    struct rxtx_transmission_config {
+        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+        uint16_t nb_ports;
+        uint16_t nb_lcores;
+    };
+
+The Lcores Processing Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For receiving packets on each port an ``ioat_rx_port()`` function is used.
+The function receives packets on each configured Rx queue. Depending on mode
+the user chose, it will enqueue packets to IOAT rawdev channels and then invoke
+copy process (hardware copy), or perform software copy of each packet using
+``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring:
+
+.. code-block:: c
+
+    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+    static void
+    ioat_rx_port(struct rxtx_port_config *rx_config)
+    {
+        uint32_t nb_rx, nb_enq, i, j;
+        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+        for (i = 0; i < rx_config->nb_queues; i++) {
+
+            nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+                pkts_burst, MAX_PKT_BURST);
+
+            if (nb_rx == 0)
+                continue;
+
+            port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
+            if (copy_mode == COPY_MODE_IOAT_NUM) {
+                /* Perform packet hardware copy */
+                nb_enq = ioat_enqueue_packets(pkts_burst,
+                    nb_rx, rx_config->ioat_ids[i]);
+                if (nb_enq > 0)
+                    rte_ioat_do_copies(rx_config->ioat_ids[i]);
+            } else {
+                /* Perform packet software copy, free source packets */
+                int ret;
+                struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+                ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst_copy, nb_rx);
+
+                if (unlikely(ret < 0))
+                    rte_exit(EXIT_FAILURE,
+                        "Unable to allocate memory.\n");
+
+                for (j = 0; j < nb_rx; j++)
+                    pktmbuf_sw_copy(pkts_burst[j],
+                        pkts_burst_copy[j]);
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst, nb_rx);
+
+                nb_enq = rte_ring_enqueue_burst(
+                    rx_config->rx_to_tx_ring,
+                    (void *)pkts_burst_copy, nb_rx, NULL);
+
+                /* Free any not enqueued packets. */
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&pkts_burst_copy[nb_enq],
+                    nb_rx - nb_enq);
+            }
+
+            port_statistics.copy_dropped[rx_config->rxtx_port] +=
+                (nb_rx - nb_enq);
+        }
+    }
+
+The packets are received in burst mode using ``rte_eth_rx_burst()``
+function. When using hardware copy mode the packets are enqueued in
+copying device's buffer using ``ioat_enqueue_packets()`` which calls
+``rte_ioat_enqueue_copy()``. When all received packets are in the
+buffer the copies are invoked by calling ``rte_ioat_do_copies()``.
+Function ``rte_ioat_enqueue_copy()`` operates on physical address of
+the packet. Structure ``rte_mbuf`` contains only physical address to
+start of the data buffer (``buf_iova``). Thus the address is shifted
+by ``addr_offset`` value in order to get pointer to ``rearm_data``
+member of ``rte_mbuf``. That way the packet is copied all at once
+(with data and metadata).
+
+.. code-block:: c
+
+    static uint32_t
+    ioat_enqueue_packets(struct rte_mbuf **pkts,
+        uint32_t nb_rx, uint16_t dev_id)
+    {
+        int ret;
+        uint32_t i;
+        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+            &pkts[0]->rearm_data);
+
+        ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                (void *)pkts_copy, nb_rx);
+
+        if (unlikely(ret < 0))
+            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+        for (i = 0; i < nb_rx; i++) {
+            /* Perform data copy */
+            ret = rte_ioat_enqueue_copy(dev_id,
+                pkts[i]->buf_iova
+                    - addr_offset,
+                pkts_copy[i]->buf_iova
+                    - addr_offset,
+                rte_pktmbuf_data_len(pkts[i])
+                    + addr_offset,
+                (uintptr_t)pkts[i],
+                (uintptr_t)pkts_copy[i],
+                0 /* nofence */);
+
+            if (ret != 1)
+                break;
+        }
+
+        ret = i;
+        /* Free any not enqueued packets. */
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+            nb_rx - i);
+
+        return ret;
+    }
+
+
+All done copies are processed by ``ioat_tx_port()`` function. When using
+hardware copy mode the function invokes ``rte_ioat_completed_copies()``
+on each assigned IOAT channel to gather copied packets. If software copy
+mode is used the function dequeues copied packets from the rte_ring. Then each
+packet MAC address is changed if it was enabled. After that copies are sent
+in burst mode using `` rte_eth_tx_burst()``.
+
+
+.. code-block:: c
+
+    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
+    static void
+    ioat_tx_port(struct rxtx_port_config *tx_config)
+    {
+        uint32_t i, j, nb_dq = 0;
+        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
+        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+        if (copy_mode == COPY_MODE_IOAT_NUM) {
+            for (i = 0; i < tx_config->nb_queues; i++) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ioat_completed_copies(
+                    tx_config->ioat_ids[i], MAX_PKT_BURST,
+                    (void *)mbufs_src, (void *)mbufs_dst);
+
+                if (nb_dq == 0)
+                    break;
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)mbufs_src, nb_dq);
+
+                /* Update macs if enabled */
+                if (mac_updating) {
+                    for (j = 0; j < nb_dq; j++)
+                        update_mac_addrs(mbufs_dst[j],
+                            tx_config->rxtx_port);
+                }
+
+                const uint16_t nb_tx = rte_eth_tx_burst(
+                    tx_config->rxtx_port, 0,
+                    (void *)mbufs_dst, nb_dq);
+
+                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+                /* Free any unsent packets. */
+                if (unlikely(nb_tx < nb_dq))
+                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&mbufs_dst[nb_tx],
+                        nb_dq - nb_tx);
+            }
+        }
+        else {
+            for (i = 0; i < tx_config->nb_queues; i++) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+                    (void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+                if (nb_dq == 0)
+                    return;
+
+                /* Update macs if enabled */
+                if (mac_updating) {
+                    for (j = 0; j < nb_dq; j++)
+                        update_mac_addrs(mbufs_dst[j],
+                            tx_config->rxtx_port);
+                }
+
+                const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+                    0, (void *)mbufs_dst, nb_dq);
+
+                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+                /* Free any unsent packets. */
+                if (unlikely(nb_tx < nb_dq))
+                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&mbufs_dst[nb_tx],
+                        nb_dq - nb_tx);
+            }
+        }
+    }
+
+The Packet Copying Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to perform packet copy there is a user-defined function
+``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
+metadata from source packet to new mbuf, and then copying a data
+chunk of source packet. Both memory copies are done using
+``rte_memcpy()``:
+
+.. code-block:: c
+
+    static inline void
+    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+    {
+        /* Copy packet metadata */
+        rte_memcpy(&dst->rearm_data,
+            &src->rearm_data,
+            offsetof(struct rte_mbuf, cacheline1)
+                - offsetof(struct rte_mbuf, rearm_data));
+
+        /* Copy packet data */
+        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+            rte_pktmbuf_mtod(src, char *), src->data_len);
+    }
+
+The metadata in this example is copied from ``rearm_data`` member of
+``rte_mbuf`` struct up to ``cacheline1``.
+
+In order to understand why software packet copying is done as shown
+above please refer to the "Mbuf Library" section of the
+*DPDK Programmer's Guide*.
\ No newline at end of file
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v4 1/6] examples/ioat: create sample app on ioat driver usage
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 1/6] examples/ioat: create " Marcin Baran
@ 2019-09-19 14:44         ` Bruce Richardson
  2019-09-19 14:46           ` Baran, MarcinX
  0 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-19 14:44 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, Pawel Modrak

On Thu, Sep 19, 2019 at 11:38:45AM +0200, Marcin Baran wrote:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> 
> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware
> copy, changes packets' MAC addresses (if enabled) and
> forwards them. The change covers ports initialization,
> closing connection and argument parsing.
> 
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
<snip>
> --- /dev/null
> +++ b/examples/ioat/meson.build
> @@ -0,0 +1,23 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2019 Intel Corporation
> +
> +# meson file, for building this example as part of a main DPDK build.
> +#
> +# To build this example as a standalone application with an already-installed
> +# DPDK instance, use 'make'
> +
> +if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
> +	build = false
> +	subdir_done()
> +endif
> +
> +if dpdk_conf.get('RTE_LIBRTE_PMD_IOAT_RAWDEV', 0) != 1
> +	build = false
> +	subdir_done()
> +endif
> +
> +deps += ['pmd_ioat']
> +
Since commit '54dcfdee85aa ("drivers/raw: standardize naming")' the naming
has changed a little so this needs to be "rawdev_ioat" rather than
"pmd_ioat". I believe this is causing some of the CI error reports.

Regards,
/Bruce

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v4 1/6] examples/ioat: create sample app on ioat driver usage
  2019-09-19 14:44         ` Bruce Richardson
@ 2019-09-19 14:46           ` Baran, MarcinX
  0 siblings, 0 replies; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-19 14:46 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev, Modrak, PawelX



-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Thursday, September 19, 2019 4:44 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org; Modrak, PawelX <pawelx.modrak@intel.com>
Subject: Re: [PATCH v4 1/6] examples/ioat: create sample app on ioat driver usage

On Thu, Sep 19, 2019 at 11:38:45AM +0200, Marcin Baran wrote:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> 
> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware copy, changes 
> packets' MAC addresses (if enabled) and forwards them. The change 
> covers ports initialization, closing connection and argument parsing.
> 
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
<snip>
> --- /dev/null
> +++ b/examples/ioat/meson.build
> @@ -0,0 +1,23 @@
> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019 Intel 
> +Corporation
> +
> +# meson file, for building this example as part of a main DPDK build.
> +#
> +# To build this example as a standalone application with an 
> +already-installed # DPDK instance, use 'make'
> +
> +if not dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
> +	build = false
> +	subdir_done()
> +endif
> +
> +if dpdk_conf.get('RTE_LIBRTE_PMD_IOAT_RAWDEV', 0) != 1
> +	build = false
> +	subdir_done()
> +endif
> +
> +deps += ['pmd_ioat']
> +
Since commit '54dcfdee85aa ("drivers/raw: standardize naming")' the naming has changed a little so this needs to be "rawdev_ioat" rather than "pmd_ioat". I believe this is causing some of the CI error reports.
Regards,
/Bruce

[MARCIN] Yes, I noticed that just now. I am generating new patches with the fix.

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                         ` (5 preceding siblings ...)
  2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
@ 2019-09-20  7:37       ` Marcin Baran
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 1/6] examples/ioat: create " Marcin Baran
                           ` (7 more replies)
  6 siblings, 8 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-20  7:37 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The patch includes sample application as
well as it's guide.

v5:
 - change dependency name from 'pmd_ioat' to 'rawdev_ioat'
   fixing build errors (branch diverged from master)

v4:
 - fix meson build support check

v3:
 - add check for meson build whether IOAT is supported

v2:
 - change printing total stats to be deltas
 - update documentation
 - fix 1 thread/sw copy/multiple Rx queues packet dropping
 - divide patch into several presenting functional change

Marcin Baran (4):
  examples/ioat: add software copy support
  examples/ioat: add rawdev copy mode support
  examples/ioat: add stats printing for each port
  doc/guides/: provide IOAT sample app guide

Pawel Modrak (2):
  examples/ioat: create sample app on ioat driver usage
  examples/ioat: add two threads configuration

 doc/guides/sample_app_ug/index.rst |    1 +
 doc/guides/sample_app_ug/intro.rst |    4 +
 doc/guides/sample_app_ug/ioat.rst  |  764 +++++++++++++++++++++
 examples/Makefile                  |    3 +
 examples/ioat/Makefile             |   54 ++
 examples/ioat/ioatfwd.c            | 1025 ++++++++++++++++++++++++++++
 examples/ioat/meson.build          |   15 +
 examples/meson.build               |    1 +
 8 files changed, 1867 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v5 1/6] examples/ioat: create sample app on ioat driver usage
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
@ 2019-09-20  7:37         ` Marcin Baran
  2019-09-27  9:58           ` Bruce Richardson
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support Marcin Baran
                           ` (6 subsequent siblings)
  7 siblings, 1 reply; 76+ messages in thread
From: Marcin Baran @ 2019-09-20  7:37 UTC (permalink / raw)
  To: dev; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The change covers ports initialization,
closing connection and argument parsing.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/Makefile         |   3 +
 examples/ioat/Makefile    |  54 +++++
 examples/ioat/ioatfwd.c   | 439 ++++++++++++++++++++++++++++++++++++++
 examples/ioat/meson.build |  15 ++
 examples/meson.build      |   1 +
 5 files changed, 512 insertions(+)
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

diff --git a/examples/Makefile b/examples/Makefile
index de11dd487..3cb313d7d 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation
 DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify
 DIRS-y += flow_filtering
 DIRS-y += helloworld
+ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y)
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat
+endif
 DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline
 ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
 DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
diff --git a/examples/ioat/Makefile b/examples/ioat/Makefile
new file mode 100644
index 000000000..2a4d1da2d
--- /dev/null
+++ b/examples/ioat/Makefile
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# binary name
+APP = ioatfwd
+
+# all source are stored in SRCS-y
+SRCS-y := ioatfwd.c
+
+# Build using pkg-config variables if possible
+ifeq ($(shell pkg-config --exists libdpdk && echo 0),0)
+
+all: shared
+.PHONY: shared static
+shared: build/$(APP)-shared
+	ln -sf $(APP)-shared build/$(APP)
+static: build/$(APP)-static
+	ln -sf $(APP)-static build/$(APP)
+
+PC_FILE := $(shell pkg-config --path libdpdk)
+CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
+LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
+
+build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
+
+build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
+
+build:
+	@mkdir -p $@
+
+.PHONY: clean
+clean:
+	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
+	test -d build && rmdir -p build || true
+
+else # Build using legacy build system
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, detect a build directory, by looking for a path with a .config
+RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config)))))
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+endif
diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
new file mode 100644
index 000000000..977ea6a61
--- /dev/null
+++ b/examples/ioat/ioatfwd.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#include <stdint.h>
+#include <getopt.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+#include <rte_ethdev.h>
+#include <rte_rawdev.h>
+#include <rte_ioat_rawdev.h>
+
+/* size of ring used for software copying between rx and tx. */
+#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
+#define MAX_PKT_BURST 32
+#define MEMPOOL_CACHE_SIZE 512
+#define MIN_POOL_SIZE 65536U
+#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
+#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
+#define CMD_LINE_OPT_PORTMASK "portmask"
+#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
+#define CMD_LINE_OPT_COPY_TYPE "copy-type"
+#define CMD_LINE_OPT_RING_SIZE "ring-size"
+
+/* configurable number of RX/TX ring descriptors */
+#define RX_DEFAULT_RINGSIZE 1024
+#define TX_DEFAULT_RINGSIZE 1024
+
+/* max number of RX queues per port */
+#define MAX_RX_QUEUES_COUNT 8
+
+struct rxtx_port_config {
+	/* common config */
+	uint16_t rxtx_port;
+	uint16_t nb_queues;
+	/* for software copy mode */
+	struct rte_ring *rx_to_tx_ring;
+	/* for IOAT rawdev copy mode */
+	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT];
+};
+
+struct rxtx_transmission_config {
+	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+	uint16_t nb_ports;
+	uint16_t nb_lcores;
+};
+
+typedef enum copy_mode_t {
+#define COPY_MODE_SW "sw"
+	COPY_MODE_SW_NUM,
+#define COPY_MODE_IOAT "rawdev"
+	COPY_MODE_IOAT_NUM,
+	COPY_MODE_INVALID_NUM,
+	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
+} copy_mode_t;
+
+/* mask of enabled ports */
+static uint32_t ioat_enabled_port_mask;
+
+/* number of RX queues per port */
+static uint16_t nb_queues = 1;
+
+/* MAC updating enabled by default. */
+static int mac_updating = 1;
+
+/* hardare copy mode enabled by default. */
+static copy_mode_t copy_mode = COPY_MODE_IOAT_NUM;
+
+/* size of IOAT rawdev ring for hardware copy mode or
+ * rte_ring for software copy mode
+ */
+static unsigned short ring_size = 2048;
+
+/* global transmission config */
+struct rxtx_transmission_config cfg;
+
+/* configurable number of RX/TX ring descriptors */
+static uint16_t nb_rxd = RX_DEFAULT_RINGSIZE;
+static uint16_t nb_txd = TX_DEFAULT_RINGSIZE;
+
+static volatile bool force_quit;
+
+/* ethernet addresses of ports */
+static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
+struct rte_mempool *ioat_pktmbuf_pool;
+
+/* Display usage */
+static void
+ioat_usage(const char *prgname)
+{
+	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
+		"  -p --portmask: hexadecimal bitmask of ports to configure\n"
+		"  -q NQ: number of RX queues per port (default is 1)\n"
+		"  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
+		"      When enabled:\n"
+		"       - The source MAC address is replaced by the TX port MAC address\n"
+		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
+		"  -c --copy-type CT: type of copy: sw|rawdev\n"
+		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
+	       prgname);
+}
+
+static int
+ioat_parse_portmask(const char *portmask)
+{
+	char *end = NULL;
+	unsigned long pm;
+
+	/* Parse hexadecimal string */
+	pm = strtoul(portmask, &end, 16);
+	if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+
+	return pm;
+}
+
+static copy_mode_t
+ioat_parse_copy_mode(const char *copy_mode)
+{
+	if (strcmp(copy_mode, COPY_MODE_SW) == 0)
+		return COPY_MODE_SW_NUM;
+	else if (strcmp(copy_mode, COPY_MODE_IOAT) == 0)
+		return COPY_MODE_IOAT_NUM;
+
+	return COPY_MODE_INVALID_NUM;
+}
+
+/* Parse the argument given in the command line of the application */
+static int
+ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
+{
+	static const char short_options[] =
+		"p:"  /* portmask */
+		"q:"  /* number of RX queues per port */
+		"c:"  /* copy type (sw|rawdev) */
+		"s:"  /* ring size */
+		;
+
+	static const struct option lgopts[] = {
+		{CMD_LINE_OPT_MAC_UPDATING, no_argument, &mac_updating, 1},
+		{CMD_LINE_OPT_NO_MAC_UPDATING, no_argument, &mac_updating, 0},
+		{CMD_LINE_OPT_PORTMASK, required_argument, NULL, 'p'},
+		{CMD_LINE_OPT_NB_QUEUE, required_argument, NULL, 'q'},
+		{CMD_LINE_OPT_COPY_TYPE, required_argument, NULL, 'c'},
+		{CMD_LINE_OPT_RING_SIZE, required_argument, NULL, 's'},
+		{NULL, 0, 0, 0}
+	};
+
+	const unsigned int default_port_mask = (1 << nb_ports) - 1;
+	int opt, ret;
+	char **argvopt;
+	int option_index;
+	char *prgname = argv[0];
+
+	ioat_enabled_port_mask = default_port_mask;
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, short_options,
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		/* portmask */
+		case 'p':
+			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
+			if (ioat_enabled_port_mask & ~default_port_mask ||
+					ioat_enabled_port_mask <= 0) {
+				printf("Invalid portmask, %s, suggest 0x%x\n",
+						optarg, default_port_mask);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'q':
+			nb_queues = atoi(optarg);
+			if (nb_queues == 0 || nb_queues > MAX_RX_QUEUES_COUNT) {
+				printf("Invalid RX queues number %s. Max %u\n",
+					optarg, MAX_RX_QUEUES_COUNT);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'c':
+			copy_mode = ioat_parse_copy_mode(optarg);
+			if (copy_mode == COPY_MODE_INVALID_NUM) {
+				printf("Invalid copy type. Use: sw, rawdev\n");
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 's':
+			ring_size = atoi(optarg);
+			if (ring_size == 0) {
+				printf("Invalid ring size, %s.\n", optarg);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		/* long options */
+		case 0:
+			break;
+
+		default:
+			ioat_usage(prgname);
+			return -1;
+		}
+	}
+
+	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
+	if (optind >= 0)
+		argv[optind-1] = prgname;
+
+	ret = optind-1;
+	optind = 1; /* reset getopt lib */
+	return ret;
+}
+
+/* check link status, return true if at least one port is up */
+static int
+check_link_status(uint32_t port_mask)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	int retval = 0;
+
+	printf("\nChecking link status\n");
+	RTE_ETH_FOREACH_DEV(portid) {
+		if ((port_mask & (1 << portid)) == 0)
+			continue;
+
+		memset(&link, 0, sizeof(link));
+		rte_eth_link_get(portid, &link);
+
+		/* Print link status */
+		if (link.link_status) {
+			printf(
+				"Port %d Link Up. Speed %u Mbps - %s\n",
+				portid, link.link_speed,
+				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+				("full-duplex") : ("half-duplex\n"));
+			retval = 1;
+		} else
+			printf("Port %d Link Down\n", portid);
+	}
+	return retval;
+}
+
+/*
+ * Initializes a given port using global settings and with the RX buffers
+ * coming from the mbuf_pool passed as a parameter.
+ */
+static inline void
+port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+{
+	/* configuring port to use RSS for multiple RX queues */
+	static const struct rte_eth_conf port_conf = {
+		.rxmode = {
+			.mq_mode        = ETH_MQ_RX_RSS,
+			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
+		},
+		.rx_adv_conf = {
+			.rss_conf = {
+				.rss_key = NULL,
+				.rss_hf = ETH_RSS_PROTO_MASK,
+			}
+		}
+	};
+
+	struct rte_eth_rxconf rxq_conf;
+	struct rte_eth_txconf txq_conf;
+	struct rte_eth_conf local_port_conf = port_conf;
+	struct rte_eth_dev_info dev_info;
+	int ret, i;
+
+	/* Skip ports that are not enabled */
+	if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+		printf("Skipping disabled port %u\n", portid);
+		return;
+	}
+
+	/* Init port */
+	printf("Initializing port %u... ", portid);
+	fflush(stdout);
+	rte_eth_dev_info_get(portid, &dev_info);
+	local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+		dev_info.flow_type_rss_offloads;
+	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+		local_port_conf.txmode.offloads |=
+			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Cannot configure device:"
+			" err=%d, port=%u\n", ret, portid);
+
+	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+						&nb_txd);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot adjust number of descriptors: err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+	/* Init RX queues */
+	rxq_conf = dev_info.default_rxconf;
+	rxq_conf.offloads = local_port_conf.rxmode.offloads;
+	for (i = 0; i < nb_queues; i++) {
+		ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+			rte_eth_dev_socket_id(portid), &rxq_conf,
+			mbuf_pool);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE,
+				"rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+				ret, portid, i);
+	}
+
+	/* Init one TX queue on each port */
+	txq_conf = dev_info.default_txconf;
+	txq_conf.offloads = local_port_conf.txmode.offloads;
+	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+			rte_eth_dev_socket_id(portid),
+			&txq_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
+			ret, portid);
+
+	/* Initialize TX buffers */
+	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+			rte_eth_dev_socket_id(portid));
+	if (tx_buffer[portid] == NULL)
+		rte_exit(EXIT_FAILURE,
+			"Cannot allocate buffer for tx on port %u\n",
+			portid);
+
+	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+	/* Start device */
+	ret = rte_eth_dev_start(portid);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_dev_start:err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_promiscuous_enable(portid);
+
+	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+			portid,
+			ioat_ports_eth_addr[portid].addr_bytes[0],
+			ioat_ports_eth_addr[portid].addr_bytes[1],
+			ioat_ports_eth_addr[portid].addr_bytes[2],
+			ioat_ports_eth_addr[portid].addr_bytes[3],
+			ioat_ports_eth_addr[portid].addr_bytes[4],
+			ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+	cfg.ports[cfg.nb_ports].rxtx_port = portid;
+	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\n\nSignal %d received, preparing to exit...\n",
+			signum);
+		force_quit = true;
+	}
+}
+
+int
+main(int argc, char **argv)
+{
+	int ret;
+	uint16_t nb_ports, portid;
+	uint32_t i;
+	unsigned int nb_mbufs;
+
+	/* Init EAL */
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+	argc -= ret;
+	argv += ret;
+
+	force_quit = false;
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	nb_ports = rte_eth_dev_count_avail();
+	if (nb_ports == 0)
+		rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
+
+	/* Parse application arguments (after the EAL ones) */
+	ret = ioat_parse_args(argc, argv, nb_ports);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid IOAT arguments\n");
+
+	nb_mbufs = RTE_MAX(nb_ports * (nb_queues * (nb_rxd + nb_txd +
+		4 * MAX_PKT_BURST) + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+		MIN_POOL_SIZE);
+
+	/* Create the mbuf pool */
+	ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+		MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+		rte_socket_id());
+	if (ioat_pktmbuf_pool == NULL)
+		rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+	/* Initialise each port */
+	cfg.nb_ports = 0;
+	RTE_ETH_FOREACH_DEV(portid)
+		port_init(portid, ioat_pktmbuf_pool, nb_queues);
+
+	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
+		sleep(1);
+
+	/* Check if there is enough lcores for all ports. */
+	cfg.nb_lcores = rte_lcore_count() - 1;
+	if (cfg.nb_lcores < 1)
+		rte_exit(EXIT_FAILURE,
+			"There should be at least one slave lcore.\n");
+	for (i = 0; i < cfg.nb_ports; i++) {
+		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
+		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
+		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+	}
+
+	printf("Bye...\n");
+	return 0;
+}
diff --git a/examples/ioat/meson.build b/examples/ioat/meson.build
new file mode 100644
index 000000000..ed8328963
--- /dev/null
+++ b/examples/ioat/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+build = dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
+
+deps += ['rawdev_ioat']
+
+sources = files(
+	'ioatfwd.c'
+)
diff --git a/examples/meson.build b/examples/meson.build
index a046b74ad..c2e18b59b 100644
--- a/examples/meson.build
+++ b/examples/meson.build
@@ -16,6 +16,7 @@ all_examples = [
 	'eventdev_pipeline', 'exception_path',
 	'fips_validation', 'flow_classify',
 	'flow_filtering', 'helloworld',
+	'ioat',
 	'ip_fragmentation', 'ip_pipeline',
 	'ip_reassembly', 'ipsec-secgw',
 	'ipv4_multicast', 'kni',
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 1/6] examples/ioat: create " Marcin Baran
@ 2019-09-20  7:37         ` Marcin Baran
  2019-09-27 10:01           ` Bruce Richardson
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
                           ` (5 subsequent siblings)
  7 siblings, 1 reply; 76+ messages in thread
From: Marcin Baran @ 2019-09-20  7:37 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 188 insertions(+), 23 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 977ea6a61..3a092c6cf 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -13,7 +13,7 @@
 #include <rte_rawdev.h>
 #include <rte_ioat_rawdev.h>
 
-/* size of ring used for software copying between rx and tx. */
+ /* size of ring used for software copying between rx and tx. */
 #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
 #define MAX_PKT_BURST 32
 #define MEMPOOL_CACHE_SIZE 512
@@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+static void
+update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
+{
+	struct rte_ether_hdr *eth;
+	void *tmp;
+
+	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
+
+	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
+	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
+	 */
+	tmp = &eth->d_addr.addr_bytes[0];
+	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
+
+	/* src addr */
+	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], &eth->s_addr);
+}
+
+static inline void
+pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+{
+	/* Copy packet metadata */
+	rte_memcpy(&dst->rearm_data,
+		&src->rearm_data,
+		offsetof(struct rte_mbuf, cacheline1)
+		- offsetof(struct rte_mbuf, rearm_data));
+
+	/* Copy packet data */
+	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+		rte_pktmbuf_mtod(src, char *), src->data_len);
+}
+
+/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+static void
+ioat_rx_port(struct rxtx_port_config *rx_config)
+{
+	uint32_t nb_rx, nb_enq, i, j;
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+
+	for (i = 0; i < rx_config->nb_queues; i++) {
+
+		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+			pkts_burst, MAX_PKT_BURST);
+
+		if (nb_rx == 0)
+			continue;
+
+		/* Perform packet software copy, free source packets */
+		int ret;
+		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+			(void *)pkts_burst_copy, nb_rx);
+
+		if (unlikely(ret < 0))
+			rte_exit(EXIT_FAILURE,
+				"Unable to allocate memory.\n");
+
+		for (j = 0; j < nb_rx; j++)
+			pktmbuf_sw_copy(pkts_burst[j],
+				pkts_burst_copy[j]);
+
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)pkts_burst, nb_rx);
+
+		nb_enq = rte_ring_enqueue_burst(
+			rx_config->rx_to_tx_ring,
+			(void *)pkts_burst_copy, nb_rx, NULL);
+
+		/* Free any not enqueued packets. */
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&pkts_burst_copy[nb_enq],
+			nb_rx - nb_enq);
+	}
+}
+
+/* Transmit packets from IOAT rawdev/rte_ring for one port. */
+static void
+ioat_tx_port(struct rxtx_port_config *tx_config)
+{
+	uint32_t i, nb_dq = 0;
+	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+	/* Deque the mbufs from rx_to_tx_ring. */
+	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+	if (nb_dq == 0)
+		return;
+
+	/* Update macs if enabled */
+	if (mac_updating) {
+		for (i = 0; i < nb_dq; i++)
+			update_mac_addrs(mbufs_dst[i],
+				tx_config->rxtx_port);
+	}
+
+	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+		0, (void *)mbufs_dst, nb_dq);
+
+	/* Free any unsent packets. */
+	if (unlikely(nb_tx < nb_dq))
+		rte_mempool_put_bulk(ioat_pktmbuf_pool,
+		(void *)&mbufs_dst[nb_tx],
+			nb_dq - nb_tx);
+}
+
+/* Main rx and tx loop if only one slave lcore available */
+static void
+rxtx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
+		" lcore %u\n", rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++) {
+			ioat_rx_port(&cfg.ports[i]);
+			ioat_tx_port(&cfg.ports[i]);
+		}
+}
+
+static void start_forwarding_cores(void)
+{
+	uint32_t lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+		__func__, rte_lcore_id());
+
+	lcore_id = rte_get_next_lcore(lcore_id, true, true);
+	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+		NULL, lcore_id);
+}
+
 /* Display usage */
 static void
 ioat_usage(const char *prgname)
@@ -102,7 +238,7 @@ ioat_usage(const char *prgname)
 		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
 		"  -c --copy-type CT: type of copy: sw|rawdev\n"
 		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
-	       prgname);
+		prgname);
 }
 
 static int
@@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 	argvopt = argv;
 
 	while ((opt = getopt_long(argc, argvopt, short_options,
-				  lgopts, &option_index)) != EOF) {
+		lgopts, &option_index)) != EOF) {
 
 		switch (opt) {
-		/* portmask */
+			/* portmask */
 		case 'p':
 			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
 			if (ioat_enabled_port_mask & ~default_port_mask ||
-					ioat_enabled_port_mask <= 0) {
+				ioat_enabled_port_mask <= 0) {
 				printf("Invalid portmask, %s, suggest 0x%x\n",
-						optarg, default_port_mask);
+					optarg, default_port_mask);
 				ioat_usage(prgname);
 				return -1;
 			}
@@ -204,7 +340,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 			}
 			break;
 
-		/* long options */
+			/* long options */
 		case 0:
 			break;
 
@@ -216,9 +352,9 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 
 	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
 	if (optind >= 0)
-		argv[optind-1] = prgname;
+		argv[optind - 1] = prgname;
 
-	ret = optind-1;
+	ret = optind - 1;
 	optind = 1; /* reset getopt lib */
 	return ret;
 }
@@ -253,6 +389,26 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+assign_rings(void)
+{
+	uint32_t i;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		char ring_name[RTE_RING_NAMESIZE];
+
+		snprintf(ring_name, sizeof(ring_name), "rx_to_tx_ring_%u", i);
+		/* Create ring for inter core communication */
+		cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+			ring_name, ring_size,
+			rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
+
+		if (cfg.ports[i].rx_to_tx_ring == NULL)
+			rte_exit(EXIT_FAILURE, "Ring create failed: %s\n",
+				rte_strerror(rte_errno));
+	}
+}
+
 /*
  * Initializes a given port using global settings and with the RX buffers
  * coming from the mbuf_pool passed as a parameter.
@@ -263,7 +419,7 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	/* configuring port to use RSS for multiple RX queues */
 	static const struct rte_eth_conf port_conf = {
 		.rxmode = {
-			.mq_mode        = ETH_MQ_RX_RSS,
+			.mq_mode = ETH_MQ_RX_RSS,
 			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
 		},
 		.rx_adv_conf = {
@@ -294,14 +450,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 		dev_info.flow_type_rss_offloads;
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
 		local_port_conf.txmode.offloads |=
-			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+		DEV_TX_OFFLOAD_MBUF_FAST_FREE;
 	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE, "Cannot configure device:"
 			" err=%d, port=%u\n", ret, portid);
 
 	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
-						&nb_txd);
+		&nb_txd);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE,
 			"Cannot adjust number of descriptors: err=%d, port=%u\n",
@@ -326,8 +482,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	txq_conf = dev_info.default_txconf;
 	txq_conf.offloads = local_port_conf.txmode.offloads;
 	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
-			rte_eth_dev_socket_id(portid),
-			&txq_conf);
+		rte_eth_dev_socket_id(portid),
+		&txq_conf);
 	if (ret < 0)
 		rte_exit(EXIT_FAILURE,
 			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
@@ -335,8 +491,8 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	/* Initialize TX buffers */
 	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
-			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
-			rte_eth_dev_socket_id(portid));
+		RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+		rte_eth_dev_socket_id(portid));
 	if (tx_buffer[portid] == NULL)
 		rte_exit(EXIT_FAILURE,
 			"Cannot allocate buffer for tx on port %u\n",
@@ -354,13 +510,13 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 	rte_eth_promiscuous_enable(portid);
 
 	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
-			portid,
-			ioat_ports_eth_addr[portid].addr_bytes[0],
-			ioat_ports_eth_addr[portid].addr_bytes[1],
-			ioat_ports_eth_addr[portid].addr_bytes[2],
-			ioat_ports_eth_addr[portid].addr_bytes[3],
-			ioat_ports_eth_addr[portid].addr_bytes[4],
-			ioat_ports_eth_addr[portid].addr_bytes[5]);
+		portid,
+		ioat_ports_eth_addr[portid].addr_bytes[0],
+		ioat_ports_eth_addr[portid].addr_bytes[1],
+		ioat_ports_eth_addr[portid].addr_bytes[2],
+		ioat_ports_eth_addr[portid].addr_bytes[3],
+		ioat_ports_eth_addr[portid].addr_bytes[4],
+		ioat_ports_eth_addr[portid].addr_bytes[5]);
 
 	cfg.ports[cfg.nb_ports].rxtx_port = portid;
 	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
@@ -428,10 +584,19 @@ main(int argc, char **argv)
 	if (cfg.nb_lcores < 1)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
+
+	assign_rings();
+
+	start_forwarding_cores();
+
+	/* force_quit is true when we get here */
+	rte_eal_mp_wait_lcore();
+
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 1/6] examples/ioat: create " Marcin Baran
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support Marcin Baran
@ 2019-09-20  7:37         ` Marcin Baran
  2019-09-27 10:05           ` Bruce Richardson
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration Marcin Baran
                           ` (4 subsequent siblings)
  7 siblings, 1 reply; 76+ messages in thread
From: Marcin Baran @ 2019-09-20  7:37 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++--------
 1 file changed, 189 insertions(+), 47 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 3a092c6cf..c66ce7e49 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
 		rte_pktmbuf_mtod(src, char *), src->data_len);
 }
 
+static uint32_t
+ioat_enqueue_packets(struct rte_mbuf **pkts,
+	uint32_t nb_rx, uint16_t dev_id)
+{
+	int ret;
+	uint32_t i;
+	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+		&pkts[0]->rearm_data);
+
+	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+		(void *)pkts_copy, nb_rx);
+
+	if (unlikely(ret < 0))
+		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+	for (i = 0; i < nb_rx; i++) {
+		/* Perform data copy */
+		ret = rte_ioat_enqueue_copy(dev_id,
+			pkts[i]->buf_iova
+			- addr_offset,
+			pkts_copy[i]->buf_iova
+			- addr_offset,
+			rte_pktmbuf_data_len(pkts[i])
+			+ addr_offset,
+			(uintptr_t)pkts[i],
+			(uintptr_t)pkts_copy[i],
+			0 /* nofence */);
+
+		if (ret != 1)
+			break;
+	}
+
+	ret = i;
+	/* Free any not enqueued packets. */
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+		nb_rx - i);
+
+
+	return ret;
+}
+
 /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
 static void
 ioat_rx_port(struct rxtx_port_config *rx_config)
@@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
-		/* Perform packet software copy, free source packets */
-		int ret;
-		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
-
-		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
-			(void *)pkts_burst_copy, nb_rx);
-
-		if (unlikely(ret < 0))
-			rte_exit(EXIT_FAILURE,
-				"Unable to allocate memory.\n");
-
-		for (j = 0; j < nb_rx; j++)
-			pktmbuf_sw_copy(pkts_burst[j],
-				pkts_burst_copy[j]);
-
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)pkts_burst, nb_rx);
-
-		nb_enq = rte_ring_enqueue_burst(
-			rx_config->rx_to_tx_ring,
-			(void *)pkts_burst_copy, nb_rx, NULL);
-
-		/* Free any not enqueued packets. */
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)&pkts_burst_copy[nb_enq],
-			nb_rx - nb_enq);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			/* Perform packet hardware copy */
+			nb_enq = ioat_enqueue_packets(pkts_burst,
+				nb_rx, rx_config->ioat_ids[i]);
+			if (nb_enq > 0)
+				rte_ioat_do_copies(rx_config->ioat_ids[i]);
+		} else {
+			/* Perform packet software copy, free source packets */
+			int ret;
+			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst_copy, nb_rx);
+
+			if (unlikely(ret < 0))
+				rte_exit(EXIT_FAILURE,
+					"Unable to allocate memory.\n");
+
+			for (j = 0; j < nb_rx; j++)
+				pktmbuf_sw_copy(pkts_burst[j],
+					pkts_burst_copy[j]);
+
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst, nb_rx);
+
+			nb_enq = rte_ring_enqueue_burst(
+				rx_config->rx_to_tx_ring,
+				(void *)pkts_burst_copy, nb_rx, NULL);
+
+			/* Free any not enqueued packets. */
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&pkts_burst_copy[nb_enq],
+				nb_rx - nb_enq);
+		}
 	}
 }
 
@@ -169,31 +221,64 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 static void
 ioat_tx_port(struct rxtx_port_config *tx_config)
 {
-	uint32_t i, nb_dq = 0;
+	uint32_t i, j, nb_dq = 0;
+	struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
-	/* Deque the mbufs from rx_to_tx_ring. */
-	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
-		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		/* Deque the mbufs from IOAT device. */
+		for (i = 0; i < tx_config->nb_queues; i++) {
+			nb_dq = rte_ioat_completed_copies(
+				tx_config->ioat_ids[i], MAX_PKT_BURST,
+				(void *)mbufs_src, (void *)mbufs_dst);
 
-	if (nb_dq == 0)
-		return;
+			if (nb_dq == 0)
+				break;
 
-	/* Update macs if enabled */
-	if (mac_updating) {
-		for (i = 0; i < nb_dq; i++)
-			update_mac_addrs(mbufs_dst[i],
-				tx_config->rxtx_port);
-	}
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)mbufs_src, nb_dq);
 
-	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
-		0, (void *)mbufs_dst, nb_dq);
+			/* Update macs if enabled */
+			if (mac_updating) {
+				for (j = 0; j < nb_dq; j++)
+					update_mac_addrs(mbufs_dst[j],
+						tx_config->rxtx_port);
+			}
+
+			const uint16_t nb_tx = rte_eth_tx_burst(
+				tx_config->rxtx_port, 0,
+				(void *)mbufs_dst, nb_dq);
 
-	/* Free any unsent packets. */
-	if (unlikely(nb_tx < nb_dq))
-		rte_mempool_put_bulk(ioat_pktmbuf_pool,
-		(void *)&mbufs_dst[nb_tx],
-			nb_dq - nb_tx);
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_dq))
+				rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&mbufs_dst[nb_tx],
+					nb_dq - nb_tx);
+		}
+	} else {
+		/* Deque the mbufs from rx_to_tx_ring. */
+		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+		if (nb_dq == 0)
+			return;
+
+		/* Update macs if enabled */
+		if (mac_updating) {
+			for (i = 0; i < nb_dq; i++)
+				update_mac_addrs(mbufs_dst[i],
+					tx_config->rxtx_port);
+		}
+
+		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+			0, (void *)mbufs_dst, nb_dq);
+
+		/* Free any unsent packets. */
+		if (unlikely(nb_tx < nb_dq))
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&mbufs_dst[nb_tx],
+				nb_dq - nb_tx);
+	}
 }
 
 /* Main rx and tx loop if only one slave lcore available */
@@ -389,6 +474,52 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+configure_rawdev_queue(uint32_t dev_id)
+{
+	struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
+	struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+	if (rte_rawdev_configure(dev_id, &info) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_configure()\n");
+	}
+	if (rte_rawdev_start(dev_id) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_start()\n");
+	}
+}
+
+static void
+assign_rawdevs(void)
+{
+	uint16_t nb_rawdev = 0, rdev_id = 0;
+	uint32_t i, j;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+			struct rte_rawdev_info rdev_info = { 0 };
+
+			do {
+				if (rdev_id == rte_rawdev_count())
+					goto end;
+				rte_rawdev_info_get(rdev_id++, &rdev_info);
+			} while (strcmp(rdev_info.driver_name,
+				IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+			cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+			configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+			++nb_rawdev;
+		}
+	}
+end:
+	if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+		rte_exit(EXIT_FAILURE,
+			"Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+			nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+	RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+}
+
 static void
 assign_rings(void)
 {
@@ -585,18 +716,29 @@ main(int argc, char **argv)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
 
-	assign_rings();
+	if (copy_mode == COPY_MODE_IOAT_NUM)
+		assign_rawdevs();
+	else /* copy_mode == COPY_MODE_SW_NUM */
+		assign_rings();
 
 	start_forwarding_cores();
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
 
+	uint32_t j;
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
-		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+				printf("Stopping rawdev %d\n",
+					cfg.ports[i].ioat_ids[j]);
+				rte_rawdev_stop(cfg.ports[i].ioat_ids[j]);
+			}
+		} else /* copy_mode == COPY_MODE_SW_NUM */
+			rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                           ` (2 preceding siblings ...)
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
@ 2019-09-20  7:37         ` Marcin Baran
  2019-09-27 10:08           ` Bruce Richardson
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port Marcin Baran
                           ` (3 subsequent siblings)
  7 siblings, 1 reply; 76+ messages in thread
From: Marcin Baran @ 2019-09-20  7:37 UTC (permalink / raw)
  To: dev; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++-----------
 1 file changed, 67 insertions(+), 25 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index c66ce7e49..4c51db6bd 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,
 	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
 		nb_rx - i);
 
-
 	return ret;
 }
 
@@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
 	if (copy_mode == COPY_MODE_IOAT_NUM) {
-		/* Deque the mbufs from IOAT device. */
 		for (i = 0; i < tx_config->nb_queues; i++) {
+			/* Deque the mbufs from IOAT device. */
 			nb_dq = rte_ioat_completed_copies(
 				tx_config->ioat_ids[i], MAX_PKT_BURST,
 				(void *)mbufs_src, (void *)mbufs_dst);
@@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 					nb_dq - nb_tx);
 		}
 	} else {
-		/* Deque the mbufs from rx_to_tx_ring. */
-		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
-			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
-
-		if (nb_dq == 0)
-			return;
-
-		/* Update macs if enabled */
-		if (mac_updating) {
-			for (i = 0; i < nb_dq; i++)
-				update_mac_addrs(mbufs_dst[i],
-					tx_config->rxtx_port);
-		}
+		for (i = 0; i < tx_config->nb_queues; i++) {
+			/* Deque the mbufs from IOAT device. */
+			nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+				(void *)mbufs_dst, MAX_PKT_BURST, NULL);
 
-		const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
-			0, (void *)mbufs_dst, nb_dq);
+			if (nb_dq == 0)
+				return;
 
-		/* Free any unsent packets. */
-		if (unlikely(nb_tx < nb_dq))
-			rte_mempool_put_bulk(ioat_pktmbuf_pool,
-			(void *)&mbufs_dst[nb_tx],
-				nb_dq - nb_tx);
+			/* Update macs if enabled */
+			if (mac_updating) {
+				for (j = 0; j < nb_dq; j++)
+					update_mac_addrs(mbufs_dst[j],
+						tx_config->rxtx_port);
+			}
+
+			const uint16_t nb_tx =
+				rte_eth_tx_burst(tx_config->rxtx_port, 0,
+					(void *)mbufs_dst, nb_dq);
+
+			/* Free any unsent packets. */
+			if (unlikely(nb_tx < nb_dq))
+				rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&mbufs_dst[nb_tx],
+					nb_dq - nb_tx);
+		}
 	}
 }
 
+/* Main rx processing loop for IOAT rawdev. */
+static void
+rx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_rx_port(&cfg.ports[i]);
+}
+
+/* Main tx processing loop for hardware copy. */
+static void
+tx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main tx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_tx_port(&cfg.ports[i]);
+}
+
 /* Main rx and tx loop if only one slave lcore available */
 static void
 rxtx_main_loop(void)
@@ -305,9 +337,19 @@ static void start_forwarding_cores(void)
 	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
 		__func__, rte_lcore_id());
 
-	lcore_id = rte_get_next_lcore(lcore_id, true, true);
-	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
-		NULL, lcore_id);
+	if (cfg.nb_lcores == 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+			NULL, lcore_id);
+	} else if (cfg.nb_lcores > 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+			NULL, lcore_id);
+
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+			lcore_id);
+	}
 }
 
 /* Display usage */
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                           ` (3 preceding siblings ...)
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration Marcin Baran
@ 2019-09-20  7:37         ` Marcin Baran
  2019-09-27 10:12           ` Bruce Richardson
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
                           ` (2 subsequent siblings)
  7 siblings, 1 reply; 76+ messages in thread
From: Marcin Baran @ 2019-09-20  7:37 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak

Added printing stats of ports each second.
The stats printing is done using master core.
The information provided informs about packets
received, dropped and send as well as statistics
of rawdev devices.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 241 insertions(+), 4 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 4c51db6bd..c6b994832 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -48,10 +48,27 @@ struct rxtx_transmission_config {
 	uint16_t nb_lcores;
 };
 
+/* per-port statistics struct */
+struct ioat_port_statistics {
+	uint64_t rx[RTE_MAX_ETHPORTS];
+	uint64_t tx[RTE_MAX_ETHPORTS];
+	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
+	uint64_t copy_dropped[RTE_MAX_ETHPORTS];
+};
+struct ioat_port_statistics port_statistics;
+
+struct total_statistics {
+	uint64_t total_packets_dropped;
+	uint64_t total_packets_tx;
+	uint64_t total_packets_rx;
+	uint64_t total_successful_enqueues;
+	uint64_t total_failed_enqueues;
+};
+
 typedef enum copy_mode_t {
 #define COPY_MODE_SW "sw"
 	COPY_MODE_SW_NUM,
-#define COPY_MODE_IOAT "rawdev"
+#define COPY_MODE_IOAT "hw"
 	COPY_MODE_IOAT_NUM,
 	COPY_MODE_INVALID_NUM,
 	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+/* Print out statistics for one port. */
+static void
+print_port_stats(uint16_t port_id)
+{
+	printf("\nStatistics for port %u ------------------------------"
+		"\nPackets sent: %34"PRIu64
+		"\nPackets received: %30"PRIu64
+		"\nPackets dropped on tx: %25"PRIu64
+		"\nPackets dropped on copy: %23"PRIu64,
+		port_id,
+		port_statistics.tx[port_id],
+		port_statistics.rx[port_id],
+		port_statistics.tx_dropped[port_id],
+		port_statistics.copy_dropped[port_id]);
+}
+
+/* Print out statistics for one IOAT rawdev device. */
+static void
+print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
+	unsigned int *ids_xstats, uint16_t nb_xstats,
+	struct rte_rawdev_xstats_name *names_xstats)
+{
+	uint16_t i;
+
+	printf("\nIOAT channel %u", dev_id);
+	for (i = 0; i < nb_xstats; i++)
+		printf("\n\t %s: %*"PRIu64,
+			names_xstats[ids_xstats[i]].name,
+			(int)(37 - strlen(names_xstats[ids_xstats[i]].name)),
+			xstats[i]);
+}
+
+static void
+print_total_stats(struct total_statistics *ts)
+{
+	printf("\nAggregate statistics ==============================="
+		"\nTotal packets Tx: %24"PRIu64" [pps]"
+		"\nTotal packets Rx: %24"PRIu64" [pps]"
+		"\nTotal packets dropped: %19"PRIu64" [pps]",
+		ts->total_packets_tx,
+		ts->total_packets_rx,
+		ts->total_packets_dropped);
+
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		printf("\nTotal IOAT successful enqueues: %8"PRIu64" [enq/s]"
+			"\nTotal IOAT failed enqueues: %12"PRIu64" [enq/s]",
+			ts->total_successful_enqueues,
+			ts->total_failed_enqueues);
+	}
+
+	printf("\n====================================================\n");
+}
+
+/* Print out statistics on packets dropped. */
+static void
+print_stats(char *prgname)
+{
+	struct total_statistics ts, delta_ts;
+	uint32_t i, port_id, dev_id;
+	struct rte_rawdev_xstats_name *names_xstats;
+	uint64_t *xstats;
+	unsigned int *ids_xstats, nb_xstats;
+	char status_string[120]; /* to print at the top of the output */
+	int status_strlen;
+
+
+	const char clr[] = { 27, '[', '2', 'J', '\0' };
+	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+	status_strlen = snprintf(status_string, sizeof(status_string),
+		"%s, ", prgname);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Worker Threads = %d, ",
+		rte_lcore_count() > 2 ? 2 : 1);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
+		COPY_MODE_SW : COPY_MODE_IOAT);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Updating MAC = %s, ", mac_updating ?
+		"enabled" : "disabled");
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Rx Queues = %d, ", nb_queues);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Ring Size = %d\n", ring_size);
+
+	/* Allocate memory for xstats names and values */
+	nb_xstats = rte_rawdev_xstats_names_get(
+		cfg.ports[0].ioat_ids[0], NULL, 0);
+
+	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
+	if (names_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat names memory\n");
+	}
+	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
+		names_xstats, nb_xstats);
+
+	ids_xstats = malloc(sizeof(*ids_xstats) * 2);
+	if (ids_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat ids_xstats memory\n");
+	}
+
+	xstats = malloc(sizeof(*xstats) * 2);
+	if (xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat memory\n");
+	}
+
+	/* Get failed/successful enqueues stats index */
+	ids_xstats[0] = ids_xstats[1] = nb_xstats;
+	for (i = 0; i < nb_xstats; i++) {
+		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
+			ids_xstats[0] = i;
+		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
+			ids_xstats[1] = i;
+		if (ids_xstats[0] < nb_xstats && ids_xstats[1] < nb_xstats)
+			break;
+	}
+	if (ids_xstats[0] == nb_xstats || ids_xstats[1] == nb_xstats) {
+		rte_exit(EXIT_FAILURE,
+			"Error getting failed/successful enqueues stats index\n");
+	}
+
+	memset(&ts, 0, sizeof(struct total_statistics));
+
+	while (!force_quit) {
+		/* Sleep for 1 second each round - init sleep allows reading
+		 * messages from app startup.
+		 */
+		sleep(1);
+
+		/* Clear screen and move to top left */
+		printf("%s%s", clr, topLeft);
+
+		memset(&delta_ts, 0, sizeof(struct total_statistics));
+
+		printf("%s", status_string);
+
+		for (i = 0; i < cfg.nb_ports; i++) {
+			port_id = cfg.ports[i].rxtx_port;
+			print_port_stats(port_id);
+
+			delta_ts.total_packets_dropped +=
+				port_statistics.tx_dropped[port_id]
+				+ port_statistics.copy_dropped[port_id];
+			delta_ts.total_packets_tx +=
+				port_statistics.tx[port_id];
+			delta_ts.total_packets_rx +=
+				port_statistics.rx[port_id];
+
+			if (copy_mode == COPY_MODE_IOAT_NUM) {
+				uint32_t j;
+
+				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+					dev_id = cfg.ports[i].ioat_ids[j];
+					rte_rawdev_xstats_get(dev_id,
+						ids_xstats, xstats, 2);
+
+					print_rawdev_stats(dev_id, xstats,
+						ids_xstats, 2, names_xstats);
+
+					delta_ts.total_failed_enqueues +=
+						xstats[ids_xstats[0]];
+					delta_ts.total_successful_enqueues +=
+						xstats[ids_xstats[1]];
+				}
+			}
+		}
+
+		delta_ts.total_packets_tx -= ts.total_packets_tx;
+		delta_ts.total_packets_rx -= ts.total_packets_rx;
+		delta_ts.total_packets_dropped -= ts.total_packets_dropped;
+		delta_ts.total_failed_enqueues -= ts.total_failed_enqueues;
+		delta_ts.total_successful_enqueues -=
+			ts.total_successful_enqueues;
+
+		printf("\n");
+		print_total_stats(&delta_ts);
+
+		ts.total_packets_tx += delta_ts.total_packets_tx;
+		ts.total_packets_rx += delta_ts.total_packets_rx;
+		ts.total_packets_dropped += delta_ts.total_packets_dropped;
+		ts.total_failed_enqueues += delta_ts.total_failed_enqueues;
+		ts.total_successful_enqueues +=
+			delta_ts.total_successful_enqueues;
+	}
+
+	free(names_xstats);
+	free(xstats);
+	free(ids_xstats);
+}
+
 static void
 update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
 {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
+		port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Perform packet hardware copy */
 			nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 				(void *)&pkts_burst_copy[nb_enq],
 				nb_rx - nb_enq);
 		}
+
+		port_statistics.copy_dropped[rx_config->rxtx_port] +=
+			(nb_rx - nb_enq);
 	}
 }
 
@@ -248,6 +468,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 				tx_config->rxtx_port, 0,
 				(void *)mbufs_dst, nb_dq);
 
+			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 			/* Free any unsent packets. */
 			if (unlikely(nb_tx < nb_dq))
 				rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -274,6 +496,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 				rte_eth_tx_burst(tx_config->rxtx_port, 0,
 					(void *)mbufs_dst, nb_dq);
 
+			port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 			/* Free any unsent packets. */
 			if (unlikely(nb_tx < nb_dq))
 				rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -363,7 +587,7 @@ ioat_usage(const char *prgname)
 		"      When enabled:\n"
 		"       - The source MAC address is replaced by the TX port MAC address\n"
 		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
-		"  -c --copy-type CT: type of copy: sw|rawdev\n"
+		"  -c --copy-type CT: type of copy: sw|hw\n"
 		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
 		prgname);
 }
@@ -400,7 +624,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 	static const char short_options[] =
 		"p:"  /* portmask */
 		"q:"  /* number of RX queues per port */
-		"c:"  /* copy type (sw|rawdev) */
+		"c:"  /* copy type (sw|hw) */
 		"s:"  /* ring size */
 		;
 
@@ -452,7 +676,7 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
 		case 'c':
 			copy_mode = ioat_parse_copy_mode(optarg);
 			if (copy_mode == COPY_MODE_INVALID_NUM) {
-				printf("Invalid copy type. Use: sw, rawdev\n");
+				printf("Invalid copy type. Use: sw, hw\n");
 				ioat_usage(prgname);
 				return -1;
 			}
@@ -673,6 +897,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
 
+	ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+		rte_eth_tx_buffer_count_callback,
+		&port_statistics.tx_dropped[portid]);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot set error callback for tx buffer on port %u\n",
+			portid);
+
 	/* Start device */
 	ret = rte_eth_dev_start(portid);
 	if (ret < 0)
@@ -749,6 +981,9 @@ main(int argc, char **argv)
 	RTE_ETH_FOREACH_DEV(portid)
 		port_init(portid, ioat_pktmbuf_pool, nb_queues);
 
+	/* Initialize port xstats */
+	memset(&port_statistics, 0, sizeof(port_statistics));
+
 	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
 		sleep(1);
 
@@ -764,6 +999,8 @@ main(int argc, char **argv)
 		assign_rings();
 
 	start_forwarding_cores();
+	/* master core prints stats while other cores forward */
+	print_stats(argv[0]);
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                           ` (4 preceding siblings ...)
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port Marcin Baran
@ 2019-09-20  7:37         ` Marcin Baran
  2019-09-27 10:36           ` Bruce Richardson
                             ` (3 more replies)
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
  7 siblings, 4 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-20  7:37 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran

Added guide for IOAT sample app usage and
code description.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 doc/guides/sample_app_ug/index.rst |   1 +
 doc/guides/sample_app_ug/intro.rst |   4 +
 doc/guides/sample_app_ug/ioat.rst  | 764 +++++++++++++++++++++++++++++
 3 files changed, 769 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst

diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index f23f8f59e..a6a1d9e7a 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -23,6 +23,7 @@ Sample Applications User Guides
     ip_reassembly
     kernel_nic_interface
     keep_alive
+    ioat
     l2_forward_crypto
     l2_forward_job_stats
     l2_forward_real_virtual
diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
index 90704194a..74462312f 100644
--- a/doc/guides/sample_app_ug/intro.rst
+++ b/doc/guides/sample_app_ug/intro.rst
@@ -91,6 +91,10 @@ examples are highlighted below.
   forwarding, or ``l3fwd`` application does forwarding based on Internet
   Protocol, IPv4 or IPv6 like a simple router.
 
+* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
+  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
+  copying packets between two threads.
+
 * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
   demonstrates how to distribute packets arriving on an Rx port to different
   cores for processing and transmission.
diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
new file mode 100644
index 000000000..69621673b
--- /dev/null
+++ b/doc/guides/sample_app_ug/ioat.rst
@@ -0,0 +1,764 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Intel Corporation.
+
+Sample Application of packet copying using Intel\ |reg| QuickData Technology
+============================================================================
+
+Overview
+--------
+
+This sample is intended as a demonstration of the basic components of a DPDK
+forwarding application and example of how to use IOAT driver API to make
+packets copies.
+
+Also while forwarding, the MAC addresses are affected as follows:
+
+*   The source MAC address is replaced by the TX port MAC address
+
+*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
+
+This application can be used to compare performance of using software packet
+copy with copy done using a DMA device for different sizes of packets.
+The example will print out statistics each second. The stats shows
+received/send packets and packets dropped or failed to copy.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``ioat`` sub-directory.
+
+
+Running the Application
+-----------------------
+
+In order to run the hardware copy application, the copying device
+needs to be bound to user-space IO driver.
+
+Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology*
+guide for information on using the driver.
+
+The application requires a number of command line options:
+
+.. code-block:: console
+
+    ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>]
+        [--[no-]mac-updating]
+
+where,
+
+*   p MASK: A hexadecimal bitmask of the ports to configure
+
+*   q NQ: Number of Rx queues used per port equivalent to CBDMA channels
+    per port
+
+*   c CT: Performed packet copy type: software (sw) or hardware using
+    DMA (hw)
+
+*   s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for
+    software copy mode
+
+*   --[no-]mac-updating: Whether MAC address of packets should be changed
+    or not
+
+The application can be launched in various configurations depending on
+provided parameters. Each port can use up to 2 lcores: one of them receives
+incoming traffic and makes a copy of each packet. The second lcore then
+updates MAC address and sends the copy. If one lcore per port is used,
+both operations are done sequentially. For each configuration an additional
+lcore is needed since master lcore in use which is responsible for
+configuration, statistics printing and safe deinitialization of all ports
+and devices.
+
+The application can use a maximum of 8 ports.
+
+To run the application in a Linux environment with 3 lcores (one of them
+is master lcore), 1 port (port 0), software copying and MAC updating issue
+the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
+
+To run the application in a Linux environment with 2 lcores (one of them
+is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC
+updating issue the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw
+
+Refer to the *DPDK Getting Started Guide* for general information on
+running applications and the Environment Abstraction Layer (EAL) options.
+
+Explanation
+-----------
+
+The following sections provide an explanation of the main components of the
+code.
+
+All DPDK library functions used in the sample code are prefixed with
+``rte_`` and are explained in detail in the *DPDK API Documentation*.
+
+
+The Main Function
+~~~~~~~~~~~~~~~~~
+
+The ``main()`` function performs the initialization and calls the execution
+threads for each lcore.
+
+The first task is to initialize the Environment Abstraction Layer (EAL).
+The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
+function. The value returned is the number of parsed arguments:
+
+.. code-block:: c
+
+    /* init EAL */
+    ret = rte_eal_init(argc, argv);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+
+
+The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
+used by the application:
+
+.. code-block:: c
+
+    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
+        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+        MIN_POOL_SIZE);
+
+    /* Create the mbuf pool */
+    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+        rte_socket_id());
+    if (ioat_pktmbuf_pool == NULL)
+        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+Mbufs are the packet buffer structure used by DPDK. They are explained in
+detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
+
+The ``main()`` function also initializes the ports:
+
+.. code-block:: c
+
+    /* Initialise each port */
+    RTE_ETH_FOREACH_DEV(portid) {
+        port_init(portid, ioat_pktmbuf_pool);
+    }
+
+Each port is configured using ``port_init()``:
+
+.. code-block:: c
+
+     /*
+     * Initializes a given port using global settings and with the RX buffers
+     * coming from the mbuf_pool passed as a parameter.
+     */
+    static inline void
+    port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+    {
+        /* configuring port to use RSS for multiple RX queues */
+        static const struct rte_eth_conf port_conf = {
+            .rxmode = {
+                .mq_mode        = ETH_MQ_RX_RSS,
+                .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+            },
+            .rx_adv_conf = {
+                .rss_conf = {
+                    .rss_key = NULL,
+                    .rss_hf = ETH_RSS_PROTO_MASK,
+                }
+            }
+        };
+
+        struct rte_eth_rxconf rxq_conf;
+        struct rte_eth_txconf txq_conf;
+        struct rte_eth_conf local_port_conf = port_conf;
+        struct rte_eth_dev_info dev_info;
+        int ret, i;
+
+        /* Skip ports that are not enabled */
+        if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+            printf("Skipping disabled port %u\n", portid);
+            return;
+        }
+
+        /* Init port */
+        printf("Initializing port %u... ", portid);
+        fflush(stdout);
+        rte_eth_dev_info_get(portid, &dev_info);
+        local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+            dev_info.flow_type_rss_offloads;
+        if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+            local_port_conf.txmode.offloads |=
+                DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+        ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE, "Cannot configure device:"
+                " err=%d, port=%u\n", ret, portid);
+
+        ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+                            &nb_txd);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "Cannot adjust number of descriptors: err=%d, port=%u\n",
+                ret, portid);
+
+        rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+        /* Init Rx queues */
+        rxq_conf = dev_info.default_rxconf;
+        rxq_conf.offloads = local_port_conf.rxmode.offloads;
+        for (i = 0; i < nb_queues; i++) {
+            ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+                rte_eth_dev_socket_id(portid), &rxq_conf,
+                mbuf_pool);
+            if (ret < 0)
+                rte_exit(EXIT_FAILURE,
+                    "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+                    ret, portid, i);
+        }
+
+        /* Init one TX queue on each port */
+        txq_conf = dev_info.default_txconf;
+        txq_conf.offloads = local_port_conf.txmode.offloads;
+        ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+                rte_eth_dev_socket_id(portid),
+                &txq_conf);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "rte_eth_tx_queue_setup:err=%d,port=%u\n",
+                ret, portid);
+
+        /* Initialize TX buffers */
+        tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+                RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+                rte_eth_dev_socket_id(portid));
+        if (tx_buffer[portid] == NULL)
+            rte_exit(EXIT_FAILURE,
+                "Cannot allocate buffer for tx on port %u\n",
+                portid);
+
+        rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+        ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+                rte_eth_tx_buffer_count_callback,
+                &port_statistics.tx_dropped[portid]);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "Cannot set error callback for tx buffer on port %u\n",
+                portid);
+
+        /* Start device */
+        ret = rte_eth_dev_start(portid);
+        if (ret < 0)
+            rte_exit(EXIT_FAILURE,
+                "rte_eth_dev_start:err=%d, port=%u\n",
+                ret, portid);
+
+        rte_eth_promiscuous_enable(portid);
+
+        printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+                portid,
+                ioat_ports_eth_addr[portid].addr_bytes[0],
+                ioat_ports_eth_addr[portid].addr_bytes[1],
+                ioat_ports_eth_addr[portid].addr_bytes[2],
+                ioat_ports_eth_addr[portid].addr_bytes[3],
+                ioat_ports_eth_addr[portid].addr_bytes[4],
+                ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+        cfg.ports[cfg.nb_ports].rxtx_port = portid;
+        cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+    }
+
+The Ethernet ports are configured with local settings using the
+``rte_eth_dev_configure()`` function and the ``port_conf`` struct.
+The RSS is enabled so that multiple Rx queues could be used for
+packet receiving and copying by multiple CBDMA channels per port:
+
+.. code-block:: c
+
+    /* configuring port to use RSS for multiple RX queues */
+    static const struct rte_eth_conf port_conf = {
+        .rxmode = {
+            .mq_mode        = ETH_MQ_RX_RSS,
+            .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+        },
+        .rx_adv_conf = {
+            .rss_conf = {
+                .rss_key = NULL,
+                .rss_hf = ETH_RSS_PROTO_MASK,
+            }
+        }
+    };
+
+For this example the ports are set up with the number of Rx queues provided
+with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()``
+and ``rte_eth_tx_queue_setup()`` functions.
+
+The Ethernet port is then started:
+
+.. code-block:: c
+
+    ret = rte_eth_dev_start(portid);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
+            ret, portid);
+
+
+Finally the Rx port is set in promiscuous mode:
+
+.. code-block:: c
+
+    rte_eth_promiscuous_enable(portid);
+
+
+After that each port application assigns resources needed.
+
+.. code-block:: c
+
+    check_link_status(ioat_enabled_port_mask);
+
+    if (!cfg.nb_ports) {
+        rte_exit(EXIT_FAILURE,
+            "All available ports are disabled. Please set portmask.\n");
+    }
+
+    /* Check if there is enough lcores for all ports. */
+    cfg.nb_lcores = rte_lcore_count() - 1;
+    if (cfg.nb_lcores < 1)
+        rte_exit(EXIT_FAILURE,
+            "There should be at least one slave lcore.\n");
+
+    ret = 0;
+
+    if (copy_mode == COPY_MODE_IOAT_NUM) {
+        assign_rawdevs();
+    } else /* copy_mode == COPY_MODE_SW_NUM */ {
+        assign_rings();
+    }
+
+A link status is checked of each port enabled by port mask
+using ``check_link_status()`` function.
+
+.. code-block:: c
+
+    /* check link status, return true if at least one port is up */
+    static int
+    check_link_status(uint32_t port_mask)
+    {
+        uint16_t portid;
+        struct rte_eth_link link;
+        int retval = 0;
+
+        printf("\nChecking link status\n");
+        RTE_ETH_FOREACH_DEV(portid) {
+            if ((port_mask & (1 << portid)) == 0)
+                continue;
+
+            memset(&link, 0, sizeof(link));
+            rte_eth_link_get(portid, &link);
+
+            /* Print link status */
+            if (link.link_status) {
+                printf(
+                    "Port %d Link Up. Speed %u Mbps - %s\n",
+                    portid, link.link_speed,
+                    (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+                    ("full-duplex") : ("half-duplex\n"));
+                retval = 1;
+            } else
+                printf("Port %d Link Down\n", portid);
+        }
+        return retval;
+    }
+
+Depending on mode set (whether copy should be done by software or by hardware)
+special structures are assigned to each port. If software copy was chosen,
+application have to assign ring structures for packet exchanging between lcores
+assigned to ports.
+
+.. code-block:: c
+
+    static void
+    assign_rings(void)
+    {
+        uint32_t i;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            char ring_name[20];
+
+            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
+            /* Create ring for inter core communication */
+            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+                    ring_name, ring_size,
+                    rte_socket_id(), RING_F_SP_ENQ);
+
+            if (cfg.ports[i].rx_to_tx_ring == NULL)
+                rte_exit(EXIT_FAILURE, "%s\n",
+                        rte_strerror(rte_errno));
+        }
+    }
+
+
+When using hardware copy each Rx queue of the port is assigned an
+IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API
+functions:
+
+.. code-block:: c
+
+    static void
+    assign_rawdevs(void)
+    {
+        uint16_t nb_rawdev = 0, rdev_id = 0;
+        uint32_t i, j;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+                struct rte_rawdev_info rdev_info = { 0 };
+
+                do {
+                    if (rdev_id == rte_rawdev_count())
+                        goto end;
+                    rte_rawdev_info_get(rdev_id++, &rdev_info);
+                } while (strcmp(rdev_info.driver_name,
+                    IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+                cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+                configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+                ++nb_rawdev;
+            }
+        }
+    end:
+        if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+            rte_exit(EXIT_FAILURE,
+                "Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+                nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+    }
+
+
+The initialization of hardware device is done by ``rte_rawdev_configure()``
+function and ``rte_rawdev_info`` struct. After configuration the device is
+started using ``rte_rawdev_start()`` function. Each of the above operations
+is done in ``configure_rawdev_queue()``.
+
+.. code-block:: c
+
+    static void
+    configure_rawdev_queue(uint32_t dev_id)
+    {
+        struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+        /* Configure hardware copy device */
+        dev_config.ring_size = ring_size;
+
+        if (rte_rawdev_configure(dev_id, &info) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_configure()\n");
+        }
+        rte_rawdev_info_get(dev_id, &info);
+        if (dev_config.ring_size != ring_size) {
+            rte_exit(EXIT_FAILURE,
+                "Error, ring size is not %d (%d)\n",
+                ring_size, (int)dev_config.ring_size);
+        }
+        if (rte_rawdev_start(dev_id) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_start()\n");
+        }
+    }
+
+If initialization is successful memory for hardware device
+statistics is allocated.
+
+Finally ``main()`` functions starts all processing lcores and starts
+printing stats in a loop on master lcore. The application can be
+interrupted and closed using ``Ctrl-C``. The master lcore waits for
+all slave processes to finish, deallocates resources and exits.
+
+The processing lcores launching function are described below.
+
+The Lcores Launching Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As described above ``main()`` function invokes ``start_forwarding_cores()``
+function in order to start processing for each lcore:
+
+.. code-block:: c
+
+    static void start_forwarding_cores(void)
+    {
+        uint32_t lcore_id = rte_lcore_id();
+
+        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+                __func__, rte_lcore_id());
+
+        if (cfg.nb_lcores == 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+                NULL, lcore_id);
+        } else if (cfg.nb_lcores > 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+                NULL, lcore_id);
+
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+                lcore_id);
+        }
+    }
+
+The function launches Rx/Tx processing functions on configured lcores
+for each port using ``rte_eal_remote_launch()``. The configured ports,
+their number and number of assigned lcores are stored in user-defined
+``rxtx_transmission_config`` struct that is initialized before launching
+tasks:
+
+.. code-block:: c
+
+    struct rxtx_transmission_config {
+        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+        uint16_t nb_ports;
+        uint16_t nb_lcores;
+    };
+
+The Lcores Processing Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For receiving packets on each port an ``ioat_rx_port()`` function is used.
+The function receives packets on each configured Rx queue. Depending on mode
+the user chose, it will enqueue packets to IOAT rawdev channels and then invoke
+copy process (hardware copy), or perform software copy of each packet using
+``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring:
+
+.. code-block:: c
+
+    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+    static void
+    ioat_rx_port(struct rxtx_port_config *rx_config)
+    {
+        uint32_t nb_rx, nb_enq, i, j;
+        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+        for (i = 0; i < rx_config->nb_queues; i++) {
+
+            nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+                pkts_burst, MAX_PKT_BURST);
+
+            if (nb_rx == 0)
+                continue;
+
+            port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
+            if (copy_mode == COPY_MODE_IOAT_NUM) {
+                /* Perform packet hardware copy */
+                nb_enq = ioat_enqueue_packets(pkts_burst,
+                    nb_rx, rx_config->ioat_ids[i]);
+                if (nb_enq > 0)
+                    rte_ioat_do_copies(rx_config->ioat_ids[i]);
+            } else {
+                /* Perform packet software copy, free source packets */
+                int ret;
+                struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+                ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst_copy, nb_rx);
+
+                if (unlikely(ret < 0))
+                    rte_exit(EXIT_FAILURE,
+                        "Unable to allocate memory.\n");
+
+                for (j = 0; j < nb_rx; j++)
+                    pktmbuf_sw_copy(pkts_burst[j],
+                        pkts_burst_copy[j]);
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst, nb_rx);
+
+                nb_enq = rte_ring_enqueue_burst(
+                    rx_config->rx_to_tx_ring,
+                    (void *)pkts_burst_copy, nb_rx, NULL);
+
+                /* Free any not enqueued packets. */
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&pkts_burst_copy[nb_enq],
+                    nb_rx - nb_enq);
+            }
+
+            port_statistics.copy_dropped[rx_config->rxtx_port] +=
+                (nb_rx - nb_enq);
+        }
+    }
+
+The packets are received in burst mode using ``rte_eth_rx_burst()``
+function. When using hardware copy mode the packets are enqueued in
+copying device's buffer using ``ioat_enqueue_packets()`` which calls
+``rte_ioat_enqueue_copy()``. When all received packets are in the
+buffer the copies are invoked by calling ``rte_ioat_do_copies()``.
+Function ``rte_ioat_enqueue_copy()`` operates on physical address of
+the packet. Structure ``rte_mbuf`` contains only physical address to
+start of the data buffer (``buf_iova``). Thus the address is shifted
+by ``addr_offset`` value in order to get pointer to ``rearm_data``
+member of ``rte_mbuf``. That way the packet is copied all at once
+(with data and metadata).
+
+.. code-block:: c
+
+    static uint32_t
+    ioat_enqueue_packets(struct rte_mbuf **pkts,
+        uint32_t nb_rx, uint16_t dev_id)
+    {
+        int ret;
+        uint32_t i;
+        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+            &pkts[0]->rearm_data);
+
+        ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                (void *)pkts_copy, nb_rx);
+
+        if (unlikely(ret < 0))
+            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+        for (i = 0; i < nb_rx; i++) {
+            /* Perform data copy */
+            ret = rte_ioat_enqueue_copy(dev_id,
+                pkts[i]->buf_iova
+                    - addr_offset,
+                pkts_copy[i]->buf_iova
+                    - addr_offset,
+                rte_pktmbuf_data_len(pkts[i])
+                    + addr_offset,
+                (uintptr_t)pkts[i],
+                (uintptr_t)pkts_copy[i],
+                0 /* nofence */);
+
+            if (ret != 1)
+                break;
+        }
+
+        ret = i;
+        /* Free any not enqueued packets. */
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+            nb_rx - i);
+
+        return ret;
+    }
+
+
+All done copies are processed by ``ioat_tx_port()`` function. When using
+hardware copy mode the function invokes ``rte_ioat_completed_copies()``
+on each assigned IOAT channel to gather copied packets. If software copy
+mode is used the function dequeues copied packets from the rte_ring. Then each
+packet MAC address is changed if it was enabled. After that copies are sent
+in burst mode using `` rte_eth_tx_burst()``.
+
+
+.. code-block:: c
+
+    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
+    static void
+    ioat_tx_port(struct rxtx_port_config *tx_config)
+    {
+        uint32_t i, j, nb_dq = 0;
+        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
+        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+        if (copy_mode == COPY_MODE_IOAT_NUM) {
+            for (i = 0; i < tx_config->nb_queues; i++) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ioat_completed_copies(
+                    tx_config->ioat_ids[i], MAX_PKT_BURST,
+                    (void *)mbufs_src, (void *)mbufs_dst);
+
+                if (nb_dq == 0)
+                    break;
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)mbufs_src, nb_dq);
+
+                /* Update macs if enabled */
+                if (mac_updating) {
+                    for (j = 0; j < nb_dq; j++)
+                        update_mac_addrs(mbufs_dst[j],
+                            tx_config->rxtx_port);
+                }
+
+                const uint16_t nb_tx = rte_eth_tx_burst(
+                    tx_config->rxtx_port, 0,
+                    (void *)mbufs_dst, nb_dq);
+
+                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+                /* Free any unsent packets. */
+                if (unlikely(nb_tx < nb_dq))
+                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&mbufs_dst[nb_tx],
+                        nb_dq - nb_tx);
+            }
+        }
+        else {
+            for (i = 0; i < tx_config->nb_queues; i++) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
+                    (void *)mbufs_dst, MAX_PKT_BURST, NULL);
+
+                if (nb_dq == 0)
+                    return;
+
+                /* Update macs if enabled */
+                if (mac_updating) {
+                    for (j = 0; j < nb_dq; j++)
+                        update_mac_addrs(mbufs_dst[j],
+                            tx_config->rxtx_port);
+                }
+
+                const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
+                    0, (void *)mbufs_dst, nb_dq);
+
+                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+                /* Free any unsent packets. */
+                if (unlikely(nb_tx < nb_dq))
+                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&mbufs_dst[nb_tx],
+                        nb_dq - nb_tx);
+            }
+        }
+    }
+
+The Packet Copying Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to perform packet copy there is a user-defined function
+``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
+metadata from source packet to new mbuf, and then copying a data
+chunk of source packet. Both memory copies are done using
+``rte_memcpy()``:
+
+.. code-block:: c
+
+    static inline void
+    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+    {
+        /* Copy packet metadata */
+        rte_memcpy(&dst->rearm_data,
+            &src->rearm_data,
+            offsetof(struct rte_mbuf, cacheline1)
+                - offsetof(struct rte_mbuf, rearm_data));
+
+        /* Copy packet data */
+        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+            rte_pktmbuf_mtod(src, char *), src->data_len);
+    }
+
+The metadata in this example is copied from ``rearm_data`` member of
+``rte_mbuf`` struct up to ``cacheline1``.
+
+In order to understand why software packet copying is done as shown
+above please refer to the "Mbuf Library" section of the
+*DPDK Programmer's Guide*.
\ No newline at end of file
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 1/6] examples/ioat: create sample app on ioat driver usage
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 1/6] examples/ioat: create " Marcin Baran
@ 2019-09-27  9:58           ` Bruce Richardson
  0 siblings, 0 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27  9:58 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, Pawel Modrak

On Fri, Sep 20, 2019 at 09:37:09AM +0200, Marcin Baran wrote:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> 
> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware
> copy, changes packets' MAC addresses (if enabled) and
> forwards them. The change covers ports initialization,
> closing connection and argument parsing.
> 
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support Marcin Baran
@ 2019-09-27 10:01           ` Bruce Richardson
  2019-09-27 14:01             ` Baran, MarcinX
  0 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27 10:01 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, Pawel Modrak

On Fri, Sep 20, 2019 at 09:37:10AM +0200, Marcin Baran wrote:
> Added support for copying packets
> using software copy mode and MAC address
> changing. The copies are processed using
> one lcore.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> ---
>  examples/ioat/ioatfwd.c | 211 +++++++++++++++++++++++++++++++++++-----
>  1 file changed, 188 insertions(+), 23 deletions(-)
> 
> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
> index 977ea6a61..3a092c6cf 100644
> --- a/examples/ioat/ioatfwd.c
> +++ b/examples/ioat/ioatfwd.c
> @@ -13,7 +13,7 @@
>  #include <rte_rawdev.h>
>  #include <rte_ioat_rawdev.h>
>  
> -/* size of ring used for software copying between rx and tx. */
> + /* size of ring used for software copying between rx and tx. */
>  #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
>  #define MAX_PKT_BURST 32
>  #define MEMPOOL_CACHE_SIZE 512
> @@ -89,6 +89,142 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
>  static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
>  struct rte_mempool *ioat_pktmbuf_pool;
>  
> +static void
> +update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
> +{
> +	struct rte_ether_hdr *eth;
> +	void *tmp;
> +
> +	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
> +
> +	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
> +	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
> +	 */
> +	tmp = &eth->d_addr.addr_bytes[0];
> +	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
> +
> +	/* src addr */
> +	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], &eth->s_addr);
> +}
> +
> +static inline void
> +pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
> +{
> +	/* Copy packet metadata */
> +	rte_memcpy(&dst->rearm_data,
> +		&src->rearm_data,
> +		offsetof(struct rte_mbuf, cacheline1)
> +		- offsetof(struct rte_mbuf, rearm_data));
> +
> +	/* Copy packet data */
> +	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
> +		rte_pktmbuf_mtod(src, char *), src->data_len);
> +}
> +
> +/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
> +static void
> +ioat_rx_port(struct rxtx_port_config *rx_config)
> +{
> +	uint32_t nb_rx, nb_enq, i, j;
> +	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
> +
> +	for (i = 0; i < rx_config->nb_queues; i++) {
> +
> +		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
> +			pkts_burst, MAX_PKT_BURST);
> +
> +		if (nb_rx == 0)
> +			continue;
> +
> +		/* Perform packet software copy, free source packets */
> +		int ret;
> +		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> +
> +		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +			(void *)pkts_burst_copy, nb_rx);
> +
> +		if (unlikely(ret < 0))
> +			rte_exit(EXIT_FAILURE,
> +				"Unable to allocate memory.\n");
> +
> +		for (j = 0; j < nb_rx; j++)
> +			pktmbuf_sw_copy(pkts_burst[j],
> +				pkts_burst_copy[j]);
> +
> +		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +			(void *)pkts_burst, nb_rx);
> +
> +		nb_enq = rte_ring_enqueue_burst(
> +			rx_config->rx_to_tx_ring,
> +			(void *)pkts_burst_copy, nb_rx, NULL);
> +
> +		/* Free any not enqueued packets. */
> +		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +			(void *)&pkts_burst_copy[nb_enq],
> +			nb_rx - nb_enq);
> +	}
> +}
> +
> +/* Transmit packets from IOAT rawdev/rte_ring for one port. */
> +static void
> +ioat_tx_port(struct rxtx_port_config *tx_config)
> +{
> +	uint32_t i, nb_dq = 0;
> +	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
> +
> +	/* Deque the mbufs from rx_to_tx_ring. */
> +	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
> +		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
> +
> +	if (nb_dq == 0)
> +		return;
> +
> +	/* Update macs if enabled */
> +	if (mac_updating) {
> +		for (i = 0; i < nb_dq; i++)
> +			update_mac_addrs(mbufs_dst[i],
> +				tx_config->rxtx_port);
> +	}
> +
> +	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
> +		0, (void *)mbufs_dst, nb_dq);
> +
> +	/* Free any unsent packets. */
> +	if (unlikely(nb_tx < nb_dq))
> +		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +		(void *)&mbufs_dst[nb_tx],
> +			nb_dq - nb_tx);
> +}
> +
> +/* Main rx and tx loop if only one slave lcore available */
> +static void
> +rxtx_main_loop(void)
> +{
> +	uint16_t i;
> +	uint16_t nb_ports = cfg.nb_ports;
> +
> +	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
> +		" lcore %u\n", rte_lcore_id());
> +
> +	while (!force_quit)
> +		for (i = 0; i < nb_ports; i++) {
> +			ioat_rx_port(&cfg.ports[i]);
> +			ioat_tx_port(&cfg.ports[i]);
> +		}
> +}
> +
> +static void start_forwarding_cores(void)
> +{
> +	uint32_t lcore_id = rte_lcore_id();
> +
> +	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
> +		__func__, rte_lcore_id());
> +
> +	lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
> +		NULL, lcore_id);
> +}
> +
>  /* Display usage */
>  static void
>  ioat_usage(const char *prgname)
> @@ -102,7 +238,7 @@ ioat_usage(const char *prgname)
>  		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
>  		"  -c --copy-type CT: type of copy: sw|rawdev\n"
>  		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
> -	       prgname);
> +		prgname);
>  }
>  
>  static int
> @@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
>  	argvopt = argv;
>  
>  	while ((opt = getopt_long(argc, argvopt, short_options,
> -				  lgopts, &option_index)) != EOF) {
> +		lgopts, &option_index)) != EOF) {

These and the other whitespace changes in this patch should be fixed in
patch 1 rather than having them as modifications here.

Without these whitespace changes merged into original patch:

Acked-by: Bruce Richardson <bruce.richardson@intel.com>


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
@ 2019-09-27 10:05           ` Bruce Richardson
  2019-09-27 14:03             ` Baran, MarcinX
  0 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27 10:05 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, Pawel Modrak

On Fri, Sep 20, 2019 at 09:37:11AM +0200, Marcin Baran wrote:
> Added support for copying packets using
> rawdev device. Each port's Rx queue is
> assigned DMA channel for copy.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> ---
>  examples/ioat/ioatfwd.c | 236 ++++++++++++++++++++++++++++++++--------
>  1 file changed, 189 insertions(+), 47 deletions(-)
> 
> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
> index 3a092c6cf..c66ce7e49 100644
> --- a/examples/ioat/ioatfwd.c
> +++ b/examples/ioat/ioatfwd.c
> @@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
>  		rte_pktmbuf_mtod(src, char *), src->data_len);
>  }
>  
> +static uint32_t
> +ioat_enqueue_packets(struct rte_mbuf **pkts,
> +	uint32_t nb_rx, uint16_t dev_id)
> +{
> +	int ret;
> +	uint32_t i;
> +	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
> +
> +	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
> +		&pkts[0]->rearm_data);
> +
> +	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +		(void *)pkts_copy, nb_rx);
> +
> +	if (unlikely(ret < 0))
> +		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
> +
> +	for (i = 0; i < nb_rx; i++) {
> +		/* Perform data copy */
> +		ret = rte_ioat_enqueue_copy(dev_id,
> +			pkts[i]->buf_iova
> +			- addr_offset,
> +			pkts_copy[i]->buf_iova
> +			- addr_offset,
> +			rte_pktmbuf_data_len(pkts[i])
> +			+ addr_offset,
> +			(uintptr_t)pkts[i],
> +			(uintptr_t)pkts_copy[i],
> +			0 /* nofence */);
> +
> +		if (ret != 1)
> +			break;
> +	}
> +
> +	ret = i;
> +	/* Free any not enqueued packets. */
> +	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
> +	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
> +		nb_rx - i);
> +
> +
> +	return ret;
> +}
> +
>  /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
>  static void
>  ioat_rx_port(struct rxtx_port_config *rx_config)
> @@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
>  		if (nb_rx == 0)
>  			continue;
>  
> -		/* Perform packet software copy, free source packets */
> -		int ret;
> -		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> -
> -		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> -			(void *)pkts_burst_copy, nb_rx);
> -
> -		if (unlikely(ret < 0))
> -			rte_exit(EXIT_FAILURE,
> -				"Unable to allocate memory.\n");
> -
> -		for (j = 0; j < nb_rx; j++)
> -			pktmbuf_sw_copy(pkts_burst[j],
> -				pkts_burst_copy[j]);
> -
> -		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> -			(void *)pkts_burst, nb_rx);
> -
> -		nb_enq = rte_ring_enqueue_burst(
> -			rx_config->rx_to_tx_ring,
> -			(void *)pkts_burst_copy, nb_rx, NULL);
> -
> -		/* Free any not enqueued packets. */
> -		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> -			(void *)&pkts_burst_copy[nb_enq],
> -			nb_rx - nb_enq);
> +		if (copy_mode == COPY_MODE_IOAT_NUM) {
> +			/* Perform packet hardware copy */
> +			nb_enq = ioat_enqueue_packets(pkts_burst,
> +				nb_rx, rx_config->ioat_ids[i]);
> +			if (nb_enq > 0)
> +				rte_ioat_do_copies(rx_config->ioat_ids[i]);
> +		} else {
> +			/* Perform packet software copy, free source packets */
> +			int ret;
> +			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> +
> +			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +				(void *)pkts_burst_copy, nb_rx);
> +
> +			if (unlikely(ret < 0))
> +				rte_exit(EXIT_FAILURE,
> +					"Unable to allocate memory.\n");
> +
> +			for (j = 0; j < nb_rx; j++)
> +				pktmbuf_sw_copy(pkts_burst[j],
> +					pkts_burst_copy[j]);
> +
> +			rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +				(void *)pkts_burst, nb_rx);
> +
> +			nb_enq = rte_ring_enqueue_burst(
> +				rx_config->rx_to_tx_ring,
> +				(void *)pkts_burst_copy, nb_rx, NULL);
> +
> +			/* Free any not enqueued packets. */
> +			rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +				(void *)&pkts_burst_copy[nb_enq],
> +				nb_rx - nb_enq);
> +		}

Would the diff in this patch be smaller if you switched the order of the
branches so that the SW copy leg, which was added first, was processed
first? You could even add in a dummy branch in patch 2, so that the
indentation for that section remains unchanged.

/Bruce

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration Marcin Baran
@ 2019-09-27 10:08           ` Bruce Richardson
  2019-09-27 14:03             ` Baran, MarcinX
  0 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27 10:08 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, Pawel Modrak

On Fri, Sep 20, 2019 at 09:37:12AM +0200, Marcin Baran wrote:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> 
> Added possibility to use two lcores: first for
> packet receiving and copying, second for packets
> sending.
> 
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  examples/ioat/ioatfwd.c | 92 ++++++++++++++++++++++++++++++-----------
>  1 file changed, 67 insertions(+), 25 deletions(-)
> 
> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
> index c66ce7e49..4c51db6bd 100644
> --- a/examples/ioat/ioatfwd.c
> +++ b/examples/ioat/ioatfwd.c
> @@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,
>  	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
>  		nb_rx - i);
>  
> -
>  	return ret;
>  }
>  
> @@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
>  	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
>  
>  	if (copy_mode == COPY_MODE_IOAT_NUM) {
> -		/* Deque the mbufs from IOAT device. */
>  		for (i = 0; i < tx_config->nb_queues; i++) {
> +			/* Deque the mbufs from IOAT device. */
>  			nb_dq = rte_ioat_completed_copies(
>  				tx_config->ioat_ids[i], MAX_PKT_BURST,
>  				(void *)mbufs_src, (void *)mbufs_dst);
> @@ -256,31 +255,64 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
>  					nb_dq - nb_tx);
>  		}
>  	} else {
> -		/* Deque the mbufs from rx_to_tx_ring. */
> -		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
> -			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
> -
> -		if (nb_dq == 0)
> -			return;
> -
> -		/* Update macs if enabled */
> -		if (mac_updating) {
> -			for (i = 0; i < nb_dq; i++)
> -				update_mac_addrs(mbufs_dst[i],
> -					tx_config->rxtx_port);
> -		}
> +		for (i = 0; i < tx_config->nb_queues; i++) {

This commit refers to adding support for 2 threads rather than 1, but this
change seems related to having multiple queues rather than threads. Shoudl
it be in a previous patch?

/Bruce


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port Marcin Baran
@ 2019-09-27 10:12           ` Bruce Richardson
  2019-09-27 14:04             ` Baran, MarcinX
  0 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27 10:12 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, Pawel Modrak

On Fri, Sep 20, 2019 at 09:37:13AM +0200, Marcin Baran wrote:
> Added printing stats of ports each second.
> The stats printing is done using master core.
> The information provided informs about packets
> received, dropped and send as well as statistics
> of rawdev devices.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> ---
>  examples/ioat/ioatfwd.c | 245 +++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 241 insertions(+), 4 deletions(-)
> 
> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
> index 4c51db6bd..c6b994832 100644
> --- a/examples/ioat/ioatfwd.c
> +++ b/examples/ioat/ioatfwd.c
> @@ -48,10 +48,27 @@ struct rxtx_transmission_config {
>  	uint16_t nb_lcores;
>  };
>  

<snip stats code>

> @@ -363,7 +587,7 @@ ioat_usage(const char *prgname)
>  		"      When enabled:\n"
>  		"       - The source MAC address is replaced by the TX port MAC address\n"
>  		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
> -		"  -c --copy-type CT: type of copy: sw|rawdev\n"
> +		"  -c --copy-type CT: type of copy: sw|hw\n"
>  		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
>  		prgname);
>  }
This change is not relevant to this patch, and should be corrected in the
original patch of the series. Try to avoid adding a line and then modifying
it in a subsequent patch in the same set.

/Bruce

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
@ 2019-09-27 10:36           ` Bruce Richardson
  2019-09-27 14:14             ` Baran, MarcinX
  2019-09-27 10:37           ` Bruce Richardson
                             ` (2 subsequent siblings)
  3 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27 10:36 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev

On Fri, Sep 20, 2019 at 09:37:14AM +0200, Marcin Baran wrote:
> Added guide for IOAT sample app usage and
> code description.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  doc/guides/sample_app_ug/index.rst |   1 +
>  doc/guides/sample_app_ug/intro.rst |   4 +
>  doc/guides/sample_app_ug/ioat.rst  | 764 +++++++++++++++++++++++++++++
>  3 files changed, 769 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> 
> diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
> index f23f8f59e..a6a1d9e7a 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -23,6 +23,7 @@ Sample Applications User Guides
>      ip_reassembly
>      kernel_nic_interface
>      keep_alive
> +    ioat
>      l2_forward_crypto
>      l2_forward_job_stats
>      l2_forward_real_virtual
> diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
> index 90704194a..74462312f 100644
> --- a/doc/guides/sample_app_ug/intro.rst
> +++ b/doc/guides/sample_app_ug/intro.rst
> @@ -91,6 +91,10 @@ examples are highlighted below.
>    forwarding, or ``l3fwd`` application does forwarding based on Internet
>    Protocol, IPv4 or IPv6 like a simple router.
>  
> +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
> +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
> +  copying packets between two threads.
> +
>  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
>    demonstrates how to distribute packets arriving on an Rx port to different
>    cores for processing and transmission.
> diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
> new file mode 100644
> index 000000000..69621673b
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/ioat.rst
> @@ -0,0 +1,764 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Sample Application of packet copying using Intel\ |reg| QuickData Technology
> +============================================================================
> +

Title is too long, you can drop the "Sample Application" at minimum, since
this is part of the example applications guide document. Call the section
"Packet Copying Using ..."

In order to get the proper (R) symbol, you also need to add an include to
the file. Therefore add the following line just after the copyright:

.. include:: <isonum.txt>

/Bruce

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
  2019-09-27 10:36           ` Bruce Richardson
@ 2019-09-27 10:37           ` Bruce Richardson
  2019-09-27 11:01           ` Bruce Richardson
  2019-09-27 13:22           ` Bruce Richardson
  3 siblings, 0 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27 10:37 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, John McNamara, Marko Kovacevic

Adding John and Marko on CC as documentation maintainers.

On Fri, Sep 20, 2019 at 09:37:14AM +0200, Marcin Baran wrote:
> Added guide for IOAT sample app usage and
> code description.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  doc/guides/sample_app_ug/index.rst |   1 +
>  doc/guides/sample_app_ug/intro.rst |   4 +
>  doc/guides/sample_app_ug/ioat.rst  | 764 +++++++++++++++++++++++++++++
>  3 files changed, 769 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> 
> diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
> index f23f8f59e..a6a1d9e7a 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -23,6 +23,7 @@ Sample Applications User Guides
>      ip_reassembly
>      kernel_nic_interface
>      keep_alive
> +    ioat
>      l2_forward_crypto
>      l2_forward_job_stats
>      l2_forward_real_virtual
> diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
> index 90704194a..74462312f 100644
> --- a/doc/guides/sample_app_ug/intro.rst
> +++ b/doc/guides/sample_app_ug/intro.rst
> @@ -91,6 +91,10 @@ examples are highlighted below.
>    forwarding, or ``l3fwd`` application does forwarding based on Internet
>    Protocol, IPv4 or IPv6 like a simple router.
>  
> +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
> +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
> +  copying packets between two threads.
> +
>  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
>    demonstrates how to distribute packets arriving on an Rx port to different
>    cores for processing and transmission.
> diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
> new file mode 100644
> index 000000000..69621673b
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/ioat.rst
> @@ -0,0 +1,764 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Sample Application of packet copying using Intel\ |reg| QuickData Technology
> +============================================================================
> +
> +Overview
> +--------
> +
> +This sample is intended as a demonstration of the basic components of a DPDK
> +forwarding application and example of how to use IOAT driver API to make
> +packets copies.
> +
> +Also while forwarding, the MAC addresses are affected as follows:
> +
> +*   The source MAC address is replaced by the TX port MAC address
> +
> +*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
> +
> +This application can be used to compare performance of using software packet
> +copy with copy done using a DMA device for different sizes of packets.
> +The example will print out statistics each second. The stats shows
> +received/send packets and packets dropped or failed to copy.
> +
> +Compiling the Application
> +-------------------------
> +
> +To compile the sample application see :doc:`compiling`.
> +
> +The application is located in the ``ioat`` sub-directory.
> +
> +
> +Running the Application
> +-----------------------
> +
> +In order to run the hardware copy application, the copying device
> +needs to be bound to user-space IO driver.
> +
> +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology*
> +guide for information on using the driver.
> +
> +The application requires a number of command line options:
> +
> +.. code-block:: console
> +
> +    ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>]
> +        [--[no-]mac-updating]
> +
> +where,
> +
> +*   p MASK: A hexadecimal bitmask of the ports to configure
> +
> +*   q NQ: Number of Rx queues used per port equivalent to CBDMA channels
> +    per port
> +
> +*   c CT: Performed packet copy type: software (sw) or hardware using
> +    DMA (hw)
> +
> +*   s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for
> +    software copy mode
> +
> +*   --[no-]mac-updating: Whether MAC address of packets should be changed
> +    or not
> +
> +The application can be launched in various configurations depending on
> +provided parameters. Each port can use up to 2 lcores: one of them receives
> +incoming traffic and makes a copy of each packet. The second lcore then
> +updates MAC address and sends the copy. If one lcore per port is used,
> +both operations are done sequentially. For each configuration an additional
> +lcore is needed since master lcore in use which is responsible for
> +configuration, statistics printing and safe deinitialization of all ports
> +and devices.
> +
> +The application can use a maximum of 8 ports.
> +
> +To run the application in a Linux environment with 3 lcores (one of them
> +is master lcore), 1 port (port 0), software copying and MAC updating issue
> +the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
> +
> +To run the application in a Linux environment with 2 lcores (one of them
> +is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC
> +updating issue the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw
> +
> +Refer to the *DPDK Getting Started Guide* for general information on
> +running applications and the Environment Abstraction Layer (EAL) options.
> +
> +Explanation
> +-----------
> +
> +The following sections provide an explanation of the main components of the
> +code.
> +
> +All DPDK library functions used in the sample code are prefixed with
> +``rte_`` and are explained in detail in the *DPDK API Documentation*.
> +
> +
> +The Main Function
> +~~~~~~~~~~~~~~~~~
> +
> +The ``main()`` function performs the initialization and calls the execution
> +threads for each lcore.
> +
> +The first task is to initialize the Environment Abstraction Layer (EAL).
> +The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
> +function. The value returned is the number of parsed arguments:
> +
> +.. code-block:: c
> +
> +    /* init EAL */
> +    ret = rte_eal_init(argc, argv);
> +    if (ret < 0)
> +        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
> +
> +
> +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
> +used by the application:
> +
> +.. code-block:: c
> +
> +    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
> +        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
> +        MIN_POOL_SIZE);
> +
> +    /* Create the mbuf pool */
> +    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
> +        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
> +        rte_socket_id());
> +    if (ioat_pktmbuf_pool == NULL)
> +        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
> +
> +Mbufs are the packet buffer structure used by DPDK. They are explained in
> +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
> +
> +The ``main()`` function also initializes the ports:
> +
> +.. code-block:: c
> +
> +    /* Initialise each port */
> +    RTE_ETH_FOREACH_DEV(portid) {
> +        port_init(portid, ioat_pktmbuf_pool);
> +    }
> +
> +Each port is configured using ``port_init()``:
> +
> +.. code-block:: c
> +
> +     /*
> +     * Initializes a given port using global settings and with the RX buffers
> +     * coming from the mbuf_pool passed as a parameter.
> +     */
> +    static inline void
> +    port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
> +    {
> +        /* configuring port to use RSS for multiple RX queues */
> +        static const struct rte_eth_conf port_conf = {
> +            .rxmode = {
> +                .mq_mode        = ETH_MQ_RX_RSS,
> +                .max_rx_pkt_len = RTE_ETHER_MAX_LEN
> +            },
> +            .rx_adv_conf = {
> +                .rss_conf = {
> +                    .rss_key = NULL,
> +                    .rss_hf = ETH_RSS_PROTO_MASK,
> +                }
> +            }
> +        };
> +
> +        struct rte_eth_rxconf rxq_conf;
> +        struct rte_eth_txconf txq_conf;
> +        struct rte_eth_conf local_port_conf = port_conf;
> +        struct rte_eth_dev_info dev_info;
> +        int ret, i;
> +
> +        /* Skip ports that are not enabled */
> +        if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
> +            printf("Skipping disabled port %u\n", portid);
> +            return;
> +        }
> +
> +        /* Init port */
> +        printf("Initializing port %u... ", portid);
> +        fflush(stdout);
> +        rte_eth_dev_info_get(portid, &dev_info);
> +        local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
> +            dev_info.flow_type_rss_offloads;
> +        if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
> +            local_port_conf.txmode.offloads |=
> +                DEV_TX_OFFLOAD_MBUF_FAST_FREE;
> +        ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE, "Cannot configure device:"
> +                " err=%d, port=%u\n", ret, portid);
> +
> +        ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
> +                            &nb_txd);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "Cannot adjust number of descriptors: err=%d, port=%u\n",
> +                ret, portid);
> +
> +        rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
> +
> +        /* Init Rx queues */
> +        rxq_conf = dev_info.default_rxconf;
> +        rxq_conf.offloads = local_port_conf.rxmode.offloads;
> +        for (i = 0; i < nb_queues; i++) {
> +            ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
> +                rte_eth_dev_socket_id(portid), &rxq_conf,
> +                mbuf_pool);
> +            if (ret < 0)
> +                rte_exit(EXIT_FAILURE,
> +                    "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
> +                    ret, portid, i);
> +        }
> +
> +        /* Init one TX queue on each port */
> +        txq_conf = dev_info.default_txconf;
> +        txq_conf.offloads = local_port_conf.txmode.offloads;
> +        ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
> +                rte_eth_dev_socket_id(portid),
> +                &txq_conf);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "rte_eth_tx_queue_setup:err=%d,port=%u\n",
> +                ret, portid);
> +
> +        /* Initialize TX buffers */
> +        tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
> +                RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
> +                rte_eth_dev_socket_id(portid));
> +        if (tx_buffer[portid] == NULL)
> +            rte_exit(EXIT_FAILURE,
> +                "Cannot allocate buffer for tx on port %u\n",
> +                portid);
> +
> +        rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
> +
> +        ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
> +                rte_eth_tx_buffer_count_callback,
> +                &port_statistics.tx_dropped[portid]);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "Cannot set error callback for tx buffer on port %u\n",
> +                portid);
> +
> +        /* Start device */
> +        ret = rte_eth_dev_start(portid);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "rte_eth_dev_start:err=%d, port=%u\n",
> +                ret, portid);
> +
> +        rte_eth_promiscuous_enable(portid);
> +
> +        printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
> +                portid,
> +                ioat_ports_eth_addr[portid].addr_bytes[0],
> +                ioat_ports_eth_addr[portid].addr_bytes[1],
> +                ioat_ports_eth_addr[portid].addr_bytes[2],
> +                ioat_ports_eth_addr[portid].addr_bytes[3],
> +                ioat_ports_eth_addr[portid].addr_bytes[4],
> +                ioat_ports_eth_addr[portid].addr_bytes[5]);
> +
> +        cfg.ports[cfg.nb_ports].rxtx_port = portid;
> +        cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
> +    }
> +
> +The Ethernet ports are configured with local settings using the
> +``rte_eth_dev_configure()`` function and the ``port_conf`` struct.
> +The RSS is enabled so that multiple Rx queues could be used for
> +packet receiving and copying by multiple CBDMA channels per port:
> +
> +.. code-block:: c
> +
> +    /* configuring port to use RSS for multiple RX queues */
> +    static const struct rte_eth_conf port_conf = {
> +        .rxmode = {
> +            .mq_mode        = ETH_MQ_RX_RSS,
> +            .max_rx_pkt_len = RTE_ETHER_MAX_LEN
> +        },
> +        .rx_adv_conf = {
> +            .rss_conf = {
> +                .rss_key = NULL,
> +                .rss_hf = ETH_RSS_PROTO_MASK,
> +            }
> +        }
> +    };
> +
> +For this example the ports are set up with the number of Rx queues provided
> +with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()``
> +and ``rte_eth_tx_queue_setup()`` functions.
> +
> +The Ethernet port is then started:
> +
> +.. code-block:: c
> +
> +    ret = rte_eth_dev_start(portid);
> +    if (ret < 0)
> +        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
> +            ret, portid);
> +
> +
> +Finally the Rx port is set in promiscuous mode:
> +
> +.. code-block:: c
> +
> +    rte_eth_promiscuous_enable(portid);
> +
> +
> +After that each port application assigns resources needed.
> +
> +.. code-block:: c
> +
> +    check_link_status(ioat_enabled_port_mask);
> +
> +    if (!cfg.nb_ports) {
> +        rte_exit(EXIT_FAILURE,
> +            "All available ports are disabled. Please set portmask.\n");
> +    }
> +
> +    /* Check if there is enough lcores for all ports. */
> +    cfg.nb_lcores = rte_lcore_count() - 1;
> +    if (cfg.nb_lcores < 1)
> +        rte_exit(EXIT_FAILURE,
> +            "There should be at least one slave lcore.\n");
> +
> +    ret = 0;
> +
> +    if (copy_mode == COPY_MODE_IOAT_NUM) {
> +        assign_rawdevs();
> +    } else /* copy_mode == COPY_MODE_SW_NUM */ {
> +        assign_rings();
> +    }
> +
> +A link status is checked of each port enabled by port mask
> +using ``check_link_status()`` function.
> +
> +.. code-block:: c
> +
> +    /* check link status, return true if at least one port is up */
> +    static int
> +    check_link_status(uint32_t port_mask)
> +    {
> +        uint16_t portid;
> +        struct rte_eth_link link;
> +        int retval = 0;
> +
> +        printf("\nChecking link status\n");
> +        RTE_ETH_FOREACH_DEV(portid) {
> +            if ((port_mask & (1 << portid)) == 0)
> +                continue;
> +
> +            memset(&link, 0, sizeof(link));
> +            rte_eth_link_get(portid, &link);
> +
> +            /* Print link status */
> +            if (link.link_status) {
> +                printf(
> +                    "Port %d Link Up. Speed %u Mbps - %s\n",
> +                    portid, link.link_speed,
> +                    (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
> +                    ("full-duplex") : ("half-duplex\n"));
> +                retval = 1;
> +            } else
> +                printf("Port %d Link Down\n", portid);
> +        }
> +        return retval;
> +    }
> +
> +Depending on mode set (whether copy should be done by software or by hardware)
> +special structures are assigned to each port. If software copy was chosen,
> +application have to assign ring structures for packet exchanging between lcores
> +assigned to ports.
> +
> +.. code-block:: c
> +
> +    static void
> +    assign_rings(void)
> +    {
> +        uint32_t i;
> +
> +        for (i = 0; i < cfg.nb_ports; i++) {
> +            char ring_name[20];
> +
> +            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
> +            /* Create ring for inter core communication */
> +            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
> +                    ring_name, ring_size,
> +                    rte_socket_id(), RING_F_SP_ENQ);
> +
> +            if (cfg.ports[i].rx_to_tx_ring == NULL)
> +                rte_exit(EXIT_FAILURE, "%s\n",
> +                        rte_strerror(rte_errno));
> +        }
> +    }
> +
> +
> +When using hardware copy each Rx queue of the port is assigned an
> +IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API
> +functions:
> +
> +.. code-block:: c
> +
> +    static void
> +    assign_rawdevs(void)
> +    {
> +        uint16_t nb_rawdev = 0, rdev_id = 0;
> +        uint32_t i, j;
> +
> +        for (i = 0; i < cfg.nb_ports; i++) {
> +            for (j = 0; j < cfg.ports[i].nb_queues; j++) {
> +                struct rte_rawdev_info rdev_info = { 0 };
> +
> +                do {
> +                    if (rdev_id == rte_rawdev_count())
> +                        goto end;
> +                    rte_rawdev_info_get(rdev_id++, &rdev_info);
> +                } while (strcmp(rdev_info.driver_name,
> +                    IOAT_PMD_RAWDEV_NAME_STR) != 0);
> +
> +                cfg.ports[i].ioat_ids[j] = rdev_id - 1;
> +                configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
> +                ++nb_rawdev;
> +            }
> +        }
> +    end:
> +        if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
> +            rte_exit(EXIT_FAILURE,
> +                "Not enough IOAT rawdevs (%u) for all queues (%u).\n",
> +                nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
> +        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
> +    }
> +
> +
> +The initialization of hardware device is done by ``rte_rawdev_configure()``
> +function and ``rte_rawdev_info`` struct. After configuration the device is
> +started using ``rte_rawdev_start()`` function. Each of the above operations
> +is done in ``configure_rawdev_queue()``.
> +
> +.. code-block:: c
> +
> +    static void
> +    configure_rawdev_queue(uint32_t dev_id)
> +    {
> +        struct rte_rawdev_info info = { .dev_private = &dev_config };
> +
> +        /* Configure hardware copy device */
> +        dev_config.ring_size = ring_size;
> +
> +        if (rte_rawdev_configure(dev_id, &info) != 0) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error with rte_rawdev_configure()\n");
> +        }
> +        rte_rawdev_info_get(dev_id, &info);
> +        if (dev_config.ring_size != ring_size) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error, ring size is not %d (%d)\n",
> +                ring_size, (int)dev_config.ring_size);
> +        }
> +        if (rte_rawdev_start(dev_id) != 0) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error with rte_rawdev_start()\n");
> +        }
> +    }
> +
> +If initialization is successful memory for hardware device
> +statistics is allocated.
> +
> +Finally ``main()`` functions starts all processing lcores and starts
> +printing stats in a loop on master lcore. The application can be
> +interrupted and closed using ``Ctrl-C``. The master lcore waits for
> +all slave processes to finish, deallocates resources and exits.
> +
> +The processing lcores launching function are described below.
> +
> +The Lcores Launching Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +As described above ``main()`` function invokes ``start_forwarding_cores()``
> +function in order to start processing for each lcore:
> +
> +.. code-block:: c
> +
> +    static void start_forwarding_cores(void)
> +    {
> +        uint32_t lcore_id = rte_lcore_id();
> +
> +        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
> +                __func__, rte_lcore_id());
> +
> +        if (cfg.nb_lcores == 1) {
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
> +                NULL, lcore_id);
> +        } else if (cfg.nb_lcores > 1) {
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
> +                NULL, lcore_id);
> +
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
> +                lcore_id);
> +        }
> +    }
> +
> +The function launches Rx/Tx processing functions on configured lcores
> +for each port using ``rte_eal_remote_launch()``. The configured ports,
> +their number and number of assigned lcores are stored in user-defined
> +``rxtx_transmission_config`` struct that is initialized before launching
> +tasks:
> +
> +.. code-block:: c
> +
> +    struct rxtx_transmission_config {
> +        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
> +        uint16_t nb_ports;
> +        uint16_t nb_lcores;
> +    };
> +
> +The Lcores Processing Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +For receiving packets on each port an ``ioat_rx_port()`` function is used.
> +The function receives packets on each configured Rx queue. Depending on mode
> +the user chose, it will enqueue packets to IOAT rawdev channels and then invoke
> +copy process (hardware copy), or perform software copy of each packet using
> +``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring:
> +
> +.. code-block:: c
> +
> +    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
> +    static void
> +    ioat_rx_port(struct rxtx_port_config *rx_config)
> +    {
> +        uint32_t nb_rx, nb_enq, i, j;
> +        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
> +        for (i = 0; i < rx_config->nb_queues; i++) {
> +
> +            nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
> +                pkts_burst, MAX_PKT_BURST);
> +
> +            if (nb_rx == 0)
> +                continue;
> +
> +            port_statistics.rx[rx_config->rxtx_port] += nb_rx;
> +
> +            if (copy_mode == COPY_MODE_IOAT_NUM) {
> +                /* Perform packet hardware copy */
> +                nb_enq = ioat_enqueue_packets(pkts_burst,
> +                    nb_rx, rx_config->ioat_ids[i]);
> +                if (nb_enq > 0)
> +                    rte_ioat_do_copies(rx_config->ioat_ids[i]);
> +            } else {
> +                /* Perform packet software copy, free source packets */
> +                int ret;
> +                struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> +
> +                ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +                    (void *)pkts_burst_copy, nb_rx);
> +
> +                if (unlikely(ret < 0))
> +                    rte_exit(EXIT_FAILURE,
> +                        "Unable to allocate memory.\n");
> +
> +                for (j = 0; j < nb_rx; j++)
> +                    pktmbuf_sw_copy(pkts_burst[j],
> +                        pkts_burst_copy[j]);
> +
> +                rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)pkts_burst, nb_rx);
> +
> +                nb_enq = rte_ring_enqueue_burst(
> +                    rx_config->rx_to_tx_ring,
> +                    (void *)pkts_burst_copy, nb_rx, NULL);
> +
> +                /* Free any not enqueued packets. */
> +                rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)&pkts_burst_copy[nb_enq],
> +                    nb_rx - nb_enq);
> +            }
> +
> +            port_statistics.copy_dropped[rx_config->rxtx_port] +=
> +                (nb_rx - nb_enq);
> +        }
> +    }
> +
> +The packets are received in burst mode using ``rte_eth_rx_burst()``
> +function. When using hardware copy mode the packets are enqueued in
> +copying device's buffer using ``ioat_enqueue_packets()`` which calls
> +``rte_ioat_enqueue_copy()``. When all received packets are in the
> +buffer the copies are invoked by calling ``rte_ioat_do_copies()``.
> +Function ``rte_ioat_enqueue_copy()`` operates on physical address of
> +the packet. Structure ``rte_mbuf`` contains only physical address to
> +start of the data buffer (``buf_iova``). Thus the address is shifted
> +by ``addr_offset`` value in order to get pointer to ``rearm_data``
> +member of ``rte_mbuf``. That way the packet is copied all at once
> +(with data and metadata).
> +
> +.. code-block:: c
> +
> +    static uint32_t
> +    ioat_enqueue_packets(struct rte_mbuf **pkts,
> +        uint32_t nb_rx, uint16_t dev_id)
> +    {
> +        int ret;
> +        uint32_t i;
> +        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
> +
> +        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
> +            &pkts[0]->rearm_data);
> +
> +        ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +                (void *)pkts_copy, nb_rx);
> +
> +        if (unlikely(ret < 0))
> +            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
> +
> +        for (i = 0; i < nb_rx; i++) {
> +            /* Perform data copy */
> +            ret = rte_ioat_enqueue_copy(dev_id,
> +                pkts[i]->buf_iova
> +                    - addr_offset,
> +                pkts_copy[i]->buf_iova
> +                    - addr_offset,
> +                rte_pktmbuf_data_len(pkts[i])
> +                    + addr_offset,
> +                (uintptr_t)pkts[i],
> +                (uintptr_t)pkts_copy[i],
> +                0 /* nofence */);
> +
> +            if (ret != 1)
> +                break;
> +        }
> +
> +        ret = i;
> +        /* Free any not enqueued packets. */
> +        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
> +        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
> +            nb_rx - i);
> +
> +        return ret;
> +    }
> +
> +
> +All done copies are processed by ``ioat_tx_port()`` function. When using
> +hardware copy mode the function invokes ``rte_ioat_completed_copies()``
> +on each assigned IOAT channel to gather copied packets. If software copy
> +mode is used the function dequeues copied packets from the rte_ring. Then each
> +packet MAC address is changed if it was enabled. After that copies are sent
> +in burst mode using `` rte_eth_tx_burst()``.
> +
> +
> +.. code-block:: c
> +
> +    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
> +    static void
> +    ioat_tx_port(struct rxtx_port_config *tx_config)
> +    {
> +        uint32_t i, j, nb_dq = 0;
> +        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
> +        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
> +
> +        if (copy_mode == COPY_MODE_IOAT_NUM) {
> +            for (i = 0; i < tx_config->nb_queues; i++) {
> +                /* Deque the mbufs from IOAT device. */
> +                nb_dq = rte_ioat_completed_copies(
> +                    tx_config->ioat_ids[i], MAX_PKT_BURST,
> +                    (void *)mbufs_src, (void *)mbufs_dst);
> +
> +                if (nb_dq == 0)
> +                    break;
> +
> +                rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)mbufs_src, nb_dq);
> +
> +                /* Update macs if enabled */
> +                if (mac_updating) {
> +                    for (j = 0; j < nb_dq; j++)
> +                        update_mac_addrs(mbufs_dst[j],
> +                            tx_config->rxtx_port);
> +                }
> +
> +                const uint16_t nb_tx = rte_eth_tx_burst(
> +                    tx_config->rxtx_port, 0,
> +                    (void *)mbufs_dst, nb_dq);
> +
> +                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
> +
> +                /* Free any unsent packets. */
> +                if (unlikely(nb_tx < nb_dq))
> +                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)&mbufs_dst[nb_tx],
> +                        nb_dq - nb_tx);
> +            }
> +        }
> +        else {
> +            for (i = 0; i < tx_config->nb_queues; i++) {
> +                /* Deque the mbufs from IOAT device. */
> +                nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
> +                    (void *)mbufs_dst, MAX_PKT_BURST, NULL);
> +
> +                if (nb_dq == 0)
> +                    return;
> +
> +                /* Update macs if enabled */
> +                if (mac_updating) {
> +                    for (j = 0; j < nb_dq; j++)
> +                        update_mac_addrs(mbufs_dst[j],
> +                            tx_config->rxtx_port);
> +                }
> +
> +                const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
> +                    0, (void *)mbufs_dst, nb_dq);
> +
> +                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
> +
> +                /* Free any unsent packets. */
> +                if (unlikely(nb_tx < nb_dq))
> +                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)&mbufs_dst[nb_tx],
> +                        nb_dq - nb_tx);
> +            }
> +        }
> +    }
> +
> +The Packet Copying Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In order to perform packet copy there is a user-defined function
> +``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
> +metadata from source packet to new mbuf, and then copying a data
> +chunk of source packet. Both memory copies are done using
> +``rte_memcpy()``:
> +
> +.. code-block:: c
> +
> +    static inline void
> +    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
> +    {
> +        /* Copy packet metadata */
> +        rte_memcpy(&dst->rearm_data,
> +            &src->rearm_data,
> +            offsetof(struct rte_mbuf, cacheline1)
> +                - offsetof(struct rte_mbuf, rearm_data));
> +
> +        /* Copy packet data */
> +        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
> +            rte_pktmbuf_mtod(src, char *), src->data_len);
> +    }
> +
> +The metadata in this example is copied from ``rearm_data`` member of
> +``rte_mbuf`` struct up to ``cacheline1``.
> +
> +In order to understand why software packet copying is done as shown
> +above please refer to the "Mbuf Library" section of the
> +*DPDK Programmer's Guide*.
> \ No newline at end of file
> -- 
> 2.22.0.windows.1
> 

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
  2019-09-27 10:36           ` Bruce Richardson
  2019-09-27 10:37           ` Bruce Richardson
@ 2019-09-27 11:01           ` Bruce Richardson
  2019-09-27 14:51             ` Baran, MarcinX
  2019-09-27 13:22           ` Bruce Richardson
  3 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27 11:01 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev

On Fri, Sep 20, 2019 at 09:37:14AM +0200, Marcin Baran wrote:
> Added guide for IOAT sample app usage and
> code description.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  doc/guides/sample_app_ug/index.rst |   1 +
>  doc/guides/sample_app_ug/intro.rst |   4 +
>  doc/guides/sample_app_ug/ioat.rst  | 764 +++++++++++++++++++++++++++++
>  3 files changed, 769 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> 
> diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
> index f23f8f59e..a6a1d9e7a 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -23,6 +23,7 @@ Sample Applications User Guides
>      ip_reassembly
>      kernel_nic_interface
>      keep_alive
> +    ioat
>      l2_forward_crypto
>      l2_forward_job_stats
>      l2_forward_real_virtual
> diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
> index 90704194a..74462312f 100644
> --- a/doc/guides/sample_app_ug/intro.rst
> +++ b/doc/guides/sample_app_ug/intro.rst
> @@ -91,6 +91,10 @@ examples are highlighted below.
>    forwarding, or ``l3fwd`` application does forwarding based on Internet
>    Protocol, IPv4 or IPv6 like a simple router.
>  
> +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
> +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
> +  copying packets between two threads.
> +
>  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
>    demonstrates how to distribute packets arriving on an Rx port to different
>    cores for processing and transmission.
> diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
> new file mode 100644
> index 000000000..69621673b
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/ioat.rst
> @@ -0,0 +1,764 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Sample Application of packet copying using Intel\ |reg| QuickData Technology
> +============================================================================
> +
> +Overview
> +--------
> +
> +This sample is intended as a demonstration of the basic components of a DPDK
> +forwarding application and example of how to use IOAT driver API to make
> +packets copies.
> +
> +Also while forwarding, the MAC addresses are affected as follows:
> +
> +*   The source MAC address is replaced by the TX port MAC address
> +
> +*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
> +
> +This application can be used to compare performance of using software packet
> +copy with copy done using a DMA device for different sizes of packets.
> +The example will print out statistics each second. The stats shows
> +received/send packets and packets dropped or failed to copy.
> +
> +Compiling the Application
> +-------------------------
> +
> +To compile the sample application see :doc:`compiling`.
> +
> +The application is located in the ``ioat`` sub-directory.
> +
> +
> +Running the Application
> +-----------------------
> +
> +In order to run the hardware copy application, the copying device
> +needs to be bound to user-space IO driver.
> +
> +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData Technology*
> +guide for information on using the driver.
> +

The document is not called that, as the IOAT guide is just part of the
overall rawdev document. So I suggest you just reference the rawdev guide.

> +The application requires a number of command line options:
> +
> +.. code-block:: console
> +
> +    ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>]
> +        [--[no-]mac-updating]
> +
> +where,
> +
> +*   p MASK: A hexadecimal bitmask of the ports to configure

Is this a mandatory parameter, or does the app use all detected ports by
default, e.g. like testpmd?

> +
> +*   q NQ: Number of Rx queues used per port equivalent to CBDMA channels
> +    per port
> +
> +*   c CT: Performed packet copy type: software (sw) or hardware using
> +    DMA (hw)

What is the default? Same for next two parameters.

> +
> +*   s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for
> +    software copy mode
> +
> +*   --[no-]mac-updating: Whether MAC address of packets should be changed
> +    or not
> +
> +The application can be launched in various configurations depending on
> +provided parameters. Each port can use up to 2 lcores: one of them receives

The app uses 2 data plane cores, total, rather than 2 per-port, I believe.
It would be good to explain the difference here that with 2 cores the
copies are done on one core, and the mac updates on the second one.

> +incoming traffic and makes a copy of each packet. The second lcore then
> +updates MAC address and sends the copy. If one lcore per port is used,
> +both operations are done sequentially. For each configuration an additional
> +lcore is needed since master lcore in use which is responsible for

... since the master lcore does not handle traffic but is responsible for

> +configuration, statistics printing and safe deinitialization of all ports
> +and devices.

s/deinitialization/shutdown/

> +
> +The application can use a maximum of 8 ports.

Is this a hard limit in the app, if so explain why. I see the stats
arrays are limited by "RTE_MAX_ETHPORTS".

> +
> +To run the application in a Linux environment with 3 lcores (one of them
> +is master lcore), 1 port (port 0), software copying and MAC updating issue
> +the command:

s/1 port/a single port/

s/one of them is master lcore/the master lcore, plus two forwarding cores/

Similar comments would apply to text immediately below too.

> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
> +
> +To run the application in a Linux environment with 2 lcores (one of them
> +is master lcore), 2 ports (ports 0 and 1), hardware copying and no MAC
> +updating issue the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw
> +
> +Refer to the *DPDK Getting Started Guide* for general information on
> +running applications and the Environment Abstraction Layer (EAL) options.
> +
> +Explanation
> +-----------
> +
> +The following sections provide an explanation of the main components of the
> +code.
> +
> +All DPDK library functions used in the sample code are prefixed with
> +``rte_`` and are explained in detail in the *DPDK API Documentation*.
> +
> +
> +The Main Function
> +~~~~~~~~~~~~~~~~~
> +
> +The ``main()`` function performs the initialization and calls the execution
> +threads for each lcore.
> +
> +The first task is to initialize the Environment Abstraction Layer (EAL).
> +The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
> +function. The value returned is the number of parsed arguments:
> +
> +.. code-block:: c
> +
> +    /* init EAL */
> +    ret = rte_eal_init(argc, argv);
> +    if (ret < 0)
> +        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
> +
> +
> +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
> +used by the application:
> +
> +.. code-block:: c
> +
> +    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
> +        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
> +        MIN_POOL_SIZE);
> +
> +    /* Create the mbuf pool */
> +    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
> +        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
> +        rte_socket_id());
> +    if (ioat_pktmbuf_pool == NULL)
> +        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
> +
> +Mbufs are the packet buffer structure used by DPDK. They are explained in
> +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
> +
> +The ``main()`` function also initializes the ports:
> +
> +.. code-block:: c
> +
> +    /* Initialise each port */
> +    RTE_ETH_FOREACH_DEV(portid) {
> +        port_init(portid, ioat_pktmbuf_pool);
> +    }
> +
> +Each port is configured using ``port_init()``:
> +
> +.. code-block:: c
> +
> +     /*
> +     * Initializes a given port using global settings and with the RX buffers
> +     * coming from the mbuf_pool passed as a parameter.
> +     */
> +    static inline void
> +    port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
> +    {
> +        /* configuring port to use RSS for multiple RX queues */
> +        static const struct rte_eth_conf port_conf = {
> +            .rxmode = {
> +                .mq_mode        = ETH_MQ_RX_RSS,
> +                .max_rx_pkt_len = RTE_ETHER_MAX_LEN
> +            },
> +            .rx_adv_conf = {
> +                .rss_conf = {
> +                    .rss_key = NULL,
> +                    .rss_hf = ETH_RSS_PROTO_MASK,
> +                }
> +            }
> +        };
> +
> +        struct rte_eth_rxconf rxq_conf;
> +        struct rte_eth_txconf txq_conf;
> +        struct rte_eth_conf local_port_conf = port_conf;
> +        struct rte_eth_dev_info dev_info;
> +        int ret, i;
> +
> +        /* Skip ports that are not enabled */
> +        if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
> +            printf("Skipping disabled port %u\n", portid);
> +            return;
> +        }
> +
> +        /* Init port */
> +        printf("Initializing port %u... ", portid);
> +        fflush(stdout);
> +        rte_eth_dev_info_get(portid, &dev_info);
> +        local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
> +            dev_info.flow_type_rss_offloads;
> +        if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
> +            local_port_conf.txmode.offloads |=
> +                DEV_TX_OFFLOAD_MBUF_FAST_FREE;
> +        ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE, "Cannot configure device:"
> +                " err=%d, port=%u\n", ret, portid);
> +
> +        ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
> +                            &nb_txd);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "Cannot adjust number of descriptors: err=%d, port=%u\n",
> +                ret, portid);
> +
> +        rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
> +
> +        /* Init Rx queues */
> +        rxq_conf = dev_info.default_rxconf;
> +        rxq_conf.offloads = local_port_conf.rxmode.offloads;
> +        for (i = 0; i < nb_queues; i++) {
> +            ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
> +                rte_eth_dev_socket_id(portid), &rxq_conf,
> +                mbuf_pool);
> +            if (ret < 0)
> +                rte_exit(EXIT_FAILURE,
> +                    "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
> +                    ret, portid, i);
> +        }
> +
> +        /* Init one TX queue on each port */
> +        txq_conf = dev_info.default_txconf;
> +        txq_conf.offloads = local_port_conf.txmode.offloads;
> +        ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
> +                rte_eth_dev_socket_id(portid),
> +                &txq_conf);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "rte_eth_tx_queue_setup:err=%d,port=%u\n",
> +                ret, portid);
> +
> +        /* Initialize TX buffers */
> +        tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
> +                RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
> +                rte_eth_dev_socket_id(portid));
> +        if (tx_buffer[portid] == NULL)
> +            rte_exit(EXIT_FAILURE,
> +                "Cannot allocate buffer for tx on port %u\n",
> +                portid);
> +
> +        rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
> +
> +        ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
> +                rte_eth_tx_buffer_count_callback,
> +                &port_statistics.tx_dropped[portid]);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "Cannot set error callback for tx buffer on port %u\n",
> +                portid);
> +
> +        /* Start device */
> +        ret = rte_eth_dev_start(portid);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "rte_eth_dev_start:err=%d, port=%u\n",
> +                ret, portid);
> +
> +        rte_eth_promiscuous_enable(portid);
> +
> +        printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
> +                portid,
> +                ioat_ports_eth_addr[portid].addr_bytes[0],
> +                ioat_ports_eth_addr[portid].addr_bytes[1],
> +                ioat_ports_eth_addr[portid].addr_bytes[2],
> +                ioat_ports_eth_addr[portid].addr_bytes[3],
> +                ioat_ports_eth_addr[portid].addr_bytes[4],
> +                ioat_ports_eth_addr[portid].addr_bytes[5]);
> +
> +        cfg.ports[cfg.nb_ports].rxtx_port = portid;
> +        cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
> +    }
> +

This code is probably quite similar to that in other sample apps, so I
don't think we need to include the full function here. It makes updating
the code more difficult, so just refer to the function as doing the port
init and leave it at that, I think. The snippets below give enough detail.

> +The Ethernet ports are configured with local settings using the
> +``rte_eth_dev_configure()`` function and the ``port_conf`` struct.
> +The RSS is enabled so that multiple Rx queues could be used for
> +packet receiving and copying by multiple CBDMA channels per port:
> +
> +.. code-block:: c
> +
> +    /* configuring port to use RSS for multiple RX queues */
> +    static const struct rte_eth_conf port_conf = {
> +        .rxmode = {
> +            .mq_mode        = ETH_MQ_RX_RSS,
> +            .max_rx_pkt_len = RTE_ETHER_MAX_LEN
> +        },
> +        .rx_adv_conf = {
> +            .rss_conf = {
> +                .rss_key = NULL,
> +                .rss_hf = ETH_RSS_PROTO_MASK,
> +            }
> +        }
> +    };
> +
> +For this example the ports are set up with the number of Rx queues provided
> +with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()``
> +and ``rte_eth_tx_queue_setup()`` functions.
> +
> +The Ethernet port is then started:
> +
> +.. code-block:: c
> +
> +    ret = rte_eth_dev_start(portid);
> +    if (ret < 0)
> +        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
> +            ret, portid);
> +
> +
> +Finally the Rx port is set in promiscuous mode:
> +
> +.. code-block:: c
> +
> +    rte_eth_promiscuous_enable(portid);
> +
> +
> +After that each port application assigns resources needed.
> +
> +.. code-block:: c
> +
> +    check_link_status(ioat_enabled_port_mask);
> +
> +    if (!cfg.nb_ports) {
> +        rte_exit(EXIT_FAILURE,
> +            "All available ports are disabled. Please set portmask.\n");
> +    }
> +
> +    /* Check if there is enough lcores for all ports. */
> +    cfg.nb_lcores = rte_lcore_count() - 1;
> +    if (cfg.nb_lcores < 1)
> +        rte_exit(EXIT_FAILURE,
> +            "There should be at least one slave lcore.\n");
> +
> +    ret = 0;
> +
> +    if (copy_mode == COPY_MODE_IOAT_NUM) {
> +        assign_rawdevs();
> +    } else /* copy_mode == COPY_MODE_SW_NUM */ {
> +        assign_rings();
> +    }
> +
> +A link status is checked of each port enabled by port mask
> +using ``check_link_status()`` function.
> +

I don't think this block needs to be covered. No need to go into everything
in detail, just focus on the key parts of the app that are unique to it,
i.e. the copying and passing mbufs between threads parts.

> +.. code-block:: c
> +
> +    /* check link status, return true if at least one port is up */
> +    static int
> +    check_link_status(uint32_t port_mask)
> +    {
> +        uint16_t portid;
> +        struct rte_eth_link link;
> +        int retval = 0;
> +
> +        printf("\nChecking link status\n");
> +        RTE_ETH_FOREACH_DEV(portid) {
> +            if ((port_mask & (1 << portid)) == 0)
> +                continue;
> +
> +            memset(&link, 0, sizeof(link));
> +            rte_eth_link_get(portid, &link);
> +
> +            /* Print link status */
> +            if (link.link_status) {
> +                printf(
> +                    "Port %d Link Up. Speed %u Mbps - %s\n",
> +                    portid, link.link_speed,
> +                    (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
> +                    ("full-duplex") : ("half-duplex\n"));
> +                retval = 1;
> +            } else
> +                printf("Port %d Link Down\n", portid);
> +        }
> +        return retval;
> +    }
> +


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
                             ` (2 preceding siblings ...)
  2019-09-27 11:01           ` Bruce Richardson
@ 2019-09-27 13:22           ` Bruce Richardson
  2019-09-27 15:13             ` Baran, MarcinX
  3 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27 13:22 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev, John McNamara, Marko Kovacevic

On Fri, Sep 20, 2019 at 09:37:14AM +0200, Marcin Baran wrote:
> Added guide for IOAT sample app usage and
> code description.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  doc/guides/sample_app_ug/index.rst |   1 +
>  doc/guides/sample_app_ug/intro.rst |   4 +
>  doc/guides/sample_app_ug/ioat.rst  | 764 +++++++++++++++++++++++++++++
>  3 files changed, 769 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> 

<snip>

> +Depending on mode set (whether copy should be done by software or by hardware)
> +special structures are assigned to each port. If software copy was chosen,
> +application have to assign ring structures for packet exchanging between lcores
> +assigned to ports.
> +
> +.. code-block:: c
> +
> +    static void
> +    assign_rings(void)
> +    {
> +        uint32_t i;
> +
> +        for (i = 0; i < cfg.nb_ports; i++) {
> +            char ring_name[20];
> +
> +            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
> +            /* Create ring for inter core communication */
> +            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
> +                    ring_name, ring_size,
> +                    rte_socket_id(), RING_F_SP_ENQ);
> +
> +            if (cfg.ports[i].rx_to_tx_ring == NULL)
> +                rte_exit(EXIT_FAILURE, "%s\n",
> +                        rte_strerror(rte_errno));
> +        }
> +    }
> +
> +
> +When using hardware copy each Rx queue of the port is assigned an
> +IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API
> +functions:
> +
> +.. code-block:: c
> +
> +    static void
> +    assign_rawdevs(void)
> +    {
> +        uint16_t nb_rawdev = 0, rdev_id = 0;
> +        uint32_t i, j;
> +
> +        for (i = 0; i < cfg.nb_ports; i++) {
> +            for (j = 0; j < cfg.ports[i].nb_queues; j++) {
> +                struct rte_rawdev_info rdev_info = { 0 };
> +
> +                do {
> +                    if (rdev_id == rte_rawdev_count())
> +                        goto end;
> +                    rte_rawdev_info_get(rdev_id++, &rdev_info);
> +                } while (strcmp(rdev_info.driver_name,
> +                    IOAT_PMD_RAWDEV_NAME_STR) != 0);
> +
> +                cfg.ports[i].ioat_ids[j] = rdev_id - 1;
> +                configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
> +                ++nb_rawdev;
> +            }
> +        }
> +    end:
> +        if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
> +            rte_exit(EXIT_FAILURE,
> +                "Not enough IOAT rawdevs (%u) for all queues (%u).\n",
> +                nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
> +        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
> +    }
> +
> +
> +The initialization of hardware device is done by ``rte_rawdev_configure()``
> +function and ``rte_rawdev_info`` struct.

... using ``rte_rawdev_info`` struct

> After configuration the device is
> +started using ``rte_rawdev_start()`` function. Each of the above operations
> +is done in ``configure_rawdev_queue()``.

In the block below, there is no mention of where dev_config structure comes
from. Presume it's a global variable, so maybe mention that in the text.

> +
> +.. code-block:: c
> +
> +    static void
> +    configure_rawdev_queue(uint32_t dev_id)
> +    {
> +        struct rte_rawdev_info info = { .dev_private = &dev_config };
> +
> +        /* Configure hardware copy device */
> +        dev_config.ring_size = ring_size;
> +
> +        if (rte_rawdev_configure(dev_id, &info) != 0) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error with rte_rawdev_configure()\n");
> +        }
> +        rte_rawdev_info_get(dev_id, &info);
> +        if (dev_config.ring_size != ring_size) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error, ring size is not %d (%d)\n",
> +                ring_size, (int)dev_config.ring_size);
> +        }
> +        if (rte_rawdev_start(dev_id) != 0) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error with rte_rawdev_start()\n");
> +        }
> +    }
> +
> +If initialization is successful memory for hardware device
> +statistics is allocated.

Missing "," after successful.
Where is this memory allocated? It is done in main or elsewhere?
> +
> +Finally ``main()`` functions starts all processing lcores and starts

s/functions/function/
s/processing lcores/packet handling lcores/

> +printing stats in a loop on master lcore. The application can be

s/master lcore/the master lcore/

> +interrupted and closed using ``Ctrl-C``. The master lcore waits for
> +all slave processes to finish, deallocates resources and exits.
> +
> +The processing lcores launching function are described below.
> +
> +The Lcores Launching Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +As described above ``main()`` function invokes ``start_forwarding_cores()``

Missing "," after above.

> +function in order to start processing for each lcore:
> +
> +.. code-block:: c
> +
> +    static void start_forwarding_cores(void)
> +    {
> +        uint32_t lcore_id = rte_lcore_id();
> +
> +        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
> +                __func__, rte_lcore_id());
> +
> +        if (cfg.nb_lcores == 1) {
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
> +                NULL, lcore_id);
> +        } else if (cfg.nb_lcores > 1) {
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
> +                NULL, lcore_id);
> +
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
> +                lcore_id);
> +        }
> +    }
> +
> +The function launches Rx/Tx processing functions on configured lcores
> +for each port using ``rte_eal_remote_launch()``. The configured ports,

Remove "for each port"

> +their number and number of assigned lcores are stored in user-defined
> +``rxtx_transmission_config`` struct that is initialized before launching

s/is/has been/
Did you describe how that structure was set up previously?

> +tasks:
> +
> +.. code-block:: c
> +
> +    struct rxtx_transmission_config {
> +        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
> +        uint16_t nb_ports;
> +        uint16_t nb_lcores;
> +    };
> +
> +The Lcores Processing Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +For receiving packets on each port an ``ioat_rx_port()`` function is used.

Missing "," after port.
s/an/the/

> +The function receives packets on each configured Rx queue. Depending on mode

s/mode/the mode/

> +the user chose, it will enqueue packets to IOAT rawdev channels and then invoke
> +copy process (hardware copy), or perform software copy of each packet using
> +``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring:

s/1 rte_ring/an rte_ring/

> +
> +.. code-block:: c
> +
> +    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
> +    static void
> +    ioat_rx_port(struct rxtx_port_config *rx_config)
> +    {
> +        uint32_t nb_rx, nb_enq, i, j;
> +        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
> +        for (i = 0; i < rx_config->nb_queues; i++) {
> +
> +            nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
> +                pkts_burst, MAX_PKT_BURST);
> +
> +            if (nb_rx == 0)
> +                continue;
> +
> +            port_statistics.rx[rx_config->rxtx_port] += nb_rx;
> +
> +            if (copy_mode == COPY_MODE_IOAT_NUM) {
> +                /* Perform packet hardware copy */
> +                nb_enq = ioat_enqueue_packets(pkts_burst,
> +                    nb_rx, rx_config->ioat_ids[i]);
> +                if (nb_enq > 0)
> +                    rte_ioat_do_copies(rx_config->ioat_ids[i]);
> +            } else {
> +                /* Perform packet software copy, free source packets */
> +                int ret;
> +                struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> +
> +                ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +                    (void *)pkts_burst_copy, nb_rx);
> +
> +                if (unlikely(ret < 0))
> +                    rte_exit(EXIT_FAILURE,
> +                        "Unable to allocate memory.\n");
> +
> +                for (j = 0; j < nb_rx; j++)
> +                    pktmbuf_sw_copy(pkts_burst[j],
> +                        pkts_burst_copy[j]);
> +
> +                rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)pkts_burst, nb_rx);
> +
> +                nb_enq = rte_ring_enqueue_burst(
> +                    rx_config->rx_to_tx_ring,
> +                    (void *)pkts_burst_copy, nb_rx, NULL);
> +
> +                /* Free any not enqueued packets. */
> +                rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)&pkts_burst_copy[nb_enq],
> +                    nb_rx - nb_enq);
> +            }
> +
> +            port_statistics.copy_dropped[rx_config->rxtx_port] +=
> +                (nb_rx - nb_enq);
> +        }
> +    }
> +
> +The packets are received in burst mode using ``rte_eth_rx_burst()``
> +function. When using hardware copy mode the packets are enqueued in
> +copying device's buffer using ``ioat_enqueue_packets()`` which calls
> +``rte_ioat_enqueue_copy()``. When all received packets are in the
> +buffer the copies are invoked by calling ``rte_ioat_do_copies()``.

s/copies are invoked/copy operations are started/

> +Function ``rte_ioat_enqueue_copy()`` operates on physical address of
> +the packet. Structure ``rte_mbuf`` contains only physical address to
> +start of the data buffer (``buf_iova``). Thus the address is shifted

s/shifted/adjusted/

> +by ``addr_offset`` value in order to get pointer to ``rearm_data``

s/pointer to/the address of/

> +member of ``rte_mbuf``. That way the packet is copied all at once
> +(with data and metadata).

"That way the both the packet data and metadata can be copied in a single
operation".
Should also note that this shortcut can be used because the mbufs are
"direct" mbufs allocated by the apps. If another app uses external buffers,
or indirect mbufs, then multiple copy operations must be used.

> +
> +.. code-block:: c
> +
> +    static uint32_t
> +    ioat_enqueue_packets(struct rte_mbuf **pkts,
> +        uint32_t nb_rx, uint16_t dev_id)
> +    {
> +        int ret;
> +        uint32_t i;
> +        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
> +
> +        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
> +            &pkts[0]->rearm_data);
> +
> +        ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +                (void *)pkts_copy, nb_rx);
> +
> +        if (unlikely(ret < 0))
> +            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
> +
> +        for (i = 0; i < nb_rx; i++) {
> +            /* Perform data copy */
> +            ret = rte_ioat_enqueue_copy(dev_id,
> +                pkts[i]->buf_iova
> +                    - addr_offset,
> +                pkts_copy[i]->buf_iova
> +                    - addr_offset,
> +                rte_pktmbuf_data_len(pkts[i])
> +                    + addr_offset,
> +                (uintptr_t)pkts[i],
> +                (uintptr_t)pkts_copy[i],
> +                0 /* nofence */);
> +
> +            if (ret != 1)
> +                break;
> +        }
> +
> +        ret = i;
> +        /* Free any not enqueued packets. */
> +        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
> +        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
> +            nb_rx - i);
> +
> +        return ret;
> +    }
> +
> +
> +All done copies are processed by ``ioat_tx_port()`` function. When using

s/done/completed/

> +hardware copy mode the function invokes ``rte_ioat_completed_copies()``
> +on each assigned IOAT channel to gather copied packets. If software copy
> +mode is used the function dequeues copied packets from the rte_ring. Then each
> +packet MAC address is changed if it was enabled. After that copies are sent
> +in burst mode using `` rte_eth_tx_burst()``.
> +
> +
> +.. code-block:: c
> +
> +    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
> +    static void
> +    ioat_tx_port(struct rxtx_port_config *tx_config)
> +    {
> +        uint32_t i, j, nb_dq = 0;
> +        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
> +        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
> +
> +        if (copy_mode == COPY_MODE_IOAT_NUM) {
> +            for (i = 0; i < tx_config->nb_queues; i++) {
> +                /* Deque the mbufs from IOAT device. */
> +                nb_dq = rte_ioat_completed_copies(
> +                    tx_config->ioat_ids[i], MAX_PKT_BURST,
> +                    (void *)mbufs_src, (void *)mbufs_dst);
> +
> +                if (nb_dq == 0)
> +                    break;
> +
> +                rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)mbufs_src, nb_dq);
> +
> +                /* Update macs if enabled */
> +                if (mac_updating) {
> +                    for (j = 0; j < nb_dq; j++)
> +                        update_mac_addrs(mbufs_dst[j],
> +                            tx_config->rxtx_port);
> +                }
> +
> +                const uint16_t nb_tx = rte_eth_tx_burst(
> +                    tx_config->rxtx_port, 0,
> +                    (void *)mbufs_dst, nb_dq);
> +
> +                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
> +
> +                /* Free any unsent packets. */
> +                if (unlikely(nb_tx < nb_dq))
> +                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)&mbufs_dst[nb_tx],
> +                        nb_dq - nb_tx);
> +            }
> +        }
> +        else {
> +            for (i = 0; i < tx_config->nb_queues; i++) {
> +                /* Deque the mbufs from IOAT device. */
> +                nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
> +                    (void *)mbufs_dst, MAX_PKT_BURST, NULL);
> +
> +                if (nb_dq == 0)
> +                    return;
> +
> +                /* Update macs if enabled */
> +                if (mac_updating) {
> +                    for (j = 0; j < nb_dq; j++)
> +                        update_mac_addrs(mbufs_dst[j],
> +                            tx_config->rxtx_port);
> +                }
> +
> +                const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
> +                    0, (void *)mbufs_dst, nb_dq);
> +
> +                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
> +
> +                /* Free any unsent packets. */
> +                if (unlikely(nb_tx < nb_dq))
> +                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)&mbufs_dst[nb_tx],
> +                        nb_dq - nb_tx);
> +            }
> +        }
> +    }
> +
> +The Packet Copying Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In order to perform packet copy there is a user-defined function
> +``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
> +metadata from source packet to new mbuf, and then copying a data
> +chunk of source packet. Both memory copies are done using
> +``rte_memcpy()``:
> +
> +.. code-block:: c
> +
> +    static inline void
> +    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
> +    {
> +        /* Copy packet metadata */
> +        rte_memcpy(&dst->rearm_data,
> +            &src->rearm_data,
> +            offsetof(struct rte_mbuf, cacheline1)
> +                - offsetof(struct rte_mbuf, rearm_data));
> +
> +        /* Copy packet data */
> +        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
> +            rte_pktmbuf_mtod(src, char *), src->data_len);
> +    }
> +
> +The metadata in this example is copied from ``rearm_data`` member of
> +``rte_mbuf`` struct up to ``cacheline1``.
> +
> +In order to understand why software packet copying is done as shown
> +above please refer to the "Mbuf Library" section of the
> +*DPDK Programmer's Guide*.
> \ No newline at end of file

Use a text editor that adds a newline automatically :-)

/Bruce

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support
  2019-09-27 10:01           ` Bruce Richardson
@ 2019-09-27 14:01             ` Baran, MarcinX
  0 siblings, 0 replies; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-27 14:01 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev, Modrak, PawelX

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Friday, September 27, 2019 12:01 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org; Modrak, PawelX <pawelx.modrak@intel.com>
Subject: Re: [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support

On Fri, Sep 20, 2019 at 09:37:10AM +0200, Marcin Baran wrote:
> Added support for copying packets
> using software copy mode and MAC address changing. The copies are 
> processed using one lcore.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> ---
>  examples/ioat/ioatfwd.c | 211 
> +++++++++++++++++++++++++++++++++++-----
>  1 file changed, 188 insertions(+), 23 deletions(-)
> 
> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c index 
> 977ea6a61..3a092c6cf 100644
> --- a/examples/ioat/ioatfwd.c
> +++ b/examples/ioat/ioatfwd.c
> @@ -13,7 +13,7 @@
>  #include <rte_rawdev.h>
>  #include <rte_ioat_rawdev.h>
>  
> -/* size of ring used for software copying between rx and tx. */
> + /* size of ring used for software copying between rx and tx. */
>  #define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1  #define MAX_PKT_BURST 32  
> #define MEMPOOL_CACHE_SIZE 512 @@ -89,6 +89,142 @@ static struct 
> rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
>  static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];  
> struct rte_mempool *ioat_pktmbuf_pool;
>  
> +static void
> +update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid) {
> +	struct rte_ether_hdr *eth;
> +	void *tmp;
> +
> +	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
> +
> +	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
> +	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
> +	 */
> +	tmp = &eth->d_addr.addr_bytes[0];
> +	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
> +
> +	/* src addr */
> +	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], 
> +&eth->s_addr); }
> +
> +static inline void
> +pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) {
> +	/* Copy packet metadata */
> +	rte_memcpy(&dst->rearm_data,
> +		&src->rearm_data,
> +		offsetof(struct rte_mbuf, cacheline1)
> +		- offsetof(struct rte_mbuf, rearm_data));
> +
> +	/* Copy packet data */
> +	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
> +		rte_pktmbuf_mtod(src, char *), src->data_len); }
> +
> +/* Receive packets on one port and enqueue to IOAT rawdev or 
> +rte_ring. */ static void ioat_rx_port(struct rxtx_port_config 
> +*rx_config) {
> +	uint32_t nb_rx, nb_enq, i, j;
> +	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
> +
> +	for (i = 0; i < rx_config->nb_queues; i++) {
> +
> +		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
> +			pkts_burst, MAX_PKT_BURST);
> +
> +		if (nb_rx == 0)
> +			continue;
> +
> +		/* Perform packet software copy, free source packets */
> +		int ret;
> +		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> +
> +		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +			(void *)pkts_burst_copy, nb_rx);
> +
> +		if (unlikely(ret < 0))
> +			rte_exit(EXIT_FAILURE,
> +				"Unable to allocate memory.\n");
> +
> +		for (j = 0; j < nb_rx; j++)
> +			pktmbuf_sw_copy(pkts_burst[j],
> +				pkts_burst_copy[j]);
> +
> +		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +			(void *)pkts_burst, nb_rx);
> +
> +		nb_enq = rte_ring_enqueue_burst(
> +			rx_config->rx_to_tx_ring,
> +			(void *)pkts_burst_copy, nb_rx, NULL);
> +
> +		/* Free any not enqueued packets. */
> +		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +			(void *)&pkts_burst_copy[nb_enq],
> +			nb_rx - nb_enq);
> +	}
> +}
> +
> +/* Transmit packets from IOAT rawdev/rte_ring for one port. */ static 
> +void ioat_tx_port(struct rxtx_port_config *tx_config) {
> +	uint32_t i, nb_dq = 0;
> +	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
> +
> +	/* Deque the mbufs from rx_to_tx_ring. */
> +	nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
> +		(void *)mbufs_dst, MAX_PKT_BURST, NULL);
> +
> +	if (nb_dq == 0)
> +		return;
> +
> +	/* Update macs if enabled */
> +	if (mac_updating) {
> +		for (i = 0; i < nb_dq; i++)
> +			update_mac_addrs(mbufs_dst[i],
> +				tx_config->rxtx_port);
> +	}
> +
> +	const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
> +		0, (void *)mbufs_dst, nb_dq);
> +
> +	/* Free any unsent packets. */
> +	if (unlikely(nb_tx < nb_dq))
> +		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +		(void *)&mbufs_dst[nb_tx],
> +			nb_dq - nb_tx);
> +}
> +
> +/* Main rx and tx loop if only one slave lcore available */ static 
> +void
> +rxtx_main_loop(void)
> +{
> +	uint16_t i;
> +	uint16_t nb_ports = cfg.nb_ports;
> +
> +	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
> +		" lcore %u\n", rte_lcore_id());
> +
> +	while (!force_quit)
> +		for (i = 0; i < nb_ports; i++) {
> +			ioat_rx_port(&cfg.ports[i]);
> +			ioat_tx_port(&cfg.ports[i]);
> +		}
> +}
> +
> +static void start_forwarding_cores(void) {
> +	uint32_t lcore_id = rte_lcore_id();
> +
> +	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
> +		__func__, rte_lcore_id());
> +
> +	lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
> +		NULL, lcore_id);
> +}
> +
>  /* Display usage */
>  static void
>  ioat_usage(const char *prgname)
> @@ -102,7 +238,7 @@ ioat_usage(const char *prgname)
>  		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
>  		"  -c --copy-type CT: type of copy: sw|rawdev\n"
>  		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
> -	       prgname);
> +		prgname);
>  }
>  
>  static int
> @@ -161,16 +297,16 @@ ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
>  	argvopt = argv;
>  
>  	while ((opt = getopt_long(argc, argvopt, short_options,
> -				  lgopts, &option_index)) != EOF) {
> +		lgopts, &option_index)) != EOF) {

These and the other whitespace changes in this patch should be fixed in patch 1 rather than having them as modifications here.

Without these whitespace changes merged into original patch:

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

[Marcin]: Whitespace changes moved to previous commit for v6.


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support
  2019-09-27 10:05           ` Bruce Richardson
@ 2019-09-27 14:03             ` Baran, MarcinX
  0 siblings, 0 replies; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-27 14:03 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev, Modrak, PawelX

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Friday, September 27, 2019 12:06 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org; Modrak, PawelX <pawelx.modrak@intel.com>
Subject: Re: [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support

On Fri, Sep 20, 2019 at 09:37:11AM +0200, Marcin Baran wrote:
> Added support for copying packets using rawdev device. Each port's Rx 
> queue is assigned DMA channel for copy.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> ---
>  examples/ioat/ioatfwd.c | 236 
> ++++++++++++++++++++++++++++++++--------
>  1 file changed, 189 insertions(+), 47 deletions(-)
> 
> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c index 
> 3a092c6cf..c66ce7e49 100644
> --- a/examples/ioat/ioatfwd.c
> +++ b/examples/ioat/ioatfwd.c
> @@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
>  		rte_pktmbuf_mtod(src, char *), src->data_len);  }
>  
> +static uint32_t
> +ioat_enqueue_packets(struct rte_mbuf **pkts,
> +	uint32_t nb_rx, uint16_t dev_id)
> +{
> +	int ret;
> +	uint32_t i;
> +	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
> +
> +	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
> +		&pkts[0]->rearm_data);
> +
> +	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +		(void *)pkts_copy, nb_rx);
> +
> +	if (unlikely(ret < 0))
> +		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
> +
> +	for (i = 0; i < nb_rx; i++) {
> +		/* Perform data copy */
> +		ret = rte_ioat_enqueue_copy(dev_id,
> +			pkts[i]->buf_iova
> +			- addr_offset,
> +			pkts_copy[i]->buf_iova
> +			- addr_offset,
> +			rte_pktmbuf_data_len(pkts[i])
> +			+ addr_offset,
> +			(uintptr_t)pkts[i],
> +			(uintptr_t)pkts_copy[i],
> +			0 /* nofence */);
> +
> +		if (ret != 1)
> +			break;
> +	}
> +
> +	ret = i;
> +	/* Free any not enqueued packets. */
> +	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
> +	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
> +		nb_rx - i);
> +
> +
> +	return ret;
> +}
> +
>  /* Receive packets on one port and enqueue to IOAT rawdev or 
> rte_ring. */  static void  ioat_rx_port(struct rxtx_port_config 
> *rx_config) @@ -136,32 +180,40 @@ ioat_rx_port(struct rxtx_port_config 
> *rx_config)
>  		if (nb_rx == 0)
>  			continue;
>  
> -		/* Perform packet software copy, free source packets */
> -		int ret;
> -		struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> -
> -		ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> -			(void *)pkts_burst_copy, nb_rx);
> -
> -		if (unlikely(ret < 0))
> -			rte_exit(EXIT_FAILURE,
> -				"Unable to allocate memory.\n");
> -
> -		for (j = 0; j < nb_rx; j++)
> -			pktmbuf_sw_copy(pkts_burst[j],
> -				pkts_burst_copy[j]);
> -
> -		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> -			(void *)pkts_burst, nb_rx);
> -
> -		nb_enq = rte_ring_enqueue_burst(
> -			rx_config->rx_to_tx_ring,
> -			(void *)pkts_burst_copy, nb_rx, NULL);
> -
> -		/* Free any not enqueued packets. */
> -		rte_mempool_put_bulk(ioat_pktmbuf_pool,
> -			(void *)&pkts_burst_copy[nb_enq],
> -			nb_rx - nb_enq);
> +		if (copy_mode == COPY_MODE_IOAT_NUM) {
> +			/* Perform packet hardware copy */
> +			nb_enq = ioat_enqueue_packets(pkts_burst,
> +				nb_rx, rx_config->ioat_ids[i]);
> +			if (nb_enq > 0)
> +				rte_ioat_do_copies(rx_config->ioat_ids[i]);
> +		} else {
> +			/* Perform packet software copy, free source packets */
> +			int ret;
> +			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> +
> +			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +				(void *)pkts_burst_copy, nb_rx);
> +
> +			if (unlikely(ret < 0))
> +				rte_exit(EXIT_FAILURE,
> +					"Unable to allocate memory.\n");
> +
> +			for (j = 0; j < nb_rx; j++)
> +				pktmbuf_sw_copy(pkts_burst[j],
> +					pkts_burst_copy[j]);
> +
> +			rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +				(void *)pkts_burst, nb_rx);
> +
> +			nb_enq = rte_ring_enqueue_burst(
> +				rx_config->rx_to_tx_ring,
> +				(void *)pkts_burst_copy, nb_rx, NULL);
> +
> +			/* Free any not enqueued packets. */
> +			rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +				(void *)&pkts_burst_copy[nb_enq],
> +				nb_rx - nb_enq);
> +		}

Would the diff in this patch be smaller if you switched the order of the branches so that the SW copy leg, which was added first, was processed first? You could even add in a dummy branch in patch 2, so that the indentation for that section remains unchanged.

/Bruce
[Marcin] Switched the order and added dummy branch in patch 2. Also changed ioat_tx_port() function the same way in v6.

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration
  2019-09-27 10:08           ` Bruce Richardson
@ 2019-09-27 14:03             ` Baran, MarcinX
  0 siblings, 0 replies; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-27 14:03 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev, Modrak, PawelX

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Friday, September 27, 2019 12:09 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org; Modrak, PawelX <pawelx.modrak@intel.com>
Subject: Re: [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration

On Fri, Sep 20, 2019 at 09:37:12AM +0200, Marcin Baran wrote:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> 
> Added possibility to use two lcores: first for packet receiving and 
> copying, second for packets sending.
> 
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  examples/ioat/ioatfwd.c | 92 
> ++++++++++++++++++++++++++++++-----------
>  1 file changed, 67 insertions(+), 25 deletions(-)
> 
> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c index 
> c66ce7e49..4c51db6bd 100644
> --- a/examples/ioat/ioatfwd.c
> +++ b/examples/ioat/ioatfwd.c
> @@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,
>  	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
>  		nb_rx - i);
>  
> -
>  	return ret;
>  }
>  
> @@ -226,8 +225,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
>  	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
>  
>  	if (copy_mode == COPY_MODE_IOAT_NUM) {
> -		/* Deque the mbufs from IOAT device. */
>  		for (i = 0; i < tx_config->nb_queues; i++) {
> +			/* Deque the mbufs from IOAT device. */
>  			nb_dq = rte_ioat_completed_copies(
>  				tx_config->ioat_ids[i], MAX_PKT_BURST,
>  				(void *)mbufs_src, (void *)mbufs_dst); @@ -256,31 +255,64 @@ 
> ioat_tx_port(struct rxtx_port_config *tx_config)
>  					nb_dq - nb_tx);
>  		}
>  	} else {
> -		/* Deque the mbufs from rx_to_tx_ring. */
> -		nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
> -			(void *)mbufs_dst, MAX_PKT_BURST, NULL);
> -
> -		if (nb_dq == 0)
> -			return;
> -
> -		/* Update macs if enabled */
> -		if (mac_updating) {
> -			for (i = 0; i < nb_dq; i++)
> -				update_mac_addrs(mbufs_dst[i],
> -					tx_config->rxtx_port);
> -		}
> +		for (i = 0; i < tx_config->nb_queues; i++) {

This commit refers to adding support for 2 threads rather than 1, but this change seems related to having multiple queues rather than threads. Shoudl it be in a previous patch?

/Bruce

[Marcin] Moved the change to 2nd patch for v6.


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port
  2019-09-27 10:12           ` Bruce Richardson
@ 2019-09-27 14:04             ` Baran, MarcinX
  0 siblings, 0 replies; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-27 14:04 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev, Modrak, PawelX



-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Friday, September 27, 2019 12:13 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org; Modrak, PawelX <pawelx.modrak@intel.com>
Subject: Re: [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port

On Fri, Sep 20, 2019 at 09:37:13AM +0200, Marcin Baran wrote:
> Added printing stats of ports each second.
> The stats printing is done using master core.
> The information provided informs about packets received, dropped and 
> send as well as statistics of rawdev devices.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> ---
>  examples/ioat/ioatfwd.c | 245 
> +++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 241 insertions(+), 4 deletions(-)
> 
> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c index 
> 4c51db6bd..c6b994832 100644
> --- a/examples/ioat/ioatfwd.c
> +++ b/examples/ioat/ioatfwd.c
> @@ -48,10 +48,27 @@ struct rxtx_transmission_config {
>  	uint16_t nb_lcores;
>  };
>  

<snip stats code>

> @@ -363,7 +587,7 @@ ioat_usage(const char *prgname)
>  		"      When enabled:\n"
>  		"       - The source MAC address is replaced by the TX port MAC address\n"
>  		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
> -		"  -c --copy-type CT: type of copy: sw|rawdev\n"
> +		"  -c --copy-type CT: type of copy: sw|hw\n"
>  		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
>  		prgname);
>  }
This change is not relevant to this patch, and should be corrected in the original patch of the series. Try to avoid adding a line and then modifying it in a subsequent patch in the same set.

/Bruce

[Marcin] Moved change to first patch.

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-27 10:36           ` Bruce Richardson
@ 2019-09-27 14:14             ` Baran, MarcinX
  0 siblings, 0 replies; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-27 14:14 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev



-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Friday, September 27, 2019 12:36 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide

On Fri, Sep 20, 2019 at 09:37:14AM +0200, Marcin Baran wrote:
> Added guide for IOAT sample app usage and code description.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  doc/guides/sample_app_ug/index.rst |   1 +
>  doc/guides/sample_app_ug/intro.rst |   4 +
>  doc/guides/sample_app_ug/ioat.rst  | 764 
> +++++++++++++++++++++++++++++
>  3 files changed, 769 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> 
> diff --git a/doc/guides/sample_app_ug/index.rst 
> b/doc/guides/sample_app_ug/index.rst
> index f23f8f59e..a6a1d9e7a 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -23,6 +23,7 @@ Sample Applications User Guides
>      ip_reassembly
>      kernel_nic_interface
>      keep_alive
> +    ioat
>      l2_forward_crypto
>      l2_forward_job_stats
>      l2_forward_real_virtual
> diff --git a/doc/guides/sample_app_ug/intro.rst 
> b/doc/guides/sample_app_ug/intro.rst
> index 90704194a..74462312f 100644
> --- a/doc/guides/sample_app_ug/intro.rst
> +++ b/doc/guides/sample_app_ug/intro.rst
> @@ -91,6 +91,10 @@ examples are highlighted below.
>    forwarding, or ``l3fwd`` application does forwarding based on Internet
>    Protocol, IPv4 or IPv6 like a simple router.
>  
> +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
> +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev 
> +driver for
> +  copying packets between two threads.
> +
>  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
>    demonstrates how to distribute packets arriving on an Rx port to different
>    cores for processing and transmission.
> diff --git a/doc/guides/sample_app_ug/ioat.rst 
> b/doc/guides/sample_app_ug/ioat.rst
> new file mode 100644
> index 000000000..69621673b
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/ioat.rst
> @@ -0,0 +1,764 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Sample Application of packet copying using Intel\ |reg| QuickData 
> +Technology 
> +=====================================================================
> +=======
> +

Title is too long, you can drop the "Sample Application" at minimum, since this is part of the example applications guide document. Call the section "Packet Copying Using ..."

In order to get the proper (R) symbol, you also need to add an include to the file. Therefore add the following line just after the copyright:

.. include:: <isonum.txt>

/Bruce
[Marcin] Dropped "Sample Application" and added include.

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-27 11:01           ` Bruce Richardson
@ 2019-09-27 14:51             ` Baran, MarcinX
  2019-09-27 15:00               ` Bruce Richardson
  0 siblings, 1 reply; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-27 14:51 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Friday, September 27, 2019 1:02 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide

On Fri, Sep 20, 2019 at 09:37:14AM +0200, Marcin Baran wrote:
> Added guide for IOAT sample app usage and code description.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  doc/guides/sample_app_ug/index.rst |   1 +
>  doc/guides/sample_app_ug/intro.rst |   4 +
>  doc/guides/sample_app_ug/ioat.rst  | 764 
> +++++++++++++++++++++++++++++
>  3 files changed, 769 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> 
> diff --git a/doc/guides/sample_app_ug/index.rst 
> b/doc/guides/sample_app_ug/index.rst
> index f23f8f59e..a6a1d9e7a 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -23,6 +23,7 @@ Sample Applications User Guides
>      ip_reassembly
>      kernel_nic_interface
>      keep_alive
> +    ioat
>      l2_forward_crypto
>      l2_forward_job_stats
>      l2_forward_real_virtual
> diff --git a/doc/guides/sample_app_ug/intro.rst 
> b/doc/guides/sample_app_ug/intro.rst
> index 90704194a..74462312f 100644
> --- a/doc/guides/sample_app_ug/intro.rst
> +++ b/doc/guides/sample_app_ug/intro.rst
> @@ -91,6 +91,10 @@ examples are highlighted below.
>    forwarding, or ``l3fwd`` application does forwarding based on Internet
>    Protocol, IPv4 or IPv6 like a simple router.
>  
> +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
> +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev 
> +driver for
> +  copying packets between two threads.
> +
>  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
>    demonstrates how to distribute packets arriving on an Rx port to different
>    cores for processing and transmission.
> diff --git a/doc/guides/sample_app_ug/ioat.rst 
> b/doc/guides/sample_app_ug/ioat.rst
> new file mode 100644
> index 000000000..69621673b
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/ioat.rst
> @@ -0,0 +1,764 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Sample Application of packet copying using Intel\ |reg| QuickData 
> +Technology 
> +=====================================================================
> +=======
> +
> +Overview
> +--------
> +
> +This sample is intended as a demonstration of the basic components of 
> +a DPDK forwarding application and example of how to use IOAT driver 
> +API to make packets copies.
> +
> +Also while forwarding, the MAC addresses are affected as follows:
> +
> +*   The source MAC address is replaced by the TX port MAC address
> +
> +*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
> +
> +This application can be used to compare performance of using software 
> +packet copy with copy done using a DMA device for different sizes of packets.
> +The example will print out statistics each second. The stats shows 
> +received/send packets and packets dropped or failed to copy.
> +
> +Compiling the Application
> +-------------------------
> +
> +To compile the sample application see :doc:`compiling`.
> +
> +The application is located in the ``ioat`` sub-directory.
> +
> +
> +Running the Application
> +-----------------------
> +
> +In order to run the hardware copy application, the copying device 
> +needs to be bound to user-space IO driver.
> +
> +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData 
> +Technology* guide for information on using the driver.
> +

The document is not called that, as the IOAT guide is just part of the overall rawdev document. So I suggest you just reference the rawdev guide.
[Marcin] I wanted to refer to the ioat guide in /doc/guides/rawdevs/ioat.rst which has that title. Is there another document or I referenced this one
incorrectly?

> +The application requires a number of command line options:
> +
> +.. code-block:: console
> +
> +    ./build/ioatfwd [EAL options] -- -p MASK [-q NQ] [-s RS] [-c <sw|hw>]
> +        [--[no-]mac-updating]
> +
> +where,
> +
> +*   p MASK: A hexadecimal bitmask of the ports to configure

Is this a mandatory parameter, or does the app use all detected ports by default, e.g. like testpmd?
[Marcin] Optional, the app use all detected ports, added default value comment and tagged -p as
optional.

> +
> +*   q NQ: Number of Rx queues used per port equivalent to CBDMA channels
> +    per port
> +
> +*   c CT: Performed packet copy type: software (sw) or hardware using
> +    DMA (hw)

What is the default? Same for next two parameters.
[Marcin] Added default values description.

> +
> +*   s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for
> +    software copy mode
> +
> +*   --[no-]mac-updating: Whether MAC address of packets should be changed
> +    or not
> +
> +The application can be launched in various configurations depending 
> +on provided parameters. Each port can use up to 2 lcores: one of them 
> +receives

The app uses 2 data plane cores, total, rather than 2 per-port, I believe.
It would be good to explain the difference here that with 2 cores the copies are done on one core, and the mac updates on the second one.
[Marcin] Changed the description accordingly.

> +incoming traffic and makes a copy of each packet. The second lcore 
> +then updates MAC address and sends the copy. If one lcore per port is 
> +used, both operations are done sequentially. For each configuration 
> +an additional lcore is needed since master lcore in use which is 
> +responsible for

... since the master lcore does not handle traffic but is responsible for
[Marcin] Changed the description accordingly.

> +configuration, statistics printing and safe deinitialization of all 
> +ports and devices.

s/deinitialization/shutdown/
[Marcin] Changed the description accordingly.

> +
> +The application can use a maximum of 8 ports.

Is this a hard limit in the app, if so explain why. I see the stats arrays are limited by "RTE_MAX_ETHPORTS".
[Marcin] The limit was set for simplicity but also because on testing board there was 16 CBDMA channels total
so when there are 8 ports used, they can be set to work with more than one Rx queue each.

> +
> +To run the application in a Linux environment with 3 lcores (one of 
> +them is master lcore), 1 port (port 0), software copying and MAC 
> +updating issue the command:

s/1 port/a single port/

s/one of them is master lcore/the master lcore, plus two forwarding cores/

Similar comments would apply to text immediately below too.
[Marcin] Changed the description accordingly.

> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
> +
> +To run the application in a Linux environment with 2 lcores (one of 
> +them is master lcore), 2 ports (ports 0 and 1), hardware copying and 
> +no MAC updating issue the command:
> +
> +.. code-block:: console
> +
> +    $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw
> +
> +Refer to the *DPDK Getting Started Guide* for general information on 
> +running applications and the Environment Abstraction Layer (EAL) options.
> +
> +Explanation
> +-----------
> +
> +The following sections provide an explanation of the main components 
> +of the code.
> +
> +All DPDK library functions used in the sample code are prefixed with 
> +``rte_`` and are explained in detail in the *DPDK API Documentation*.
> +
> +
> +The Main Function
> +~~~~~~~~~~~~~~~~~
> +
> +The ``main()`` function performs the initialization and calls the 
> +execution threads for each lcore.
> +
> +The first task is to initialize the Environment Abstraction Layer (EAL).
> +The ``argc`` and ``argv`` arguments are provided to the 
> +``rte_eal_init()`` function. The value returned is the number of parsed arguments:
> +
> +.. code-block:: c
> +
> +    /* init EAL */
> +    ret = rte_eal_init(argc, argv);
> +    if (ret < 0)
> +        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
> +
> +
> +The ``main()`` also allocates a mempool to hold the mbufs (Message 
> +Buffers) used by the application:
> +
> +.. code-block:: c
> +
> +    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
> +        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
> +        MIN_POOL_SIZE);
> +
> +    /* Create the mbuf pool */
> +    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
> +        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
> +        rte_socket_id());
> +    if (ioat_pktmbuf_pool == NULL)
> +        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
> +
> +Mbufs are the packet buffer structure used by DPDK. They are 
> +explained in detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
> +
> +The ``main()`` function also initializes the ports:
> +
> +.. code-block:: c
> +
> +    /* Initialise each port */
> +    RTE_ETH_FOREACH_DEV(portid) {
> +        port_init(portid, ioat_pktmbuf_pool);
> +    }
> +
> +Each port is configured using ``port_init()``:
> +
> +.. code-block:: c
> +
> +     /*
> +     * Initializes a given port using global settings and with the RX buffers
> +     * coming from the mbuf_pool passed as a parameter.
> +     */
> +    static inline void
> +    port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
> +    {
> +        /* configuring port to use RSS for multiple RX queues */
> +        static const struct rte_eth_conf port_conf = {
> +            .rxmode = {
> +                .mq_mode        = ETH_MQ_RX_RSS,
> +                .max_rx_pkt_len = RTE_ETHER_MAX_LEN
> +            },
> +            .rx_adv_conf = {
> +                .rss_conf = {
> +                    .rss_key = NULL,
> +                    .rss_hf = ETH_RSS_PROTO_MASK,
> +                }
> +            }
> +        };
> +
> +        struct rte_eth_rxconf rxq_conf;
> +        struct rte_eth_txconf txq_conf;
> +        struct rte_eth_conf local_port_conf = port_conf;
> +        struct rte_eth_dev_info dev_info;
> +        int ret, i;
> +
> +        /* Skip ports that are not enabled */
> +        if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
> +            printf("Skipping disabled port %u\n", portid);
> +            return;
> +        }
> +
> +        /* Init port */
> +        printf("Initializing port %u... ", portid);
> +        fflush(stdout);
> +        rte_eth_dev_info_get(portid, &dev_info);
> +        local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
> +            dev_info.flow_type_rss_offloads;
> +        if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
> +            local_port_conf.txmode.offloads |=
> +                DEV_TX_OFFLOAD_MBUF_FAST_FREE;
> +        ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE, "Cannot configure device:"
> +                " err=%d, port=%u\n", ret, portid);
> +
> +        ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
> +                            &nb_txd);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "Cannot adjust number of descriptors: err=%d, port=%u\n",
> +                ret, portid);
> +
> +        rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
> +
> +        /* Init Rx queues */
> +        rxq_conf = dev_info.default_rxconf;
> +        rxq_conf.offloads = local_port_conf.rxmode.offloads;
> +        for (i = 0; i < nb_queues; i++) {
> +            ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
> +                rte_eth_dev_socket_id(portid), &rxq_conf,
> +                mbuf_pool);
> +            if (ret < 0)
> +                rte_exit(EXIT_FAILURE,
> +                    "rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
> +                    ret, portid, i);
> +        }
> +
> +        /* Init one TX queue on each port */
> +        txq_conf = dev_info.default_txconf;
> +        txq_conf.offloads = local_port_conf.txmode.offloads;
> +        ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
> +                rte_eth_dev_socket_id(portid),
> +                &txq_conf);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "rte_eth_tx_queue_setup:err=%d,port=%u\n",
> +                ret, portid);
> +
> +        /* Initialize TX buffers */
> +        tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
> +                RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
> +                rte_eth_dev_socket_id(portid));
> +        if (tx_buffer[portid] == NULL)
> +            rte_exit(EXIT_FAILURE,
> +                "Cannot allocate buffer for tx on port %u\n",
> +                portid);
> +
> +        rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
> +
> +        ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
> +                rte_eth_tx_buffer_count_callback,
> +                &port_statistics.tx_dropped[portid]);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "Cannot set error callback for tx buffer on port %u\n",
> +                portid);
> +
> +        /* Start device */
> +        ret = rte_eth_dev_start(portid);
> +        if (ret < 0)
> +            rte_exit(EXIT_FAILURE,
> +                "rte_eth_dev_start:err=%d, port=%u\n",
> +                ret, portid);
> +
> +        rte_eth_promiscuous_enable(portid);
> +
> +        printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
> +                portid,
> +                ioat_ports_eth_addr[portid].addr_bytes[0],
> +                ioat_ports_eth_addr[portid].addr_bytes[1],
> +                ioat_ports_eth_addr[portid].addr_bytes[2],
> +                ioat_ports_eth_addr[portid].addr_bytes[3],
> +                ioat_ports_eth_addr[portid].addr_bytes[4],
> +                ioat_ports_eth_addr[portid].addr_bytes[5]);
> +
> +        cfg.ports[cfg.nb_ports].rxtx_port = portid;
> +        cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
> +    }
> +

This code is probably quite similar to that in other sample apps, so I don't think we need to include the full function here. It makes updating the code more difficult, so just refer to the function as doing the port init and leave it at that, I think. The snippets below give enough detail.
[Marcin] Changed the description accordingly.

> +The Ethernet ports are configured with local settings using the 
> +``rte_eth_dev_configure()`` function and the ``port_conf`` struct.
> +The RSS is enabled so that multiple Rx queues could be used for 
> +packet receiving and copying by multiple CBDMA channels per port:
> +
> +.. code-block:: c
> +
> +    /* configuring port to use RSS for multiple RX queues */
> +    static const struct rte_eth_conf port_conf = {
> +        .rxmode = {
> +            .mq_mode        = ETH_MQ_RX_RSS,
> +            .max_rx_pkt_len = RTE_ETHER_MAX_LEN
> +        },
> +        .rx_adv_conf = {
> +            .rss_conf = {
> +                .rss_key = NULL,
> +                .rss_hf = ETH_RSS_PROTO_MASK,
> +            }
> +        }
> +    };
> +
> +For this example the ports are set up with the number of Rx queues 
> +provided with -q option and 1 Tx queue using the 
> +``rte_eth_rx_queue_setup()`` and ``rte_eth_tx_queue_setup()`` functions.
> +
> +The Ethernet port is then started:
> +
> +.. code-block:: c
> +
> +    ret = rte_eth_dev_start(portid);
> +    if (ret < 0)
> +        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
> +            ret, portid);
> +
> +
> +Finally the Rx port is set in promiscuous mode:
> +
> +.. code-block:: c
> +
> +    rte_eth_promiscuous_enable(portid);
> +
> +
> +After that each port application assigns resources needed.
> +
> +.. code-block:: c
> +
> +    check_link_status(ioat_enabled_port_mask);
> +
> +    if (!cfg.nb_ports) {
> +        rte_exit(EXIT_FAILURE,
> +            "All available ports are disabled. Please set portmask.\n");
> +    }
> +
> +    /* Check if there is enough lcores for all ports. */
> +    cfg.nb_lcores = rte_lcore_count() - 1;
> +    if (cfg.nb_lcores < 1)
> +        rte_exit(EXIT_FAILURE,
> +            "There should be at least one slave lcore.\n");
> +
> +    ret = 0;
> +
> +    if (copy_mode == COPY_MODE_IOAT_NUM) {
> +        assign_rawdevs();
> +    } else /* copy_mode == COPY_MODE_SW_NUM */ {
> +        assign_rings();
> +    }
> +
> +A link status is checked of each port enabled by port mask using 
> +``check_link_status()`` function.
> +

I don't think this block needs to be covered. No need to go into everything in detail, just focus on the key parts of the app that are unique to it, i.e. the copying and passing mbufs between threads parts.
[Marcin] Changed the description accordingly.

> +.. code-block:: c
> +
> +    /* check link status, return true if at least one port is up */
> +    static int
> +    check_link_status(uint32_t port_mask)
> +    {
> +        uint16_t portid;
> +        struct rte_eth_link link;
> +        int retval = 0;
> +
> +        printf("\nChecking link status\n");
> +        RTE_ETH_FOREACH_DEV(portid) {
> +            if ((port_mask & (1 << portid)) == 0)
> +                continue;
> +
> +            memset(&link, 0, sizeof(link));
> +            rte_eth_link_get(portid, &link);
> +
> +            /* Print link status */
> +            if (link.link_status) {
> +                printf(
> +                    "Port %d Link Up. Speed %u Mbps - %s\n",
> +                    portid, link.link_speed,
> +                    (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
> +                    ("full-duplex") : ("half-duplex\n"));
> +                retval = 1;
> +            } else
> +                printf("Port %d Link Down\n", portid);
> +        }
> +        return retval;
> +    }
> +


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-27 14:51             ` Baran, MarcinX
@ 2019-09-27 15:00               ` Bruce Richardson
  2019-09-27 15:16                 ` Baran, MarcinX
  0 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-09-27 15:00 UTC (permalink / raw)
  To: Baran, MarcinX; +Cc: dev

On Fri, Sep 27, 2019 at 03:51:48PM +0100, Baran, MarcinX wrote:
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com> 
> Sent: Friday, September 27, 2019 1:02 PM
> To: Baran, MarcinX <marcinx.baran@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
> 
> On Fri, Sep 20, 2019 at 09:37:14AM +0200, Marcin Baran wrote:
> > Added guide for IOAT sample app usage and code description.
> > 
> > Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> > ---
> >  doc/guides/sample_app_ug/index.rst |   1 +
> >  doc/guides/sample_app_ug/intro.rst |   4 +
> >  doc/guides/sample_app_ug/ioat.rst  | 764 
> > +++++++++++++++++++++++++++++
> >  3 files changed, 769 insertions(+)
> >  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> > 
> > diff --git a/doc/guides/sample_app_ug/index.rst 
> > b/doc/guides/sample_app_ug/index.rst
> > index f23f8f59e..a6a1d9e7a 100644
> > --- a/doc/guides/sample_app_ug/index.rst
> > +++ b/doc/guides/sample_app_ug/index.rst
> > @@ -23,6 +23,7 @@ Sample Applications User Guides
> >      ip_reassembly
> >      kernel_nic_interface
> >      keep_alive
> > +    ioat
> >      l2_forward_crypto
> >      l2_forward_job_stats
> >      l2_forward_real_virtual
> > diff --git a/doc/guides/sample_app_ug/intro.rst 
> > b/doc/guides/sample_app_ug/intro.rst
> > index 90704194a..74462312f 100644
> > --- a/doc/guides/sample_app_ug/intro.rst
> > +++ b/doc/guides/sample_app_ug/intro.rst
> > @@ -91,6 +91,10 @@ examples are highlighted below.
> >    forwarding, or ``l3fwd`` application does forwarding based on Internet
> >    Protocol, IPv4 or IPv6 like a simple router.
> >  
> > +* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
> > +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev 
> > +driver for
> > +  copying packets between two threads.
> > +
> >  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
> >    demonstrates how to distribute packets arriving on an Rx port to different
> >    cores for processing and transmission.
> > diff --git a/doc/guides/sample_app_ug/ioat.rst 
> > b/doc/guides/sample_app_ug/ioat.rst
> > new file mode 100644
> > index 000000000..69621673b
> > --- /dev/null
> > +++ b/doc/guides/sample_app_ug/ioat.rst
> > @@ -0,0 +1,764 @@
> > +..  SPDX-License-Identifier: BSD-3-Clause
> > +    Copyright(c) 2019 Intel Corporation.
> > +
> > +Sample Application of packet copying using Intel\ |reg| QuickData 
> > +Technology 
> > +=====================================================================
> > +=======
> > +
> > +Overview
> > +--------
> > +
> > +This sample is intended as a demonstration of the basic components of 
> > +a DPDK forwarding application and example of how to use IOAT driver 
> > +API to make packets copies.
> > +
> > +Also while forwarding, the MAC addresses are affected as follows:
> > +
> > +*   The source MAC address is replaced by the TX port MAC address
> > +
> > +*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
> > +
> > +This application can be used to compare performance of using software 
> > +packet copy with copy done using a DMA device for different sizes of packets.
> > +The example will print out statistics each second. The stats shows 
> > +received/send packets and packets dropped or failed to copy.
> > +
> > +Compiling the Application
> > +-------------------------
> > +
> > +To compile the sample application see :doc:`compiling`.
> > +
> > +The application is located in the ``ioat`` sub-directory.
> > +
> > +
> > +Running the Application
> > +-----------------------
> > +
> > +In order to run the hardware copy application, the copying device 
> > +needs to be bound to user-space IO driver.
> > +
> > +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData 
> > +Technology* guide for information on using the driver.
> > +
> 
> The document is not called that, as the IOAT guide is just part of the overall rawdev document. So I suggest you just reference the rawdev guide.
> [Marcin] I wanted to refer to the ioat guide in /doc/guides/rawdevs/ioat.rst which has that title. Is there another document or I referenced this one
> incorrectly?
> 

It's one chapter in the overall "Rawdev Drivers" document:
https://doc.dpdk.org/guides/rawdevs/index.html

Since the (R) symbol doesn't seem to show up correctly in what you have
above (though it looks correct to me), I suggest just referring to the
"IOAT Rawdev Driver" chapter in the "Rawdev Drivers" document.


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-27 13:22           ` Bruce Richardson
@ 2019-09-27 15:13             ` Baran, MarcinX
  0 siblings, 0 replies; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-27 15:13 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev, Mcnamara, John, Kovacevic, Marko

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Friday, September 27, 2019 3:23 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org; Mcnamara, John <john.mcnamara@intel.com>; Kovacevic, Marko <marko.kovacevic@intel.com>
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide

On Fri, Sep 20, 2019 at 09:37:14AM +0200, Marcin Baran wrote:
> Added guide for IOAT sample app usage and code description.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> ---
>  doc/guides/sample_app_ug/index.rst |   1 +
>  doc/guides/sample_app_ug/intro.rst |   4 +
>  doc/guides/sample_app_ug/ioat.rst  | 764 
> +++++++++++++++++++++++++++++
>  3 files changed, 769 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> 

<snip>

> +Depending on mode set (whether copy should be done by software or by 
> +hardware) special structures are assigned to each port. If software 
> +copy was chosen, application have to assign ring structures for 
> +packet exchanging between lcores assigned to ports.
> +
> +.. code-block:: c
> +
> +    static void
> +    assign_rings(void)
> +    {
> +        uint32_t i;
> +
> +        for (i = 0; i < cfg.nb_ports; i++) {
> +            char ring_name[20];
> +
> +            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
> +            /* Create ring for inter core communication */
> +            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
> +                    ring_name, ring_size,
> +                    rte_socket_id(), RING_F_SP_ENQ);
> +
> +            if (cfg.ports[i].rx_to_tx_ring == NULL)
> +                rte_exit(EXIT_FAILURE, "%s\n",
> +                        rte_strerror(rte_errno));
> +        }
> +    }
> +
> +
> +When using hardware copy each Rx queue of the port is assigned an 
> +IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API
> +functions:
> +
> +.. code-block:: c
> +
> +    static void
> +    assign_rawdevs(void)
> +    {
> +        uint16_t nb_rawdev = 0, rdev_id = 0;
> +        uint32_t i, j;
> +
> +        for (i = 0; i < cfg.nb_ports; i++) {
> +            for (j = 0; j < cfg.ports[i].nb_queues; j++) {
> +                struct rte_rawdev_info rdev_info = { 0 };
> +
> +                do {
> +                    if (rdev_id == rte_rawdev_count())
> +                        goto end;
> +                    rte_rawdev_info_get(rdev_id++, &rdev_info);
> +                } while (strcmp(rdev_info.driver_name,
> +                    IOAT_PMD_RAWDEV_NAME_STR) != 0);
> +
> +                cfg.ports[i].ioat_ids[j] = rdev_id - 1;
> +                configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
> +                ++nb_rawdev;
> +            }
> +        }
> +    end:
> +        if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
> +            rte_exit(EXIT_FAILURE,
> +                "Not enough IOAT rawdevs (%u) for all queues (%u).\n",
> +                nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
> +        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
> +    }
> +
> +
> +The initialization of hardware device is done by 
> +``rte_rawdev_configure()`` function and ``rte_rawdev_info`` struct.

... using ``rte_rawdev_info`` struct
[Marcin] Changed the description accordingly.

> After configuration the device is
> +started using ``rte_rawdev_start()`` function. Each of the above 
> +operations is done in ``configure_rawdev_queue()``.

In the block below, there is no mention of where dev_config structure comes from. Presume it's a global variable, so maybe mention that in the text.
[Marcin] Actually this code snipped was not updated, it now uses local dev_config variable of type struct rte_ioat_rawdev_config, so I updated the snippet.

> +
> +.. code-block:: c
> +
> +    static void
> +    configure_rawdev_queue(uint32_t dev_id)
> +    {
> +        struct rte_rawdev_info info = { .dev_private = &dev_config };
> +
> +        /* Configure hardware copy device */
> +        dev_config.ring_size = ring_size;
> +
> +        if (rte_rawdev_configure(dev_id, &info) != 0) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error with rte_rawdev_configure()\n");
> +        }
> +        rte_rawdev_info_get(dev_id, &info);
> +        if (dev_config.ring_size != ring_size) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error, ring size is not %d (%d)\n",
> +                ring_size, (int)dev_config.ring_size);
> +        }
> +        if (rte_rawdev_start(dev_id) != 0) {
> +            rte_exit(EXIT_FAILURE,
> +                "Error with rte_rawdev_start()\n");
> +        }
> +    }
> +
> +If initialization is successful memory for hardware device statistics 
> +is allocated.

Missing "," after successful.
Where is this memory allocated? It is done in main or elsewhere?
[Marcin] This comment is invalid now as code has been updated, I removed it.
> +
> +Finally ``main()`` functions starts all processing lcores and starts

s/functions/function/
s/processing lcores/packet handling lcores/
[Marcin] Changed the description accordingly

> +printing stats in a loop on master lcore. The application can be

s/master lcore/the master lcore/
[Marcin] Changed the description accordingly

> +interrupted and closed using ``Ctrl-C``. The master lcore waits for 
> +all slave processes to finish, deallocates resources and exits.
> +
> +The processing lcores launching function are described below.
> +
> +The Lcores Launching Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +As described above ``main()`` function invokes 
> +``start_forwarding_cores()``

Missing "," after above.
[Marcin] Changed the description accordingly

> +function in order to start processing for each lcore:
> +
> +.. code-block:: c
> +
> +    static void start_forwarding_cores(void)
> +    {
> +        uint32_t lcore_id = rte_lcore_id();
> +
> +        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
> +                __func__, rte_lcore_id());
> +
> +        if (cfg.nb_lcores == 1) {
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
> +                NULL, lcore_id);
> +        } else if (cfg.nb_lcores > 1) {
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
> +                NULL, lcore_id);
> +
> +            lcore_id = rte_get_next_lcore(lcore_id, true, true);
> +            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
> +                lcore_id);
> +        }
> +    }
> +
> +The function launches Rx/Tx processing functions on configured lcores 
> +for each port using ``rte_eal_remote_launch()``. The configured 
> +ports,

Remove "for each port"
[Marcin] Removed

> +their number and number of assigned lcores are stored in user-defined 
> +``rxtx_transmission_config`` struct that is initialized before 
> +launching

s/is/has been/
Did you describe how that structure was set up previously?
[Marcin] Added description as to how and when it is set up.

> +tasks:
> +
> +.. code-block:: c
> +
> +    struct rxtx_transmission_config {
> +        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
> +        uint16_t nb_ports;
> +        uint16_t nb_lcores;
> +    };
> +
> +The Lcores Processing Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +For receiving packets on each port an ``ioat_rx_port()`` function is used.

Missing "," after port.
s/an/the/
[Marcin] Changed the description accordingly

> +The function receives packets on each configured Rx queue. Depending 
> +on mode

s/mode/the mode/
[Marcin] Changed the description accordingly

> +the user chose, it will enqueue packets to IOAT rawdev channels and 
> +then invoke copy process (hardware copy), or perform software copy of 
> +each packet using ``pktmbuf_sw_copy()`` function and enqueue them to 1 rte_ring:

s/1 rte_ring/an rte_ring/
[Marcin] Changed the description accordingly

> +
> +.. code-block:: c
> +
> +    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
> +    static void
> +    ioat_rx_port(struct rxtx_port_config *rx_config)
> +    {
> +        uint32_t nb_rx, nb_enq, i, j;
> +        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
> +        for (i = 0; i < rx_config->nb_queues; i++) {
> +
> +            nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
> +                pkts_burst, MAX_PKT_BURST);
> +
> +            if (nb_rx == 0)
> +                continue;
> +
> +            port_statistics.rx[rx_config->rxtx_port] += nb_rx;
> +
> +            if (copy_mode == COPY_MODE_IOAT_NUM) {
> +                /* Perform packet hardware copy */
> +                nb_enq = ioat_enqueue_packets(pkts_burst,
> +                    nb_rx, rx_config->ioat_ids[i]);
> +                if (nb_enq > 0)
> +                    rte_ioat_do_copies(rx_config->ioat_ids[i]);
> +            } else {
> +                /* Perform packet software copy, free source packets */
> +                int ret;
> +                struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
> +
> +                ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +                    (void *)pkts_burst_copy, nb_rx);
> +
> +                if (unlikely(ret < 0))
> +                    rte_exit(EXIT_FAILURE,
> +                        "Unable to allocate memory.\n");
> +
> +                for (j = 0; j < nb_rx; j++)
> +                    pktmbuf_sw_copy(pkts_burst[j],
> +                        pkts_burst_copy[j]);
> +
> +                rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)pkts_burst, nb_rx);
> +
> +                nb_enq = rte_ring_enqueue_burst(
> +                    rx_config->rx_to_tx_ring,
> +                    (void *)pkts_burst_copy, nb_rx, NULL);
> +
> +                /* Free any not enqueued packets. */
> +                rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)&pkts_burst_copy[nb_enq],
> +                    nb_rx - nb_enq);
> +            }
> +
> +            port_statistics.copy_dropped[rx_config->rxtx_port] +=
> +                (nb_rx - nb_enq);
> +        }
> +    }
> +
> +The packets are received in burst mode using ``rte_eth_rx_burst()`` 
> +function. When using hardware copy mode the packets are enqueued in 
> +copying device's buffer using ``ioat_enqueue_packets()`` which calls 
> +``rte_ioat_enqueue_copy()``. When all received packets are in the 
> +buffer the copies are invoked by calling ``rte_ioat_do_copies()``.

s/copies are invoked/copy operations are started/
[Marcin] Changed the description accordingly

> +Function ``rte_ioat_enqueue_copy()`` operates on physical address of 
> +the packet. Structure ``rte_mbuf`` contains only physical address to 
> +start of the data buffer (``buf_iova``). Thus the address is shifted

s/shifted/adjusted/
[Marcin] Changed the description accordingly

> +by ``addr_offset`` value in order to get pointer to ``rearm_data``

s/pointer to/the address of/
[Marcin] Changed the description accordingly

> +member of ``rte_mbuf``. That way the packet is copied all at once 
> +(with data and metadata).

"That way the both the packet data and metadata can be copied in a single operation".
Should also note that this shortcut can be used because the mbufs are "direct" mbufs allocated by the apps. If another app uses external buffers, or indirect mbufs, then multiple copy operations must be used.
[Marcin] Changed the description accordingly and added additional informations.

> +
> +.. code-block:: c
> +
> +    static uint32_t
> +    ioat_enqueue_packets(struct rte_mbuf **pkts,
> +        uint32_t nb_rx, uint16_t dev_id)
> +    {
> +        int ret;
> +        uint32_t i;
> +        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
> +
> +        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
> +            &pkts[0]->rearm_data);
> +
> +        ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
> +                (void *)pkts_copy, nb_rx);
> +
> +        if (unlikely(ret < 0))
> +            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
> +
> +        for (i = 0; i < nb_rx; i++) {
> +            /* Perform data copy */
> +            ret = rte_ioat_enqueue_copy(dev_id,
> +                pkts[i]->buf_iova
> +                    - addr_offset,
> +                pkts_copy[i]->buf_iova
> +                    - addr_offset,
> +                rte_pktmbuf_data_len(pkts[i])
> +                    + addr_offset,
> +                (uintptr_t)pkts[i],
> +                (uintptr_t)pkts_copy[i],
> +                0 /* nofence */);
> +
> +            if (ret != 1)
> +                break;
> +        }
> +
> +        ret = i;
> +        /* Free any not enqueued packets. */
> +        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
> +        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
> +            nb_rx - i);
> +
> +        return ret;
> +    }
> +
> +
> +All done copies are processed by ``ioat_tx_port()`` function. When 
> +using

s/done/completed/
[Marcin] Changed the description accordingly

> +hardware copy mode the function invokes 
> +``rte_ioat_completed_copies()`` on each assigned IOAT channel to 
> +gather copied packets. If software copy mode is used the function 
> +dequeues copied packets from the rte_ring. Then each packet MAC 
> +address is changed if it was enabled. After that copies are sent in burst mode using `` rte_eth_tx_burst()``.
> +
> +
> +.. code-block:: c
> +
> +    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
> +    static void
> +    ioat_tx_port(struct rxtx_port_config *tx_config)
> +    {
> +        uint32_t i, j, nb_dq = 0;
> +        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
> +        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
> +
> +        if (copy_mode == COPY_MODE_IOAT_NUM) {
> +            for (i = 0; i < tx_config->nb_queues; i++) {
> +                /* Deque the mbufs from IOAT device. */
> +                nb_dq = rte_ioat_completed_copies(
> +                    tx_config->ioat_ids[i], MAX_PKT_BURST,
> +                    (void *)mbufs_src, (void *)mbufs_dst);
> +
> +                if (nb_dq == 0)
> +                    break;
> +
> +                rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)mbufs_src, nb_dq);
> +
> +                /* Update macs if enabled */
> +                if (mac_updating) {
> +                    for (j = 0; j < nb_dq; j++)
> +                        update_mac_addrs(mbufs_dst[j],
> +                            tx_config->rxtx_port);
> +                }
> +
> +                const uint16_t nb_tx = rte_eth_tx_burst(
> +                    tx_config->rxtx_port, 0,
> +                    (void *)mbufs_dst, nb_dq);
> +
> +                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
> +
> +                /* Free any unsent packets. */
> +                if (unlikely(nb_tx < nb_dq))
> +                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)&mbufs_dst[nb_tx],
> +                        nb_dq - nb_tx);
> +            }
> +        }
> +        else {
> +            for (i = 0; i < tx_config->nb_queues; i++) {
> +                /* Deque the mbufs from IOAT device. */
> +                nb_dq = rte_ring_dequeue_burst(tx_config->rx_to_tx_ring,
> +                    (void *)mbufs_dst, MAX_PKT_BURST, NULL);
> +
> +                if (nb_dq == 0)
> +                    return;
> +
> +                /* Update macs if enabled */
> +                if (mac_updating) {
> +                    for (j = 0; j < nb_dq; j++)
> +                        update_mac_addrs(mbufs_dst[j],
> +                            tx_config->rxtx_port);
> +                }
> +
> +                const uint16_t nb_tx = rte_eth_tx_burst(tx_config->rxtx_port,
> +                    0, (void *)mbufs_dst, nb_dq);
> +
> +                port_statistics.tx[tx_config->rxtx_port] += nb_tx;
> +
> +                /* Free any unsent packets. */
> +                if (unlikely(nb_tx < nb_dq))
> +                    rte_mempool_put_bulk(ioat_pktmbuf_pool,
> +                    (void *)&mbufs_dst[nb_tx],
> +                        nb_dq - nb_tx);
> +            }
> +        }
> +    }
> +
> +The Packet Copying Functions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In order to perform packet copy there is a user-defined function 
> +``pktmbuf_sw_copy()`` used. It copies a whole packet by copying 
> +metadata from source packet to new mbuf, and then copying a data 
> +chunk of source packet. Both memory copies are done using
> +``rte_memcpy()``:
> +
> +.. code-block:: c
> +
> +    static inline void
> +    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
> +    {
> +        /* Copy packet metadata */
> +        rte_memcpy(&dst->rearm_data,
> +            &src->rearm_data,
> +            offsetof(struct rte_mbuf, cacheline1)
> +                - offsetof(struct rte_mbuf, rearm_data));
> +
> +        /* Copy packet data */
> +        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
> +            rte_pktmbuf_mtod(src, char *), src->data_len);
> +    }
> +
> +The metadata in this example is copied from ``rearm_data`` member of 
> +``rte_mbuf`` struct up to ``cacheline1``.
> +
> +In order to understand why software packet copying is done as shown 
> +above please refer to the "Mbuf Library" section of the *DPDK 
> +Programmer's Guide*.
> \ No newline at end of file

Use a text editor that adds a newline automatically :-)
[Marcin] Added new line at end of file.

/Bruce

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-27 15:00               ` Bruce Richardson
@ 2019-09-27 15:16                 ` Baran, MarcinX
  0 siblings, 0 replies; 76+ messages in thread
From: Baran, MarcinX @ 2019-09-27 15:16 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: dev



-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Friday, September 27, 2019 5:01 PM
To: Baran, MarcinX <marcinx.baran@intel.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide

On Fri, Sep 27, 2019 at 03:51:48PM +0100, Baran, MarcinX wrote:
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Friday, September 27, 2019 1:02 PM
> To: Baran, MarcinX <marcinx.baran@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT 
> sample app guide
> 
> On Fri, Sep 20, 2019 at 09:37:14AM +0200, Marcin Baran wrote:
> > Added guide for IOAT sample app usage and code description.
> > 
> > Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> > ---
> >  doc/guides/sample_app_ug/index.rst |   1 +
> >  doc/guides/sample_app_ug/intro.rst |   4 +
> >  doc/guides/sample_app_ug/ioat.rst  | 764
> > +++++++++++++++++++++++++++++
> >  3 files changed, 769 insertions(+)
> >  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> > 
> > diff --git a/doc/guides/sample_app_ug/index.rst
> > b/doc/guides/sample_app_ug/index.rst
> > index f23f8f59e..a6a1d9e7a 100644
> > --- a/doc/guides/sample_app_ug/index.rst
> > +++ b/doc/guides/sample_app_ug/index.rst
> > @@ -23,6 +23,7 @@ Sample Applications User Guides
> >      ip_reassembly
> >      kernel_nic_interface
> >      keep_alive
> > +    ioat
> >      l2_forward_crypto
> >      l2_forward_job_stats
> >      l2_forward_real_virtual
> > diff --git a/doc/guides/sample_app_ug/intro.rst
> > b/doc/guides/sample_app_ug/intro.rst
> > index 90704194a..74462312f 100644
> > --- a/doc/guides/sample_app_ug/intro.rst
> > +++ b/doc/guides/sample_app_ug/intro.rst
> > @@ -91,6 +91,10 @@ examples are highlighted below.
> >    forwarding, or ``l3fwd`` application does forwarding based on Internet
> >    Protocol, IPv4 or IPv6 like a simple router.
> >  
> > +* :doc:`Hardware packet copying<ioat>`: The Hardware packet 
> > +copying,
> > +  or ``ioatfwd`` application demonstrates how to use IOAT rawdev 
> > +driver for
> > +  copying packets between two threads.
> > +
> >  * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
> >    demonstrates how to distribute packets arriving on an Rx port to different
> >    cores for processing and transmission.
> > diff --git a/doc/guides/sample_app_ug/ioat.rst
> > b/doc/guides/sample_app_ug/ioat.rst
> > new file mode 100644
> > index 000000000..69621673b
> > --- /dev/null
> > +++ b/doc/guides/sample_app_ug/ioat.rst
> > @@ -0,0 +1,764 @@
> > +..  SPDX-License-Identifier: BSD-3-Clause
> > +    Copyright(c) 2019 Intel Corporation.
> > +
> > +Sample Application of packet copying using Intel\ |reg| QuickData 
> > +Technology 
> > +===================================================================
> > +==
> > +=======
> > +
> > +Overview
> > +--------
> > +
> > +This sample is intended as a demonstration of the basic components 
> > +of a DPDK forwarding application and example of how to use IOAT 
> > +driver API to make packets copies.
> > +
> > +Also while forwarding, the MAC addresses are affected as follows:
> > +
> > +*   The source MAC address is replaced by the TX port MAC address
> > +
> > +*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
> > +
> > +This application can be used to compare performance of using 
> > +software packet copy with copy done using a DMA device for different sizes of packets.
> > +The example will print out statistics each second. The stats shows 
> > +received/send packets and packets dropped or failed to copy.
> > +
> > +Compiling the Application
> > +-------------------------
> > +
> > +To compile the sample application see :doc:`compiling`.
> > +
> > +The application is located in the ``ioat`` sub-directory.
> > +
> > +
> > +Running the Application
> > +-----------------------
> > +
> > +In order to run the hardware copy application, the copying device 
> > +needs to be bound to user-space IO driver.
> > +
> > +Refer to the *IOAT Rawdev Driver for Intel\ |reg| QuickData
> > +Technology* guide for information on using the driver.
> > +
> 
> The document is not called that, as the IOAT guide is just part of the overall rawdev document. So I suggest you just reference the rawdev guide.
> [Marcin] I wanted to refer to the ioat guide in 
> /doc/guides/rawdevs/ioat.rst which has that title. Is there another document or I referenced this one incorrectly?
> 

It's one chapter in the overall "Rawdev Drivers" document:
https://doc.dpdk.org/guides/rawdevs/index.html

Since the (R) symbol doesn't seem to show up correctly in what you have above (though it looks correct to me), I suggest just referring to the "IOAT Rawdev Driver" chapter in the "Rawdev Drivers" document.
[Marcin] Ok, I will change it like that.

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                           ` (5 preceding siblings ...)
  2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
@ 2019-09-30  7:50         ` Marcin Baran
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 1/6] examples/ioat: create " Marcin Baran
                             ` (7 more replies)
  2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
  7 siblings, 8 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-30  7:50 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The patch includes sample application as
well as it's guide.

v6:
 - rearrange 'ioat_tx_port()' to remove unnecessary code
 - improve documentation
 - format patches for better readability

v5:
 - change dependency name from 'pmd_ioat' to 'rawdev_ioat'
   fixing build errors (branch diverged from master)

v4:
 - fix meson build support check

v3:
 - add check for meson build whether IOAT is supported

v2:
 - change printing total stats to be deltas
 - update documentation
 - fix 1 thread/sw copy/multiple Rx queues packet dropping
 - divide patch into several presenting functional change


Marcin Baran (4):
  examples/ioat: add software copy support
  examples/ioat: add rawdev copy mode support
  examples/ioat: add stats printing for each port
  doc/guides/: provide IOAT sample app guide

Pawel Modrak (2):
  examples/ioat: create sample app on ioat driver usage
  examples/ioat: add two threads configuration

 doc/guides/sample_app_ug/index.rst |    1 +
 doc/guides/sample_app_ug/intro.rst |    4 +
 doc/guides/sample_app_ug/ioat.rst  |  581 ++++++++++++++++
 examples/Makefile                  |    3 +
 examples/ioat/Makefile             |   54 ++
 examples/ioat/ioatfwd.c            | 1003 ++++++++++++++++++++++++++++
 examples/ioat/meson.build          |   15 +
 examples/meson.build               |    1 +
 8 files changed, 1662 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v6 1/6] examples/ioat: create sample app on ioat driver usage
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
@ 2019-09-30  7:50           ` Marcin Baran
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 2/6] examples/ioat: add software copy support Marcin Baran
                             ` (6 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-30  7:50 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

A new sample app demonstrating use of driver for CBDMA.
The app receives packets, performs software or hardware
copy, changes packets' MAC addresses (if enabled) and
forwards them. The change covers ports initialization,
closing connection and argument parsing.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/Makefile         |   3 +
 examples/ioat/Makefile    |  54 +++++
 examples/ioat/ioatfwd.c   | 439 ++++++++++++++++++++++++++++++++++++++
 examples/ioat/meson.build |  15 ++
 examples/meson.build      |   1 +
 5 files changed, 512 insertions(+)
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

diff --git a/examples/Makefile b/examples/Makefile
index de11dd487..3cb313d7d 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation
 DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify
 DIRS-y += flow_filtering
 DIRS-y += helloworld
+ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y)
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat
+endif
 DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline
 ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
 DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
diff --git a/examples/ioat/Makefile b/examples/ioat/Makefile
new file mode 100644
index 000000000..2a4d1da2d
--- /dev/null
+++ b/examples/ioat/Makefile
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# binary name
+APP = ioatfwd
+
+# all source are stored in SRCS-y
+SRCS-y := ioatfwd.c
+
+# Build using pkg-config variables if possible
+ifeq ($(shell pkg-config --exists libdpdk && echo 0),0)
+
+all: shared
+.PHONY: shared static
+shared: build/$(APP)-shared
+	ln -sf $(APP)-shared build/$(APP)
+static: build/$(APP)-static
+	ln -sf $(APP)-static build/$(APP)
+
+PC_FILE := $(shell pkg-config --path libdpdk)
+CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
+LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
+
+build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
+
+build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
+
+build:
+	@mkdir -p $@
+
+.PHONY: clean
+clean:
+	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
+	test -d build && rmdir -p build || true
+
+else # Build using legacy build system
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, detect a build directory, by looking for a path with a .config
+RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config)))))
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+endif
diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
new file mode 100644
index 000000000..b33c1eeb4
--- /dev/null
+++ b/examples/ioat/ioatfwd.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#include <stdint.h>
+#include <getopt.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+#include <rte_ethdev.h>
+#include <rte_rawdev.h>
+#include <rte_ioat_rawdev.h>
+
+/* size of ring used for software copying between rx and tx. */
+#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
+#define MAX_PKT_BURST 32
+#define MEMPOOL_CACHE_SIZE 512
+#define MIN_POOL_SIZE 65536U
+#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
+#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
+#define CMD_LINE_OPT_PORTMASK "portmask"
+#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
+#define CMD_LINE_OPT_COPY_TYPE "copy-type"
+#define CMD_LINE_OPT_RING_SIZE "ring-size"
+
+/* configurable number of RX/TX ring descriptors */
+#define RX_DEFAULT_RINGSIZE 1024
+#define TX_DEFAULT_RINGSIZE 1024
+
+/* max number of RX queues per port */
+#define MAX_RX_QUEUES_COUNT 8
+
+struct rxtx_port_config {
+	/* common config */
+	uint16_t rxtx_port;
+	uint16_t nb_queues;
+	/* for software copy mode */
+	struct rte_ring *rx_to_tx_ring;
+	/* for IOAT rawdev copy mode */
+	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT];
+};
+
+struct rxtx_transmission_config {
+	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+	uint16_t nb_ports;
+	uint16_t nb_lcores;
+};
+
+typedef enum copy_mode_t {
+#define COPY_MODE_SW "sw"
+	COPY_MODE_SW_NUM,
+#define COPY_MODE_IOAT "hw"
+	COPY_MODE_IOAT_NUM,
+	COPY_MODE_INVALID_NUM,
+	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
+} copy_mode_t;
+
+/* mask of enabled ports */
+static uint32_t ioat_enabled_port_mask;
+
+/* number of RX queues per port */
+static uint16_t nb_queues = 1;
+
+/* MAC updating enabled by default. */
+static int mac_updating = 1;
+
+/* hardare copy mode enabled by default. */
+static copy_mode_t copy_mode = COPY_MODE_IOAT_NUM;
+
+/* size of IOAT rawdev ring for hardware copy mode or
+ * rte_ring for software copy mode
+ */
+static unsigned short ring_size = 2048;
+
+/* global transmission config */
+struct rxtx_transmission_config cfg;
+
+/* configurable number of RX/TX ring descriptors */
+static uint16_t nb_rxd = RX_DEFAULT_RINGSIZE;
+static uint16_t nb_txd = TX_DEFAULT_RINGSIZE;
+
+static volatile bool force_quit;
+
+/* ethernet addresses of ports */
+static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
+struct rte_mempool *ioat_pktmbuf_pool;
+
+/* Display usage */
+static void
+ioat_usage(const char *prgname)
+{
+	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
+		"  -p --portmask: hexadecimal bitmask of ports to configure\n"
+		"  -q NQ: number of RX queues per port (default is 1)\n"
+		"  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
+		"      When enabled:\n"
+		"       - The source MAC address is replaced by the TX port MAC address\n"
+		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
+		"  -c --copy-type CT: type of copy: sw|hw\n"
+		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
+			prgname);
+}
+
+static int
+ioat_parse_portmask(const char *portmask)
+{
+	char *end = NULL;
+	unsigned long pm;
+
+	/* Parse hexadecimal string */
+	pm = strtoul(portmask, &end, 16);
+	if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+
+	return pm;
+}
+
+static copy_mode_t
+ioat_parse_copy_mode(const char *copy_mode)
+{
+	if (strcmp(copy_mode, COPY_MODE_SW) == 0)
+		return COPY_MODE_SW_NUM;
+	else if (strcmp(copy_mode, COPY_MODE_IOAT) == 0)
+		return COPY_MODE_IOAT_NUM;
+
+	return COPY_MODE_INVALID_NUM;
+}
+
+/* Parse the argument given in the command line of the application */
+static int
+ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
+{
+	static const char short_options[] =
+		"p:"  /* portmask */
+		"q:"  /* number of RX queues per port */
+		"c:"  /* copy type (sw|hw) */
+		"s:"  /* ring size */
+		;
+
+	static const struct option lgopts[] = {
+		{CMD_LINE_OPT_MAC_UPDATING, no_argument, &mac_updating, 1},
+		{CMD_LINE_OPT_NO_MAC_UPDATING, no_argument, &mac_updating, 0},
+		{CMD_LINE_OPT_PORTMASK, required_argument, NULL, 'p'},
+		{CMD_LINE_OPT_NB_QUEUE, required_argument, NULL, 'q'},
+		{CMD_LINE_OPT_COPY_TYPE, required_argument, NULL, 'c'},
+		{CMD_LINE_OPT_RING_SIZE, required_argument, NULL, 's'},
+		{NULL, 0, 0, 0}
+	};
+
+	const unsigned int default_port_mask = (1 << nb_ports) - 1;
+	int opt, ret;
+	char **argvopt;
+	int option_index;
+	char *prgname = argv[0];
+
+	ioat_enabled_port_mask = default_port_mask;
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, short_options,
+			lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		/* portmask */
+		case 'p':
+			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
+			if (ioat_enabled_port_mask & ~default_port_mask ||
+					ioat_enabled_port_mask <= 0) {
+				printf("Invalid portmask, %s, suggest 0x%x\n",
+						optarg, default_port_mask);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'q':
+			nb_queues = atoi(optarg);
+			if (nb_queues == 0 || nb_queues > MAX_RX_QUEUES_COUNT) {
+				printf("Invalid RX queues number %s. Max %u\n",
+					optarg, MAX_RX_QUEUES_COUNT);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'c':
+			copy_mode = ioat_parse_copy_mode(optarg);
+			if (copy_mode == COPY_MODE_INVALID_NUM) {
+				printf("Invalid copy type. Use: sw, hw\n");
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 's':
+			ring_size = atoi(optarg);
+			if (ring_size == 0) {
+				printf("Invalid ring size, %s.\n", optarg);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		/* long options */
+		case 0:
+			break;
+
+		default:
+			ioat_usage(prgname);
+			return -1;
+		}
+	}
+
+	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
+	if (optind >= 0)
+		argv[optind - 1] = prgname;
+
+	ret = optind - 1;
+	optind = 1; /* reset getopt lib */
+	return ret;
+}
+
+/* check link status, return true if at least one port is up */
+static int
+check_link_status(uint32_t port_mask)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	int retval = 0;
+
+	printf("\nChecking link status\n");
+	RTE_ETH_FOREACH_DEV(portid) {
+		if ((port_mask & (1 << portid)) == 0)
+			continue;
+
+		memset(&link, 0, sizeof(link));
+		rte_eth_link_get(portid, &link);
+
+		/* Print link status */
+		if (link.link_status) {
+			printf(
+				"Port %d Link Up. Speed %u Mbps - %s\n",
+				portid, link.link_speed,
+				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+				("full-duplex") : ("half-duplex\n"));
+			retval = 1;
+		} else
+			printf("Port %d Link Down\n", portid);
+	}
+	return retval;
+}
+
+/*
+ * Initializes a given port using global settings and with the RX buffers
+ * coming from the mbuf_pool passed as a parameter.
+ */
+static inline void
+port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+{
+	/* configuring port to use RSS for multiple RX queues */
+	static const struct rte_eth_conf port_conf = {
+		.rxmode = {
+			.mq_mode = ETH_MQ_RX_RSS,
+			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
+		},
+		.rx_adv_conf = {
+			.rss_conf = {
+				.rss_key = NULL,
+				.rss_hf = ETH_RSS_PROTO_MASK,
+			}
+		}
+	};
+
+	struct rte_eth_rxconf rxq_conf;
+	struct rte_eth_txconf txq_conf;
+	struct rte_eth_conf local_port_conf = port_conf;
+	struct rte_eth_dev_info dev_info;
+	int ret, i;
+
+	/* Skip ports that are not enabled */
+	if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+		printf("Skipping disabled port %u\n", portid);
+		return;
+	}
+
+	/* Init port */
+	printf("Initializing port %u... ", portid);
+	fflush(stdout);
+	rte_eth_dev_info_get(portid, &dev_info);
+	local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+		dev_info.flow_type_rss_offloads;
+	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+		local_port_conf.txmode.offloads |=
+			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Cannot configure device:"
+			" err=%d, port=%u\n", ret, portid);
+
+	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+			&nb_txd);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot adjust number of descriptors: err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+	/* Init RX queues */
+	rxq_conf = dev_info.default_rxconf;
+	rxq_conf.offloads = local_port_conf.rxmode.offloads;
+	for (i = 0; i < nb_queues; i++) {
+		ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+			rte_eth_dev_socket_id(portid), &rxq_conf,
+			mbuf_pool);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE,
+				"rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+				ret, portid, i);
+	}
+
+	/* Init one TX queue on each port */
+	txq_conf = dev_info.default_txconf;
+	txq_conf.offloads = local_port_conf.txmode.offloads;
+	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+			rte_eth_dev_socket_id(portid),
+			&txq_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
+			ret, portid);
+
+	/* Initialize TX buffers */
+	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+			rte_eth_dev_socket_id(portid));
+	if (tx_buffer[portid] == NULL)
+		rte_exit(EXIT_FAILURE,
+			"Cannot allocate buffer for tx on port %u\n",
+			portid);
+
+	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+	/* Start device */
+	ret = rte_eth_dev_start(portid);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_dev_start:err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_promiscuous_enable(portid);
+
+	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+			portid,
+			ioat_ports_eth_addr[portid].addr_bytes[0],
+			ioat_ports_eth_addr[portid].addr_bytes[1],
+			ioat_ports_eth_addr[portid].addr_bytes[2],
+			ioat_ports_eth_addr[portid].addr_bytes[3],
+			ioat_ports_eth_addr[portid].addr_bytes[4],
+			ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+	cfg.ports[cfg.nb_ports].rxtx_port = portid;
+	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\n\nSignal %d received, preparing to exit...\n",
+			signum);
+		force_quit = true;
+	}
+}
+
+int
+main(int argc, char **argv)
+{
+	int ret;
+	uint16_t nb_ports, portid;
+	uint32_t i;
+	unsigned int nb_mbufs;
+
+	/* Init EAL */
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+	argc -= ret;
+	argv += ret;
+
+	force_quit = false;
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	nb_ports = rte_eth_dev_count_avail();
+	if (nb_ports == 0)
+		rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
+
+	/* Parse application arguments (after the EAL ones) */
+	ret = ioat_parse_args(argc, argv, nb_ports);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid IOAT arguments\n");
+
+	nb_mbufs = RTE_MAX(nb_ports * (nb_queues * (nb_rxd + nb_txd +
+		4 * MAX_PKT_BURST) + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+		MIN_POOL_SIZE);
+
+	/* Create the mbuf pool */
+	ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+		MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+		rte_socket_id());
+	if (ioat_pktmbuf_pool == NULL)
+		rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+	/* Initialise each port */
+	cfg.nb_ports = 0;
+	RTE_ETH_FOREACH_DEV(portid)
+		port_init(portid, ioat_pktmbuf_pool, nb_queues);
+
+	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
+		sleep(1);
+
+	/* Check if there is enough lcores for all ports. */
+	cfg.nb_lcores = rte_lcore_count() - 1;
+	if (cfg.nb_lcores < 1)
+		rte_exit(EXIT_FAILURE,
+			"There should be at least one slave lcore.\n");
+	for (i = 0; i < cfg.nb_ports; i++) {
+		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
+		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
+		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+	}
+
+	printf("Bye...\n");
+	return 0;
+}
diff --git a/examples/ioat/meson.build b/examples/ioat/meson.build
new file mode 100644
index 000000000..ed8328963
--- /dev/null
+++ b/examples/ioat/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+build = dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
+
+deps += ['rawdev_ioat']
+
+sources = files(
+	'ioatfwd.c'
+)
diff --git a/examples/meson.build b/examples/meson.build
index a046b74ad..c2e18b59b 100644
--- a/examples/meson.build
+++ b/examples/meson.build
@@ -16,6 +16,7 @@ all_examples = [
 	'eventdev_pipeline', 'exception_path',
 	'fips_validation', 'flow_classify',
 	'flow_filtering', 'helloworld',
+	'ioat',
 	'ip_fragmentation', 'ip_pipeline',
 	'ip_reassembly', 'ipsec-secgw',
 	'ipv4_multicast', 'kni',
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v6 2/6] examples/ioat: add software copy support
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 1/6] examples/ioat: create " Marcin Baran
@ 2019-09-30  7:50           ` Marcin Baran
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
                             ` (5 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-30  7:50 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets
using software copy mode and MAC address
changing. The copies are processed using
one lcore.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 177 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 177 insertions(+)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index b33c1eeb4..2a4da948a 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -89,6 +89,154 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+static void
+update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
+{
+	struct rte_ether_hdr *eth;
+	void *tmp;
+
+	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
+
+	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
+	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
+	 */
+	tmp = &eth->d_addr.addr_bytes[0];
+	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
+
+	/* src addr */
+	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], &eth->s_addr);
+}
+
+static inline void
+pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+{
+	/* Copy packet metadata */
+	rte_memcpy(&dst->rearm_data,
+		&src->rearm_data,
+		offsetof(struct rte_mbuf, cacheline1)
+		- offsetof(struct rte_mbuf, rearm_data));
+
+	/* Copy packet data */
+	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+		rte_pktmbuf_mtod(src, char *), src->data_len);
+}
+
+/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+static void
+ioat_rx_port(struct rxtx_port_config *rx_config)
+{
+	uint32_t nb_rx, nb_enq, i, j;
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+
+	for (i = 0; i < rx_config->nb_queues; i++) {
+
+		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+			pkts_burst, MAX_PKT_BURST);
+
+		if (nb_rx == 0)
+			continue;
+
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			/* Perform packet hardware copy */
+		} else {
+			/* Perform packet software copy, free source packets */
+			int ret;
+			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst_copy, nb_rx);
+
+			if (unlikely(ret < 0))
+				rte_exit(EXIT_FAILURE,
+					"Unable to allocate memory.\n");
+
+			for (j = 0; j < nb_rx; j++)
+				pktmbuf_sw_copy(pkts_burst[j],
+					pkts_burst_copy[j]);
+
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst, nb_rx);
+
+			nb_enq = rte_ring_enqueue_burst(
+				rx_config->rx_to_tx_ring,
+				(void *)pkts_burst_copy, nb_rx, NULL);
+
+			/* Free any not enqueued packets. */
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&pkts_burst_copy[nb_enq],
+				nb_rx - nb_enq);
+		}
+	}
+}
+
+/* Transmit packets from IOAT rawdev/rte_ring for one port. */
+static void
+ioat_tx_port(struct rxtx_port_config *tx_config)
+{
+	uint32_t i, j, nb_dq = 0;
+	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+	for (i = 0; i < tx_config->nb_queues; i++) {
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			/* Deque the mbufs from IOAT device. */
+		} else {
+			/* Deque the mbufs from rx_to_tx_ring. */
+			nb_dq = rte_ring_dequeue_burst(
+				tx_config->rx_to_tx_ring, (void *)mbufs_dst,
+				MAX_PKT_BURST, NULL);
+		}
+
+		if (nb_dq == 0)
+			return;
+
+		/* Update macs if enabled */
+		if (mac_updating) {
+			for (j = 0; j < nb_dq; j++)
+				update_mac_addrs(mbufs_dst[j],
+					tx_config->rxtx_port);
+		}
+
+		const uint16_t nb_tx = rte_eth_tx_burst(
+			tx_config->rxtx_port, 0,
+			(void *)mbufs_dst, nb_dq);
+
+		/* Free any unsent packets. */
+		if (unlikely(nb_tx < nb_dq))
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&mbufs_dst[nb_tx],
+				nb_dq - nb_tx);
+	}
+}
+
+/* Main rx and tx loop if only one slave lcore available */
+static void
+rxtx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
+		" lcore %u\n", rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++) {
+			ioat_rx_port(&cfg.ports[i]);
+			ioat_tx_port(&cfg.ports[i]);
+		}
+}
+
+static void start_forwarding_cores(void)
+{
+	uint32_t lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+		__func__, rte_lcore_id());
+
+	lcore_id = rte_get_next_lcore(lcore_id, true, true);
+	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+		NULL, lcore_id);
+}
+
 /* Display usage */
 static void
 ioat_usage(const char *prgname)
@@ -253,6 +401,26 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+assign_rings(void)
+{
+	uint32_t i;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		char ring_name[RTE_RING_NAMESIZE];
+
+		snprintf(ring_name, sizeof(ring_name), "rx_to_tx_ring_%u", i);
+		/* Create ring for inter core communication */
+		cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+			ring_name, ring_size,
+			rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
+
+		if (cfg.ports[i].rx_to_tx_ring == NULL)
+			rte_exit(EXIT_FAILURE, "Ring create failed: %s\n",
+				rte_strerror(rte_errno));
+	}
+}
+
 /*
  * Initializes a given port using global settings and with the RX buffers
  * coming from the mbuf_pool passed as a parameter.
@@ -428,10 +596,19 @@ main(int argc, char **argv)
 	if (cfg.nb_lcores < 1)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
+
+	assign_rings();
+
+	start_forwarding_cores();
+
+	/* force_quit is true when we get here */
+	rte_eal_mp_wait_lcore();
+
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v6 3/6] examples/ioat: add rawdev copy mode support
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 1/6] examples/ioat: create " Marcin Baran
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 2/6] examples/ioat: add software copy support Marcin Baran
@ 2019-09-30  7:50           ` Marcin Baran
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 4/6] examples/ioat: add two threads configuration Marcin Baran
                             ` (4 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-30  7:50 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added support for copying packets using
rawdev device. Each port's Rx queue is
assigned DMA channel for copy.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 117 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 115 insertions(+), 2 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 2a4da948a..a38b6700b 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
 		rte_pktmbuf_mtod(src, char *), src->data_len);
 }
 
+static uint32_t
+ioat_enqueue_packets(struct rte_mbuf **pkts,
+	uint32_t nb_rx, uint16_t dev_id)
+{
+	int ret;
+	uint32_t i;
+	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+		&pkts[0]->rearm_data);
+
+	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+		(void *)pkts_copy, nb_rx);
+
+	if (unlikely(ret < 0))
+		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+	for (i = 0; i < nb_rx; i++) {
+		/* Perform data copy */
+		ret = rte_ioat_enqueue_copy(dev_id,
+			pkts[i]->buf_iova
+			- addr_offset,
+			pkts_copy[i]->buf_iova
+			- addr_offset,
+			rte_pktmbuf_data_len(pkts[i])
+			+ addr_offset,
+			(uintptr_t)pkts[i],
+			(uintptr_t)pkts_copy[i],
+			0 /* nofence */);
+
+		if (ret != 1)
+			break;
+	}
+
+	ret = i;
+	/* Free any not enqueued packets. */
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+		nb_rx - i);
+
+
+	return ret;
+}
+
 /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
 static void
 ioat_rx_port(struct rxtx_port_config *rx_config)
@@ -138,6 +182,10 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Perform packet hardware copy */
+			nb_enq = ioat_enqueue_packets(pkts_burst,
+				nb_rx, rx_config->ioat_ids[i]);
+			if (nb_enq > 0)
+				rte_ioat_do_copies(rx_config->ioat_ids[i]);
 		} else {
 			/* Perform packet software copy, free source packets */
 			int ret;
@@ -174,11 +222,15 @@ static void
 ioat_tx_port(struct rxtx_port_config *tx_config)
 {
 	uint32_t i, j, nb_dq = 0;
+	struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
 	for (i = 0; i < tx_config->nb_queues; i++) {
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Deque the mbufs from IOAT device. */
+			nb_dq = rte_ioat_completed_copies(
+				tx_config->ioat_ids[i], MAX_PKT_BURST,
+				(void *)mbufs_src, (void *)mbufs_dst);
 		} else {
 			/* Deque the mbufs from rx_to_tx_ring. */
 			nb_dq = rte_ring_dequeue_burst(
@@ -189,6 +241,10 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 		if (nb_dq == 0)
 			return;
 
+		if (copy_mode == COPY_MODE_IOAT_NUM)
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)mbufs_src, nb_dq);
+
 		/* Update macs if enabled */
 		if (mac_updating) {
 			for (j = 0; j < nb_dq; j++)
@@ -401,6 +457,52 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+configure_rawdev_queue(uint32_t dev_id)
+{
+	struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
+	struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+	if (rte_rawdev_configure(dev_id, &info) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_configure()\n");
+	}
+	if (rte_rawdev_start(dev_id) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_start()\n");
+	}
+}
+
+static void
+assign_rawdevs(void)
+{
+	uint16_t nb_rawdev = 0, rdev_id = 0;
+	uint32_t i, j;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+			struct rte_rawdev_info rdev_info = { 0 };
+
+			do {
+				if (rdev_id == rte_rawdev_count())
+					goto end;
+				rte_rawdev_info_get(rdev_id++, &rdev_info);
+			} while (strcmp(rdev_info.driver_name,
+				IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+			cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+			configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+			++nb_rawdev;
+		}
+	}
+end:
+	if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+		rte_exit(EXIT_FAILURE,
+			"Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+			nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+	RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+}
+
 static void
 assign_rings(void)
 {
@@ -597,18 +699,29 @@ main(int argc, char **argv)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
 
-	assign_rings();
+	if (copy_mode == COPY_MODE_IOAT_NUM)
+		assign_rawdevs();
+	else /* copy_mode == COPY_MODE_SW_NUM */
+		assign_rings();
 
 	start_forwarding_cores();
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
 
+	uint32_t j;
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
-		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+				printf("Stopping rawdev %d\n",
+					cfg.ports[i].ioat_ids[j]);
+				rte_rawdev_stop(cfg.ports[i].ioat_ids[j]);
+			}
+		} else /* copy_mode == COPY_MODE_SW_NUM */
+			rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v6 4/6] examples/ioat: add two threads configuration
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                             ` (2 preceding siblings ...)
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
@ 2019-09-30  7:50           ` Marcin Baran
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 5/6] examples/ioat: add stats printing for each port Marcin Baran
                             ` (3 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-30  7:50 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Pawel Modrak, Marcin Baran

From: Pawel Modrak <pawelx.modrak@intel.com>

Added possibility to use two lcores: first for
packet receiving and copying, second for packets
sending.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 examples/ioat/ioatfwd.c | 47 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 4 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index a38b6700b..2fc0b1173 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,
 	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
 		nb_rx - i);
 
-
 	return ret;
 }
 
@@ -264,6 +263,36 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 	}
 }
 
+/* Main rx processing loop for IOAT rawdev. */
+static void
+rx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_rx_port(&cfg.ports[i]);
+}
+
+/* Main tx processing loop for hardware copy. */
+static void
+tx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main tx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_tx_port(&cfg.ports[i]);
+}
+
 /* Main rx and tx loop if only one slave lcore available */
 static void
 rxtx_main_loop(void)
@@ -288,9 +317,19 @@ static void start_forwarding_cores(void)
 	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
 		__func__, rte_lcore_id());
 
-	lcore_id = rte_get_next_lcore(lcore_id, true, true);
-	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
-		NULL, lcore_id);
+	if (cfg.nb_lcores == 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+			NULL, lcore_id);
+	} else if (cfg.nb_lcores > 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+			NULL, lcore_id);
+
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+			lcore_id);
+	}
 }
 
 /* Display usage */
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v6 5/6] examples/ioat: add stats printing for each port
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                             ` (3 preceding siblings ...)
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 4/6] examples/ioat: add two threads configuration Marcin Baran
@ 2019-09-30  7:50           ` Marcin Baran
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
                             ` (2 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-30  7:50 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran, Pawel Modrak

Added printing stats of ports each second.
The stats printing is done using master core.
The information provided informs about packets
received, dropped and send as well as statistics
of rawdev devices.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
---
 examples/ioat/ioatfwd.c | 235 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 235 insertions(+)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 2fc0b1173..5e623a4d5 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -48,6 +48,23 @@ struct rxtx_transmission_config {
 	uint16_t nb_lcores;
 };
 
+/* per-port statistics struct */
+struct ioat_port_statistics {
+	uint64_t rx[RTE_MAX_ETHPORTS];
+	uint64_t tx[RTE_MAX_ETHPORTS];
+	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
+	uint64_t copy_dropped[RTE_MAX_ETHPORTS];
+};
+struct ioat_port_statistics port_statistics;
+
+struct total_statistics {
+	uint64_t total_packets_dropped;
+	uint64_t total_packets_tx;
+	uint64_t total_packets_rx;
+	uint64_t total_successful_enqueues;
+	uint64_t total_failed_enqueues;
+};
+
 typedef enum copy_mode_t {
 #define COPY_MODE_SW "sw"
 	COPY_MODE_SW_NUM,
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+/* Print out statistics for one port. */
+static void
+print_port_stats(uint16_t port_id)
+{
+	printf("\nStatistics for port %u ------------------------------"
+		"\nPackets sent: %34"PRIu64
+		"\nPackets received: %30"PRIu64
+		"\nPackets dropped on tx: %25"PRIu64
+		"\nPackets dropped on copy: %23"PRIu64,
+		port_id,
+		port_statistics.tx[port_id],
+		port_statistics.rx[port_id],
+		port_statistics.tx_dropped[port_id],
+		port_statistics.copy_dropped[port_id]);
+}
+
+/* Print out statistics for one IOAT rawdev device. */
+static void
+print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
+	unsigned int *ids_xstats, uint16_t nb_xstats,
+	struct rte_rawdev_xstats_name *names_xstats)
+{
+	uint16_t i;
+
+	printf("\nIOAT channel %u", dev_id);
+	for (i = 0; i < nb_xstats; i++)
+		printf("\n\t %s: %*"PRIu64,
+			names_xstats[ids_xstats[i]].name,
+			(int)(37 - strlen(names_xstats[ids_xstats[i]].name)),
+			xstats[i]);
+}
+
+static void
+print_total_stats(struct total_statistics *ts)
+{
+	printf("\nAggregate statistics ==============================="
+		"\nTotal packets Tx: %24"PRIu64" [pps]"
+		"\nTotal packets Rx: %24"PRIu64" [pps]"
+		"\nTotal packets dropped: %19"PRIu64" [pps]",
+		ts->total_packets_tx,
+		ts->total_packets_rx,
+		ts->total_packets_dropped);
+
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		printf("\nTotal IOAT successful enqueues: %8"PRIu64" [enq/s]"
+			"\nTotal IOAT failed enqueues: %12"PRIu64" [enq/s]",
+			ts->total_successful_enqueues,
+			ts->total_failed_enqueues);
+	}
+
+	printf("\n====================================================\n");
+}
+
+/* Print out statistics on packets dropped. */
+static void
+print_stats(char *prgname)
+{
+	struct total_statistics ts, delta_ts;
+	uint32_t i, port_id, dev_id;
+	struct rte_rawdev_xstats_name *names_xstats;
+	uint64_t *xstats;
+	unsigned int *ids_xstats, nb_xstats;
+	char status_string[120]; /* to print at the top of the output */
+	int status_strlen;
+
+
+	const char clr[] = { 27, '[', '2', 'J', '\0' };
+	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+	status_strlen = snprintf(status_string, sizeof(status_string),
+		"%s, ", prgname);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Worker Threads = %d, ",
+		rte_lcore_count() > 2 ? 2 : 1);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
+		COPY_MODE_SW : COPY_MODE_IOAT);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Updating MAC = %s, ", mac_updating ?
+		"enabled" : "disabled");
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Rx Queues = %d, ", nb_queues);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Ring Size = %d\n", ring_size);
+
+	/* Allocate memory for xstats names and values */
+	nb_xstats = rte_rawdev_xstats_names_get(
+		cfg.ports[0].ioat_ids[0], NULL, 0);
+
+	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
+	if (names_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat names memory\n");
+	}
+	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
+		names_xstats, nb_xstats);
+
+	ids_xstats = malloc(sizeof(*ids_xstats) * 2);
+	if (ids_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat ids_xstats memory\n");
+	}
+
+	xstats = malloc(sizeof(*xstats) * 2);
+	if (xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat memory\n");
+	}
+
+	/* Get failed/successful enqueues stats index */
+	ids_xstats[0] = ids_xstats[1] = nb_xstats;
+	for (i = 0; i < nb_xstats; i++) {
+		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
+			ids_xstats[0] = i;
+		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
+			ids_xstats[1] = i;
+		if (ids_xstats[0] < nb_xstats && ids_xstats[1] < nb_xstats)
+			break;
+	}
+	if (ids_xstats[0] == nb_xstats || ids_xstats[1] == nb_xstats) {
+		rte_exit(EXIT_FAILURE,
+			"Error getting failed/successful enqueues stats index\n");
+	}
+
+	memset(&ts, 0, sizeof(struct total_statistics));
+
+	while (!force_quit) {
+		/* Sleep for 1 second each round - init sleep allows reading
+		 * messages from app startup.
+		 */
+		sleep(1);
+
+		/* Clear screen and move to top left */
+		printf("%s%s", clr, topLeft);
+
+		memset(&delta_ts, 0, sizeof(struct total_statistics));
+
+		printf("%s", status_string);
+
+		for (i = 0; i < cfg.nb_ports; i++) {
+			port_id = cfg.ports[i].rxtx_port;
+			print_port_stats(port_id);
+
+			delta_ts.total_packets_dropped +=
+				port_statistics.tx_dropped[port_id]
+				+ port_statistics.copy_dropped[port_id];
+			delta_ts.total_packets_tx +=
+				port_statistics.tx[port_id];
+			delta_ts.total_packets_rx +=
+				port_statistics.rx[port_id];
+
+			if (copy_mode == COPY_MODE_IOAT_NUM) {
+				uint32_t j;
+
+				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+					dev_id = cfg.ports[i].ioat_ids[j];
+					rte_rawdev_xstats_get(dev_id,
+						ids_xstats, xstats, 2);
+
+					print_rawdev_stats(dev_id, xstats,
+						ids_xstats, 2, names_xstats);
+
+					delta_ts.total_failed_enqueues +=
+						xstats[ids_xstats[0]];
+					delta_ts.total_successful_enqueues +=
+						xstats[ids_xstats[1]];
+				}
+			}
+		}
+
+		delta_ts.total_packets_tx -= ts.total_packets_tx;
+		delta_ts.total_packets_rx -= ts.total_packets_rx;
+		delta_ts.total_packets_dropped -= ts.total_packets_dropped;
+		delta_ts.total_failed_enqueues -= ts.total_failed_enqueues;
+		delta_ts.total_successful_enqueues -=
+			ts.total_successful_enqueues;
+
+		printf("\n");
+		print_total_stats(&delta_ts);
+
+		ts.total_packets_tx += delta_ts.total_packets_tx;
+		ts.total_packets_rx += delta_ts.total_packets_rx;
+		ts.total_packets_dropped += delta_ts.total_packets_dropped;
+		ts.total_failed_enqueues += delta_ts.total_failed_enqueues;
+		ts.total_successful_enqueues +=
+			delta_ts.total_successful_enqueues;
+	}
+
+	free(names_xstats);
+	free(xstats);
+	free(ids_xstats);
+}
+
 static void
 update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
 {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
+		port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Perform packet hardware copy */
 			nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 				(void *)&pkts_burst_copy[nb_enq],
 				nb_rx - nb_enq);
 		}
+
+		port_statistics.copy_dropped[rx_config->rxtx_port] +=
+			(nb_rx - nb_enq);
 	}
 }
 
@@ -255,6 +475,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 			tx_config->rxtx_port, 0,
 			(void *)mbufs_dst, nb_dq);
 
+		port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 		/* Free any unsent packets. */
 		if (unlikely(nb_tx < nb_dq))
 			rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -653,6 +875,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
 
+	ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+		rte_eth_tx_buffer_count_callback,
+		&port_statistics.tx_dropped[portid]);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot set error callback for tx buffer on port %u\n",
+			portid);
+
 	/* Start device */
 	ret = rte_eth_dev_start(portid);
 	if (ret < 0)
@@ -729,6 +959,9 @@ main(int argc, char **argv)
 	RTE_ETH_FOREACH_DEV(portid)
 		port_init(portid, ioat_pktmbuf_pool, nb_queues);
 
+	/* Initialize port xstats */
+	memset(&port_statistics, 0, sizeof(port_statistics));
+
 	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
 		sleep(1);
 
@@ -744,6 +977,8 @@ main(int argc, char **argv)
 		assign_rings();
 
 	start_forwarding_cores();
+	/* master core prints stats while other cores forward */
+	print_stats(argv[0]);
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v6 6/6] doc/guides/: provide IOAT sample app guide
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                             ` (4 preceding siblings ...)
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 5/6] examples/ioat: add stats printing for each port Marcin Baran
@ 2019-09-30  7:50           ` Marcin Baran
  2019-10-03  9:48           ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Bruce Richardson
  2019-10-04 17:16           ` Kevin Traynor
  7 siblings, 0 replies; 76+ messages in thread
From: Marcin Baran @ 2019-09-30  7:50 UTC (permalink / raw)
  To: dev, bruce.richardson; +Cc: Marcin Baran

Added guide for IOAT sample app usage and
code description.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
---
 doc/guides/sample_app_ug/index.rst |   1 +
 doc/guides/sample_app_ug/intro.rst |   4 +
 doc/guides/sample_app_ug/ioat.rst  | 581 +++++++++++++++++++++++++++++
 3 files changed, 586 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst

diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index f23f8f59e..a6a1d9e7a 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -23,6 +23,7 @@ Sample Applications User Guides
     ip_reassembly
     kernel_nic_interface
     keep_alive
+    ioat
     l2_forward_crypto
     l2_forward_job_stats
     l2_forward_real_virtual
diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
index 90704194a..74462312f 100644
--- a/doc/guides/sample_app_ug/intro.rst
+++ b/doc/guides/sample_app_ug/intro.rst
@@ -91,6 +91,10 @@ examples are highlighted below.
   forwarding, or ``l3fwd`` application does forwarding based on Internet
   Protocol, IPv4 or IPv6 like a simple router.
 
+* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
+  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
+  copying packets between two threads.
+
 * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
   demonstrates how to distribute packets arriving on an Rx port to different
   cores for processing and transmission.
diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
new file mode 100644
index 000000000..bab7654b8
--- /dev/null
+++ b/doc/guides/sample_app_ug/ioat.rst
@@ -0,0 +1,581 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Intel Corporation.
+
+.. include:: <isonum.txt>
+
+Packet copying using Intel\ |reg| QuickData Technology
+======================================================
+
+Overview
+--------
+
+This sample is intended as a demonstration of the basic components of a DPDK
+forwarding application and example of how to use IOAT driver API to make
+packets copies.
+
+Also while forwarding, the MAC addresses are affected as follows:
+
+*   The source MAC address is replaced by the TX port MAC address
+
+*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
+
+This application can be used to compare performance of using software packet
+copy with copy done using a DMA device for different sizes of packets.
+The example will print out statistics each second. The stats shows
+received/send packets and packets dropped or failed to copy.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``ioat`` sub-directory.
+
+
+Running the Application
+-----------------------
+
+In order to run the hardware copy application, the copying device
+needs to be bound to user-space IO driver.
+
+Refer to the "IOAT Rawdev Driver" chapter in the "Rawdev Drivers" document
+for information on using the driver.
+
+The application requires a number of command line options:
+
+.. code-block:: console
+
+    ./build/ioatfwd [EAL options] -- [-p MASK] [-q NQ] [-s RS] [-c <sw|hw>]
+        [--[no-]mac-updating]
+
+where,
+
+*   p MASK: A hexadecimal bitmask of the ports to configure (default is all)
+
+*   q NQ: Number of Rx queues used per port equivalent to CBDMA channels
+    per port (default is 1)
+
+*   c CT: Performed packet copy type: software (sw) or hardware using
+    DMA (hw) (default is hw)
+
+*   s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for
+    software copy mode (default is 2048)
+
+*   --[no-]mac-updating: Whether MAC address of packets should be changed
+    or not (default is mac-updating)
+
+The application can be launched in various configurations depending on
+provided parameters. The app can use up to 2 lcores: one of them receives
+incoming traffic and makes a copy of each packet. The second lcore then
+updates MAC address and sends the copy. If one lcore per port is used,
+both operations are done sequentially. For each configuration an additional
+lcore is needed since the master lcore does not handle traffic but is
+responsible for configuration, statistics printing and safe shutdown of
+all ports and devices.
+
+The application can use a maximum of 8 ports.
+
+To run the application in a Linux environment with 3 lcores (the master lcore,
+plus two forwarding cores), a single port (port 0), software copying and MAC
+updating issue the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
+
+To run the application in a Linux environment with 2 lcores (the master lcore,
+plus one forwarding core), 2 ports (ports 0 and 1), hardware copying and no MAC
+updating issue the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw
+
+Refer to the *DPDK Getting Started Guide* for general information on
+running applications and the Environment Abstraction Layer (EAL) options.
+
+Explanation
+-----------
+
+The following sections provide an explanation of the main components of the
+code.
+
+All DPDK library functions used in the sample code are prefixed with
+``rte_`` and are explained in detail in the *DPDK API Documentation*.
+
+
+The Main Function
+~~~~~~~~~~~~~~~~~
+
+The ``main()`` function performs the initialization and calls the execution
+threads for each lcore.
+
+The first task is to initialize the Environment Abstraction Layer (EAL).
+The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
+function. The value returned is the number of parsed arguments:
+
+.. code-block:: c
+
+    /* init EAL */
+    ret = rte_eal_init(argc, argv);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+
+
+The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
+used by the application:
+
+.. code-block:: c
+
+    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
+        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+        MIN_POOL_SIZE);
+
+    /* Create the mbuf pool */
+    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+        rte_socket_id());
+    if (ioat_pktmbuf_pool == NULL)
+        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+Mbufs are the packet buffer structure used by DPDK. They are explained in
+detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
+
+The ``main()`` function also initializes the ports:
+
+.. code-block:: c
+
+    /* Initialise each port */
+    RTE_ETH_FOREACH_DEV(portid) {
+        port_init(portid, ioat_pktmbuf_pool);
+    }
+
+Each port is configured using ``port_init()`` function. The Ethernet
+ports are configured with local settings using the ``rte_eth_dev_configure()``
+function and the ``port_conf`` struct. The RSS is enabled so that
+multiple Rx queues could be used for packet receiving and copying by
+multiple CBDMA channels per port:
+
+.. code-block:: c
+
+    /* configuring port to use RSS for multiple RX queues */
+    static const struct rte_eth_conf port_conf = {
+        .rxmode = {
+            .mq_mode        = ETH_MQ_RX_RSS,
+            .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+        },
+        .rx_adv_conf = {
+            .rss_conf = {
+                .rss_key = NULL,
+                .rss_hf = ETH_RSS_PROTO_MASK,
+            }
+        }
+    };
+
+For this example the ports are set up with the number of Rx queues provided
+with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()``
+and ``rte_eth_tx_queue_setup()`` functions.
+
+The Ethernet port is then started:
+
+.. code-block:: c
+
+    ret = rte_eth_dev_start(portid);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
+            ret, portid);
+
+
+Finally the Rx port is set in promiscuous mode:
+
+.. code-block:: c
+
+    rte_eth_promiscuous_enable(portid);
+
+
+After that each port application assigns resources needed.
+
+.. code-block:: c
+
+    check_link_status(ioat_enabled_port_mask);
+
+    if (!cfg.nb_ports) {
+        rte_exit(EXIT_FAILURE,
+            "All available ports are disabled. Please set portmask.\n");
+    }
+
+    /* Check if there is enough lcores for all ports. */
+    cfg.nb_lcores = rte_lcore_count() - 1;
+    if (cfg.nb_lcores < 1)
+        rte_exit(EXIT_FAILURE,
+            "There should be at least one slave lcore.\n");
+
+    ret = 0;
+
+    if (copy_mode == COPY_MODE_IOAT_NUM) {
+        assign_rawdevs();
+    } else /* copy_mode == COPY_MODE_SW_NUM */ {
+        assign_rings();
+    }
+
+Depending on mode set (whether copy should be done by software or by hardware)
+special structures are assigned to each port. If software copy was chosen,
+application have to assign ring structures for packet exchanging between lcores
+assigned to ports.
+
+.. code-block:: c
+
+    static void
+    assign_rings(void)
+    {
+        uint32_t i;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            char ring_name[20];
+
+            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
+            /* Create ring for inter core communication */
+            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+                    ring_name, ring_size,
+                    rte_socket_id(), RING_F_SP_ENQ);
+
+            if (cfg.ports[i].rx_to_tx_ring == NULL)
+                rte_exit(EXIT_FAILURE, "%s\n",
+                        rte_strerror(rte_errno));
+        }
+    }
+
+
+When using hardware copy each Rx queue of the port is assigned an
+IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API
+functions:
+
+.. code-block:: c
+
+    static void
+    assign_rawdevs(void)
+    {
+        uint16_t nb_rawdev = 0, rdev_id = 0;
+        uint32_t i, j;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+                struct rte_rawdev_info rdev_info = { 0 };
+
+                do {
+                    if (rdev_id == rte_rawdev_count())
+                        goto end;
+                    rte_rawdev_info_get(rdev_id++, &rdev_info);
+                } while (strcmp(rdev_info.driver_name,
+                    IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+                cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+                configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+                ++nb_rawdev;
+            }
+        }
+    end:
+        if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+            rte_exit(EXIT_FAILURE,
+                "Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+                nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+    }
+
+
+The initialization of hardware device is done by ``rte_rawdev_configure()``
+function using ``rte_rawdev_info`` struct. After configuration the device is
+started using ``rte_rawdev_start()`` function. Each of the above operations
+is done in ``configure_rawdev_queue()``.
+
+.. code-block:: c
+
+    static void
+    configure_rawdev_queue(uint32_t dev_id)
+    {
+        struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
+        struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+        if (rte_rawdev_configure(dev_id, &info) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_configure()\n");
+        }
+        if (rte_rawdev_start(dev_id) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_start()\n");
+        }
+    }
+
+If initialization is successful, memory for hardware device
+statistics is allocated.
+
+Finally ``main()`` function starts all packet handling lcores and starts
+printing stats in a loop on the master lcore. The application can be
+interrupted and closed using ``Ctrl-C``. The master lcore waits for
+all slave processes to finish, deallocates resources and exits.
+
+The processing lcores launching function are described below.
+
+The Lcores Launching Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As described above, ``main()`` function invokes ``start_forwarding_cores()``
+function in order to start processing for each lcore:
+
+.. code-block:: c
+
+    static void start_forwarding_cores(void)
+    {
+        uint32_t lcore_id = rte_lcore_id();
+
+        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+                __func__, rte_lcore_id());
+
+        if (cfg.nb_lcores == 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+                NULL, lcore_id);
+        } else if (cfg.nb_lcores > 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+                NULL, lcore_id);
+
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+                lcore_id);
+        }
+    }
+
+The function launches Rx/Tx processing functions on configured lcores
+using ``rte_eal_remote_launch()``. The configured ports, their number
+and number of assigned lcores are stored in user-defined
+``rxtx_transmission_config`` struct:
+
+.. code-block:: c
+
+    struct rxtx_transmission_config {
+        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+        uint16_t nb_ports;
+        uint16_t nb_lcores;
+    };
+
+The structure is initialized in 'main()' function with the values
+corresponding to ports and lcores configuration provided by the user.
+
+The Lcores Processing Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For receiving packets on each port, the ``ioat_rx_port()`` function is used.
+The function receives packets on each configured Rx queue. Depending on the
+mode the user chose, it will enqueue packets to IOAT rawdev channels and
+then invoke copy process (hardware copy), or perform software copy of each
+packet using ``pktmbuf_sw_copy()`` function and enqueue them to an rte_ring:
+
+.. code-block:: c
+
+    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+    static void
+    ioat_rx_port(struct rxtx_port_config *rx_config)
+    {
+        uint32_t nb_rx, nb_enq, i, j;
+        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+        for (i = 0; i < rx_config->nb_queues; i++) {
+
+            nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+                pkts_burst, MAX_PKT_BURST);
+
+            if (nb_rx == 0)
+                continue;
+
+            port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
+            if (copy_mode == COPY_MODE_IOAT_NUM) {
+                /* Perform packet hardware copy */
+                nb_enq = ioat_enqueue_packets(pkts_burst,
+                    nb_rx, rx_config->ioat_ids[i]);
+                if (nb_enq > 0)
+                    rte_ioat_do_copies(rx_config->ioat_ids[i]);
+            } else {
+                /* Perform packet software copy, free source packets */
+                int ret;
+                struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+                ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst_copy, nb_rx);
+
+                if (unlikely(ret < 0))
+                    rte_exit(EXIT_FAILURE,
+                        "Unable to allocate memory.\n");
+
+                for (j = 0; j < nb_rx; j++)
+                    pktmbuf_sw_copy(pkts_burst[j],
+                        pkts_burst_copy[j]);
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst, nb_rx);
+
+                nb_enq = rte_ring_enqueue_burst(
+                    rx_config->rx_to_tx_ring,
+                    (void *)pkts_burst_copy, nb_rx, NULL);
+
+                /* Free any not enqueued packets. */
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&pkts_burst_copy[nb_enq],
+                    nb_rx - nb_enq);
+            }
+
+            port_statistics.copy_dropped[rx_config->rxtx_port] +=
+                (nb_rx - nb_enq);
+        }
+    }
+
+The packets are received in burst mode using ``rte_eth_rx_burst()``
+function. When using hardware copy mode the packets are enqueued in
+copying device's buffer using ``ioat_enqueue_packets()`` which calls
+``rte_ioat_enqueue_copy()``. When all received packets are in the
+buffer the copy operations are started by calling ``rte_ioat_do_copies()``.
+Function ``rte_ioat_enqueue_copy()`` operates on physical address of
+the packet. Structure ``rte_mbuf`` contains only physical address to
+start of the data buffer (``buf_iova``). Thus the address is adjusted
+by ``addr_offset`` value in order to get the address of ``rearm_data``
+member of ``rte_mbuf``. That way both the packet data and metadata can
+be copied in a single operation. This method can be used because the mbufs
+are direct mbufs allocated by the apps. If another app uses external buffers,
+or indirect mbufs, then multiple copy operations must be used.
+
+.. code-block:: c
+
+    static uint32_t
+    ioat_enqueue_packets(struct rte_mbuf **pkts,
+        uint32_t nb_rx, uint16_t dev_id)
+    {
+        int ret;
+        uint32_t i;
+        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+            &pkts[0]->rearm_data);
+
+        ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                (void *)pkts_copy, nb_rx);
+
+        if (unlikely(ret < 0))
+            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+        for (i = 0; i < nb_rx; i++) {
+            /* Perform data copy */
+            ret = rte_ioat_enqueue_copy(dev_id,
+                pkts[i]->buf_iova
+                    - addr_offset,
+                pkts_copy[i]->buf_iova
+                    - addr_offset,
+                rte_pktmbuf_data_len(pkts[i])
+                    + addr_offset,
+                (uintptr_t)pkts[i],
+                (uintptr_t)pkts_copy[i],
+                0 /* nofence */);
+
+            if (ret != 1)
+                break;
+        }
+
+        ret = i;
+        /* Free any not enqueued packets. */
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+            nb_rx - i);
+
+        return ret;
+    }
+
+
+All completed copies are processed by ``ioat_tx_port()`` function. When using
+hardware copy mode the function invokes ``rte_ioat_completed_copies()``
+on each assigned IOAT channel to gather copied packets. If software copy
+mode is used the function dequeues copied packets from the rte_ring. Then each
+packet MAC address is changed if it was enabled. After that copies are sent
+in burst mode using `` rte_eth_tx_burst()``.
+
+
+.. code-block:: c
+
+    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
+    static void
+    ioat_tx_port(struct rxtx_port_config *tx_config)
+    {
+        uint32_t i, j, nb_dq = 0;
+        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
+        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+        for (i = 0; i < tx_config->nb_queues; i++) {
+            if (copy_mode == COPY_MODE_IOAT_NUM) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ioat_completed_copies(
+                    tx_config->ioat_ids[i], MAX_PKT_BURST,
+                    (void *)mbufs_src, (void *)mbufs_dst);
+            } else {
+                /* Deque the mbufs from rx_to_tx_ring. */
+                nb_dq = rte_ring_dequeue_burst(
+                    tx_config->rx_to_tx_ring, (void *)mbufs_dst,
+                    MAX_PKT_BURST, NULL);
+            }
+
+            if (nb_dq == 0)
+                return;
+
+            if (copy_mode == COPY_MODE_IOAT_NUM)
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)mbufs_src, nb_dq);
+
+            /* Update macs if enabled */
+            if (mac_updating) {
+                for (j = 0; j < nb_dq; j++)
+                    update_mac_addrs(mbufs_dst[j],
+                        tx_config->rxtx_port);
+            }
+
+            const uint16_t nb_tx = rte_eth_tx_burst(
+                tx_config->rxtx_port, 0,
+                (void *)mbufs_dst, nb_dq);
+
+            port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+            /* Free any unsent packets. */
+            if (unlikely(nb_tx < nb_dq))
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                (void *)&mbufs_dst[nb_tx],
+                    nb_dq - nb_tx);
+        }
+    }
+
+The Packet Copying Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to perform packet copy there is a user-defined function
+``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
+metadata from source packet to new mbuf, and then copying a data
+chunk of source packet. Both memory copies are done using
+``rte_memcpy()``:
+
+.. code-block:: c
+
+    static inline void
+    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+    {
+        /* Copy packet metadata */
+        rte_memcpy(&dst->rearm_data,
+            &src->rearm_data,
+            offsetof(struct rte_mbuf, cacheline1)
+                - offsetof(struct rte_mbuf, rearm_data));
+
+        /* Copy packet data */
+        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+            rte_pktmbuf_mtod(src, char *), src->data_len);
+    }
+
+The metadata in this example is copied from ``rearm_data`` member of
+``rte_mbuf`` struct up to ``cacheline1``.
+
+In order to understand why software packet copying is done as shown
+above please refer to the "Mbuf Library" section of the
+*DPDK Programmer's Guide*.
-- 
2.22.0.windows.1


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                             ` (5 preceding siblings ...)
  2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
@ 2019-10-03  9:48           ` Bruce Richardson
  2019-10-04 17:16           ` Kevin Traynor
  7 siblings, 0 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-10-03  9:48 UTC (permalink / raw)
  To: Marcin Baran; +Cc: dev

On Mon, Sep 30, 2019 at 09:50:28AM +0200, Marcin Baran wrote:
> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware
> copy, changes packets' MAC addresses (if enabled) and
> forwards them. The patch includes sample application as
> well as it's guide.
> 
> v6:
>  - rearrange 'ioat_tx_port()' to remove unnecessary code
>  - improve documentation
>  - format patches for better readability
> 
> v5:
>  - change dependency name from 'pmd_ioat' to 'rawdev_ioat'
>    fixing build errors (branch diverged from master)
> 
> v4:
>  - fix meson build support check
> 
> v3:
>  - add check for meson build whether IOAT is supported
> 
> v2:
>  - change printing total stats to be deltas
>  - update documentation
>  - fix 1 thread/sw copy/multiple Rx queues packet dropping
>  - divide patch into several presenting functional change
> 
> 
> Marcin Baran (4):
>   examples/ioat: add software copy support
>   examples/ioat: add rawdev copy mode support
>   examples/ioat: add stats printing for each port
>   doc/guides/: provide IOAT sample app guide
> 
> Pawel Modrak (2):
>   examples/ioat: create sample app on ioat driver usage
>   examples/ioat: add two threads configuration
> 
Series-acked-by: Bruce Richardson <bruce.richardson@intel.com>


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                             ` (6 preceding siblings ...)
  2019-10-03  9:48           ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Bruce Richardson
@ 2019-10-04 17:16           ` Kevin Traynor
  2019-10-07 11:10             ` Bruce Richardson
  7 siblings, 1 reply; 76+ messages in thread
From: Kevin Traynor @ 2019-10-04 17:16 UTC (permalink / raw)
  To: Marcin Baran, dev, bruce.richardson

On 30/09/2019 08:50, Marcin Baran wrote:
> A new sample app demonstrating use of driver for CBDMA.
> The app receives packets, performs software or hardware
> copy, changes packets' MAC addresses (if enabled) and
> forwards them. The patch includes sample application as
> well as it's guide.
> 
> v6:
>  - rearrange 'ioat_tx_port()' to remove unnecessary code
>  - improve documentation
>  - format patches for better readability
> 
> v5:
>  - change dependency name from 'pmd_ioat' to 'rawdev_ioat'
>    fixing build errors (branch diverged from master)
> 
> v4:
>  - fix meson build support check
> 
> v3:
>  - add check for meson build whether IOAT is supported
> 
> v2:
>  - change printing total stats to be deltas
>  - update documentation
>  - fix 1 thread/sw copy/multiple Rx queues packet dropping
>  - divide patch into several presenting functional change
> 
> 
> Marcin Baran (4):
>   examples/ioat: add software copy support
>   examples/ioat: add rawdev copy mode support
>   examples/ioat: add stats printing for each port
>   doc/guides/: provide IOAT sample app guide
> 
> Pawel Modrak (2):
>   examples/ioat: create sample app on ioat driver usage
>   examples/ioat: add two threads configuration
> 
>  doc/guides/sample_app_ug/index.rst |    1 +
>  doc/guides/sample_app_ug/intro.rst |    4 +
>  doc/guides/sample_app_ug/ioat.rst  |  581 ++++++++++++++++
>  examples/Makefile                  |    3 +
>  examples/ioat/Makefile             |   54 ++
>  examples/ioat/ioatfwd.c            | 1003 ++++++++++++++++++++++++++++
>  examples/ioat/meson.build          |   15 +
>  examples/meson.build               |    1 +
>  8 files changed, 1662 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/ioat.rst
>  create mode 100644 examples/ioat/Makefile
>  create mode 100644 examples/ioat/ioatfwd.c
>  create mode 100644 examples/ioat/meson.build
> 

Just a nit, but I think you need to add something to the MAINTAINERS
file for this example app, maybe it can be a follow up patch if it's the
only rework

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver
  2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
                           ` (6 preceding siblings ...)
  2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
@ 2019-10-07 11:08         ` Bruce Richardson
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 1/6] examples/ioat: new " Bruce Richardson
                             ` (6 more replies)
  7 siblings, 7 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-10-07 11:08 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak, Bruce Richardson

A new sample app demonstrating use of driver for CBDMA.  The app
receives packets, performs software or hardware copy, changes packets'
MAC addresses (if enabled) and forwards them. The patch includes sample
application as well as it's guide.

v7:
 - MAINTAINERS file entry added

v6:
 - rearrange 'ioat_tx_port()' to remove unnecessary code
 - improve documentation
 - format patches for better readability

v5:
 - change dependency name from 'pmd_ioat' to 'rawdev_ioat'
   fixing build errors (branch diverged from master)

v4:
 - fix meson build support check

v3:
 - add check for meson build whether IOAT is supported

v2:
 - change printing total stats to be deltas
 - update documentation
 - fix 1 thread/sw copy/multiple Rx queues packet dropping
 - divide patch into several presenting functional change



Marcin Baran (4):
  examples/ioat: add software copy support
  examples/ioat: add rawdev copy mode support
  examples/ioat: add stats printing for each port
  doc/guides/: provide IOAT sample app guide

Pawel Modrak (2):
  examples/ioat: create sample app on ioat driver usage
  examples/ioat: add two threads configuration

 MAINTAINERS                        |    1 +
 doc/guides/sample_app_ug/index.rst |    1 +
 doc/guides/sample_app_ug/intro.rst |    4 +
 doc/guides/sample_app_ug/ioat.rst  |  581 ++++++++++++++++
 examples/Makefile                  |    3 +
 examples/ioat/Makefile             |   54 ++
 examples/ioat/ioatfwd.c            | 1003 ++++++++++++++++++++++++++++
 examples/ioat/meson.build          |   15 +
 examples/meson.build               |    1 +
 9 files changed, 1663 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

-- 
2.21.0


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v7 1/6] examples/ioat: new sample app for ioat driver
  2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
@ 2019-10-07 11:08           ` Bruce Richardson
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 2/6] examples/ioat: add software copy support Bruce Richardson
                             ` (5 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-10-07 11:08 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak, Bruce Richardson

From: Pawel Modrak <pawelx.modrak@intel.com>

A new sample app demonstrating use of driver for CBDMA.  The app receives
packets, performs software or hardware copy, changes packets' MAC addresses
(if enabled) and forwards them. The change covers ports initialization,
closing connection and argument parsing.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

---
V6: add maintainers file entry for this new app
---
 MAINTAINERS               |   1 +
 examples/Makefile         |   3 +
 examples/ioat/Makefile    |  54 +++++
 examples/ioat/ioatfwd.c   | 439 ++++++++++++++++++++++++++++++++++++++
 examples/ioat/meson.build |  15 ++
 examples/meson.build      |   1 +
 6 files changed, 513 insertions(+)
 create mode 100644 examples/ioat/Makefile
 create mode 100644 examples/ioat/ioatfwd.c
 create mode 100644 examples/ioat/meson.build

diff --git a/MAINTAINERS b/MAINTAINERS
index 37672b7f6..7e1e6948e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1107,6 +1107,7 @@ IOAT Rawdev
 M: Bruce Richardson <bruce.richardson@intel.com>
 F: drivers/raw/ioat/
 F: doc/guides/rawdevs/ioat.rst
+F: examples/ioat/
 
 NXP DPAA2 QDMA
 M: Nipun Gupta <nipun.gupta@nxp.com>
diff --git a/examples/Makefile b/examples/Makefile
index de11dd487..3cb313d7d 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -23,6 +23,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += fips_validation
 DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += flow_classify
 DIRS-y += flow_filtering
 DIRS-y += helloworld
+ifeq ($(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV),y)
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat
+endif
 DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += ip_pipeline
 ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
 DIRS-$(CONFIG_RTE_IP_FRAG) += ip_reassembly
diff --git a/examples/ioat/Makefile b/examples/ioat/Makefile
new file mode 100644
index 000000000..2a4d1da2d
--- /dev/null
+++ b/examples/ioat/Makefile
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# binary name
+APP = ioatfwd
+
+# all source are stored in SRCS-y
+SRCS-y := ioatfwd.c
+
+# Build using pkg-config variables if possible
+ifeq ($(shell pkg-config --exists libdpdk && echo 0),0)
+
+all: shared
+.PHONY: shared static
+shared: build/$(APP)-shared
+	ln -sf $(APP)-shared build/$(APP)
+static: build/$(APP)-static
+	ln -sf $(APP)-static build/$(APP)
+
+PC_FILE := $(shell pkg-config --path libdpdk)
+CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
+LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
+
+build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
+
+build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
+
+build:
+	@mkdir -p $@
+
+.PHONY: clean
+clean:
+	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
+	test -d build && rmdir -p build || true
+
+else # Build using legacy build system
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, detect a build directory, by looking for a path with a .config
+RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config)))))
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+endif
diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
new file mode 100644
index 000000000..b33c1eeb4
--- /dev/null
+++ b/examples/ioat/ioatfwd.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#include <stdint.h>
+#include <getopt.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+#include <rte_ethdev.h>
+#include <rte_rawdev.h>
+#include <rte_ioat_rawdev.h>
+
+/* size of ring used for software copying between rx and tx. */
+#define RTE_LOGTYPE_IOAT RTE_LOGTYPE_USER1
+#define MAX_PKT_BURST 32
+#define MEMPOOL_CACHE_SIZE 512
+#define MIN_POOL_SIZE 65536U
+#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"
+#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating"
+#define CMD_LINE_OPT_PORTMASK "portmask"
+#define CMD_LINE_OPT_NB_QUEUE "nb-queue"
+#define CMD_LINE_OPT_COPY_TYPE "copy-type"
+#define CMD_LINE_OPT_RING_SIZE "ring-size"
+
+/* configurable number of RX/TX ring descriptors */
+#define RX_DEFAULT_RINGSIZE 1024
+#define TX_DEFAULT_RINGSIZE 1024
+
+/* max number of RX queues per port */
+#define MAX_RX_QUEUES_COUNT 8
+
+struct rxtx_port_config {
+	/* common config */
+	uint16_t rxtx_port;
+	uint16_t nb_queues;
+	/* for software copy mode */
+	struct rte_ring *rx_to_tx_ring;
+	/* for IOAT rawdev copy mode */
+	uint16_t ioat_ids[MAX_RX_QUEUES_COUNT];
+};
+
+struct rxtx_transmission_config {
+	struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+	uint16_t nb_ports;
+	uint16_t nb_lcores;
+};
+
+typedef enum copy_mode_t {
+#define COPY_MODE_SW "sw"
+	COPY_MODE_SW_NUM,
+#define COPY_MODE_IOAT "hw"
+	COPY_MODE_IOAT_NUM,
+	COPY_MODE_INVALID_NUM,
+	COPY_MODE_SIZE_NUM = COPY_MODE_INVALID_NUM
+} copy_mode_t;
+
+/* mask of enabled ports */
+static uint32_t ioat_enabled_port_mask;
+
+/* number of RX queues per port */
+static uint16_t nb_queues = 1;
+
+/* MAC updating enabled by default. */
+static int mac_updating = 1;
+
+/* hardare copy mode enabled by default. */
+static copy_mode_t copy_mode = COPY_MODE_IOAT_NUM;
+
+/* size of IOAT rawdev ring for hardware copy mode or
+ * rte_ring for software copy mode
+ */
+static unsigned short ring_size = 2048;
+
+/* global transmission config */
+struct rxtx_transmission_config cfg;
+
+/* configurable number of RX/TX ring descriptors */
+static uint16_t nb_rxd = RX_DEFAULT_RINGSIZE;
+static uint16_t nb_txd = TX_DEFAULT_RINGSIZE;
+
+static volatile bool force_quit;
+
+/* ethernet addresses of ports */
+static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
+
+static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
+struct rte_mempool *ioat_pktmbuf_pool;
+
+/* Display usage */
+static void
+ioat_usage(const char *prgname)
+{
+	printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
+		"  -p --portmask: hexadecimal bitmask of ports to configure\n"
+		"  -q NQ: number of RX queues per port (default is 1)\n"
+		"  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n"
+		"      When enabled:\n"
+		"       - The source MAC address is replaced by the TX port MAC address\n"
+		"       - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n"
+		"  -c --copy-type CT: type of copy: sw|hw\n"
+		"  -s --ring-size RS: size of IOAT rawdev ring for hardware copy mode or rte_ring for software copy mode\n",
+			prgname);
+}
+
+static int
+ioat_parse_portmask(const char *portmask)
+{
+	char *end = NULL;
+	unsigned long pm;
+
+	/* Parse hexadecimal string */
+	pm = strtoul(portmask, &end, 16);
+	if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
+		return -1;
+
+	return pm;
+}
+
+static copy_mode_t
+ioat_parse_copy_mode(const char *copy_mode)
+{
+	if (strcmp(copy_mode, COPY_MODE_SW) == 0)
+		return COPY_MODE_SW_NUM;
+	else if (strcmp(copy_mode, COPY_MODE_IOAT) == 0)
+		return COPY_MODE_IOAT_NUM;
+
+	return COPY_MODE_INVALID_NUM;
+}
+
+/* Parse the argument given in the command line of the application */
+static int
+ioat_parse_args(int argc, char **argv, unsigned int nb_ports)
+{
+	static const char short_options[] =
+		"p:"  /* portmask */
+		"q:"  /* number of RX queues per port */
+		"c:"  /* copy type (sw|hw) */
+		"s:"  /* ring size */
+		;
+
+	static const struct option lgopts[] = {
+		{CMD_LINE_OPT_MAC_UPDATING, no_argument, &mac_updating, 1},
+		{CMD_LINE_OPT_NO_MAC_UPDATING, no_argument, &mac_updating, 0},
+		{CMD_LINE_OPT_PORTMASK, required_argument, NULL, 'p'},
+		{CMD_LINE_OPT_NB_QUEUE, required_argument, NULL, 'q'},
+		{CMD_LINE_OPT_COPY_TYPE, required_argument, NULL, 'c'},
+		{CMD_LINE_OPT_RING_SIZE, required_argument, NULL, 's'},
+		{NULL, 0, 0, 0}
+	};
+
+	const unsigned int default_port_mask = (1 << nb_ports) - 1;
+	int opt, ret;
+	char **argvopt;
+	int option_index;
+	char *prgname = argv[0];
+
+	ioat_enabled_port_mask = default_port_mask;
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, short_options,
+			lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		/* portmask */
+		case 'p':
+			ioat_enabled_port_mask = ioat_parse_portmask(optarg);
+			if (ioat_enabled_port_mask & ~default_port_mask ||
+					ioat_enabled_port_mask <= 0) {
+				printf("Invalid portmask, %s, suggest 0x%x\n",
+						optarg, default_port_mask);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'q':
+			nb_queues = atoi(optarg);
+			if (nb_queues == 0 || nb_queues > MAX_RX_QUEUES_COUNT) {
+				printf("Invalid RX queues number %s. Max %u\n",
+					optarg, MAX_RX_QUEUES_COUNT);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 'c':
+			copy_mode = ioat_parse_copy_mode(optarg);
+			if (copy_mode == COPY_MODE_INVALID_NUM) {
+				printf("Invalid copy type. Use: sw, hw\n");
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		case 's':
+			ring_size = atoi(optarg);
+			if (ring_size == 0) {
+				printf("Invalid ring size, %s.\n", optarg);
+				ioat_usage(prgname);
+				return -1;
+			}
+			break;
+
+		/* long options */
+		case 0:
+			break;
+
+		default:
+			ioat_usage(prgname);
+			return -1;
+		}
+	}
+
+	printf("MAC updating %s\n", mac_updating ? "enabled" : "disabled");
+	if (optind >= 0)
+		argv[optind - 1] = prgname;
+
+	ret = optind - 1;
+	optind = 1; /* reset getopt lib */
+	return ret;
+}
+
+/* check link status, return true if at least one port is up */
+static int
+check_link_status(uint32_t port_mask)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	int retval = 0;
+
+	printf("\nChecking link status\n");
+	RTE_ETH_FOREACH_DEV(portid) {
+		if ((port_mask & (1 << portid)) == 0)
+			continue;
+
+		memset(&link, 0, sizeof(link));
+		rte_eth_link_get(portid, &link);
+
+		/* Print link status */
+		if (link.link_status) {
+			printf(
+				"Port %d Link Up. Speed %u Mbps - %s\n",
+				portid, link.link_speed,
+				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+				("full-duplex") : ("half-duplex\n"));
+			retval = 1;
+		} else
+			printf("Port %d Link Down\n", portid);
+	}
+	return retval;
+}
+
+/*
+ * Initializes a given port using global settings and with the RX buffers
+ * coming from the mbuf_pool passed as a parameter.
+ */
+static inline void
+port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
+{
+	/* configuring port to use RSS for multiple RX queues */
+	static const struct rte_eth_conf port_conf = {
+		.rxmode = {
+			.mq_mode = ETH_MQ_RX_RSS,
+			.max_rx_pkt_len = RTE_ETHER_MAX_LEN
+		},
+		.rx_adv_conf = {
+			.rss_conf = {
+				.rss_key = NULL,
+				.rss_hf = ETH_RSS_PROTO_MASK,
+			}
+		}
+	};
+
+	struct rte_eth_rxconf rxq_conf;
+	struct rte_eth_txconf txq_conf;
+	struct rte_eth_conf local_port_conf = port_conf;
+	struct rte_eth_dev_info dev_info;
+	int ret, i;
+
+	/* Skip ports that are not enabled */
+	if ((ioat_enabled_port_mask & (1 << portid)) == 0) {
+		printf("Skipping disabled port %u\n", portid);
+		return;
+	}
+
+	/* Init port */
+	printf("Initializing port %u... ", portid);
+	fflush(stdout);
+	rte_eth_dev_info_get(portid, &dev_info);
+	local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
+		dev_info.flow_type_rss_offloads;
+	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MBUF_FAST_FREE)
+		local_port_conf.txmode.offloads |=
+			DEV_TX_OFFLOAD_MBUF_FAST_FREE;
+	ret = rte_eth_dev_configure(portid, nb_queues, 1, &local_port_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Cannot configure device:"
+			" err=%d, port=%u\n", ret, portid);
+
+	ret = rte_eth_dev_adjust_nb_rx_tx_desc(portid, &nb_rxd,
+			&nb_txd);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot adjust number of descriptors: err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_macaddr_get(portid, &ioat_ports_eth_addr[portid]);
+
+	/* Init RX queues */
+	rxq_conf = dev_info.default_rxconf;
+	rxq_conf.offloads = local_port_conf.rxmode.offloads;
+	for (i = 0; i < nb_queues; i++) {
+		ret = rte_eth_rx_queue_setup(portid, i, nb_rxd,
+			rte_eth_dev_socket_id(portid), &rxq_conf,
+			mbuf_pool);
+		if (ret < 0)
+			rte_exit(EXIT_FAILURE,
+				"rte_eth_rx_queue_setup:err=%d,port=%u, queue_id=%u\n",
+				ret, portid, i);
+	}
+
+	/* Init one TX queue on each port */
+	txq_conf = dev_info.default_txconf;
+	txq_conf.offloads = local_port_conf.txmode.offloads;
+	ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
+			rte_eth_dev_socket_id(portid),
+			&txq_conf);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_tx_queue_setup:err=%d,port=%u\n",
+			ret, portid);
+
+	/* Initialize TX buffers */
+	tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
+			RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
+			rte_eth_dev_socket_id(portid));
+	if (tx_buffer[portid] == NULL)
+		rte_exit(EXIT_FAILURE,
+			"Cannot allocate buffer for tx on port %u\n",
+			portid);
+
+	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
+
+	/* Start device */
+	ret = rte_eth_dev_start(portid);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"rte_eth_dev_start:err=%d, port=%u\n",
+			ret, portid);
+
+	rte_eth_promiscuous_enable(portid);
+
+	printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n",
+			portid,
+			ioat_ports_eth_addr[portid].addr_bytes[0],
+			ioat_ports_eth_addr[portid].addr_bytes[1],
+			ioat_ports_eth_addr[portid].addr_bytes[2],
+			ioat_ports_eth_addr[portid].addr_bytes[3],
+			ioat_ports_eth_addr[portid].addr_bytes[4],
+			ioat_ports_eth_addr[portid].addr_bytes[5]);
+
+	cfg.ports[cfg.nb_ports].rxtx_port = portid;
+	cfg.ports[cfg.nb_ports++].nb_queues = nb_queues;
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\n\nSignal %d received, preparing to exit...\n",
+			signum);
+		force_quit = true;
+	}
+}
+
+int
+main(int argc, char **argv)
+{
+	int ret;
+	uint16_t nb_ports, portid;
+	uint32_t i;
+	unsigned int nb_mbufs;
+
+	/* Init EAL */
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+	argc -= ret;
+	argv += ret;
+
+	force_quit = false;
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	nb_ports = rte_eth_dev_count_avail();
+	if (nb_ports == 0)
+		rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
+
+	/* Parse application arguments (after the EAL ones) */
+	ret = ioat_parse_args(argc, argv, nb_ports);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Invalid IOAT arguments\n");
+
+	nb_mbufs = RTE_MAX(nb_ports * (nb_queues * (nb_rxd + nb_txd +
+		4 * MAX_PKT_BURST) + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+		MIN_POOL_SIZE);
+
+	/* Create the mbuf pool */
+	ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+		MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+		rte_socket_id());
+	if (ioat_pktmbuf_pool == NULL)
+		rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+	/* Initialise each port */
+	cfg.nb_ports = 0;
+	RTE_ETH_FOREACH_DEV(portid)
+		port_init(portid, ioat_pktmbuf_pool, nb_queues);
+
+	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
+		sleep(1);
+
+	/* Check if there is enough lcores for all ports. */
+	cfg.nb_lcores = rte_lcore_count() - 1;
+	if (cfg.nb_lcores < 1)
+		rte_exit(EXIT_FAILURE,
+			"There should be at least one slave lcore.\n");
+	for (i = 0; i < cfg.nb_ports; i++) {
+		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
+		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
+		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+	}
+
+	printf("Bye...\n");
+	return 0;
+}
diff --git a/examples/ioat/meson.build b/examples/ioat/meson.build
new file mode 100644
index 000000000..ed8328963
--- /dev/null
+++ b/examples/ioat/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+build = dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV')
+
+deps += ['rawdev_ioat']
+
+sources = files(
+	'ioatfwd.c'
+)
diff --git a/examples/meson.build b/examples/meson.build
index a046b74ad..c2e18b59b 100644
--- a/examples/meson.build
+++ b/examples/meson.build
@@ -16,6 +16,7 @@ all_examples = [
 	'eventdev_pipeline', 'exception_path',
 	'fips_validation', 'flow_classify',
 	'flow_filtering', 'helloworld',
+	'ioat',
 	'ip_fragmentation', 'ip_pipeline',
 	'ip_reassembly', 'ipsec-secgw',
 	'ipv4_multicast', 'kni',
-- 
2.21.0


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v7 2/6] examples/ioat: add software copy support
  2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 1/6] examples/ioat: new " Bruce Richardson
@ 2019-10-07 11:08           ` Bruce Richardson
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 3/6] examples/ioat: add rawdev copy mode support Bruce Richardson
                             ` (4 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-10-07 11:08 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak, Bruce Richardson

From: Marcin Baran <marcinx.baran@intel.com>

Added support for copying packets using software copy mode and MAC address
changing. The copies are processed using one lcore.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/ioat/ioatfwd.c | 177 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 177 insertions(+)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index b33c1eeb4..2a4da948a 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -89,6 +89,154 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+static void
+update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
+{
+	struct rte_ether_hdr *eth;
+	void *tmp;
+
+	eth = rte_pktmbuf_mtod(m, struct rte_ether_hdr *);
+
+	/* 02:00:00:00:00:xx - overwriting 2 bytes of source address but
+	 * it's acceptable cause it gets overwritten by rte_ether_addr_copy
+	 */
+	tmp = &eth->d_addr.addr_bytes[0];
+	*((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dest_portid << 40);
+
+	/* src addr */
+	rte_ether_addr_copy(&ioat_ports_eth_addr[dest_portid], &eth->s_addr);
+}
+
+static inline void
+pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+{
+	/* Copy packet metadata */
+	rte_memcpy(&dst->rearm_data,
+		&src->rearm_data,
+		offsetof(struct rte_mbuf, cacheline1)
+		- offsetof(struct rte_mbuf, rearm_data));
+
+	/* Copy packet data */
+	rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+		rte_pktmbuf_mtod(src, char *), src->data_len);
+}
+
+/* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+static void
+ioat_rx_port(struct rxtx_port_config *rx_config)
+{
+	uint32_t nb_rx, nb_enq, i, j;
+	struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+
+	for (i = 0; i < rx_config->nb_queues; i++) {
+
+		nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+			pkts_burst, MAX_PKT_BURST);
+
+		if (nb_rx == 0)
+			continue;
+
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			/* Perform packet hardware copy */
+		} else {
+			/* Perform packet software copy, free source packets */
+			int ret;
+			struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+			ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst_copy, nb_rx);
+
+			if (unlikely(ret < 0))
+				rte_exit(EXIT_FAILURE,
+					"Unable to allocate memory.\n");
+
+			for (j = 0; j < nb_rx; j++)
+				pktmbuf_sw_copy(pkts_burst[j],
+					pkts_burst_copy[j]);
+
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)pkts_burst, nb_rx);
+
+			nb_enq = rte_ring_enqueue_burst(
+				rx_config->rx_to_tx_ring,
+				(void *)pkts_burst_copy, nb_rx, NULL);
+
+			/* Free any not enqueued packets. */
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)&pkts_burst_copy[nb_enq],
+				nb_rx - nb_enq);
+		}
+	}
+}
+
+/* Transmit packets from IOAT rawdev/rte_ring for one port. */
+static void
+ioat_tx_port(struct rxtx_port_config *tx_config)
+{
+	uint32_t i, j, nb_dq = 0;
+	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+	for (i = 0; i < tx_config->nb_queues; i++) {
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			/* Deque the mbufs from IOAT device. */
+		} else {
+			/* Deque the mbufs from rx_to_tx_ring. */
+			nb_dq = rte_ring_dequeue_burst(
+				tx_config->rx_to_tx_ring, (void *)mbufs_dst,
+				MAX_PKT_BURST, NULL);
+		}
+
+		if (nb_dq == 0)
+			return;
+
+		/* Update macs if enabled */
+		if (mac_updating) {
+			for (j = 0; j < nb_dq; j++)
+				update_mac_addrs(mbufs_dst[j],
+					tx_config->rxtx_port);
+		}
+
+		const uint16_t nb_tx = rte_eth_tx_burst(
+			tx_config->rxtx_port, 0,
+			(void *)mbufs_dst, nb_dq);
+
+		/* Free any unsent packets. */
+		if (unlikely(nb_tx < nb_dq))
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+			(void *)&mbufs_dst[nb_tx],
+				nb_dq - nb_tx);
+	}
+}
+
+/* Main rx and tx loop if only one slave lcore available */
+static void
+rxtx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx and tx loop for copy on"
+		" lcore %u\n", rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++) {
+			ioat_rx_port(&cfg.ports[i]);
+			ioat_tx_port(&cfg.ports[i]);
+		}
+}
+
+static void start_forwarding_cores(void)
+{
+	uint32_t lcore_id = rte_lcore_id();
+
+	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+		__func__, rte_lcore_id());
+
+	lcore_id = rte_get_next_lcore(lcore_id, true, true);
+	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+		NULL, lcore_id);
+}
+
 /* Display usage */
 static void
 ioat_usage(const char *prgname)
@@ -253,6 +401,26 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+assign_rings(void)
+{
+	uint32_t i;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		char ring_name[RTE_RING_NAMESIZE];
+
+		snprintf(ring_name, sizeof(ring_name), "rx_to_tx_ring_%u", i);
+		/* Create ring for inter core communication */
+		cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+			ring_name, ring_size,
+			rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
+
+		if (cfg.ports[i].rx_to_tx_ring == NULL)
+			rte_exit(EXIT_FAILURE, "Ring create failed: %s\n",
+				rte_strerror(rte_errno));
+	}
+}
+
 /*
  * Initializes a given port using global settings and with the RX buffers
  * coming from the mbuf_pool passed as a parameter.
@@ -428,10 +596,19 @@ main(int argc, char **argv)
 	if (cfg.nb_lcores < 1)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
+
+	assign_rings();
+
+	start_forwarding_cores();
+
+	/* force_quit is true when we get here */
+	rte_eal_mp_wait_lcore();
+
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
+		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.21.0


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v7 3/6] examples/ioat: add rawdev copy mode support
  2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 1/6] examples/ioat: new " Bruce Richardson
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 2/6] examples/ioat: add software copy support Bruce Richardson
@ 2019-10-07 11:08           ` Bruce Richardson
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 4/6] examples/ioat: add two threads configuration Bruce Richardson
                             ` (3 subsequent siblings)
  6 siblings, 0 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-10-07 11:08 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak, Bruce Richardson

From: Marcin Baran <marcinx.baran@intel.com>

Added support for copying packets using rawdev device. Each port's Rx queue
is assigned DMA channel for copy.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/ioat/ioatfwd.c | 117 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 115 insertions(+), 2 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 2a4da948a..a38b6700b 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -121,6 +121,50 @@ pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
 		rte_pktmbuf_mtod(src, char *), src->data_len);
 }
 
+static uint32_t
+ioat_enqueue_packets(struct rte_mbuf **pkts,
+	uint32_t nb_rx, uint16_t dev_id)
+{
+	int ret;
+	uint32_t i;
+	struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+	const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+		&pkts[0]->rearm_data);
+
+	ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+		(void *)pkts_copy, nb_rx);
+
+	if (unlikely(ret < 0))
+		rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+	for (i = 0; i < nb_rx; i++) {
+		/* Perform data copy */
+		ret = rte_ioat_enqueue_copy(dev_id,
+			pkts[i]->buf_iova
+			- addr_offset,
+			pkts_copy[i]->buf_iova
+			- addr_offset,
+			rte_pktmbuf_data_len(pkts[i])
+			+ addr_offset,
+			(uintptr_t)pkts[i],
+			(uintptr_t)pkts_copy[i],
+			0 /* nofence */);
+
+		if (ret != 1)
+			break;
+	}
+
+	ret = i;
+	/* Free any not enqueued packets. */
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+		nb_rx - i);
+
+
+	return ret;
+}
+
 /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
 static void
 ioat_rx_port(struct rxtx_port_config *rx_config)
@@ -138,6 +182,10 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Perform packet hardware copy */
+			nb_enq = ioat_enqueue_packets(pkts_burst,
+				nb_rx, rx_config->ioat_ids[i]);
+			if (nb_enq > 0)
+				rte_ioat_do_copies(rx_config->ioat_ids[i]);
 		} else {
 			/* Perform packet software copy, free source packets */
 			int ret;
@@ -174,11 +222,15 @@ static void
 ioat_tx_port(struct rxtx_port_config *tx_config)
 {
 	uint32_t i, j, nb_dq = 0;
+	struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
 	struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
 
 	for (i = 0; i < tx_config->nb_queues; i++) {
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Deque the mbufs from IOAT device. */
+			nb_dq = rte_ioat_completed_copies(
+				tx_config->ioat_ids[i], MAX_PKT_BURST,
+				(void *)mbufs_src, (void *)mbufs_dst);
 		} else {
 			/* Deque the mbufs from rx_to_tx_ring. */
 			nb_dq = rte_ring_dequeue_burst(
@@ -189,6 +241,10 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 		if (nb_dq == 0)
 			return;
 
+		if (copy_mode == COPY_MODE_IOAT_NUM)
+			rte_mempool_put_bulk(ioat_pktmbuf_pool,
+				(void *)mbufs_src, nb_dq);
+
 		/* Update macs if enabled */
 		if (mac_updating) {
 			for (j = 0; j < nb_dq; j++)
@@ -401,6 +457,52 @@ check_link_status(uint32_t port_mask)
 	return retval;
 }
 
+static void
+configure_rawdev_queue(uint32_t dev_id)
+{
+	struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
+	struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+	if (rte_rawdev_configure(dev_id, &info) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_configure()\n");
+	}
+	if (rte_rawdev_start(dev_id) != 0) {
+		rte_exit(EXIT_FAILURE,
+			"Error with rte_rawdev_start()\n");
+	}
+}
+
+static void
+assign_rawdevs(void)
+{
+	uint16_t nb_rawdev = 0, rdev_id = 0;
+	uint32_t i, j;
+
+	for (i = 0; i < cfg.nb_ports; i++) {
+		for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+			struct rte_rawdev_info rdev_info = { 0 };
+
+			do {
+				if (rdev_id == rte_rawdev_count())
+					goto end;
+				rte_rawdev_info_get(rdev_id++, &rdev_info);
+			} while (strcmp(rdev_info.driver_name,
+				IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+			cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+			configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+			++nb_rawdev;
+		}
+	}
+end:
+	if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+		rte_exit(EXIT_FAILURE,
+			"Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+			nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+	RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+}
+
 static void
 assign_rings(void)
 {
@@ -597,18 +699,29 @@ main(int argc, char **argv)
 		rte_exit(EXIT_FAILURE,
 			"There should be at least one slave lcore.\n");
 
-	assign_rings();
+	if (copy_mode == COPY_MODE_IOAT_NUM)
+		assign_rawdevs();
+	else /* copy_mode == COPY_MODE_SW_NUM */
+		assign_rings();
 
 	start_forwarding_cores();
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
 
+	uint32_t j;
 	for (i = 0; i < cfg.nb_ports; i++) {
 		printf("Closing port %d\n", cfg.ports[i].rxtx_port);
 		rte_eth_dev_stop(cfg.ports[i].rxtx_port);
 		rte_eth_dev_close(cfg.ports[i].rxtx_port);
-		rte_ring_free(cfg.ports[i].rx_to_tx_ring);
+		if (copy_mode == COPY_MODE_IOAT_NUM) {
+			for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+				printf("Stopping rawdev %d\n",
+					cfg.ports[i].ioat_ids[j]);
+				rte_rawdev_stop(cfg.ports[i].ioat_ids[j]);
+			}
+		} else /* copy_mode == COPY_MODE_SW_NUM */
+			rte_ring_free(cfg.ports[i].rx_to_tx_ring);
 	}
 
 	printf("Bye...\n");
-- 
2.21.0


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v7 4/6] examples/ioat: add two threads configuration
  2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
                             ` (2 preceding siblings ...)
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 3/6] examples/ioat: add rawdev copy mode support Bruce Richardson
@ 2019-10-07 11:08           ` Bruce Richardson
  2019-10-27 17:04             ` Thomas Monjalon
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 5/6] examples/ioat: add stats printing for each port Bruce Richardson
                             ` (2 subsequent siblings)
  6 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-10-07 11:08 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak, Bruce Richardson

From: Pawel Modrak <pawelx.modrak@intel.com>

Added possibility to use two lcores: first for packet receiving and
copying, second for packets sending.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/ioat/ioatfwd.c | 47 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 4 deletions(-)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index a38b6700b..2fc0b1173 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,
 	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
 		nb_rx - i);
 
-
 	return ret;
 }
 
@@ -264,6 +263,36 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 	}
 }
 
+/* Main rx processing loop for IOAT rawdev. */
+static void
+rx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main rx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_rx_port(&cfg.ports[i]);
+}
+
+/* Main tx processing loop for hardware copy. */
+static void
+tx_main_loop(void)
+{
+	uint16_t i;
+	uint16_t nb_ports = cfg.nb_ports;
+
+	RTE_LOG(INFO, IOAT, "Entering main tx loop for copy on lcore %u\n",
+		rte_lcore_id());
+
+	while (!force_quit)
+		for (i = 0; i < nb_ports; i++)
+			ioat_tx_port(&cfg.ports[i]);
+}
+
 /* Main rx and tx loop if only one slave lcore available */
 static void
 rxtx_main_loop(void)
@@ -288,9 +317,19 @@ static void start_forwarding_cores(void)
 	RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
 		__func__, rte_lcore_id());
 
-	lcore_id = rte_get_next_lcore(lcore_id, true, true);
-	rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
-		NULL, lcore_id);
+	if (cfg.nb_lcores == 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+			NULL, lcore_id);
+	} else if (cfg.nb_lcores > 1) {
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+			NULL, lcore_id);
+
+		lcore_id = rte_get_next_lcore(lcore_id, true, true);
+		rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+			lcore_id);
+	}
 }
 
 /* Display usage */
-- 
2.21.0


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v7 5/6] examples/ioat: add stats printing for each port
  2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
                             ` (3 preceding siblings ...)
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 4/6] examples/ioat: add two threads configuration Bruce Richardson
@ 2019-10-07 11:08           ` Bruce Richardson
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 6/6] doc/guides/: provide IOAT sample app guide Bruce Richardson
  2019-10-27 17:07           ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Thomas Monjalon
  6 siblings, 0 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-10-07 11:08 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak, Bruce Richardson

From: Marcin Baran <marcinx.baran@intel.com>

Added printing stats of ports each second.  The stats printing is done
using master core.  The information provided informs about packets
received, dropped and send as well as statistics of rawdev devices.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/ioat/ioatfwd.c | 235 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 235 insertions(+)

diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c
index 2fc0b1173..5e623a4d5 100644
--- a/examples/ioat/ioatfwd.c
+++ b/examples/ioat/ioatfwd.c
@@ -48,6 +48,23 @@ struct rxtx_transmission_config {
 	uint16_t nb_lcores;
 };
 
+/* per-port statistics struct */
+struct ioat_port_statistics {
+	uint64_t rx[RTE_MAX_ETHPORTS];
+	uint64_t tx[RTE_MAX_ETHPORTS];
+	uint64_t tx_dropped[RTE_MAX_ETHPORTS];
+	uint64_t copy_dropped[RTE_MAX_ETHPORTS];
+};
+struct ioat_port_statistics port_statistics;
+
+struct total_statistics {
+	uint64_t total_packets_dropped;
+	uint64_t total_packets_tx;
+	uint64_t total_packets_rx;
+	uint64_t total_successful_enqueues;
+	uint64_t total_failed_enqueues;
+};
+
 typedef enum copy_mode_t {
 #define COPY_MODE_SW "sw"
 	COPY_MODE_SW_NUM,
@@ -89,6 +106,204 @@ static struct rte_ether_addr ioat_ports_eth_addr[RTE_MAX_ETHPORTS];
 static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
 struct rte_mempool *ioat_pktmbuf_pool;
 
+/* Print out statistics for one port. */
+static void
+print_port_stats(uint16_t port_id)
+{
+	printf("\nStatistics for port %u ------------------------------"
+		"\nPackets sent: %34"PRIu64
+		"\nPackets received: %30"PRIu64
+		"\nPackets dropped on tx: %25"PRIu64
+		"\nPackets dropped on copy: %23"PRIu64,
+		port_id,
+		port_statistics.tx[port_id],
+		port_statistics.rx[port_id],
+		port_statistics.tx_dropped[port_id],
+		port_statistics.copy_dropped[port_id]);
+}
+
+/* Print out statistics for one IOAT rawdev device. */
+static void
+print_rawdev_stats(uint32_t dev_id, uint64_t *xstats,
+	unsigned int *ids_xstats, uint16_t nb_xstats,
+	struct rte_rawdev_xstats_name *names_xstats)
+{
+	uint16_t i;
+
+	printf("\nIOAT channel %u", dev_id);
+	for (i = 0; i < nb_xstats; i++)
+		printf("\n\t %s: %*"PRIu64,
+			names_xstats[ids_xstats[i]].name,
+			(int)(37 - strlen(names_xstats[ids_xstats[i]].name)),
+			xstats[i]);
+}
+
+static void
+print_total_stats(struct total_statistics *ts)
+{
+	printf("\nAggregate statistics ==============================="
+		"\nTotal packets Tx: %24"PRIu64" [pps]"
+		"\nTotal packets Rx: %24"PRIu64" [pps]"
+		"\nTotal packets dropped: %19"PRIu64" [pps]",
+		ts->total_packets_tx,
+		ts->total_packets_rx,
+		ts->total_packets_dropped);
+
+	if (copy_mode == COPY_MODE_IOAT_NUM) {
+		printf("\nTotal IOAT successful enqueues: %8"PRIu64" [enq/s]"
+			"\nTotal IOAT failed enqueues: %12"PRIu64" [enq/s]",
+			ts->total_successful_enqueues,
+			ts->total_failed_enqueues);
+	}
+
+	printf("\n====================================================\n");
+}
+
+/* Print out statistics on packets dropped. */
+static void
+print_stats(char *prgname)
+{
+	struct total_statistics ts, delta_ts;
+	uint32_t i, port_id, dev_id;
+	struct rte_rawdev_xstats_name *names_xstats;
+	uint64_t *xstats;
+	unsigned int *ids_xstats, nb_xstats;
+	char status_string[120]; /* to print at the top of the output */
+	int status_strlen;
+
+
+	const char clr[] = { 27, '[', '2', 'J', '\0' };
+	const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
+
+	status_strlen = snprintf(status_string, sizeof(status_string),
+		"%s, ", prgname);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Worker Threads = %d, ",
+		rte_lcore_count() > 2 ? 2 : 1);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Copy Mode = %s,\n", copy_mode == COPY_MODE_SW_NUM ?
+		COPY_MODE_SW : COPY_MODE_IOAT);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Updating MAC = %s, ", mac_updating ?
+		"enabled" : "disabled");
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Rx Queues = %d, ", nb_queues);
+	status_strlen += snprintf(status_string + status_strlen,
+		sizeof(status_string) - status_strlen,
+		"Ring Size = %d\n", ring_size);
+
+	/* Allocate memory for xstats names and values */
+	nb_xstats = rte_rawdev_xstats_names_get(
+		cfg.ports[0].ioat_ids[0], NULL, 0);
+
+	names_xstats = malloc(sizeof(*names_xstats) * nb_xstats);
+	if (names_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat names memory\n");
+	}
+	rte_rawdev_xstats_names_get(cfg.ports[0].ioat_ids[0],
+		names_xstats, nb_xstats);
+
+	ids_xstats = malloc(sizeof(*ids_xstats) * 2);
+	if (ids_xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat ids_xstats memory\n");
+	}
+
+	xstats = malloc(sizeof(*xstats) * 2);
+	if (xstats == NULL) {
+		rte_exit(EXIT_FAILURE,
+			"Error allocating xstat memory\n");
+	}
+
+	/* Get failed/successful enqueues stats index */
+	ids_xstats[0] = ids_xstats[1] = nb_xstats;
+	for (i = 0; i < nb_xstats; i++) {
+		if (!strcmp(names_xstats[i].name, "failed_enqueues"))
+			ids_xstats[0] = i;
+		else if (!strcmp(names_xstats[i].name, "successful_enqueues"))
+			ids_xstats[1] = i;
+		if (ids_xstats[0] < nb_xstats && ids_xstats[1] < nb_xstats)
+			break;
+	}
+	if (ids_xstats[0] == nb_xstats || ids_xstats[1] == nb_xstats) {
+		rte_exit(EXIT_FAILURE,
+			"Error getting failed/successful enqueues stats index\n");
+	}
+
+	memset(&ts, 0, sizeof(struct total_statistics));
+
+	while (!force_quit) {
+		/* Sleep for 1 second each round - init sleep allows reading
+		 * messages from app startup.
+		 */
+		sleep(1);
+
+		/* Clear screen and move to top left */
+		printf("%s%s", clr, topLeft);
+
+		memset(&delta_ts, 0, sizeof(struct total_statistics));
+
+		printf("%s", status_string);
+
+		for (i = 0; i < cfg.nb_ports; i++) {
+			port_id = cfg.ports[i].rxtx_port;
+			print_port_stats(port_id);
+
+			delta_ts.total_packets_dropped +=
+				port_statistics.tx_dropped[port_id]
+				+ port_statistics.copy_dropped[port_id];
+			delta_ts.total_packets_tx +=
+				port_statistics.tx[port_id];
+			delta_ts.total_packets_rx +=
+				port_statistics.rx[port_id];
+
+			if (copy_mode == COPY_MODE_IOAT_NUM) {
+				uint32_t j;
+
+				for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+					dev_id = cfg.ports[i].ioat_ids[j];
+					rte_rawdev_xstats_get(dev_id,
+						ids_xstats, xstats, 2);
+
+					print_rawdev_stats(dev_id, xstats,
+						ids_xstats, 2, names_xstats);
+
+					delta_ts.total_failed_enqueues +=
+						xstats[ids_xstats[0]];
+					delta_ts.total_successful_enqueues +=
+						xstats[ids_xstats[1]];
+				}
+			}
+		}
+
+		delta_ts.total_packets_tx -= ts.total_packets_tx;
+		delta_ts.total_packets_rx -= ts.total_packets_rx;
+		delta_ts.total_packets_dropped -= ts.total_packets_dropped;
+		delta_ts.total_failed_enqueues -= ts.total_failed_enqueues;
+		delta_ts.total_successful_enqueues -=
+			ts.total_successful_enqueues;
+
+		printf("\n");
+		print_total_stats(&delta_ts);
+
+		ts.total_packets_tx += delta_ts.total_packets_tx;
+		ts.total_packets_rx += delta_ts.total_packets_rx;
+		ts.total_packets_dropped += delta_ts.total_packets_dropped;
+		ts.total_failed_enqueues += delta_ts.total_failed_enqueues;
+		ts.total_successful_enqueues +=
+			delta_ts.total_successful_enqueues;
+	}
+
+	free(names_xstats);
+	free(xstats);
+	free(ids_xstats);
+}
+
 static void
 update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid)
 {
@@ -179,6 +394,8 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 		if (nb_rx == 0)
 			continue;
 
+		port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
 		if (copy_mode == COPY_MODE_IOAT_NUM) {
 			/* Perform packet hardware copy */
 			nb_enq = ioat_enqueue_packets(pkts_burst,
@@ -213,6 +430,9 @@ ioat_rx_port(struct rxtx_port_config *rx_config)
 				(void *)&pkts_burst_copy[nb_enq],
 				nb_rx - nb_enq);
 		}
+
+		port_statistics.copy_dropped[rx_config->rxtx_port] +=
+			(nb_rx - nb_enq);
 	}
 }
 
@@ -255,6 +475,8 @@ ioat_tx_port(struct rxtx_port_config *tx_config)
 			tx_config->rxtx_port, 0,
 			(void *)mbufs_dst, nb_dq);
 
+		port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
 		/* Free any unsent packets. */
 		if (unlikely(nb_tx < nb_dq))
 			rte_mempool_put_bulk(ioat_pktmbuf_pool,
@@ -653,6 +875,14 @@ port_init(uint16_t portid, struct rte_mempool *mbuf_pool, uint16_t nb_queues)
 
 	rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
 
+	ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
+		rte_eth_tx_buffer_count_callback,
+		&port_statistics.tx_dropped[portid]);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Cannot set error callback for tx buffer on port %u\n",
+			portid);
+
 	/* Start device */
 	ret = rte_eth_dev_start(portid);
 	if (ret < 0)
@@ -729,6 +959,9 @@ main(int argc, char **argv)
 	RTE_ETH_FOREACH_DEV(portid)
 		port_init(portid, ioat_pktmbuf_pool, nb_queues);
 
+	/* Initialize port xstats */
+	memset(&port_statistics, 0, sizeof(port_statistics));
+
 	while (!check_link_status(ioat_enabled_port_mask) && !force_quit)
 		sleep(1);
 
@@ -744,6 +977,8 @@ main(int argc, char **argv)
 		assign_rings();
 
 	start_forwarding_cores();
+	/* master core prints stats while other cores forward */
+	print_stats(argv[0]);
 
 	/* force_quit is true when we get here */
 	rte_eal_mp_wait_lcore();
-- 
2.21.0


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [dpdk-dev] [PATCH v7 6/6] doc/guides/: provide IOAT sample app guide
  2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
                             ` (4 preceding siblings ...)
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 5/6] examples/ioat: add stats printing for each port Bruce Richardson
@ 2019-10-07 11:08           ` Bruce Richardson
  2019-10-27 17:07             ` Thomas Monjalon
  2019-10-27 17:07           ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Thomas Monjalon
  6 siblings, 1 reply; 76+ messages in thread
From: Bruce Richardson @ 2019-10-07 11:08 UTC (permalink / raw)
  To: dev; +Cc: Marcin Baran, Pawel Modrak, Bruce Richardson

From: Marcin Baran <marcinx.baran@intel.com>

Added guide for IOAT sample app usage and code description.

Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/guides/sample_app_ug/index.rst |   1 +
 doc/guides/sample_app_ug/intro.rst |   4 +
 doc/guides/sample_app_ug/ioat.rst  | 581 +++++++++++++++++++++++++++++
 3 files changed, 586 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/ioat.rst

diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index f23f8f59e..a6a1d9e7a 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -23,6 +23,7 @@ Sample Applications User Guides
     ip_reassembly
     kernel_nic_interface
     keep_alive
+    ioat
     l2_forward_crypto
     l2_forward_job_stats
     l2_forward_real_virtual
diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
index 90704194a..74462312f 100644
--- a/doc/guides/sample_app_ug/intro.rst
+++ b/doc/guides/sample_app_ug/intro.rst
@@ -91,6 +91,10 @@ examples are highlighted below.
   forwarding, or ``l3fwd`` application does forwarding based on Internet
   Protocol, IPv4 or IPv6 like a simple router.
 
+* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
+  or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
+  copying packets between two threads.
+
 * :doc:`Packet Distributor<dist_app>`: The Packet Distributor
   demonstrates how to distribute packets arriving on an Rx port to different
   cores for processing and transmission.
diff --git a/doc/guides/sample_app_ug/ioat.rst b/doc/guides/sample_app_ug/ioat.rst
new file mode 100644
index 000000000..bab7654b8
--- /dev/null
+++ b/doc/guides/sample_app_ug/ioat.rst
@@ -0,0 +1,581 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Intel Corporation.
+
+.. include:: <isonum.txt>
+
+Packet copying using Intel\ |reg| QuickData Technology
+======================================================
+
+Overview
+--------
+
+This sample is intended as a demonstration of the basic components of a DPDK
+forwarding application and example of how to use IOAT driver API to make
+packets copies.
+
+Also while forwarding, the MAC addresses are affected as follows:
+
+*   The source MAC address is replaced by the TX port MAC address
+
+*   The destination MAC address is replaced by  02:00:00:00:00:TX_PORT_ID
+
+This application can be used to compare performance of using software packet
+copy with copy done using a DMA device for different sizes of packets.
+The example will print out statistics each second. The stats shows
+received/send packets and packets dropped or failed to copy.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``ioat`` sub-directory.
+
+
+Running the Application
+-----------------------
+
+In order to run the hardware copy application, the copying device
+needs to be bound to user-space IO driver.
+
+Refer to the "IOAT Rawdev Driver" chapter in the "Rawdev Drivers" document
+for information on using the driver.
+
+The application requires a number of command line options:
+
+.. code-block:: console
+
+    ./build/ioatfwd [EAL options] -- [-p MASK] [-q NQ] [-s RS] [-c <sw|hw>]
+        [--[no-]mac-updating]
+
+where,
+
+*   p MASK: A hexadecimal bitmask of the ports to configure (default is all)
+
+*   q NQ: Number of Rx queues used per port equivalent to CBDMA channels
+    per port (default is 1)
+
+*   c CT: Performed packet copy type: software (sw) or hardware using
+    DMA (hw) (default is hw)
+
+*   s RS: Size of IOAT rawdev ring for hardware copy mode or rte_ring for
+    software copy mode (default is 2048)
+
+*   --[no-]mac-updating: Whether MAC address of packets should be changed
+    or not (default is mac-updating)
+
+The application can be launched in various configurations depending on
+provided parameters. The app can use up to 2 lcores: one of them receives
+incoming traffic and makes a copy of each packet. The second lcore then
+updates MAC address and sends the copy. If one lcore per port is used,
+both operations are done sequentially. For each configuration an additional
+lcore is needed since the master lcore does not handle traffic but is
+responsible for configuration, statistics printing and safe shutdown of
+all ports and devices.
+
+The application can use a maximum of 8 ports.
+
+To run the application in a Linux environment with 3 lcores (the master lcore,
+plus two forwarding cores), a single port (port 0), software copying and MAC
+updating issue the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-2 -n 2 -- -p 0x1 --mac-updating -c sw
+
+To run the application in a Linux environment with 2 lcores (the master lcore,
+plus one forwarding core), 2 ports (ports 0 and 1), hardware copying and no MAC
+updating issue the command:
+
+.. code-block:: console
+
+    $ ./build/ioatfwd -l 0-1 -n 1 -- -p 0x3 --no-mac-updating -c hw
+
+Refer to the *DPDK Getting Started Guide* for general information on
+running applications and the Environment Abstraction Layer (EAL) options.
+
+Explanation
+-----------
+
+The following sections provide an explanation of the main components of the
+code.
+
+All DPDK library functions used in the sample code are prefixed with
+``rte_`` and are explained in detail in the *DPDK API Documentation*.
+
+
+The Main Function
+~~~~~~~~~~~~~~~~~
+
+The ``main()`` function performs the initialization and calls the execution
+threads for each lcore.
+
+The first task is to initialize the Environment Abstraction Layer (EAL).
+The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()``
+function. The value returned is the number of parsed arguments:
+
+.. code-block:: c
+
+    /* init EAL */
+    ret = rte_eal_init(argc, argv);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
+
+
+The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers)
+used by the application:
+
+.. code-block:: c
+
+    nb_mbufs = RTE_MAX(rte_eth_dev_count_avail() * (nb_rxd + nb_txd
+        + MAX_PKT_BURST + rte_lcore_count() * MEMPOOL_CACHE_SIZE),
+        MIN_POOL_SIZE);
+
+    /* Create the mbuf pool */
+    ioat_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", nb_mbufs,
+        MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE,
+        rte_socket_id());
+    if (ioat_pktmbuf_pool == NULL)
+        rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
+
+Mbufs are the packet buffer structure used by DPDK. They are explained in
+detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*.
+
+The ``main()`` function also initializes the ports:
+
+.. code-block:: c
+
+    /* Initialise each port */
+    RTE_ETH_FOREACH_DEV(portid) {
+        port_init(portid, ioat_pktmbuf_pool);
+    }
+
+Each port is configured using ``port_init()`` function. The Ethernet
+ports are configured with local settings using the ``rte_eth_dev_configure()``
+function and the ``port_conf`` struct. The RSS is enabled so that
+multiple Rx queues could be used for packet receiving and copying by
+multiple CBDMA channels per port:
+
+.. code-block:: c
+
+    /* configuring port to use RSS for multiple RX queues */
+    static const struct rte_eth_conf port_conf = {
+        .rxmode = {
+            .mq_mode        = ETH_MQ_RX_RSS,
+            .max_rx_pkt_len = RTE_ETHER_MAX_LEN
+        },
+        .rx_adv_conf = {
+            .rss_conf = {
+                .rss_key = NULL,
+                .rss_hf = ETH_RSS_PROTO_MASK,
+            }
+        }
+    };
+
+For this example the ports are set up with the number of Rx queues provided
+with -q option and 1 Tx queue using the ``rte_eth_rx_queue_setup()``
+and ``rte_eth_tx_queue_setup()`` functions.
+
+The Ethernet port is then started:
+
+.. code-block:: c
+
+    ret = rte_eth_dev_start(portid);
+    if (ret < 0)
+        rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err=%d, port=%u\n",
+            ret, portid);
+
+
+Finally the Rx port is set in promiscuous mode:
+
+.. code-block:: c
+
+    rte_eth_promiscuous_enable(portid);
+
+
+After that each port application assigns resources needed.
+
+.. code-block:: c
+
+    check_link_status(ioat_enabled_port_mask);
+
+    if (!cfg.nb_ports) {
+        rte_exit(EXIT_FAILURE,
+            "All available ports are disabled. Please set portmask.\n");
+    }
+
+    /* Check if there is enough lcores for all ports. */
+    cfg.nb_lcores = rte_lcore_count() - 1;
+    if (cfg.nb_lcores < 1)
+        rte_exit(EXIT_FAILURE,
+            "There should be at least one slave lcore.\n");
+
+    ret = 0;
+
+    if (copy_mode == COPY_MODE_IOAT_NUM) {
+        assign_rawdevs();
+    } else /* copy_mode == COPY_MODE_SW_NUM */ {
+        assign_rings();
+    }
+
+Depending on mode set (whether copy should be done by software or by hardware)
+special structures are assigned to each port. If software copy was chosen,
+application have to assign ring structures for packet exchanging between lcores
+assigned to ports.
+
+.. code-block:: c
+
+    static void
+    assign_rings(void)
+    {
+        uint32_t i;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            char ring_name[20];
+
+            snprintf(ring_name, 20, "rx_to_tx_ring_%u", i);
+            /* Create ring for inter core communication */
+            cfg.ports[i].rx_to_tx_ring = rte_ring_create(
+                    ring_name, ring_size,
+                    rte_socket_id(), RING_F_SP_ENQ);
+
+            if (cfg.ports[i].rx_to_tx_ring == NULL)
+                rte_exit(EXIT_FAILURE, "%s\n",
+                        rte_strerror(rte_errno));
+        }
+    }
+
+
+When using hardware copy each Rx queue of the port is assigned an
+IOAT device (``assign_rawdevs()``) using IOAT Rawdev Driver API
+functions:
+
+.. code-block:: c
+
+    static void
+    assign_rawdevs(void)
+    {
+        uint16_t nb_rawdev = 0, rdev_id = 0;
+        uint32_t i, j;
+
+        for (i = 0; i < cfg.nb_ports; i++) {
+            for (j = 0; j < cfg.ports[i].nb_queues; j++) {
+                struct rte_rawdev_info rdev_info = { 0 };
+
+                do {
+                    if (rdev_id == rte_rawdev_count())
+                        goto end;
+                    rte_rawdev_info_get(rdev_id++, &rdev_info);
+                } while (strcmp(rdev_info.driver_name,
+                    IOAT_PMD_RAWDEV_NAME_STR) != 0);
+
+                cfg.ports[i].ioat_ids[j] = rdev_id - 1;
+                configure_rawdev_queue(cfg.ports[i].ioat_ids[j]);
+                ++nb_rawdev;
+            }
+        }
+    end:
+        if (nb_rawdev < cfg.nb_ports * cfg.ports[0].nb_queues)
+            rte_exit(EXIT_FAILURE,
+                "Not enough IOAT rawdevs (%u) for all queues (%u).\n",
+                nb_rawdev, cfg.nb_ports * cfg.ports[0].nb_queues);
+        RTE_LOG(INFO, IOAT, "Number of used rawdevs: %u.\n", nb_rawdev);
+    }
+
+
+The initialization of hardware device is done by ``rte_rawdev_configure()``
+function using ``rte_rawdev_info`` struct. After configuration the device is
+started using ``rte_rawdev_start()`` function. Each of the above operations
+is done in ``configure_rawdev_queue()``.
+
+.. code-block:: c
+
+    static void
+    configure_rawdev_queue(uint32_t dev_id)
+    {
+        struct rte_ioat_rawdev_config dev_config = { .ring_size = ring_size };
+        struct rte_rawdev_info info = { .dev_private = &dev_config };
+
+        if (rte_rawdev_configure(dev_id, &info) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_configure()\n");
+        }
+        if (rte_rawdev_start(dev_id) != 0) {
+            rte_exit(EXIT_FAILURE,
+                "Error with rte_rawdev_start()\n");
+        }
+    }
+
+If initialization is successful, memory for hardware device
+statistics is allocated.
+
+Finally ``main()`` function starts all packet handling lcores and starts
+printing stats in a loop on the master lcore. The application can be
+interrupted and closed using ``Ctrl-C``. The master lcore waits for
+all slave processes to finish, deallocates resources and exits.
+
+The processing lcores launching function are described below.
+
+The Lcores Launching Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As described above, ``main()`` function invokes ``start_forwarding_cores()``
+function in order to start processing for each lcore:
+
+.. code-block:: c
+
+    static void start_forwarding_cores(void)
+    {
+        uint32_t lcore_id = rte_lcore_id();
+
+        RTE_LOG(INFO, IOAT, "Entering %s on lcore %u\n",
+                __func__, rte_lcore_id());
+
+        if (cfg.nb_lcores == 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rxtx_main_loop,
+                NULL, lcore_id);
+        } else if (cfg.nb_lcores > 1) {
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)rx_main_loop,
+                NULL, lcore_id);
+
+            lcore_id = rte_get_next_lcore(lcore_id, true, true);
+            rte_eal_remote_launch((lcore_function_t *)tx_main_loop, NULL,
+                lcore_id);
+        }
+    }
+
+The function launches Rx/Tx processing functions on configured lcores
+using ``rte_eal_remote_launch()``. The configured ports, their number
+and number of assigned lcores are stored in user-defined
+``rxtx_transmission_config`` struct:
+
+.. code-block:: c
+
+    struct rxtx_transmission_config {
+        struct rxtx_port_config ports[RTE_MAX_ETHPORTS];
+        uint16_t nb_ports;
+        uint16_t nb_lcores;
+    };
+
+The structure is initialized in 'main()' function with the values
+corresponding to ports and lcores configuration provided by the user.
+
+The Lcores Processing Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For receiving packets on each port, the ``ioat_rx_port()`` function is used.
+The function receives packets on each configured Rx queue. Depending on the
+mode the user chose, it will enqueue packets to IOAT rawdev channels and
+then invoke copy process (hardware copy), or perform software copy of each
+packet using ``pktmbuf_sw_copy()`` function and enqueue them to an rte_ring:
+
+.. code-block:: c
+
+    /* Receive packets on one port and enqueue to IOAT rawdev or rte_ring. */
+    static void
+    ioat_rx_port(struct rxtx_port_config *rx_config)
+    {
+        uint32_t nb_rx, nb_enq, i, j;
+        struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
+        for (i = 0; i < rx_config->nb_queues; i++) {
+
+            nb_rx = rte_eth_rx_burst(rx_config->rxtx_port, i,
+                pkts_burst, MAX_PKT_BURST);
+
+            if (nb_rx == 0)
+                continue;
+
+            port_statistics.rx[rx_config->rxtx_port] += nb_rx;
+
+            if (copy_mode == COPY_MODE_IOAT_NUM) {
+                /* Perform packet hardware copy */
+                nb_enq = ioat_enqueue_packets(pkts_burst,
+                    nb_rx, rx_config->ioat_ids[i]);
+                if (nb_enq > 0)
+                    rte_ioat_do_copies(rx_config->ioat_ids[i]);
+            } else {
+                /* Perform packet software copy, free source packets */
+                int ret;
+                struct rte_mbuf *pkts_burst_copy[MAX_PKT_BURST];
+
+                ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst_copy, nb_rx);
+
+                if (unlikely(ret < 0))
+                    rte_exit(EXIT_FAILURE,
+                        "Unable to allocate memory.\n");
+
+                for (j = 0; j < nb_rx; j++)
+                    pktmbuf_sw_copy(pkts_burst[j],
+                        pkts_burst_copy[j]);
+
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)pkts_burst, nb_rx);
+
+                nb_enq = rte_ring_enqueue_burst(
+                    rx_config->rx_to_tx_ring,
+                    (void *)pkts_burst_copy, nb_rx, NULL);
+
+                /* Free any not enqueued packets. */
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)&pkts_burst_copy[nb_enq],
+                    nb_rx - nb_enq);
+            }
+
+            port_statistics.copy_dropped[rx_config->rxtx_port] +=
+                (nb_rx - nb_enq);
+        }
+    }
+
+The packets are received in burst mode using ``rte_eth_rx_burst()``
+function. When using hardware copy mode the packets are enqueued in
+copying device's buffer using ``ioat_enqueue_packets()`` which calls
+``rte_ioat_enqueue_copy()``. When all received packets are in the
+buffer the copy operations are started by calling ``rte_ioat_do_copies()``.
+Function ``rte_ioat_enqueue_copy()`` operates on physical address of
+the packet. Structure ``rte_mbuf`` contains only physical address to
+start of the data buffer (``buf_iova``). Thus the address is adjusted
+by ``addr_offset`` value in order to get the address of ``rearm_data``
+member of ``rte_mbuf``. That way both the packet data and metadata can
+be copied in a single operation. This method can be used because the mbufs
+are direct mbufs allocated by the apps. If another app uses external buffers,
+or indirect mbufs, then multiple copy operations must be used.
+
+.. code-block:: c
+
+    static uint32_t
+    ioat_enqueue_packets(struct rte_mbuf **pkts,
+        uint32_t nb_rx, uint16_t dev_id)
+    {
+        int ret;
+        uint32_t i;
+        struct rte_mbuf *pkts_copy[MAX_PKT_BURST];
+
+        const uint64_t addr_offset = RTE_PTR_DIFF(pkts[0]->buf_addr,
+            &pkts[0]->rearm_data);
+
+        ret = rte_mempool_get_bulk(ioat_pktmbuf_pool,
+                (void *)pkts_copy, nb_rx);
+
+        if (unlikely(ret < 0))
+            rte_exit(EXIT_FAILURE, "Unable to allocate memory.\n");
+
+        for (i = 0; i < nb_rx; i++) {
+            /* Perform data copy */
+            ret = rte_ioat_enqueue_copy(dev_id,
+                pkts[i]->buf_iova
+                    - addr_offset,
+                pkts_copy[i]->buf_iova
+                    - addr_offset,
+                rte_pktmbuf_data_len(pkts[i])
+                    + addr_offset,
+                (uintptr_t)pkts[i],
+                (uintptr_t)pkts_copy[i],
+                0 /* nofence */);
+
+            if (ret != 1)
+                break;
+        }
+
+        ret = i;
+        /* Free any not enqueued packets. */
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts[i], nb_rx - i);
+        rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
+            nb_rx - i);
+
+        return ret;
+    }
+
+
+All completed copies are processed by ``ioat_tx_port()`` function. When using
+hardware copy mode the function invokes ``rte_ioat_completed_copies()``
+on each assigned IOAT channel to gather copied packets. If software copy
+mode is used the function dequeues copied packets from the rte_ring. Then each
+packet MAC address is changed if it was enabled. After that copies are sent
+in burst mode using `` rte_eth_tx_burst()``.
+
+
+.. code-block:: c
+
+    /* Transmit packets from IOAT rawdev/rte_ring for one port. */
+    static void
+    ioat_tx_port(struct rxtx_port_config *tx_config)
+    {
+        uint32_t i, j, nb_dq = 0;
+        struct rte_mbuf *mbufs_src[MAX_PKT_BURST];
+        struct rte_mbuf *mbufs_dst[MAX_PKT_BURST];
+
+        for (i = 0; i < tx_config->nb_queues; i++) {
+            if (copy_mode == COPY_MODE_IOAT_NUM) {
+                /* Deque the mbufs from IOAT device. */
+                nb_dq = rte_ioat_completed_copies(
+                    tx_config->ioat_ids[i], MAX_PKT_BURST,
+                    (void *)mbufs_src, (void *)mbufs_dst);
+            } else {
+                /* Deque the mbufs from rx_to_tx_ring. */
+                nb_dq = rte_ring_dequeue_burst(
+                    tx_config->rx_to_tx_ring, (void *)mbufs_dst,
+                    MAX_PKT_BURST, NULL);
+            }
+
+            if (nb_dq == 0)
+                return;
+
+            if (copy_mode == COPY_MODE_IOAT_NUM)
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                    (void *)mbufs_src, nb_dq);
+
+            /* Update macs if enabled */
+            if (mac_updating) {
+                for (j = 0; j < nb_dq; j++)
+                    update_mac_addrs(mbufs_dst[j],
+                        tx_config->rxtx_port);
+            }
+
+            const uint16_t nb_tx = rte_eth_tx_burst(
+                tx_config->rxtx_port, 0,
+                (void *)mbufs_dst, nb_dq);
+
+            port_statistics.tx[tx_config->rxtx_port] += nb_tx;
+
+            /* Free any unsent packets. */
+            if (unlikely(nb_tx < nb_dq))
+                rte_mempool_put_bulk(ioat_pktmbuf_pool,
+                (void *)&mbufs_dst[nb_tx],
+                    nb_dq - nb_tx);
+        }
+    }
+
+The Packet Copying Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to perform packet copy there is a user-defined function
+``pktmbuf_sw_copy()`` used. It copies a whole packet by copying
+metadata from source packet to new mbuf, and then copying a data
+chunk of source packet. Both memory copies are done using
+``rte_memcpy()``:
+
+.. code-block:: c
+
+    static inline void
+    pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst)
+    {
+        /* Copy packet metadata */
+        rte_memcpy(&dst->rearm_data,
+            &src->rearm_data,
+            offsetof(struct rte_mbuf, cacheline1)
+                - offsetof(struct rte_mbuf, rearm_data));
+
+        /* Copy packet data */
+        rte_memcpy(rte_pktmbuf_mtod(dst, char *),
+            rte_pktmbuf_mtod(src, char *), src->data_len);
+    }
+
+The metadata in this example is copied from ``rearm_data`` member of
+``rte_mbuf`` struct up to ``cacheline1``.
+
+In order to understand why software packet copying is done as shown
+above please refer to the "Mbuf Library" section of the
+*DPDK Programmer's Guide*.
-- 
2.21.0


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage
  2019-10-04 17:16           ` Kevin Traynor
@ 2019-10-07 11:10             ` Bruce Richardson
  0 siblings, 0 replies; 76+ messages in thread
From: Bruce Richardson @ 2019-10-07 11:10 UTC (permalink / raw)
  To: Kevin Traynor; +Cc: Marcin Baran, dev

On Fri, Oct 04, 2019 at 06:16:55PM +0100, Kevin Traynor wrote:
> On 30/09/2019 08:50, Marcin Baran wrote:
> > A new sample app demonstrating use of driver for CBDMA.
> > The app receives packets, performs software or hardware
> > copy, changes packets' MAC addresses (if enabled) and
> > forwards them. The patch includes sample application as
> > well as it's guide.
> > 
> > v6:
> >  - rearrange 'ioat_tx_port()' to remove unnecessary code
> >  - improve documentation
> >  - format patches for better readability
> > 
> > v5:
> >  - change dependency name from 'pmd_ioat' to 'rawdev_ioat'
> >    fixing build errors (branch diverged from master)
> > 
> > v4:
> >  - fix meson build support check
> > 
> > v3:
> >  - add check for meson build whether IOAT is supported
> > 
> > v2:
> >  - change printing total stats to be deltas
> >  - update documentation
> >  - fix 1 thread/sw copy/multiple Rx queues packet dropping
> >  - divide patch into several presenting functional change
> > 
> > 
> > Marcin Baran (4):
> >   examples/ioat: add software copy support
> >   examples/ioat: add rawdev copy mode support
> >   examples/ioat: add stats printing for each port
> >   doc/guides/: provide IOAT sample app guide
> > 
> > Pawel Modrak (2):
> >   examples/ioat: create sample app on ioat driver usage
> >   examples/ioat: add two threads configuration
> > 
> >  doc/guides/sample_app_ug/index.rst |    1 +
> >  doc/guides/sample_app_ug/intro.rst |    4 +
> >  doc/guides/sample_app_ug/ioat.rst  |  581 ++++++++++++++++
> >  examples/Makefile                  |    3 +
> >  examples/ioat/Makefile             |   54 ++
> >  examples/ioat/ioatfwd.c            | 1003 ++++++++++++++++++++++++++++
> >  examples/ioat/meson.build          |   15 +
> >  examples/meson.build               |    1 +
> >  8 files changed, 1662 insertions(+)
> >  create mode 100644 doc/guides/sample_app_ug/ioat.rst
> >  create mode 100644 examples/ioat/Makefile
> >  create mode 100644 examples/ioat/ioatfwd.c
> >  create mode 100644 examples/ioat/meson.build
> > 
> 
> Just a nit, but I think you need to add something to the MAINTAINERS
> file for this example app, maybe it can be a follow up patch if it's the
> only rework

Good catch. For completeness I've sent up a V7 of this set, adding this
example under the existing ioat section in MAINTAINERS.

/Bruce

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v7 4/6] examples/ioat: add two threads configuration
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 4/6] examples/ioat: add two threads configuration Bruce Richardson
@ 2019-10-27 17:04             ` Thomas Monjalon
  0 siblings, 0 replies; 76+ messages in thread
From: Thomas Monjalon @ 2019-10-27 17:04 UTC (permalink / raw)
  To: Bruce Richardson, Pawel Modrak; +Cc: dev, Marcin Baran

07/10/2019 13:08, Bruce Richardson:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> --- a/examples/ioat/ioatfwd.c
> +++ b/examples/ioat/ioatfwd.c
> @@ -161,7 +161,6 @@ ioat_enqueue_packets(struct rte_mbuf **pkts,
>  	rte_mempool_put_bulk(ioat_pktmbuf_pool, (void *)&pkts_copy[i],
>  		nb_rx - i);
>  
> -
>  	return ret;
>  }

This line can be removed in previous patch :)
(will do while merging)




^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v7 6/6] doc/guides/: provide IOAT sample app guide
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 6/6] doc/guides/: provide IOAT sample app guide Bruce Richardson
@ 2019-10-27 17:07             ` Thomas Monjalon
  0 siblings, 0 replies; 76+ messages in thread
From: Thomas Monjalon @ 2019-10-27 17:07 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev, Marcin Baran, Pawel Modrak

07/10/2019 13:08, Bruce Richardson:
> From: Marcin Baran <marcinx.baran@intel.com>
> 
> Added guide for IOAT sample app usage and code description.
> 
> Signed-off-by: Marcin Baran <marcinx.baran@intel.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  doc/guides/sample_app_ug/index.rst |   1 +
>  doc/guides/sample_app_ug/intro.rst |   4 +
>  doc/guides/sample_app_ug/ioat.rst  | 581 +++++++++++++++++++++++++++++

For completeness, this file should be added in MAINTAINERS (will do).



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver
  2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
                             ` (5 preceding siblings ...)
  2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 6/6] doc/guides/: provide IOAT sample app guide Bruce Richardson
@ 2019-10-27 17:07           ` Thomas Monjalon
  6 siblings, 0 replies; 76+ messages in thread
From: Thomas Monjalon @ 2019-10-27 17:07 UTC (permalink / raw)
  To: Bruce Richardson, Marcin Baran, Pawel Modrak; +Cc: dev

07/10/2019 13:08, Bruce Richardson:
> Marcin Baran (4):
>   examples/ioat: add software copy support
>   examples/ioat: add rawdev copy mode support
>   examples/ioat: add stats printing for each port
>   doc/guides/: provide IOAT sample app guide
> 
> Pawel Modrak (2):
>   examples/ioat: create sample app on ioat driver usage
>   examples/ioat: add two threads configuration

Applied (with minor updates), thanks




^ permalink raw reply	[flat|nested] 76+ messages in thread

end of thread, other threads:[~2019-10-27 17:07 UTC | newest]

Thread overview: 76+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-09  8:29 [dpdk-dev] [PATCH] examples/ioat: create sample app on ioat driver usage Marcin Baran
2019-09-09 13:12 ` Aaron Conole
2019-09-09 13:58   ` Bruce Richardson
2019-09-12  9:52 ` Bruce Richardson
2019-09-12 12:18   ` Baran, MarcinX
2019-09-13 14:39 ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: " Marcin Baran
2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 1/6] examples/ioat: create " Marcin Baran
2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 2/6] examples/ioat: add software copy support Marcin Baran
2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 4/6] examples/ioat: add two threads configuration Marcin Baran
2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 5/6] examples/ioat: add stats printing for each port Marcin Baran
2019-09-13 14:39   ` [dpdk-dev] [PATCH v2 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
2019-09-13 18:45   ` [dpdk-dev] [PATCH v2 0/6] examples/ioat: sample app on ioat driver usage Aaron Conole
2019-09-16  9:42     ` Baran, MarcinX
2019-09-19  9:19       ` Aaron Conole
2019-09-18  9:11   ` [dpdk-dev] [PATCH v3 " Marcin Baran
2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 1/6] examples/ioat: create " Marcin Baran
2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 2/6] examples/ioat: add software copy support Marcin Baran
2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 4/6] examples/ioat: add two threads configuration Marcin Baran
2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 5/6] examples/ioat: add stats printing for each port Marcin Baran
2019-09-18  9:11     ` [dpdk-dev] [PATCH v3 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
2019-09-19  9:38     ` [dpdk-dev] [PATCH v4 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 1/6] examples/ioat: create " Marcin Baran
2019-09-19 14:44         ` Bruce Richardson
2019-09-19 14:46           ` Baran, MarcinX
2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 2/6] examples/ioat: add software copy support Marcin Baran
2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 4/6] examples/ioat: add two threads configuration Marcin Baran
2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 5/6] examples/ioat: add stats printing for each port Marcin Baran
2019-09-19  9:38       ` [dpdk-dev] [PATCH v4 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
2019-09-20  7:37       ` [dpdk-dev] [PATCH v5 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 1/6] examples/ioat: create " Marcin Baran
2019-09-27  9:58           ` Bruce Richardson
2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 2/6] examples/ioat: add software copy support Marcin Baran
2019-09-27 10:01           ` Bruce Richardson
2019-09-27 14:01             ` Baran, MarcinX
2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
2019-09-27 10:05           ` Bruce Richardson
2019-09-27 14:03             ` Baran, MarcinX
2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 4/6] examples/ioat: add two threads configuration Marcin Baran
2019-09-27 10:08           ` Bruce Richardson
2019-09-27 14:03             ` Baran, MarcinX
2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 5/6] examples/ioat: add stats printing for each port Marcin Baran
2019-09-27 10:12           ` Bruce Richardson
2019-09-27 14:04             ` Baran, MarcinX
2019-09-20  7:37         ` [dpdk-dev] [PATCH v5 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
2019-09-27 10:36           ` Bruce Richardson
2019-09-27 14:14             ` Baran, MarcinX
2019-09-27 10:37           ` Bruce Richardson
2019-09-27 11:01           ` Bruce Richardson
2019-09-27 14:51             ` Baran, MarcinX
2019-09-27 15:00               ` Bruce Richardson
2019-09-27 15:16                 ` Baran, MarcinX
2019-09-27 13:22           ` Bruce Richardson
2019-09-27 15:13             ` Baran, MarcinX
2019-09-30  7:50         ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Marcin Baran
2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 1/6] examples/ioat: create " Marcin Baran
2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 2/6] examples/ioat: add software copy support Marcin Baran
2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 3/6] examples/ioat: add rawdev copy mode support Marcin Baran
2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 4/6] examples/ioat: add two threads configuration Marcin Baran
2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 5/6] examples/ioat: add stats printing for each port Marcin Baran
2019-09-30  7:50           ` [dpdk-dev] [PATCH v6 6/6] doc/guides/: provide IOAT sample app guide Marcin Baran
2019-10-03  9:48           ` [dpdk-dev] [PATCH v6 0/6] examples/ioat: sample app on ioat driver usage Bruce Richardson
2019-10-04 17:16           ` Kevin Traynor
2019-10-07 11:10             ` Bruce Richardson
2019-10-07 11:08         ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Bruce Richardson
2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 1/6] examples/ioat: new " Bruce Richardson
2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 2/6] examples/ioat: add software copy support Bruce Richardson
2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 3/6] examples/ioat: add rawdev copy mode support Bruce Richardson
2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 4/6] examples/ioat: add two threads configuration Bruce Richardson
2019-10-27 17:04             ` Thomas Monjalon
2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 5/6] examples/ioat: add stats printing for each port Bruce Richardson
2019-10-07 11:08           ` [dpdk-dev] [PATCH v7 6/6] doc/guides/: provide IOAT sample app guide Bruce Richardson
2019-10-27 17:07             ` Thomas Monjalon
2019-10-27 17:07           ` [dpdk-dev] [PATCH v7 0/6] examples/ioat: sample app for ioat driver Thomas Monjalon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).