[PATCH 1/2] doc/howto: rework section on virtio-user as exception path

[PATCH 1/2] doc/howto: rework section on virtio-user as exception path

[Bruce Richardson]

This patch extensively reworks the howto guide on using virtio-user for
exception packets. Changes include:

* rename "exceptional path" to "exception path"
* remove references to uio and just reference vfio-pci
* simplify testpmd command-lines, giving a basic usage example first
  before adding on detail about checksum or TSO parameters
* give a complete working example showing traffic flowing through the
  whole system from a testpmd loopback using the created TAP netdev
* replace use of "ifconfig" with Linux standard "ip" command
* other general rewording.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 .../howto/virtio_user_as_exceptional_path.rst | 159 +++++++++++-------
 1 file changed, 100 insertions(+), 59 deletions(-)

diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst b/doc/guides/howto/virtio_user_as_exceptional_path.rst
index ec021af399..0709eeff10 100644
--- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
+++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
@@ -3,116 +3,157 @@
 
 .. _virtio_user_as_exceptional_path:
 
-Virtio_user as Exceptional Path
-===============================
+Virtio_user as Exception Path
+=============================
 
-The virtual device, virtio-user, was originally introduced with vhost-user
-backend, as a high performance solution for IPC (Inter-Process Communication)
+.. note::
+
+   This solution is only applicable to Linux systems.
+
+The virtual device, virtio-user, was originally introduced with the vhost-user
+backend as a high performance solution for IPC (Inter-Process Communication)
 and user space container networking.
 
-Virtio_user with vhost-kernel backend is a solution for exceptional path,
-such as KNI which exchanges packets with kernel networking stack. This
-solution is very promising in:
+Beyond this originally intended use, virtio-user can be used in conjunction with the vhost-kernel
+backend as a solution for dealing with exception path packets which need to be injected into the
+Linux kernel for processing there.
+In this regard, virtio-user and vhost in kernel space are an alternative to DPDK KNI for
+transferring packets between a DPDK packet processing application and the kernel stack.
+
+This solution has a number of advantages over alternatives such as KNI:
 
 *   Maintenance
 
     All kernel modules needed by this solution, vhost and vhost-net (kernel),
-    are upstreamed and extensively used kernel module.
+    are upstreamed and extensively used.
 
 *   Features
 
-    vhost-net is born to be a networking solution, which has lots of networking
-    related features, like multi queue, tso, multi-seg mbuf, etc.
+    vhost-net is designed to be a networking solution, and, as such, has lots of networking
+    related features, such as multi queue support, TSO, multi-segment buffer support, etc.
 
 *   Performance
 
-    similar to KNI, this solution would use one or more kthreads to
-    send/receive packets to/from user space DPDK applications, which has little
-    impact on user space polling thread (except that it might enter into kernel
-    space to wake up those kthreads if necessary).
+    similar to KNI, this solution would uses one or more kthreads to
+    send/receive packets to/from user space DPDK applications, which minimises the impact
+    on the polling DPDK threads.
 
-The overview of an application using virtio-user as exceptional path is shown
+The overview of an application using virtio-user as exception path is shown
 in :numref:`figure_virtio_user_as_exceptional_path`.
 
 .. _figure_virtio_user_as_exceptional_path:
 
 .. figure:: img/virtio_user_as_exceptional_path.*
 
-   Overview of a DPDK app using virtio-user as exceptional path
+   Overview of a DPDK app using virtio-user as exception path
+
 
+Example Usage With Testpmd
+---------------------------
 
-Sample Usage
-------------
+.. note::
+
+   These instruction assume that the vhost/vhost-net kernel modules are available and have already
+   been loaded into the running kernel.
+   It also assumes that the DPDK virtio driver has not been disabled in the DPDK build.
 
-As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen before
-compiling the kernel and those kernel modules should be inserted.
+To run a simple test of virtio-user as exception path using testpmd:
 
-#.  Compile DPDK and bind a physical NIC to igb_uio/uio_pci_generic/vfio-pci.
+#.  Compile DPDK and bind a NIC to vfio-pci as documented in :ref:`linux_gsg_linux_drivers`.
 
-    This physical NIC is for communicating with outside.
+    This physical NIC is for communicating with the outside world,
+    and serves as a packet source in this example.
 
-#.  Run testpmd.
+#.  Run testpmd to forward packets from NIC to kernel,
+    passing in a suitable list of logical cores to run on  (``-l``.parameter),
+    and optionally the PCI address of the physical NIC to use (``-a`` parameter).
+    The virtio-user device for interfacing to the kernel is specified via a ``-vdev`` argument,
+    taking the parameters described below.
 
     .. code-block:: console
 
-        $(testpmd) -l 2-3 -n 4 \
-		--vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 \
-		-- -i --tx-offloads=0x0000002c --enable-lro \
-		--txd=1024 --rxd=1024
+        /path/to/dpdk-testpmd -l <cores> -a <pci BDF> \
+            --vdev=virtio_user0,path=/dev/vhost-net,queues=1,queue_size=1024
 
-    This command runs testpmd with two ports, one physical NIC to communicate
-    with outside, and one virtio-user to communicate with kernel.
+    * ``path``
 
-* ``--enable-lro``
+       The path to the kernel vhost-net device.
 
-    This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
-    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can be
-    transmitted to DPDK application and further TSOed by physical NIC.
+    * ``queue_size``
 
-* ``queue_size``
+       256 by default. To avoid shortage of descriptors, we can increase it to 1024.
 
-    256 by default. To avoid shortage of descriptors, we can increase it to 1024.
+    * ``queues``
 
-* ``queues``
+       Number of virt-queues. Each queue will be served by a kthread.
 
-    Number of multi-queues. Each queue will be served by a kthread. For example:
+#. Once testpmd is running, a new network interface - called ``tap0`` by default -
+   will be present on the system.
+   This should be configured with an IP address and then enabled for use:
 
     .. code-block:: console
 
-        $(testpmd) -l 2-3 -n 4 \
-		--vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 \
-		-- -i --tx-offloads=0x0000002c --enable-lro \
-		--txq=2 --rxq=2 --txd=1024 --rxd=1024
+        ip addr add 192.168.1.1/24 dev tap0
+        ip link set dev tap0 up
 
-#. Enable Rx checksum offloads in testpmd:
+#. To observe packet forwarding through the kernel,
+   a second testpmd instance can be run on the system,
+   taking packets from the kernel using an ``af_packet`` socket on the ``tap0`` interface.
 
-    .. code-block:: console
+   .. code-block:: console
 
-        (testpmd) port stop 0
-        (testpmd) port config 0 rx_offload tcp_cksum on
-        (testpmd) port config 0 rx_offload udp_cksum on
-        (testpmd) port start 0
+       /path/to/dpdk-testpmd -l <cores> --vdev=net_af_packet0,iface=tap0 --in-memory --no-pci
 
-#. Start testpmd:
+   When running this instance,
+   we can use ``--in-memory`` flag to avoid hugepage naming conflicts with the previous instance,
+   and we also use ``--no-pci`` flag to only use the ``af_packet`` interface for all traffic forwarding.
 
-    .. code-block:: console
+#. Running traffic into the system through the NIC should see that traffic returned back again,
+   having been forwarded through both testpmd instances.
+   This can be confirmed by checking the testpmd statistics on testpmd exit.
 
-        (testpmd) start
+For more advanced use of virtio-user with testpmd in this scenario,
+some other more advanced options may also be used.
+For example:
 
-#.  Configure IP address and start tap:
+* ``--tx-offloads=0x02c``
+
+    This testpmd option enables TX offloads for UDP and TCP checksum on transmit,
+    as well as TCP TSO support.
+    The list of the offload flag values can be seen in header `rte_ethdev.h
+    <http://doc.dpdk.org/api/rte__ethdev_8h.html>`_.
+
+* ``--enable-lro``
+
+    This testpmd option is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
+    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from the kernel can be
+    transmitted to the DPDK application and further TSOed by physical NIC.
+    If unsupported by the physical NIC, errors may be reported by testpmd with this option.
+
+
+* Enabling Rx checksum offloads for physical port:
+
+   Within testpmd, you can enable and disable offloads on a per-port basis,
+   rather than enabling them for both ports.
+   For the physical NIC, it may be desirable to enable checksum offload on packet RX.
+   This may be done as below, if testpmd is run with ``-i`` flag for interactive mode.
 
     .. code-block:: console
 
-        ifconfig tap0 1.1.1.1/24 up
+        testpmd> port stop 0
+        testpmd> port config 0 rx_offload tcp_cksum on
+        testpmd> port config 0 rx_offload udp_cksum on
+        testpmd> port start 0
 
-.. note::
+* Multiple queue support
 
-    The tap device will be named tap0, tap1, etc, by kernel.
+    Better performance may be achived by using multiple queues,
+    so that multiple kernel threads are handling the traffic on the kernel side.
+    For example, to use 2 queues on both NIC and virtio ports,
+    while also enabling TX offloads and LRO support:
 
-Then, all traffic from physical NIC can be forwarded into kernel stack, and all
-traffic on the tap0 can be sent out from physical NIC.
+    .. code-block:: console
 
-Limitations
------------
+        /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 -- \
+            -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --txd=1024 --rxd=1024
 
-This solution is only available on Linux systems.
-- 
2.34.1

[PATCH 2/2] doc/howto: add code example to virtio-user exception path doc

[Bruce Richardson]

The HOWTO guide for using virtio-user as an exception path to the kernel
only provided an example of how testpmd may be used for that purpose.
However, a real application wanting to use virtio-user as exception path
would likely want to create such devices from code within the app
itself. Therefore, we update the doc with instructions and a code
snippet showing how this may be done.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 .../howto/virtio_user_as_exceptional_path.rst | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst b/doc/guides/howto/virtio_user_as_exceptional_path.rst
index 0709eeff10..f4bd909458 100644
--- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
+++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
@@ -157,3 +157,58 @@ For example:
         /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 -- \
             -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --txd=1024 --rxd=1024
 
+
+Creating Virtio-User Ports within an Application
+------------------------------------------------
+
+To use virtio-user ports within an application,
+it is not necessary to explicitly initialize those ports using EAL arguments at startup.
+Instead, one can use the generic EAL API
+`rte_eal_hotplug_add <http://doc.dpdk.org/api/rte__dev_8h.html#ad32e8eebf1f81ef9f290cb296b0c90bb>`_
+function to create a new instance at startup.
+For example, to create a basic virtio-user port, the following code could be used:
+
+.. code-block:: C
+
+   rte_eal_hotplug_add("vdev", "virtio_user0", "path=/dev/vhost-net");
+
+A fuller code example is shown below, where a virtio-user port, and hence kernel netdev,
+is created for each NIC port discovered by DPDK.
+Each virtio-user port is given the MAC address of its matching physical port
+(assuming app was run without vdev args on commandline, so all ports auto-discovered are HW ones).
+These new virtio-user netdevs will appear in the kernel port listings as ``virtio_user0``,
+``virtio_user1``, etc.,
+based on the names passed in as ``iface=`` via the ``portargs`` parameter.
+
+.. code-block:: C
+
+    nb_ports = rte_eth_dev_count_avail();
+
+    /* Create a vhost_user port for each physical port */
+    unsigned port_count = 0;
+    RTE_ETH_FOREACH_DEV(portid) {
+        char portname[32];
+        char portargs[256];
+        struct rte_ether_addr addr = {0};
+
+        /* don't create virtio_user ports for other virtio_user ports */
+        if (++port_count > nb_ports)
+            break;
+
+        /* get mac address of physical port to use as mac of virtio_user port */
+        rte_eth_macaddr_get(portid, &addr);
+
+        /* set the name and arguments */
+        snprintf(portname, sizeof(portname), "virtio_user%u", portid);
+        snprintf(portargs, sizeof(portargs),
+                "path=/dev/vhost-net,queues=1,queue_size=%u,iface=%s,mac=" RTE_ETHER_ADDR_PRT_FMT,
+                RX_RING_SIZE, portname, RTE_ETHER_ADDR_BYTES(&addr));
+
+        /* add the vdev for virtio_user */
+        if (rte_eal_hotplug_add("vdev", portname, portargs) < 0)
+            rte_exit(EXIT_FAILURE, "Cannot create paired port for port %u\n", portid);
+
+    }
+
+Once these virtio-user ports have been created in the loop, all ports, both physical and virtual,
+may be initialized and used as normal in the application.
-- 
2.34.1

[PATCH v2 1/2] doc/howto: rework section on virtio-user as exception path

[Bruce Richardson]

This patch extensively reworks the howto guide on using virtio-user for
exception packets. Changes include:

* rename "exceptional path" to "exception path"
* remove references to uio and just reference vfio-pci
* simplify testpmd command-lines, giving a basic usage example first
  before adding on detail about checksum or TSO parameters
* give a complete working example showing traffic flowing through the
  whole system from a testpmd loopback using the created TAP netdev
* replace use of "ifconfig" with Linux standard "ip" command
* other general rewording.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

---
v2: fix checkpatch warnings
---
 .../howto/virtio_user_as_exceptional_path.rst | 159 +++++++++++-------
 1 file changed, 100 insertions(+), 59 deletions(-)

diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst b/doc/guides/howto/virtio_user_as_exceptional_path.rst
index ec021af399..100376c32d 100644
--- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
+++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
@@ -3,116 +3,157 @@

 .. _virtio_user_as_exceptional_path:

-Virtio_user as Exceptional Path
-===============================
+Virtio_user as Exception Path
+=============================

-The virtual device, virtio-user, was originally introduced with vhost-user
-backend, as a high performance solution for IPC (Inter-Process Communication)
+.. note::
+
+   This solution is only applicable to Linux systems.
+
+The virtual device, virtio-user, was originally introduced with the vhost-user
+backend as a high performance solution for IPC (Inter-Process Communication)
 and user space container networking.

-Virtio_user with vhost-kernel backend is a solution for exceptional path,
-such as KNI which exchanges packets with kernel networking stack. This
-solution is very promising in:
+Beyond this originally intended use, virtio-user can be used in conjunction with the vhost-kernel
+backend as a solution for dealing with exception path packets which need to be injected into the
+Linux kernel for processing there.
+In this regard, virtio-user and vhost in kernel space are an alternative to DPDK KNI for
+transferring packets between a DPDK packet processing application and the kernel stack.
+
+This solution has a number of advantages over alternatives such as KNI:

 *   Maintenance

     All kernel modules needed by this solution, vhost and vhost-net (kernel),
-    are upstreamed and extensively used kernel module.
+    are upstreamed and extensively used.

 *   Features

-    vhost-net is born to be a networking solution, which has lots of networking
-    related features, like multi queue, tso, multi-seg mbuf, etc.
+    vhost-net is designed to be a networking solution, and, as such, has lots of networking
+    related features, such as multi queue support, TSO, multi-segment buffer support, etc.

 *   Performance

-    similar to KNI, this solution would use one or more kthreads to
-    send/receive packets to/from user space DPDK applications, which has little
-    impact on user space polling thread (except that it might enter into kernel
-    space to wake up those kthreads if necessary).
+    similar to KNI, this solution would uses one or more kthreads to
+    send/receive packets to/from user space DPDK applications, which minimises the impact
+    on the polling DPDK threads.

-The overview of an application using virtio-user as exceptional path is shown
+The overview of an application using virtio-user as exception path is shown
 in :numref:`figure_virtio_user_as_exceptional_path`.

 .. _figure_virtio_user_as_exceptional_path:

 .. figure:: img/virtio_user_as_exceptional_path.*

-   Overview of a DPDK app using virtio-user as exceptional path
+   Overview of a DPDK app using virtio-user as exception path
+

+Example Usage With Testpmd
+---------------------------

-Sample Usage
-------------
+.. note::
+
+   These instruction assume that the vhost/vhost-net kernel modules are available and have already
+   been loaded into the running kernel.
+   It also assumes that the DPDK virtio driver has not been disabled in the DPDK build.

-As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen before
-compiling the kernel and those kernel modules should be inserted.
+To run a simple test of virtio-user as exception path using testpmd:

-#.  Compile DPDK and bind a physical NIC to igb_uio/uio_pci_generic/vfio-pci.
+#.  Compile DPDK and bind a NIC to vfio-pci as documented in :ref:`linux_gsg_linux_drivers`.

-    This physical NIC is for communicating with outside.
+    This physical NIC is for communicating with the outside world,
+    and serves as a packet source in this example.

-#.  Run testpmd.
+#.  Run testpmd to forward packets from NIC to kernel,
+    passing in a suitable list of logical cores to run on  (``-l``.parameter),
+    and optionally the PCI address of the physical NIC to use (``-a`` parameter).
+    The virtio-user device for interfacing to the kernel is specified via a ``-vdev`` argument,
+    taking the parameters described below.

     .. code-block:: console

-        $(testpmd) -l 2-3 -n 4 \
-		--vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 \
-		-- -i --tx-offloads=0x0000002c --enable-lro \
-		--txd=1024 --rxd=1024
+        /path/to/dpdk-testpmd -l <cores> -a <pci BDF> \
+            --vdev=virtio_user0,path=/dev/vhost-net,queues=1,queue_size=1024

-    This command runs testpmd with two ports, one physical NIC to communicate
-    with outside, and one virtio-user to communicate with kernel.
+    * ``path``

-* ``--enable-lro``
+       The path to the kernel vhost-net device.

-    This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
-    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can be
-    transmitted to DPDK application and further TSOed by physical NIC.
+    * ``queue_size``

-* ``queue_size``
+       256 by default. To avoid shortage of descriptors, we can increase it to 1024.

-    256 by default. To avoid shortage of descriptors, we can increase it to 1024.
+    * ``queues``

-* ``queues``
+       Number of virt-queues. Each queue will be served by a kthread.

-    Number of multi-queues. Each queue will be served by a kthread. For example:
+#. Once testpmd is running, a new network interface - called ``tap0`` by default -
+   will be present on the system.
+   This should be configured with an IP address and then enabled for use:

     .. code-block:: console

-        $(testpmd) -l 2-3 -n 4 \
-		--vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 \
-		-- -i --tx-offloads=0x0000002c --enable-lro \
-		--txq=2 --rxq=2 --txd=1024 --rxd=1024
+        ip addr add 192.168.1.1/24 dev tap0
+        ip link set dev tap0 up

-#. Enable Rx checksum offloads in testpmd:
+#. To observe packet forwarding through the kernel,
+   a second testpmd instance can be run on the system,
+   taking packets from the kernel using an ``af_packet`` socket on the ``tap0`` interface.

-    .. code-block:: console
+   .. code-block:: console

-        (testpmd) port stop 0
-        (testpmd) port config 0 rx_offload tcp_cksum on
-        (testpmd) port config 0 rx_offload udp_cksum on
-        (testpmd) port start 0
+       /path/to/dpdk-testpmd -l <cores> --vdev=net_af_packet0,iface=tap0 --in-memory --no-pci

-#. Start testpmd:
+   When running this instance,
+   we can use ``--in-memory`` flag to avoid hugepage naming conflicts with the previous instance,
+   and we also use ``--no-pci`` flag to only use the ``af_packet`` interface for all traffic forwarding.

-    .. code-block:: console
+#. Running traffic into the system through the NIC should see that traffic returned back again,
+   having been forwarded through both testpmd instances.
+   This can be confirmed by checking the testpmd statistics on testpmd exit.

-        (testpmd) start
+For more advanced use of virtio-user with testpmd in this scenario,
+some other more advanced options may also be used.
+For example:

-#.  Configure IP address and start tap:
+* ``--tx-offloads=0x02c``
+
+    This testpmd option enables TX offloads for UDP and TCP checksum on transmit,
+    as well as TCP TSO support.
+    The list of the offload flag values can be seen in header `rte_ethdev.h
+    <https://doc.dpdk.org/api/rte__ethdev_8h.html>`_.
+
+* ``--enable-lro``
+
+    This testpmd option is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
+    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from the kernel can be
+    transmitted to the DPDK application and further TSOed by physical NIC.
+    If unsupported by the physical NIC, errors may be reported by testpmd with this option.
+
+
+* Enabling Rx checksum offloads for physical port:
+
+   Within testpmd, you can enable and disable offloads on a per-port basis,
+   rather than enabling them for both ports.
+   For the physical NIC, it may be desirable to enable checksum offload on packet RX.
+   This may be done as below, if testpmd is run with ``-i`` flag for interactive mode.

     .. code-block:: console

-        ifconfig tap0 1.1.1.1/24 up
+        testpmd> port stop 0
+        testpmd> port config 0 rx_offload tcp_cksum on
+        testpmd> port config 0 rx_offload udp_cksum on
+        testpmd> port start 0

-.. note::
+* Multiple queue support

-    The tap device will be named tap0, tap1, etc, by kernel.
+    Better performance may be achieved by using multiple queues,
+    so that multiple kernel threads are handling the traffic on the kernel side.
+    For example, to use 2 queues on both NIC and virtio ports,
+    while also enabling TX offloads and LRO support:

-Then, all traffic from physical NIC can be forwarded into kernel stack, and all
-traffic on the tap0 can be sent out from physical NIC.
+    .. code-block:: console

-Limitations
------------
+        /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 -- \
+            -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --txd=1024 --rxd=1024

-This solution is only available on Linux systems.
--
2.34.1

[PATCH v2 2/2] doc/howto: add code example to virtio-user exception path doc

[Bruce Richardson]

The HOWTO guide for using virtio-user as an exception path to the kernel
only provided an example of how testpmd may be used for that purpose.
However, a real application wanting to use virtio-user as exception path
would likely want to create such devices from code within the app
itself. Therefore, we update the doc with instructions and a code
snippet showing how this may be done.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

---

 v2: fix http link to https
---
 .../howto/virtio_user_as_exceptional_path.rst | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst b/doc/guides/howto/virtio_user_as_exceptional_path.rst
index 100376c32d..45d4ebd284 100644
--- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
+++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
@@ -157,3 +157,58 @@ For example:
         /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 -- \
             -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --txd=1024 --rxd=1024
 
+
+Creating Virtio-User Ports within an Application
+------------------------------------------------
+
+To use virtio-user ports within an application,
+it is not necessary to explicitly initialize those ports using EAL arguments at startup.
+Instead, one can use the generic EAL API
+`rte_eal_hotplug_add <https://doc.dpdk.org/api/rte__dev_8h.html#ad32e8eebf1f81ef9f290cb296b0c90bb>`_
+function to create a new instance at startup.
+For example, to create a basic virtio-user port, the following code could be used:
+
+.. code-block:: C
+
+   rte_eal_hotplug_add("vdev", "virtio_user0", "path=/dev/vhost-net");
+
+A fuller code example is shown below, where a virtio-user port, and hence kernel netdev,
+is created for each NIC port discovered by DPDK.
+Each virtio-user port is given the MAC address of its matching physical port
+(assuming app was run without vdev args on commandline, so all ports auto-discovered are HW ones).
+These new virtio-user netdevs will appear in the kernel port listings as ``virtio_user0``,
+``virtio_user1``, etc.,
+based on the names passed in as ``iface=`` via the ``portargs`` parameter.
+
+.. code-block:: C
+
+    nb_ports = rte_eth_dev_count_avail();
+
+    /* Create a vhost_user port for each physical port */
+    unsigned port_count = 0;
+    RTE_ETH_FOREACH_DEV(portid) {
+        char portname[32];
+        char portargs[256];
+        struct rte_ether_addr addr = {0};
+
+        /* don't create virtio_user ports for other virtio_user ports */
+        if (++port_count > nb_ports)
+            break;
+
+        /* get mac address of physical port to use as mac of virtio_user port */
+        rte_eth_macaddr_get(portid, &addr);
+
+        /* set the name and arguments */
+        snprintf(portname, sizeof(portname), "virtio_user%u", portid);
+        snprintf(portargs, sizeof(portargs),
+                "path=/dev/vhost-net,queues=1,queue_size=%u,iface=%s,mac=" RTE_ETHER_ADDR_PRT_FMT,
+                RX_RING_SIZE, portname, RTE_ETHER_ADDR_BYTES(&addr));
+
+        /* add the vdev for virtio_user */
+        if (rte_eal_hotplug_add("vdev", portname, portargs) < 0)
+            rte_exit(EXIT_FAILURE, "Cannot create paired port for port %u\n", portid);
+
+    }
+
+Once these virtio-user ports have been created in the loop, all ports, both physical and virtual,
+may be initialized and used as normal in the application.
-- 
2.34.1

Re: [PATCH v2 2/2] doc/howto: add code example to virtio-user exception path doc

[Stephen Hemminger]

On Fri, 27 May 2022 17:36:43 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> The HOWTO guide for using virtio-user as an exception path to the kernel
> only provided an example of how testpmd may be used for that purpose.
> However, a real application wanting to use virtio-user as exception path
> would likely want to create such devices from code within the app
> itself. Therefore, we update the doc with instructions and a code
> snippet showing how this may be done.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

Acked-by: Stephen Hemminger <stephen@networkplumber.org>

RE: [PATCH v2 1/2] doc/howto: rework section on virtio-user as exception path

[Xia, Chenbo]

Hi Bruce,

> -----Original Message-----
> From: Richardson, Bruce <bruce.richardson@intel.com>
> Sent: Saturday, May 28, 2022 12:37 AM
> To: dev@dpdk.org
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: [PATCH v2 1/2] doc/howto: rework section on virtio-user as
> exception path
> 
> This patch extensively reworks the howto guide on using virtio-user for
> exception packets. Changes include:
> 
> * rename "exceptional path" to "exception path"
> * remove references to uio and just reference vfio-pci
> * simplify testpmd command-lines, giving a basic usage example first
>   before adding on detail about checksum or TSO parameters
> * give a complete working example showing traffic flowing through the
>   whole system from a testpmd loopback using the created TAP netdev
> * replace use of "ifconfig" with Linux standard "ip" command
> * other general rewording.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> ---
> v2: fix checkpatch warnings
> ---
>  .../howto/virtio_user_as_exceptional_path.rst | 159 +++++++++++-------

I agree with the renaming. And should we rename this file, and if possible, the
attached img too. 

>  1 file changed, 100 insertions(+), 59 deletions(-)
> 
> diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> index ec021af399..100376c32d 100644
> --- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> +++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> @@ -3,116 +3,157 @@
> 
>  .. _virtio_user_as_exceptional_path:
> 
> -Virtio_user as Exceptional Path
> -===============================
> +Virtio_user as Exception Path
> +=============================
> 
> -The virtual device, virtio-user, was originally introduced with vhost-
> user
> -backend, as a high performance solution for IPC (Inter-Process
> Communication)
> +.. note::
> +
> +   This solution is only applicable to Linux systems.
> +
> +The virtual device, virtio-user, was originally introduced with the
> vhost-user
> +backend as a high performance solution for IPC (Inter-Process
> Communication)
>  and user space container networking.
> 
> -Virtio_user with vhost-kernel backend is a solution for exceptional path,
> -such as KNI which exchanges packets with kernel networking stack. This
> -solution is very promising in:
> +Beyond this originally intended use, virtio-user can be used in
> conjunction with the vhost-kernel
> +backend as a solution for dealing with exception path packets which need
> to be injected into the
> +Linux kernel for processing there.
> +In this regard, virtio-user and vhost in kernel space are an alternative
> to DPDK KNI for
> +transferring packets between a DPDK packet processing application and the
> kernel stack.
> +
> +This solution has a number of advantages over alternatives such as KNI:
> 
>  *   Maintenance
> 
>      All kernel modules needed by this solution, vhost and vhost-net
> (kernel),
> -    are upstreamed and extensively used kernel module.
> +    are upstreamed and extensively used.
> 
>  *   Features
> 
> -    vhost-net is born to be a networking solution, which has lots of
> networking
> -    related features, like multi queue, tso, multi-seg mbuf, etc.
> +    vhost-net is designed to be a networking solution, and, as such, has
> lots of networking
> +    related features, such as multi queue support, TSO, multi-segment
> buffer support, etc.
> 
>  *   Performance
> 
> -    similar to KNI, this solution would use one or more kthreads to
> -    send/receive packets to/from user space DPDK applications, which has
> little
> -    impact on user space polling thread (except that it might enter into
> kernel
> -    space to wake up those kthreads if necessary).
> +    similar to KNI, this solution would uses one or more kthreads to
> +    send/receive packets to/from user space DPDK applications, which
> minimises the impact
> +    on the polling DPDK threads.
> 
> -The overview of an application using virtio-user as exceptional path is
> shown
> +The overview of an application using virtio-user as exception path is
> shown
>  in :numref:`figure_virtio_user_as_exceptional_path`.
> 
>  .. _figure_virtio_user_as_exceptional_path:
> 
>  .. figure:: img/virtio_user_as_exceptional_path.*
> 
> -   Overview of a DPDK app using virtio-user as exceptional path
> +   Overview of a DPDK app using virtio-user as exception path
> +
> 
> +Example Usage With Testpmd
> +---------------------------
> 
> -Sample Usage
> -------------
> +.. note::
> +
> +   These instruction assume that the vhost/vhost-net kernel modules are
> available and have already
> +   been loaded into the running kernel.
> +   It also assumes that the DPDK virtio driver has not been disabled in
> the DPDK build.
> 
> -As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen
> before
> -compiling the kernel and those kernel modules should be inserted.
> +To run a simple test of virtio-user as exception path using testpmd:
> 
> -#.  Compile DPDK and bind a physical NIC to igb_uio/uio_pci_generic/vfio-
> pci.
> +#.  Compile DPDK and bind a NIC to vfio-pci as documented
> in :ref:`linux_gsg_linux_drivers`.
> 
> -    This physical NIC is for communicating with outside.
> +    This physical NIC is for communicating with the outside world,
> +    and serves as a packet source in this example.
> 
> -#.  Run testpmd.
> +#.  Run testpmd to forward packets from NIC to kernel,
> +    passing in a suitable list of logical cores to run on  (``-
> l``.parameter),

This '.' should be a space?

Overall, the changes LGTM.

Thanks!
Chenbo

> +    and optionally the PCI address of the physical NIC to use (``-a``
> parameter).
> +    The virtio-user device for interfacing to the kernel is specified via
> a ``-vdev`` argument,
> +    taking the parameters described below.
> 
>      .. code-block:: console
> 
> -        $(testpmd) -l 2-3 -n 4 \
> -		--vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 \
> -		-- -i --tx-offloads=0x0000002c --enable-lro \
> -		--txd=1024 --rxd=1024
> +        /path/to/dpdk-testpmd -l <cores> -a <pci BDF> \
> +            --vdev=virtio_user0,path=/dev/vhost-
> net,queues=1,queue_size=1024
> 
> -    This command runs testpmd with two ports, one physical NIC to
> communicate
> -    with outside, and one virtio-user to communicate with kernel.
> +    * ``path``
> 
> -* ``--enable-lro``
> +       The path to the kernel vhost-net device.
> 
> -    This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
> -    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can
> be
> -    transmitted to DPDK application and further TSOed by physical NIC.
> +    * ``queue_size``
> 
> -* ``queue_size``
> +       256 by default. To avoid shortage of descriptors, we can increase
> it to 1024.
> 
> -    256 by default. To avoid shortage of descriptors, we can increase it
> to 1024.
> +    * ``queues``
> 
> -* ``queues``
> +       Number of virt-queues. Each queue will be served by a kthread.
> 
> -    Number of multi-queues. Each queue will be served by a kthread. For
> example:
> +#. Once testpmd is running, a new network interface - called ``tap0`` by
> default -
> +   will be present on the system.
> +   This should be configured with an IP address and then enabled for use:
> 
>      .. code-block:: console
> 
> -        $(testpmd) -l 2-3 -n 4 \
> -		--vdev=virtio_user0,path=/dev/vhost-
> net,queues=2,queue_size=1024 \
> -		-- -i --tx-offloads=0x0000002c --enable-lro \
> -		--txq=2 --rxq=2 --txd=1024 --rxd=1024
> +        ip addr add 192.168.1.1/24 dev tap0
> +        ip link set dev tap0 up
> 
> -#. Enable Rx checksum offloads in testpmd:
> +#. To observe packet forwarding through the kernel,
> +   a second testpmd instance can be run on the system,
> +   taking packets from the kernel using an ``af_packet`` socket on the
> ``tap0`` interface.
> 
> -    .. code-block:: console
> +   .. code-block:: console
> 
> -        (testpmd) port stop 0
> -        (testpmd) port config 0 rx_offload tcp_cksum on
> -        (testpmd) port config 0 rx_offload udp_cksum on
> -        (testpmd) port start 0
> +       /path/to/dpdk-testpmd -l <cores> --vdev=net_af_packet0,iface=tap0
> --in-memory --no-pci
> 
> -#. Start testpmd:
> +   When running this instance,
> +   we can use ``--in-memory`` flag to avoid hugepage naming conflicts
> with the previous instance,
> +   and we also use ``--no-pci`` flag to only use the ``af_packet``
> interface for all traffic forwarding.
> 
> -    .. code-block:: console
> +#. Running traffic into the system through the NIC should see that
> traffic returned back again,
> +   having been forwarded through both testpmd instances.
> +   This can be confirmed by checking the testpmd statistics on testpmd
> exit.
> 
> -        (testpmd) start
> +For more advanced use of virtio-user with testpmd in this scenario,
> +some other more advanced options may also be used.
> +For example:
> 
> -#.  Configure IP address and start tap:
> +* ``--tx-offloads=0x02c``
> +
> +    This testpmd option enables TX offloads for UDP and TCP checksum on
> transmit,
> +    as well as TCP TSO support.
> +    The list of the offload flag values can be seen in header
> `rte_ethdev.h
> +    <https://doc.dpdk.org/api/rte__ethdev_8h.html>`_.
> +
> +* ``--enable-lro``
> +
> +    This testpmd option is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
> +    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from the kernel
> can be
> +    transmitted to the DPDK application and further TSOed by physical NIC.
> +    If unsupported by the physical NIC, errors may be reported by testpmd
> with this option.
> +
> +
> +* Enabling Rx checksum offloads for physical port:
> +
> +   Within testpmd, you can enable and disable offloads on a per-port
> basis,
> +   rather than enabling them for both ports.
> +   For the physical NIC, it may be desirable to enable checksum offload
> on packet RX.
> +   This may be done as below, if testpmd is run with ``-i`` flag for
> interactive mode.
> 
>      .. code-block:: console
> 
> -        ifconfig tap0 1.1.1.1/24 up
> +        testpmd> port stop 0
> +        testpmd> port config 0 rx_offload tcp_cksum on
> +        testpmd> port config 0 rx_offload udp_cksum on
> +        testpmd> port start 0
> 
> -.. note::
> +* Multiple queue support
> 
> -    The tap device will be named tap0, tap1, etc, by kernel.
> +    Better performance may be achieved by using multiple queues,
> +    so that multiple kernel threads are handling the traffic on the
> kernel side.
> +    For example, to use 2 queues on both NIC and virtio ports,
> +    while also enabling TX offloads and LRO support:
> 
> -Then, all traffic from physical NIC can be forwarded into kernel stack,
> and all
> -traffic on the tap0 can be sent out from physical NIC.
> +    .. code-block:: console
> 
> -Limitations
> ------------
> +        /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-
> net,queues=2,queue_size=1024 -- \
> +            -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --
> txd=1024 --rxd=1024
> 
> -This solution is only available on Linux systems.
> --
> 2.34.1

RE: [PATCH v2 2/2] doc/howto: add code example to virtio-user exception path doc

[Xia, Chenbo]

> -----Original Message-----
> From: Richardson, Bruce <bruce.richardson@intel.com>
> Sent: Saturday, May 28, 2022 12:37 AM
> To: dev@dpdk.org
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: [PATCH v2 2/2] doc/howto: add code example to virtio-user
> exception path doc
> 
> The HOWTO guide for using virtio-user as an exception path to the kernel
> only provided an example of how testpmd may be used for that purpose.
> However, a real application wanting to use virtio-user as exception path
> would likely want to create such devices from code within the app
> itself. Therefore, we update the doc with instructions and a code
> snippet showing how this may be done.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> ---
> 
>  v2: fix http link to https
> ---
>  .../howto/virtio_user_as_exceptional_path.rst | 55 +++++++++++++++++++
>  1 file changed, 55 insertions(+)
> 
> diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> index 100376c32d..45d4ebd284 100644
> --- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> +++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> @@ -157,3 +157,58 @@ For example:
>          /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-
> net,queues=2,queue_size=1024 -- \
>              -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --
> txd=1024 --rxd=1024
> 
> +
> +Creating Virtio-User Ports within an Application
> +------------------------------------------------
> +
> +To use virtio-user ports within an application,
> +it is not necessary to explicitly initialize those ports using EAL
> arguments at startup.
> +Instead, one can use the generic EAL API
> +`rte_eal_hotplug_add
> <https://doc.dpdk.org/api/rte__dev_8h.html#ad32e8eebf1f81ef9f290cb296b0c90
> bb>`_
> +function to create a new instance at startup.
> +For example, to create a basic virtio-user port, the following code could
> be used:
> +
> +.. code-block:: C
> +
> +   rte_eal_hotplug_add("vdev", "virtio_user0", "path=/dev/vhost-net");
> +
> +A fuller code example is shown below, where a virtio-user port, and hence
> kernel netdev,
> +is created for each NIC port discovered by DPDK.
> +Each virtio-user port is given the MAC address of its matching physical
> port
> +(assuming app was run without vdev args on commandline, so all ports
> auto-discovered are HW ones).
> +These new virtio-user netdevs will appear in the kernel port listings as
> ``virtio_user0``,

You mean vhost_tap netdev here? Or?

Thanks,
Chenbo

> +``virtio_user1``, etc.,
> +based on the names passed in as ``iface=`` via the ``portargs`` parameter.
> +
> +.. code-block:: C
> +
> +    nb_ports = rte_eth_dev_count_avail();
> +
> +    /* Create a vhost_user port for each physical port */
> +    unsigned port_count = 0;
> +    RTE_ETH_FOREACH_DEV(portid) {
> +        char portname[32];
> +        char portargs[256];
> +        struct rte_ether_addr addr = {0};
> +
> +        /* don't create virtio_user ports for other virtio_user ports */
> +        if (++port_count > nb_ports)
> +            break;
> +
> +        /* get mac address of physical port to use as mac of virtio_user
> port */
> +        rte_eth_macaddr_get(portid, &addr);
> +
> +        /* set the name and arguments */
> +        snprintf(portname, sizeof(portname), "virtio_user%u", portid);
> +        snprintf(portargs, sizeof(portargs),
> +                "path=/dev/vhost-
> net,queues=1,queue_size=%u,iface=%s,mac=" RTE_ETHER_ADDR_PRT_FMT,
> +                RX_RING_SIZE, portname, RTE_ETHER_ADDR_BYTES(&addr));
> +
> +        /* add the vdev for virtio_user */
> +        if (rte_eal_hotplug_add("vdev", portname, portargs) < 0)
> +            rte_exit(EXIT_FAILURE, "Cannot create paired port for
> port %u\n", portid);
> +
> +    }
> +
> +Once these virtio-user ports have been created in the loop, all ports,
> both physical and virtual,
> +may be initialized and used as normal in the application.
> --
> 2.34.1

Re: [PATCH v2 2/2] doc/howto: add code example to virtio-user exception path doc

[Bruce Richardson]

On Mon, May 30, 2022 at 06:44:27AM +0100, Xia, Chenbo wrote:
> > -----Original Message-----
> > From: Richardson, Bruce <bruce.richardson@intel.com>
> > Sent: Saturday, May 28, 2022 12:37 AM
> > To: dev@dpdk.org
> > Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> > <chenbo.xia@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
> > Subject: [PATCH v2 2/2] doc/howto: add code example to virtio-user
> > exception path doc
> >
> > The HOWTO guide for using virtio-user as an exception path to the kernel
> > only provided an example of how testpmd may be used for that purpose.
> > However, a real application wanting to use virtio-user as exception path
> > would likely want to create such devices from code within the app
> > itself. Therefore, we update the doc with instructions and a code
> > snippet showing how this may be done.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> >
> > ---
> >
> >  v2: fix http link to https
> > ---
> >  .../howto/virtio_user_as_exceptional_path.rst | 55 +++++++++++++++++++
> >  1 file changed, 55 insertions(+)
> >
> > diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> > b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> > index 100376c32d..45d4ebd284 100644
> > --- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> > +++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> > @@ -157,3 +157,58 @@ For example:
> >          /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-
> > net,queues=2,queue_size=1024 -- \
> >              -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --
> > txd=1024 --rxd=1024
> >
> > +
> > +Creating Virtio-User Ports within an Application
> > +------------------------------------------------
> > +
> > +To use virtio-user ports within an application,
> > +it is not necessary to explicitly initialize those ports using EAL
> > arguments at startup.
> > +Instead, one can use the generic EAL API
> > +`rte_eal_hotplug_add
> > <https://doc.dpdk.org/api/rte__dev_8h.html#ad32e8eebf1f81ef9f290cb296b0c90
> > bb>`_
> > +function to create a new instance at startup.
> > +For example, to create a basic virtio-user port, the following code could
> > be used:
> > +
> > +.. code-block:: C
> > +
> > +   rte_eal_hotplug_add("vdev", "virtio_user0", "path=/dev/vhost-net");
> > +
> > +A fuller code example is shown below, where a virtio-user port, and hence
> > kernel netdev,
> > +is created for each NIC port discovered by DPDK.
> > +Each virtio-user port is given the MAC address of its matching physical
> > port
> > +(assuming app was run without vdev args on commandline, so all ports
> > auto-discovered are HW ones).
> > +These new virtio-user netdevs will appear in the kernel port listings as
> > ``virtio_user0``,
> 
> You mean vhost_tap netdev here? Or?
> 

Yes, I am referring to the vhost tap port. If you look at the code below,
the parameters provide the name of the kernel netdev to be the same as the
name of the vdev created, meaning any ports will appear as "vhost_user0",
"vhost_user1" etc. in your ip-link/ifconfig listings.

/Bruce

> 
> > +``virtio_user1``, etc.,
> > +based on the names passed in as ``iface=`` via the ``portargs`` parameter.
> > +
> > +.. code-block:: C
> > +
> > +    nb_ports = rte_eth_dev_count_avail();
> > +
> > +    /* Create a vhost_user port for each physical port */
> > +    unsigned port_count = 0;
> > +    RTE_ETH_FOREACH_DEV(portid) {
> > +        char portname[32];
> > +        char portargs[256];
> > +        struct rte_ether_addr addr = {0};
> > +
> > +        /* don't create virtio_user ports for other virtio_user ports */
> > +        if (++port_count > nb_ports)
> > +            break;
> > +
> > +        /* get mac address of physical port to use as mac of virtio_user
> > port */
> > +        rte_eth_macaddr_get(portid, &addr);
> > +
> > +        /* set the name and arguments */
> > +        snprintf(portname, sizeof(portname), "virtio_user%u", portid);
> > +        snprintf(portargs, sizeof(portargs),
> > +                "path=/dev/vhost-
> > net,queues=1,queue_size=%u,iface=%s,mac=" RTE_ETHER_ADDR_PRT_FMT,
> > +                RX_RING_SIZE, portname, RTE_ETHER_ADDR_BYTES(&addr));
> > +
> > +        /* add the vdev for virtio_user */
> > +        if (rte_eal_hotplug_add("vdev", portname, portargs) < 0)
> > +            rte_exit(EXIT_FAILURE, "Cannot create paired port for
> > port %u\n", portid);
> > +
> > +    }
> > +
> > +Once these virtio-user ports have been created in the loop, all ports,
> > both physical and virtual,
> > +may be initialized and used as normal in the application.
> > --
> > 2.34.1
> 

Re: [PATCH v2 1/2] doc/howto: rework section on virtio-user as exception path

[Bruce Richardson]

On Mon, May 30, 2022 at 06:33:14AM +0100, Xia, Chenbo wrote:
> Hi Bruce,
> 
> > -----Original Message-----
> > From: Richardson, Bruce <bruce.richardson@intel.com>
> > Sent: Saturday, May 28, 2022 12:37 AM
> > To: dev@dpdk.org
> > Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> > <chenbo.xia@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
> > Subject: [PATCH v2 1/2] doc/howto: rework section on virtio-user as
> > exception path
> >
> > This patch extensively reworks the howto guide on using virtio-user for
> > exception packets. Changes include:
> >
> > * rename "exceptional path" to "exception path"
> > * remove references to uio and just reference vfio-pci
> > * simplify testpmd command-lines, giving a basic usage example first
> >   before adding on detail about checksum or TSO parameters
> > * give a complete working example showing traffic flowing through the
> >   whole system from a testpmd loopback using the created TAP netdev
> > * replace use of "ifconfig" with Linux standard "ip" command
> > * other general rewording.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> >
> > ---
> > v2: fix checkpatch warnings
> > ---
> >  .../howto/virtio_user_as_exceptional_path.rst | 159 +++++++++++-------
> 
> I agree with the renaming. And should we rename this file, and if possible, the
> attached img too.
> 

I wasn't sure of the value of doing such a rename - afraid it might clobber
up the git history. I'll add it as a separate patch in a v3.

> >  1 file changed, 100 insertions(+), 59 deletions(-)
> >
> > diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> > b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> > index ec021af399..100376c32d 100644
> > --- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> > +++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> > @@ -3,116 +3,157 @@
> >
> >  .. _virtio_user_as_exceptional_path:
> >
> > -Virtio_user as Exceptional Path
> > -===============================
> > +Virtio_user as Exception Path
> > +=============================
> >
> > -The virtual device, virtio-user, was originally introduced with vhost-
> > user
> > -backend, as a high performance solution for IPC (Inter-Process
> > Communication)
> > +.. note::
> > +
> > +   This solution is only applicable to Linux systems.
> > +
> > +The virtual device, virtio-user, was originally introduced with the
> > vhost-user
> > +backend as a high performance solution for IPC (Inter-Process
> > Communication)
> >  and user space container networking.
> >
> > -Virtio_user with vhost-kernel backend is a solution for exceptional path,
> > -such as KNI which exchanges packets with kernel networking stack. This
> > -solution is very promising in:
> > +Beyond this originally intended use, virtio-user can be used in
> > conjunction with the vhost-kernel
> > +backend as a solution for dealing with exception path packets which need
> > to be injected into the
> > +Linux kernel for processing there.
> > +In this regard, virtio-user and vhost in kernel space are an alternative
> > to DPDK KNI for
> > +transferring packets between a DPDK packet processing application and the
> > kernel stack.
> > +
> > +This solution has a number of advantages over alternatives such as KNI:
> >
> >  *   Maintenance
> >
> >      All kernel modules needed by this solution, vhost and vhost-net
> > (kernel),
> > -    are upstreamed and extensively used kernel module.
> > +    are upstreamed and extensively used.
> >
> >  *   Features
> >
> > -    vhost-net is born to be a networking solution, which has lots of
> > networking
> > -    related features, like multi queue, tso, multi-seg mbuf, etc.
> > +    vhost-net is designed to be a networking solution, and, as such, has
> > lots of networking
> > +    related features, such as multi queue support, TSO, multi-segment
> > buffer support, etc.
> >
> >  *   Performance
> >
> > -    similar to KNI, this solution would use one or more kthreads to
> > -    send/receive packets to/from user space DPDK applications, which has
> > little
> > -    impact on user space polling thread (except that it might enter into
> > kernel
> > -    space to wake up those kthreads if necessary).
> > +    similar to KNI, this solution would uses one or more kthreads to
> > +    send/receive packets to/from user space DPDK applications, which
> > minimises the impact
> > +    on the polling DPDK threads.
> >
> > -The overview of an application using virtio-user as exceptional path is
> > shown
> > +The overview of an application using virtio-user as exception path is
> > shown
> >  in :numref:`figure_virtio_user_as_exceptional_path`.
> >
> >  .. _figure_virtio_user_as_exceptional_path:
> >
> >  .. figure:: img/virtio_user_as_exceptional_path.*
> >
> > -   Overview of a DPDK app using virtio-user as exceptional path
> > +   Overview of a DPDK app using virtio-user as exception path
> > +
> >
> > +Example Usage With Testpmd
> > +---------------------------
> >
> > -Sample Usage
> > -------------
> > +.. note::
> > +
> > +   These instruction assume that the vhost/vhost-net kernel modules are
> > available and have already
> > +   been loaded into the running kernel.
> > +   It also assumes that the DPDK virtio driver has not been disabled in
> > the DPDK build.
> >
> > -As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen
> > before
> > -compiling the kernel and those kernel modules should be inserted.
> > +To run a simple test of virtio-user as exception path using testpmd:
> >
> > -#.  Compile DPDK and bind a physical NIC to igb_uio/uio_pci_generic/vfio-
> > pci.
> > +#.  Compile DPDK and bind a NIC to vfio-pci as documented
> > in :ref:`linux_gsg_linux_drivers`.
> >
> > -    This physical NIC is for communicating with outside.
> > +    This physical NIC is for communicating with the outside world,
> > +    and serves as a packet source in this example.
> >
> > -#.  Run testpmd.
> > +#.  Run testpmd to forward packets from NIC to kernel,
> > +    passing in a suitable list of logical cores to run on  (``-
> > l``.parameter),
> 
> This '.' should be a space?
>
Thanks, good catch.

[PATCH v3 0/4] Enhance docs on virtio-user as KNI replacement

[Bruce Richardson]

Since use of KNI is no longer advised (due to known issues and the fact
it's an out-of-tree module), virtio-user is recommended for new
developments. To help encourage this, we improve the doc HOWTO section
on using virtio-user and add a link to that document from the main KNI
section in the programmers guide.

v3:
* Added two extra patches:
  - rename document files to match new section name
  - add note to KNI programmers guide section
* minor updates to patch 2, and added stable on CC for that patch

Bruce Richardson (4):
  doc/howto: rework section on virtio-user as exception path
  doc/howto: rename files to match new content name
  doc/howto: add code example to virtio-user exception path doc
  doc/prog_guide: add reference to virtio-user from KNI doc

 ....svg => virtio_user_as_exception_path.svg} |   0
 doc/guides/howto/index.rst                    |   2 +-
 .../howto/virtio_user_as_exception_path.rst   | 214 ++++++++++++++++++
 .../howto/virtio_user_as_exceptional_path.rst | 118 ----------
 .../prog_guide/kernel_nic_interface.rst       |   6 +
 5 files changed, 221 insertions(+), 119 deletions(-)
 rename doc/guides/howto/img/{virtio_user_as_exceptional_path.svg => virtio_user_as_exception_path.svg} (100%)
 create mode 100644 doc/guides/howto/virtio_user_as_exception_path.rst
 delete mode 100644 doc/guides/howto/virtio_user_as_exceptional_path.rst

--
2.34.1

[PATCH v3 1/4] doc/howto: rework section on virtio-user as exception path

[Bruce Richardson]

This patch extensively reworks the howto guide on using virtio-user for
exception packets. Changes include:

* rename "exceptional path" to "exception path"
* remove references to uio and just reference vfio-pci
* simplify testpmd command-lines, giving a basic usage example first
  before adding on detail about checksum or TSO parameters
* give a complete working example showing traffic flowing through the
  whole system from a testpmd loopback using the created TAP netdev
* replace use of "ifconfig" with Linux standard "ip" command
* other general rewording.

CC: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

---
V3:
* fix error reported by Chenbo on review.
* add stable on CC, since this rework could be applicable for older
  releases too, if desired for backport.
---
 .../howto/virtio_user_as_exceptional_path.rst | 159 +++++++++++-------
 1 file changed, 100 insertions(+), 59 deletions(-)

diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst b/doc/guides/howto/virtio_user_as_exceptional_path.rst
index ec021af399..3d3d1b1219 100644
--- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
+++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
@@ -3,116 +3,157 @@

 .. _virtio_user_as_exceptional_path:

-Virtio_user as Exceptional Path
-===============================
+Virtio_user as Exception Path
+=============================

-The virtual device, virtio-user, was originally introduced with vhost-user
-backend, as a high performance solution for IPC (Inter-Process Communication)
+.. note::
+
+   This solution is only applicable to Linux systems.
+
+The virtual device, virtio-user, was originally introduced with the vhost-user
+backend as a high performance solution for IPC (Inter-Process Communication)
 and user space container networking.

-Virtio_user with vhost-kernel backend is a solution for exceptional path,
-such as KNI which exchanges packets with kernel networking stack. This
-solution is very promising in:
+Beyond this originally intended use, virtio-user can be used in conjunction with the vhost-kernel
+backend as a solution for dealing with exception path packets which need to be injected into the
+Linux kernel for processing there.
+In this regard, virtio-user and vhost in kernel space are an alternative to DPDK KNI for
+transferring packets between a DPDK packet processing application and the kernel stack.
+
+This solution has a number of advantages over alternatives such as KNI:

 *   Maintenance

     All kernel modules needed by this solution, vhost and vhost-net (kernel),
-    are upstreamed and extensively used kernel module.
+    are upstreamed and extensively used.

 *   Features

-    vhost-net is born to be a networking solution, which has lots of networking
-    related features, like multi queue, tso, multi-seg mbuf, etc.
+    vhost-net is designed to be a networking solution, and, as such, has lots of networking
+    related features, such as multi queue support, TSO, multi-segment buffer support, etc.

 *   Performance

-    similar to KNI, this solution would use one or more kthreads to
-    send/receive packets to/from user space DPDK applications, which has little
-    impact on user space polling thread (except that it might enter into kernel
-    space to wake up those kthreads if necessary).
+    similar to KNI, this solution would uses one or more kthreads to
+    send/receive packets to/from user space DPDK applications, which minimises the impact
+    on the polling DPDK threads.

-The overview of an application using virtio-user as exceptional path is shown
+The overview of an application using virtio-user as exception path is shown
 in :numref:`figure_virtio_user_as_exceptional_path`.

 .. _figure_virtio_user_as_exceptional_path:

 .. figure:: img/virtio_user_as_exceptional_path.*

-   Overview of a DPDK app using virtio-user as exceptional path
+   Overview of a DPDK app using virtio-user as exception path
+

+Example Usage With Testpmd
+---------------------------

-Sample Usage
-------------
+.. note::
+
+   These instruction assume that the vhost/vhost-net kernel modules are available and have already
+   been loaded into the running kernel.
+   It also assumes that the DPDK virtio driver has not been disabled in the DPDK build.

-As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen before
-compiling the kernel and those kernel modules should be inserted.
+To run a simple test of virtio-user as exception path using testpmd:

-#.  Compile DPDK and bind a physical NIC to igb_uio/uio_pci_generic/vfio-pci.
+#.  Compile DPDK and bind a NIC to vfio-pci as documented in :ref:`linux_gsg_linux_drivers`.

-    This physical NIC is for communicating with outside.
+    This physical NIC is for communicating with the outside world,
+    and serves as a packet source in this example.

-#.  Run testpmd.
+#.  Run testpmd to forward packets from NIC to kernel,
+    passing in a suitable list of logical cores to run on  (``-l`` parameter),
+    and optionally the PCI address of the physical NIC to use (``-a`` parameter).
+    The virtio-user device for interfacing to the kernel is specified via a ``-vdev`` argument,
+    taking the parameters described below.

     .. code-block:: console

-        $(testpmd) -l 2-3 -n 4 \
-		--vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 \
-		-- -i --tx-offloads=0x0000002c --enable-lro \
-		--txd=1024 --rxd=1024
+        /path/to/dpdk-testpmd -l <cores> -a <pci BDF> \
+            --vdev=virtio_user0,path=/dev/vhost-net,queues=1,queue_size=1024

-    This command runs testpmd with two ports, one physical NIC to communicate
-    with outside, and one virtio-user to communicate with kernel.
+    * ``path``

-* ``--enable-lro``
+       The path to the kernel vhost-net device.

-    This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
-    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can be
-    transmitted to DPDK application and further TSOed by physical NIC.
+    * ``queue_size``

-* ``queue_size``
+       256 by default. To avoid shortage of descriptors, we can increase it to 1024.

-    256 by default. To avoid shortage of descriptors, we can increase it to 1024.
+    * ``queues``

-* ``queues``
+       Number of virt-queues. Each queue will be served by a kthread.

-    Number of multi-queues. Each queue will be served by a kthread. For example:
+#. Once testpmd is running, a new network interface - called ``tap0`` by default -
+   will be present on the system.
+   This should be configured with an IP address and then enabled for use:

     .. code-block:: console

-        $(testpmd) -l 2-3 -n 4 \
-		--vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 \
-		-- -i --tx-offloads=0x0000002c --enable-lro \
-		--txq=2 --rxq=2 --txd=1024 --rxd=1024
+        ip addr add 192.168.1.1/24 dev tap0
+        ip link set dev tap0 up

-#. Enable Rx checksum offloads in testpmd:
+#. To observe packet forwarding through the kernel,
+   a second testpmd instance can be run on the system,
+   taking packets from the kernel using an ``af_packet`` socket on the ``tap0`` interface.

-    .. code-block:: console
+   .. code-block:: console

-        (testpmd) port stop 0
-        (testpmd) port config 0 rx_offload tcp_cksum on
-        (testpmd) port config 0 rx_offload udp_cksum on
-        (testpmd) port start 0
+       /path/to/dpdk-testpmd -l <cores> --vdev=net_af_packet0,iface=tap0 --in-memory --no-pci

-#. Start testpmd:
+   When running this instance,
+   we can use ``--in-memory`` flag to avoid hugepage naming conflicts with the previous instance,
+   and we also use ``--no-pci`` flag to only use the ``af_packet`` interface for all traffic forwarding.

-    .. code-block:: console
+#. Running traffic into the system through the NIC should see that traffic returned back again,
+   having been forwarded through both testpmd instances.
+   This can be confirmed by checking the testpmd statistics on testpmd exit.

-        (testpmd) start
+For more advanced use of virtio-user with testpmd in this scenario,
+some other more advanced options may also be used.
+For example:

-#.  Configure IP address and start tap:
+* ``--tx-offloads=0x02c``
+
+    This testpmd option enables TX offloads for UDP and TCP checksum on transmit,
+    as well as TCP TSO support.
+    The list of the offload flag values can be seen in header `rte_ethdev.h
+    <https://doc.dpdk.org/api/rte__ethdev_8h.html>`_.
+
+* ``--enable-lro``
+
+    This testpmd option is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
+    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from the kernel can be
+    transmitted to the DPDK application and further TSOed by physical NIC.
+    If unsupported by the physical NIC, errors may be reported by testpmd with this option.
+
+
+* Enabling Rx checksum offloads for physical port:
+
+   Within testpmd, you can enable and disable offloads on a per-port basis,
+   rather than enabling them for both ports.
+   For the physical NIC, it may be desirable to enable checksum offload on packet RX.
+   This may be done as below, if testpmd is run with ``-i`` flag for interactive mode.

     .. code-block:: console

-        ifconfig tap0 1.1.1.1/24 up
+        testpmd> port stop 0
+        testpmd> port config 0 rx_offload tcp_cksum on
+        testpmd> port config 0 rx_offload udp_cksum on
+        testpmd> port start 0

-.. note::
+* Multiple queue support

-    The tap device will be named tap0, tap1, etc, by kernel.
+    Better performance may be achieved by using multiple queues,
+    so that multiple kernel threads are handling the traffic on the kernel side.
+    For example, to use 2 queues on both NIC and virtio ports,
+    while also enabling TX offloads and LRO support:

-Then, all traffic from physical NIC can be forwarded into kernel stack, and all
-traffic on the tap0 can be sent out from physical NIC.
+    .. code-block:: console

-Limitations
------------
+        /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 -- \
+            -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --txd=1024 --rxd=1024

-This solution is only available on Linux systems.
--
2.34.1

[PATCH v3 2/4] doc/howto: rename files to match new content name

[Bruce Richardson]

Since the section in the "howto" has been renamed from "exceptional
path" to "exception path", we can rename the relevant files to match
this new name too.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 ...ptional_path.svg => virtio_user_as_exception_path.svg} | 0
 doc/guides/howto/index.rst                                | 2 +-
 ...ptional_path.rst => virtio_user_as_exception_path.rst} | 8 ++++----
 3 files changed, 5 insertions(+), 5 deletions(-)
 rename doc/guides/howto/img/{virtio_user_as_exceptional_path.svg => virtio_user_as_exception_path.svg} (100%)
 rename doc/guides/howto/{virtio_user_as_exceptional_path.rst => virtio_user_as_exception_path.rst} (96%)

diff --git a/doc/guides/howto/img/virtio_user_as_exceptional_path.svg b/doc/guides/howto/img/virtio_user_as_exception_path.svg
similarity index 100%
rename from doc/guides/howto/img/virtio_user_as_exceptional_path.svg
rename to doc/guides/howto/img/virtio_user_as_exception_path.svg
diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst
index c2a2c60ddb..bf6337d021 100644
--- a/doc/guides/howto/index.rst
+++ b/doc/guides/howto/index.rst
@@ -15,7 +15,7 @@ HowTo Guides
     pvp_reference_benchmark
     vfd
     virtio_user_for_container_networking
-    virtio_user_as_exceptional_path
+    virtio_user_as_exception_path
     packet_capture_framework
     telemetry
     debug_troubleshoot
diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst b/doc/guides/howto/virtio_user_as_exception_path.rst
similarity index 96%
rename from doc/guides/howto/virtio_user_as_exceptional_path.rst
rename to doc/guides/howto/virtio_user_as_exception_path.rst
index 3d3d1b1219..a7f0d366fe 100644
--- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
+++ b/doc/guides/howto/virtio_user_as_exception_path.rst
@@ -1,7 +1,7 @@
 ..  SPDX-License-Identifier: BSD-3-Clause
     Copyright(c) 2016 Intel Corporation.
 
-.. _virtio_user_as_exceptional_path:
+.. _virtio_user_as_exception_path:
 
 Virtio_user as Exception Path
 =============================
@@ -39,11 +39,11 @@ This solution has a number of advantages over alternatives such as KNI:
     on the polling DPDK threads.
 
 The overview of an application using virtio-user as exception path is shown
-in :numref:`figure_virtio_user_as_exceptional_path`.
+in :numref:`figure_virtio_user_as_exception_path`.
 
-.. _figure_virtio_user_as_exceptional_path:
+.. _figure_virtio_user_as_exception_path:
 
-.. figure:: img/virtio_user_as_exceptional_path.*
+.. figure:: img/virtio_user_as_exception_path.*
 
    Overview of a DPDK app using virtio-user as exception path
 
-- 
2.34.1

[PATCH v3 3/4] doc/howto: add code example to virtio-user exception path doc

[Bruce Richardson]

The HOWTO guide for using virtio-user as an exception path to the kernel
only provided an example of how testpmd may be used for that purpose.
However, a real application wanting to use virtio-user as exception path
would likely want to create such devices from code within the app
itself. Therefore, we update the doc with instructions and a code
snippet showing how this may be done.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
 .../howto/virtio_user_as_exception_path.rst   | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/doc/guides/howto/virtio_user_as_exception_path.rst b/doc/guides/howto/virtio_user_as_exception_path.rst
index a7f0d366fe..2af1778578 100644
--- a/doc/guides/howto/virtio_user_as_exception_path.rst
+++ b/doc/guides/howto/virtio_user_as_exception_path.rst
@@ -157,3 +157,58 @@ For example:
         /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 -- \
             -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --txd=1024 --rxd=1024

+
+Creating Virtio-User Ports within an Application
+------------------------------------------------
+
+To use virtio-user ports within an application,
+it is not necessary to explicitly initialize those ports using EAL arguments at startup.
+Instead, one can use the generic EAL API
+`rte_eal_hotplug_add <https://doc.dpdk.org/api/rte__dev_8h.html#ad32e8eebf1f81ef9f290cb296b0c90bb>`_
+function to create a new instance at startup.
+For example, to create a basic virtio-user port, the following code could be used:
+
+.. code-block:: C
+
+   rte_eal_hotplug_add("vdev", "virtio_user0", "path=/dev/vhost-net");
+
+A fuller code example is shown below, where a virtio-user port, and hence kernel netdev,
+is created for each NIC port discovered by DPDK.
+Each virtio-user port is given the MAC address of its matching physical port
+(assuming app was run without vdev args on commandline, so all ports auto-discovered are HW ones).
+These new virtio-user netdevs will appear in the kernel port listings as ``virtio_user0``,
+``virtio_user1``, etc.,
+based on the names passed in as ``iface=`` via the ``portargs`` parameter.
+
+.. code-block:: C
+
+    nb_ports = rte_eth_dev_count_avail();
+
+    /* Create a vhost_user port for each physical port */
+    unsigned port_count = 0;
+    RTE_ETH_FOREACH_DEV(portid) {
+        char portname[32];
+        char portargs[256];
+        struct rte_ether_addr addr = {0};
+
+        /* don't create virtio_user ports for other virtio_user ports */
+        if (++port_count > nb_ports)
+            break;
+
+        /* get mac address of physical port to use as mac of virtio_user port */
+        rte_eth_macaddr_get(portid, &addr);
+
+        /* set the name and arguments */
+        snprintf(portname, sizeof(portname), "virtio_user%u", portid);
+        snprintf(portargs, sizeof(portargs),
+                "path=/dev/vhost-net,queues=1,queue_size=%u,iface=%s,mac=" RTE_ETHER_ADDR_PRT_FMT,
+                RX_RING_SIZE, portname, RTE_ETHER_ADDR_BYTES(&addr));
+
+        /* add the vdev for virtio_user */
+        if (rte_eal_hotplug_add("vdev", portname, portargs) < 0)
+            rte_exit(EXIT_FAILURE, "Cannot create paired port for port %u\n", portid);
+
+    }
+
+Once these virtio-user ports have been created in the loop, all ports, both physical and virtual,
+may be initialized and used as normal in the application.
--
2.34.1

[PATCH v3 4/4] doc/prog_guide: add reference to virtio-user from KNI doc

[Bruce Richardson]

To help encourage use of virtio-user in place of KNI, put a reference to
the relevant howto section at the top of the KNI doc.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/guides/prog_guide/kernel_nic_interface.rst | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst
index 5248f9622b..e021cc69b6 100644
--- a/doc/guides/prog_guide/kernel_nic_interface.rst
+++ b/doc/guides/prog_guide/kernel_nic_interface.rst
@@ -6,6 +6,12 @@
 Kernel NIC Interface
 ====================
 
+.. note::
+
+   For an alternative to KNI, that does not require any out-of-tree Linux kernel modules,
+   or a custom library, see :ref:`virtio_user_as_exception_path`.
+
+
 The DPDK Kernel NIC Interface (KNI) allows userspace applications access to the Linux* control plane.
 
 The benefits of using the DPDK KNI are:
-- 
2.34.1

RE: [PATCH v3 0/4] Enhance docs on virtio-user as KNI replacement

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Friday, 10 June 2022 17.36
> 
> Since use of KNI is no longer advised (due to known issues and the fact
> it's an out-of-tree module), virtio-user is recommended for new
> developments. To help encourage this, we improve the doc HOWTO section
> on using virtio-user and add a link to that document from the main KNI
> section in the programmers guide.
> 
> v3:
> * Added two extra patches:
>   - rename document files to match new section name
>   - add note to KNI programmers guide section
> * minor updates to patch 2, and added stable on CC for that patch
> 
> Bruce Richardson (4):
>   doc/howto: rework section on virtio-user as exception path
>   doc/howto: rename files to match new content name
>   doc/howto: add code example to virtio-user exception path doc
>   doc/prog_guide: add reference to virtio-user from KNI doc
> 
>  ....svg => virtio_user_as_exception_path.svg} |   0
>  doc/guides/howto/index.rst                    |   2 +-
>  .../howto/virtio_user_as_exception_path.rst   | 214 ++++++++++++++++++
>  .../howto/virtio_user_as_exceptional_path.rst | 118 ----------
>  .../prog_guide/kernel_nic_interface.rst       |   6 +
>  5 files changed, 221 insertions(+), 119 deletions(-)
>  rename doc/guides/howto/img/{virtio_user_as_exceptional_path.svg =>
> virtio_user_as_exception_path.svg} (100%)
>  create mode 100644 doc/guides/howto/virtio_user_as_exception_path.rst
>  delete mode 100644
> doc/guides/howto/virtio_user_as_exceptional_path.rst
> 
> --
> 2.34.1
> 

Yes.

Series-Acked-by: Morten Brørup <mb@smartsharesystems.com>

RE: [PATCH v3 1/4] doc/howto: rework section on virtio-user as exception path

[Xia, Chenbo]

Hi Bruce,

> -----Original Message-----
> From: Richardson, Bruce <bruce.richardson@intel.com>
> Sent: Friday, June 10, 2022 11:36 PM
> To: dev@dpdk.org
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> stable@dpdk.org
> Subject: [PATCH v3 1/4] doc/howto: rework section on virtio-user as
> exception path
> 
> This patch extensively reworks the howto guide on using virtio-user for
> exception packets. Changes include:
> 
> * rename "exceptional path" to "exception path"
> * remove references to uio and just reference vfio-pci
> * simplify testpmd command-lines, giving a basic usage example first
>   before adding on detail about checksum or TSO parameters
> * give a complete working example showing traffic flowing through the
>   whole system from a testpmd loopback using the created TAP netdev
> * replace use of "ifconfig" with Linux standard "ip" command
> * other general rewording.
> 
> CC: stable@dpdk.org

Not sure you want to backport or not as there's no fixes tag.

For this patch:

Acked-by: Chenbo Xia <chenbo.xia@intel.com>

> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> ---
> V3:
> * fix error reported by Chenbo on review.
> * add stable on CC, since this rework could be applicable for older
>   releases too, if desired for backport.
> ---
>  .../howto/virtio_user_as_exceptional_path.rst | 159 +++++++++++-------
>  1 file changed, 100 insertions(+), 59 deletions(-)
> 
> diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> index ec021af399..3d3d1b1219 100644
> --- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> +++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> @@ -3,116 +3,157 @@
> 
>  .. _virtio_user_as_exceptional_path:
> 
> -Virtio_user as Exceptional Path
> -===============================
> +Virtio_user as Exception Path
> +=============================
> 
> -The virtual device, virtio-user, was originally introduced with vhost-
> user
> -backend, as a high performance solution for IPC (Inter-Process
> Communication)
> +.. note::
> +
> +   This solution is only applicable to Linux systems.
> +
> +The virtual device, virtio-user, was originally introduced with the
> vhost-user
> +backend as a high performance solution for IPC (Inter-Process
> Communication)
>  and user space container networking.
> 
> -Virtio_user with vhost-kernel backend is a solution for exceptional path,
> -such as KNI which exchanges packets with kernel networking stack. This
> -solution is very promising in:
> +Beyond this originally intended use, virtio-user can be used in
> conjunction with the vhost-kernel
> +backend as a solution for dealing with exception path packets which need
> to be injected into the
> +Linux kernel for processing there.
> +In this regard, virtio-user and vhost in kernel space are an alternative
> to DPDK KNI for
> +transferring packets between a DPDK packet processing application and the
> kernel stack.
> +
> +This solution has a number of advantages over alternatives such as KNI:
> 
>  *   Maintenance
> 
>      All kernel modules needed by this solution, vhost and vhost-net
> (kernel),
> -    are upstreamed and extensively used kernel module.
> +    are upstreamed and extensively used.
> 
>  *   Features
> 
> -    vhost-net is born to be a networking solution, which has lots of
> networking
> -    related features, like multi queue, tso, multi-seg mbuf, etc.
> +    vhost-net is designed to be a networking solution, and, as such, has
> lots of networking
> +    related features, such as multi queue support, TSO, multi-segment
> buffer support, etc.
> 
>  *   Performance
> 
> -    similar to KNI, this solution would use one or more kthreads to
> -    send/receive packets to/from user space DPDK applications, which has
> little
> -    impact on user space polling thread (except that it might enter into
> kernel
> -    space to wake up those kthreads if necessary).
> +    similar to KNI, this solution would uses one or more kthreads to
> +    send/receive packets to/from user space DPDK applications, which
> minimises the impact
> +    on the polling DPDK threads.
> 
> -The overview of an application using virtio-user as exceptional path is
> shown
> +The overview of an application using virtio-user as exception path is
> shown
>  in :numref:`figure_virtio_user_as_exceptional_path`.
> 
>  .. _figure_virtio_user_as_exceptional_path:
> 
>  .. figure:: img/virtio_user_as_exceptional_path.*
> 
> -   Overview of a DPDK app using virtio-user as exceptional path
> +   Overview of a DPDK app using virtio-user as exception path
> +
> 
> +Example Usage With Testpmd
> +---------------------------
> 
> -Sample Usage
> -------------
> +.. note::
> +
> +   These instruction assume that the vhost/vhost-net kernel modules are
> available and have already
> +   been loaded into the running kernel.
> +   It also assumes that the DPDK virtio driver has not been disabled in
> the DPDK build.
> 
> -As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen
> before
> -compiling the kernel and those kernel modules should be inserted.
> +To run a simple test of virtio-user as exception path using testpmd:
> 
> -#.  Compile DPDK and bind a physical NIC to igb_uio/uio_pci_generic/vfio-
> pci.
> +#.  Compile DPDK and bind a NIC to vfio-pci as documented
> in :ref:`linux_gsg_linux_drivers`.
> 
> -    This physical NIC is for communicating with outside.
> +    This physical NIC is for communicating with the outside world,
> +    and serves as a packet source in this example.
> 
> -#.  Run testpmd.
> +#.  Run testpmd to forward packets from NIC to kernel,
> +    passing in a suitable list of logical cores to run on  (``-l``
> parameter),
> +    and optionally the PCI address of the physical NIC to use (``-a``
> parameter).
> +    The virtio-user device for interfacing to the kernel is specified via
> a ``-vdev`` argument,
> +    taking the parameters described below.
> 
>      .. code-block:: console
> 
> -        $(testpmd) -l 2-3 -n 4 \
> -		--vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 \
> -		-- -i --tx-offloads=0x0000002c --enable-lro \
> -		--txd=1024 --rxd=1024
> +        /path/to/dpdk-testpmd -l <cores> -a <pci BDF> \
> +            --vdev=virtio_user0,path=/dev/vhost-
> net,queues=1,queue_size=1024
> 
> -    This command runs testpmd with two ports, one physical NIC to
> communicate
> -    with outside, and one virtio-user to communicate with kernel.
> +    * ``path``
> 
> -* ``--enable-lro``
> +       The path to the kernel vhost-net device.
> 
> -    This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
> -    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can
> be
> -    transmitted to DPDK application and further TSOed by physical NIC.
> +    * ``queue_size``
> 
> -* ``queue_size``
> +       256 by default. To avoid shortage of descriptors, we can increase
> it to 1024.
> 
> -    256 by default. To avoid shortage of descriptors, we can increase it
> to 1024.
> +    * ``queues``
> 
> -* ``queues``
> +       Number of virt-queues. Each queue will be served by a kthread.
> 
> -    Number of multi-queues. Each queue will be served by a kthread. For
> example:
> +#. Once testpmd is running, a new network interface - called ``tap0`` by
> default -
> +   will be present on the system.
> +   This should be configured with an IP address and then enabled for use:
> 
>      .. code-block:: console
> 
> -        $(testpmd) -l 2-3 -n 4 \
> -		--vdev=virtio_user0,path=/dev/vhost-
> net,queues=2,queue_size=1024 \
> -		-- -i --tx-offloads=0x0000002c --enable-lro \
> -		--txq=2 --rxq=2 --txd=1024 --rxd=1024
> +        ip addr add 192.168.1.1/24 dev tap0
> +        ip link set dev tap0 up
> 
> -#. Enable Rx checksum offloads in testpmd:
> +#. To observe packet forwarding through the kernel,
> +   a second testpmd instance can be run on the system,
> +   taking packets from the kernel using an ``af_packet`` socket on the
> ``tap0`` interface.
> 
> -    .. code-block:: console
> +   .. code-block:: console
> 
> -        (testpmd) port stop 0
> -        (testpmd) port config 0 rx_offload tcp_cksum on
> -        (testpmd) port config 0 rx_offload udp_cksum on
> -        (testpmd) port start 0
> +       /path/to/dpdk-testpmd -l <cores> --vdev=net_af_packet0,iface=tap0
> --in-memory --no-pci
> 
> -#. Start testpmd:
> +   When running this instance,
> +   we can use ``--in-memory`` flag to avoid hugepage naming conflicts
> with the previous instance,
> +   and we also use ``--no-pci`` flag to only use the ``af_packet``
> interface for all traffic forwarding.
> 
> -    .. code-block:: console
> +#. Running traffic into the system through the NIC should see that
> traffic returned back again,
> +   having been forwarded through both testpmd instances.
> +   This can be confirmed by checking the testpmd statistics on testpmd
> exit.
> 
> -        (testpmd) start
> +For more advanced use of virtio-user with testpmd in this scenario,
> +some other more advanced options may also be used.
> +For example:
> 
> -#.  Configure IP address and start tap:
> +* ``--tx-offloads=0x02c``
> +
> +    This testpmd option enables TX offloads for UDP and TCP checksum on
> transmit,
> +    as well as TCP TSO support.
> +    The list of the offload flag values can be seen in header
> `rte_ethdev.h
> +    <https://doc.dpdk.org/api/rte__ethdev_8h.html>`_.
> +
> +* ``--enable-lro``
> +
> +    This testpmd option is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
> +    VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from the kernel
> can be
> +    transmitted to the DPDK application and further TSOed by physical NIC.
> +    If unsupported by the physical NIC, errors may be reported by testpmd
> with this option.
> +
> +
> +* Enabling Rx checksum offloads for physical port:
> +
> +   Within testpmd, you can enable and disable offloads on a per-port
> basis,
> +   rather than enabling them for both ports.
> +   For the physical NIC, it may be desirable to enable checksum offload
> on packet RX.
> +   This may be done as below, if testpmd is run with ``-i`` flag for
> interactive mode.
> 
>      .. code-block:: console
> 
> -        ifconfig tap0 1.1.1.1/24 up
> +        testpmd> port stop 0
> +        testpmd> port config 0 rx_offload tcp_cksum on
> +        testpmd> port config 0 rx_offload udp_cksum on
> +        testpmd> port start 0
> 
> -.. note::
> +* Multiple queue support
> 
> -    The tap device will be named tap0, tap1, etc, by kernel.
> +    Better performance may be achieved by using multiple queues,
> +    so that multiple kernel threads are handling the traffic on the
> kernel side.
> +    For example, to use 2 queues on both NIC and virtio ports,
> +    while also enabling TX offloads and LRO support:
> 
> -Then, all traffic from physical NIC can be forwarded into kernel stack,
> and all
> -traffic on the tap0 can be sent out from physical NIC.
> +    .. code-block:: console
> 
> -Limitations
> ------------
> +        /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-
> net,queues=2,queue_size=1024 -- \
> +            -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --
> txd=1024 --rxd=1024
> 
> -This solution is only available on Linux systems.
> --
> 2.34.1

RE: [PATCH v3 2/4] doc/howto: rename files to match new content name

[Xia, Chenbo]

> -----Original Message-----
> From: Richardson, Bruce <bruce.richardson@intel.com>
> Sent: Friday, June 10, 2022 11:36 PM
> To: dev@dpdk.org
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: [PATCH v3 2/4] doc/howto: rename files to match new content name
> 
> Since the section in the "howto" has been renamed from "exceptional
> path" to "exception path", we can rename the relevant files to match
> this new name too.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  ...ptional_path.svg => virtio_user_as_exception_path.svg} | 0
>  doc/guides/howto/index.rst                                | 2 +-
>  ...ptional_path.rst => virtio_user_as_exception_path.rst} | 8 ++++----
>  3 files changed, 5 insertions(+), 5 deletions(-)
>  rename doc/guides/howto/img/{virtio_user_as_exceptional_path.svg =>
> virtio_user_as_exception_path.svg} (100%)
>  rename doc/guides/howto/{virtio_user_as_exceptional_path.rst =>
> virtio_user_as_exception_path.rst} (96%)
> 
> diff --git a/doc/guides/howto/img/virtio_user_as_exceptional_path.svg
> b/doc/guides/howto/img/virtio_user_as_exception_path.svg
> similarity index 100%
> rename from doc/guides/howto/img/virtio_user_as_exceptional_path.svg
> rename to doc/guides/howto/img/virtio_user_as_exception_path.svg
> diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst
> index c2a2c60ddb..bf6337d021 100644
> --- a/doc/guides/howto/index.rst
> +++ b/doc/guides/howto/index.rst
> @@ -15,7 +15,7 @@ HowTo Guides
>      pvp_reference_benchmark
>      vfd
>      virtio_user_for_container_networking
> -    virtio_user_as_exceptional_path
> +    virtio_user_as_exception_path
>      packet_capture_framework
>      telemetry
>      debug_troubleshoot
> diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> b/doc/guides/howto/virtio_user_as_exception_path.rst
> similarity index 96%
> rename from doc/guides/howto/virtio_user_as_exceptional_path.rst
> rename to doc/guides/howto/virtio_user_as_exception_path.rst
> index 3d3d1b1219..a7f0d366fe 100644
> --- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> +++ b/doc/guides/howto/virtio_user_as_exception_path.rst
> @@ -1,7 +1,7 @@
>  ..  SPDX-License-Identifier: BSD-3-Clause
>      Copyright(c) 2016 Intel Corporation.
> 
> -.. _virtio_user_as_exceptional_path:
> +.. _virtio_user_as_exception_path:
> 
>  Virtio_user as Exception Path
>  =============================
> @@ -39,11 +39,11 @@ This solution has a number of advantages over
> alternatives such as KNI:
>      on the polling DPDK threads.
> 
>  The overview of an application using virtio-user as exception path is
> shown
> -in :numref:`figure_virtio_user_as_exceptional_path`.
> +in :numref:`figure_virtio_user_as_exception_path`.
> 
> -.. _figure_virtio_user_as_exceptional_path:
> +.. _figure_virtio_user_as_exception_path:
> 
> -.. figure:: img/virtio_user_as_exceptional_path.*
> +.. figure:: img/virtio_user_as_exception_path.*
> 
>     Overview of a DPDK app using virtio-user as exception path
> 
> --
> 2.34.1

Acked-by: Chenbo Xia <chenbo.xia@intel.com>

RE: [PATCH v3 3/4] doc/howto: add code example to virtio-user exception path doc

[Xia, Chenbo]

> -----Original Message-----
> From: Richardson, Bruce <bruce.richardson@intel.com>
> Sent: Friday, June 10, 2022 11:36 PM
> To: dev@dpdk.org
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> Stephen Hemminger <stephen@networkplumber.org>
> Subject: [PATCH v3 3/4] doc/howto: add code example to virtio-user
> exception path doc
> 
> The HOWTO guide for using virtio-user as an exception path to the kernel
> only provided an example of how testpmd may be used for that purpose.
> However, a real application wanting to use virtio-user as exception path
> would likely want to create such devices from code within the app
> itself. Therefore, we update the doc with instructions and a code
> snippet showing how this may be done.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
>  .../howto/virtio_user_as_exception_path.rst   | 55 +++++++++++++++++++
>  1 file changed, 55 insertions(+)
> 
> diff --git a/doc/guides/howto/virtio_user_as_exception_path.rst
> b/doc/guides/howto/virtio_user_as_exception_path.rst
> index a7f0d366fe..2af1778578 100644
> --- a/doc/guides/howto/virtio_user_as_exception_path.rst
> +++ b/doc/guides/howto/virtio_user_as_exception_path.rst
> @@ -157,3 +157,58 @@ For example:
>          /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-
> net,queues=2,queue_size=1024 -- \
>              -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --
> txd=1024 --rxd=1024
> 
> +
> +Creating Virtio-User Ports within an Application
> +------------------------------------------------
> +
> +To use virtio-user ports within an application,
> +it is not necessary to explicitly initialize those ports using EAL
> arguments at startup.
> +Instead, one can use the generic EAL API
> +`rte_eal_hotplug_add
> <https://doc.dpdk.org/api/rte__dev_8h.html#ad32e8eebf1f81ef9f290cb296b0c90
> bb>`_
> +function to create a new instance at startup.
> +For example, to create a basic virtio-user port, the following code could
> be used:
> +
> +.. code-block:: C
> +
> +   rte_eal_hotplug_add("vdev", "virtio_user0", "path=/dev/vhost-net");
> +
> +A fuller code example is shown below, where a virtio-user port, and hence
> kernel netdev,
> +is created for each NIC port discovered by DPDK.
> +Each virtio-user port is given the MAC address of its matching physical
> port
> +(assuming app was run without vdev args on commandline, so all ports
> auto-discovered are HW ones).
> +These new virtio-user netdevs will appear in the kernel port listings as
> ``virtio_user0``,
> +``virtio_user1``, etc.,
> +based on the names passed in as ``iface=`` via the ``portargs`` parameter.
> +
> +.. code-block:: C
> +
> +    nb_ports = rte_eth_dev_count_avail();
> +
> +    /* Create a vhost_user port for each physical port */
> +    unsigned port_count = 0;
> +    RTE_ETH_FOREACH_DEV(portid) {
> +        char portname[32];
> +        char portargs[256];
> +        struct rte_ether_addr addr = {0};
> +
> +        /* don't create virtio_user ports for other virtio_user ports */
> +        if (++port_count > nb_ports)
> +            break;
> +
> +        /* get mac address of physical port to use as mac of virtio_user
> port */
> +        rte_eth_macaddr_get(portid, &addr);
> +
> +        /* set the name and arguments */
> +        snprintf(portname, sizeof(portname), "virtio_user%u", portid);
> +        snprintf(portargs, sizeof(portargs),
> +                "path=/dev/vhost-
> net,queues=1,queue_size=%u,iface=%s,mac=" RTE_ETHER_ADDR_PRT_FMT,
> +                RX_RING_SIZE, portname, RTE_ETHER_ADDR_BYTES(&addr));
> +
> +        /* add the vdev for virtio_user */
> +        if (rte_eal_hotplug_add("vdev", portname, portargs) < 0)
> +            rte_exit(EXIT_FAILURE, "Cannot create paired port for
> port %u\n", portid);
> +
> +    }
> +
> +Once these virtio-user ports have been created in the loop, all ports,
> both physical and virtual,
> +may be initialized and used as normal in the application.
> --
> 2.34.1

Acked-by: Chenbo Xia <chenbo.xia@intel.com>

RE: [PATCH v3 4/4] doc/prog_guide: add reference to virtio-user from KNI doc

[Xia, Chenbo]

> -----Original Message-----
> From: Richardson, Bruce <bruce.richardson@intel.com>
> Sent: Friday, June 10, 2022 11:36 PM
> To: dev@dpdk.org
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: [PATCH v3 4/4] doc/prog_guide: add reference to virtio-user from
> KNI doc
> 
> To help encourage use of virtio-user in place of KNI, put a reference to
> the relevant howto section at the top of the KNI doc.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  doc/guides/prog_guide/kernel_nic_interface.rst | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst
> b/doc/guides/prog_guide/kernel_nic_interface.rst
> index 5248f9622b..e021cc69b6 100644
> --- a/doc/guides/prog_guide/kernel_nic_interface.rst
> +++ b/doc/guides/prog_guide/kernel_nic_interface.rst
> @@ -6,6 +6,12 @@
>  Kernel NIC Interface
>  ====================
> 
> +.. note::
> +
> +   For an alternative to KNI, that does not require any out-of-tree Linux
> kernel modules,
> +   or a custom library, see :ref:`virtio_user_as_exception_path`.
> +
> +
>  The DPDK Kernel NIC Interface (KNI) allows userspace applications access
> to the Linux* control plane.
> 
>  The benefits of using the DPDK KNI are:
> --
> 2.34.1

Acked-by: Chenbo Xia <chenbo.xia@intel.com>

Re: [PATCH v3 1/4] doc/howto: rework section on virtio-user as exception path

[Maxime Coquelin]

On 6/10/22 17:35, Bruce Richardson wrote:
> This patch extensively reworks the howto guide on using virtio-user for
> exception packets. Changes include:
> 
> * rename "exceptional path" to "exception path"
> * remove references to uio and just reference vfio-pci
> * simplify testpmd command-lines, giving a basic usage example first
>    before adding on detail about checksum or TSO parameters
> * give a complete working example showing traffic flowing through the
>    whole system from a testpmd loopback using the created TAP netdev
> * replace use of "ifconfig" with Linux standard "ip" command
> * other general rewording.
> 
> CC: stable@dpdk.org
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> ---
> V3:
> * fix error reported by Chenbo on review.
> * add stable on CC, since this rework could be applicable for older
>    releases too, if desired for backport.
> ---
>   .../howto/virtio_user_as_exceptional_path.rst | 159 +++++++++++-------
>   1 file changed, 100 insertions(+), 59 deletions(-)
> 

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

Re: [PATCH v3 2/4] doc/howto: rename files to match new content name

[Maxime Coquelin]

On 6/10/22 17:35, Bruce Richardson wrote:
> Since the section in the "howto" has been renamed from "exceptional
> path" to "exception path", we can rename the relevant files to match
> this new name too.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>   ...ptional_path.svg => virtio_user_as_exception_path.svg} | 0
>   doc/guides/howto/index.rst                                | 2 +-
>   ...ptional_path.rst => virtio_user_as_exception_path.rst} | 8 ++++----
>   3 files changed, 5 insertions(+), 5 deletions(-)
>   rename doc/guides/howto/img/{virtio_user_as_exceptional_path.svg => virtio_user_as_exception_path.svg} (100%)
>   rename doc/guides/howto/{virtio_user_as_exceptional_path.rst => virtio_user_as_exception_path.rst} (96%)
> 

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

Re: [PATCH v3 3/4] doc/howto: add code example to virtio-user exception path doc

[Maxime Coquelin]

On 6/10/22 17:35, Bruce Richardson wrote:
> The HOWTO guide for using virtio-user as an exception path to the kernel
> only provided an example of how testpmd may be used for that purpose.
> However, a real application wanting to use virtio-user as exception path
> would likely want to create such devices from code within the app
> itself. Therefore, we update the doc with instructions and a code
> snippet showing how this may be done.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
>   .../howto/virtio_user_as_exception_path.rst   | 55 +++++++++++++++++++
>   1 file changed, 55 insertions(+)
> 

Nice addition!

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

Re: [PATCH v3 4/4] doc/prog_guide: add reference to virtio-user from KNI doc

[Maxime Coquelin]

On 6/10/22 17:35, Bruce Richardson wrote:
> To help encourage use of virtio-user in place of KNI, put a reference to
> the relevant howto section at the top of the KNI doc.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>   doc/guides/prog_guide/kernel_nic_interface.rst | 6 ++++++
>   1 file changed, 6 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst
> index 5248f9622b..e021cc69b6 100644
> --- a/doc/guides/prog_guide/kernel_nic_interface.rst
> +++ b/doc/guides/prog_guide/kernel_nic_interface.rst
> @@ -6,6 +6,12 @@
>   Kernel NIC Interface
>   ====================
>   
> +.. note::
> +
> +   For an alternative to KNI, that does not require any out-of-tree Linux kernel modules,
> +   or a custom library, see :ref:`virtio_user_as_exception_path`.
> +
> +
>   The DPDK Kernel NIC Interface (KNI) allows userspace applications access to the Linux* control plane.
>   
>   The benefits of using the DPDK KNI are:

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

Re: [PATCH v3 0/4] Enhance docs on virtio-user as KNI replacement

[Maxime Coquelin]

On 6/10/22 17:35, Bruce Richardson wrote:
> Since use of KNI is no longer advised (due to known issues and the fact
> it's an out-of-tree module), virtio-user is recommended for new
> developments. To help encourage this, we improve the doc HOWTO section
> on using virtio-user and add a link to that document from the main KNI
> section in the programmers guide.
> 
> v3:
> * Added two extra patches:
>    - rename document files to match new section name
>    - add note to KNI programmers guide section
> * minor updates to patch 2, and added stable on CC for that patch
> 
> Bruce Richardson (4):
>    doc/howto: rework section on virtio-user as exception path
>    doc/howto: rename files to match new content name
>    doc/howto: add code example to virtio-user exception path doc
>    doc/prog_guide: add reference to virtio-user from KNI doc
> 
>   ....svg => virtio_user_as_exception_path.svg} |   0
>   doc/guides/howto/index.rst                    |   2 +-
>   .../howto/virtio_user_as_exception_path.rst   | 214 ++++++++++++++++++
>   .../howto/virtio_user_as_exceptional_path.rst | 118 ----------
>   .../prog_guide/kernel_nic_interface.rst       |   6 +
>   5 files changed, 221 insertions(+), 119 deletions(-)
>   rename doc/guides/howto/img/{virtio_user_as_exceptional_path.svg => virtio_user_as_exception_path.svg} (100%)
>   create mode 100644 doc/guides/howto/virtio_user_as_exception_path.rst
>   delete mode 100644 doc/guides/howto/virtio_user_as_exceptional_path.rst
> 
> --
> 2.34.1
> 

Applied to dpdk-next-virtio/main.

Thanks,
Maxime

Re: [PATCH v3 1/4] doc/howto: rework section on virtio-user as exception path

[Thomas Monjalon]

10/06/2022 17:35, Bruce Richardson:
> This patch extensively reworks the howto guide on using virtio-user for
> exception packets. Changes include:
> 
> * rename "exceptional path" to "exception path"
> * remove references to uio and just reference vfio-pci
> * simplify testpmd command-lines, giving a basic usage example first
>   before adding on detail about checksum or TSO parameters
> * give a complete working example showing traffic flowing through the
>   whole system from a testpmd loopback using the created TAP netdev
> * replace use of "ifconfig" with Linux standard "ip" command
> * other general rewording.
> 
> CC: stable@dpdk.org

I don't think we should keep this for backport.
This is an enhancement.
Anyway new developments are probably not looking in old branches.

Re: [PATCH v3 1/4] doc/howto: rework section on virtio-user as exception path

[Bruce Richardson]

On Mon, Jul 11, 2022 at 03:10:56PM +0200, Thomas Monjalon wrote:
> 10/06/2022 17:35, Bruce Richardson:
> > This patch extensively reworks the howto guide on using virtio-user for
> > exception packets. Changes include:
> > 
> > * rename "exceptional path" to "exception path"
> > * remove references to uio and just reference vfio-pci
> > * simplify testpmd command-lines, giving a basic usage example first
> >   before adding on detail about checksum or TSO parameters
> > * give a complete working example showing traffic flowing through the
> >   whole system from a testpmd loopback using the created TAP netdev
> > * replace use of "ifconfig" with Linux standard "ip" command
> > * other general rewording.
> > 
> > CC: stable@dpdk.org
> 
> I don't think we should keep this for backport.
> This is an enhancement.
> Anyway new developments are probably not looking in old branches.
> 
Sure, that's fine with me.