From: "Xia, Chenbo" <chenbo.xia@intel.com>
To: "Richardson, Bruce" <bruce.richardson@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
Subject: RE: [PATCH v2 1/2] doc/howto: rework section on virtio-user as exception path
Date: Mon, 30 May 2022 05:33:14 +0000 [thread overview]
Message-ID: <SN6PR11MB3504ECD484225EF41C43E9819CDD9@SN6PR11MB3504.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20220527163643.130679-1-bruce.richardson@intel.com>
Hi Bruce,
> -----Original Message-----
> From: Richardson, Bruce <bruce.richardson@intel.com>
> Sent: Saturday, May 28, 2022 12:37 AM
> To: dev@dpdk.org
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: [PATCH v2 1/2] doc/howto: rework section on virtio-user as
> exception path
>
> This patch extensively reworks the howto guide on using virtio-user for
> exception packets. Changes include:
>
> * rename "exceptional path" to "exception path"
> * remove references to uio and just reference vfio-pci
> * simplify testpmd command-lines, giving a basic usage example first
> before adding on detail about checksum or TSO parameters
> * give a complete working example showing traffic flowing through the
> whole system from a testpmd loopback using the created TAP netdev
> * replace use of "ifconfig" with Linux standard "ip" command
> * other general rewording.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
>
> ---
> v2: fix checkpatch warnings
> ---
> .../howto/virtio_user_as_exceptional_path.rst | 159 +++++++++++-------
I agree with the renaming. And should we rename this file, and if possible, the
attached img too.
> 1 file changed, 100 insertions(+), 59 deletions(-)
>
> diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> index ec021af399..100376c32d 100644
> --- a/doc/guides/howto/virtio_user_as_exceptional_path.rst
> +++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst
> @@ -3,116 +3,157 @@
>
> .. _virtio_user_as_exceptional_path:
>
> -Virtio_user as Exceptional Path
> -===============================
> +Virtio_user as Exception Path
> +=============================
>
> -The virtual device, virtio-user, was originally introduced with vhost-
> user
> -backend, as a high performance solution for IPC (Inter-Process
> Communication)
> +.. note::
> +
> + This solution is only applicable to Linux systems.
> +
> +The virtual device, virtio-user, was originally introduced with the
> vhost-user
> +backend as a high performance solution for IPC (Inter-Process
> Communication)
> and user space container networking.
>
> -Virtio_user with vhost-kernel backend is a solution for exceptional path,
> -such as KNI which exchanges packets with kernel networking stack. This
> -solution is very promising in:
> +Beyond this originally intended use, virtio-user can be used in
> conjunction with the vhost-kernel
> +backend as a solution for dealing with exception path packets which need
> to be injected into the
> +Linux kernel for processing there.
> +In this regard, virtio-user and vhost in kernel space are an alternative
> to DPDK KNI for
> +transferring packets between a DPDK packet processing application and the
> kernel stack.
> +
> +This solution has a number of advantages over alternatives such as KNI:
>
> * Maintenance
>
> All kernel modules needed by this solution, vhost and vhost-net
> (kernel),
> - are upstreamed and extensively used kernel module.
> + are upstreamed and extensively used.
>
> * Features
>
> - vhost-net is born to be a networking solution, which has lots of
> networking
> - related features, like multi queue, tso, multi-seg mbuf, etc.
> + vhost-net is designed to be a networking solution, and, as such, has
> lots of networking
> + related features, such as multi queue support, TSO, multi-segment
> buffer support, etc.
>
> * Performance
>
> - similar to KNI, this solution would use one or more kthreads to
> - send/receive packets to/from user space DPDK applications, which has
> little
> - impact on user space polling thread (except that it might enter into
> kernel
> - space to wake up those kthreads if necessary).
> + similar to KNI, this solution would uses one or more kthreads to
> + send/receive packets to/from user space DPDK applications, which
> minimises the impact
> + on the polling DPDK threads.
>
> -The overview of an application using virtio-user as exceptional path is
> shown
> +The overview of an application using virtio-user as exception path is
> shown
> in :numref:`figure_virtio_user_as_exceptional_path`.
>
> .. _figure_virtio_user_as_exceptional_path:
>
> .. figure:: img/virtio_user_as_exceptional_path.*
>
> - Overview of a DPDK app using virtio-user as exceptional path
> + Overview of a DPDK app using virtio-user as exception path
> +
>
> +Example Usage With Testpmd
> +---------------------------
>
> -Sample Usage
> -------------
> +.. note::
> +
> + These instruction assume that the vhost/vhost-net kernel modules are
> available and have already
> + been loaded into the running kernel.
> + It also assumes that the DPDK virtio driver has not been disabled in
> the DPDK build.
>
> -As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen
> before
> -compiling the kernel and those kernel modules should be inserted.
> +To run a simple test of virtio-user as exception path using testpmd:
>
> -#. Compile DPDK and bind a physical NIC to igb_uio/uio_pci_generic/vfio-
> pci.
> +#. Compile DPDK and bind a NIC to vfio-pci as documented
> in :ref:`linux_gsg_linux_drivers`.
>
> - This physical NIC is for communicating with outside.
> + This physical NIC is for communicating with the outside world,
> + and serves as a packet source in this example.
>
> -#. Run testpmd.
> +#. Run testpmd to forward packets from NIC to kernel,
> + passing in a suitable list of logical cores to run on (``-
> l``.parameter),
This '.' should be a space?
Overall, the changes LGTM.
Thanks!
Chenbo
> + and optionally the PCI address of the physical NIC to use (``-a``
> parameter).
> + The virtio-user device for interfacing to the kernel is specified via
> a ``-vdev`` argument,
> + taking the parameters described below.
>
> .. code-block:: console
>
> - $(testpmd) -l 2-3 -n 4 \
> - --vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 \
> - -- -i --tx-offloads=0x0000002c --enable-lro \
> - --txd=1024 --rxd=1024
> + /path/to/dpdk-testpmd -l <cores> -a <pci BDF> \
> + --vdev=virtio_user0,path=/dev/vhost-
> net,queues=1,queue_size=1024
>
> - This command runs testpmd with two ports, one physical NIC to
> communicate
> - with outside, and one virtio-user to communicate with kernel.
> + * ``path``
>
> -* ``--enable-lro``
> + The path to the kernel vhost-net device.
>
> - This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
> - VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can
> be
> - transmitted to DPDK application and further TSOed by physical NIC.
> + * ``queue_size``
>
> -* ``queue_size``
> + 256 by default. To avoid shortage of descriptors, we can increase
> it to 1024.
>
> - 256 by default. To avoid shortage of descriptors, we can increase it
> to 1024.
> + * ``queues``
>
> -* ``queues``
> + Number of virt-queues. Each queue will be served by a kthread.
>
> - Number of multi-queues. Each queue will be served by a kthread. For
> example:
> +#. Once testpmd is running, a new network interface - called ``tap0`` by
> default -
> + will be present on the system.
> + This should be configured with an IP address and then enabled for use:
>
> .. code-block:: console
>
> - $(testpmd) -l 2-3 -n 4 \
> - --vdev=virtio_user0,path=/dev/vhost-
> net,queues=2,queue_size=1024 \
> - -- -i --tx-offloads=0x0000002c --enable-lro \
> - --txq=2 --rxq=2 --txd=1024 --rxd=1024
> + ip addr add 192.168.1.1/24 dev tap0
> + ip link set dev tap0 up
>
> -#. Enable Rx checksum offloads in testpmd:
> +#. To observe packet forwarding through the kernel,
> + a second testpmd instance can be run on the system,
> + taking packets from the kernel using an ``af_packet`` socket on the
> ``tap0`` interface.
>
> - .. code-block:: console
> + .. code-block:: console
>
> - (testpmd) port stop 0
> - (testpmd) port config 0 rx_offload tcp_cksum on
> - (testpmd) port config 0 rx_offload udp_cksum on
> - (testpmd) port start 0
> + /path/to/dpdk-testpmd -l <cores> --vdev=net_af_packet0,iface=tap0
> --in-memory --no-pci
>
> -#. Start testpmd:
> + When running this instance,
> + we can use ``--in-memory`` flag to avoid hugepage naming conflicts
> with the previous instance,
> + and we also use ``--no-pci`` flag to only use the ``af_packet``
> interface for all traffic forwarding.
>
> - .. code-block:: console
> +#. Running traffic into the system through the NIC should see that
> traffic returned back again,
> + having been forwarded through both testpmd instances.
> + This can be confirmed by checking the testpmd statistics on testpmd
> exit.
>
> - (testpmd) start
> +For more advanced use of virtio-user with testpmd in this scenario,
> +some other more advanced options may also be used.
> +For example:
>
> -#. Configure IP address and start tap:
> +* ``--tx-offloads=0x02c``
> +
> + This testpmd option enables TX offloads for UDP and TCP checksum on
> transmit,
> + as well as TCP TSO support.
> + The list of the offload flag values can be seen in header
> `rte_ethdev.h
> + <https://doc.dpdk.org/api/rte__ethdev_8h.html>`_.
> +
> +* ``--enable-lro``
> +
> + This testpmd option is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and
> + VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from the kernel
> can be
> + transmitted to the DPDK application and further TSOed by physical NIC.
> + If unsupported by the physical NIC, errors may be reported by testpmd
> with this option.
> +
> +
> +* Enabling Rx checksum offloads for physical port:
> +
> + Within testpmd, you can enable and disable offloads on a per-port
> basis,
> + rather than enabling them for both ports.
> + For the physical NIC, it may be desirable to enable checksum offload
> on packet RX.
> + This may be done as below, if testpmd is run with ``-i`` flag for
> interactive mode.
>
> .. code-block:: console
>
> - ifconfig tap0 1.1.1.1/24 up
> + testpmd> port stop 0
> + testpmd> port config 0 rx_offload tcp_cksum on
> + testpmd> port config 0 rx_offload udp_cksum on
> + testpmd> port start 0
>
> -.. note::
> +* Multiple queue support
>
> - The tap device will be named tap0, tap1, etc, by kernel.
> + Better performance may be achieved by using multiple queues,
> + so that multiple kernel threads are handling the traffic on the
> kernel side.
> + For example, to use 2 queues on both NIC and virtio ports,
> + while also enabling TX offloads and LRO support:
>
> -Then, all traffic from physical NIC can be forwarded into kernel stack,
> and all
> -traffic on the tap0 can be sent out from physical NIC.
> + .. code-block:: console
>
> -Limitations
> ------------
> + /path/to/dpdk-testpmd --vdev=virtio_user0,path=/dev/vhost-
> net,queues=2,queue_size=1024 -- \
> + -i --tx-offloads=0x002c --enable-lro --txq=2 --rxq=2 --
> txd=1024 --rxd=1024
>
> -This solution is only available on Linux systems.
> --
> 2.34.1
next prev parent reply other threads:[~2022-05-30 5:33 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-27 16:26 [PATCH " Bruce Richardson
2022-05-27 16:26 ` [PATCH 2/2] doc/howto: add code example to virtio-user exception path doc Bruce Richardson
2022-05-27 16:36 ` [PATCH v2 1/2] doc/howto: rework section on virtio-user as exception path Bruce Richardson
2022-05-27 16:36 ` [PATCH v2 2/2] doc/howto: add code example to virtio-user exception path doc Bruce Richardson
2022-05-27 18:03 ` Stephen Hemminger
2022-05-30 5:44 ` Xia, Chenbo
2022-06-10 14:33 ` Bruce Richardson
2022-05-30 5:33 ` Xia, Chenbo [this message]
2022-06-10 14:36 ` [PATCH v2 1/2] doc/howto: rework section on virtio-user as exception path Bruce Richardson
2022-06-10 15:35 ` [PATCH v3 0/4] Enhance docs on virtio-user as KNI replacement Bruce Richardson
2022-06-10 15:35 ` [PATCH v3 1/4] doc/howto: rework section on virtio-user as exception path Bruce Richardson
2022-06-15 1:44 ` Xia, Chenbo
2022-07-01 12:59 ` Maxime Coquelin
2022-07-11 13:10 ` Thomas Monjalon
2022-07-11 13:18 ` Bruce Richardson
2022-06-10 15:35 ` [PATCH v3 2/4] doc/howto: rename files to match new content name Bruce Richardson
2022-06-15 1:45 ` Xia, Chenbo
2022-07-07 15:38 ` Maxime Coquelin
2022-06-10 15:35 ` [PATCH v3 3/4] doc/howto: add code example to virtio-user exception path doc Bruce Richardson
2022-06-15 1:47 ` Xia, Chenbo
2022-07-07 15:50 ` Maxime Coquelin
2022-06-10 15:35 ` [PATCH v3 4/4] doc/prog_guide: add reference to virtio-user from KNI doc Bruce Richardson
2022-06-15 1:47 ` Xia, Chenbo
2022-07-07 15:52 ` Maxime Coquelin
2022-06-10 15:58 ` [PATCH v3 0/4] Enhance docs on virtio-user as KNI replacement Morten Brørup
2022-07-08 9:12 ` Maxime Coquelin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=SN6PR11MB3504ECD484225EF41C43E9819CDD9@SN6PR11MB3504.namprd11.prod.outlook.com \
--to=chenbo.xia@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=maxime.coquelin@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).