Re: [dpdk-dev] [PATCH v4] doc: clarify virtio PMD path selection

[Tiwei Bie <tiwei.bie@intel.com>]

On Tue, Nov 26, 2019 at 11:08:14AM -0500, Yinan wrote:
> From: Wang Yinan <yinan.wang@intel.com>
> 
> add virtio paths selection and usage introduction for better

s/add/Add/

> virtio usability.
> 
> Signed-off-by: Wang Yinan <yinan.wang@intel.com>
> ---
>  doc/guides/howto/index.rst                    |   1 +
>  .../virtio_paths_selection_and_usage.rst      | 142 ++++++++++++++++++
>  2 files changed, 143 insertions(+)
>  create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rst
> 
> diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst
> index a4c131652..6edb8d5be 100644
> --- a/doc/guides/howto/index.rst
> +++ b/doc/guides/howto/index.rst
> @@ -16,6 +16,7 @@ HowTo Guides
>      vfd
>      virtio_user_for_container_networking
>      virtio_user_as_exceptional_path
> +    virtio_paths_selection_and_usage
>      packet_capture_framework
>      telemetry
>      debug_troubleshoot
> diff --git a/doc/guides/howto/virtio_paths_selection_and_usage.rst b/doc/guides/howto/virtio_paths_selection_and_usage.rst
> new file mode 100644
> index 000000000..e22b18e14
> --- /dev/null
> +++ b/doc/guides/howto/virtio_paths_selection_and_usage.rst
> @@ -0,0 +1,142 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Virtio paths Selection and Usage
> +================================
> +
> +Logically virtio-PMD has 9 paths based on the combination of virtio features
> +(Rx mergeable, In-order, Packed virtqueue), below is an introduction of virtio
> +common features:

s/virtio common features/these features/

> +
> +*   `Rx mergeable <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/
> +    virtio-v1.1-cs01.html#x1-2140004>`_: With this feature negotiated, device
> +    can receive larger packets by combining individual descriptors.
> +*   `In-order <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/
> +    virtio-v1.1-cs01.html#x1-690008>`_: Some devices always use descriptors
> +    in the same order in which they have been made available, these
> +    devices can offer the VIRTIO_F_IN_ORDER feature. If this feature negotiated,
> +    driver will use descriptors in order. Meanwhile, this knowledge allows device
> +    operate used ring in batches and driver operate available ring in batches and
> +    such can decrease cache miss rate.
> +*   `Packed virtqueue <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/
> +    virtio-v1.1-cs01.html#x1-610007>`_: The structure of packed virtqueue is
> +    different from split virtqueue, split virtqueue is composed of available ring,
> +    used ring and descriptor table, while packed virtqueue is composed of descriptor
> +    ring, driver event suppression and device event suppression. The idea behind
> +    this is to improve performance by avoiding cache misses and and make it easier

s/and and/and/

> +    for devices to implement.
> +
> +Virtio paths Selection
> +----------------------
> +
> +If packed virtqueue is not negotiated, below split virtqueue paths will be selected
> +according to below configuration:
> +
> +#. Split virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature is
> +   not negotiated, this path will be selected.
> +#. Split virtqueue non-mergeable path: If Rx mergeable and in-order feature are not
> +   negotiated, also Rx offload(s) are requested, this path will be selected.
> +#. Split virtqueue in-order mergeable path: If Rx mergeable and in-order feature are
> +   both negotiated, this path will be selected.
> +#. Split virtqueue in-order non-mergeable path: If in-order feature is negotiated and
> +   Rx mergeable is not negotiated, this path will be selected.
> +#. Split virtqueue vectorized RX path: If Rx mergeable is disabled and no Rx offload

s/RX/Rx/

> +   requested, this path will be selected.
> +
> +If packed virtqueue is negotiated, below packed virtqueue paths will be selected
> +according to below configuration:
> +
> +#. Packed virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature
> +   is not negotiated, this path will be selected.
> +#. Packed virtqueue non-mergeable path: If Rx mergeable and in-order feature are not
> +   negotiated, this path will be selected.
> +#. Packed virtqueue in-order mergeable path: If in-order and Rx mergeable feature are
> +   both negotiated, this path will be selected.
> +#. Packed virtqueue in-order non-mergeable path: If in-order feature is negotiated and
> +   Rx mergeable is not negotiated, this path will be selected.
> +
> +Rx/Tx callbacks of each Virtio path
> +-----------------------------------
> +
> +Refer to above descriptions, virtio path and Rx/TX callbacks are auto selected by

s/TX/Tx/

> +different parameters of vdev and workloads. Rx callbacks and Tx callbacks name for
> +each Virtio Path are shown in following tables::

s/in following tables::/in below table:/

> +
> +   +----------------------------------------------------------------------------------------------------------+
> +   |       Virtio path                          |   Rx callbacks                   |    TX callbacks          |

s/TX/Tx/

> +   +----------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue mergeable path              |virtio_recv_mergeable_pkts        | virtio_xmit_pkts         |
> +   +----------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue non-mergeable path          | virtio_recv_pkts                 |  virtio_xmit_pkts        |
> +   +----------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue in-order mergeable path     | virtio_recv_pkts_inorder         |  virtio_xmit_pkts_inorder|
> +   +----------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue in-order non-mergeable path | virtio_recv_pkts_inorder         |  virtio_xmit_pkts_inorder|
> +   +----------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue vectorized RX path          | virtio_recv_pkts_vec             |  virtio_xmit_pkts        |

s/RX/Rx/

> +   +----------------------------------------------------------------------------------------------------------+
> +   |Packed virtqueue mergeable path             | virtio_recv_mergeable_pkts_packed|  virtio_xmit_pkts_packed |
> +   +----------------------------------------------------------------------------------------------------------+
> +   |Packed virtqueue normal path                | virtio_recv_pkts_packed          |  virtio_xmit_pkts_packed |

s/normal/non-mergeable/

> +   +----------------------------------------------------------------------------------------------------------+
> +   |Packed virtqueue in-order mergeable path    | virtio_recv_mergeable_pkts_packed|  virtio_xmit_pkts_packed |
> +   +----------------------------------------------------------------------------------------------------------+
> +   |Packed virtqueue in-order normal path       | virtio_recv_pkts_packed          |  virtio_xmit_pkts_packed |

s/normal/non-mergeable/

> +   +----------------------------------------------------------------------------------------------------------+

It seems above table will be interpreted as code block.
You can use below table:

.. table:: Virtio Paths and Callbacks

   ============================================ ================================= ========================
                 Virtio paths                            Rx callbacks                   Tx callbacks
   ============================================ ================================= ========================
   Split virtqueue mergeable path               virtio_recv_mergeable_pkts        virtio_xmit_pkts
   Split virtqueue non-mergeable path           virtio_recv_pkts                  virtio_xmit_pkts
   Split virtqueue in-order mergeable path      virtio_recv_pkts_inorder          virtio_xmit_pkts_inorder
   Split virtqueue in-order non-mergeable path  virtio_recv_pkts_inorder          virtio_xmit_pkts_inorder
   Split virtqueue vectorized Rx path           virtio_recv_pkts_vec              virtio_xmit_pkts
   Packed virtqueue mergeable path              virtio_recv_mergeable_pkts_packed virtio_xmit_pkts_packed
   Packed virtqueue non-meregable path          virtio_recv_pkts_packed           virtio_xmit_pkts_packed
   Packed virtqueue in-order mergeable path     virtio_recv_mergeable_pkts_packed virtio_xmit_pkts_packed
   Packed virtqueue in-order non-mergeable path virtio_recv_pkts_packed           virtio_xmit_pkts_packed
   ============================================ ================================= ========================


> +
> +Virtio paths Support Status from Release to Release
> +---------------------------------------------------
> +
> +Virtio feature implementation:
> +
> +*   In-order feature implemented in DPDK 18.08 by adding new Rx/TX callbacks

s/TX/Tx/
s/implemented in/is supported since/

> +    ``virtio_recv_pkts_inorder`` and ``virtio_xmit_pkts_inorder``.
> +*   Packed virtqueue implemented in DPDK 19.02 by adding new Rx/TX callbacks

Ditto.

> +    ``virtio_recv_pkts_packed`` , ``virtio_recv_mergeable_pkts_packed`` and
> +    ``virtio_xmit_pkts_packed``.
> +
> +Virtio path number changes from release to release, all virtio paths support
> +status are shown in below table::
> +
> +   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Virtio path\ DPDK version                   | v16.11 | v17.02 | v17.05 | v17.08 | v17.11 | v18.02 | v18.05 | v18.08 | v18.11 | v19.02 | v19.05 | v19.08 |
> +   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue mergeable path              |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   Y    |   Y    |  Y     |   Y    |   Y    |
> +   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue non-mergeable path          |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   Y    |   Y    |  Y     |   Y    |   Y    |
> +   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue vectorized RX path          |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   Y    |   Y    |  Y     |   Y    |   Y    |
> +   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue simple TX path              |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   N    |   N    |  N     |   N    |   N    |
> +   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue in-order non-mergeable path |        |        |         |       |        |        |        |   Y    |   Y    |  Y     |   Y    |   Y    |
> +   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Split virtqueue in-order mergeable path     |        |        |         |       |        |        |        |   Y    |   Y    |  Y     |   Y    |   Y    |
> +   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Packed virtqueue mergeable path             |        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
> +   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Packed virtqueue non-mergeable path         |        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
> +   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Packed virtqueue in-order mergeable path    |        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
> +   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
> +   |Packed virtqueue in-order non-mergeable path|        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
> +   ---------------------------------------------------------------------------------------------------------------------------------------------------------+

19.11 can be included in above table as well.
Columns can be merged if possible.
You can do it like this:

.. table:: Virtio Paths and Releases

   ============================================ ============= ============= =============
                  Virtio paths                  16.11 ~ 18.05 18.08 ~ 18.11 19.02 ~ 19.11
   ============================================ ============= ============= =============
   Split virtqueue mergeable path                     Y             Y             Y
   Split virtqueue non-mergeable path                 Y             Y             Y
   Split virtqueue vectorized Rx path                 Y             Y             Y
   Split virtqueue simple Tx path                     Y             N             N
   Split virtqueue in-order mergeable path                          Y             Y
   Split virtqueue in-order non-mergeable path                      Y             Y
   Packed virtqueue mergeable path                                                Y
   Packed virtqueue non-mergeable path                                            Y
   Packed virtqueue in-order mergeable path                                       Y
   Packed virtqueue in-order non-mergeable path                                   Y
   ============================================ ============= ============= =============

> +
> +QEMU Support Status
> +-------------------
> +
> +*   Qemu now supports three paths of split virtqueue: Split virtqueue mergeable path,
> +    Split virtqueue non-mergeable path, Split virtqueue vectorized RX path.

s/RX/Rx/

> +*   Since qemu 4.2.0, Packed virtqueue mergeable path and Packed virtqueue non-mergeable
> +    path can be supported.
> +
> +How to Debug
> +------------
> +
> +If you meet performance drop or some other issues after upgrading the driver
> +or configuration, below steps can help you identify which path you selected and
> +root cause faster.
> +
> +#. Run vhost/virtio test case;
> +#. Run "perf top" and check virtio Rx/tx callback names;
> +#. Identify which virtio path is selected refer to above table.
> -- 
> 2.17.1
>