Re: [dpdk-dev] [PATCH v2] doc: fix update release notes for Mellanox drivers

[Shahaf Shuler <shahafs@mellanox.com>]

Monday, May 13, 2019 9:43 AM, Ori Kam:
> Subject: RE: [dpdk-dev] [PATCH v2] doc: fix update release notes for
> Mellanox drivers
> 
> Hi,
> 
> PSB
> 
> > -----Original Message-----
> > From: Shahaf Shuler
> > Sent: Monday, May 13, 2019 8:26 AM
> > To: Ori Kam <orika@mellanox.com>; Yongseok Koh
> <yskoh@mellanox.com>;
> > Matan Azrad <matan@mellanox.com>; Thomas Monjalon
> > <thomas@monjalon.net>
> > Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>
> > Subject: RE: [dpdk-dev] [PATCH v2] doc: fix update release notes for
> > Mellanox drivers
> >
> > Hi Ori,
> >
> > See some comments below.
> >
> > Sunday, May 12, 2019 11:33 PM, Ori Kam:
> > > Subject: [dpdk-dev] [PATCH v2] doc: fix update release notes for
> > > Mellanox drivers
> > >
> > > This patch adds some missing features to Mellanox drivers release notes.
> > > It also updates the mlx5/mlx4 documentations.
> > >
> > > Fixes: d85b204b5dba ("doc: update release notes for Mellanox
> > > drivers")
> > > Cc: yskoh@mellanox.com
> > >
> > > Signed-off-by: Ori Kam <orika@mellanox.com>
> > >
> > > ---
> > >
> > > v2:
> > >  * fix checkpatch warning.
> > >
> > > ---
> > >  doc/guides/nics/mlx4.rst               |   2 +-
> > >  doc/guides/nics/mlx5.rst               | 150 ++++++++++++++++++++++-------
> ----
> > >  doc/guides/rel_notes/release_19_05.rst |  10 ++-
> > >  3 files changed, 109 insertions(+), 53 deletions(-)
> > >
> > > diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst
> > > index f6d7a16..5c6bbde 100644
> > > --- a/doc/guides/nics/mlx4.rst
> > > +++ b/doc/guides/nics/mlx4.rst
> > > @@ -253,7 +253,7 @@ thanks to these environment variables:
> > >  Mellanox OFED as a fallback
> > >  ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >
> > > -- `Mellanox OFED`_ version: **4.4, 4.5**.
> > > +- `Mellanox OFED`_ version: **4.4, 4.5, 4.6**.
> > >  - firmware version: **2.42.5000** and above.
> > >
> > >  .. _`Mellanox OFED`:
> > >
> http://www.mellanox.com/page/products_dyn?product_family=26&mtag=li
> > > nux_sw_drivers
> > > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
> > > index
> > > 325e9f6..5cf7744 100644
> > > --- a/doc/guides/nics/mlx5.rst
> > > +++ b/doc/guides/nics/mlx5.rst
> > > @@ -7,7 +7,7 @@ MLX5 poll mode driver
> > >
> > >  The MLX5 poll mode driver library (**librte_pmd_mlx5**) provides
> > > support for **Mellanox ConnectX-4**, **Mellanox ConnectX-4 Lx** ,
> > > **Mellanox - ConnectX-5**, **Mellanox ConnectX-6** and **Mellanox
> > > Bluefield** families
> > > +ConnectX-5**, **Mellanox ConnectX-6** and **Mellanox BlueField**
> > > +families
> > >  of 10/25/40/50/100/200 Gb/s adapters as well as their virtual
> > > functions (VF) in SR-IOV context.
> > >
> > > @@ -62,8 +62,8 @@ Features
> > >  - RX VLAN stripping.
> > >  - TX VLAN insertion.
> > >  - RX CRC stripping configuration.
> > > -- Promiscuous mode.
> > > -- Multicast promiscuous mode.
> > > +- Promiscuous mode on PF and VF.
> > > +- Multicast promiscuous mode on PF and VF.
> > >  - Hardware checksum offloads.
> > >  - Flow director (RTE_FDIR_MODE_PERFECT,
> > > RTE_FDIR_MODE_PERFECT_MAC_VLAN and
> > >    RTE_ETH_FDIR_REJECT).
> > > @@ -78,6 +78,11 @@ Features
> > >  - Rx HW timestamp.
> > >  - Tunnel types: VXLAN, L3 VXLAN, VXLAN-GPE, GRE, MPLSoGRE,
> MPLSoUDP.
> > >  - Tunnel HW offloads: packet type, inner/outer RSS, IP and UDP
> > > checksum verification.
> > > +- Nic HW offloads: encapsulation (vxlan, gre, mplsoudp, mplsogre),
> > > +NAT, routing, TTL
> > > +  increment/decrement, count, drop, mark. For details please see
> > > :ref:`Supported hardware offloads using rte_flow API`.
> > > +- Flow insertion rate of more then million flows per second. When
> > > +using
> > > Direct Rules.
> >
> > Replace '.' with ',' .
> >
> 
> O.K
> 
> > > +- Support groups.
> >
> > Support for multiple rte_flow groups.
> >
> 
> O.K
> 
> > > +- Support raw encapsulation offload.
> >
> > Can't it be part of the encapsulation part above?
> >
> 
> Do you mean the tunnel HW offloads?
> If yes I guess it can be there.

As part of  "Nic HW offloads: encapsulation (..)"

> 
> > >
> > >  Limitations
> > >  -----------
> > > @@ -112,8 +117,6 @@ Limitations
> > >    is set to multi-packet send or Enhanced multi-packet send.
> > > Otherwise it must have
> > >    less than 50 segments.
> > >
> > > -- Count action for RTE flow is **only supported in Mellanox OFED**.
> > > -
> > >  - Flows with a VXLAN Network Identifier equal (or ends to be equal)
> > >    to 0 are not supported.
> > >
> > > @@ -124,13 +127,6 @@ Limitations
> > >  - VF: flow rules created on VF devices can only match traffic targeted at
> the
> > >    configured MAC addresses (see ``rte_eth_dev_mac_addr_add()``).
> > >
> > > -.. note::
> > > -
> > > -   MAC addresses not already present in the bridge table of the
> associated
> > > -   kernel network device will be added and cleaned up by the PMD when
> > > closing
> > > -   the device. In case of ungraceful program termination, some entries
> may
> > > -   remain present and should be removed manually by other means.
> > > -
> >
> > This is still correct, why removed?
> >
> 
> Will return the code.
> 
> > >  - When Multi-Packet Rx queue is configured (``mprq_en``), a Rx
> > > packet can be
> > >    externally attached to a user-provided mbuf with having
> > > EXT_ATTACHED_MBUF in
> > >    ol_flags. As the mempool for the external buffer is managed by
> > > PMD, all the @@ -147,30 +143,18 @@ Limitations
> > >    To receive IPv6 Multicast messages on VM, explicitly set the relevant
> > >    MAC address using rte_eth_dev_mac_addr_add() API.
> > >
> > > -- E-Switch VXLAN tunnel is not supported together with outer VLAN.
> > > -
> > > -- E-Switch Flows with VNI pattern must include the VXLAN
> > > decapsulation action.
> > > -
> > > -- E-Switch VXLAN decapsulation Flow:
> > > +- E-Switch decapsulation Flow:
> > >
> > >    - can be applied to PF port only.
> > >    - must specify VF port action (packet redirection from PF to VF).
> > > -  - must specify tunnel outer UDP local (destination) port,
> > > wildcards not allowed.
> > > -  - must specify tunnel outer VNI, wildcards not allowed.
> > > -  - must specify tunnel outer local (destination)  IPv4 or IPv6
> > > address, wildcards not allowed.
> > > -  - optionally may specify tunnel outer remote (source) IPv4 or
> > > IPv6, wildcards or group IPs allowed.
> > >    - optionally may specify tunnel inner source and destination MAC
> > > addresses.
> > >
> > > -- E-Switch VXLAN encapsulation Flow:
> > > +- E-Switch  encapsulation Flow:
> > >
> > >    - can be applied to VF ports only.
> > >    - must specify PF port action (packet redirection from VF to PF).
> > > -  - must specify the VXLAN item with tunnel outer parameters.
> > > -  - must specify the tunnel outer VNI in the VXLAN item.
> > > -  - must specify the tunnel outer remote (destination) UDP port in
> > > the VXLAN item.
> > > -  - must specify the tunnel outer local (source) IPv4 or IPv6 in
> > > the , this address will locally (with scope link) assigned to the
> > > outer network interface, wildcards not allowed.
> > > -  - must specify the tunnel outer remote (destination) IPv4 or IPv6
> > > in the VXLAN item, group IPs allowed.
> > > -  - must specify the tunnel outer destination MAC address in the
> > > VXLAN item, this address will be used to create neigh rule.
> > > +
> > > +- Groups, E-Switch steering, fast inseration rate are supported
> > > +only with
> > > Mellanox OFED 4.6.2 and above.
> >
> > I think the right place for such limitations is the table below
> > (Supported hardware offloads using rte_flow API).
> >
> 
> O.K will update to something like this:
> Action | Nic                                             | E-Switch
> Count  | DPDK > 19.02                          | DPDK > 19.05
>              | OFED > 4.6.2 / RDMA > xxx  | OFED > 4.6.2 / RDMA > xxx
> 
> Is that O.K?

We can skip the >, just mention the version are the minimal versions. 
Need to also add the minimum NIC. 

> 
> > >
> > >  Statistics
> > >  ----------
> > > @@ -227,7 +211,7 @@ These options can be modified in the ``.config``
> file.
> > >
> > >  .. note::
> > >
> > > -   For Bluefield, target should be set to ``arm64-bluefield-linux-gcc``. This
> > > +   For BlueField, target should be set to
> > > + ``arm64-bluefield-linux-gcc``. This
> > >     will enable ``CONFIG_RTE_LIBRTE_MLX5_PMD`` and set
> > > ``RTE_CACHE_LINE_SIZE`` to
> > >     64. Default armv8a configuration of make build and meson build
> > > set it to
> > > 128
> > >     then brings performance degradation.
> > > @@ -277,8 +261,8 @@ Run-time configuration
> > >
> > >    Supported on:
> > >
> > > -  - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6
> > > and Bluefield.
> > > -  - POWER8 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and
> > > Bluefield.
> > > +  - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6
> > > + and
> > > BlueField.
> > > +  - POWER9 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6
> and
> > > BlueField.
> > >
> > >  - ``rxq_cqe_pad_en`` parameter [int]
> > >
> > > @@ -296,7 +280,7 @@ Run-time configuration
> > >
> > >    Supported on:
> > >
> > > -  - CPU having 128B cacheline with ConnectX-5 and Bluefield.
> > > +  - CPU having 128B cacheline with ConnectX-5 and BlueField.
> > >
> > >  - ``rxq_pkt_pad_en`` parameter [int]
> > >
> > > @@ -308,8 +292,8 @@ Run-time configuration
> > >
> > >    Supported on:
> > >
> > > -  - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6
> > > and Bluefield.
> > > -  - POWER8 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and
> > > Bluefield.
> > > +  - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6
> > > + and
> > > BlueField.
> > > +  - POWER8 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6
> and
> > > BlueField.
> > >
> > >  - ``mprq_en`` parameter [int]
> > >
> > > @@ -375,13 +359,13 @@ Run-time configuration
> > >
> > >    This option should be used in combination with ``txq_inline`` above.
> > >
> > > -  On ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and
> > > Bluefield without
> > > +  On ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and
> > > + BlueField without
> > >    Enhanced MPW:
> > >
> > >          - Disabled by default.
> > >          - In case ``txq_inline`` is set recommendation is 4.
> > >
> > > -  On ConnectX-5, ConnectX-6 and Bluefield with Enhanced MPW:
> > > +  On ConnectX-5, ConnectX-6 and BlueField with Enhanced MPW:
> > >
> > >          - Set to 8 by default.
> > >
> > > @@ -395,14 +379,14 @@ Run-time configuration
> > >          - Set to 8 by default on ARMv8.
> > >          - Set to 4 by default otherwise.
> > >
> > > -  On Bluefield
> > > +  On BlueField
> > >
> > >          - Set to 16 by default.
> > >
> > >  - ``txq_mpw_en`` parameter [int]
> > >
> > >    A nonzero value enables multi-packet send (MPS) for ConnectX-4 Lx
> > > and
> > > -  enhanced multi-packet send (Enhanced MPS) for ConnectX-5,
> > > ConnectX-6 and Bluefield.
> > > +  enhanced multi-packet send (Enhanced MPS) for ConnectX-5,
> > > + ConnectX-6
> > > and BlueField.
> > >    MPS allows the TX burst function to pack up multiple packets in a
> > >    single descriptor session in order to save PCI bandwidth and improve
> > >    performance at the cost of a slightly higher CPU usage. When @@
> > > -417,13
> > > +401,13 @@ Run-time configuration
> > >    DEV_TX_OFFLOAD_VXLAN_TNL_TSO,
> DEV_TX_OFFLOAD_GRE_TNL_TSO,
> > > DEV_TX_OFFLOAD_VLAN_INSERT``.
> > >    When those offloads are requested the MPS send function will not
> > > be used.
> > >
> > > -  It is currently only supported on the ConnectX-4 Lx, ConnectX-5,
> > > ConnectX-
> > > 6 and Bluefield
> > > +  It is currently only supported on the ConnectX-4 Lx, ConnectX-5,
> > > + ConnectX-6 and BlueField
> > >    families of adapters.
> > >    On ConnectX-4 Lx the MPW is considered un-secure hence disabled
> > > by default.
> > >    Users which enable the MPW should be aware that application which
> > > provides incorrect
> > >    mbuf descriptors in the Tx burst can lead to serious errors in
> > > the host including, on some cases,
> > >    NIC to get stuck.
> > > -  On ConnectX-5, ConnectX-6 and Bluefield the MPW is secure and
> > > enabled by default.
> > > +  On ConnectX-5, ConnectX-6 and BlueField the MPW is secure and
> > > + enabled
> > > by default.
> > >
> > >  - ``txq_mpw_hdr_dseg_en`` parameter [int]
> > >
> > > @@ -443,14 +427,14 @@ Run-time configuration
> > >
> > >  - ``tx_vec_en`` parameter [int]
> > >
> > > -  A nonzero value enables Tx vector on ConnectX-5, ConnectX-6 and
> > > Bluefield NICs if the number of
> > > +  A nonzero value enables Tx vector on ConnectX-5, ConnectX-6 and
> > > + BlueField NICs if the number of
> > >    global Tx queues on the port is less than ``txqs_max_vec``.
> > >
> > >    This option cannot be used with certain offloads such as
> > > ``DEV_TX_OFFLOAD_TCP_TSO,
> > >    DEV_TX_OFFLOAD_VXLAN_TNL_TSO,
> DEV_TX_OFFLOAD_GRE_TNL_TSO,
> > > DEV_TX_OFFLOAD_VLAN_INSERT``.
> > >    When those offloads are requested the MPS send function will not
> > > be used.
> > >
> > > -  Enabled by default on ConnectX-5, ConnectX-6 and Bluefield.
> > > +  Enabled by default on ConnectX-5, ConnectX-6 and BlueField.
> > >
> > >  - ``rx_vec_en`` parameter [int]
> > >
> > > @@ -480,10 +464,15 @@ Run-time configuration
> > >
> > >    A nonzero value enables the DV flow steering assuming it is supported
> > >    by the driver.
> > > -  The DV flow steering is not supported on switchdev mode.
> > >
> > >    Disabled by default.
> > >
> > > +- ``dv_esw_en`` parameter [int]
> > > +
> > > +  A nonzero value enables E-Switch using Direct Rules.
> > > +
> > > +  Enabled by default if supported.
> > > +
> > >  - ``mr_ext_memseg_en`` parameter [int]
> > >
> > >    A nonzero value enables extending memseg when registering DMA
> > > memory. If @@ -545,7 +534,7 @@ DPDK and must be installed
> separately:
> > >  - **libmlx5**
> > >
> > >    Low-level user space driver library for Mellanox
> > > -  ConnectX-4/ConnectX-5/ConnectX-6/Bluefield devices, it is
> > > automatically loaded
> > > +  ConnectX-4/ConnectX-5/ConnectX-6/BlueField devices, it is
> > > + automatically loaded
> > >    by libibverbs.
> > >
> > >    This library basically implements send/receive calls to the
> > > hardware @@ -
> > > 567,7 +556,7 @@ DPDK and must be installed separately:
> > >    their devices:
> > >
> > >    - mlx5_core: hardware driver managing Mellanox
> > > -    ConnectX-4/ConnectX-5/ConnectX-6/Bluefield devices and related
> > > Ethernet kernel
> > > +    ConnectX-4/ConnectX-5/ConnectX-6/BlueField devices and related
> > > + Ethernet kernel
> > >      network devices.
> > >    - mlx5_ib: InifiniBand device driver.
> > >    - ib_uverbs: user space driver for Verbs (entry point for libibverbs).
> > > @@ -575,7 +564,7 @@ DPDK and must be installed separately:
> > >  - **Firmware update**
> > >
> > >    Mellanox OFED/EN releases include firmware updates for
> > > -  ConnectX-4/ConnectX-5/ConnectX-6/Bluefield adapters.
> > > +  ConnectX-4/ConnectX-5/ConnectX-6/BlueField adapters.
> > >
> > >    Because each release provides new features, these updates must be
> > > applied to
> > >    match the kernel modules and libraries they come with.
> > > @@ -595,7 +584,7 @@ releases.
> > >  RDMA Core with Linux Kernel
> > >  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > >
> > > -- Minimal kernel version : v4.14 or the most recent 4.14-rc (see
> > > `Linux installation documentation`_)
> > > +- Minimal kernel version : v4.14 or the most recent 5.1.0-rc (see
> > > +`Linux installation documentation`_)
> > >  - Minimal rdma-core version: v15+ commit 0c5f5765213a ("Merge pull
> > > request #227 from yishaih/tm")
> > >    (see `RDMA Core installation documentation`_)
> > >  - When building for i686 use:
> > > @@ -622,7 +611,8 @@ thanks to these environment variables:
> > >  Mellanox OFED/EN
> > >  ^^^^^^^^^^^^^^^^
> > >
> > > -- Mellanox OFED version: **4.4, 4.5** / Mellanox EN version:
> > > **4.5**
> > > +- Mellanox OFED version: ** 4.5, 4.6** /
> > > +  Mellanox EN version: **4.5, 4.6**
> > >  - firmware version:
> > >
> > >    - ConnectX-4: **12.21.1000** and above.
> > > @@ -630,7 +620,7 @@ Mellanox OFED/EN
> > >    - ConnectX-5: **16.21.1000** and above.
> > >    - ConnectX-5 Ex: **16.21.1000** and above.
> > >    - ConnectX-6: **20.99.5374** and above.
> > > -  - Bluefield: **18.99.3950** and above.
> > > +  - BlueField: **18.25.1010** and above.
> > >
> > >  While these libraries and kernel modules are available on
> > > OpenFabrics Alliance's `website
> > >
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw
> > > w
> > >
> w.openfabrics.org%2F&amp;data=02%7C01%7Cshahafs%40mellanox.com%7
> > >
> >
> C6432343b75294926c94c08d6d71918dc%7Ca652971c7d2e4d9ba6a4d149256f4
> > >
> >
> 61b%7C0%7C0%7C636932900128053421&amp;sdata=d8coRBxwMI62n%2FB1I
> > > E4mOcoVHs4lsiDl7kSgVljJBeY%3D&amp;reserved=0>`__ and provided by
> > > package @@ -766,6 +756,34 @@ Quick Start Guide on OFED/EN  6.
> > > Compile DPDK and you are ready to go. See instructions on
> > >     :ref:`Development Kit Build System
> > > <Development_Kit_Build_System>`
> > >
> > > +Enable E-Switch mode
> > > +--------------------
> >
> > It is how to enable switchdev mode right? Not e-switch mode.
> > Need to put a few words on what it is switchdev mode and why it is
> needed.
> >
> 
> O.K will add some words.
> 
> > > +
> > > +1. Enable SRIOV mode:
> > > +
> > > +  .. code-block:: console
> > > +
> > > +        mlxconfig -d <mst device> set SRIOV_EN=true
> > > +
> > > +2. Configure the number of VFs:
> >
> > This is the max number of VFs.
> >
> 
> Will fix.
> 
> > > +
> > > +  .. code-block:: console
> > > +
> > > +        mlxconfig -d <mst device> set NUM_OF_VFS=<num of vfs>
> > > +        echo <num of vfs > /sys/class/net/<net
> > > + device>/device/sriov_numvfs
> >
> > Need to reset FW after it.
> >
> I don't think reset FW is required.

After you change the FW configuration w/ mlxconfig you must reset the FW. 

> 
> > Also - where do you create the actual VFs?
> >
> 
> The  in the second line:
>  echo <num of vfs > /sys/class/net/<netdevice>/device/sriov_numvfs

Sorry missed that. 

> 
> > > +
> > > +3. Unbind the device (can be rebind after the switchdev mode):
> > > +
> > > +  .. code-block:: console
> > > +
> > > +        echo -n "<device pci address" >
> > > + /sys/bus/pci/drivers/mlx5_core/unbind
> > > +
> > > +4. Enbale switchdev mode:
> > > +
> > > +  .. code-block:: console
> > > +
> > > +        echo switchdev > /sys/class/net/<net
> > > + device>/compat/devlink/mode
> > > +
> > >  Performance tuning
> > >  ------------------
> > >
> > > @@ -842,6 +860,38 @@ Performance tuning
> > >     - Configure per-lcore cache when creating Mempools for packet buffer.
> > >     - Refrain from dynamically allocating/freeing memory in run-time.
> > >
> > > +Supported hardware offloads using rte_flow API
> > > +----------------------------------------------
> > > +
> > > +.. _Supported hardware offloads using rte_flow API:
> > > +
> > > +.. table:: Supported hardware offloads using rte_flow API
> > > +
> > > +   +---------------+-----------+-------------+-----------------+----------------+
> > > +   | Offload       | E-Switch  | NIC PF      | VF representor  | PF representor
> |
> > > +   |               | (SRIOV)   | (non SRIOV) | (SRIOV)         | (SRIOV)        |
> > > +
> > >
> +===============+===========+=============+===============
> > > ==+================+
> > > +   | Encapsulation |     V     |     V       |        V        |       V*       |
> > > +   +---------------+-----------+-------------+-----------------+----------------+
> > > +   | Multi table   |     V     |     V       |        V        |       V*       |
> > > +   | support       |           |             |                 |                |
> > > +   +---------------+-----------+-------------+-----------------+----------------+
> > > +   | NAT           |     V     |     V       |        V        |       V*       |
> > > +   +---------------+-----------+-------------+-----------------+----------------+
> > > +   | Routing       |     V     |     V       |        V        |       V*       |
> > > +   +---------------+-----------+-------------+-----------------+----------------+
> > > +   | TTL inc/dec   |     V     |     V       |        V        |       V*       |
> > > +   +---------------+-----------+-------------+-----------------+----------------+
> > > +   | Count         |     V     |     V       |        V        |       V*       |
> > > +   +---------------+-----------+-------------+-----------------+----------------+
> > > +   | Drop          |     V     |     V       |        V        |       V*       |
> > > +   +---------------+-----------+-------------+-----------------+----------------+
> > > +   | Mark          |    N/A    |     V       |        V        |       V*       |
> > > +
> > > + +---------------+-----------+-------------+-----------------+-----
> > > + +---------------+-----------+-------------+-----------------+----
> > > + -------+
> > > +
> > > +| V ConnectX-5
> > > +| V* ConnectX-5 & BlueField
> > > +
> >
> > What is the difference between E-Switch (SRIOV), VF representor
> > (SRIOV) and PF representor (SRIOV)? On switchdev mode every rule is
> > either for the VF/uplink representor.
> >
> > The matrix is a bit more complex than what you show here.
> > IMO-  the matrix has 2 column (E-Switch and NIC) and 2/3 dimensions
> > (minimal NIC that support, minimal DPDK that support, minimal
> > OFED/upstream kernel which support(?)).
> > The columns need to be RTE_FLOW actions (let's leave the RTE_FLOW
> > items aside for a moment). Because NAT, for example, may require
> > different actions on different implementations.
> >
> 
> PSB my answer above.
> 
> >
> > >  Notes for testpmd
> > >  -----------------
> > >
> > > @@ -863,7 +913,7 @@ Usage example
> > >  -------------
> > >
> > >  This section demonstrates how to launch **testpmd** with Mellanox -
> > > ConnectX-4/ConnectX-5/ConnectX-6/Bluefield devices managed by
> > > librte_pmd_mlx5.
> > > +ConnectX-4/ConnectX-5/ConnectX-6/BlueField devices managed by
> > > librte_pmd_mlx5.
> > >
> > >  #. Load the kernel modules:
> > >
> > > diff --git a/doc/guides/rel_notes/release_19_05.rst
> > > b/doc/guides/rel_notes/release_19_05.rst
> > > index 4e0eed5..489273b 100644
> > > --- a/doc/guides/rel_notes/release_19_05.rst
> > > +++ b/doc/guides/rel_notes/release_19_05.rst
> > > @@ -148,10 +148,16 @@ New Features
> > >     * Added support for multiport InfiniBand device.
> > >     * Added control of excessive memory pinning by kernel.
> > >     * Added support of DMA memory registration by secondary process.
> > > -   * Added Direct Rule support in Direct Verbs flow driver.
> > >     * Added support of per-process device registers, reserving
> > > identical VA space
> > >       is not needed anymore.
> > > -   * Added E-Switch support in Direct Verbs flow driver.
> > > +   * Added Direct Rule support for Nic steering, in Direct Verbs driver.
> > > +   * Added Direct Rule support for E-Switch steering, in Direct Verbs
> driver.
> >
> > People who read the release notes doesn't necessarily read the entire
> > mlx5 manual. It is not clear what Direct rule is and why it is good.
> > I would drop this part.
> >
> >
> 
> This was done by Olga request.
> But I don't care to remove it.
> 
> > > +   * Added support for jump action for both E-Switch and Nic when using
> > > +     Direct Rules.
> >
> > How can user know if it uses Direct rule or not? I would just stick with:
> > "Support rte_flow jump action"
> >
> 
> O.K
> 
> > > +   * Added Group Support for Nic steering when using Direct Rules.
> >
> > Again:
> > "Support multiple rte_flow groups"
> >
> > > +   * Support millions of offloaded flow rules.
> > > +   * Improved flow insertion rate to millions of flows per second when
> using
> > > +     Direct Rules and group larger then 0.
> >
> > I would say:
> > * flow engine re-design to support large scale deployments. This include:
> > 1. support millions of rte_flow rules
> > 2. fast flow insertion and deletion up to 1M flow update per second.
> >
> >
> 
> O.K.
> 
> 
> > >
> > >  * **Renamed avf to iavf.**
> > >
> > > --
> > > 1.8.3.1