From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00064.outbound.protection.outlook.com [40.107.0.64]) by dpdk.org (Postfix) with ESMTP id 61FAD3DC for ; Mon, 13 May 2019 08:43:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DE1EaMWaNxsu+UK21UUzqUSWMieOyvaZ+bBpJVSdp70=; b=h8E9BLcjfngy9cAA4IMx8qPygs3TIiD6bot4SuYTplj+K0HweehaOXe3xf+DYhok0BoAujTKkILSaxVA1aYA31tnA5pZX3UD9MHrfqGRWdPxsgEcK7NWYEXU/nM+zoQZpAAPGStsxtTB+Ew5AO/mxnVmOy6LYUJDwKuuZqfszpc= Received: from AM4PR05MB3425.eurprd05.prod.outlook.com (10.171.190.15) by AM4PR05MB3489.eurprd05.prod.outlook.com (10.170.126.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1878.21; Mon, 13 May 2019 06:43:22 +0000 Received: from AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::c9ea:c445:68e5:942]) by AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::c9ea:c445:68e5:942%4]) with mapi id 15.20.1878.024; Mon, 13 May 2019 06:43:22 +0000 From: Ori Kam To: Shahaf Shuler , Yongseok Koh , Matan Azrad , Thomas Monjalon CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v2] doc: fix update release notes for Mellanox drivers Thread-Index: AQHVCQH58QJ8qRjqYEyP1YsrsfA2YaZohskAgAAOXIA= Date: Mon, 13 May 2019 06:43:22 +0000 Message-ID: References: <1557685859-121772-1-git-send-email-orika@mellanox.com> <1557693197-129982-1-git-send-email-orika@mellanox.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a2585eda-a532-436a-6360-08d6d76e4b26 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(4618075)(2017052603328)(7193020); SRVR:AM4PR05MB3489; x-ms-traffictypediagnostic: AM4PR05MB3489: x-ms-exchange-purlcount: 2 x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7219; x-forefront-prvs: 0036736630 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(136003)(396003)(376002)(366004)(346002)(199004)(189003)(13464003)(8936002)(25786009)(316002)(486006)(186003)(76116006)(66946007)(446003)(73956011)(26005)(66446008)(64756008)(66556008)(66476007)(33656002)(476003)(11346002)(81166006)(55016002)(2906002)(229853002)(53946003)(6306002)(9686003)(305945005)(6116002)(3846002)(19627235002)(6436002)(52536014)(7736002)(81156014)(256004)(14444005)(5024004)(76176011)(6506007)(71190400001)(71200400001)(53936002)(478600001)(14454004)(966005)(68736007)(7110500001)(2420400007)(15650500001)(6246003)(86362001)(66066001)(74316002)(53546011)(102836004)(30864003)(110136005)(5660300002)(99286004)(7696005)(4326008)(21314003)(579004); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3489; H:AM4PR05MB3425.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 44jiUGeRqT5tVs21xs9kW4RJd6N3zJHBDgt2b2+ZWUUon4KUGL3mqkr+BAtOoQznWL3zpUga1NiqcCBx6GiR5026u7l4NqhJ7zVtacO6YLJ1qNoWFpUX/FJ/OO6dh+7o/RoVM/ga7Go2opSUyqYBM4eN00vP2Yuy3hrkaFrCQEjVKywP8bVXNAynQatHvFqdhpLNsanxgFlMGaLerqZ0aZBCs4sFRLegVV6pq6GEAr4X+MFSxT1nUmWrkjKoNX6ac/v+YpEGf2ZbpY5sV/FFalWZ9IaEvRqvB6kr+UJN3gXhA2qPcIh4jSqtkPP7s/hie+kFTucIL7O+BjcvEC74HC6y2r4cmWgGbCoN7KXmGraIGMJikSUlXvrMeHWvoA9RNZcSX+vz3+h1hRX+zLlje9+YR/YFUqdUNkTJb835LIE= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: a2585eda-a532-436a-6360-08d6d76e4b26 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 May 2019 06:43:22.3946 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3489 Subject: Re: [dpdk-dev] [PATCH v2] doc: fix update release notes for Mellanox drivers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2019 06:43:24 -0000 Hi, PSB > -----Original Message----- > From: Shahaf Shuler > Sent: Monday, May 13, 2019 8:26 AM > To: Ori Kam ; Yongseok Koh ; > Matan Azrad ; Thomas Monjalon > > Cc: dev@dpdk.org; Ori Kam > Subject: RE: [dpdk-dev] [PATCH v2] doc: fix update release notes for Mell= anox > drivers >=20 > Hi Ori, >=20 > See some comments below. >=20 > Sunday, May 12, 2019 11:33 PM, Ori Kam: > > Subject: [dpdk-dev] [PATCH v2] doc: fix update release notes for Mellan= ox > > drivers > > > > This patch adds some missing features to Mellanox drivers release notes= . > > It also updates the mlx5/mlx4 documentations. > > > > Fixes: d85b204b5dba ("doc: update release notes for Mellanox drivers") > > Cc: yskoh@mellanox.com > > > > Signed-off-by: Ori Kam > > > > --- > > > > v2: > > * fix checkpatch warning. > > > > --- > > doc/guides/nics/mlx4.rst | 2 +- > > doc/guides/nics/mlx5.rst | 150 ++++++++++++++++++++++---= -------- > > doc/guides/rel_notes/release_19_05.rst | 10 ++- > > 3 files changed, 109 insertions(+), 53 deletions(-) > > > > diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst index > > f6d7a16..5c6bbde 100644 > > --- a/doc/guides/nics/mlx4.rst > > +++ b/doc/guides/nics/mlx4.rst > > @@ -253,7 +253,7 @@ thanks to these environment variables: > > Mellanox OFED as a fallback > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > -- `Mellanox OFED`_ version: **4.4, 4.5**. > > +- `Mellanox OFED`_ version: **4.4, 4.5, 4.6**. > > - firmware version: **2.42.5000** and above. > > > > .. _`Mellanox OFED`: > > http://www.mellanox.com/page/products_dyn?product_family=3D26&mtag=3Dli > > nux_sw_drivers > > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index > > 325e9f6..5cf7744 100644 > > --- a/doc/guides/nics/mlx5.rst > > +++ b/doc/guides/nics/mlx5.rst > > @@ -7,7 +7,7 @@ MLX5 poll mode driver > > > > The MLX5 poll mode driver library (**librte_pmd_mlx5**) provides suppo= rt > > for **Mellanox ConnectX-4**, **Mellanox ConnectX-4 Lx** , **Mellanox - > > ConnectX-5**, **Mellanox ConnectX-6** and **Mellanox Bluefield** > > families > > +ConnectX-5**, **Mellanox ConnectX-6** and **Mellanox BlueField** > > +families > > of 10/25/40/50/100/200 Gb/s adapters as well as their virtual function= s (VF) > > in SR-IOV context. > > > > @@ -62,8 +62,8 @@ Features > > - RX VLAN stripping. > > - TX VLAN insertion. > > - RX CRC stripping configuration. > > -- Promiscuous mode. > > -- Multicast promiscuous mode. > > +- Promiscuous mode on PF and VF. > > +- Multicast promiscuous mode on PF and VF. > > - Hardware checksum offloads. > > - Flow director (RTE_FDIR_MODE_PERFECT, > > RTE_FDIR_MODE_PERFECT_MAC_VLAN and > > RTE_ETH_FDIR_REJECT). > > @@ -78,6 +78,11 @@ Features > > - Rx HW timestamp. > > - Tunnel types: VXLAN, L3 VXLAN, VXLAN-GPE, GRE, MPLSoGRE, MPLSoUDP. > > - Tunnel HW offloads: packet type, inner/outer RSS, IP and UDP checksu= m > > verification. > > +- Nic HW offloads: encapsulation (vxlan, gre, mplsoudp, mplsogre), NAT= , > > +routing, TTL > > + increment/decrement, count, drop, mark. For details please see > > :ref:`Supported hardware offloads using rte_flow API`. > > +- Flow insertion rate of more then million flows per second. When usin= g > > Direct Rules. >=20 > Replace '.' with ',' . >=20 O.K > > +- Support groups. >=20 > Support for multiple rte_flow groups. >=20 O.K > > +- Support raw encapsulation offload. >=20 > Can't it be part of the encapsulation part above? >=20 Do you mean the tunnel HW offloads? If yes I guess it can be there. > > > > Limitations > > ----------- > > @@ -112,8 +117,6 @@ Limitations > > is set to multi-packet send or Enhanced multi-packet send. Otherwise= it > > must have > > less than 50 segments. > > > > -- Count action for RTE flow is **only supported in Mellanox OFED**. > > - > > - Flows with a VXLAN Network Identifier equal (or ends to be equal) > > to 0 are not supported. > > > > @@ -124,13 +127,6 @@ Limitations > > - VF: flow rules created on VF devices can only match traffic targeted= at the > > configured MAC addresses (see ``rte_eth_dev_mac_addr_add()``). > > > > -.. note:: > > - > > - MAC addresses not already present in the bridge table of the associ= ated > > - kernel network device will be added and cleaned up by the PMD when > > closing > > - the device. In case of ungraceful program termination, some entries= may > > - remain present and should be removed manually by other means. > > - >=20 > This is still correct, why removed? >=20 Will return the code. > > - When Multi-Packet Rx queue is configured (``mprq_en``), a Rx packet = can > > be > > externally attached to a user-provided mbuf with having > > EXT_ATTACHED_MBUF in > > ol_flags. As the mempool for the external buffer is managed by PMD, = all > > the @@ -147,30 +143,18 @@ Limitations > > To receive IPv6 Multicast messages on VM, explicitly set the relevan= t > > MAC address using rte_eth_dev_mac_addr_add() API. > > > > -- E-Switch VXLAN tunnel is not supported together with outer VLAN. > > - > > -- E-Switch Flows with VNI pattern must include the VXLAN decapsulation > > action. > > - > > -- E-Switch VXLAN decapsulation Flow: > > +- E-Switch decapsulation Flow: > > > > - can be applied to PF port only. > > - must specify VF port action (packet redirection from PF to VF). > > - - must specify tunnel outer UDP local (destination) port, wildcards = not > > allowed. > > - - must specify tunnel outer VNI, wildcards not allowed. > > - - must specify tunnel outer local (destination) IPv4 or IPv6 addres= s, > > wildcards not allowed. > > - - optionally may specify tunnel outer remote (source) IPv4 or IPv6, > > wildcards or group IPs allowed. > > - optionally may specify tunnel inner source and destination MAC > > addresses. > > > > -- E-Switch VXLAN encapsulation Flow: > > +- E-Switch encapsulation Flow: > > > > - can be applied to VF ports only. > > - must specify PF port action (packet redirection from VF to PF). > > - - must specify the VXLAN item with tunnel outer parameters. > > - - must specify the tunnel outer VNI in the VXLAN item. > > - - must specify the tunnel outer remote (destination) UDP port in the > > VXLAN item. > > - - must specify the tunnel outer local (source) IPv4 or IPv6 in the ,= this > > address will locally (with scope link) assigned to the outer network in= terface, > > wildcards not allowed. > > - - must specify the tunnel outer remote (destination) IPv4 or IPv6 in= the > > VXLAN item, group IPs allowed. > > - - must specify the tunnel outer destination MAC address in the VXLAN > > item, this address will be used to create neigh rule. > > + > > +- Groups, E-Switch steering, fast inseration rate are supported only w= ith > > Mellanox OFED 4.6.2 and above. >=20 > I think the right place for such limitations is the table below (Supporte= d > hardware offloads using rte_flow API). >=20 O.K will update to something like this: Action | Nic | E-Switch Count | DPDK > 19.02 | DPDK > 19.05 | OFED > 4.6.2 / RDMA > xxx | OFED > 4.6.2 / RDMA > xxx Is that O.K? =20 > > > > Statistics > > ---------- > > @@ -227,7 +211,7 @@ These options can be modified in the ``.config`` fi= le. > > > > .. note:: > > > > - For Bluefield, target should be set to ``arm64-bluefield-linux-gcc`= `. This > > + For BlueField, target should be set to > > + ``arm64-bluefield-linux-gcc``. This > > will enable ``CONFIG_RTE_LIBRTE_MLX5_PMD`` and set > > ``RTE_CACHE_LINE_SIZE`` to > > 64. Default armv8a configuration of make build and meson build set = it to > > 128 > > then brings performance degradation. > > @@ -277,8 +261,8 @@ Run-time configuration > > > > Supported on: > > > > - - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > Bluefield. > > - - POWER8 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > Bluefield. > > + - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > BlueField. > > + - POWER9 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > BlueField. > > > > - ``rxq_cqe_pad_en`` parameter [int] > > > > @@ -296,7 +280,7 @@ Run-time configuration > > > > Supported on: > > > > - - CPU having 128B cacheline with ConnectX-5 and Bluefield. > > + - CPU having 128B cacheline with ConnectX-5 and BlueField. > > > > - ``rxq_pkt_pad_en`` parameter [int] > > > > @@ -308,8 +292,8 @@ Run-time configuration > > > > Supported on: > > > > - - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > Bluefield. > > - - POWER8 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > Bluefield. > > + - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > BlueField. > > + - POWER8 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > BlueField. > > > > - ``mprq_en`` parameter [int] > > > > @@ -375,13 +359,13 @@ Run-time configuration > > > > This option should be used in combination with ``txq_inline`` above. > > > > - On ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and Bluefield > > without > > + On ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and BlueField > > + without > > Enhanced MPW: > > > > - Disabled by default. > > - In case ``txq_inline`` is set recommendation is 4. > > > > - On ConnectX-5, ConnectX-6 and Bluefield with Enhanced MPW: > > + On ConnectX-5, ConnectX-6 and BlueField with Enhanced MPW: > > > > - Set to 8 by default. > > > > @@ -395,14 +379,14 @@ Run-time configuration > > - Set to 8 by default on ARMv8. > > - Set to 4 by default otherwise. > > > > - On Bluefield > > + On BlueField > > > > - Set to 16 by default. > > > > - ``txq_mpw_en`` parameter [int] > > > > A nonzero value enables multi-packet send (MPS) for ConnectX-4 Lx an= d > > - enhanced multi-packet send (Enhanced MPS) for ConnectX-5, ConnectX-6 > > and Bluefield. > > + enhanced multi-packet send (Enhanced MPS) for ConnectX-5, ConnectX-6 > > and BlueField. > > MPS allows the TX burst function to pack up multiple packets in a > > single descriptor session in order to save PCI bandwidth and improve > > performance at the cost of a slightly higher CPU usage. When @@ -417= ,13 > > +401,13 @@ Run-time configuration > > DEV_TX_OFFLOAD_VXLAN_TNL_TSO, DEV_TX_OFFLOAD_GRE_TNL_TSO, > > DEV_TX_OFFLOAD_VLAN_INSERT``. > > When those offloads are requested the MPS send function will not be > > used. > > > > - It is currently only supported on the ConnectX-4 Lx, ConnectX-5, Con= nectX- > > 6 and Bluefield > > + It is currently only supported on the ConnectX-4 Lx, ConnectX-5, > > + ConnectX-6 and BlueField > > families of adapters. > > On ConnectX-4 Lx the MPW is considered un-secure hence disabled by > > default. > > Users which enable the MPW should be aware that application which > > provides incorrect > > mbuf descriptors in the Tx burst can lead to serious errors in the h= ost > > including, on some cases, > > NIC to get stuck. > > - On ConnectX-5, ConnectX-6 and Bluefield the MPW is secure and enable= d > > by default. > > + On ConnectX-5, ConnectX-6 and BlueField the MPW is secure and enable= d > > by default. > > > > - ``txq_mpw_hdr_dseg_en`` parameter [int] > > > > @@ -443,14 +427,14 @@ Run-time configuration > > > > - ``tx_vec_en`` parameter [int] > > > > - A nonzero value enables Tx vector on ConnectX-5, ConnectX-6 and > > Bluefield NICs if the number of > > + A nonzero value enables Tx vector on ConnectX-5, ConnectX-6 and > > + BlueField NICs if the number of > > global Tx queues on the port is less than ``txqs_max_vec``. > > > > This option cannot be used with certain offloads such as > > ``DEV_TX_OFFLOAD_TCP_TSO, > > DEV_TX_OFFLOAD_VXLAN_TNL_TSO, DEV_TX_OFFLOAD_GRE_TNL_TSO, > > DEV_TX_OFFLOAD_VLAN_INSERT``. > > When those offloads are requested the MPS send function will not be > > used. > > > > - Enabled by default on ConnectX-5, ConnectX-6 and Bluefield. > > + Enabled by default on ConnectX-5, ConnectX-6 and BlueField. > > > > - ``rx_vec_en`` parameter [int] > > > > @@ -480,10 +464,15 @@ Run-time configuration > > > > A nonzero value enables the DV flow steering assuming it is supporte= d > > by the driver. > > - The DV flow steering is not supported on switchdev mode. > > > > Disabled by default. > > > > +- ``dv_esw_en`` parameter [int] > > + > > + A nonzero value enables E-Switch using Direct Rules. > > + > > + Enabled by default if supported. > > + > > - ``mr_ext_memseg_en`` parameter [int] > > > > A nonzero value enables extending memseg when registering DMA > > memory. If @@ -545,7 +534,7 @@ DPDK and must be installed separately: > > - **libmlx5** > > > > Low-level user space driver library for Mellanox > > - ConnectX-4/ConnectX-5/ConnectX-6/Bluefield devices, it is automatica= lly > > loaded > > + ConnectX-4/ConnectX-5/ConnectX-6/BlueField devices, it is > > + automatically loaded > > by libibverbs. > > > > This library basically implements send/receive calls to the hardware= @@ - > > 567,7 +556,7 @@ DPDK and must be installed separately: > > their devices: > > > > - mlx5_core: hardware driver managing Mellanox > > - ConnectX-4/ConnectX-5/ConnectX-6/Bluefield devices and related > > Ethernet kernel > > + ConnectX-4/ConnectX-5/ConnectX-6/BlueField devices and related > > + Ethernet kernel > > network devices. > > - mlx5_ib: InifiniBand device driver. > > - ib_uverbs: user space driver for Verbs (entry point for libibverbs= ). > > @@ -575,7 +564,7 @@ DPDK and must be installed separately: > > - **Firmware update** > > > > Mellanox OFED/EN releases include firmware updates for > > - ConnectX-4/ConnectX-5/ConnectX-6/Bluefield adapters. > > + ConnectX-4/ConnectX-5/ConnectX-6/BlueField adapters. > > > > Because each release provides new features, these updates must be > > applied to > > match the kernel modules and libraries they come with. > > @@ -595,7 +584,7 @@ releases. > > RDMA Core with Linux Kernel > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > > -- Minimal kernel version : v4.14 or the most recent 4.14-rc (see `Linu= x > > installation documentation`_) > > +- Minimal kernel version : v4.14 or the most recent 5.1.0-rc (see > > +`Linux installation documentation`_) > > - Minimal rdma-core version: v15+ commit 0c5f5765213a ("Merge pull > > request #227 from yishaih/tm") > > (see `RDMA Core installation documentation`_) > > - When building for i686 use: > > @@ -622,7 +611,8 @@ thanks to these environment variables: > > Mellanox OFED/EN > > ^^^^^^^^^^^^^^^^ > > > > -- Mellanox OFED version: **4.4, 4.5** / Mellanox EN version: **4.5** > > +- Mellanox OFED version: ** 4.5, 4.6** / > > + Mellanox EN version: **4.5, 4.6** > > - firmware version: > > > > - ConnectX-4: **12.21.1000** and above. > > @@ -630,7 +620,7 @@ Mellanox OFED/EN > > - ConnectX-5: **16.21.1000** and above. > > - ConnectX-5 Ex: **16.21.1000** and above. > > - ConnectX-6: **20.99.5374** and above. > > - - Bluefield: **18.99.3950** and above. > > + - BlueField: **18.25.1010** and above. > > > > While these libraries and kernel modules are available on OpenFabrics > > Alliance's `website > > > w.openfabrics.org%2F&data=3D02%7C01%7Cshahafs%40mellanox.com%7 > > > C6432343b75294926c94c08d6d71918dc%7Ca652971c7d2e4d9ba6a4d149256f4 > > > 61b%7C0%7C0%7C636932900128053421&sdata=3Dd8coRBxwMI62n%2FB1I > > E4mOcoVHs4lsiDl7kSgVljJBeY%3D&reserved=3D0>`__ and provided by > > package @@ -766,6 +756,34 @@ Quick Start Guide on OFED/EN 6. Compile > > DPDK and you are ready to go. See instructions on > > :ref:`Development Kit Build System ` > > > > +Enable E-Switch mode > > +-------------------- >=20 > It is how to enable switchdev mode right? Not e-switch mode. > Need to put a few words on what it is switchdev mode and why it is needed= . >=20 O.K will add some words. > > + > > +1. Enable SRIOV mode: > > + > > + .. code-block:: console > > + > > + mlxconfig -d set SRIOV_EN=3Dtrue > > + > > +2. Configure the number of VFs: >=20 > This is the max number of VFs. > Will fix. =20 > > + > > + .. code-block:: console > > + > > + mlxconfig -d set NUM_OF_VFS=3D > > + echo /sys/class/net/ > + device>/device/sriov_numvfs >=20 > Need to reset FW after it. >=20 I don't think reset FW is required. > Also - where do you create the actual VFs? >=20 The in the second line: echo /sys/class/net//device/sriov_numvfs > > + > > +3. Unbind the device (can be rebind after the switchdev mode): > > + > > + .. code-block:: console > > + > > + echo -n " > > + /sys/bus/pci/drivers/mlx5_core/unbind > > + > > +4. Enbale switchdev mode: > > + > > + .. code-block:: console > > + > > + echo switchdev > /sys/class/net/ > + device>/compat/devlink/mode > > + > > Performance tuning > > ------------------ > > > > @@ -842,6 +860,38 @@ Performance tuning > > - Configure per-lcore cache when creating Mempools for packet buffe= r. > > - Refrain from dynamically allocating/freeing memory in run-time. > > > > +Supported hardware offloads using rte_flow API > > +---------------------------------------------- > > + > > +.. _Supported hardware offloads using rte_flow API: > > + > > +.. table:: Supported hardware offloads using rte_flow API > > + > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Offload | E-Switch | NIC PF | VF representor | PF re= presentor | > > + | | (SRIOV) | (non SRIOV) | (SRIOV) | (SRIO= V) | > > + > > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > > =3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > > + | Encapsulation | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Multi table | V | V | V | = V* | > > + | support | | | | = | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | NAT | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Routing | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | TTL inc/dec | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Count | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Drop | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Mark | N/A | V | V | = V* | > > + > > + +---------------+-----------+-------------+-----------------+--------= - > > + -------+ > > + > > +| V ConnectX-5 > > +| V* ConnectX-5 & BlueField > > + >=20 > What is the difference between E-Switch (SRIOV), VF representor (SRIOV) a= nd > PF representor (SRIOV)? On switchdev mode every rule is either for the > VF/uplink representor. >=20 > The matrix is a bit more complex than what you show here. > IMO- the matrix has 2 column (E-Switch and NIC) and 2/3 dimensions (mini= mal > NIC that support, minimal DPDK that support, minimal OFED/upstream kernel > which support(?)). > The columns need to be RTE_FLOW actions (let's leave the RTE_FLOW items > aside for a moment). Because NAT, for example, may require different acti= ons > on different implementations. >=20 PSB my answer above. >=20 > > Notes for testpmd > > ----------------- > > > > @@ -863,7 +913,7 @@ Usage example > > ------------- > > > > This section demonstrates how to launch **testpmd** with Mellanox - > > ConnectX-4/ConnectX-5/ConnectX-6/Bluefield devices managed by > > librte_pmd_mlx5. > > +ConnectX-4/ConnectX-5/ConnectX-6/BlueField devices managed by > > librte_pmd_mlx5. > > > > #. Load the kernel modules: > > > > diff --git a/doc/guides/rel_notes/release_19_05.rst > > b/doc/guides/rel_notes/release_19_05.rst > > index 4e0eed5..489273b 100644 > > --- a/doc/guides/rel_notes/release_19_05.rst > > +++ b/doc/guides/rel_notes/release_19_05.rst > > @@ -148,10 +148,16 @@ New Features > > * Added support for multiport InfiniBand device. > > * Added control of excessive memory pinning by kernel. > > * Added support of DMA memory registration by secondary process. > > - * Added Direct Rule support in Direct Verbs flow driver. > > * Added support of per-process device registers, reserving identica= l VA > > space > > is not needed anymore. > > - * Added E-Switch support in Direct Verbs flow driver. > > + * Added Direct Rule support for Nic steering, in Direct Verbs drive= r. > > + * Added Direct Rule support for E-Switch steering, in Direct Verbs = driver. >=20 > People who read the release notes doesn't necessarily read the entire mlx= 5 > manual. It is not clear what Direct rule is and why it is good. > I would drop this part. >=20 > This was done by Olga request. But I don't care to remove it. =20 > > + * Added support for jump action for both E-Switch and Nic when usin= g > > + Direct Rules. >=20 > How can user know if it uses Direct rule or not? I would just stick with: > "Support rte_flow jump action" >=20 O.K=20 > > + * Added Group Support for Nic steering when using Direct Rules. >=20 > Again: > "Support multiple rte_flow groups" >=20 > > + * Support millions of offloaded flow rules. > > + * Improved flow insertion rate to millions of flows per second when= using > > + Direct Rules and group larger then 0. >=20 > I would say: > * flow engine re-design to support large scale deployments. This include: > 1. support millions of rte_flow rules > 2. fast flow insertion and deletion up to 1M flow update per second. >=20 > O.K. =20 > > > > * **Renamed avf to iavf.** > > > > -- > > 1.8.3.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id E2A49A00E6 for ; Mon, 13 May 2019 08:43:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 868A62082; Mon, 13 May 2019 08:43:25 +0200 (CEST) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00064.outbound.protection.outlook.com [40.107.0.64]) by dpdk.org (Postfix) with ESMTP id 61FAD3DC for ; Mon, 13 May 2019 08:43:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DE1EaMWaNxsu+UK21UUzqUSWMieOyvaZ+bBpJVSdp70=; b=h8E9BLcjfngy9cAA4IMx8qPygs3TIiD6bot4SuYTplj+K0HweehaOXe3xf+DYhok0BoAujTKkILSaxVA1aYA31tnA5pZX3UD9MHrfqGRWdPxsgEcK7NWYEXU/nM+zoQZpAAPGStsxtTB+Ew5AO/mxnVmOy6LYUJDwKuuZqfszpc= Received: from AM4PR05MB3425.eurprd05.prod.outlook.com (10.171.190.15) by AM4PR05MB3489.eurprd05.prod.outlook.com (10.170.126.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1878.21; Mon, 13 May 2019 06:43:22 +0000 Received: from AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::c9ea:c445:68e5:942]) by AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::c9ea:c445:68e5:942%4]) with mapi id 15.20.1878.024; Mon, 13 May 2019 06:43:22 +0000 From: Ori Kam To: Shahaf Shuler , Yongseok Koh , Matan Azrad , Thomas Monjalon CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v2] doc: fix update release notes for Mellanox drivers Thread-Index: AQHVCQH58QJ8qRjqYEyP1YsrsfA2YaZohskAgAAOXIA= Date: Mon, 13 May 2019 06:43:22 +0000 Message-ID: References: <1557685859-121772-1-git-send-email-orika@mellanox.com> <1557693197-129982-1-git-send-email-orika@mellanox.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a2585eda-a532-436a-6360-08d6d76e4b26 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(4618075)(2017052603328)(7193020); SRVR:AM4PR05MB3489; x-ms-traffictypediagnostic: AM4PR05MB3489: x-ms-exchange-purlcount: 2 x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7219; x-forefront-prvs: 0036736630 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(136003)(396003)(376002)(366004)(346002)(199004)(189003)(13464003)(8936002)(25786009)(316002)(486006)(186003)(76116006)(66946007)(446003)(73956011)(26005)(66446008)(64756008)(66556008)(66476007)(33656002)(476003)(11346002)(81166006)(55016002)(2906002)(229853002)(53946003)(6306002)(9686003)(305945005)(6116002)(3846002)(19627235002)(6436002)(52536014)(7736002)(81156014)(256004)(14444005)(5024004)(76176011)(6506007)(71190400001)(71200400001)(53936002)(478600001)(14454004)(966005)(68736007)(7110500001)(2420400007)(15650500001)(6246003)(86362001)(66066001)(74316002)(53546011)(102836004)(30864003)(110136005)(5660300002)(99286004)(7696005)(4326008)(21314003)(579004); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3489; H:AM4PR05MB3425.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 44jiUGeRqT5tVs21xs9kW4RJd6N3zJHBDgt2b2+ZWUUon4KUGL3mqkr+BAtOoQznWL3zpUga1NiqcCBx6GiR5026u7l4NqhJ7zVtacO6YLJ1qNoWFpUX/FJ/OO6dh+7o/RoVM/ga7Go2opSUyqYBM4eN00vP2Yuy3hrkaFrCQEjVKywP8bVXNAynQatHvFqdhpLNsanxgFlMGaLerqZ0aZBCs4sFRLegVV6pq6GEAr4X+MFSxT1nUmWrkjKoNX6ac/v+YpEGf2ZbpY5sV/FFalWZ9IaEvRqvB6kr+UJN3gXhA2qPcIh4jSqtkPP7s/hie+kFTucIL7O+BjcvEC74HC6y2r4cmWgGbCoN7KXmGraIGMJikSUlXvrMeHWvoA9RNZcSX+vz3+h1hRX+zLlje9+YR/YFUqdUNkTJb835LIE= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: a2585eda-a532-436a-6360-08d6d76e4b26 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 May 2019 06:43:22.3946 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3489 Subject: Re: [dpdk-dev] [PATCH v2] doc: fix update release notes for Mellanox drivers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190513064322.MZvpGtqItzXuNwakMvKh3VY4TL5fejKLQJTS2N2CFII@z> Hi, PSB > -----Original Message----- > From: Shahaf Shuler > Sent: Monday, May 13, 2019 8:26 AM > To: Ori Kam ; Yongseok Koh ; > Matan Azrad ; Thomas Monjalon > > Cc: dev@dpdk.org; Ori Kam > Subject: RE: [dpdk-dev] [PATCH v2] doc: fix update release notes for Mell= anox > drivers >=20 > Hi Ori, >=20 > See some comments below. >=20 > Sunday, May 12, 2019 11:33 PM, Ori Kam: > > Subject: [dpdk-dev] [PATCH v2] doc: fix update release notes for Mellan= ox > > drivers > > > > This patch adds some missing features to Mellanox drivers release notes= . > > It also updates the mlx5/mlx4 documentations. > > > > Fixes: d85b204b5dba ("doc: update release notes for Mellanox drivers") > > Cc: yskoh@mellanox.com > > > > Signed-off-by: Ori Kam > > > > --- > > > > v2: > > * fix checkpatch warning. > > > > --- > > doc/guides/nics/mlx4.rst | 2 +- > > doc/guides/nics/mlx5.rst | 150 ++++++++++++++++++++++---= -------- > > doc/guides/rel_notes/release_19_05.rst | 10 ++- > > 3 files changed, 109 insertions(+), 53 deletions(-) > > > > diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst index > > f6d7a16..5c6bbde 100644 > > --- a/doc/guides/nics/mlx4.rst > > +++ b/doc/guides/nics/mlx4.rst > > @@ -253,7 +253,7 @@ thanks to these environment variables: > > Mellanox OFED as a fallback > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > -- `Mellanox OFED`_ version: **4.4, 4.5**. > > +- `Mellanox OFED`_ version: **4.4, 4.5, 4.6**. > > - firmware version: **2.42.5000** and above. > > > > .. _`Mellanox OFED`: > > http://www.mellanox.com/page/products_dyn?product_family=3D26&mtag=3Dli > > nux_sw_drivers > > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index > > 325e9f6..5cf7744 100644 > > --- a/doc/guides/nics/mlx5.rst > > +++ b/doc/guides/nics/mlx5.rst > > @@ -7,7 +7,7 @@ MLX5 poll mode driver > > > > The MLX5 poll mode driver library (**librte_pmd_mlx5**) provides suppo= rt > > for **Mellanox ConnectX-4**, **Mellanox ConnectX-4 Lx** , **Mellanox - > > ConnectX-5**, **Mellanox ConnectX-6** and **Mellanox Bluefield** > > families > > +ConnectX-5**, **Mellanox ConnectX-6** and **Mellanox BlueField** > > +families > > of 10/25/40/50/100/200 Gb/s adapters as well as their virtual function= s (VF) > > in SR-IOV context. > > > > @@ -62,8 +62,8 @@ Features > > - RX VLAN stripping. > > - TX VLAN insertion. > > - RX CRC stripping configuration. > > -- Promiscuous mode. > > -- Multicast promiscuous mode. > > +- Promiscuous mode on PF and VF. > > +- Multicast promiscuous mode on PF and VF. > > - Hardware checksum offloads. > > - Flow director (RTE_FDIR_MODE_PERFECT, > > RTE_FDIR_MODE_PERFECT_MAC_VLAN and > > RTE_ETH_FDIR_REJECT). > > @@ -78,6 +78,11 @@ Features > > - Rx HW timestamp. > > - Tunnel types: VXLAN, L3 VXLAN, VXLAN-GPE, GRE, MPLSoGRE, MPLSoUDP. > > - Tunnel HW offloads: packet type, inner/outer RSS, IP and UDP checksu= m > > verification. > > +- Nic HW offloads: encapsulation (vxlan, gre, mplsoudp, mplsogre), NAT= , > > +routing, TTL > > + increment/decrement, count, drop, mark. For details please see > > :ref:`Supported hardware offloads using rte_flow API`. > > +- Flow insertion rate of more then million flows per second. When usin= g > > Direct Rules. >=20 > Replace '.' with ',' . >=20 O.K > > +- Support groups. >=20 > Support for multiple rte_flow groups. >=20 O.K > > +- Support raw encapsulation offload. >=20 > Can't it be part of the encapsulation part above? >=20 Do you mean the tunnel HW offloads? If yes I guess it can be there. > > > > Limitations > > ----------- > > @@ -112,8 +117,6 @@ Limitations > > is set to multi-packet send or Enhanced multi-packet send. Otherwise= it > > must have > > less than 50 segments. > > > > -- Count action for RTE flow is **only supported in Mellanox OFED**. > > - > > - Flows with a VXLAN Network Identifier equal (or ends to be equal) > > to 0 are not supported. > > > > @@ -124,13 +127,6 @@ Limitations > > - VF: flow rules created on VF devices can only match traffic targeted= at the > > configured MAC addresses (see ``rte_eth_dev_mac_addr_add()``). > > > > -.. note:: > > - > > - MAC addresses not already present in the bridge table of the associ= ated > > - kernel network device will be added and cleaned up by the PMD when > > closing > > - the device. In case of ungraceful program termination, some entries= may > > - remain present and should be removed manually by other means. > > - >=20 > This is still correct, why removed? >=20 Will return the code. > > - When Multi-Packet Rx queue is configured (``mprq_en``), a Rx packet = can > > be > > externally attached to a user-provided mbuf with having > > EXT_ATTACHED_MBUF in > > ol_flags. As the mempool for the external buffer is managed by PMD, = all > > the @@ -147,30 +143,18 @@ Limitations > > To receive IPv6 Multicast messages on VM, explicitly set the relevan= t > > MAC address using rte_eth_dev_mac_addr_add() API. > > > > -- E-Switch VXLAN tunnel is not supported together with outer VLAN. > > - > > -- E-Switch Flows with VNI pattern must include the VXLAN decapsulation > > action. > > - > > -- E-Switch VXLAN decapsulation Flow: > > +- E-Switch decapsulation Flow: > > > > - can be applied to PF port only. > > - must specify VF port action (packet redirection from PF to VF). > > - - must specify tunnel outer UDP local (destination) port, wildcards = not > > allowed. > > - - must specify tunnel outer VNI, wildcards not allowed. > > - - must specify tunnel outer local (destination) IPv4 or IPv6 addres= s, > > wildcards not allowed. > > - - optionally may specify tunnel outer remote (source) IPv4 or IPv6, > > wildcards or group IPs allowed. > > - optionally may specify tunnel inner source and destination MAC > > addresses. > > > > -- E-Switch VXLAN encapsulation Flow: > > +- E-Switch encapsulation Flow: > > > > - can be applied to VF ports only. > > - must specify PF port action (packet redirection from VF to PF). > > - - must specify the VXLAN item with tunnel outer parameters. > > - - must specify the tunnel outer VNI in the VXLAN item. > > - - must specify the tunnel outer remote (destination) UDP port in the > > VXLAN item. > > - - must specify the tunnel outer local (source) IPv4 or IPv6 in the ,= this > > address will locally (with scope link) assigned to the outer network in= terface, > > wildcards not allowed. > > - - must specify the tunnel outer remote (destination) IPv4 or IPv6 in= the > > VXLAN item, group IPs allowed. > > - - must specify the tunnel outer destination MAC address in the VXLAN > > item, this address will be used to create neigh rule. > > + > > +- Groups, E-Switch steering, fast inseration rate are supported only w= ith > > Mellanox OFED 4.6.2 and above. >=20 > I think the right place for such limitations is the table below (Supporte= d > hardware offloads using rte_flow API). >=20 O.K will update to something like this: Action | Nic | E-Switch Count | DPDK > 19.02 | DPDK > 19.05 | OFED > 4.6.2 / RDMA > xxx | OFED > 4.6.2 / RDMA > xxx Is that O.K? =20 > > > > Statistics > > ---------- > > @@ -227,7 +211,7 @@ These options can be modified in the ``.config`` fi= le. > > > > .. note:: > > > > - For Bluefield, target should be set to ``arm64-bluefield-linux-gcc`= `. This > > + For BlueField, target should be set to > > + ``arm64-bluefield-linux-gcc``. This > > will enable ``CONFIG_RTE_LIBRTE_MLX5_PMD`` and set > > ``RTE_CACHE_LINE_SIZE`` to > > 64. Default armv8a configuration of make build and meson build set = it to > > 128 > > then brings performance degradation. > > @@ -277,8 +261,8 @@ Run-time configuration > > > > Supported on: > > > > - - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > Bluefield. > > - - POWER8 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > Bluefield. > > + - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > BlueField. > > + - POWER9 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > BlueField. > > > > - ``rxq_cqe_pad_en`` parameter [int] > > > > @@ -296,7 +280,7 @@ Run-time configuration > > > > Supported on: > > > > - - CPU having 128B cacheline with ConnectX-5 and Bluefield. > > + - CPU having 128B cacheline with ConnectX-5 and BlueField. > > > > - ``rxq_pkt_pad_en`` parameter [int] > > > > @@ -308,8 +292,8 @@ Run-time configuration > > > > Supported on: > > > > - - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > Bluefield. > > - - POWER8 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > Bluefield. > > + - x86_64 with ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > BlueField. > > + - POWER8 and ARMv8 with ConnectX-4 LX, ConnectX-5, ConnectX-6 and > > BlueField. > > > > - ``mprq_en`` parameter [int] > > > > @@ -375,13 +359,13 @@ Run-time configuration > > > > This option should be used in combination with ``txq_inline`` above. > > > > - On ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and Bluefield > > without > > + On ConnectX-4, ConnectX-4 LX, ConnectX-5, ConnectX-6 and BlueField > > + without > > Enhanced MPW: > > > > - Disabled by default. > > - In case ``txq_inline`` is set recommendation is 4. > > > > - On ConnectX-5, ConnectX-6 and Bluefield with Enhanced MPW: > > + On ConnectX-5, ConnectX-6 and BlueField with Enhanced MPW: > > > > - Set to 8 by default. > > > > @@ -395,14 +379,14 @@ Run-time configuration > > - Set to 8 by default on ARMv8. > > - Set to 4 by default otherwise. > > > > - On Bluefield > > + On BlueField > > > > - Set to 16 by default. > > > > - ``txq_mpw_en`` parameter [int] > > > > A nonzero value enables multi-packet send (MPS) for ConnectX-4 Lx an= d > > - enhanced multi-packet send (Enhanced MPS) for ConnectX-5, ConnectX-6 > > and Bluefield. > > + enhanced multi-packet send (Enhanced MPS) for ConnectX-5, ConnectX-6 > > and BlueField. > > MPS allows the TX burst function to pack up multiple packets in a > > single descriptor session in order to save PCI bandwidth and improve > > performance at the cost of a slightly higher CPU usage. When @@ -417= ,13 > > +401,13 @@ Run-time configuration > > DEV_TX_OFFLOAD_VXLAN_TNL_TSO, DEV_TX_OFFLOAD_GRE_TNL_TSO, > > DEV_TX_OFFLOAD_VLAN_INSERT``. > > When those offloads are requested the MPS send function will not be > > used. > > > > - It is currently only supported on the ConnectX-4 Lx, ConnectX-5, Con= nectX- > > 6 and Bluefield > > + It is currently only supported on the ConnectX-4 Lx, ConnectX-5, > > + ConnectX-6 and BlueField > > families of adapters. > > On ConnectX-4 Lx the MPW is considered un-secure hence disabled by > > default. > > Users which enable the MPW should be aware that application which > > provides incorrect > > mbuf descriptors in the Tx burst can lead to serious errors in the h= ost > > including, on some cases, > > NIC to get stuck. > > - On ConnectX-5, ConnectX-6 and Bluefield the MPW is secure and enable= d > > by default. > > + On ConnectX-5, ConnectX-6 and BlueField the MPW is secure and enable= d > > by default. > > > > - ``txq_mpw_hdr_dseg_en`` parameter [int] > > > > @@ -443,14 +427,14 @@ Run-time configuration > > > > - ``tx_vec_en`` parameter [int] > > > > - A nonzero value enables Tx vector on ConnectX-5, ConnectX-6 and > > Bluefield NICs if the number of > > + A nonzero value enables Tx vector on ConnectX-5, ConnectX-6 and > > + BlueField NICs if the number of > > global Tx queues on the port is less than ``txqs_max_vec``. > > > > This option cannot be used with certain offloads such as > > ``DEV_TX_OFFLOAD_TCP_TSO, > > DEV_TX_OFFLOAD_VXLAN_TNL_TSO, DEV_TX_OFFLOAD_GRE_TNL_TSO, > > DEV_TX_OFFLOAD_VLAN_INSERT``. > > When those offloads are requested the MPS send function will not be > > used. > > > > - Enabled by default on ConnectX-5, ConnectX-6 and Bluefield. > > + Enabled by default on ConnectX-5, ConnectX-6 and BlueField. > > > > - ``rx_vec_en`` parameter [int] > > > > @@ -480,10 +464,15 @@ Run-time configuration > > > > A nonzero value enables the DV flow steering assuming it is supporte= d > > by the driver. > > - The DV flow steering is not supported on switchdev mode. > > > > Disabled by default. > > > > +- ``dv_esw_en`` parameter [int] > > + > > + A nonzero value enables E-Switch using Direct Rules. > > + > > + Enabled by default if supported. > > + > > - ``mr_ext_memseg_en`` parameter [int] > > > > A nonzero value enables extending memseg when registering DMA > > memory. If @@ -545,7 +534,7 @@ DPDK and must be installed separately: > > - **libmlx5** > > > > Low-level user space driver library for Mellanox > > - ConnectX-4/ConnectX-5/ConnectX-6/Bluefield devices, it is automatica= lly > > loaded > > + ConnectX-4/ConnectX-5/ConnectX-6/BlueField devices, it is > > + automatically loaded > > by libibverbs. > > > > This library basically implements send/receive calls to the hardware= @@ - > > 567,7 +556,7 @@ DPDK and must be installed separately: > > their devices: > > > > - mlx5_core: hardware driver managing Mellanox > > - ConnectX-4/ConnectX-5/ConnectX-6/Bluefield devices and related > > Ethernet kernel > > + ConnectX-4/ConnectX-5/ConnectX-6/BlueField devices and related > > + Ethernet kernel > > network devices. > > - mlx5_ib: InifiniBand device driver. > > - ib_uverbs: user space driver for Verbs (entry point for libibverbs= ). > > @@ -575,7 +564,7 @@ DPDK and must be installed separately: > > - **Firmware update** > > > > Mellanox OFED/EN releases include firmware updates for > > - ConnectX-4/ConnectX-5/ConnectX-6/Bluefield adapters. > > + ConnectX-4/ConnectX-5/ConnectX-6/BlueField adapters. > > > > Because each release provides new features, these updates must be > > applied to > > match the kernel modules and libraries they come with. > > @@ -595,7 +584,7 @@ releases. > > RDMA Core with Linux Kernel > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > > -- Minimal kernel version : v4.14 or the most recent 4.14-rc (see `Linu= x > > installation documentation`_) > > +- Minimal kernel version : v4.14 or the most recent 5.1.0-rc (see > > +`Linux installation documentation`_) > > - Minimal rdma-core version: v15+ commit 0c5f5765213a ("Merge pull > > request #227 from yishaih/tm") > > (see `RDMA Core installation documentation`_) > > - When building for i686 use: > > @@ -622,7 +611,8 @@ thanks to these environment variables: > > Mellanox OFED/EN > > ^^^^^^^^^^^^^^^^ > > > > -- Mellanox OFED version: **4.4, 4.5** / Mellanox EN version: **4.5** > > +- Mellanox OFED version: ** 4.5, 4.6** / > > + Mellanox EN version: **4.5, 4.6** > > - firmware version: > > > > - ConnectX-4: **12.21.1000** and above. > > @@ -630,7 +620,7 @@ Mellanox OFED/EN > > - ConnectX-5: **16.21.1000** and above. > > - ConnectX-5 Ex: **16.21.1000** and above. > > - ConnectX-6: **20.99.5374** and above. > > - - Bluefield: **18.99.3950** and above. > > + - BlueField: **18.25.1010** and above. > > > > While these libraries and kernel modules are available on OpenFabrics > > Alliance's `website > > > w.openfabrics.org%2F&data=3D02%7C01%7Cshahafs%40mellanox.com%7 > > > C6432343b75294926c94c08d6d71918dc%7Ca652971c7d2e4d9ba6a4d149256f4 > > > 61b%7C0%7C0%7C636932900128053421&sdata=3Dd8coRBxwMI62n%2FB1I > > E4mOcoVHs4lsiDl7kSgVljJBeY%3D&reserved=3D0>`__ and provided by > > package @@ -766,6 +756,34 @@ Quick Start Guide on OFED/EN 6. Compile > > DPDK and you are ready to go. See instructions on > > :ref:`Development Kit Build System ` > > > > +Enable E-Switch mode > > +-------------------- >=20 > It is how to enable switchdev mode right? Not e-switch mode. > Need to put a few words on what it is switchdev mode and why it is needed= . >=20 O.K will add some words. > > + > > +1. Enable SRIOV mode: > > + > > + .. code-block:: console > > + > > + mlxconfig -d set SRIOV_EN=3Dtrue > > + > > +2. Configure the number of VFs: >=20 > This is the max number of VFs. > Will fix. =20 > > + > > + .. code-block:: console > > + > > + mlxconfig -d set NUM_OF_VFS=3D > > + echo /sys/class/net/ > + device>/device/sriov_numvfs >=20 > Need to reset FW after it. >=20 I don't think reset FW is required. > Also - where do you create the actual VFs? >=20 The in the second line: echo /sys/class/net//device/sriov_numvfs > > + > > +3. Unbind the device (can be rebind after the switchdev mode): > > + > > + .. code-block:: console > > + > > + echo -n " > > + /sys/bus/pci/drivers/mlx5_core/unbind > > + > > +4. Enbale switchdev mode: > > + > > + .. code-block:: console > > + > > + echo switchdev > /sys/class/net/ > + device>/compat/devlink/mode > > + > > Performance tuning > > ------------------ > > > > @@ -842,6 +860,38 @@ Performance tuning > > - Configure per-lcore cache when creating Mempools for packet buffe= r. > > - Refrain from dynamically allocating/freeing memory in run-time. > > > > +Supported hardware offloads using rte_flow API > > +---------------------------------------------- > > + > > +.. _Supported hardware offloads using rte_flow API: > > + > > +.. table:: Supported hardware offloads using rte_flow API > > + > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Offload | E-Switch | NIC PF | VF representor | PF re= presentor | > > + | | (SRIOV) | (non SRIOV) | (SRIOV) | (SRIO= V) | > > + > > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > > =3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > > + | Encapsulation | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Multi table | V | V | V | = V* | > > + | support | | | | = | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | NAT | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Routing | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | TTL inc/dec | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Count | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Drop | V | V | V | = V* | > > + +---------------+-----------+-------------+-----------------+------= ----------+ > > + | Mark | N/A | V | V | = V* | > > + > > + +---------------+-----------+-------------+-----------------+--------= - > > + -------+ > > + > > +| V ConnectX-5 > > +| V* ConnectX-5 & BlueField > > + >=20 > What is the difference between E-Switch (SRIOV), VF representor (SRIOV) a= nd > PF representor (SRIOV)? On switchdev mode every rule is either for the > VF/uplink representor. >=20 > The matrix is a bit more complex than what you show here. > IMO- the matrix has 2 column (E-Switch and NIC) and 2/3 dimensions (mini= mal > NIC that support, minimal DPDK that support, minimal OFED/upstream kernel > which support(?)). > The columns need to be RTE_FLOW actions (let's leave the RTE_FLOW items > aside for a moment). Because NAT, for example, may require different acti= ons > on different implementations. >=20 PSB my answer above. >=20 > > Notes for testpmd > > ----------------- > > > > @@ -863,7 +913,7 @@ Usage example > > ------------- > > > > This section demonstrates how to launch **testpmd** with Mellanox - > > ConnectX-4/ConnectX-5/ConnectX-6/Bluefield devices managed by > > librte_pmd_mlx5. > > +ConnectX-4/ConnectX-5/ConnectX-6/BlueField devices managed by > > librte_pmd_mlx5. > > > > #. Load the kernel modules: > > > > diff --git a/doc/guides/rel_notes/release_19_05.rst > > b/doc/guides/rel_notes/release_19_05.rst > > index 4e0eed5..489273b 100644 > > --- a/doc/guides/rel_notes/release_19_05.rst > > +++ b/doc/guides/rel_notes/release_19_05.rst > > @@ -148,10 +148,16 @@ New Features > > * Added support for multiport InfiniBand device. > > * Added control of excessive memory pinning by kernel. > > * Added support of DMA memory registration by secondary process. > > - * Added Direct Rule support in Direct Verbs flow driver. > > * Added support of per-process device registers, reserving identica= l VA > > space > > is not needed anymore. > > - * Added E-Switch support in Direct Verbs flow driver. > > + * Added Direct Rule support for Nic steering, in Direct Verbs drive= r. > > + * Added Direct Rule support for E-Switch steering, in Direct Verbs = driver. >=20 > People who read the release notes doesn't necessarily read the entire mlx= 5 > manual. It is not clear what Direct rule is and why it is good. > I would drop this part. >=20 > This was done by Olga request. But I don't care to remove it. =20 > > + * Added support for jump action for both E-Switch and Nic when usin= g > > + Direct Rules. >=20 > How can user know if it uses Direct rule or not? I would just stick with: > "Support rte_flow jump action" >=20 O.K=20 > > + * Added Group Support for Nic steering when using Direct Rules. >=20 > Again: > "Support multiple rte_flow groups" >=20 > > + * Support millions of offloaded flow rules. > > + * Improved flow insertion rate to millions of flows per second when= using > > + Direct Rules and group larger then 0. >=20 > I would say: > * flow engine re-design to support large scale deployments. This include: > 1. support millions of rte_flow rules > 2. fast flow insertion and deletion up to 1M flow update per second. >=20 > O.K. =20 > > > > * **Renamed avf to iavf.** > > > > -- > > 1.8.3.1