From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E9885A0093; Tue, 19 May 2020 00:04:22 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 586371D515; Tue, 19 May 2020 00:04:22 +0200 (CEST) Received: from mail-ot1-f48.google.com (mail-ot1-f48.google.com [209.85.210.48]) by dpdk.org (Postfix) with ESMTP id A0A2C1D509 for ; Tue, 19 May 2020 00:04:20 +0200 (CEST) Received: by mail-ot1-f48.google.com with SMTP id 63so9526762oto.8 for ; Mon, 18 May 2020 15:04:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=4Tzwx7ru34VN5XP/P9HeG7vM9u6DAZWiACl/EzcuZng=; b=W6xI+iE+Bqi4uesxFKnoaimZBjCbGwdbycmBUY8AAfhedMkgvtdBtsN9y9uTj0ReiS 5l8h4UPJs4IjpYY+LoRvpaQWqbx94roNA4DbAQRbcWNQowSk8Fo0hs/Rc2LRdJ7NgLJz eXRNrFqawf9/AiTaqKBjjKdOtNx/CIIUEDFBU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=4Tzwx7ru34VN5XP/P9HeG7vM9u6DAZWiACl/EzcuZng=; b=IXzs/7rYYNahFTQDP/9hzMIVvxjQE/F6mhQ6yz9kuGrmI3QB+YrMvnTQZXq875r3IL TMx3gAc78t51PwZI0edgtmdISZsha8I/IgsWxANb/Czh60HFwXarRrPr/eAgBFFBltTs bu1R647AyVfqYc9UKPXuuRheWIutc19D0ctb3MX0UH3zoZqqKUEED9ukuOrPlUvUcwYd aO1njnAsgWl1Mk4H2Pf1IpGq//62csE1HnboOj5jXUwg1fEsoYScy/Wfhcy0CTHVVE+C 0meW55tvQxOVp0cGHRYAjp2hwSJzOCGzZWxvVsZIkYlbF9prH/Sv221Ndv+1GuFdHVdb QnVQ== X-Gm-Message-State: AOAM5313prQDURRj+d4i/6PzBheXNwhAUl1Urd9JD/pJa+IAULYe0Mp1 auHbLH5ugJOpqkZY11czgYMdH20un3wVfcsVAYQFAUX+Rro= X-Google-Smtp-Source: ABdhPJydsPZhjdy3RKSfZ+JKO6tSOYNDlGYLZ9JS90u3aSjsa1IehuGW19M9vjEusnimDKUhCnP8A0iJ6cpwCnvidw8= X-Received: by 2002:a9d:5f04:: with SMTP id f4mr15248961oti.154.1589839459163; Mon, 18 May 2020 15:04:19 -0700 (PDT) MIME-Version: 1.0 References: <2f0d843f-ad72-154c-ded4-e48c2690ad10@intel.com> <20200518175820.32340-1-ajit.khaparde@broadcom.com> In-Reply-To: <20200518175820.32340-1-ajit.khaparde@broadcom.com> From: Ajit Khaparde Date: Mon, 18 May 2020 15:04:01 -0700 Message-ID: To: dpdk-dev Cc: Ferruh Yigit , JP Lee , Kovacevic Marko Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH v4] doc: update bnxt guide X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, May 18, 2020 at 10:58 AM Ajit Khaparde wrote: > - Update list of supported adapters. > - Update list of supported features. > - Add some details to describe the features. > - Remove obsolete limitations. > - Fix and update links. > > Signed-off-by: JP Lee > Signed-off-by: Ajit Khaparde > Acked-by: Kovacevic Marko > --- > v1->v2: Some lines were too long in v1. Made then shorter. Checked for > typos. > v2->v3: Removed list of extended stats. > v3->v4: Removed an irrelevant line. > Patch applied to dpdk-next-net-brcm. Thanks > --- > doc/guides/nics/bnxt.rst | 893 +++++++++++++++++++++++++++++++++------ > 1 file changed, 763 insertions(+), 130 deletions(-) > > diff --git a/doc/guides/nics/bnxt.rst b/doc/guides/nics/bnxt.rst > index 434ba9d6c..3aad7ea4a 100644 > --- a/doc/guides/nics/bnxt.rst > +++ b/doc/guides/nics/bnxt.rst > @@ -1,138 +1,771 @@ > -.. SPDX-License-Identifier: BSD-3-Clause > - Copyright 2016-2019 Broadcom > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright 2020 Broadcom Inc. > > BNXT Poll Mode Driver > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > -The BNXT PMD (**librte_pmd_bnxt**) implements support for adapters based > on > -Ethernet controllers and SoCs belonging to the **Broadcom BCM5730X > NetXtreme-C=C2=AE > -Family of Ethernet Network Controllers**, the **Broadcom BCM574XX/BCM575= XX > -NetXtreme-E=C2=AE Family of Ethernet Network Controllers**, the **Broadc= om > BCM588XX > -Stingray Family of SmartNIC Adapters**, and the **Broadcom StrataGX=C2= =AE > BCM5871X > -Series of Communications Processors**. A complete list with links to > reference > -material is included below. > +The Broadcom BNXT PMD (Poll Mode Driver, librte_pmd_bnxt) implements > support for adapters based on Ethernet controllers and SoCs belonging to > the Broadcom BCM574XX/BCM575XX NetXtreme-E=C2=AE Family of Ethernet Netwo= rk > Controllers, the Broadcom BCM588XX Stingray Family of Smart NIC Adapters, > and the Broadcom StrataGX=C2=AE BCM5873X Series of Communications Process= ors. > > +A complete list with links to reference material is in the Appendix > section. > > -BNXT PMD Features > +CPU Support > +----------- > + > +BNXT PMD supports multiple CPU architectures, including x86-32, x86-64, > and ARMv8. > + > +Kernel Dependency > +----------------- > + > +BNXT PMD requires a kernel module (VFIO or UIO) for setting up a device, > mapping device memory to userspace, registering interrupts, etc. > +VFIO is more secure than UIO, relying on IOMMU protection. > +UIO requires the IOMMU disabled or configured to pass-through mode. > + > +Operating Systems supported: > + > +* Red Hat Enterprise Linux release 8.1 (Ootpa) > +* Red Hat Enterprise Linux release 8.0 (Ootpa) > +* Red Hat Enterprise Linux Server release 7.7 (Maipo) > +* Red Hat Enterprise Linux Server release 7.6 (Maipo) > +* Red Hat Enterprise Linux Server release 7.5 (Maipo) > +* Red Hat Enterprise Linux Server release 7.4 (Maipo) > +* Red Hat Enterprise Linux Server release 7.3 (Maipo) > +* Red Hat Enterprise Linux Server release 7.2 (Maipo) > +* CentOS Linux release 8.0 > +* CentOS Linux release 7.7 > +* CentOS Linux release 7.6.1810 > +* CentOS Linux release 7.5.1804 > +* CentOS Linux release 7.4.1708 > +* Fedora 31 > +* FreeBSD 12.1 > +* Suse 15SP1 > +* Ubuntu 19.04 > +* Ubuntu 18.04 > +* Ubuntu 16.10 > +* Ubuntu 16.04 > +* Ubuntu 14.04 > + > +The BNXT PMD supports operating with: > + > +* Linux vfio-pci > +* Linux uio_pci_generic > +* Linux igb_uio > +* BSD nic_uio > + > +Compiling BNXT PMD > +------------------ > + > +To compile the BNXT PMD: > + > +.. code-block:: console > + > + make config T=3Dx86_64-native-linux-gcc && make // for x86-64 > + make config T=3Dx86_32-native-linux-gcc && make // for x86-32 > + make config T=3Darmv8a-linux-gcc && make // for ARMv8 > + > +Bind the device to one of the kernel modules listed above > + > +.. code-block:: console > + > + ./dpdk-devbind.py -b vfio-pci|igb_uio|uio_pci_generic > bus_id:device_id.function_id > + > +Load an application (e.g. testpmd) with a default configuration (e.g. a > single TX /RX queue): > + > +.. code-block:: console > + > + ./testpmd -c 0xF -n 4 -- -i --portmask=3D0x1 --nb-cores=3D2 > + > +Running BNXT PMD > +---------------- > + > +The BNXT PMD can run on PF or VF. > + > +PCI-SIG Single Root I/O Virtualization (SR-IOV) involves the direct > assignment of part of the network port resources to guest operating syste= ms > using the SR-IOV standard. > +NIC is logically distributed among multiple virtual machines (VMs), whil= e > still having global data in common to share with the PF and other VFs. > + > +Sysadmin can create and configure VFs: > + > +.. code-block:: console > + > + echo num_vfs > > /sys/bus/pci/devices/domain_id:bus_id:device_id:function_id/sriov_numvfs > + (ex) echo 4 > /sys/bus/pci/devices/0000:82:00:0/sriov_numvfs > + > +Sysadmin also can change the VF property such as MAC address, transparen= t > VLAN, TX rate limit, and trusted VF: > + > +.. code-block:: console > + > + ip link set pf_id vf vf_id mac (mac_address) vlan (vlan_id) txrate > (rate_value) trust (enable|disable) > + (ex) ip link set 0 vf 0 mac 00:11:22:33:44:55 vlan 0x100 txrate 100 > trust disable > + > +Running on VF > +~~~~~~~~~~~~~ > + > +Flow Bifurcation > +^^^^^^^^^^^^^^^^ > + > +The Flow Bifurcation splits the incoming data traffic to user space > applications (such as DPDK applications) and/or kernel space programs (su= ch > as the Linux kernel stack). > +It can direct some traffic, for example data plane traffic, to DPDK. > +Rest of the traffic, for example control plane traffic, would be > redirected to to the traditional Linux networking stack. > + > +Refer to https://doc.dpdk.org/guides/howto/flow_bifurcation.html > + > +Benefits of the flow bifurcation include: > + > +* Better performance with less CPU overhead, as user application can > directly access the NIC for data path > +* NIC is still being controlled by the kernel, as control traffic is > forwarded only to the kernel driver > +* Control commands, e.g. ethtool, will work as usual > + > +Running on a VF, the BXNT PMD supports the flow bifurcation with a > combination of SR-IOV and packet classification and/or forwarding > capability. > +In the simplest case of flow bifurcation, a PF driver configures a NIC t= o > forward all user traffic directly to VFs with matching destination MAC > address, while the rest of the traffic is forwarded to a PF. > +Note that the broadcast packets will be forwarded to both PF and VF. > + > +.. code-block:: console > + > + (ex) ethtool --config-ntuple ens2f0 flow-type ether dst > 00:01:02:03:00:01 vlan 10 vlan-mask 0xf000 action 0x100000000 > + > +Trusted VF > +^^^^^^^^^^ > + > +By default, VFs are *not* allowed to perform privileged operations, such > as modifying the VF=E2=80=99s MAC address in the guest. These security me= asures are > designed to prevent possible attacks. > +However, when a DPDK application can be trusted (e.g., OVS-DPDK, here), > these operations performed by a VF would be legitimate and can be allowed= . > + > +To enable VF to request "trusted mode," a new trusted VF concept was > introduced in Linux kernel 4.4 and allowed VFs to become =E2=80=9Ctrusted= =E2=80=9D and > perform some privileged operations. > + > +The BNXT PMD supports the trusted VF mode of operation. Only a PF can > enable trusted attribute on the VF. It is preferable to enable the Truste= d > setting on a VF before starting applications. > +However, the BNXT PMD handles dynamic changes in trusted settings as wel= l. > + > +Note that control commands, e.g., ethtool, will work via the kernel PF > driver, *not* via the trusted VF driver. > + > +Operations supported by trusted VF: > + > +* MAC address configuration > +* Flow rule creation > + > +Operations *not* supported by trusted VF: > + > +* Firmware upgrade > +* Promiscuous mode setting > + > +Running on PF > +~~~~~~~~~~~~~ > + > +Unlike the VF when BNXT PMD runs on a PF there are no restrictions place= d > on the features which the PF can enable or request. > +In a multiport NIC, each port will have a corresponding PF. Also > depending on the configuration of the NIC there can be more than one PF > associated per port. > +A sysadmin can load the kernel driver on one PF, and run BNXT PMD on the > other PF or run the PMD on both the PFs. In such cases, the firmware pick= s > one of the PFs as a master PF. > + > +Much like in the trusted VF, the DPDK application must be *trusted* and > expected to be *well-behaved*. > + > +Features > +-------- > + > +The BNXT PMD supports the following features: > + > +* Port Control > + * Port MTU > + * LED > + * Flow Control and Autoneg > +* Packet Filtering > + * Unicast MAC Filter > + * Multicast MAC Filter > + * VLAN Filtering > + * Allmulticast Mode > + * Promiscuous Mode > +* Stateless Offloads > + * CRC Offload > + * Checksum Offload (IPv4, TCP, and UDP) > + * Multi-Queue (TSS and RSS) > + * Segmentation and Reassembly (TSO and LRO) > +* VLAN insert strip > +* Stats Collection > +* Generic Flow Offload > + > +Port Control > +~~~~~~~~~~~~ > + > +**Port MTU**: BNXT PMD supports the MTU (Maximum Transmission Unit) up t= o > 9,574 bytes: > + > +.. code-block:: console > + > + testpmd> port config mtu (port_id) mtu_value > + testpmd> show port info (port_id) > + > +**LED**: Application tunes on (or off) a port LED, typically for a port > identification: > + > +.. code-block:: console > + > + int rte_eth_led_on (uint16_t port_id) > + int rte_eth_led_off (uint16_t port_id) > + > +**Flow Control and Autoneg**: Application tunes on (or off) flow control > and/or auto-negotiation on a port: > + > +.. code-block:: console > + > + testpmd> set flow_ctrl rx (on|off) (port_id) > + testpmd> set flow_ctrl tx (on|off) (port_id) > + testpmd> set flow_ctrl autoneg (on|off) (port_id) > + > +Note that the BNXT PMD does *not* support some options and ignores them > when requested: > + > +* high_water > +* low_water > +* pause_time > +* mac_ctrl_frame_fwd > +* send_xon > + > +Packet Filtering > +~~~~~~~~~~~~~~~~ > + > +Applications control the packet-forwarding behaviors with packet filters= . > + > +The BNXT PMD supports hardware-based packet filtering: > + > +* UC (Unicast) MAC Filters > + * No unicast packets are forwarded to an application except the one > with DMAC address added to the port > + * At initialization, the station MAC address is added to the port > +* MC (Multicast) MAC Filters > + * No multicast packets are forwarded to an application except the on= e > with MC address added to the port > + * When the application listens to a multicast group, it adds the MC > address to the port > +* VLAN Filtering Mode > + * When enabled, no packets are forwarded to an application except th= e > ones with the VLAN tag assigned to the port > +* Allmulticast Mode > + * When enabled, every multicast packet received on the port is > forwarded to the application > + * Typical usage is routing applications > +* Promiscuous Mode > + * When enabled, every packet received on the port is forwarded to th= e > application > + > +Unicast MAC Filter > +^^^^^^^^^^^^^^^^^^ > + > +The application adds (or removes) MAC addresses to enable (or disable) > whitelist filtering to accept packets. > + > +.. code-block:: console > + > + testpmd> show port (port_id) macs > + testpmd> mac_addr (add|remove) (port_id) (XX:XX:XX:XX:XX:XX) > + > +Multicast MAC Filter > +^^^^^^^^^^^^^^^^^^^^ > + > +Application adds (or removes) Multicast addresses to enable (or disable) > whitelist filtering to accept packets. > + > +.. code-block:: console > + > + testpmd> show port (port_id) mcast_macs > + testpmd> mcast_addr (add|remove) (port_id) (XX:XX:XX:XX:XX:XX) > + > +Application adds (or removes) Multicast addresses to enable (or disable) > whitelist filtering to accept packets. > + > +Note that the BNXT PMD supports up to 16 MC MAC filters. if the user add= s > more than 16 MC MACs, the BNXT PMD puts the port into the Allmulticast mo= de. > + > +VLAN Filtering > +^^^^^^^^^^^^^^ > + > +The application enables (or disables) VLAN filtering mode. When the mode > is enabled, no packets are forwarded to an application except ones with > VLAN tag assigned for the application. > + > +.. code-block:: console > + > + testpmd> vlan set filter (on|off) (port_id) > + testpmd> rx_vlan (add|rm) (vlan_id) (port_id) > + > +Allmulticast Mode > +^^^^^^^^^^^^^^^^^ > + > +The application enables (or disables) the allmulticast mode. When the > mode is enabled, every multicast packet received is forwarded to the > application. > + > +.. code-block:: console > + > + testpmd> show port info (port_id) > + testpmd> set allmulti (port_id) (on|off) > + > +Promiscuous Mode > +^^^^^^^^^^^^^^^^ > + > +The application enables (or disables) the promiscuous mode. When the mod= e > is enabled on a port, every packet received on the port is forwarded to t= he > application. > + > +.. code-block:: console > + > + testpmd> show port info (port_id) > + testpmd> set promisc port_id (on|off) > + > +Stateless Offloads > +~~~~~~~~~~~~~~~~~~ > + > +Like Linux, DPDK provides enabling hardware offload of some stateless > processing (such as checksum calculation) of the stack, alleviating the C= PU > from having to burn cycles on every packet. > + > +Listed below are the stateless offloads supported by the BNXT PMD: > + > +* CRC offload (for both TX and RX packets) > +* Checksum Offload (for both TX and RX packets) > + * IPv4 Checksum Offload > + * TCP Checksum Offload > + * UDP Checksum Offload > +* Segmentation/Reassembly Offloads > + * TCP Segmentation Offload (TSO) > + * Large Receive Offload (LRO) > +* Multi-Queue > + * Transmit Side Scaling (TSS) > + * Receive Side Scaling (RSS) > + > +Also, the BNXT PMD supports stateless offloads on inner frames for > tunneled packets. Listed below are the tunneling protocols supported by t= he > BNXT PMD: > + > +* VXLAN > +* GRE > +* NVGRE > + > +Note that enabling (or disabling) stateless offloads requires > applications to stop DPDK before changing configuration. > + > +CRC Offload > +^^^^^^^^^^^ > + > +The FCS (Frame Check Sequence) in the Ethernet frame is a four-octet CRC > (Cyclic Redundancy Check) that allows detection of corrupted data within > the entire frame as received on the receiver side. > + > +The BNXT PMD supports hardware-based CRC offload: > + > +* TX: calculate and insert CRC > +* RX: check and remove CRC, notify the application on CRC error > + > +Note that the CRC offload is always turned on. > + > +Checksum Offload > +^^^^^^^^^^^^^^^^ > + > +The application enables hardware checksum calculation for IPv4, TCP, and > UDP. > + > +.. code-block:: console > + > + testpmd> port stop (port_id) > + testpmd> csum set (ip|tcp|udp|outer-ip|outer-udp) (sw|hw) (port_id) > + testpmd> set fwd csum > + > +Multi-Queue > +^^^^^^^^^^^ > + > +Multi-Queue, also known as TSS (Transmit Side Scaling) or RSS (Receive > Side Scaling), is a common networking technique that allows for more > efficient load balancing across multiple CPU cores. > + > +The application enables multiple TX and RX queues when starts. > + > +.. code-block:: console > + > + testpmd -l 1,3,5 --master-lcore 1 --txq=3D2 =E2=80=93rxq=3D2 --nb-co= res=3D2 > + > +**TSS** > + > +TSS distributes network transmit processing across several hardware-base= d > transmit queues, allowing outbound network traffic to be processed by > multiple CPU cores. > + > +**RSS** > + > +RSS distributes network receive processing across several hardware-based > receive queues, allowing inbound network traffic to be processed by > multiple CPU cores. > + > +The application can select the RSS mode, i.e. select the header fields > that are included for hash calculation. The BNXT PMD supports the RSS mod= e > of ``default|ip|tcp|udp|none``, where default mode is L3 and L4. > + > +For tunneled packets, RSS hash is calculated over inner frame header > fields. Applications may want to select the tunnel header fields for hash > calculation, and it will be supported in 20.08 using RSS level. > + > +.. code-block:: console > + > + testpmd> port config (port_id) rss (all|default|ip|tcp|udp|none) > + > + // note that the testpmd defaults the RSS mode to ip > + // ensure to issue the command below to enable L4 header (TCP or UDP= ) > along with IPv4 header > + testpmd> port config (port_id) rss default > + > + // to check the current RSS configuration, such as RSS function and > RSS key > + testpmd> show port (port_id) rss-hash key > + > + // RSS is enabled by default. However, application can disable RSS a= s > follows > + testpmd> port config (port_id) rss none > + > +Application can change the flow distribution, i.e. remap the received > traffic to CPU cores, using RSS RETA (Redirection Table). > + > +.. code-block:: console > + > + // application queries the current RSS RETA configuration > + testpmd> show port (port_id) rss reta size (mask0, mask1) > + > + // application changes the RSS RETA configuration > + testpmd> port config (port_id) rss reta (hash, queue) [, (hash, > queue)] > + > +TSO > +^^^ > + > +TSO (TCP Segmentation Offload), also known as LSO (Large Send Offload), > enables the TCP/IP stack to pass to the NIC a larger datagram than the MT= U > (Maximum Transmit Unit). NIC breaks it into multiple segments before > sending it to the network. > + > +The BNXT PMD supports hardware-based TSO. > + > +.. code-block:: console > + > + // display the status of TSO > + testpmd> tso show (port_id) > + > + // enable/disable TSO > + testpmd> port config (port_id) tx_offload tcp_tso (on|off) > + > + // set TSO segment size > + testpmd> tso set segment_size (port_id) > + > +The BNXT PMD also supports hardware-based tunneled TSO. > + > +.. code-block:: console > + > + // display the status of tunneled TSO > + testpmd> tunnel_tso show (port_id) > + > + // enable/disable tunneled TSO > + testpmd> port config (port_id) tx_offload vxlan_tnl_tso|gre_tnl_tso > (on|off) > + > + // set tunneled TSO segment size > + testpmd> tunnel_tso set segment_size (port_id) > + > +Note that the checksum offload is always assumed to be enabled for TSO. > + > +LRO > +^^^ > + > +LRO (Large Receive Offload) enables NIC to aggregate multiple incoming > TCP/IP packets from a single stream into a larger buffer, before passing = to > the networking stack. > + > +The BNXT PMD supports hardware-based LRO. > + > +.. code-block:: console > + > + // display the status of LRO > + testpmd> show port (port_id) rx_offload capabilities > + testpmd> show port (port_id) rx_offload configuration > + > + // enable/disable LRO > + testpmd> port config (port_id) rx_offload tcp_lro (on|off) > + > + // set max LRO packet (datagram) size > + testpmd> port config (port_id) max-lro-pkt-size (max_size) > + > +The BNXT PMD also supports tunneled LRO. > + > +Some applications, such as routing, should *not* change the packet > headers as they pass through (i.e. received from and sent back to the > network). In such a case, GRO (Generic Receive Offload) should be used > instead of LRO. > + > +VLAN Insert/Strip > +~~~~~~~~~~~~~~~~~ > + > +DPDK application offloads VLAN insert/strip to improve performance. The > BNXT PMD supports hardware-based VLAN insert/strip offload for both singl= e > and double VLAN packets. > + > + > +VLAN Insert > +^^^^^^^^^^^ > + > +Application configures the VLAN TPID (Tag Protocol ID). By default, the > TPID is 0x8100. > + > +.. code-block:: console > + > + // configure outer TPID value for a port > + testpmd> vlan set outer tpid (tpid_value) (port_id) > + > +The inner TPID set will be rejected as the BNXT PMD supports inserting > only an outer VLAN. Note that when a packet has a single VLAN, the tag is > considered as outer, i.e. the inner VLAN is relevant only when a packet i= s > double-tagged. > + > +The BNXT PMD supports various TPID values shown below. Any other values > will be rejected. > + > +* ``0x8100`` > +* ``0x88a8`` > +* ``0x9100`` > +* ``0x9200`` > +* ``0x9300`` > + > +The BNXT PMD supports the VLAN insert offload per-packet basis. The > application provides the TCI (Tag Control Info) for a packet via mbuf. In > turn, the BNXT PMD inserts the VLAN tag (via hardware) using the provided > TCI along with the configured TPID. > + > +.. code-block:: console > + > + // enable VLAN insert offload > + testpmd> port config (port_id) rx_offload vlan_insert|qinq_insert > (on|off) > + > + if (mbuf->ol_flags && PKT_TX_QINQ) // case-1: insert VLAN to > single-tagged packet > + tci_value =3D mbuf->vlan_tci_outer > + else if (mbuf->ol_flags && PKT_TX_VLAN) // case-2: insert VLAN to > untagged packet > + tci_value =3D mbuf->vlan_tci > + > +VLAN Strip > +^^^^^^^^^^ > + > +The application configures the per-port VLAN strip offload. > + > +.. code-block:: console > + > + // enable VLAN strip on a port > + testpmd> port config (port_id) tx_offload vlan_strip (on|off) > + > + // notify application VLAN strip via mbuf > + mbuf->ol_flags |=3D PKT_RX_VLAN | PKT_RX_STRIPPED // outer VLAN is > found and stripped > + mbuf->vlan_tci =3D tci_value // TCI of the > stripped VLAN > + > +Time Synchronization > +~~~~~~~~~~~~~~~~~~~~ > + > +System operators may run a PTP (Precision Time Protocol) client > application to synchronize the time on the NIC (and optionally, on the > system) to a PTP master. > + > +The BNXT PMD supports a PTP client application to communicate with a PTP > master clock using DPDK IEEE1588 APIs. Note that the PTP client applicati= on > need to run on PF and vector mode needs to be disabled. > + > +For the PTP time synchronization support, the BNXT PMD must be compiled > with ``CONFIG_RTE_LIBRTE_IEEE1588=3Dy`` (this compilation flag is current= ly > pending). > + > +.. code-block:: console > + > + testpmd> set fwd ieee1588 // enable IEEE 1588 mode > + > +When enabled, the BNXT PMD configures hardware to insert IEEE 1588 > timestamps to the outgoing PTP packets and reports IEEE 1588 timestamps > from the incoming PTP packets to application via mbuf. > + > +.. code-block:: console > + > + // RX packet completion will indicate whether the packet is PTP > + mbuf->ol_flags |=3D PKT_RX_IEEE1588_PTP > + > +Statistics Collection > +~~~~~~~~~~~~~~~~~~~~~ > + > +In Linux, the *ethtool -S* enables us to query the NIC stats. DPDK > provides the similar functionalities via rte_eth_stats and rte_eth_xstats= . > + > +The BNXT PMD supports both basic and extended stats collection: > + > +* Basic stats > +* Extended stats > + > +Basic Stats > +^^^^^^^^^^^ > + > +The application collects per-port and per-queue stats using rte_eth_stat= s > APIs. > + > +.. code-block:: console > + > + testpmd> show port stats (port_id) > + > +Basic stats include: > + > +* ipackets > +* ibytes > +* opackets > +* obytes > +* imissed > +* ierrors > +* oerrors > + > +By default, the BNXT PMD supports per-queue stats for 16 queues. For mor= e > than 16 queues, BNXT PMD should be compiled with > ``CONFIG_RTE_ETHDEV_QUEUE_STAT_CNTRS`` set to the desired number of queue= s. > + > +Extended Stats > +^^^^^^^^^^^^^^ > + > +Unlike basic stats, the extended stats are vendor-specific, i.e. each > vendor provides its own set of counters. > + > +The BNXT PMD provides a rich set of counters, including per-flow > counters, per-cos counters, per-priority counters, etc. > + > +.. code-block:: console > + > + testpmd> show port xstats (port_id) > + > +Shown below is the elaborated sequence to retrieve extended stats: > + > +.. code-block:: console > + > + // application queries the number of xstats > + len =3D rte_eth_xstats_get(port_id, NULL, 0); > + // BNXT PMD returns the size of xstats array (i.e. the number of > entries) > + // BNXT PMD returns 0, if the feature is compiled out or disabled > + > + // application allocates memory for xstats > + struct rte_eth_xstats_name *names; // name is 64 character or less > + struct rte_eth_xstats *xstats; > + names =3D calloc(len, sizeof(*names)); > + xstats =3D calloc(len, sizeof(*xstats)); > + > + // application retrieves xstats // names and values > + ret =3D rte_eth_xstats_get_names(port_id, *names, len); > + ret =3D rte_eth_xstats_get(port_id, *xstats, len); > + > + // application checks the xstats > + // application may repeat the below: > + len =3D rte_eth_xstats_reset(port_id); // reset the xstats > + > + // reset can be skipped, if application wants to see accumulated sta= ts > + // run traffic > + // probably stop the traffic > + // retrieve xstats // no need to retrieve xstats names again > + // check xstats > + > +Generic Flow Offload > +~~~~~~~~~~~~~~~~~~~~ > + > +Applications can get benefit by offloading all or part of flow processin= g > to hardware. For example, applications can offload packet classification > only (partial offload) or whole match-action (full offload). > + > +DPDK offers the Generic Flow API (rte_flow API) to configure hardware to > perform flow processing. > + > +Listed below are the rte_flow APIs BNXT PMD supports: > + > +* rte_flow_validate > +* rte_flow_create > +* rte_flow_destroy > +* rte_flow_flush > + > +Host Based Flow Table Management > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +Starting with 20.05 BNXT PMD supports host based flow table management. > This is a new mechanism that should allow higher flow scalability than wh= at > is currently supported. This new approach also defines a new rte_flow > parser, and mapper which currently supports basic packet classification i= n > the receive path. > + > +The feature uses a newly implemented control-plane firmware interface > which optimizes flow insertions and deletions. > + > +This is a tech preview feature, and is disabled by default. It can be > enabled using bnxt devargs. For ex: "-w 0000:0d:00.0,host-based-truflow= =3D1=E2=80=9D. > + > +Application Support > +------------------- > + > +Firmware > +~~~~~~~~ > + > +The BNXT PMD supports the application to retrieve the firmware version. > + > +.. code-block:: console > + > + testpmd> show port info (port_id) > + > +Note that the applications cannot update the firmware using BNXT PMD. > + > +Multiple Processes > +~~~~~~~~~~~~~~~~~~ > + > +When two or more DPDK applications (e.g., testpmd and dpdk-pdump) share = a > single instance of DPDK, the BNXT PMD supports a single primary applicati= on > and one or more secondary applications. Note that the DPDK-layer (*not* t= he > PMD-layer) ensures there is only one primary application. > + > +There are two modes: > + > +Manual mode > + > +* Application notifies whether it is primary or secondary using > *proc-type* flag > +* 1st process should be spawned with ``--proc-type=3Dprimary`` > +* All subsequent processes should be spawned with > ``--proc-type=3Dsecondary`` > + > +Auto detection mode > + > +* Application is using ``proc-type=3Dauto`` flag > +* A process is spawned as a secondary if a primary is already running > + > +The BNXT PMD uses the info to skip a device initialization, i.e. perform= s > a device initialization only when being brought up by a primary applicati= on. > + > +Runtime Queue Setup > +~~~~~~~~~~~~~~~~~~~ > + > +Typically, a DPDK application allocates TX and RX queues statically: i.e= . > queues are allocated at start. However, an application may want to increa= se > (or decrease) the number of queues dynamically for various reasons, e.g. > power savings. > + > +The BNXT PMD supports applications to increase or decrease queues at > runtime. > + > +.. code-block:: console > + > + testpmd> port config all (rxq|txq) (num_queues) > + > +Note that a DPDK application must allocate default queues (one for TX an= d > one for RX at minimum) at initialization. > + > +Descriptor Status > +~~~~~~~~~~~~~~~~~ > + > +Applications may use the descriptor status for various reasons, e.g. for > power savings. For example, an application may stop polling and change to > interrupt mode when the descriptor status shows no packets to service for= a > while. > + > +The BNXT PMD supports the application to retrieve both TX and RX > descriptor status. > + > +.. code-block:: console > + > + testpmd> show port (port_id) (rxq|txq) (queue_id) desc (desc_id) > status > + > +Bonding > +~~~~~~~ > + > +DPDK implements a light-weight library to allow PMDs to be bonded > together and provide a single logical PMD to the application. > + > +.. code-block:: console > + > + testpmd -l 0-3 -n4 --vdev 'net_bonding0,mode=3D0,slave=3D device 1>,slave=3D,mac=3DXX:XX:XX:XX:XX:XX=E2=80=99 = =E2=80=93 > --socket_num=3D1 =E2=80=93 -i --port-topology=3Dchained > + (ex) testpmd -l 1,3,5,7,9 -n4 --vdev > 'net_bonding0,mode=3D0,slave=3D0000:82:00.0,slave=3D0000:82:00.1,mac=3D00= :1e:67:1d:fd:1d' > =E2=80=93 --socket-num=3D1 =E2=80=93 -i --port-topology=3Dchained > + > +Vector Processing > ----------------- > > -The BNXT PMD includes support for the following features: > - > - * Multiple transmit and receive queues > - * Queue start/stop > - * RSS hash > - * RSS key configuration > - * RSS reta configuration > - * VMDq > - * Packet type parsing > - * Configurable RX CRC stripping > - * L3/L4 checksum offload > - * LRO offload > - * TSO offload > - * VLAN offload > - * SR-IOV VF > - * Basic and extended port statistics > - * Link state reporting > - * Flow control > - * Ethertype filtering > - * N-tuple filtering > - * Promiscuous mode > - * Unicast and multicast MAC filtering > - * Scatter/gather transmit and receive > - * Jumbo frames > - * Vector PMD > - > -BNXT Vector PMD > ---------------- > - > -The BNXT PMD includes support for SSE vector mode on x86 platforms. Vect= or > -provides significantly improved performance over the base implementation= , > -however it does not support all of the features that are supported by th= e > -base (non-vector) implementation. Vector mode will be selected and enabl= ed > -automatically when the port is started if allowed by the current > configuration. > - > -RX Requirements for Vector Mode > -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > - > -Vector mode receive will be enabled if the following constrainsts are me= t: > - * Packets must fit within a single mbuf (no scatter RX). > - * LRO offload must be disabled. > - > -TX Requirements for Vector Mode > -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > - > -Vector mode transmit will be enabled if the following constrainsts are > met: > - * Packets must be contained within a single mbuf (no gather TX). > - * All transmit offloads other than VLAN insertion must be disabled. > - > -BNXT PMD Supported Chipsets and Adapters > ----------------------------------------- > - > -Chipsets and adapters supported by the bnxt PMD include: > - > - * **Broadcom BCM5730X NetXtreme-C=C2=AE Family of Ethernet Network > Controllers** > - > - * M150c - Single-port 40/50 Gigabit Ethernet Adapter > - * P150c - Single-port 40/50 Gigabit Ethernet Adapter > - * P225c - Dual-port 10/25 Gigabit Ethernet Adapter > - > - * **Broadcom BCM574XX/BCM575XX NetXtreme-E=C2=AE Family of Ethernet Ne= twork > Controllers** > - > - * M125P - Single-port OCP 2.0 10/25 Gigabit Ethernet Adapter > - * M150P - Single-port OCP 2.0 50 Gigabit Ethernet Adapter > - * M150PM - Single-port OCP 2.0 Multi-Host 50 Gigabit Ethernet > Adapter > - * M210P - Dual-port OCP 2.0 10 Gigabit Ethernet Adapter > - * M210TP - Dual-port OCP 2.0 10 Gigabit Ethernet Adapter > - * M11000G - Single-port OCP 2.0 10/25/50/100 Gigabit Ethernet > Adapter > - * N150G - Single-port OCP 3.0 50 Gigabit Ethernet Adapter > - * M225P - Dual-port OCP 2.0 10/25 Gigabit Ethernet Adapter > - * N210P - Dual-port OCP 3.0 10 Gigabit Ethernet Adapter > - * N210TP - Dual-port OCP 3.0 10 Gigabit Ethernet Adapter > - * N225P - Dual-port OCP 3.0 10/25 Gigabit Ethernet Adapter > - * N250G - Dual-port OCP 3.0 50 Gigabit Ethernet Adapter > - * N410SG - Quad-port OCP 3.0 10 Gigabit Ethernet Adapter > - * N410SGBT - Quad-port OCP 3.0 10 Gigabit Ethernet Adapter > - * N425G - Quad-port OCP 3.0 10/25 Gigabit Ethernet Adapter > - * N1100G - Single-port OCP 3.0 10/25/50/100 Gigabit Ethernet > Adapter > - * N2100G - Dual-port OCP 3.0 10/25/50/100 Gigabit Ethernet Adapte= r > - * N2200G - Dual-port OCP 3.0 10/25/50/100/200 Gigabit Ethernet > Adapter > - * P150P - Single-port 50 Gigabit Ethernet Adapter > - * P210P - Dual-port 10 Gigabit Ethernet Adapter > - * P210TP - Dual-port 10 Gigabit Ethernet Adapter > - * P225P - Dual-port 10/25 Gigabit Ethernet Adapter > - * P410SG - Quad-port 10 Gigabit Ethernet Adapter > - * P410SGBT - Quad-port 10 Gigabit Ethernet Adapter > - * P425G - Quad-port 10/25 Gigabit Ethernet Adapter > - * P1100G - Single-port 10/25/50/100 Gigabit Ethernet Adapter > - * P2100G - Dual-port 10/25/50/100 Gigabit Ethernet Adapter > - * P2200G - Dual-port 10/25/50/100/200 Gigabit Ethernet Adapter > - > - Information about Ethernet adapters in the NetXtreme family of > - adapters can be found in the `NetXtreme=C2=AE Brand section > - < > https://www.broadcom.com/products/ethernet-connectivity/network-adapters/ > >`_ > - of the `Broadcom website `_. > - > - * **Broadcom BCM588XX Stingray Family of SmartNIC Adapters** > - > - * PS410T - Quad-port 10 Gigabit Ethernet SmartNIC > - * PS225 - Dual-port 25 Gigabit Ethernet SmartNIC > - * PS250 - Dual-Port 50 Gigabit Ethernet SmartNIC > - > - Information about the Stingray family of SmartNIC adapters can be > found in the > - `Stingray=C2=AE Brand section > - `= _ > - of the `Broadcom website `_. > - > - * **Broadcom StrataGX=C2=AE BCM5871X Series of Communucations Processo= rs** > - > - These ARM based processors target a broad range of networking > applications > - including virtual CPE (vCPE) and NFV appliances, 10G service routers > and > - gateways, control plane processing for Ethernet switches and network > - attached storage (NAS). > - > - Information about the StrataGX family of adapters can be found in th= e > - `StrataGX=C2=AE BCM58712 > - < > http://www.broadcom.com/products/embedded-and-networking-processors/commu= nications/bcm58712 > >`_ > - and `StrataGX=C2=AE BCM58713 > - < > http://www.broadcom.com/products/embedded-and-networking-processors/commu= nications/bcm58713 > >`_ > - sections of the `Broadcom website `_. > +Vector processing provides significantly improved performance over scala= r > processing (see Vector Processor, here). > + > +The BNXT PMD supports the vector processing using SSE (Streaming SIMD > Extensions) instructions on x86 platforms. The BNXT vPMD (vector mode PMD= ) > is currently limited to Intel/AMD CPU architecture. Support for ARM is > *not* currently implemented. > + > +This improved performance comes from several optimizations: > + > +* Batching > + * TX: processing completions in bulk > + * RX: allocating mbufs in bulk > +* Chained mbufs are *not* supported, i.e. a packet should fit a single > mbuf > +* Some stateless offloads are *not* supported with vector processing > + * TX: no offloads will be supported > + * RX: reduced RX offloads (listed below) will be supported:: > + > + DEV_RX_OFFLOAD_VLAN_STRIP > + DEV_RX_OFFLOAD_KEEP_CRC > + DEV_RX_OFFLOAD_JUMBO_FRAME > + DEV_RX_OFFLOAD_IPV4_CKSUM > + DEV_RX_OFFLOAD_UDP_CKSUM > + DEV_RX_OFFLOAD_TCP_CKSUM > + DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM > + DEV_RX_OFFLOAD_RSS_HASH > + DEV_RX_OFFLOAD_VLAN_FILTER > + > +The BNXT Vector PMD is enabled in DPDK builds by default. When required, > it has to be disabled in the DPDK build configuration by setting > ``CONFIG_RTE_LIBRTE_BNXT_INC_VECTOR=3Dn``. > + > +However, a decision to enable vector mode will be made when the port > transitions from stopped to started. Any TX offloads or some RX offloads > (other than listed above) will disable the vector mode. > +Offload configuration changes that impact vector mode must be made when > the port is stopped. > + > +Note that TX (or RX) vector mode can be enabled independently from RX (o= r > TX) vector mode. > + > +Appendix > +-------- > + > +Supported Chipsets and Adapters > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +BCM5730x NetXtreme-C=C2=AE Family of Ethernet Network Controllers > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +Information about Ethernet adapters in the NetXtreme family of adapters > can be found in the `NetXtreme=C2=AE Brand section < > https://www.broadcom.com/products/ethernet-connectivity/network-adapters/= >`_ > of the `Broadcom website `_. > + > +* ``M150c ... Single-port 40/50 Gigabit Ethernet Adapter`` > +* ``P150c ... Single-port 40/50 Gigabit Ethernet Adapter`` > +* ``P225c ... Dual-port 10/25 Gigabit Ethernet Adapter`` > + > +BCM574xx/575xx NetXtreme-E=C2=AE Family of Ethernet Network Controllers > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +Information about Ethernet adapters in the NetXtreme family of adapters > can be found in the `NetXtreme=C2=AE Brand section < > https://www.broadcom.com/products/ethernet-connectivity/network-adapters/= >`_ > of the `Broadcom website `_. > + > +* ``M125P .... Single-port OCP 2.0 10/25 Gigabit Ethernet Adapter`` > +* ``M150P .... Single-port OCP 2.0 50 Gigabit Ethernet Adapter`` > +* ``M150PM ... Single-port OCP 2.0 Multi-Host 50 Gigabit Ethernet > Adapter`` > +* ``M210P .... Dual-port OCP 2.0 10 Gigabit Ethernet Adapter`` > +* ``M210TP ... Dual-port OCP 2.0 10 Gigabit Ethernet Adapter`` > +* ``M1100G ... Single-port OCP 2.0 10/25/50/100 Gigabit Ethernet Adapter= `` > +* ``N150G .... Single-port OCP 3.0 50 Gigabit Ethernet Adapter`` > +* ``M225P .... Dual-port OCP 2.0 10/25 Gigabit Ethernet Adapter`` > +* ``N210P .... Dual-port OCP 3.0 10 Gigabit Ethernet Adapter`` > +* ``N210TP ... Dual-port OCP 3.0 10 Gigabit Ethernet Adapter`` > +* ``N225P .... Dual-port OCP 3.0 10/25 Gigabit Ethernet Adapter`` > +* ``N250G .... Dual-port OCP 3.0 50 Gigabit Ethernet Adapter`` > +* ``N410SG ... Quad-port OCP 3.0 10 Gigabit Ethernet Adapter`` > +* ``N410SGBT . Quad-port OCP 3.0 10 Gigabit Ethernet Adapter`` > +* ``N425G .... Quad-port OCP 3.0 10/25 Gigabit Ethernet Adapter`` > +* ``N1100G ... Single-port OCP 3.0 10/25/50/100 Gigabit Ethernet Adapter= `` > +* ``N2100G ... Dual-port OCP 3.0 10/25/50/100 Gigabit Ethernet Adapter`` > +* ``N2200G ... Dual-port OCP 3.0 10/25/50/100/200 Gigabit Ethernet > Adapter`` > +* ``P150P .... Single-port 50 Gigabit Ethernet Adapter`` > +* ``P210P .... Dual-port 10 Gigabit Ethernet Adapter`` > +* ``P210TP ... Dual-port 10 Gigabit Ethernet Adapter`` > +* ``P225P .... Dual-port 10/25 Gigabit Ethernet Adapter`` > +* ``P410SG ... Quad-port 10 Gigabit Ethernet Adapter`` > +* ``P410SGBT . Quad-port 10 Gigabit Ethernet Adapter`` > +* ``P425G .... Quad-port 10/25 Gigabit Ethernet Adapter`` > +* ``P1100G ... Single-port 10/25/50/100 Gigabit Ethernet Adapter`` > +* ``P2100G ... Dual-port 10/25/50/100 Gigabit Ethernet Adapter`` > +* ``P2200G ... Dual-port 10/25/50/100/200 Gigabit Ethernet Adapter`` > + > +BCM588xx NetXtreme-S=C2=AE Family of SmartNIC Network Controllers > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +Information about the Stingray family of SmartNIC adapters can be found > in the `Stingray=C2=AE Brand section < > https://www.broadcom.com/products/ethernet-connectivity/smartnic/>`_ of > the `Broadcom website `_. > + > +* ``PS225 ... Dual-port 25 Gigabit Ethernet SmartNIC`` > + > +BCM5873x StrataGX=C2=AE Family of Communications Processors > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +These ARM-based processors target a broad range of networking > applications, including virtual CPE (vCPE) and NFV appliances, 10G servic= e > routers and gateways, control plane processing for Ethernet switches, and > network-attached storage (NAS). > + > +* ``StrataGX BCM58732 ... Octal-Core 3.0GHz 64-bit ARM=C2=AEv8 Cortex=C2= =AE-A72 > based SoC`` > -- > 2.21.1 (Apple Git-122.3) > >