From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ubuntu (host217-39-174-19.in-addr.btopenworld.com [217.39.174.19]) by dpdk.org (Postfix) with SMTP id E17228E8B for ; Mon, 2 Nov 2015 13:25:31 +0100 (CET) Received: by ubuntu (Postfix, from userid 5466) id 440C2EA780; Mon, 2 Nov 2015 12:25:30 +0000 (GMT) From: "Alejandro.Lucero" To: dev@dpdk.org Date: Mon, 2 Nov 2015 12:25:28 +0000 Message-Id: <1446467130-300-9-git-send-email-alejandro.lucero@netronome.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1446467130-300-1-git-send-email-alejandro.lucero@netronome.com> References: <1446467130-300-1-git-send-email-alejandro.lucero@netronome.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: [dpdk-dev] =?utf-8?q?=5BPATCH_v5_8/9=5D_nfp=3A_adding_nic_guide?= X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2015 12:25:32 -0000 From: "Alejandro.Lucero" Signed-off-by: Alejandro.Lucero Signed-off-by: Rolf.Neugebauer --- doc/guides/nics/index.rst | 1 + doc/guides/nics/nfp.rst | 189 +++++++++++++++++++++++++++++++++++++++= ++++++ 2 files changed, 190 insertions(+) create mode 100644 doc/guides/nics/nfp.rst diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 2d4936d..1a7bffe 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -46,6 +46,7 @@ Network Interface Controller Drivers intel_vf mlx4 mlx5 + nfp virtio vmxnet3 pcap_ring diff --git a/doc/guides/nics/nfp.rst b/doc/guides/nics/nfp.rst new file mode 100644 index 0000000..bb2afda --- /dev/null +++ b/doc/guides/nics/nfp.rst @@ -0,0 +1,189 @@ +.. BSD LICENSE + Copyright(c) 2015 Netronome Systems, Inc. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FO= R + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL= , + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE= , + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON AN= Y + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE US= E + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +NFP poll mode driver library +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D + +Netronome's sixth generation of flow processors pack 216 programmable +cores and over 100 hardware accelerators that uniquely combine packet, +flow, security and content processing in a single device that scales +up to 400 Gbps. + +This document explains how to use DPDK with the Netronome Poll Mode +Driver (PMD) supporting Netronome's Network Flow Processor 6xxx +(NFP-6xxx). + +Currently the driver supports virtual functions (VFs) only. + +Dependencies +------------ + +Before using the Netronome's DPDK PMD some NFP-6xxx configuration, +which is not related to DPDK, is required. The system requires +installation of **Netronome's BSP (Board Support Package)** which includ= es +Linux drivers, programs and libraries. + +If you have a NFP-6xxx device you should already have the code and +documentation for doing this configuration. Contact +**support@netronome.com** to obtain the latest available firmware. + +The NFP Linux kernel drivers (including the required PF driver for the +NFP) are available on Github at +**https://github.com/Netronome/nfp-drv-kmods** along with build +instructions. + +Using Netronome=C2=B4s NFP PMD requires to have the Netronome=C2=B4s BSP= module +loaded. + +Building the software +--------------------- + +Netronome's PMD code is provided in the **drivers/net/nfp** directory. +This PMD is included as part of the DPDK **common_linuxapp configuration= ** +file, but it is not enabled by default. If it is enabled without a BSP +installed in the system, the compilation will fail. + +For enabling the PMD, just modifies the **common_linuxapp** file with: + +- **CONFIG_RTE_LIBRTE_NFP_PMD=3Dy** + +Once DPDK is built all the DPDK apps and examples include support for +the NFP PMD. + +System configuration +-------------------- + +Using the NFP PMD is not different to using other PMDs. Usual steps are: + +#. **Configure hugepages:** All major Linux distributions have the hugep= ages + functionality enabled by default. By default this allows the system u= ses for + working with transparent hugepages. But in this case some hugepages n= eed to + be created/reserved for use with the DPDK through the hugetlbfs file = system. + First the virtual file system need to be mounted: + + .. code-block:: console + + mount -t hugetlbfs none /mnt/hugetlbfs + + The command uses the common mount point for this file system and it n= eeds to + be created if necessary. + + Configuring hugepages is performed via sysfs: + + .. code-block:: console + + /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + + This sysfs file is used to specify the number of hugepages to reserve= . + For example: + + .. code-block:: console + + echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + + This will reserve 2GB of memory using 1024 2MB hugepages. The file ma= y be + read to see if the operation was performed correctly: + + .. code-block:: console + + cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + + The number of unused hugepages may also be inspected. + + Before executing the DPDK app it should match the value of nr_hugepag= es. + + .. code-block:: console + + cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages + + The hugepages reservation should be performed at system initialisatio= n and + it is usual to use a kernel parameter for configuration. If the reser= vation + is attempted on a busy system it will likely fail. Reserving memory f= or + hugepages may be done adding the following to the grub kernel command= line: + + .. code-block:: console + + default_hugepagesz=3D1M hugepagesz=3D2M hugepages=3D1024 + + This will reserve 2GBytes of memory using 2Mbytes huge pages. + + Finally, for a NUMA system the allocation needs to be made on the cor= rect + NUMA node. In a DPDK app there is a master core which will (usually) = perform + memory allocation. It is important that some of the hugepages are res= erved + on the NUMA memory node where the network device is attached. This is= because + of a restriction in DPDK by which TX and RX descriptors rings must be= created + on the master code. + + Per-node allocation of hugepages may be inspected and controlled usin= g sysfs. + For example: + + .. code-block:: console + + cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_h= ugepages + + For a NUMA system there will be a specific hugepage directory per nod= e + allowing control of hugepage reservation. A common problem may occur = when + hugepages reservation is performed after the system has been working = for + some time. Configuration using the global sysfs hugepage interface wi= ll + succeed but the per-node allocations may be unsatisfactory. + + The number of hugepages that need to be reserved depends on how the a= pp uses + TX and RX descriptors, and packets mbufs. + +#. **Enable SR-IOV on the NFP-6xxx device:** The current NFP PMD works w= ith + Virtual Functions (VFs) on a NFP device. Make sure that one of the Ph= ysical + Function (PF) drivers from the above Github repository is installed a= nd + loaded. + + Virtual Functions need to be enabled before they can be used with the= PMD. + Before enabling the VFs it is useful to obtain information about the + current NFP PCI device detected by the system: + + .. code-block:: console + + lspci -d19ee: + + Now, for example, configure two virtual functions on a NFP-6xxx devic= e + whose PCI system identity is "0000:03:00.0": + + .. code-block:: console + + echo 2 > /sys/bus/pci/devices/0000:03:00.0/sriov_numvfs + + The result of this command may be shown using lspci again: + + .. code-block:: console + + lspci -d19ee: -k + + Two new PCI devices should appear in the output of the above command.= The + -k option shows the device driver, if any, that devices are bound to. + Depending on the modules loaded at this point the new PCI devices may= be + bound to nfp_netvf driver. --=20 1.7.9.5