From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0b-0016ce01.pphosted.com (mx0b-0016ce01.pphosted.com [67.231.156.153]) by dpdk.org (Postfix) with ESMTP id B2E93C608 for ; Sat, 20 Feb 2016 16:40:55 +0100 (CET) Received: from pps.filterd (m0085408.ppops.net [127.0.0.1]) by mx0b-0016ce01.pphosted.com (8.16.0.11/8.16.0.11) with SMTP id u1KFetSp000759 for ; Sat, 20 Feb 2016 07:40:55 -0800 Received: from avcashub1.qlogic.com ([198.186.0.115]) by mx0b-0016ce01.pphosted.com with ESMTP id 216r0t85fs-1 (version=TLSv1 cipher=AES128-SHA bits=128 verify=NOT) for ; Sat, 20 Feb 2016 07:40:55 -0800 Received: from avluser01.qlc.com (10.1.113.203) by qlc.com (10.1.4.190) with Microsoft SMTP Server id 14.3.235.1; Sat, 20 Feb 2016 07:40:53 -0800 Received: (from hpatil@localhost) by avluser01.qlc.com (8.14.4/8.14.4/Submit) id u1KFerWP027506; Sat, 20 Feb 2016 07:40:53 -0800 From: Harish Patil To: Date: Sat, 20 Feb 2016 07:40:27 -0800 Message-ID: <1455982831-21682-3-git-send-email-harish.patil@qlogic.com> X-Mailer: git-send-email 1.7.10.3 In-Reply-To: <1455982831-21682-1-git-send-email-harish.patil@qlogic.com> References: <1455982831-21682-1-git-send-email-harish.patil@qlogic.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=nai engine=5800 definitions=8080 signatures=670693 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=4 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1601100000 definitions=main-1602200234 Cc: Sony Chacko Subject: [dpdk-dev] [PATCH 2/6] qede: add documentation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2016 15:40:56 -0000 Signed-off-by: Harish Patil Signed-off-by: Rasesh Mody Signed-off-by: Sony Chacko --- doc/guides/nics/index.rst | 1 + doc/guides/nics/qede.rst | 344 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 345 insertions(+) create mode 100644 doc/guides/nics/qede.rst diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 8618114..f092f75 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -50,6 +50,7 @@ Network Interface Controller Drivers virtio vmxnet3 pcap_ring + qede **Figures** diff --git a/doc/guides/nics/qede.rst b/doc/guides/nics/qede.rst new file mode 100644 index 0000000..de4fecd --- /dev/null +++ b/doc/guides/nics/qede.rst @@ -0,0 +1,344 @@ +.. BSD LICENSE + Copyright (c) 2016 QLogic Corporation + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of QLogic Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +QEDE Poll Mode Driver +====================== + +The QEDE poll mode driver library (**librte_pmd_qede**) implements support +for **QLogic FastLinQ QL4xxxx 25G/40G CNA** family of adapters as well +as their virtual functions (VF) in SR-IOV context. It is supported on +several standard Linux distros like RHEL7.x and SLES12.x OS. +It is compile-tested under FreeBSD OS. + +More information can be found at `QLogic Corporation's Official Website +`_. + +Supported Features +------------------ + +- Unicast/multicast/VLAN filtering +- Promiscuous mode +- Allmulti mode +- Port hardware statistics +- Jumbo frames (using single buffer) +- VLAN tag insertion/removal +- Stateless checksum offloads (IPv4/TCP/UDP) +- Multiple Rx/Tx queues (queue-pairs) +- RSS (with default table/key) +- TSS +- Multiple MAC address +- Default pause flow control +- SR-IOV VF + +Non-supported Features +---------------------- + +- Scatter-Gather Rx/Tx frames +- User configurable RETA table/key +- Unequal number of Rx/Tx queues +- MTU change (dynamic) +- SR-IOV PF +- Tunneling offloads + +Supported QLogic NICs +--------------------- + +- QLogic FastLinQ QL4xxxx 25G/40G CNAs + +Prerequisites +------------- + +- Requires storm firmware version **8.7.x.** and management + firmware version **8.7.x and higher**. Storm firmware may be available + under /usr/lib/firmware/qed/ in certain newer Linux + distributions (e.g. qed_init_values_zipped-8.7.8.0.bin). + +- If the required firmware files are not available then visit + `QLogic Driver Download Center ` + +- This driver relies on external zlib library (-lz) for uncompressing + the firmware file. + +Config File Options +~~~~~~~~~~~~~~~~~~~ + +The following options can be modified in the ``.config`` file. Please note that +enabling debugging options may affect system performance. + +- ``CONFIG_RTE_LIBRTE_QEDE_PMD`` (default **y**) + + Toggle compilation of QEDE PMD driver. + +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_INFO`` (default **n**) + + Toggle display of generic debugging messages. + +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_ECORE`` (default **n**) + + Toggle display of ecore related messages. + +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_TX`` (default **n**) + + Toggle display of transmit fast path run-time messages. + +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_RX`` (default **n**) + + Toggle display of receive fast path run-time messages. + +- ``CONFIG_RTE_LIBRTE_QEDE_RX_COAL_US`` (default **24**) + + Change Rx interrupt coalescing timer (in us). + +- ``CONFIG_RTE_LIBRTE_QEDE_TX_COAL_US`` (default **48**) + + Change Tx interrupt coalescing timer (in us). + +- ``CONFIG_RTE_LIBRTE_QEDE_TX_SWITCHING`` (default **y**) + + Toggle Tx switching + +- ``CONFIG_RTE_LIBRTE_QEDE_FW`` (default **n**) + + Path of firmware file (overrides default location) + +Driver Compilation +~~~~~~~~~~~~~~~~~~ + +To compile QEDE PMD for Linux x86_64 gcc target, run the following "make" +command:: + + cd + make config T=x86_64-native-linuxapp-gcc install + +To compile QEDE PMD for Linux x86_64 clang target, run the following "make" +command:: + + cd + make config T=x86_64-native-linuxapp-clang install + +To compile QEDE PMD for FreeBSD x86_64 clang target, run the following "gmake" +command:: + + cd + gmake config T=x86_64-native-bsdapp-clang install + +To compile QEDE PMD for FreeBSD x86_64 gcc target, run the following "gmake" +command:: + + cd + gmake config T=x86_64-native-bsdapp-gcc install -Wl,-rpath=/usr/local/lib/gcc48 CC=gcc48 + + +Sample Application Notes +~~~~~~~~~~~~~~~~~~~~~~~~ + +This section demonstrates how to launch ``testpmd`` with QLogic 579xx +devices managed by ``librte_pmd_qede`` in Linux operating system. + +#. Request huge pages: + + .. code-block:: console + + echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages + +#. Load ``igb_uio`` or ``vfio-pci`` driver: + + .. code-block:: console + + insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko + + or + + .. code-block:: console + + modprobe vfio-pci + +#. Bind the QLogic 579xx adapters to ``igb_uio`` or ``vfio-pci`` loaded in the + previous step:: + + ./tools/dpdk_nic_bind.py --bind igb_uio 0000:84:00.0 0000:84:00.1 0000:84:00.2 0000:84:00.3 + + or + + Setup VFIO permissions for regular users and then bind to ``vfio-pci``: + + .. code-block:: console + + sudo chmod a+x /dev/vfio + + sudo chmod 0666 /dev/vfio/* + + ./tools/dpdk_nic_bind.py --bind vfio-pci 0000:84:00.0 0000:84:00.1 0000:84:00.2 0000:84:00.3 + +#. Start ``testpmd`` with basic parameters: + + .. code-block:: console + + testpmd -c 0xf8000 -n 4 -- -i --nb-cores=4 --portmask=0xf --rxd=4096 --txd=4096 --txfreet=4068 --rxq=2 --txq=2 --rss-ip --rss-udp + + [...] + + EAL: PCI device 0000:84:00.0 on NUMA socket 1 + EAL: probe driver: 1077:1634 rte_qede_pmd + EAL: Not managed by a supported kernel driver, skipped + EAL: PCI device 0000:84:00.1 on NUMA socket 1 + EAL: probe driver: 1077:1634 rte_qede_pmd + EAL: Not managed by a supported kernel driver, skipped + EAL: PCI device 0000:88:00.0 on NUMA socket 1 + EAL: probe driver: 1077:1656 rte_qede_pmd + EAL: PCI memory mapped at 0x7f738b200000 + EAL: PCI memory mapped at 0x7f738b280000 + EAL: PCI memory mapped at 0x7f738b300000 + [QEDE PMD: (88:00.0:dpdk-port-0)]qed_load_firmware_data: Loading the firmware file /lib/firmware/qed/qed_init_values_zipped.bin... + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_print_adapter_info:Chip details - BB1 + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_print_adapter_info:Driver version:QEDE PMD 8.7.9.0_1.0.0 + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_print_adapter_info:Firmware version:8.7.7.0 + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_print_adapter_info:Management firmware version:8.7.8.0 + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_common_dev_init:macaddr 00:0e:1e:d2:08:e8 + EAL: PCI device 0000:88:00.1 on NUMA socket 1 + EAL: probe driver: 1077:1656 rte_qede_pmd + EAL: PCI memory mapped at 0x7f738b310000 + EAL: PCI memory mapped at 0x7f738b390000 + EAL: PCI memory mapped at 0x7f738b410000 + [QEDE PMD: (88:00.1:dpdk-port-1)]qede_common_dev_init:macaddr 00:0e:1e:d2:08:e9 + EAL: PCI device 0000:88:00.2 on NUMA socket 1 + EAL: probe driver: 1077:1656 rte_qede_pmd + EAL: PCI memory mapped at 0x7f738b420000 + EAL: PCI memory mapped at 0x7f738b4a0000 + EAL: PCI memory mapped at 0x7f738b520000 + + [QEDE PMD: (88:00.2:dpdk-port-2)]qede_common_dev_init:macaddr 00:0e:1e:d2:08:ea + EAL: PCI device 0000:88:00.3 on NUMA socket 1 + EAL: probe driver: 1077:1656 rte_qede_pmd + EAL: PCI memory mapped at 0x7f738b530000 + EAL: PCI memory mapped at 0x7f738b5b0000 + EAL: PCI memory mapped at 0x7f738b630000 + + [QEDE PMD: (88:00.3:dpdk-port-3)]qede_common_dev_init:macaddr 00:0e:1e:d2:08:eb + Interactive-mode selected + Configuring Port 0 (socket 0) + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 0 num_desc 4096 tx_free_thresh 4068 socket 0 + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 1 num_desc 4096 tx_free_thresh 4068 socket 0 + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 0 num_desc 4096 rx_buf_size=2148 socket 0 + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 1 num_desc 4096 rx_buf_size=2148 socket 0 + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_dev_start:port 0 + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_config_rss:RSS flag is set + [QEDE PMD: (88:00.0:dpdk-port-0)]qede_dev_start:link status: down + Port 0: 00:0E:1E:D2:08:E8 + Configuring Port 1 (socket 0) + [QEDE PMD: (88:00.1:dpdk-port-1)]qede_tx_queue_setup:txq 0 num_desc 4096 tx_free_thresh 4068 socket 0 + [QEDE PMD: (88:00.1:dpdk-port-1)]qede_tx_queue_setup:txq 1 num_desc 4096 tx_free_thresh 4068 socket 0 + [QEDE PMD: (88:00.1:dpdk-port-1)]qede_rx_queue_setup:rxq 0 num_desc 4096 rx_buf_size=2148 socket 0 + [QEDE PMD: (88:00.1:dpdk-port-1)]qede_rx_queue_setup:rxq 1 num_desc 4096 rx_buf_size=2148 socket 0 + [QEDE PMD: (88:00.1:dpdk-port-1)]qede_dev_start:port 1 + ... + Checking link statuses... + Port 0 Link Up - speed 25000 Mbps - full-duplex + Port 1 Link Up - speed 25000 Mbps - full-duplex + Port 2 Link Up - speed 25000 Mbps - full-duplex + Port 3 Link Up - speed 25000 Mbps - full-duplex + Done + testpmd> + + +SR-IOV: Prerequisites and Sample Application Notes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This section provides instructions to configure SR-IOV with Linux OS. + +**Note**: QEDE PMD will be used to bind to SR-IOV VF device and + Linux native kernel driver (QEDE) will function as SR-IOV PF driver. + +#. Verify SR-IOV and ARI capabilities are enabled on the adapter using ``lspci``: + + .. code-block:: console + + lspci -s -vvv + + Example output: + + .. code-block:: console + + [...] + Capabilities: [1b8 v1] Alternative Routing-ID Interpretation (ARI) + [...] + Capabilities: [1c0 v1] Single Root I/O Virtualization (SR-IOV) + [...] + Kernel driver in use: igb_uio + +#. Load the kernel module: + + .. code-block:: console + + modprobe qede + + Example output: + + .. code-block:: console + + systemd-udevd[4848]: renamed network interface eth0 to ens5f0 + systemd-udevd[4848]: renamed network interface eth1 to ens5f1 + +#. Bring up the PF ports: + + .. code-block:: console + + ifconfig ens5f0 up + ifconfig ens5f1 up + +#. Create VF device(s): + + Echo the number of VFs to be created into "sriov_numvfs" sysfs entry + of the parent PF. + + Example output: + + .. code-block:: console + + echo 2 > /sys/devices/pci0000:00/0000:00:03.0/0000:81:00.0/sriov_numvfs + + +#. Assign VF MAC address: + + Assign MAC address to the VF using iproute2 utility. The syntax is: + ip link set vf mac + + Example output: + + .. code-block:: console + + ip link set ens5f0 vf 0 mac 52:54:00:2f:9d:e8 + + +#. PCI Passthrough: + + The VF devices may be passed through to the guest VM using virt-manager or + virsh. QEDE PMD should be used to bind the VF devices in the guest VM + using the instructions outlined in the Application notes below. -- 1.8.3.1