From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-outbound-2.vmware.com (smtp-outbound-2.vmware.com [208.91.2.13]) by dpdk.org (Postfix) with ESMTP id 45D257F74 for ; Fri, 21 Nov 2014 14:25:33 +0100 (CET) Received: from sc9-mailhost2.vmware.com (sc9-mailhost2.vmware.com [10.113.161.72]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id 24C4328941 for ; Fri, 21 Nov 2014 05:36:04 -0800 (PST) Received: from EX13-CAS-004.vmware.com (EX13-CAS-004.vmware.com [10.113.191.54]) by sc9-mailhost2.vmware.com (Postfix) with ESMTP id 1D639B0A2F for ; Fri, 21 Nov 2014 05:36:04 -0800 (PST) Received: from EX13-MBX-017.vmware.com (10.113.191.37) by EX13-MBX-002.vmware.com (10.113.191.22) with Microsoft SMTP Server (TLS) id 15.0.913.22; Fri, 21 Nov 2014 05:36:15 -0800 Received: from EX13-MBX-013.vmware.com (10.113.191.33) by EX13-MBX-017.vmware.com (10.113.191.37) with Microsoft SMTP Server (TLS) id 15.0.913.22; Fri, 21 Nov 2014 05:35:47 -0800 Received: from EX13-MBX-013.vmware.com ([fe80::b82b:d0bb:9cd7:392e]) by EX13-MBX-013.vmware.com ([fe80::b82b:d0bb:9cd7:392e%15]) with mapi id 15.00.0913.011; Fri, 21 Nov 2014 05:35:47 -0800 From: Sandesh Sawant To: "dev@dpdk.org" Thread-Topic: Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs Thread-Index: AdAFj6igGDZMA1hhTZSo+kfHz7FQ+g== Date: Fri, 21 Nov 2014 13:35:46 +0000 Message-ID: <176b9372581a4f54a2fdf91b802a5538@EX13-MBX-013.vmware.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.113.160.246] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Nov 2014 13:25:34 -0000 Hello dpdk experts, I am following the documentation available on dpdk.org & the documentation = published by Intel to learn how to use DPDK libraries. I am using dpdk v1.7.1 downloaded from dpdk.org. I wish to run testpmd app = provided with this release with 2 VMXNET3 vNICs on my setup. My test setup is a 64-bit Ubunutu 14.04 (3.13.0) Linux VM running on a VMWa= re ESXi-5.5.0 hypervisor on Dell server with Intel Xeon E5-2440. I followed the following stpes to capture the vmxnet3 vNICs in DPDK (all co= mmands run as a root user): cd /home/sandesh/dpdk/dpdk-1.7.1/ export RTE_SDK=3D/home/sandesh/dpdk/dpdk-1.7.1/ export RTE_TARGET=3Dbuild modprobe uio insmod build/kmod/igb_uio.ko modprobe vfio-pci mkdir -p /mnt/huge mount -t hugetlbfs nodev /mnt/huge echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_huge= pages tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Network devices using kernel driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=3Deth2 drv= =3De1000 unused=3Digb_uio,vfio-pci *Active* 0000:03:00.0 'VMXNET3 Ethernet Controller' if=3Deth0 drv=3Dvmxnet3 unused= =3Digb_uio,vfio-pci *Active* 0000:0b:00.0 'VMXNET3 Ethernet Controller' if=3Deth1 drv=3Dvmxnet3 unused= =3Digb_uio,vfio-pci *Active* Other network devices =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Then I bind VMXNET3 vNICs to igb_uio driver: ../dpdk-1.7.1/tools/dpdk_nic_bind.py --bind=3Digb_uio eth0 eth1 ../dpdk-1.7.1/tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Network devices using kernel driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=3Deth2 drv= =3De1000 unused=3Digb_uio,vfio-pci *Active* 0000:03:00.0 'VMXNET3 Ethernet Controller' if=3Deth0 drv=3Dvmxnet3 unused= =3Digb_uio,vfio-pci *Active* 0000:0b:00.0 'VMXNET3 Ethernet Controller' if=3Deth1 drv=3Dvmxnet3 unused= =3Digb_uio,vfio-pci *Active* Other network devices =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Note: I downloaded vmxnet3-usermap extension 1.2 source from dpdk.org to wo= rk with VMXNET3. I got a few compilation errors for the same which I resolved and got the dr= iver & .so built. (I belive these errors in driver code were due to its code not being uptoda= te wrt linux kernel 3.13). Next I proceeded to replace the orignial vmxnet3 driver with vmxnet3-userma= p.ko as follows: cd /home/sandesh/dpdk/vmxnet3-usermap-1.2/ rmmod vmxnet3 insmod vmxnet3-usermap.ko enable_shm=3D2,2 num_rqs=3D1,1 num_rxds=3D512 num= _txds=3D512 and then executed: ../dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -d librte_pmd_vmxnet3.so -- -i= --rxq=3D1 --rxd=3D512 --txd=3D512 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 0 on socket 0 EAL: Detected lcore 5 as core 1 on socket 0 EAL: Detected lcore 6 as core 2 on socket 0 EAL: Detected lcore 7 as core 3 on socket 0 EAL: Support maximum 64 logical core(s) by configuration. EAL: Detected 8 lcore(s) EAL: unsupported IOMMU type! EAL: VFIO support could not be initialized EAL: Setting up memory... EAL: Ask a virtual area of 0x5800000 bytes EAL: Virtual area found at 0x7fb8f7000000 (size =3D 0x5800000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fb8f6a00000 (size =3D 0x400000) EAL: Ask a virtual area of 0x2400000 bytes EAL: Virtual area found at 0x7fb8f4400000 (size =3D 0x2400000) EAL: Requesting 64 pages of size 2MB from socket 0 EAL: TSC frequency is ~2400001 KHz EAL: open shared lib /home/sandesh/dpdk/vmxnet3-usermap-1.2/librte_pmd_vmxn= et3.so librte_pmd_vmxnet3 version 1.2 Copyright 2012-2014 6WIND S.A. provided without warranty. EAL: Master core 0 is ready (tid=3Dfd716880) EAL: Core 7 is ready (tid=3Df09ed700) EAL: Core 6 is ready (tid=3Df11ee700) EAL: Core 5 is ready (tid=3Df19ef700) EAL: Core 4 is ready (tid=3Df21f0700) EAL: Core 3 is ready (tid=3Df29f1700) EAL: Core 2 is ready (tid=3Df31f2700) EAL: Core 1 is ready (tid=3Df39f3700) EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: 0000:02:01.0 not managed by UIO driver, skipping EAL: PCI device 0000:03:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd PMD: eth_vmxnet3_dev_init(): No matching device found EAL: Error - exiting with code: 1 Cause: Requested device 0000:03:00.0 cannot be used root@pnq1-vm-eng-dhcp13:/home/sandesh/dpdk/vmxnet3-usermap-1.2# Can someone please point out what I am missing here? Am I supposed to bind VMXNET3 vNICs to the vmxnet3-usermap driver instead o= f igb_uio driver here? Note: Earlier when I was not running testpmd without vmxnet3-usermap extens= ion, when I was hitting the following error: dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -- -i . . . EAL: PCI device 0000:0b:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd EAL: PCI memory mapped at 0x7f945efe7000 EAL: PCI memory mapped at 0x7f945efe6000 EAL: PCI memory mapped at 0x7f945efe4000 Interactive-mode selected Configuring Port 0 (socket 0) Fail to configure port 0 tx queues EAL: Error - exiting with code: 1 Cause: Start ports failed Please let me know the correct method to run this application. Thanks in advance. Best, Sandesh