DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs
@ 2014-11-21 13:35 Sandesh Sawant
  0 siblings, 0 replies; 2+ messages in thread
From: Sandesh Sawant @ 2014-11-21 13:35 UTC (permalink / raw)
  To: dev

Hello dpdk experts,

I am following the documentation available on dpdk.org & the documentation published by Intel to learn how to use DPDK libraries.

I am using dpdk v1.7.1 downloaded from dpdk.org. I wish to run testpmd app provided with this release with 2 VMXNET3 vNICs on my setup.
My test setup is a 64-bit Ubunutu 14.04 (3.13.0) Linux VM running on a VMWare ESXi-5.5.0 hypervisor on Dell server with Intel Xeon E5-2440.

I followed the following stpes to capture the vmxnet3 vNICs in DPDK (all commands run as a root user):

cd /home/sandesh/dpdk/dpdk-1.7.1/
export RTE_SDK=/home/sandesh/dpdk/dpdk-1.7.1/
export RTE_TARGET=build
modprobe uio
insmod build/kmod/igb_uio.ko
modprobe vfio-pci
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active*
0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*
0000:0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*

Other network devices
=====================
<none>

Then I bind VMXNET3 vNICs to igb_uio driver:

../dpdk-1.7.1/tools/dpdk_nic_bind.py --bind=igb_uio eth0 eth1

../dpdk-1.7.1/tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active*
0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*
0000:0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*

Other network devices
=====================
<none>


Note: I downloaded vmxnet3-usermap extension 1.2 source from dpdk.org to work with VMXNET3.
I got a few compilation errors for the same which I resolved and got the driver & .so built.
(I belive these errors in driver code were due to its code not being uptodate wrt linux kernel 3.13).

Next I proceeded to replace the orignial vmxnet3 driver with vmxnet3-usermap.ko as follows:

cd /home/sandesh/dpdk/vmxnet3-usermap-1.2/
rmmod vmxnet3
insmod vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=512 num_txds=512

and then executed:

../dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -d librte_pmd_vmxnet3.so -- -i --rxq=1 --rxd=512 --txd=512

EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 0 on socket 0
EAL: Detected lcore 5 as core 1 on socket 0
EAL: Detected lcore 6 as core 2 on socket 0
EAL: Detected lcore 7 as core 3 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 8 lcore(s)
EAL:   unsupported IOMMU type!
EAL: VFIO support could not be initialized
EAL: Setting up memory...
EAL: Ask a virtual area of 0x5800000 bytes
EAL: Virtual area found at 0x7fb8f7000000 (size = 0x5800000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fb8f6a00000 (size = 0x400000)
EAL: Ask a virtual area of 0x2400000 bytes
EAL: Virtual area found at 0x7fb8f4400000 (size = 0x2400000)
EAL: Requesting 64 pages of size 2MB from socket 0
EAL: TSC frequency is ~2400001 KHz
EAL: open shared lib /home/sandesh/dpdk/vmxnet3-usermap-1.2/librte_pmd_vmxnet3.so
librte_pmd_vmxnet3 version 1.2
        Copyright 2012-2014 6WIND S.A. provided without warranty.
EAL: Master core 0 is ready (tid=fd716880)
EAL: Core 7 is ready (tid=f09ed700)
EAL: Core 6 is ready (tid=f11ee700)
EAL: Core 5 is ready (tid=f19ef700)
EAL: Core 4 is ready (tid=f21f0700)
EAL: Core 3 is ready (tid=f29f1700)
EAL: Core 2 is ready (tid=f31f2700)
EAL: Core 1 is ready (tid=f39f3700)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:01.0 not managed by UIO driver, skipping
EAL: PCI device 0000:03:00.0 on NUMA socket -1
EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
PMD: eth_vmxnet3_dev_init(): No matching device found
EAL: Error - exiting with code: 1
  Cause: Requested device 0000:03:00.0 cannot be used
root@pnq1-vm-eng-dhcp13:/home/sandesh/dpdk/vmxnet3-usermap-1.2#


Can someone please point out what I am missing here?
Am I supposed to bind VMXNET3 vNICs to the vmxnet3-usermap driver instead of igb_uio driver here?


Note: Earlier when I was not running testpmd without vmxnet3-usermap extension, when I was hitting the following error:
dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -- -i
.
.
.
EAL: PCI device 0000:0b:00.0 on NUMA socket -1
EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
EAL:   PCI memory mapped at 0x7f945efe7000
EAL:   PCI memory mapped at 0x7f945efe6000
EAL:   PCI memory mapped at 0x7f945efe4000
Interactive-mode selected
Configuring Port 0 (socket 0)
Fail to configure port 0 tx queues
EAL: Error - exiting with code: 1
  Cause: Start ports failed

Please let me know the correct method to run this application.
Thanks in advance.

Best,
Sandesh

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: [dpdk-dev] Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs
@ 2014-11-21 13:58 Sandesh Sawant
  0 siblings, 0 replies; 2+ messages in thread
From: Sandesh Sawant @ 2014-11-21 13:58 UTC (permalink / raw)
  To: dev

Just an update on the issue I was facing...

I got what was wrong in the steps which I followed. I shouldn't have bound the VNICs to igb_uio driver after replacing vmxnet3 driver. The testpdm app worked fine after eliminating that step from the earlier sequence :) Sorry to bother you all.

Regards,
Sandesh

From: Sandesh Sawant
Sent: Friday, November 21, 2014 7:06 PM
To: 'dev@dpdk.org'
Subject: Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs

Hello dpdk experts,

I am following the documentation available on dpdk.org & the documentation published by Intel to learn how to use DPDK libraries.

I am using dpdk v1.7.1 downloaded from dpdk.org. I wish to run testpmd app provided with this release with 2 VMXNET3 vNICs on my setup.
My test setup is a 64-bit Ubunutu 14.04 (3.13.0) Linux VM running on a VMWare ESXi-5.5.0 hypervisor on Dell server with Intel Xeon E5-2440.

I followed the following stpes to capture the vmxnet3 vNICs in DPDK (all commands run as a root user):

cd /home/sandesh/dpdk/dpdk-1.7.1/
export RTE_SDK=/home/sandesh/dpdk/dpdk-1.7.1/
export RTE_TARGET=build
modprobe uio
insmod build/kmod/igb_uio.ko
modprobe vfio-pci
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active*
0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*
0000:0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*

Other network devices
=====================
<none>

Then I bind VMXNET3 vNICs to igb_uio driver:

../dpdk-1.7.1/tools/dpdk_nic_bind.py --bind=igb_uio eth0 eth1

../dpdk-1.7.1/tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active*
0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*
0000:0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*

Other network devices
=====================
<none>


Note: I downloaded vmxnet3-usermap extension 1.2 source from dpdk.org to work with VMXNET3.
I got a few compilation errors for the same which I resolved and got the driver & .so built.
(I belive these errors in driver code were due to its code not being uptodate wrt linux kernel 3.13).

Next I proceeded to replace the orignial vmxnet3 driver with vmxnet3-usermap.ko as follows:

cd /home/sandesh/dpdk/vmxnet3-usermap-1.2/
rmmod vmxnet3
insmod vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=512 num_txds=512

and then executed:

../dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -d librte_pmd_vmxnet3.so -- -i --rxq=1 --rxd=512 --txd=512

EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 0 on socket 0
EAL: Detected lcore 5 as core 1 on socket 0
EAL: Detected lcore 6 as core 2 on socket 0
EAL: Detected lcore 7 as core 3 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 8 lcore(s)
EAL:   unsupported IOMMU type!
EAL: VFIO support could not be initialized
EAL: Setting up memory...
EAL: Ask a virtual area of 0x5800000 bytes
EAL: Virtual area found at 0x7fb8f7000000 (size = 0x5800000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fb8f6a00000 (size = 0x400000)
EAL: Ask a virtual area of 0x2400000 bytes
EAL: Virtual area found at 0x7fb8f4400000 (size = 0x2400000)
EAL: Requesting 64 pages of size 2MB from socket 0
EAL: TSC frequency is ~2400001 KHz
EAL: open shared lib /home/sandesh/dpdk/vmxnet3-usermap-1.2/librte_pmd_vmxnet3.so
librte_pmd_vmxnet3 version 1.2
        Copyright 2012-2014 6WIND S.A. provided without warranty.
EAL: Master core 0 is ready (tid=fd716880)
EAL: Core 7 is ready (tid=f09ed700)
EAL: Core 6 is ready (tid=f11ee700)
EAL: Core 5 is ready (tid=f19ef700)
EAL: Core 4 is ready (tid=f21f0700)
EAL: Core 3 is ready (tid=f29f1700)
EAL: Core 2 is ready (tid=f31f2700)
EAL: Core 1 is ready (tid=f39f3700)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:01.0 not managed by UIO driver, skipping
EAL: PCI device 0000:03:00.0 on NUMA socket -1
EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
PMD: eth_vmxnet3_dev_init(): No matching device found
EAL: Error - exiting with code: 1
  Cause: Requested device 0000:03:00.0 cannot be used
root@pnq1-vm-eng-dhcp13:/home/sandesh/dpdk/vmxnet3-usermap-1.2#<mailto:root@pnq1-vm-eng-dhcp13:/home/sandesh/dpdk/vmxnet3-usermap-1.2#>


Can someone please point out what I am missing here?
Am I supposed to bind VMXNET3 vNICs to the vmxnet3-usermap driver instead of igb_uio driver here?


Note: Earlier when I was not running testpmd without vmxnet3-usermap extension, when I was hitting the following error:
dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -- -i
.
.
.
EAL: PCI device 0000:0b:00.0 on NUMA socket -1
EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
EAL:   PCI memory mapped at 0x7f945efe7000
EAL:   PCI memory mapped at 0x7f945efe6000
EAL:   PCI memory mapped at 0x7f945efe4000
Interactive-mode selected
Configuring Port 0 (socket 0)
Fail to configure port 0 tx queues
EAL: Error - exiting with code: 1
  Cause: Start ports failed

Please let me know the correct method to run this application.
Thanks in advance.

Best,
Sandesh

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2014-11-21 13:47 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-11-21 13:35 [dpdk-dev] Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs Sandesh Sawant
2014-11-21 13:58 Sandesh Sawant

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).