From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by dpdk.org (Postfix) with ESMTP id 1F74358C4 for ; Fri, 7 Sep 2018 20:06:51 +0200 (CEST) Received: by mail-qk1-f176.google.com with SMTP id z125-v6so10220148qkb.12 for ; Fri, 07 Sep 2018 11:06:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0RLrF7ZfIzwtGGwumdeFAQxpK9vHohYchpAO2pHMUDo=; b=Wg7kXbrN9sfMHxSiHpk8/2TLCnRIXu6HBY3Zu/JjiBNqx2t2Eu9X2t6GenQBeR3aWz tQvfD60bVhVJcgBeUn2nRhO6ZBwZMZKLMvTyBZUHDXGHoMFY3PIwEjqtWfciumVLUS3x Km/H0FBRqKHQZxUr+NECtOpk+06RFsYe202Hu26zQity3Bum1FNEb9e/r+RcXshis60w v1Se1QNtpUWWQzWDFpzln1EIxUpKL7/Yykr76ru8PO83XOCwrGJ6KnQ+uYVt5NFCsoRe +m15roblQ8VaJweU2RNG11EBoSHLY+whKKkZ7XAdadDoYRpPbM2SK+BAMlSfQHpyCeE7 ViQg== X-Gm-Message-State: APzg51AQMd047c5C9r878MAE2uJifyZGVxr8cLvQIAK/3X4eodW05eot mu9LiVKZSVSWr0a7LvowqKZFZx6N/2/OaBQQ6VeFwFA2SHs= X-Google-Smtp-Source: ANB0Vdb7fHFQGsx2N5/3GmTCwqoyO7JoAQErZCQkv6XJaT1BeBbVCq2qPnp9Ad+iQAjBq6yeG6//TDjY+OSAKqaftqU= X-Received: by 2002:a37:6458:: with SMTP id y85-v6mr6636897qkb.59.1536343611076; Fri, 07 Sep 2018 11:06:51 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:aed:2429:0:0:0:0:0 with HTTP; Fri, 7 Sep 2018 11:06:50 -0700 (PDT) In-Reply-To: References: From: Jamie Fargen Date: Fri, 7 Sep 2018 14:06:50 -0400 Message-ID: To: Emre Eraltan Cc: users@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] TestPMD testing with VMX3NET Devices... X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2018 18:06:52 -0000 Emre- Things are looking better, but this is a ESX6.x hypervisor with CentOS 7.5 Guest, and it looks like vmxnet3 is mis-reporting the driver version. I will try and get more details about the hypervisor. # insmod /root/dpdk-stable-17.11.4/build/build/lib/librte_eal/linuxapp/igb_uio/igb_u= io.ko # ./dpdk-devbind.py -b igb_uio 0b:00.0 # ./dpdk-devbind.py -b igb_uio 13:00.0 # echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages # mount -t hugetlbfs nodev /mnt/huge # testpmd -l 1,2,3 -n 1 -- --disable-hw-vlan --forward-mode=3Dmac --eth-peer=3D0,00:00:00:00:33:33 --eth-peer=3D1,00:00:00:00:44:44 -i --nb-cores=3D2 --rxq=3D8 --txq=3D8 EAL: Detected 4 lcore(s) EAL: No free hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: PCI device 0000:0b:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15ad:7b0 net_vmxnet3 PMD: eth_vmxnet3_dev_init(): Incompatible hardware version: 0 EAL: Requested device 0000:0b:00.0 cannot be used EAL: PCI device 0000:13:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15ad:7b0 net_vmxnet3 PMD: eth_vmxnet3_dev_init(): Incompatible hardware version: 0 EAL: Requested device 0000:13:00.0 cannot be used EAL: PCI device 0000:1b:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: No probed ethernet devices Set mac packet forwarding mode Interactive-mode selected USER1: create a new mbuf pool : n=3D163456, size=3D2176= , socket=3D0 Done testpmd> show port info all testpmd> On Fri, Sep 7, 2018 at 1:24 PM, Emre Eraltan wrote= : > Hi Jamie, > > I have recently used testpmd on VMware ESX 5.5 with a Ubuntu 16.04 VM - > and it was working with vmxnet3 devices. > > Did you try lowering your RXD and TXD sizes? > I am not sure VMXNET3 supports your values "--rxd=3D8192 --txd=3D8192". > > Try removing these 2 parameters to use the default sizes or lower your > values. > > From my history: > $ wget http://fast.dpdk.org/rel/dpdk-17.11.4.tar.xz > $ tar -xf dpdk-17.11.4.tar.xz > $ cd dpdk-stable-17.11.4 > $ make config T=3Dx86_64-native-linuxapp-gcc > $ make ## had to install libnuma-dev package > $ modprobe uio > $ insmod ./build/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.ko > $ ./usertools/dpdk-devbind.py -b igb_uio 03:00.0 > $ ./usertools/dpdk-devbind.py -b igb_uio 0b:00.0 > $ echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/ > nr_hugepages > $ mkdir -p /mnt/huge > $ mount -t hugetlbfs nodev /mnt/huge > $ testpmd -l 1,2,3 -n 1 -- --disable-hw-vlan --forward-mode=3Dmac > --eth-peer=3D0,00:00:00:00:33:33 --eth-peer=3D1,00:00:00:00:44:44 -i > --nb-cores=3D2 --rxq=3D8 --txq=3D8 > > testpmd> show port info all > > ********************* Infos for port 0 ********************* > MAC address: 00:0C:29:06:04:24 > Driver name: net_vmxnet3 > Connect to socket: 0 > memory allocation on the socket: 0 > Link status: up > Link speed: 10000 Mbps > Link duplex: full-duplex > MTU: 1500 > Promiscuous mode: enabled > Allmulticast mode: disabled > Maximum number of MAC addresses: 1 > Maximum number of MAC addresses of hash filtering: 0 > VLAN offload: > strip off > filter off > qinq(extend) off > Supported flow types: > ipv4 > ipv4-tcp > ipv6 > ipv6-tcp > Max possible RX queues: 16 > Max possible number of RXDs per queue: 4096 > Min possible number of RXDs per queue: 128 > RXDs number alignment: 1 > Max possible TX queues: 8 > Max possible number of TXDs per queue: 4096 > Min possible number of TXDs per queue: 512 > TXDs number alignment: 1 > > ********************* Infos for port 1 ********************* > MAC address: 00:0C:29:06:04:2E > Driver name: net_vmxnet3 > Connect to socket: 0 > memory allocation on the socket: 0 > Link status: up > Link speed: 10000 Mbps > Link duplex: full-duplex > MTU: 1500 > Promiscuous mode: enabled > Allmulticast mode: disabled > Maximum number of MAC addresses: 1 > Maximum number of MAC addresses of hash filtering: 0 > VLAN offload: > strip off > filter off > qinq(extend) off > Supported flow types: > ipv4 > ipv4-tcp > ipv6 > ipv6-tcp > Max possible RX queues: 16 > Max possible number of RXDs per queue: 4096 > Min possible number of RXDs per queue: 128 > RXDs number alignment: 1 > Max possible TX queues: 8 > Max possible number of TXDs per queue: 4096 > Min possible number of TXDs per queue: 512 > TXDs number alignment: 1 > > Hope it helps. > > Regards, > Emre > > On Thu, Sep 6, 2018 at 9:39 AM, Jamie Fargen wrote: > >> Hello- >> >> Would like to do some performance analysis using testpmd on a RHEL7 VMWa= re >> guest using a VMX3NET network devices. Similar tests have been performed >> using RHEL7 KVM guests using VirtIO network devices, but the same proces= s >> does not work with VMX3NET network interfaces. >> >> The dpdk-stable-17.11 has been compiled and it looks like the devices ar= e >> properly bound to the uio driver, but when testpmd is started it is unab= le >> to locate the devices. >> >> This is the basic process of how the uio module is loaded, the devices o= ur >> bound to the driver, and testpmd is started. >> >> [root@redacted ~]# cat startTestPmd17.sh >> #!/bin/bash -x >> modprobe uio >> insmod /root/dpdk-stable-17.11.4/build/kmod/igb_uio.ko >> /root/dpdk-stable-17.11.4/usertools/dpdk-devbind.py -b igb_uio 0b:00.0 >> /root/dpdk-stable-17.11.4/usertools/dpdk-devbind.py -b igb_uio 13:00.0 >> mount -t hugetlbfs nodev /mnt/huge >> echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages- >> 2048kB/nr_hugepages >> testpmd -l 1,2,3 -n 1 -- --disable-hw-vlan --forward-mode=3Dmac >> --eth-peer=3D0,00:00:00:00:33:33 --eth-peer=3D1,00:00:00:00:44:44 -i >> --nb-cores=3D2 --rxq=3D8 --txq=3D8 --rxd=3D8192 --txd=3D8192 >> >> EAL: Detected 4 lcore(s) >> EAL: No free hugepages reported in hugepages-1048576kB >> EAL: Probing VFIO support... >> EAL: No probed ethernet devices >> Set mac packet forwarding mode >> Interactive-mode selected >> USER1: create a new mbuf pool : n=3D163456, size=3D2= 176, >> socket=3D0 >> Done >> testpmd> >> >> >> There are network devices bound to DPDK-compatible uio driver. >> [root@redacted ~]# dpdk-stable-17.11.4/usertools/dpdk-devbind.py -s >> >> Network devices using DPDK-compatible driver >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> 0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3Dv= mxnet3 >> 0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3Dv= mxnet3 >> >> Network devices using kernel driver >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> 0000:1b:00.0 'VMXNET3 Ethernet Controller 07b0' if=3Dens256 drv=3Dvmxnet= 3 >> unused=3Digb_uio *Active* >> >> >> If anyone has any spare cycles to help me solve this issue it would be >> greatly appreciated. >> >> >> -- >> Jamie Fargen >> Senior Consultant >> jfargen@redhat.com >> 813-817-4430 >> > > > > -- > > -- > Emre Eraltan > Lead Solutions Architect > > Direct (US): +1 408 508 6733 > Mobile (US): +1 408 329 0271 > > 2975 Scott Blvd, Suite 115 > Santa Clara 95054 CA > > Visit our web site: http://www.6wind.com > Join us on Twitter: http://twitter.com/6windsoftware > Join us on LinkedIn: https://www.linkedin.com/company/6wind > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D > This e-mail message, including any attachments, is for the sole use of th= e intended recipient(s) and contains information that is confidential and p= roprietary to 6WIND. All unauthorized review, use, disclosure or distributi= on is prohibited. If you are not the intended recipient, please contact the= sender by reply e-mail and destroy all copies of the original message. > Ce courriel ainsi que toutes les pi=C3=A8ces jointes, est uniquement dest= in=C3=A9 =C3=A0 son ou ses destinataires. Il contient des informations conf= identielles qui sont la propri=C3=A9t=C3=A9 de 6WIND. Toute r=C3=A9v=C3=A9l= ation, distribution ou copie des informations qu'il contient est strictemen= t interdite. Si vous avez re=C3=A7u ce message par erreur, veuillez imm=C3= =A9diatement le signaler =C3=A0 l'=C3=A9metteur et d=C3=A9truire toutes les= donn=C3=A9es re=C3=A7ues > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D > > --=20 Jamie Fargen Senior Consultant jfargen@redhat.com 813-817-4430