From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f170.google.com (mail-qk0-f170.google.com [209.85.220.170]) by dpdk.org (Postfix) with ESMTP id 9689C7CF1 for ; Tue, 1 Aug 2017 10:15:24 +0200 (CEST) Received: by mail-qk0-f170.google.com with SMTP id x191so4858494qka.5 for ; Tue, 01 Aug 2017 01:15:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=naphzUQyjP1vadFAb7mXOx9qtKQ3f+JBxD9CO6ITbo8=; b=GXE56zeL9xaoY3wZmfx5D3qhl/25+VAzy17YxjgSRqN+2WQ1AzIjQJVwOQA103IXnC 9X78FbZf6WpVDrbRWpdHy7RAJLIQKFhZeYu0t8LtNkHxIAq8sQq/sDD1Uam9ixuMVDdv v0o2GAjgp6pz/I7Rr7xveoxevtFmOKKeCEFIZM7WEls6i0rY/6kljhM+kIdLpm1bpxWH Zk1ZCOwpOR2T4vxfi/p+tusn5k/baoPr9JHPkrakYS+2GrkSNFRX2V/6vl8NXMc/MDDh 6R2zDshmaU4pHzYKjZYI90jzMYbUiYczkHPCxI9eP+k36mEI4yi3K83fQCP1AicZtOtF HqGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=naphzUQyjP1vadFAb7mXOx9qtKQ3f+JBxD9CO6ITbo8=; b=VKEo8JazllSsgDAPiv++t5d0nmOAAqSZExiG/vR8r6ZFwxOccy/GIISRexbN+B09Si UUJh+/6ykWChI/HREXGIzqs0JT3fgZSIGef0CJj5cmO/7Zc6G8JNonVgVwXXp/m1FrBK HhfQPh/GNCSD64ps/XTfTSfnfnUY/SfY6Nu59XRSDfukRH4Ai10Jcpr4/xjJP2mx2ZR9 +YGMz1eEdWqfWQ4DqoaCQg6TWhjaY723kk9JD8AP3YMQcuzPdwLR+ldhy70sWnXxBiLz vj631FBAu2SsB9VWJ8oQsGlS+zSXHClPnOYhDIJu5IGgUSFJkvx8ix72sWQ0l6jYhblM jxqw== X-Gm-Message-State: AIVw110pUkdvuUVJOMNnBJ16kqjFTGuE2vqzooiiK54tjS+Tz0Oeqg5J fHpLwHvA/Go3wiGz08z4TRGvL817LyDZ X-Received: by 10.55.106.197 with SMTP id f188mr20711972qkc.289.1501575323907; Tue, 01 Aug 2017 01:15:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.170.14 with HTTP; Tue, 1 Aug 2017 01:15:23 -0700 (PDT) In-Reply-To: References: From: Paul Tsvika Date: Tue, 1 Aug 2017 16:15:23 +0800 Message-ID: To: zhilong zheng Cc: "users@dpdk.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 08:15:25 -0000 Hi zheng, Thanks for the feedback. You are right. To get pktgen work with mlx4, it needs to recompile DPDK and pktgen as well. I am able to run pktgen now. By following the document from DPDK website for this NIC, I did and change the following variables I set *MLX4_INLINE_RECV_SIZE=3D64* as the environment variable in bashrc. set l*og_num_mgm_entry_size=3D-7* Disable pause frames: *ethtool -A ens2d1 rx off tx 0ff* I did not change the NIC register ( *setpci* ) at this time. and then I run the commands below in order: *test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib* *test$: /etc/init.d/openibd restart* *test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1 --proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P -m "[1-5.0]"* | Ports 0-1 of 2
Copyright (c) <2010-2017>, Intel Corporatio= n Flags:Port : P--------------:0 Link State : <--Down--> ----TotalRate---- Pkts/s Max/Rx : 0/0 0/0 Max/Tx : 844160/732319 844160/732319 MBits/s Rx/Tx : 0/*492 * 0/*492* Broadcast : 0 Multicast : 0 64 Bytes : 0 65-127 : 0 128-255 : 0 256-511 : 0 512-1023 : 0 1024-1518 : 0 Runts/Jumbos : 0/0 Errors Rx/Tx : 0/0 Total Rx Pkts : 0 Tx Pkts : 59289215 Rx MBs : 0 Tx MBs : 39842 ARP/ICMP Pkts : 0/0 Tx Overrun : 0 Cycles per Tx : 14880952/3200 Missed Rx : 0 mcasts Rx : None No Mbuf Rx : 0 : Pattern Type : abcd... Tx Count/% Rate : Forever /100% PktSize/Tx Burst : 64 / 32 Src/Dest Port : 1234 / 5678 Pkt Type:VLAN ID : IPv4 / TCP:0001 Dst IP Address : 192.168.1.1 Src IP Address : 192.168.0.1/24 Src MAC Address : e4:1d:2d:02:44:81 Powered by DPDK -------------------------- The transmitting rate should be around 40000 since it is 40G NIC. However, the rate is much lower than it and it is probably related to configurations. I actually have two mother boards and each one has 40G NICs installed. My intention is to do the tx / rx experiment. My first step is to check tx rate can reach as close to 40000 as possible. Anyone has related experience please advise. P.T 2017-08-01 12:15 GMT+08:00 zhilong zheng : > > Hi, > > After enabling the MLX4 flag and recompiling the DPDK source code, Do you > recompile the code of pktgen? I think you can recompile it by running mak= e > clean and make. > > Regards, > Zhilong > > > =E5=9C=A8 2017=E5=B9=B48=E6=9C=881=E6=97=A5=EF=BC=8C11:38=EF=BC=8CPaul = Tsvika =E5=86=99=E9=81=93=EF=BC=9A > > > > Hi Olga, > > > > 2017-08-01 0:11 GMT+08:00 Olga Shern : > > > >> Hi Paul, > >> > >> Looks like your ConnectX-3 configured to be IB and not ETH > >> > >>> pci@0000:04:00.0 ib0 network MT27500 Family [ConnectX-3] > >> > >> Use connectx_port_config script. > >> Set all the ports to Ethernet, and follow the instructions on the > screen. > >> > > > > Thanks. I followed the script and configured both ports to Ethernet. > > > > ibstat info below: > > > > CA 'mlx4_0' > > CA type: MT4099 > > Number of ports: 2 > > Firmware version: 2.40.7000 > > Hardware version: 1 > > Node GUID: 0xe41d2d0300024480 > > System image GUID: 0xe41d2d0300024483 > > Port 1: > > State: Down > > Physical state: Disabled > > Rate: 10 > > Base lid: 0 > > LMC: 0 > > SM lid: 0 > > Capability mask: 0x04010000 > > Port GUID: 0xe61d2dfffe024481 > > Link layer: Ethernet > > Port 2: > > State: Active > > Physical state: LinkUp > > Rate: 40 > > Base lid: 0 > > LMC: 0 > > SM lid: 0 > > Capability mask: 0x04010000 > > Port GUID: 0xe61d2dfffe024482 > > Link layer: Ethernet > > > > I also change the default value from n to CONFIG_RTE_LIBRTE_MLX4_PMD= =3Dy. > > Recompiled DPDK and it all worked. > > > > And Below is my ifconfig info: > > > > ens2 Link encap:Ethernet HWaddr e4:1d:2d:02:44:81 > > UP BROADCAST MULTICAST MTU:1500 Metric:1 > > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:1000 > > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > > > ens2d1 Link encap:Ethernet HWaddr e4:1d:2d:02:44:82 > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > RX packets:126 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:220 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:1000 > > RX bytes:29494 (29.4 KB) TX bytes:38063 (38.0 KB) > > > > > > Since I assume that I don't have to bind the device manually, I execute= d > > ./dpdk-devbind.py --status with the following info popped out: > > > > Network devices using kernel driver > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=3Deno= 3 > > drv=3Dixgbe unused=3D > > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=3Deno= 4 > > drv=3Dixgbe unused=3D > > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=3Dens2,ens2d1 > > drv=3Dmlx4_core unused=3D > > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=3Deno1 drv=3Digb > unused=3D > > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=3Deno2 drv=3Digb > unused=3D > > *Active* > > > > > > With these configurations, I run commands in order but still had the sa= me > > result: > > > > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib > > $test: /etc/init.d/openibd restart > > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1 > --proc-type > > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P > -m > > "[1-5.0]" > > > > > >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf > cache > > 1024 > > !PANIC!: *** Did not find any ports to use *** > > PANIC in pktgen_config_ports(): > > *** Did not find any ports to use ***6: > > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29) > > [0x447239]] > > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) > > [0x7f0ec0931830]] > > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]] > > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614) > > [0x46e324]] > > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3) > [0x43ead4]] > > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b) > [0x4b7ecb]] > > Aborted (core dumped) > > > > > > > > Is there anything I've done wrong here? > > > > > > Thanks > > > > > > P.T > > > > > >> You can use -w to specify ConnectX-3 PCI address, you don't need to > use -b > >> > >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default > >> > >> Best Regards, > >> Olga > >> > >> > >> > > > > -- > > P.T > --=20 P.T