[dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors

DPDK usage discussions
 help / color / mirror / Atom feed

* [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
@ 2017-07-31 10:14 Paul Tsvika
  2017-07-31 12:54 ` zhilong zheng
  2017-07-31 16:11 ` Olga Shern
  0 siblings, 2 replies; 17+ messages in thread
From: Paul Tsvika @ 2017-07-31 10:14 UTC (permalink / raw)
  To: users

Hi,

Below is my configuration:

AOC: Mellanox MT27500 Family [ConnectX-3]
OS: Ubuntu 16.04
DPDK version: 17.05.1
Pktgen: 3.3.8
Memory: 16G
CPU: Intel(R) Xeon(R) CPU D-1557 (12C)

I've enabled the configuration in dpdk (CONFIG_RTE_LIBRTE_MLX4_PMD=y) and
installed Mellanox OFED
<http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers>
driver and recompiled DPDK. It all worked.

Below is my pci network class information:

pci@0000:03:00.0  eno3    network        Ethernet Connection X552/X557-AT
10GBAS
pci@0000:03:00.1  eno4    network        Ethernet Connection X552/X557-AT
10GBAS
pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
pci@0000:05:00.0  eno1    network        I350 Gigabit Network Connection
pci@0000:05:00.1  eno2    network        I350 Gigabit Network Connection
pci@0000:04:00.0  ens2d1  network        Ethernet interface

And my step to run pktgen is shown in the following:

$test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
$test: /etc/init.d/openibd restart
$test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1 --proc-type
auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P -m
"[1-5.0]"


Looks like in order to use mlx4, i need to blacklist other ports so that i
-b other ports.


However, when i run the command above to execute pktgen, i got the
following error:

>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf cache
1024
!PANIC!: *** Did not find any ports to use ***
PANIC in pktgen_config_ports():
*** Did not find any ports to use ***6:
[./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29) [0x447239]]
5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
[0x7f0ec0931830]]
4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
[0x46e324]]
2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3) [0x43ead4]]
1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b) [0x4b7ecb]]
Aborted (core dumped)


Anyone might have related experience please provide advise.


Thanks


P.T


-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-07-31 10:14 [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors Paul Tsvika
@ 2017-07-31 12:54 ` zhilong zheng
  2017-07-31 13:07   ` Paul Tsvika
  2017-07-31 16:11 ` Olga Shern
  1 sibling, 1 reply; 17+ messages in thread
From: zhilong zheng @ 2017-07-31 12:54 UTC (permalink / raw)
  To: Paul Tsvika; +Cc: users

Hi Paul,

It seems that DPDK can only find the ethernet device with Mellanox. So you can check your interfaces’ link layer protocol by running command “ibstat” to ensure that they are using ethernet, but not infiniband. 

Hope it can help you.

Regards,
Zhilong

> 在 2017年7月31日，18:14，Paul Tsvika <mozloverinweb@gmail.com> 写道：
> 
> Hi,
> 
> Below is my configuration:
> 
> AOC: Mellanox MT27500 Family [ConnectX-3]
> OS: Ubuntu 16.04
> DPDK version: 17.05.1
> Pktgen: 3.3.8
> Memory: 16G
> CPU: Intel(R) Xeon(R) CPU D-1557 (12C)
> 
> I've enabled the configuration in dpdk (CONFIG_RTE_LIBRTE_MLX4_PMD=y) and
> installed Mellanox OFED

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-07-31 12:54 ` zhilong zheng
@ 2017-07-31 13:07   ` Paul Tsvika
  2017-07-31 13:14     ` zhilong zheng
  0 siblings, 1 reply; 17+ messages in thread
From: Paul Tsvika @ 2017-07-31 13:07 UTC (permalink / raw)
  To: zhilong zheng; +Cc: users

Hi zhilong,

I did not bind the device with dpdk-devbind.py.
I am not sure if this is a must because I don't see it in the tutorial.

the command I have used to test other ethernet devices was

./dpdk-devbind.py -b igb_uio pcie_addr

Since mlx4 does not use igb_uio as its driver, if this is a must, which
driver I should take instead ?


Thanks



P.T


2017-07-31 20:54 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:

> Hi Paul,
>
> It seems that DPDK can only find the ethernet device with Mellanox. So you
> can check your interfaces’ link layer protocol by running command “
> *ibstat*” to ensure that they are using ethernet, but not infiniband.
>
> Hope it can help you.
>
> Regards,
> Zhilong
>
> 在 2017年7月31日，18:14，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>
> Hi,
>
> Below is my configuration:
>
> AOC: Mellanox MT27500 Family [ConnectX-3]
> OS: Ubuntu 16.04
> DPDK version: 17.05.1
> Pktgen: 3.3.8
> Memory: 16G
> CPU: Intel(R) Xeon(R) CPU D-1557 (12C)
>
> I've enabled the configuration in dpdk (CONFIG_RTE_LIBRTE_MLX4_PMD=y) and
> installed Mellanox OFED
>
>
>


-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-07-31 13:07   ` Paul Tsvika
@ 2017-07-31 13:14     ` zhilong zheng
  2017-07-31 13:25       ` Paul Tsvika
  0 siblings, 1 reply; 17+ messages in thread
From: zhilong zheng @ 2017-07-31 13:14 UTC (permalink / raw)
  To: Paul Tsvika; +Cc: users

Yes, CX3 does not use igb_uio, it uses mlx4_core as the driver and you do not need to bind the device to igb_uio. Can you show the output when running command “./dpdk-devbind.py —status” and “ibstat"

> 在 2017年7月31日，21:07，Paul Tsvika <mozloverinweb@gmail.com> 写道：
> 
> er I should 

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-07-31 13:14     ` zhilong zheng
@ 2017-07-31 13:25       ` Paul Tsvika
  2017-07-31 13:40         ` zhilong zheng
  0 siblings, 1 reply; 17+ messages in thread
From: Paul Tsvika @ 2017-07-31 13:25 UTC (permalink / raw)
  To: zhilong zheng; +Cc: users

Sorry I am not with my testing machine now.
if it uses mlx4_core, should I bind the device to the driver by using the
following command?

*./dpdk-devbind.py -b mlx4_core xx:xx.x*


Thanks


P.T

2017-07-31 21:14 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:

> Yes, CX3 does not use igb_uio, it uses mlx4_core as the driver and you do
> not need to bind the device to igb_uio. Can you show the output when
> running command “*./dpdk-devbind.py —status*” and “*ibstat*"
>
> 在 2017年7月31日，21:07，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>
> er I should
>
>
>


-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-07-31 13:25       ` Paul Tsvika
@ 2017-07-31 13:40         ` zhilong zheng
  2017-07-31 14:00           ` Paul Tsvika
  0 siblings, 1 reply; 17+ messages in thread
From: zhilong zheng @ 2017-07-31 13:40 UTC (permalink / raw)
  To: Paul Tsvika; +Cc: users

No, you do not need any operation. This driver is bound to the device automatically.

> 在 2017年7月31日，21:25，Paul Tsvika <mozloverinweb@gmail.com> 写道：
> 
> Sorry I am not with my testing machine now. 
> if it uses mlx4_core, should I bind the device to the driver by using the following command?
> 
> ./dpdk-devbind.py -b mlx4_core xx:xx.x
> 
> 
> Thanks
> 
> 
> P.T
> 
> 2017-07-31 21:14 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com <mailto:zhengzl0715@gmail.com>>:
> Yes, CX3 does not use igb_uio, it uses mlx4_core as the driver and you do not need to bind the device to igb_uio. Can you show the output when running command “./dpdk-devbind.py —status” and “ibstat"
> 
>> 在 2017年7月31日，21:07，Paul Tsvika <mozloverinweb@gmail.com <mailto:mozloverinweb@gmail.com>> 写道：
>> 
>> er I should 
> 
> 
> 
> 
> -- 
> P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-07-31 13:40         ` zhilong zheng
@ 2017-07-31 14:00           ` Paul Tsvika
  0 siblings, 0 replies; 17+ messages in thread
From: Paul Tsvika @ 2017-07-31 14:00 UTC (permalink / raw)
  To: zhilong zheng; +Cc: users

I will provide the info for the "./dpdk-devbind.py" and "ibstat"


Regards-


P.T

2017-07-31 21:40 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:

> No, you do not need any operation. This driver is bound to the device
> automatically.
>
> 在 2017年7月31日，21:25，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>
> Sorry I am not with my testing machine now.
> if it uses mlx4_core, should I bind the device to the driver by using the
> following command?
>
> *./dpdk-devbind.py -b mlx4_core xx:xx.x*
>
>
> Thanks
>
>
> P.T
>
> 2017-07-31 21:14 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:
>
>> Yes, CX3 does not use igb_uio, it uses mlx4_core as the driver and you do
>> not need to bind the device to igb_uio. Can you show the output when
>> running command “*./dpdk-devbind.py —status*” and “*ibstat*"
>>
>> 在 2017年7月31日，21:07，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>>
>> er I should
>>
>>
>>
>
>
> --
> P.T
>
>
>


-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-07-31 10:14 [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors Paul Tsvika
  2017-07-31 12:54 ` zhilong zheng
@ 2017-07-31 16:11 ` Olga Shern
  2017-08-01  3:38   ` Paul Tsvika
  1 sibling, 1 reply; 17+ messages in thread
From: Olga Shern @ 2017-07-31 16:11 UTC (permalink / raw)
  To: Paul Tsvika, users

Hi Paul, 

Looks like your ConnectX-3 configured to be IB and not ETH 

> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]

Use connectx_port_config script.
Set all the ports to Ethernet, and follow the instructions on the screen.

You can use  -w to specify ConnectX-3 PCI address, you don't need to use -b

Make sure you compile DPDK with mlx4 PMD, it is not enabled by default  

Best Regards,
Olga


> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Paul Tsvika
> Sent: Monday, July 31, 2017 1:14 PM
> To: users@dpdk.org
> Subject: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with
> some errors
> 
> Hi,
> 
> Below is my configuration:
> 
> AOC: Mellanox MT27500 Family [ConnectX-3]
> OS: Ubuntu 16.04
> DPDK version: 17.05.1
> Pktgen: 3.3.8
> Memory: 16G
> CPU: Intel(R) Xeon(R) CPU D-1557 (12C)
> 
> I've enabled the configuration in dpdk (CONFIG_RTE_LIBRTE_MLX4_PMD=y)
> and installed Mellanox OFED
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fw
> ww.mellanox.com%2Fpage%2Fproducts_dyn%3Fproduct_family%3D26%26
> mtag%3Dlinux_sw_drivers&data=02%7C01%7Colgas%40mellanox.com%7C4f
> e2f94155994a2a579008d4d7fce66a%7Ca652971c7d2e4d9ba6a4d149256f461b
> %7C0%7C0%7C636370928586578406&sdata=%2FPnEAOJFsGtQRY%2FP5u0Hpl
> L6kDbnhwRGIgSV%2B31w40Q%3D&reserved=0>
> driver and recompiled DPDK. It all worked.
> 
> Below is my pci network class information:
> 
> pci@0000:03:00.0  eno3    network        Ethernet Connection X552/X557-AT
> 10GBAS
> pci@0000:03:00.1  eno4    network        Ethernet Connection X552/X557-AT
> 10GBAS
> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
> pci@0000:05:00.0  eno1    network        I350 Gigabit Network Connection
> pci@0000:05:00.1  eno2    network        I350 Gigabit Network Connection
> pci@0000:04:00.0  ens2d1  network        Ethernet interface
> 
> And my step to run pktgen is shown in the following:
> 
> $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
> $test: /etc/init.d/openibd restart
> $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1 --proc-type
> auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P -m "[1-
> 5.0]"
> 
> 
> Looks like in order to use mlx4, i need to blacklist other ports so that i -b
> other ports.
> 
> 
> However, when i run the command above to execute pktgen, i got the
> following error:
> 
> >>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
> >>> cache
> 1024
> !PANIC!: *** Did not find any ports to use *** PANIC in
> pktgen_config_ports():
> *** Did not find any ports to use ***6:
> [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29) [0x447239]]
> 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
> [0x7f0ec0931830]]
> 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
> 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
> [0x46e324]]
> 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3) [0x43ead4]]
> 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
> [0x4b7ecb]] Aborted (core dumped)
> 
> 
> Anyone might have related experience please provide advise.
> 
> 
> Thanks
> 
> 
> P.T
> 
> 
> --
> P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-07-31 16:11 ` Olga Shern
@ 2017-08-01  3:38   ` Paul Tsvika
  2017-08-01  4:15     ` zhilong zheng
  0 siblings, 1 reply; 17+ messages in thread
From: Paul Tsvika @ 2017-08-01  3:38 UTC (permalink / raw)
  To: Olga Shern; +Cc: users

Hi Olga,

2017-08-01 0:11 GMT+08:00 Olga Shern <olgas@mellanox.com>:

> Hi Paul,
>
> Looks like your ConnectX-3 configured to be IB and not ETH
>
> > pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
>
> Use connectx_port_config script.
> Set all the ports to Ethernet, and follow the instructions on the screen.
>

Thanks. I followed the script and configured  both ports to Ethernet.

ibstat info below:

CA 'mlx4_0'
    CA type: MT4099
    Number of ports: 2
    Firmware version: 2.40.7000
    Hardware version: 1
    Node GUID: 0xe41d2d0300024480
    System image GUID: 0xe41d2d0300024483
    Port 1:
        State: Down
        Physical state: Disabled
        Rate: 10
        Base lid: 0
        LMC: 0
        SM lid: 0
        Capability mask: 0x04010000
        Port GUID: 0xe61d2dfffe024481
        Link layer: Ethernet
    Port 2:
        State: Active
        Physical state: LinkUp
        Rate: 40
        Base lid: 0
        LMC: 0
        SM lid: 0
        Capability mask: 0x04010000
        Port GUID: 0xe61d2dfffe024482
        Link layer: Ethernet

I also change the default value from n to  CONFIG_RTE_LIBRTE_MLX4_PMD=y.
Recompiled DPDK and it all worked.

And Below is my ifconfig info:

ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)


Since I assume that I don't have to bind the device manually, I executed
./dpdk-devbind.py --status with the following info popped out:

Network devices using kernel driver
===================================
0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
drv=ixgbe unused=
0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
drv=ixgbe unused=
0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
drv=mlx4_core unused=
0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb unused=
0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb unused=
*Active*


With these configurations, I run commands in order but still had the same
result:

$test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
$test: /etc/init.d/openibd restart
$test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1 --proc-type
auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P -m
"[1-5.0]"


>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf cache
1024
!PANIC!: *** Did not find any ports to use ***
PANIC in pktgen_config_ports():
*** Did not find any ports to use ***6:
[./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
[0x447239]]
5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
[0x7f0ec0931830]]
4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
[0x46e324]]
2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3) [0x43ead4]]
1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b) [0x4b7ecb]]
Aborted (core dumped)



Is there anything I've done wrong here?


Thanks


P.T


> You can use  -w to specify ConnectX-3 PCI address, you don't need to use -b
>
> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
>
> Best Regards,
> Olga
>
>
>

-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-08-01  3:38   ` Paul Tsvika
@ 2017-08-01  4:15     ` zhilong zheng
  2017-08-01  8:15       ` Paul Tsvika
  0 siblings, 1 reply; 17+ messages in thread
From: zhilong zheng @ 2017-08-01  4:15 UTC (permalink / raw)
  To: Paul Tsvika; +Cc: users


Hi,

After enabling the MLX4 flag and recompiling the DPDK source code, Do you recompile the code of pktgen? I think you can recompile it by running make clean and make.

Regards,
Zhilong

> 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb@gmail.com> 写道：
> 
> Hi Olga,
> 
> 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas@mellanox.com>:
> 
>> Hi Paul,
>> 
>> Looks like your ConnectX-3 configured to be IB and not ETH
>> 
>>> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
>> 
>> Use connectx_port_config script.
>> Set all the ports to Ethernet, and follow the instructions on the screen.
>> 
> 
> Thanks. I followed the script and configured  both ports to Ethernet.
> 
> ibstat info below:
> 
> CA 'mlx4_0'
>    CA type: MT4099
>    Number of ports: 2
>    Firmware version: 2.40.7000
>    Hardware version: 1
>    Node GUID: 0xe41d2d0300024480
>    System image GUID: 0xe41d2d0300024483
>    Port 1:
>        State: Down
>        Physical state: Disabled
>        Rate: 10
>        Base lid: 0
>        LMC: 0
>        SM lid: 0
>        Capability mask: 0x04010000
>        Port GUID: 0xe61d2dfffe024481
>        Link layer: Ethernet
>    Port 2:
>        State: Active
>        Physical state: LinkUp
>        Rate: 40
>        Base lid: 0
>        LMC: 0
>        SM lid: 0
>        Capability mask: 0x04010000
>        Port GUID: 0xe61d2dfffe024482
>        Link layer: Ethernet
> 
> I also change the default value from n to  CONFIG_RTE_LIBRTE_MLX4_PMD=y.
> Recompiled DPDK and it all worked.
> 
> And Below is my ifconfig info:
> 
> ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
>          UP BROADCAST MULTICAST  MTU:1500  Metric:1
>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>          collisions:0 txqueuelen:1000
>          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
> 
> ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
>          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
>          collisions:0 txqueuelen:1000
>          RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)
> 
> 
> Since I assume that I don't have to bind the device manually, I executed
> ./dpdk-devbind.py --status with the following info popped out:
> 
> Network devices using kernel driver
> ===================================
> 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
> drv=ixgbe unused=
> 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
> drv=ixgbe unused=
> 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
> drv=mlx4_core unused=
> 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb unused=
> 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb unused=
> *Active*
> 
> 
> With these configurations, I run commands in order but still had the same
> result:
> 
> $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
> $test: /etc/init.d/openibd restart
> $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1 --proc-type
> auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P -m
> "[1-5.0]"
> 
> 
>>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf cache
> 1024
> !PANIC!: *** Did not find any ports to use ***
> PANIC in pktgen_config_ports():
> *** Did not find any ports to use ***6:
> [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
> [0x447239]]
> 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
> [0x7f0ec0931830]]
> 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
> 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
> [0x46e324]]
> 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3) [0x43ead4]]
> 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b) [0x4b7ecb]]
> Aborted (core dumped)
> 
> 
> 
> Is there anything I've done wrong here?
> 
> 
> Thanks
> 
> 
> P.T
> 
> 
>> You can use  -w to specify ConnectX-3 PCI address, you don't need to use -b
>> 
>> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
>> 
>> Best Regards,
>> Olga
>> 
>> 
>> 
> 
> -- 
> P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-08-01  4:15     ` zhilong zheng
@ 2017-08-01  8:15       ` Paul Tsvika
  2017-08-01  8:53         ` Zhilong Zheng
  2017-08-01 14:05         ` Wiles, Keith
  0 siblings, 2 replies; 17+ messages in thread
From: Paul Tsvika @ 2017-08-01  8:15 UTC (permalink / raw)
  To: zhilong zheng; +Cc: users

Hi zheng,

Thanks for the feedback.

You are right. To get pktgen work with mlx4, it needs to recompile DPDK and
pktgen as well.

I am able to run pktgen now.

By following the document from DPDK website for this NIC,
I did and change the following variables

I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
set l*og_num_mgm_entry_size=-7*

Disable pause frames:
*ethtool -A ens2d1 rx off tx 0ff*

I did not change the NIC register ( *setpci* ) at this time.


and then I run the commands below in order:


*test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
*test$: /etc/init.d/openibd restart*
*test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
--proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
05:00.1 -- -P -m "[1-5.0]"*

| Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel Corporation
  Flags:Port      :   P--------------:0
Link State        :          <--Down-->     ----TotalRate----
Pkts/s Max/Rx     :                 0/0                   0/0
       Max/Tx     :       844160/732319         844160/732319
MBits/s Rx/Tx     :               0/*492 *                0/*492*
Broadcast         :                   0
Multicast         :                   0
  64 Bytes        :                   0
  65-127          :                   0
  128-255         :                   0
  256-511         :                   0
  512-1023        :                   0
  1024-1518       :                   0
Runts/Jumbos      :                 0/0
Errors Rx/Tx      :                 0/0
Total Rx Pkts     :                   0
      Tx Pkts     :            59289215
      Rx MBs      :                   0
      Tx MBs      :               39842
ARP/ICMP Pkts     :                 0/0
Tx Overrun        :                   0
Cycles per Tx     :       14880952/3200
Missed Rx         :                   0
mcasts Rx         :                None
No Mbuf Rx        :                   0
                  :
Pattern Type      :             abcd...
Tx Count/% Rate   :       Forever /100%
PktSize/Tx Burst  :           64 /   32
Src/Dest Port     :         1234 / 5678
Pkt Type:VLAN ID  :     IPv4 / TCP:0001
Dst  IP Address   :         192.168.1.1
Src  IP Address   :      192.168.0.1/24
Src MAC Address   :   e4:1d:2d:02:44:81
Powered by DPDK --------------------------

The transmitting rate should be around 40000 since it is 40G NIC.
However, the rate is much lower than it and it is probably related to
configurations.

I actually have two mother boards and each one has 40G NICs installed.
My intention is to do the tx / rx experiment.
My first step is to check tx rate can reach as close to 40000 as possible.

Anyone has related experience please advise.



P.T


2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:

>
> Hi,
>
> After enabling the MLX4 flag and recompiling the DPDK source code, Do you
> recompile the code of pktgen? I think you can recompile it by running make
> clean and make.
>
> Regards,
> Zhilong
>
> > 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb@gmail.com> 写道：
> >
> > Hi Olga,
> >
> > 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas@mellanox.com>:
> >
> >> Hi Paul,
> >>
> >> Looks like your ConnectX-3 configured to be IB and not ETH
> >>
> >>> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
> >>
> >> Use connectx_port_config script.
> >> Set all the ports to Ethernet, and follow the instructions on the
> screen.
> >>
> >
> > Thanks. I followed the script and configured  both ports to Ethernet.
> >
> > ibstat info below:
> >
> > CA 'mlx4_0'
> >    CA type: MT4099
> >    Number of ports: 2
> >    Firmware version: 2.40.7000
> >    Hardware version: 1
> >    Node GUID: 0xe41d2d0300024480
> >    System image GUID: 0xe41d2d0300024483
> >    Port 1:
> >        State: Down
> >        Physical state: Disabled
> >        Rate: 10
> >        Base lid: 0
> >        LMC: 0
> >        SM lid: 0
> >        Capability mask: 0x04010000
> >        Port GUID: 0xe61d2dfffe024481
> >        Link layer: Ethernet
> >    Port 2:
> >        State: Active
> >        Physical state: LinkUp
> >        Rate: 40
> >        Base lid: 0
> >        LMC: 0
> >        SM lid: 0
> >        Capability mask: 0x04010000
> >        Port GUID: 0xe61d2dfffe024482
> >        Link layer: Ethernet
> >
> > I also change the default value from n to  CONFIG_RTE_LIBRTE_MLX4_PMD=y.
> > Recompiled DPDK and it all worked.
> >
> > And Below is my ifconfig info:
> >
> > ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
> >          UP BROADCAST MULTICAST  MTU:1500  Metric:1
> >          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >          collisions:0 txqueuelen:1000
> >          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
> >
> > ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
> >          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
> >          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
> >          collisions:0 txqueuelen:1000
> >          RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)
> >
> >
> > Since I assume that I don't have to bind the device manually, I executed
> > ./dpdk-devbind.py --status with the following info popped out:
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
> > drv=ixgbe unused=
> > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
> > drv=ixgbe unused=
> > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
> > drv=mlx4_core unused=
> > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
> unused=
> > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
> unused=
> > *Active*
> >
> >
> > With these configurations, I run commands in order but still had the same
> > result:
> >
> > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
> > $test: /etc/init.d/openibd restart
> > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
> --proc-type
> > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P
> -m
> > "[1-5.0]"
> >
> >
> >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
> cache
> > 1024
> > !PANIC!: *** Did not find any ports to use ***
> > PANIC in pktgen_config_ports():
> > *** Did not find any ports to use ***6:
> > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
> > [0x447239]]
> > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
> > [0x7f0ec0931830]]
> > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
> > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
> > [0x46e324]]
> > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
> [0x43ead4]]
> > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
> [0x4b7ecb]]
> > Aborted (core dumped)
> >
> >
> >
> > Is there anything I've done wrong here?
> >
> >
> > Thanks
> >
> >
> > P.T
> >
> >
> >> You can use  -w to specify ConnectX-3 PCI address, you don't need to
> use -b
> >>
> >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
> >>
> >> Best Regards,
> >> Olga
> >>
> >>
> >>
> >
> > --
> > P.T
>



-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-08-01  8:15       ` Paul Tsvika
@ 2017-08-01  8:53         ` Zhilong Zheng
  2017-08-01  9:04           ` Paul Tsvika
  2017-08-01 14:05         ` Wiles, Keith
  1 sibling, 1 reply; 17+ messages in thread
From: Zhilong Zheng @ 2017-08-01  8:53 UTC (permalink / raw)
  To: Paul Tsvika; +Cc: users

Hi Paul,

You can refer to the website (https://community.mellanox.
com/docs/DOC-2489#jive_content_id_Getting_started) to tune performance. By
the way, it looks like that you can not get ~40Gbps throughputs when
generating 64B packets as @Adrien mentioned (Actually I am also confused
about this problem before, http://dpdk.org/ml/archives/
users/2017-July/002226.html).

Hope it can help you.

Regards,

Zhilong


在 2017年8月1日，16:15，Paul Tsvika <mozloverinweb@gmail.com> 写道：

Hi zheng,

Thanks for the feedback.

You are right. To get pktgen work with mlx4, it needs to recompile DPDK and
pktgen as well.

I am able to run pktgen now.

By following the document from DPDK website for this NIC,
I did and change the following variables

I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
set l*og_num_mgm_entry_size=-7*

Disable pause frames:
*ethtool -A ens2d1 rx off tx 0ff*

I did not change the NIC register ( *setpci* ) at this time.


and then I run the commands below in order:


*test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
*test$: /etc/init.d/openibd restart*
*test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
--proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
05:00.1 -- -P -m "[1-5.0]"*

| Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel Corporation
  Flags:Port      :   P--------------:0
Link State        :          <--Down-->     ----TotalRate----
Pkts/s Max/Rx     :                 0/0                   0/0
       Max/Tx     :       844160/732319         844160/732319
MBits/s Rx/Tx     :               0/*492 *                0/*492*
Broadcast         :                   0
Multicast         :                   0
  64 Bytes        :                   0
  65-127          :                   0
  128-255         :                   0
  256-511         :                   0
  512-1023        :                   0
  1024-1518       :                   0
Runts/Jumbos      :                 0/0
Errors Rx/Tx      :                 0/0
Total Rx Pkts     :                   0
      Tx Pkts     :            59289215
      Rx MBs      :                   0
      Tx MBs      :               39842
ARP/ICMP Pkts     :                 0/0
Tx Overrun        :                   0
Cycles per Tx     :       14880952/3200
Missed Rx         :                   0
mcasts Rx         :                None
No Mbuf Rx        :                   0
                  :
Pattern Type      :             abcd...
Tx Count/% Rate   :       Forever /100%
PktSize/Tx Burst  :           64 /   32
Src/Dest Port     :         1234 / 5678
Pkt Type:VLAN ID  :     IPv4 / TCP:0001
Dst  IP Address   :         192.168.1.1
Src  IP Address   :      192.168.0.1/24
Src MAC Address   :   e4:1d:2d:02:44:81
Powered by DPDK --------------------------

The transmitting rate should be around 40000 since it is 40G NIC.
However, the rate is much lower than it and it is probably related to
configurations.

I actually have two mother boards and each one has 40G NICs installed.
My intention is to do the tx / rx experiment.
My first step is to check tx rate can reach as close to 40000 as possible.

Anyone has related experience please advise.



P.T


2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:

>
> Hi,
>
> After enabling the MLX4 flag and recompiling the DPDK source code, Do you
> recompile the code of pktgen? I think you can recompile it by running make
> clean and make.
>
> Regards,
> Zhilong
>
> > 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb@gmail.com> 写道：
> >
> > Hi Olga,
> >
> > 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas@mellanox.com>:
> >
> >> Hi Paul,
> >>
> >> Looks like your ConnectX-3 configured to be IB and not ETH
> >>
> >>> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
> >>
> >> Use connectx_port_config script.
> >> Set all the ports to Ethernet, and follow the instructions on the
> screen.
> >>
> >
> > Thanks. I followed the script and configured  both ports to Ethernet.
> >
> > ibstat info below:
> >
> > CA 'mlx4_0'
> >    CA type: MT4099
> >    Number of ports: 2
> >    Firmware version: 2.40.7000
> >    Hardware version: 1
> >    Node GUID: 0xe41d2d0300024480
> >    System image GUID: 0xe41d2d0300024483
> >    Port 1:
> >        State: Down
> >        Physical state: Disabled
> >        Rate: 10
> >        Base lid: 0
> >        LMC: 0
> >        SM lid: 0
> >        Capability mask: 0x04010000
> >        Port GUID: 0xe61d2dfffe024481
> >        Link layer: Ethernet
> >    Port 2:
> >        State: Active
> >        Physical state: LinkUp
> >        Rate: 40
> >        Base lid: 0
> >        LMC: 0
> >        SM lid: 0
> >        Capability mask: 0x04010000
> >        Port GUID: 0xe61d2dfffe024482
> >        Link layer: Ethernet
> >
> > I also change the default value from n to  CONFIG_RTE_LIBRTE_MLX4_PMD=y.
> > Recompiled DPDK and it all worked.
> >
> > And Below is my ifconfig info:
> >
> > ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
> >          UP BROADCAST MULTICAST  MTU:1500  Metric:1
> >          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >          collisions:0 txqueuelen:1000
> >          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
> >
> > ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
> >          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
> >          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
> >          collisions:0 txqueuelen:1000
> >          RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)
> >
> >
> > Since I assume that I don't have to bind the device manually, I executed
> > ./dpdk-devbind.py --status with the following info popped out:
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
> > drv=ixgbe unused=
> > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
> > drv=ixgbe unused=
> > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
> > drv=mlx4_core unused=
> > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
> unused=
> > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
> unused=
> > *Active*
> >
> >
> > With these configurations, I run commands in order but still had the same
> > result:
> >
> > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
> > $test: /etc/init.d/openibd restart
> > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
> --proc-type
> > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P
> -m
> > "[1-5.0]"
> >
> >
> >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
> cache
> > 1024
> > !PANIC!: *** Did not find any ports to use ***
> > PANIC in pktgen_config_ports():
> > *** Did not find any ports to use ***6:
> > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
> > [0x447239]]
> > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
> > [0x7f0ec0931830]]
> > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
> > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
> > [0x46e324]]
> > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
> [0x43ead4]]
> > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
> [0x4b7ecb]]
> > Aborted (core dumped)
> >
> >
> >
> > Is there anything I've done wrong here?
> >
> >
> > Thanks
> >
> >
> > P.T
> >
> >
> >> You can use  -w to specify ConnectX-3 PCI address, you don't need to
> use -b
> >>
> >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
> >>
> >> Best Regards,
> >> Olga
> >>
> >>
> >>
> >
> > --
> > P.T
>



-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-08-01  8:53         ` Zhilong Zheng
@ 2017-08-01  9:04           ` Paul Tsvika
  2017-08-01  9:12             ` Zhilong Zheng
  0 siblings, 1 reply; 17+ messages in thread
From: Paul Tsvika @ 2017-08-01  9:04 UTC (permalink / raw)
  To: Zhilong Zheng; +Cc: users

Hi Zheng,

What throughput you had when you tried 128 or 256B packets?


P.T

2017-08-01 16:53 GMT+08:00 Zhilong Zheng <zhengzl0715@gmail.com>:

> Hi Paul,
>
> You can refer to the website (https://community.mellanox.co
> m/docs/DOC-2489#jive_content_id_Getting_started) to tune performance. By
> the way, it looks like that you can not get ~40Gbps throughputs when
> generating 64B packets as @Adrien mentioned (Actually I am also confused
> about this problem before, http://dpdk.org/ml/archives/us
> ers/2017-July/002226.html).
>
> Hope it can help you.
>
> Regards,
>
> Zhilong
>
>
> 在 2017年8月1日，16:15，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>
> Hi zheng,
>
> Thanks for the feedback.
>
> You are right. To get pktgen work with mlx4, it needs to recompile DPDK
> and pktgen as well.
>
> I am able to run pktgen now.
>
> By following the document from DPDK website for this NIC,
> I did and change the following variables
>
> I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
> set l*og_num_mgm_entry_size=-7*
>
> Disable pause frames:
> *ethtool -A ens2d1 rx off tx 0ff*
>
> I did not change the NIC register ( *setpci* ) at this time.
>
>
> and then I run the commands below in order:
>
>
> *test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
> *test$: /etc/init.d/openibd restart*
> *test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
> --proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
> 05:00.1 -- -P -m "[1-5.0]"*
>
> | Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel
> Corporation
>   Flags:Port      :   P--------------:0
> Link State        :          <--Down-->     ----TotalRate----
> Pkts/s Max/Rx     :                 0/0                   0/0
>        Max/Tx     :       844160/732319         844160/732319
> MBits/s Rx/Tx     :               0/*492 *                0/*492*
> Broadcast         :                   0
> Multicast         :                   0
>   64 Bytes        :                   0
>   65-127          :                   0
>   128-255         :                   0
>   256-511         :                   0
>   512-1023        :                   0
>   1024-1518       :                   0
> Runts/Jumbos      :                 0/0
> Errors Rx/Tx      :                 0/0
> Total Rx Pkts     :                   0
>       Tx Pkts     :            59289215
>       Rx MBs      :                   0
>       Tx MBs      :               39842
> ARP/ICMP Pkts     :                 0/0
> Tx Overrun        :                   0
> Cycles per Tx     :       14880952/3200
> Missed Rx         :                   0
> mcasts Rx         :                None
> No Mbuf Rx        :                   0
>                   :
> Pattern Type      :             abcd...
> Tx Count/% Rate   :       Forever /100%
> PktSize/Tx Burst  :           64 /   32
> Src/Dest Port     :         1234 / 5678
> Pkt Type:VLAN ID  :     IPv4 / TCP:0001
> Dst  IP Address   :         192.168.1.1
> Src  IP Address   :      192.168.0.1/24
> Src MAC Address   :   e4:1d:2d:02:44:81
> Powered by DPDK --------------------------
>
> The transmitting rate should be around 40000 since it is 40G NIC.
> However, the rate is much lower than it and it is probably related to
> configurations.
>
> I actually have two mother boards and each one has 40G NICs installed.
> My intention is to do the tx / rx experiment.
> My first step is to check tx rate can reach as close to 40000 as possible.
>
> Anyone has related experience please advise.
>
>
>
> P.T
>
>
> 2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:
>
>>
>> Hi,
>>
>> After enabling the MLX4 flag and recompiling the DPDK source code, Do you
>> recompile the code of pktgen? I think you can recompile it by running make
>> clean and make.
>>
>> Regards,
>> Zhilong
>>
>> > 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>> >
>> > Hi Olga,
>> >
>> > 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas@mellanox.com>:
>> >
>> >> Hi Paul,
>> >>
>> >> Looks like your ConnectX-3 configured to be IB and not ETH
>> >>
>> >>> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
>> >>
>> >> Use connectx_port_config script.
>> >> Set all the ports to Ethernet, and follow the instructions on the
>> screen.
>> >>
>> >
>> > Thanks. I followed the script and configured  both ports to Ethernet.
>> >
>> > ibstat info below:
>> >
>> > CA 'mlx4_0'
>> >    CA type: MT4099
>> >    Number of ports: 2
>> >    Firmware version: 2.40.7000
>> >    Hardware version: 1
>> >    Node GUID: 0xe41d2d0300024480
>> >    System image GUID: 0xe41d2d0300024483
>> >    Port 1:
>> >        State: Down
>> >        Physical state: Disabled
>> >        Rate: 10
>> >        Base lid: 0
>> >        LMC: 0
>> >        SM lid: 0
>> >        Capability mask: 0x04010000
>> >        Port GUID: 0xe61d2dfffe024481
>> >        Link layer: Ethernet
>> >    Port 2:
>> >        State: Active
>> >        Physical state: LinkUp
>> >        Rate: 40
>> >        Base lid: 0
>> >        LMC: 0
>> >        SM lid: 0
>> >        Capability mask: 0x04010000
>> >        Port GUID: 0xe61d2dfffe024482
>> >        Link layer: Ethernet
>> >
>> > I also change the default value from n to  CONFIG_RTE_LIBRTE_MLX4_PMD=y.
>> > Recompiled DPDK and it all worked.
>> >
>> > And Below is my ifconfig info:
>> >
>> > ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
>> >          UP BROADCAST MULTICAST  MTU:1500  Metric:1
>> >          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>> >          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>> >          collisions:0 txqueuelen:1000
>> >          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>> >
>> > ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
>> >          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>> >          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
>> >          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
>> >          collisions:0 txqueuelen:1000
>> >          RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)
>> >
>> >
>> > Since I assume that I don't have to bind the device manually, I executed
>> > ./dpdk-devbind.py --status with the following info popped out:
>> >
>> > Network devices using kernel driver
>> > ===================================
>> > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
>> > drv=ixgbe unused=
>> > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
>> > drv=ixgbe unused=
>> > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
>> > drv=mlx4_core unused=
>> > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
>> unused=
>> > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
>> unused=
>> > *Active*
>> >
>> >
>> > With these configurations, I run commands in order but still had the
>> same
>> > result:
>> >
>> > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
>> > $test: /etc/init.d/openibd restart
>> > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>> --proc-type
>> > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P
>> -m
>> > "[1-5.0]"
>> >
>> >
>> >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
>> cache
>> > 1024
>> > !PANIC!: *** Did not find any ports to use ***
>> > PANIC in pktgen_config_ports():
>> > *** Did not find any ports to use ***6:
>> > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
>> > [0x447239]]
>> > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
>> > [0x7f0ec0931830]]
>> > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
>> > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
>> > [0x46e324]]
>> > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
>> [0x43ead4]]
>> > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
>> [0x4b7ecb]]
>> > Aborted (core dumped)
>> >
>> >
>> >
>> > Is there anything I've done wrong here?
>> >
>> >
>> > Thanks
>> >
>> >
>> > P.T
>> >
>> >
>> >> You can use  -w to specify ConnectX-3 PCI address, you don't need to
>> use -b
>> >>
>> >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
>> >>
>> >> Best Regards,
>> >> Olga
>> >>
>> >>
>> >>
>> >
>> > --
>> > P.T
>>
>
>
>
> --
> P.T
>
>
>


-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-08-01  9:04           ` Paul Tsvika
@ 2017-08-01  9:12             ` Zhilong Zheng
  2017-08-01  9:37               ` Paul Tsvika
  0 siblings, 1 reply; 17+ messages in thread
From: Zhilong Zheng @ 2017-08-01  9:12 UTC (permalink / raw)
  To: Paul Tsvika; +Cc: users

~35Gbps and ~40Gbps when setting to 128 and 256. I use 4 cores to tx
packets.

在 2017年8月1日，17:04，Paul Tsvika <mozloverinweb@gmail.com> 写道：

Hi Zheng,

What throughput you had when you tried 128 or 256B packets?


P.T

2017-08-01 16:53 GMT+08:00 Zhilong Zheng <zhengzl0715@gmail.com>:

> Hi Paul,
>
> You can refer to the website (https://community.mellanox.co
> m/docs/DOC-2489#jive_content_id_Getting_started) to tune performance. By
> the way, it looks like that you can not get ~40Gbps throughputs when
> generating 64B packets as @Adrien mentioned (Actually I am also confused
> about this problem before, http://dpdk.org/ml/archives/us
> ers/2017-July/002226.html).
>
> Hope it can help you.
>
> Regards,
>
> Zhilong
>
>
> 在 2017年8月1日，16:15，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>
> Hi zheng,
>
> Thanks for the feedback.
>
> You are right. To get pktgen work with mlx4, it needs to recompile DPDK
> and pktgen as well.
>
> I am able to run pktgen now.
>
> By following the document from DPDK website for this NIC,
> I did and change the following variables
>
> I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
> set l*og_num_mgm_entry_size=-7*
>
> Disable pause frames:
> *ethtool -A ens2d1 rx off tx 0ff*
>
> I did not change the NIC register ( *setpci* ) at this time.
>
>
> and then I run the commands below in order:
>
>
> *test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
> *test$: /etc/init.d/openibd restart*
> *test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
> --proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
> 05:00.1 -- -P -m "[1-5.0]"*
>
> | Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel
> Corporation
>   Flags:Port      :   P--------------:0
> Link State        :          <--Down-->     ----TotalRate----
> Pkts/s Max/Rx     :                 0/0                   0/0
>        Max/Tx     :       844160/732319         844160/732319
> MBits/s Rx/Tx     :               0/*492 *                0/*492*
> Broadcast         :                   0
> Multicast         :                   0
>   64 Bytes        :                   0
>   65-127          :                   0
>   128-255         :                   0
>   256-511         :                   0
>   512-1023        :                   0
>   1024-1518       :                   0
> Runts/Jumbos      :                 0/0
> Errors Rx/Tx      :                 0/0
> Total Rx Pkts     :                   0
>       Tx Pkts     :            59289215
>       Rx MBs      :                   0
>       Tx MBs      :               39842
> ARP/ICMP Pkts     :                 0/0
> Tx Overrun        :                   0
> Cycles per Tx     :       14880952/3200
> Missed Rx         :                   0
> mcasts Rx         :                None
> No Mbuf Rx        :                   0
>                   :
> Pattern Type      :             abcd...
> Tx Count/% Rate   :       Forever /100%
> PktSize/Tx Burst  :           64 /   32
> Src/Dest Port     :         1234 / 5678
> Pkt Type:VLAN ID  :     IPv4 / TCP:0001
> Dst  IP Address   :         192.168.1.1
> Src  IP Address   :      192.168.0.1/24
> Src MAC Address   :   e4:1d:2d:02:44:81
> Powered by DPDK --------------------------
>
> The transmitting rate should be around 40000 since it is 40G NIC.
> However, the rate is much lower than it and it is probably related to
> configurations.
>
> I actually have two mother boards and each one has 40G NICs installed.
> My intention is to do the tx / rx experiment.
> My first step is to check tx rate can reach as close to 40000 as possible.
>
> Anyone has related experience please advise.
>
>
>
> P.T
>
>
> 2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:
>
>>
>> Hi,
>>
>> After enabling the MLX4 flag and recompiling the DPDK source code, Do you
>> recompile the code of pktgen? I think you can recompile it by running make
>> clean and make.
>>
>> Regards,
>> Zhilong
>>
>> > 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>> >
>> > Hi Olga,
>> >
>> > 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas@mellanox.com>:
>> >
>> >> Hi Paul,
>> >>
>> >> Looks like your ConnectX-3 configured to be IB and not ETH
>> >>
>> >>> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
>> >>
>> >> Use connectx_port_config script.
>> >> Set all the ports to Ethernet, and follow the instructions on the
>> screen.
>> >>
>> >
>> > Thanks. I followed the script and configured  both ports to Ethernet.
>> >
>> > ibstat info below:
>> >
>> > CA 'mlx4_0'
>> >    CA type: MT4099
>> >    Number of ports: 2
>> >    Firmware version: 2.40.7000
>> >    Hardware version: 1
>> >    Node GUID: 0xe41d2d0300024480
>> >    System image GUID: 0xe41d2d0300024483
>> >    Port 1:
>> >        State: Down
>> >        Physical state: Disabled
>> >        Rate: 10
>> >        Base lid: 0
>> >        LMC: 0
>> >        SM lid: 0
>> >        Capability mask: 0x04010000
>> >        Port GUID: 0xe61d2dfffe024481
>> >        Link layer: Ethernet
>> >    Port 2:
>> >        State: Active
>> >        Physical state: LinkUp
>> >        Rate: 40
>> >        Base lid: 0
>> >        LMC: 0
>> >        SM lid: 0
>> >        Capability mask: 0x04010000
>> >        Port GUID: 0xe61d2dfffe024482
>> >        Link layer: Ethernet
>> >
>> > I also change the default value from n to  CONFIG_RTE_LIBRTE_MLX4_PMD=y.
>> > Recompiled DPDK and it all worked.
>> >
>> > And Below is my ifconfig info:
>> >
>> > ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
>> >          UP BROADCAST MULTICAST  MTU:1500  Metric:1
>> >          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>> >          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>> >          collisions:0 txqueuelen:1000
>> >          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>> >
>> > ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
>> >          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>> >          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
>> >          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
>> >          collisions:0 txqueuelen:1000
>> >          RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)
>> >
>> >
>> > Since I assume that I don't have to bind the device manually, I executed
>> > ./dpdk-devbind.py --status with the following info popped out:
>> >
>> > Network devices using kernel driver
>> > ===================================
>> > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
>> > drv=ixgbe unused=
>> > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
>> > drv=ixgbe unused=
>> > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
>> > drv=mlx4_core unused=
>> > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
>> unused=
>> > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
>> unused=
>> > *Active*
>> >
>> >
>> > With these configurations, I run commands in order but still had the
>> same
>> > result:
>> >
>> > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
>> > $test: /etc/init.d/openibd restart
>> > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>> --proc-type
>> > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P
>> -m
>> > "[1-5.0]"
>> >
>> >
>> >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
>> cache
>> > 1024
>> > !PANIC!: *** Did not find any ports to use ***
>> > PANIC in pktgen_config_ports():
>> > *** Did not find any ports to use ***6:
>> > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
>> > [0x447239]]
>> > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
>> > [0x7f0ec0931830]]
>> > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
>> > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
>> > [0x46e324]]
>> > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
>> [0x43ead4]]
>> > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
>> [0x4b7ecb]]
>> > Aborted (core dumped)
>> >
>> >
>> >
>> > Is there anything I've done wrong here?
>> >
>> >
>> > Thanks
>> >
>> >
>> > P.T
>> >
>> >
>> >> You can use  -w to specify ConnectX-3 PCI address, you don't need to
>> use -b
>> >>
>> >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
>> >>
>> >> Best Regards,
>> >> Olga
>> >>
>> >>
>> >>
>> >
>> > --
>> > P.T
>>
>
>
>
> --
> P.T
>
>
>


-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-08-01  9:12             ` Zhilong Zheng
@ 2017-08-01  9:37               ` Paul Tsvika
  2017-08-01  9:49                 ` Zhilong Zheng
  0 siblings, 1 reply; 17+ messages in thread
From: Paul Tsvika @ 2017-08-01  9:37 UTC (permalink / raw)
  To: Zhilong Zheng; +Cc: users

I found the issue with the output log when executed pktgen.

| Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel Corporation
  Flags:Port      :   P--------------:0
Link State        :      *    <--Down--> *    ----TotalRate----
Pkts/s Max/Rx     :                 0/0                   0/0


The link is down. This looks not right.


Do you have any idea to check this?




P.T

2017-08-01 17:12 GMT+08:00 Zhilong Zheng <zhengzl0715@gmail.com>:

> ~35Gbps and ~40Gbps when setting to 128 and 256. I use 4 cores to tx
> packets.
>
> 在 2017年8月1日，17:04，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>
> Hi Zheng,
>
> What throughput you had when you tried 128 or 256B packets?
>
>
> P.T
>
> 2017-08-01 16:53 GMT+08:00 Zhilong Zheng <zhengzl0715@gmail.com>:
>
>> Hi Paul,
>>
>> You can refer to the website (https://community.mellanox.co
>> m/docs/DOC-2489#jive_content_id_Getting_started) to tune performance. By
>> the way, it looks like that you can not get ~40Gbps throughputs when
>> generating 64B packets as @Adrien mentioned (Actually I am also confused
>> about this problem before, http://dpdk.org/ml/archives/us
>> ers/2017-July/002226.html).
>>
>> Hope it can help you.
>>
>> Regards,
>>
>> Zhilong
>>
>>
>> 在 2017年8月1日，16:15，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>>
>> Hi zheng,
>>
>> Thanks for the feedback.
>>
>> You are right. To get pktgen work with mlx4, it needs to recompile DPDK
>> and pktgen as well.
>>
>> I am able to run pktgen now.
>>
>> By following the document from DPDK website for this NIC,
>> I did and change the following variables
>>
>> I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
>> set l*og_num_mgm_entry_size=-7*
>>
>> Disable pause frames:
>> *ethtool -A ens2d1 rx off tx 0ff*
>>
>> I did not change the NIC register ( *setpci* ) at this time.
>>
>>
>> and then I run the commands below in order:
>>
>>
>> *test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
>> *test$: /etc/init.d/openibd restart*
>> *test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>> --proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
>> 05:00.1 -- -P -m "[1-5.0]"*
>>
>> | Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel
>> Corporation
>>   Flags:Port      :   P--------------:0
>> Link State        :          <--Down-->     ----TotalRate----
>> Pkts/s Max/Rx     :                 0/0                   0/0
>>        Max/Tx     :       844160/732319         844160/732319
>> MBits/s Rx/Tx     :               0/*492 *                0/*492*
>> Broadcast         :                   0
>> Multicast         :                   0
>>   64 Bytes        :                   0
>>   65-127          :                   0
>>   128-255         :                   0
>>   256-511         :                   0
>>   512-1023        :                   0
>>   1024-1518       :                   0
>> Runts/Jumbos      :                 0/0
>> Errors Rx/Tx      :                 0/0
>> Total Rx Pkts     :                   0
>>       Tx Pkts     :            59289215
>>       Rx MBs      :                   0
>>       Tx MBs      :               39842
>> ARP/ICMP Pkts     :                 0/0
>> Tx Overrun        :                   0
>> Cycles per Tx     :       14880952/3200
>> Missed Rx         :                   0
>> mcasts Rx         :                None
>> No Mbuf Rx        :                   0
>>                   :
>> Pattern Type      :             abcd...
>> Tx Count/% Rate   :       Forever /100%
>> PktSize/Tx Burst  :           64 /   32
>> Src/Dest Port     :         1234 / 5678
>> Pkt Type:VLAN ID  :     IPv4 / TCP:0001
>> Dst  IP Address   :         192.168.1.1
>> Src  IP Address   :      192.168.0.1/24
>> Src MAC Address   :   e4:1d:2d:02:44:81
>> Powered by DPDK --------------------------
>>
>> The transmitting rate should be around 40000 since it is 40G NIC.
>> However, the rate is much lower than it and it is probably related to
>> configurations.
>>
>> I actually have two mother boards and each one has 40G NICs installed.
>> My intention is to do the tx / rx experiment.
>> My first step is to check tx rate can reach as close to 40000 as possible.
>>
>> Anyone has related experience please advise.
>>
>>
>>
>> P.T
>>
>>
>> 2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:
>>
>>>
>>> Hi,
>>>
>>> After enabling the MLX4 flag and recompiling the DPDK source code, Do
>>> you recompile the code of pktgen? I think you can recompile it by running
>>> make clean and make.
>>>
>>> Regards,
>>> Zhilong
>>>
>>> > 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>>> >
>>> > Hi Olga,
>>> >
>>> > 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas@mellanox.com>:
>>> >
>>> >> Hi Paul,
>>> >>
>>> >> Looks like your ConnectX-3 configured to be IB and not ETH
>>> >>
>>> >>> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
>>> >>
>>> >> Use connectx_port_config script.
>>> >> Set all the ports to Ethernet, and follow the instructions on the
>>> screen.
>>> >>
>>> >
>>> > Thanks. I followed the script and configured  both ports to Ethernet.
>>> >
>>> > ibstat info below:
>>> >
>>> > CA 'mlx4_0'
>>> >    CA type: MT4099
>>> >    Number of ports: 2
>>> >    Firmware version: 2.40.7000
>>> >    Hardware version: 1
>>> >    Node GUID: 0xe41d2d0300024480
>>> >    System image GUID: 0xe41d2d0300024483
>>> >    Port 1:
>>> >        State: Down
>>> >        Physical state: Disabled
>>> >        Rate: 10
>>> >        Base lid: 0
>>> >        LMC: 0
>>> >        SM lid: 0
>>> >        Capability mask: 0x04010000
>>> >        Port GUID: 0xe61d2dfffe024481
>>> >        Link layer: Ethernet
>>> >    Port 2:
>>> >        State: Active
>>> >        Physical state: LinkUp
>>> >        Rate: 40
>>> >        Base lid: 0
>>> >        LMC: 0
>>> >        SM lid: 0
>>> >        Capability mask: 0x04010000
>>> >        Port GUID: 0xe61d2dfffe024482
>>> >        Link layer: Ethernet
>>> >
>>> > I also change the default value from n to
>>> CONFIG_RTE_LIBRTE_MLX4_PMD=y.
>>> > Recompiled DPDK and it all worked.
>>> >
>>> > And Below is my ifconfig info:
>>> >
>>> > ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
>>> >          UP BROADCAST MULTICAST  MTU:1500  Metric:1
>>> >          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>> >          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> >          collisions:0 txqueuelen:1000
>>> >          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>> >
>>> > ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
>>> >          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>> >          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
>>> >          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
>>> >          collisions:0 txqueuelen:1000
>>> >          RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)
>>> >
>>> >
>>> > Since I assume that I don't have to bind the device manually, I
>>> executed
>>> > ./dpdk-devbind.py --status with the following info popped out:
>>> >
>>> > Network devices using kernel driver
>>> > ===================================
>>> > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
>>> > drv=ixgbe unused=
>>> > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
>>> > drv=ixgbe unused=
>>> > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
>>> > drv=mlx4_core unused=
>>> > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
>>> unused=
>>> > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
>>> unused=
>>> > *Active*
>>> >
>>> >
>>> > With these configurations, I run commands in order but still had the
>>> same
>>> > result:
>>> >
>>> > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
>>> > $test: /etc/init.d/openibd restart
>>> > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>>> --proc-type
>>> > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 --
>>> -P -m
>>> > "[1-5.0]"
>>> >
>>> >
>>> >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
>>> cache
>>> > 1024
>>> > !PANIC!: *** Did not find any ports to use ***
>>> > PANIC in pktgen_config_ports():
>>> > *** Did not find any ports to use ***6:
>>> > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
>>> > [0x447239]]
>>> > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
>>> > [0x7f0ec0931830]]
>>> > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
>>> > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports
>>> +0x1614)
>>> > [0x46e324]]
>>> > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
>>> [0x43ead4]]
>>> > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
>>> [0x4b7ecb]]
>>> > Aborted (core dumped)
>>> >
>>> >
>>> >
>>> > Is there anything I've done wrong here?
>>> >
>>> >
>>> > Thanks
>>> >
>>> >
>>> > P.T
>>> >
>>> >
>>> >> You can use  -w to specify ConnectX-3 PCI address, you don't need to
>>> use -b
>>> >>
>>> >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
>>> >>
>>> >> Best Regards,
>>> >> Olga
>>> >>
>>> >>
>>> >>
>>> >
>>> > --
>>> > P.T
>>>
>>
>>
>>
>> --
>> P.T
>>
>>
>>
>
>
> --
> P.T
>
>
>


-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-08-01  9:37               ` Paul Tsvika
@ 2017-08-01  9:49                 ` Zhilong Zheng
  0 siblings, 0 replies; 17+ messages in thread
From: Zhilong Zheng @ 2017-08-01  9:49 UTC (permalink / raw)
  To: Paul Tsvika; +Cc: users

I think you can check the device status using command "ibstat" on the
sender and receiver. If both are down, it seems you should check the link
connection.

在 2017年8月1日，17:37，Paul Tsvika <mozloverinweb@gmail.com> 写道：

I found the issue with the output log when executed pktgen.

| Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel Corporation
  Flags:Port      :   P--------------:0
Link State        :      *    <--Down--> *    ----TotalRate----
Pkts/s Max/Rx     :                 0/0                   0/0


The link is down. This looks not right.


Do you have any idea to check this?




P.T

2017-08-01 17:12 GMT+08:00 Zhilong Zheng <zhengzl0715@gmail.com>:

> ~35Gbps and ~40Gbps when setting to 128 and 256. I use 4 cores to tx
> packets.
>
> 在 2017年8月1日，17:04，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>
> Hi Zheng,
>
> What throughput you had when you tried 128 or 256B packets?
>
>
> P.T
>
> 2017-08-01 16:53 GMT+08:00 Zhilong Zheng <zhengzl0715@gmail.com>:
>
>> Hi Paul,
>>
>> You can refer to the website (https://community.mellanox.co
>> m/docs/DOC-2489#jive_content_id_Getting_started) to tune performance. By
>> the way, it looks like that you can not get ~40Gbps throughputs when
>> generating 64B packets as @Adrien mentioned (Actually I am also confused
>> about this problem before, http://dpdk.org/ml/archives/us
>> ers/2017-July/002226.html).
>>
>> Hope it can help you.
>>
>> Regards,
>>
>> Zhilong
>>
>>
>> 在 2017年8月1日，16:15，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>>
>> Hi zheng,
>>
>> Thanks for the feedback.
>>
>> You are right. To get pktgen work with mlx4, it needs to recompile DPDK
>> and pktgen as well.
>>
>> I am able to run pktgen now.
>>
>> By following the document from DPDK website for this NIC,
>> I did and change the following variables
>>
>> I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
>> set l*og_num_mgm_entry_size=-7*
>>
>> Disable pause frames:
>> *ethtool -A ens2d1 rx off tx 0ff*
>>
>> I did not change the NIC register ( *setpci* ) at this time.
>>
>>
>> and then I run the commands below in order:
>>
>>
>> *test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
>> *test$: /etc/init.d/openibd restart*
>> *test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>> --proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
>> 05:00.1 -- -P -m "[1-5.0]"*
>>
>> | Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel
>> Corporation
>>   Flags:Port      :   P--------------:0
>> Link State        :          <--Down-->     ----TotalRate----
>> Pkts/s Max/Rx     :                 0/0                   0/0
>>        Max/Tx     :       844160/732319         844160/732319
>> MBits/s Rx/Tx     :               0/*492 *                0/*492*
>> Broadcast         :                   0
>> Multicast         :                   0
>>   64 Bytes        :                   0
>>   65-127          :                   0
>>   128-255         :                   0
>>   256-511         :                   0
>>   512-1023        :                   0
>>   1024-1518       :                   0
>> Runts/Jumbos      :                 0/0
>> Errors Rx/Tx      :                 0/0
>> Total Rx Pkts     :                   0
>>       Tx Pkts     :            59289215
>>       Rx MBs      :                   0
>>       Tx MBs      :               39842
>> ARP/ICMP Pkts     :                 0/0
>> Tx Overrun        :                   0
>> Cycles per Tx     :       14880952/3200
>> Missed Rx         :                   0
>> mcasts Rx         :                None
>> No Mbuf Rx        :                   0
>>                   :
>> Pattern Type      :             abcd...
>> Tx Count/% Rate   :       Forever /100%
>> PktSize/Tx Burst  :           64 /   32
>> Src/Dest Port     :         1234 / 5678
>> Pkt Type:VLAN ID  :     IPv4 / TCP:0001
>> Dst  IP Address   :         192.168.1.1
>> Src  IP Address   :      192.168.0.1/24
>> Src MAC Address   :   e4:1d:2d:02:44:81
>> Powered by DPDK --------------------------
>>
>> The transmitting rate should be around 40000 since it is 40G NIC.
>> However, the rate is much lower than it and it is probably related to
>> configurations.
>>
>> I actually have two mother boards and each one has 40G NICs installed.
>> My intention is to do the tx / rx experiment.
>> My first step is to check tx rate can reach as close to 40000 as possible.
>>
>> Anyone has related experience please advise.
>>
>>
>>
>> P.T
>>
>>
>> 2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:
>>
>>>
>>> Hi,
>>>
>>> After enabling the MLX4 flag and recompiling the DPDK source code, Do
>>> you recompile the code of pktgen? I think you can recompile it by running
>>> make clean and make.
>>>
>>> Regards,
>>> Zhilong
>>>
>>> > 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>>> >
>>> > Hi Olga,
>>> >
>>> > 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas@mellanox.com>:
>>> >
>>> >> Hi Paul,
>>> >>
>>> >> Looks like your ConnectX-3 configured to be IB and not ETH
>>> >>
>>> >>> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
>>> >>
>>> >> Use connectx_port_config script.
>>> >> Set all the ports to Ethernet, and follow the instructions on the
>>> screen.
>>> >>
>>> >
>>> > Thanks. I followed the script and configured  both ports to Ethernet.
>>> >
>>> > ibstat info below:
>>> >
>>> > CA 'mlx4_0'
>>> >    CA type: MT4099
>>> >    Number of ports: 2
>>> >    Firmware version: 2.40.7000
>>> >    Hardware version: 1
>>> >    Node GUID: 0xe41d2d0300024480
>>> >    System image GUID: 0xe41d2d0300024483
>>> >    Port 1:
>>> >        State: Down
>>> >        Physical state: Disabled
>>> >        Rate: 10
>>> >        Base lid: 0
>>> >        LMC: 0
>>> >        SM lid: 0
>>> >        Capability mask: 0x04010000
>>> >        Port GUID: 0xe61d2dfffe024481
>>> >        Link layer: Ethernet
>>> >    Port 2:
>>> >        State: Active
>>> >        Physical state: LinkUp
>>> >        Rate: 40
>>> >        Base lid: 0
>>> >        LMC: 0
>>> >        SM lid: 0
>>> >        Capability mask: 0x04010000
>>> >        Port GUID: 0xe61d2dfffe024482
>>> >        Link layer: Ethernet
>>> >
>>> > I also change the default value from n to
>>> CONFIG_RTE_LIBRTE_MLX4_PMD=y.
>>> > Recompiled DPDK and it all worked.
>>> >
>>> > And Below is my ifconfig info:
>>> >
>>> > ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
>>> >          UP BROADCAST MULTICAST  MTU:1500  Metric:1
>>> >          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>> >          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> >          collisions:0 txqueuelen:1000
>>> >          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>> >
>>> > ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
>>> >          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>> >          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
>>> >          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
>>> >          collisions:0 txqueuelen:1000
>>> >          RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)
>>> >
>>> >
>>> > Since I assume that I don't have to bind the device manually, I
>>> executed
>>> > ./dpdk-devbind.py --status with the following info popped out:
>>> >
>>> > Network devices using kernel driver
>>> > ===================================
>>> > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
>>> > drv=ixgbe unused=
>>> > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
>>> > drv=ixgbe unused=
>>> > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
>>> > drv=mlx4_core unused=
>>> > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
>>> unused=
>>> > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
>>> unused=
>>> > *Active*
>>> >
>>> >
>>> > With these configurations, I run commands in order but still had the
>>> same
>>> > result:
>>> >
>>> > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
>>> > $test: /etc/init.d/openibd restart
>>> > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>>> --proc-type
>>> > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 --
>>> -P -m
>>> > "[1-5.0]"
>>> >
>>> >
>>> >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
>>> cache
>>> > 1024
>>> > !PANIC!: *** Did not find any ports to use ***
>>> > PANIC in pktgen_config_ports():
>>> > *** Did not find any ports to use ***6:
>>> > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
>>> > [0x447239]]
>>> > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
>>> > [0x7f0ec0931830]]
>>> > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
>>> > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports
>>> +0x1614)
>>> > [0x46e324]]
>>> > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
>>> [0x43ead4]]
>>> > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
>>> [0x4b7ecb]]
>>> > Aborted (core dumped)
>>> >
>>> >
>>> >
>>> > Is there anything I've done wrong here?
>>> >
>>> >
>>> > Thanks
>>> >
>>> >
>>> > P.T
>>> >
>>> >
>>> >> You can use  -w to specify ConnectX-3 PCI address, you don't need to
>>> use -b
>>> >>
>>> >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
>>> >>
>>> >> Best Regards,
>>> >> Olga
>>> >>
>>> >>
>>> >>
>>> >
>>> > --
>>> > P.T
>>>
>>
>>
>>
>> --
>> P.T
>>
>>
>>
>
>
> --
> P.T
>
>
>


-- 
P.T

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
  2017-08-01  8:15       ` Paul Tsvika
  2017-08-01  8:53         ` Zhilong Zheng
@ 2017-08-01 14:05         ` Wiles, Keith
  1 sibling, 0 replies; 17+ messages in thread
From: Wiles, Keith @ 2017-08-01 14:05 UTC (permalink / raw)
  To: Paul Tsvika; +Cc: zhilong zheng, users


> On Aug 1, 2017, at 3:15 AM, Paul Tsvika <mozloverinweb@gmail.com> wrote:
> 
> Hi zheng,
> 
> Thanks for the feedback.
> 
> You are right. To get pktgen work with mlx4, it needs to recompile DPDK and
> pktgen as well.
> 
> I am able to run pktgen now.
> 
> By following the document from DPDK website for this NIC,
> I did and change the following variables
> 
> I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
> set l*og_num_mgm_entry_size=-7*
> 
> Disable pause frames:
> *ethtool -A ens2d1 rx off tx 0ff*
> 
> I did not change the NIC register ( *setpci* ) at this time.
> 
> 
> and then I run the commands below in order:
> 
> 
> *test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
> *test$: /etc/init.d/openibd restart*
> *test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
> --proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
> 05:00.1 -- -P -m "[1-5.0]"*
> 
> | Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel Corporation
>  Flags:Port      :   P--------------:0
> Link State        :          <--Down-->     ----TotalRate----
> Pkts/s Max/Rx     :                 0/0                   0/0
>       Max/Tx     :       844160/732319         844160/732319
> MBits/s Rx/Tx     :               0/*492 *                0/*492*


Note the link state is down and it needs to be up to determine the link speed for Pktgen.

> Broadcast         :                   0
> Multicast         :                   0
>  64 Bytes        :                   0
>  65-127          :                   0
>  128-255         :                   0
>  256-511         :                   0
>  512-1023        :                   0
>  1024-1518       :                   0
> Runts/Jumbos      :                 0/0
> Errors Rx/Tx      :                 0/0
> Total Rx Pkts     :                   0
>      Tx Pkts     :            59289215
>      Rx MBs      :                   0
>      Tx MBs      :               39842
> ARP/ICMP Pkts     :                 0/0
> Tx Overrun        :                   0
> Cycles per Tx     :       14880952/3200
> Missed Rx         :                   0
> mcasts Rx         :                None
> No Mbuf Rx        :                   0
>                  :
> Pattern Type      :             abcd...
> Tx Count/% Rate   :       Forever /100%
> PktSize/Tx Burst  :           64 /   32
> Src/Dest Port     :         1234 / 5678
> Pkt Type:VLAN ID  :     IPv4 / TCP:0001
> Dst  IP Address   :         192.168.1.1
> Src  IP Address   :      192.168.0.1/24
> Src MAC Address   :   e4:1d:2d:02:44:81
> Powered by DPDK --------------------------
> 
> The transmitting rate should be around 40000 since it is 40G NIC.
> However, the rate is much lower than it and it is probably related to
> configurations.
> 
> I actually have two mother boards and each one has 40G NICs installed.
> My intention is to do the tx / rx experiment.
> My first step is to check tx rate can reach as close to 40000 as possible.
> 
> Anyone has related experience please advise.
> 
> 
> 
> P.T
> 
> 
> 2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715@gmail.com>:
> 
>> 
>> Hi,
>> 
>> After enabling the MLX4 flag and recompiling the DPDK source code, Do you
>> recompile the code of pktgen? I think you can recompile it by running make
>> clean and make.
>> 
>> Regards,
>> Zhilong
>> 
>>> 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb@gmail.com> 写道：
>>> 
>>> Hi Olga,
>>> 
>>> 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas@mellanox.com>:
>>> 
>>>> Hi Paul,
>>>> 
>>>> Looks like your ConnectX-3 configured to be IB and not ETH
>>>> 
>>>>> pci@0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
>>>> 
>>>> Use connectx_port_config script.
>>>> Set all the ports to Ethernet, and follow the instructions on the
>> screen.
>>>> 
>>> 
>>> Thanks. I followed the script and configured  both ports to Ethernet.
>>> 
>>> ibstat info below:
>>> 
>>> CA 'mlx4_0'
>>>   CA type: MT4099
>>>   Number of ports: 2
>>>   Firmware version: 2.40.7000
>>>   Hardware version: 1
>>>   Node GUID: 0xe41d2d0300024480
>>>   System image GUID: 0xe41d2d0300024483
>>>   Port 1:
>>>       State: Down
>>>       Physical state: Disabled
>>>       Rate: 10
>>>       Base lid: 0
>>>       LMC: 0
>>>       SM lid: 0
>>>       Capability mask: 0x04010000
>>>       Port GUID: 0xe61d2dfffe024481
>>>       Link layer: Ethernet
>>>   Port 2:
>>>       State: Active
>>>       Physical state: LinkUp
>>>       Rate: 40
>>>       Base lid: 0
>>>       LMC: 0
>>>       SM lid: 0
>>>       Capability mask: 0x04010000
>>>       Port GUID: 0xe61d2dfffe024482
>>>       Link layer: Ethernet
>>> 
>>> I also change the default value from n to  CONFIG_RTE_LIBRTE_MLX4_PMD=y.
>>> Recompiled DPDK and it all worked.
>>> 
>>> And Below is my ifconfig info:
>>> 
>>> ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
>>>         UP BROADCAST MULTICAST  MTU:1500  Metric:1
>>>         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>         collisions:0 txqueuelen:1000
>>>         RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>> 
>>> ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>         RX packets:126 errors:0 dropped:0 overruns:0 frame:0
>>>         TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
>>>         collisions:0 txqueuelen:1000
>>>         RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)
>>> 
>>> 
>>> Since I assume that I don't have to bind the device manually, I executed
>>> ./dpdk-devbind.py --status with the following info popped out:
>>> 
>>> Network devices using kernel driver
>>> ===================================
>>> 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
>>> drv=ixgbe unused=
>>> 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
>>> drv=ixgbe unused=
>>> 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
>>> drv=mlx4_core unused=
>>> 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
>> unused=
>>> 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
>> unused=
>>> *Active*
>>> 
>>> 
>>> With these configurations, I run commands in order but still had the same
>>> result:
>>> 
>>> $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
>>> $test: /etc/init.d/openibd restart
>>> $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>> --proc-type
>>> auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P
>> -m
>>> "[1-5.0]"
>>> 
>>> 
>>>>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
>> cache
>>> 1024
>>> !PANIC!: *** Did not find any ports to use ***
>>> PANIC in pktgen_config_ports():
>>> *** Did not find any ports to use ***6:
>>> [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
>>> [0x447239]]
>>> 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
>>> [0x7f0ec0931830]]
>>> 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
>>> 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
>>> [0x46e324]]
>>> 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
>> [0x43ead4]]
>>> 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
>> [0x4b7ecb]]
>>> Aborted (core dumped)
>>> 
>>> 
>>> 
>>> Is there anything I've done wrong here?
>>> 
>>> 
>>> Thanks
>>> 
>>> 
>>> P.T
>>> 
>>> 
>>>> You can use  -w to specify ConnectX-3 PCI address, you don't need to
>> use -b
>>>> 
>>>> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
>>>> 
>>>> Best Regards,
>>>> Olga
>>>> 
>>>> 
>>>> 
>>> 
>>> --
>>> P.T
>> 
> 
> 
> 
> -- 
> P.T

Regards,
Keith


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2017-08-01 14:06 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-07-31 10:14 [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors Paul Tsvika
2017-07-31 12:54 ` zhilong zheng
2017-07-31 13:07   ` Paul Tsvika
2017-07-31 13:14     ` zhilong zheng
2017-07-31 13:25       ` Paul Tsvika
2017-07-31 13:40         ` zhilong zheng
2017-07-31 14:00           ` Paul Tsvika
2017-07-31 16:11 ` Olga Shern
2017-08-01  3:38   ` Paul Tsvika
2017-08-01  4:15     ` zhilong zheng
2017-08-01  8:15       ` Paul Tsvika
2017-08-01  8:53         ` Zhilong Zheng
2017-08-01  9:04           ` Paul Tsvika
2017-08-01  9:12             ` Zhilong Zheng
2017-08-01  9:37               ` Paul Tsvika
2017-08-01  9:49                 ` Zhilong Zheng
2017-08-01 14:05         ` Wiles, Keith

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).