Hi, I’d like to test the latency between different network interfaces on the same host machine using DPDK applications.
However, my DPDK application is currently unable to receive any packets, although it was successful with the same configuration previously. Here are the details:
sudo ./pktgen -l 0,2 -n 4 --proc-type auto --file-prefix pg -a 03:00.0 -- -m "[2].0" -T -P
Pktgen:/> start 0
/ Ports 0-0 of 1 <Main Page> Copyright(c) <2010-2023>, Intel Corporation
Port:Flags : 0:P------ Single
Link State : <UP-100000-FD> ---Total Rate---
Pkts/s Rx : 0 0
Tx : 22,764,800 22,764,800
MBits/s Rx/Tx : 0/14,569 0/14,569
Pkts/s Rx Max : 0 0
Tx Max : 23,168,768 23,168,768
Broadcast : 0
Multicast : 0
Sizes 64 : 0
65-127 : 0
128-255 : 0
256-511 : 0
512-1023 : 0
1024-1518 : 0
Runts/Jumbos : 0/0
ARP/ICMP Pkts : 0/0
Errors Rx/Tx : 0/0
Total Rx Pkts : 0
Tx Pkts : 1,922,118,272
Rx/Tx MBs : 0/1,230,155
TCP Flags : .A....
TCP Seq/Ack : 74616/74640
Pattern Type : abcd...
Tx Count/% Rate : Forever /100%
Pkt Size/Rx:Tx Burst: 64 / 64: 64
TTL/Port Src/Dest : 64/ 1234/ 5678
Pkt Type:VLAN ID : IPv4 / TCP:0001
802.1p CoS/DSCP/IPP : 0/ 0/ 0
VxLAN Flg/Grp/vid : 0000/ 0/ 0
IP Destination : 192.168.1.1
Source : 192.168.0.1/24
MAC Destination : 00:00:00:00:00:00
Source : b8:59:9f:42:6d:24
NUMA/Vend:ID/PCI : 0/15b3:a2d2/03:00.0
-- Pktgen 24.03.1 (DPDK 24.03.0-rc4) Powered by DPDK (pid:123993) -----------
sudo ./pktgen -l 1,3,5 -n 4 --proc-type auto --file-prefix pg-receive -a 82:00.0 -- -P -m "[3/5].0" -T
\ Ports 0-0 of 1 <Main Page> Copyright(c) <2010-2023>, Intel Corporation
Port:Flags : 0:P------ Single
Link State : <UP-100000-FD> ---Total Rate---
Pkts/s Rx : 0 0
Tx : 0 0
MBits/s Rx/Tx : 0/0 0/0
Pkts/s Rx Max : 1 1
Tx Max : 0 0
Broadcast : 0
Multicast : 0
Sizes 64 : 0
65-127 : 128
128-255 : 0
256-511 : 0
512-1023 : 0
1024-1518 : 0
Runts/Jumbos : 0/0
ARP/ICMP Pkts : 0/0
Errors Rx/Tx : 0/0
Total Rx Pkts : 2
Tx Pkts : 0
Rx/Tx MBs : 0/0
TCP Flags : .A....
TCP Seq/Ack : 74616/74640
Pattern Type : abcd...
Tx Count/% Rate : Forever /100%
Pkt Size/Rx:Tx Burst: 64 / 64: 64
TTL/Port Src/Dest : 64/ 1234/ 5678
Pkt Type:VLAN ID : IPv4 / TCP:0001
802.1p CoS/DSCP/IPP : 0/ 0/ 0
VxLAN Flg/Grp/vid : 0000/ 0/ 0
IP Destination : 192.168.1.1
Source : 192.168.0.1/24
MAC Destination : 00:00:00:00:00:00
Source : 7c:fe:90:55:db:5a
NUMA/Vend:ID/PCI : 1/15b3:1013/82:00.0
-- Pktgen 24.03.1 (DPDK 24.03.0-rc4) Powered by DPDK (pid:124523) -----------
sudo ./dpdk-testpmd -l 1,3,5 -n 4 --file-prefix pg-receive --proc-type auto -a 82:00.0 -- -i --portlist=0 --nb-cores=2 --port-topology=loop
testpmd> start
...
testpmd> stop
---------------------- Forward statistics for port 0 ----------------------
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 0 TX-dropped: 0 TX-total: 0
----------------------------------------------------------------------------
+++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 0 TX-dropped: 0 TX-total: 0
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
sudo ./build/l2fwd -l 1,3,5 -n 4 --file-prefix pg-receive --proc-type auto -a 82:00.0 -- -p 0001 -P
Port statistics ====================================
Statistics for port 0 ------------------------------
Packets sent: 0
Packets received: 0
Packets dropped: 0
Aggregate statistics ===============================
Total packets sent: 0
Total packets received: 0
Total packets dropped: 0
====================================================
sudo tcpdump -i enp130s0np0 -n
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on enp130s0np0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
06:43:06.954175 LLDP, length 59
06:43:37.007247 LLDP, length 59
06:44:07.059921 LLDP, length 59
06:44:37.112526 LLDP, length 59
06:45:07.164358 LLDP, length 59
tcpdump cannot catch any packtes generated by pktgen sender.
$ ifconfig -a
enp130s0np0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 1500
inet 192.168.3.1 netmask 255.255.255.0 broadcast 192.168.3.255
inet6 fe80::7efe:90ff:fe55:db5a prefixlen 64 scopeid 0x20<link>
ether 7c:fe:90:55:db:5a txqueuelen 1000 (Ethernet)
RX packets 65 bytes 4745 (4.7 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 14 bytes 1076 (1.0 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
(82:00.0)
enp3s0f0np0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 1500
inet 192.168.3.2 netmask 255.255.255.0 broadcast 192.168.3.255
inet6 fe80::ba59:9fff:fe42:6d24 prefixlen 64 scopeid 0x20<link>
ether b8:59:9f:42:6d:24 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15 bytes 1146 (1.1 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
(03:00.0)
sudo ./dpdk-devbind.py -s
Network devices using kernel driver
===================================
0000:03:00.0 'MT416842 BlueField integrated ConnectX-5 network controller a2d2' if=enp3s0f0np0 drv=mlx5_core unused=vfio-pci *Active*
0000:82:00.0 'MT27700 Family [ConnectX-4] 1013' if=enp130s0np0 drv=mlx5_core unused=vfio-pci *Active*
The two network cards, 03:00.0
and 82:00.0
, are directly connected using a single Ethernet cable.
Ubuntu20.04
DPDK version: 23.11.0
Pktgen version: 24.03.1