Hello,
I am trying to set up dpdk with netvsc as master pmd on Azure following
https://learn.microsoft.com/en-us/azure/virtual-network/setup-dpdk?tabs=ubuntu
and
https://doc.dpdk.org/guides-22.11/nics/netvsc.html.
On the Azure VM, I have a LAN and a WAN interface with accelerated networking enabled.I have unbound both the VMBUS devices from the kernel and bound it to uio_hv_generic.The DPDK version is 22.11 running on openwrt 5.15.150.
When I try running testpmd in io mode to send and receive traffic between LAN and WAN port, I notice very low throughput. Please find the testpmd command and stats below:
/opt/vc/bin/dpdk-testpmd -l 1-3 -n 1 -a f030:00:02.0 -a 2334:00:02.0 -- --rxq=1 --txq=1 -i
EAL: Detected CPU lcores: 8
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: Debug dataplane logs available - lower performance
EAL: VFIO support initialized
EAL: Probe PCI driver: mlx5_pci (15b3:1016) device: 2334:00:02.0 (socket -1)
mlx5_net: No available register for sampler.
EAL: Probe PCI driver: mlx5_pci (15b3:1016) device: f030:00:02.0 (socket -1)
mlx5_net: No available register for sampler.
hn_vf_attach(): found matching VF port 0
hn_vf_attach(): found matching VF port 1
TELEMETRY: No legacy callbacks, legacy socket not created
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=326912, size=2560, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 2 (socket 0)
Port 2: 00:0D:3A:42:F8:3C
Configuring Port 3 (socket 0)
Port 3: 00:0D:3A:42:FB:CD
Checking link statuses...
Done
testpmd> start tx_first
io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP allocation mode: native
Logical Core 2 (socket 0) forwards packets on 2 streams:
RX P=2/Q=0 (socket 0) -> TX P=3/Q=0 (socket 0) peer=02:00:00:00:00:03
RX P=3/Q=0 (socket 0) -> TX P=2/Q=0 (socket 0) peer=02:00:00:00:00:02
io packet forwarding packets/burst=32
nb forwarding cores=1 - nb forwarding ports=2
port 2: RX queue number: 1 Tx queue number: 1
Rx offloads=0x0 Tx offloads=0x0
RX queue: 0
RX desc=256 - RX free threshold=0
RX threshold registers: pthresh=0 hthresh=0 wthresh=0
RX Offloads=0x0
TX queue: 0
TX desc=256 - TX free threshold=0
TX threshold registers: pthresh=0 hthresh=0 wthresh=0
TX offloads=0x0 - TX RS bit threshold=0
port 3: RX queue number: 1 Tx queue number: 1
Rx offloads=0x0 Tx offloads=0x0
RX queue: 0
RX desc=256 - RX free threshold=0
RX threshold registers: pthresh=0 hthresh=0 wthresh=0
RX Offloads=0x0
TX queue: 0
TX desc=256 - TX free threshold=0
TX threshold registers: pthresh=0 hthresh=0 wthresh=0
TX offloads=0x0 - TX RS bit threshold=0
testpmd> show port stats all
######################## NIC statistics for port 2 ########################
RX-packets: 34 RX-missed: 0 RX-bytes: 2194
RX-errors: 0
RX-nombuf: 0
TX-packets: 32 TX-errors: 0 TX-bytes: 2048
Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 0 Tx-bps: 0
############################################################################
######################## NIC statistics for port 3 ########################
RX-packets: 1 RX-missed: 0 RX-bytes: 86
RX-errors: 0
RX-nombuf: 0
TX-packets: 64 TX-errors: 0 TX-bytes: 4096
Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 0 Tx-bps: 0
############################################################################
testpmd> show port stats all
######################## NIC statistics for port 2 ########################
RX-packets: 34 RX-missed: 0 RX-bytes: 2194
RX-errors: 0
RX-nombuf: 0
TX-packets: 32 TX-errors: 0 TX-bytes: 2048
Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 0 Tx-bps: 0
############################################################################
######################## NIC statistics for port 3 ########################
RX-packets: 1 RX-missed: 0 RX-bytes: 86
RX-errors: 0
RX-nombuf: 0
TX-packets: 64 TX-errors: 0 TX-bytes: 4096
Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 0 Tx-bps: 0
############################################################################
testpmd> show port stats all
######################## NIC statistics for port 2 ########################
RX-packets: 34 RX-missed: 0 RX-bytes: 2194
RX-errors: 0
RX-nombuf: 0
TX-packets: 32 TX-errors: 0 TX-bytes: 2048
Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 0 Tx-bps: 0
############################################################################
######################## NIC statistics for port 3 ########################
RX-packets: 1 RX-missed: 0 RX-bytes: 86
RX-errors: 0
RX-nombuf: 0
TX-packets: 64 TX-errors: 0 TX-bytes: 4096
Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 0 Tx-bps: 0
############################################################################
testpmd> quit
Telling cores to stop...
Waiting for lcores to finish...
---------------------- Forward statistics for port 2 ----------------------
RX-packets: 32 RX-dropped: 0 RX-total: 32
TX-packets: 32 TX-dropped: 0 TX-total: 32
----------------------------------------------------------------------------
---------------------- Forward statistics for port 3 ----------------------
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 64 TX-dropped: 0 TX-total: 64
----------------------------------------------------------------------------
+++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
RX-packets: 32 RX-dropped: 0 RX-total: 32
TX-packets: 96 TX-dropped: 0 TX-total: 96
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Done.
Stopping port 2...
Stopping ports...
Done
Stopping port 3...
Stopping ports...
Done
Shutting down port 2...
Closing ports...
Port 0 is closed
Port 2 is closed
Done
Shutting down port 3...
Closing ports...
Port 1 is closed
Port 3 is closed
Done
Bye...
If I try with 2 queues , the throughput only slightly improves. I expected to see larger values.
After enabling debug logs i observe that
1. Both the VMbus devices are being probed and matching VF devices found
2. VF devices are being configured with Rx and Tx queue setup.
Any ideas what I might be doing wrong?
Regards,
Nandini