ConnectX-7 performance anomaly

[Kamil Vojanec <vojanec@cesnet.cz>]

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]

Hello,

while benchmarking the Nvidia ConnectX-7 400Gbe NIC, I encountered some 
interesting behavior: when processing short packets (72 B), the maximum 
packet rate I can achieve is 230 Mpps. However, when I increase the 
packet size to 512 B briefly and then bring the packet size down to 72 B 
again, the maximum throughput is now 260 Mpps and seems to stabilize at 
that value. According to the Nvidia performance report 
(https://fast.dpdk.org/doc/perf/DPDK_24_07_NVIDIA_NIC_performance_report.pdf), 
the maximum achievable packet rate of ConnectX-7 NICs is 266 Mpps and 
that is what I am trying to reproduce. I have observed this behavior 
with dpdk-l3fwd as well as with my own DPDK application.

My setup is as follows:

CPU: AMD EPYC 9554P 64-Core Processor

Application commandline:

sudo dpdk_example_l3fwd -a0000:05:00.0 -l0-32 --main-lcore=32  -- -p0x1 
--config 
"(0,0,0),(0,1,1),(0,2,2),(0,3,3),(0,4,4),(0,5,5),(0,6,6),(0,7,7),(0,8,8),(0,9,9),(0,10,10),(0,11,11),(0,12,12),(0,13,13),(0,14,14),(0,15,15),(0,16,16),(0,17,17),(0,18,18),(0,19,19),(0,20,20),(0,21,21),(0,22,22),(0,23,23),(0,24,24),(0,25,25),(0,26,26),(0,27,27),(0,28,28),(0,29,29),(0,30,30),(0,31,31)" 
--eth-dest=0,00:10:94:00:00:02

Generator: Spirent 400G generator

Would anyone be able to share some insight as to what might be happening 
here?

Kamil Vojanec
CESNET z.s.p.o.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4495 bytes --]