Unable to initialize ConnectX-4 ports in DPDK 24.11 and 25.03

Unable to initialize ConnectX-4 ports in DPDK 24.11 and 25.03

[Charles Oluwaseun Oluboyo]

Hello,

I have a ConnectX-4 dual-100Gbps NIC that works with DPDK versions
24.03 and 24.07 but not with 24.11 or 25.03.

0000:04:00.0 'MT27700 Family [ConnectX-4] 1013' numa_node=0
if=enp4s0f0np0 drv=mlx5_core unused=vfio-pci
0000:04:00.1 'MT27700 Family [ConnectX-4] 1013' numa_node=0
if=enp4s0f1np1 drv=mlx5_core unused=vfio-pci

Setting the log levels to debug, the newer versions are unable to
configure more than one (1) description on any queue:

ETHDEV: Invalid value for nb_tx_desc(=1024), should be: <= 1, >= 0,
and a product of 1

Whereas using the older version, I am able to set a larger number of
descriptors. Tried 512, 1024, 2048 and 4096.

********************* Infos for port 0  *********************
MAC address: EC:0D:9A:35:1A:16
Device name: 04:00.0
Driver name: mlx5_pci
Firmware-version: 12.24.1000
Devargs:
Connect to socket: 0
memory allocation on the socket: 0
Link status: up
Link speed: 100 Gbps
Link duplex: full-duplex
Autoneg status: On
MTU: 2030
Promiscuous mode: enabled
Allmulticast mode: disabled
Maximum number of MAC addresses: 128
Maximum number of MAC addresses of hash filtering: 0
VLAN offload:
  strip off, filter off, extend off, qinq strip off
Hash key size in bytes: 40
Redirection table size: 4
Supported RSS offload flow types:
  ipv4  ipv4-frag  ipv4-tcp  ipv4-udp  ipv4-other  ipv6
  ipv6-frag  ipv6-tcp  ipv6-udp  ipv6-other  ipv6-ex
  ipv6-tcp-ex  ipv6-udp-ex  esp  l4-dst-only  l4-src-only
  l3-dst-only  l3-src-only
Minimum size of RX buffer: 32
Maximum configurable length of RX packet: 65536
Maximum configurable size of LRO aggregated packet: 65280
Current number of RX queues: 4
Max possible RX queues: 1024
Max possible number of RXDs per queue: 65535
Min possible number of RXDs per queue: 0
RXDs number alignment: 1
Current number of TX queues: 4
Max possible TX queues: 1024
Max possible number of TXDs per queue: 65535
Min possible number of TXDs per queue: 0
TXDs number alignment: 1
Max segment number per packet: 40
Max segment number per MTU/TSO: 40
Device capabilities: 0x10( FLOW_SHARED_OBJECT_KEEP )
Switch name: 04:00.0
Switch domain Id: 0
Switch Port Id: 65535
Device error handling mode: none
Device private info:
  none

Is this a known issue? Please, what should I do next?

Oluwaseun

Re: Unable to initialize ConnectX-4 ports in DPDK 24.11 and 25.03

[Dariusz Sosnowski]

Hi,

On Sat, May 24, 2025 at 01:50:45AM +0100, Charles Oluwaseun Oluboyo wrote:
> Hello,
> 
> I have a ConnectX-4 dual-100Gbps NIC that works with DPDK versions
> 24.03 and 24.07 but not with 24.11 or 25.03.
> 
> 0000:04:00.0 'MT27700 Family [ConnectX-4] 1013' numa_node=0
> if=enp4s0f0np0 drv=mlx5_core unused=vfio-pci
> 0000:04:00.1 'MT27700 Family [ConnectX-4] 1013' numa_node=0
> if=enp4s0f1np1 drv=mlx5_core unused=vfio-pci
> 
> Setting the log levels to debug, the newer versions are unable to
> configure more than one (1) description on any queue:
> 
> ETHDEV: Invalid value for nb_tx_desc(=1024), should be: <= 1, >= 0,
> and a product of 1
> 
> Whereas using the older version, I am able to set a larger number of
> descriptors. Tried 512, 1024, 2048 and 4096.
> 
> ********************* Infos for port 0  *********************
> MAC address: EC:0D:9A:35:1A:16
> Device name: 04:00.0
> Driver name: mlx5_pci
> Firmware-version: 12.24.1000
> Devargs:
> Connect to socket: 0
> memory allocation on the socket: 0
> Link status: up
> Link speed: 100 Gbps
> Link duplex: full-duplex
> Autoneg status: On
> MTU: 2030
> Promiscuous mode: enabled
> Allmulticast mode: disabled
> Maximum number of MAC addresses: 128
> Maximum number of MAC addresses of hash filtering: 0
> VLAN offload:
>   strip off, filter off, extend off, qinq strip off
> Hash key size in bytes: 40
> Redirection table size: 4
> Supported RSS offload flow types:
>   ipv4  ipv4-frag  ipv4-tcp  ipv4-udp  ipv4-other  ipv6
>   ipv6-frag  ipv6-tcp  ipv6-udp  ipv6-other  ipv6-ex
>   ipv6-tcp-ex  ipv6-udp-ex  esp  l4-dst-only  l4-src-only
>   l3-dst-only  l3-src-only
> Minimum size of RX buffer: 32
> Maximum configurable length of RX packet: 65536
> Maximum configurable size of LRO aggregated packet: 65280
> Current number of RX queues: 4
> Max possible RX queues: 1024
> Max possible number of RXDs per queue: 65535
> Min possible number of RXDs per queue: 0
> RXDs number alignment: 1
> Current number of TX queues: 4
> Max possible TX queues: 1024
> Max possible number of TXDs per queue: 65535
> Min possible number of TXDs per queue: 0
> TXDs number alignment: 1
> Max segment number per packet: 40
> Max segment number per MTU/TSO: 40
> Device capabilities: 0x10( FLOW_SHARED_OBJECT_KEEP )
> Switch name: 04:00.0
> Switch domain Id: 0
> Switch Port Id: 65535
> Device error handling mode: none
> Device private info:
>   none
> 
> Is this a known issue? Please, what should I do next?
> 
> Oluwaseun

One of the patches merged to 24.11 introduced an issue with queues
descriptor limits on older NICs.

A patch with a fix for that is pending for merge to 25.07.

Would you be able apply the following patches and test on your side?

- https://patches.dpdk.org/project/dpdk/patch/20250424133128.133900-1-viacheslavo@nvidia.com/
- https://patches.dpdk.org/project/dpdk/patch/20250514075530.419423-1-viacheslavo@nvidia.com/

Best regards,
Dariusz Sosnowski

Re: Unable to initialize ConnectX-4 ports in DPDK 24.11 and 25.03

[Charles Oluwaseun Oluboyo]

Hello Dariusz and all,

>
> One of the patches merged to 24.11 introduced an issue with queues
> descriptor limits on older NICs.
>
> A patch with a fix for that is pending for merge to 25.07.
>
> Would you be able apply the following patches and test on your side?
>
> - https://patches.dpdk.org/project/dpdk/patch/20250424133128.133900-1-viacheslavo@nvidia.com/
> - https://patches.dpdk.org/project/dpdk/patch/20250514075530.419423-1-viacheslavo@nvidia.com/
>

I applied these patches, and they have fixed the issue with the
descriptors. There are now 32768 per queue.
I have tested with testpmd, and the ports work. However, I am unable
to initialize the ports with pktgen.

Initialize Port 2 ...
   Max MTU: 65535
   Enabling Tx TCP_CKSUM offload
   Enabling Tx UDP_CKSUM offload
   Enabling Tx IPV4_CKSUM offload
   Allocate packet sequence array
   Setup latency defaults
   Setup fill pattern defaults
   Configure device
ETHDEV: Frame size (65553) > device max frame size (65536) for port_id 2
!PANIC!: Cannot configure device: port=2, Num queues 1,1.
EAL: PANIC in initialize_port_info():
Cannot configure device: port=2, Num queues 1,1.
0: /usr/local/lib/x86_64-linux-gnu/librte_eal.so.25
(rte_dump_stack+0x42) [7f16cb3de6d2]
1: /usr/local/lib/x86_64-linux-gnu/librte_eal.so.25 (__rte_panic+0xd4)
[7f16cb3b737f]
2: ./usr/local/bin/pktgen (5606908ff000+0xd26e) [56069090c26e]
3: ./usr/local/bin/pktgen (5606908ff000+0x2c61e) [56069092b61e]
4: ./usr/local/bin/pktgen (5606908ff000+0xe339) [56069090d339]
5: /lib/x86_64-linux-gnu/libc.so.6 (7f16cafc2000+0x29d90) [7f16cafebd90]
6: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0x80) [7f16cafebe40]
7: ./usr/local/bin/pktgen (5606908ff000+0xecc5) [56069090dcc5]

65553 looks very unusual, doesn't it? Do you have any suggestions as
to what to do next?

Regards,

Oluwaseun

Re: Unable to initialize ConnectX-4 ports in DPDK 24.11 and 25.03

[Charles Oluwaseun Oluboyo]

Hello all.

Appears the PMD is returning the wrong value for the max MTU.

The error about the frame size is raised in eth_dev_validate_mtu
called from rte_eth_dev_configure.
dev_info->max_rx_pktlen = 65536
dev_info->max_mtu = 65535

That is not true. From my testing, max MTU is 9960 for this ConnectX-4
Should this have been set in mlx5_dev_infos_get?

Regards,

Oluwaseun

On Tue, May 27, 2025 at 7:12 PM Charles Oluwaseun Oluboyo
<charlesboyo@gmail.com> wrote:
>
> Hello Dariusz and all,
>
> >
> > One of the patches merged to 24.11 introduced an issue with queues
> > descriptor limits on older NICs.
> >
> > A patch with a fix for that is pending for merge to 25.07.
> >
> > Would you be able apply the following patches and test on your side?
> >
> > - https://patches.dpdk.org/project/dpdk/patch/20250424133128.133900-1-viacheslavo@nvidia.com/
> > - https://patches.dpdk.org/project/dpdk/patch/20250514075530.419423-1-viacheslavo@nvidia.com/
> >
>
> I applied these patches, and they have fixed the issue with the
> descriptors. There are now 32768 per queue.
> I have tested with testpmd, and the ports work. However, I am unable
> to initialize the ports with pktgen.
>
> Initialize Port 2 ...
>    Max MTU: 65535
>    Enabling Tx TCP_CKSUM offload
>    Enabling Tx UDP_CKSUM offload
>    Enabling Tx IPV4_CKSUM offload
>    Allocate packet sequence array
>    Setup latency defaults
>    Setup fill pattern defaults
>    Configure device
> ETHDEV: Frame size (65553) > device max frame size (65536) for port_id 2
> !PANIC!: Cannot configure device: port=2, Num queues 1,1.
> EAL: PANIC in initialize_port_info():
> Cannot configure device: port=2, Num queues 1,1.
> 0: /usr/local/lib/x86_64-linux-gnu/librte_eal.so.25
> (rte_dump_stack+0x42) [7f16cb3de6d2]
> 1: /usr/local/lib/x86_64-linux-gnu/librte_eal.so.25 (__rte_panic+0xd4)
> [7f16cb3b737f]
> 2: ./usr/local/bin/pktgen (5606908ff000+0xd26e) [56069090c26e]
> 3: ./usr/local/bin/pktgen (5606908ff000+0x2c61e) [56069092b61e]
> 4: ./usr/local/bin/pktgen (5606908ff000+0xe339) [56069090d339]
> 5: /lib/x86_64-linux-gnu/libc.so.6 (7f16cafc2000+0x29d90) [7f16cafebd90]
> 6: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0x80) [7f16cafebe40]
> 7: ./usr/local/bin/pktgen (5606908ff000+0xecc5) [56069090dcc5]
>
> 65553 looks very unusual, doesn't it? Do you have any suggestions as
> to what to do next?
>
> Regards,
>
> Oluwaseun