[dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Alessandro Pagani]

Hi all,

I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5 driver).

I am specifying the tx_pp parameter to provide the packet send scheduling
on mbuf timestamps, but the testpmd fails with the following error:

# ./dpdk-testpmd -l 8-15 -n 4 -a 3b:00.0,tx_pp=500 -- --flow-isolate-all -i
EAL: Detected 36 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:3b:00.0 (socket 0)
mlx5_pci: No available register for Sampler.
mlx5_pci: WQE index ignore feature is required for packet pacing
mlx5_pci: probe of PCI device 0000:3b:00.0 aborted after encountering an
error: No such device
common_mlx5: Failed to load driver = mlx5_pci.

EAL: Requested device 0000:3b:00.0 cannot be used
EAL: Bus (pci) probe failed.
EAL: No legacy callbacks, legacy socket not created
testpmd: No probed ethernet devices



The error messages suggest that "WQE index ignore feature is required for
packet pacing".

Anyone knows the reason of this error and how to solve it?
I know that WQE is related to RDMA (InfiniBand/RoCE) but I do not
understand how it is related to DPDK.

Thank you.

Best regards
Alessandro

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Thomas Monjalon]

09/12/2020 17:03, Alessandro Pagani:
> Hi all,
> 
> I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5 driver).
> 
> I am specifying the tx_pp parameter to provide the packet send scheduling
> on mbuf timestamps, but the testpmd fails with the following error:
[...]
> EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:3b:00.0 (socket 0)
> mlx5_pci: No available register for Sampler.
> mlx5_pci: WQE index ignore feature is required for packet pacing
> mlx5_pci: probe of PCI device 0000:3b:00.0 aborted after encountering an
> error: No such device
> common_mlx5: Failed to load driver = mlx5_pci.
> 
> EAL: Requested device 0000:3b:00.0 cannot be used
[...]
> The error messages suggest that "WQE index ignore feature is required for
> packet pacing".
> 
> Anyone knows the reason of this error and how to solve it?

I think it means your device does not support this feature.
But I realize it is not documented here:
http://doc.dpdk.org/guides/nics/mlx5.html#supported-hardware-offloads

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Slava Ovsiienko]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Friday, December 11, 2020 17:48
> To: Alessandro Pagani <alessandropagani.90@gmail.com>
> Cc: users@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
> <asafp@nvidia.com>
> Subject: Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore
> feature is required for packet pacing'
> 
> 09/12/2020 17:03, Alessandro Pagani:
> > Hi all,
> >
> > I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5 driver).
> >
> > I am specifying the tx_pp parameter to provide the packet send
> > scheduling on mbuf timestamps, but the testpmd fails with the following
> error:
> [...]
> > EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:3b:00.0

This is ConnectX-4LX (DevID is 1015), it does not support scheduling.
Tx scheduling is supported since ConnectX-6DX. 

> > (socket 0)
> > mlx5_pci: No available register for Sampler.
> > mlx5_pci: WQE index ignore feature is required for packet pacing
> > mlx5_pci: probe of PCI device 0000:3b:00.0 aborted after encountering
> > an
> > error: No such device
> > common_mlx5: Failed to load driver = mlx5_pci.
> >
> > EAL: Requested device 0000:3b:00.0 cannot be used
> [...]
> > The error messages suggest that "WQE index ignore feature is required
> > for packet pacing".
> >
> > Anyone knows the reason of this error and how to solve it?
> 
> I think it means your device does not support this feature.
> But I realize it is not documented here:

Yes, indeed. I'll provide the patch, thank you for noticing that.

With best regards, Slava


> https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dp
> dk.org%2Fguides%2Fnics%2Fmlx5.html%23supported-hardware-
> offloads&amp;data=04%7C01%7Cviacheslavo%40nvidia.com%7C68077a428d9
> 140ecac4408d89dec2c93%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0
> %7C637432984987268866%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sda
> ta=2n5MI7WLln9xRPcpQzSy2Q2g3xPlYnYrK6YIeRKxS9Q%3D&amp;reserved=0
> 


> 

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Thomas Monjalon]

11/12/2020 17:19, Slava Ovsiienko:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 09/12/2020 17:03, Alessandro Pagani:
> > > Hi all,
> > >
> > > I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5 driver).
> > >
> > > I am specifying the tx_pp parameter to provide the packet send
> > > scheduling on mbuf timestamps, but the testpmd fails with the following
> > error:
> > [...]
> > > EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:3b:00.0
> 
> This is ConnectX-4LX (DevID is 1015), it does not support scheduling.
> Tx scheduling is supported since ConnectX-6DX. 
> 
> > > (socket 0)
> > > mlx5_pci: No available register for Sampler.
> > > mlx5_pci: WQE index ignore feature is required for packet pacing
> > > mlx5_pci: probe of PCI device 0000:3b:00.0 aborted after encountering
> > > an
> > > error: No such device
> > > common_mlx5: Failed to load driver = mlx5_pci.
> > >
> > > EAL: Requested device 0000:3b:00.0 cannot be used
> > [...]
> > > The error messages suggest that "WQE index ignore feature is required
> > > for packet pacing".
> > >
> > > Anyone knows the reason of this error and how to solve it?
> > 
> > I think it means your device does not support this feature.
> > But I realize it is not documented here:
> 
> Yes, indeed. I'll provide the patch, thank you for noticing that.

I think we should also improve the error message to
something like "not supported on this device".

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Slava Ovsiienko]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Friday, December 11, 2020 18:31
> To: Slava Ovsiienko <viacheslavo@nvidia.com>
> Cc: Alessandro Pagani <alessandropagani.90@gmail.com>; users@dpdk.org;
> Asaf Penso <asafp@nvidia.com>
> Subject: Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore
> feature is required for packet pacing'
> 
> 11/12/2020 17:19, Slava Ovsiienko:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 09/12/2020 17:03, Alessandro Pagani:
> > > > Hi all,
> > > >
> > > > I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5
> driver).
> > > >
> > > > I am specifying the tx_pp parameter to provide the packet send
> > > > scheduling on mbuf timestamps, but the testpmd fails with the
> > > > following
> > > error:
> > > [...]
> > > > EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:3b:00.0
> >
> > This is ConnectX-4LX (DevID is 1015), it does not support scheduling.
> > Tx scheduling is supported since ConnectX-6DX.
> >
> > > > (socket 0)
> > > > mlx5_pci: No available register for Sampler.
> > > > mlx5_pci: WQE index ignore feature is required for packet pacing
> > > > mlx5_pci: probe of PCI device 0000:3b:00.0 aborted after
> > > > encountering an
> > > > error: No such device
> > > > common_mlx5: Failed to load driver = mlx5_pci.
> > > >
> > > > EAL: Requested device 0000:3b:00.0 cannot be used
> > > [...]
> > > > The error messages suggest that "WQE index ignore feature is
> > > > required for packet pacing".
> > > >
> > > > Anyone knows the reason of this error and how to solve it?
> > >
> > > I think it means your device does not support this feature.
> > > But I realize it is not documented here:
> >
> > Yes, indeed. I'll provide the patch, thank you for noticing that.
> 
> I think we should also improve the error message to something like "not
> supported on this device".
> 
In my opinion, we should not depend on model index, but rather on
what capabilities firmware reports, this approach provides much more flexibility
and accuracy. We do not provide model index blind check for any other feature(s).

With best regards,
Slava

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Asaf Penso]

Hello Alessandro,

As you can see from the documentation - http://doc.dpdk.org/guides/nics/mlx5.html#supported-hardware-offloads
The feature is supported starting ConnectX-6 Dx, and it's not supported for ConnectX-4Lx as you tried.

Regards,
Asaf Penso

>-----Original Message-----
>From: users <users-bounces@dpdk.org> On Behalf Of Alessandro Pagani
>Sent: Wednesday, December 9, 2020 6:03 PM
>To: users@dpdk.org
>Subject: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore
>feature is required for packet pacing'
>
>Hi all,
>
>I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5 driver).
>
>I am specifying the tx_pp parameter to provide the packet send scheduling on
>mbuf timestamps, but the testpmd fails with the following error:
>
># ./dpdk-testpmd -l 8-15 -n 4 -a 3b:00.0,tx_pp=500 -- --flow-isolate-all -i
>EAL: Detected 36 lcore(s)
>EAL: Detected 2 NUMA nodes
>EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>EAL: Selected IOVA mode 'PA'
>EAL: No available hugepages reported in hugepages-1048576kB
>EAL: Probing VFIO support...
>EAL: VFIO support initialized
>EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:3b:00.0 (socket 0)
>mlx5_pci: No available register for Sampler.
>mlx5_pci: WQE index ignore feature is required for packet pacing
>mlx5_pci: probe of PCI device 0000:3b:00.0 aborted after encountering an
>error: No such device
>common_mlx5: Failed to load driver = mlx5_pci.
>
>EAL: Requested device 0000:3b:00.0 cannot be used
>EAL: Bus (pci) probe failed.
>EAL: No legacy callbacks, legacy socket not created
>testpmd: No probed ethernet devices
>
>
>
>The error messages suggest that "WQE index ignore feature is required for
>packet pacing".
>
>Anyone knows the reason of this error and how to solve it?
>I know that WQE is related to RDMA (InfiniBand/RoCE) but I do not
>understand how it is related to DPDK.
>
>Thank you.
>
>Best regards
>Alessandro

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Alessandro Pagani]

Thanks for your reply.
I did not realize from the DPDK documentation that the tx scheduling
feature is not supported by ConnectX4 Lx.

Moreover, an improved error message would be really helpful.

Thanks for the info.

Best regards
Alessandro


On Fri, Dec 11, 2020, 5:44 PM Slava Ovsiienko <viacheslavo@nvidia.com>
wrote:

> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: Friday, December 11, 2020 18:31
> > To: Slava Ovsiienko <viacheslavo@nvidia.com>
> > Cc: Alessandro Pagani <alessandropagani.90@gmail.com>; users@dpdk.org;
> > Asaf Penso <asafp@nvidia.com>
> > Subject: Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore
> > feature is required for packet pacing'
> >
> > 11/12/2020 17:19, Slava Ovsiienko:
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > 09/12/2020 17:03, Alessandro Pagani:
> > > > > Hi all,
> > > > >
> > > > > I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5
> > driver).
> > > > >
> > > > > I am specifying the tx_pp parameter to provide the packet send
> > > > > scheduling on mbuf timestamps, but the testpmd fails with the
> > > > > following
> > > > error:
> > > > [...]
> > > > > EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:3b:00.0
> > >
> > > This is ConnectX-4LX (DevID is 1015), it does not support scheduling.
> > > Tx scheduling is supported since ConnectX-6DX.
> > >
> > > > > (socket 0)
> > > > > mlx5_pci: No available register for Sampler.
> > > > > mlx5_pci: WQE index ignore feature is required for packet pacing
> > > > > mlx5_pci: probe of PCI device 0000:3b:00.0 aborted after
> > > > > encountering an
> > > > > error: No such device
> > > > > common_mlx5: Failed to load driver = mlx5_pci.
> > > > >
> > > > > EAL: Requested device 0000:3b:00.0 cannot be used
> > > > [...]
> > > > > The error messages suggest that "WQE index ignore feature is
> > > > > required for packet pacing".
> > > > >
> > > > > Anyone knows the reason of this error and how to solve it?
> > > >
> > > > I think it means your device does not support this feature.
> > > > But I realize it is not documented here:
> > >
> > > Yes, indeed. I'll provide the patch, thank you for noticing that.
> >
> > I think we should also improve the error message to something like "not
> > supported on this device".
> >
> In my opinion, we should not depend on model index, but rather on
> what capabilities firmware reports, this approach provides much more
> flexibility
> and accuracy. We do not provide model index blind check for any other
> feature(s).
>
> With best regards,
> Slava
>
>
>
>