[mlx5 driver] Usage of mlx5 with unpriviledged LXC container

[mlx5 driver] Usage of mlx5 with unpriviledged LXC container

[Julien]

[-- Attachment #1: Type: text/plain, Size: 4046 bytes --]

Hello,
I have a question about using the mlx5 driver with LXC.
I'm trying to use dpdk-testpmd in an LXC container whose root user isn't
mapped to the host's root user.
Note: The entire physical interface is given to the LXC container, not a
virtual interface.

The following error occured:
mlx5_common: DevX create TIS failed errno=22 status=0 syndrome=0
mlx5_net: Failed to create TIS 0/0 for [bonding] device mlx5_2.
mlx5_net: TIS allocation failure
mlx5_net: probe of PCI device 0000:27:00.0 aborted after encountering an
error: Cannot allocate memory
mlx5_common: Failed to load driver mlx5_eth
EAL: Requested device 0000:27:00.0 cannot be used
EAL: Bus (pci) probe failed.

The "transport_domain" is created, and the mlx5_devx_cmd_create_td()
function runs normally.
The call to mlx5dv_devx_obj_create() receives an errno of 22.

I don't encounter any problems when the container's root user is mapped to
the host's root user.
Has anyone experienced this before?
Is it possible to use the driver in an unprivileged LXC container?

Dpdk version: 23.11
Linux Kernel: 5.15
Stack:
libibverbs.so.1!execute_ioctl(struct ibv_context * context, struct
ibv_context * context@entry, struct ibv_command_buffer * cmd, struct
ibv_command_buffer * cmd@entry) (\rdma-core-49.0\libibverbs\cmd_ioctl.c:147)
libmlx5.so.1!_mlx5dv_devx_obj_create(struct ibv_context * context, const
void * in, size_t inlen, void * out, size_t outlen)
(\rdma-core-49.0\providers\mlx5\verbs.c:5794)
libmlx5.so.1!mlx5dv_devx_obj_create(struct ibv_context * context, const
void * in, size_t inlen, void * out, size_t outlen)
(\rdma-core-49.0\providers\mlx5\verbs.c:5819)
librte_common_mlx5.so.24!mlx5_glue_devx_obj_create(struct ibv_context *
ctx, const void * in, size_t inlen, void * out, size_t outlen)
(\dpdk-23.11\drivers\common\mlx5\linux\mlx5_glue.c:1045)
librte_common_mlx5.so.24!mlx5_devx_cmd_create_tis(void * ctx, struct
mlx5_devx_tis_attr * tis_attr)
(\dpdk-23.11\drivers\common\mlx5\mlx5_devx_cmds.c:2037)
librte_net_mlx5.so!mlx5_setup_tis(struct mlx5_dev_ctx_shared * sh)
(\dpdk-23.11\drivers\net\mlx5\mlx5.c:1343)
librte_net_mlx5.so!mlx5_alloc_shared_dev_ctx(const struct
mlx5_dev_spawn_data * spawn, struct mlx5_kvargs_ctrl * mkvlist)
(\dpdk-23.11\drivers\net\mlx5\mlx5.c:1784)
librte_net_mlx5.so!mlx5_dev_spawn(struct rte_device * dpdk_dev, struct
mlx5_dev_spawn_data * spawn, struct rte_eth_devargs * eth_da, struct
mlx5_kvargs_ctrl * mkvlist)
(\dpdk-23.11\drivers\net\mlx5\linux\mlx5_os.c:1169)
librte_net_mlx5.so!mlx5_os_pci_probe_pf(struct mlx5_common_device * cdev,
struct rte_eth_devargs * req_eth_da, uint16_t owner_id, struct
mlx5_kvargs_ctrl * mkvlist)
(\dpdk-23.11\drivers\net\mlx5\linux\mlx5_os.c:2648)
librte_net_mlx5.so!mlx5_os_pci_probe(struct mlx5_common_device * cdev,
struct mlx5_kvargs_ctrl * mkvlist)
(\dpdk-23.11\drivers\net\mlx5\linux\mlx5_os.c:2797)
librte_net_mlx5.so!mlx5_os_net_probe(struct mlx5_common_device * cdev,
struct mlx5_kvargs_ctrl * mkvlist)
(\dpdk-23.11\drivers\net\mlx5\linux\mlx5_os.c:2881)
librte_common_mlx5.so.24!drivers_probe(struct mlx5_common_device * cdev,
uint32_t user_classes, struct mlx5_kvargs_ctrl * mkvlist)
(\dpdk-23.11\drivers\common\mlx5\mlx5_common.c:938)
librte_common_mlx5.so.24!mlx5_common_dev_probe(struct rte_device * eal_dev)
(\dpdk-23.11\drivers\common\mlx5\mlx5_common.c:1028)
librte_common_mlx5.so.24!mlx5_common_pci_probe(struct rte_pci_driver *
pci_drv, struct rte_pci_device * pci_dev)
(\dpdk-23.11\drivers\common\mlx5\mlx5_common_pci.c:168)
librte_bus_pci.so.24!rte_pci_probe_one_driver(struct rte_pci_driver * dr,
struct rte_pci_device * dev) (\dpdk-23.11\drivers\bus\pci\pci_common.c:312)
librte_bus_pci.so.24!pci_probe_all_drivers(struct rte_pci_device * dev)
(\dpdk-23.11\drivers\bus\pci\pci_common.c:396)
librte_bus_pci.so.24!pci_probe()
(\dpdk-23.11\drivers\bus\pci\pci_common.c:423)
librte_eal.so.24!rte_bus_probe()
(\dpdk-23.11\lib\eal\common\eal_common_bus.c:78)
librte_eal.so.24!rte_eal_init(int argc, char ** argv)
(\dpdk-23.11\lib\eal\linux\eal.c:1287)

Best regards,
Julien Marcin

[-- Attachment #2: Type: text/html, Size: 6122 bytes --]

Re: [mlx5 driver] Usage of mlx5 with unpriviledged LXC container

[David Marchand]

On Mon, 25 Aug 2025 at 14:09, Julien <julien.marcin.tech@gmail.com> wrote:
>
> Hello,
> I have a question about using the mlx5 driver with LXC.
> I'm trying to use dpdk-testpmd in an LXC container whose root user isn't mapped to the host's root user.
> Note: The entire physical interface is given to the LXC container, not a virtual interface.
>
> The following error occured:
> mlx5_common: DevX create TIS failed errno=22 status=0 syndrome=0
> mlx5_net: Failed to create TIS 0/0 for [bonding] device mlx5_2.
> mlx5_net: TIS allocation failure
> mlx5_net: probe of PCI device 0000:27:00.0 aborted after encountering an error: Cannot allocate memory
> mlx5_common: Failed to load driver mlx5_eth
> EAL: Requested device 0000:27:00.0 cannot be used
> EAL: Bus (pci) probe failed.
>
> The "transport_domain" is created, and the mlx5_devx_cmd_create_td() function runs normally.
> The call to mlx5dv_devx_obj_create() receives an errno of 22.
>
> I don't encounter any problems when the container's root user is mapped to the host's root user.
> Has anyone experienced this before?
> Is it possible to use the driver in an unprivileged LXC container?

There is probably something missing in terms of capabilities.
I don't know how LXC behaves in this regard.

I suggest you look at "5.5.1.5. Run as Non-Root" in
https://doc.dpdk.org/guides/platform/mlx5.html.


-- 
David Marchand