Does the mlx5 NIC support reloading

Does the mlx5 NIC support reloading

[jinag]

[-- Attachment #1: Type: text/plain, Size: 275 bytes --]

Hi 
I am verifying the reload function of the mlx5 nic based on dpdk 19.11:
echo 1 > /sys/bus/pci/devices/$pci_address/remove
echo 1 > /sys/bus/pci/rescan
rte_bus_probe();   //
rte_eth_dev_stop(); //
rte_eth_dev_start(); 
    net_mlx5: port 0 TX queue 0 CQ creation failure


[-- Attachment #2: Type: text/html, Size: 702 bytes --]

RE: Does the mlx5 NIC support reloading

[Slava Ovsiienko]

[-- Attachment #1: Type: text/plain, Size: 2120 bytes --]

Hi, Jinag

5 cents to add: please see how FailSafe PMD handles device removal events and probes the devices periodically.
This is good example to understand how device hotplug works (Failsafe uses rte_eal_hotplug_add())
Also,  to add the device the previous instance should closed (RTE_ETH_EVENT_INTR_RMV) shoud be handled accordingly.

With best regards,
Slava

From: Matan Azrad <matan@nvidia.com>
Sent: Monday, August 14, 2023 9:41 AM
To: jinag <15720603159@163.com>; users@dpdk.org; Shahaf Shuler <shahafs@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>
Subject: RE: Does the mlx5 NIC support reloading

Hi Jinag

After plugging out the device from the bus, you need to get the event RTE_ETH_EVENT_INTR_RMV.
You need to listen to this event and close the port when you see it.

After plugging in the device, you need to scan the bus again and attach the mlx5 device, so it will be probed again and a new ethdev port will be created.

Then, you need to reconfigure the port, as regular, and reuse the device.

Thanks,
Matan


From: jinag <15720603159@163.com<mailto:15720603159@163.com>>
Sent: Monday, 14 August 2023 6:07
To: users@dpdk.org<mailto:users@dpdk.org>; Matan Azrad <matan@nvidia.com<mailto:matan@nvidia.com>>; Shahaf Shuler <shahafs@nvidia.com<mailto:shahafs@nvidia.com>>; Slava Ovsiienko <viacheslavo@nvidia.com<mailto:viacheslavo@nvidia.com>>
Subject: Does the mlx5 NIC support reloading

External email: Use caution opening links or attachments

Hi
I am verifying the reload function of the mlx5 nic based on dpdk 19.11:
echo 1 > /sys/bus/pci/devices/$pci_address/remove
echo 1 > /sys/bus/pci/rescan
rte_bus_probe();
rte_eth_dev_stop();
rte_eth_dev_start();
    net_mlx5: port 0 TX queue 0 CQ creation failure
    net_mlx5: port 0 TX queue allocation failed: cannot allocate memory
The nic cannot be reinitialized.

I am not sure if the above operation is correct. Could you please tell me if the mlx5 nic supports reloading(for example, the network card is down during normal  operation) and which dpdk functions need to be called.

Thanks!




[-- Attachment #2: Type: text/html, Size: 11311 bytes --]

RE: Does the mlx5 NIC support reloading

[Matan Azrad]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

Hi Jinag

After plugging out the device from the bus, you need to get the event RTE_ETH_EVENT_INTR_RMV.
You need to listen to this event and close the port when you see it.

After plugging in the device, you need to scan the bus again and attach the mlx5 device, so it will be probed again and a new ethdev port will be created.

Then, you need to reconfigure the port, as regular, and reuse the device.

Thanks,
Matan

From: jinag <15720603159@163.com>
Sent: Monday, 14 August 2023 6:07
To: users@dpdk.org; Matan Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>
Subject: Does the mlx5 NIC support reloading

External email: Use caution opening links or attachments

Hi
I am verifying the reload function of the mlx5 nic based on dpdk 19.11:
echo 1 > /sys/bus/pci/devices/$pci_address/remove
echo 1 > /sys/bus/pci/rescan
rte_bus_probe();
rte_eth_dev_stop();
rte_eth_dev_start();
    net_mlx5: port 0 TX queue 0 CQ creation failure
    net_mlx5: port 0 TX queue allocation failed: cannot allocate memory
The nic cannot be reinitialized.

I am not sure if the above operation is correct. Could you please tell me if the mlx5 nic supports reloading(for example, the network card is down during normal  operation) and which dpdk functions need to be called.

Thanks!




[-- Attachment #2: Type: text/html, Size: 8903 bytes --]

Does the mlx5 NIC support reloading

[jinag]

[-- Attachment #1: Type: text/plain, Size: 608 bytes --]

Hi 
I am verifying the reload function of the mlx5 nic based on dpdk 19.11:
echo 1 > /sys/bus/pci/devices/$pci_address/remove
echo 1 > /sys/bus/pci/rescan
rte_bus_probe();  
rte_eth_dev_stop(); 
rte_eth_dev_start(); 
    net_mlx5: port 0 TX queue 0 CQ creation failure
    net_mlx5: port 0 TX queue allocation failed: cannot allocate memory
The nic cannot be reinitialized.


I am not sure if the above operation is correct. Could you please tell me if the mlx5 nic supports reloading(for example, the network card is down during normal  operation) and which dpdk functions need to be called.


Thanks!






[-- Attachment #2: Type: text/html, Size: 2112 bytes --]