ethdev use from secondary process

[Ashish Sadanandan <ashish.sadanandan@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2734 bytes --]

Hi everyone,
I'm trying to use a secondary process to rx/tx packets through a NIC and
running into segfaults due to some data structures not being populated on
the secondary. I've figured out a solution, but it involves the use of
internal APIs and I wanted to check with the community if there's a better
way to do this.

I'm using DPDK 24.11 LTS for this test. The test code is at
https://gist.github.com/praetorian20/0c1b69abbc7843d958da72fdf611a5d7. I'll
describe the steps I'm following here.

Primary process:

   1. Command line: sudo LD_LIBRARY_PATH=/path/to/dpdk/shared/libs/
   ./test-ethdev --main-lcore 0 --file-prefix=mpdemo --socket-mem=1024 -d
   /path/to/dpdk/shared/libs/dpdk/pmds-25.0 --proc-type primary
   2. Calls rte_eal_init
   3. Registers message handler using  rte_mp_action_register
   4. Waits indefinitely for CTRL+C

Secondary process:

   1. Command line: Same as primary, except --proc-type=secondary
   2. Calls rte_eal_init
   3. Registers message handler using  rte_mp_action_register
   4. Creates a mempool using rte_pktmbuf_pool_create
   5. Sends request message to primary using  rte_mp_request_async

Primary process (upon receiving the async request):

   1. Look up the mempool created by the secondary
   2. Get port id for a particular ethdev using PCI address
   3. Call rte_eth_dev_configure,  rte_eth_rx_queue_setup,
rte_eth_tx_queue_setup
   and  rte_eth_dev_start to start the ethdev
   4. Send response to secondary

Secondary process (upon receiving response):

   1. Call  rte_eth_dev_get_port_by_name to fetch port id
   2. Call  rte_eth_dev_attach_secondary and  rte_eth_dev_probing_finish to
   initialize data structures. Without this the next step has segv
   because rte_eth_fp_ops.rxq.data is nullptr
   3. Call rte_eth_rx_burst to receive packets


The above works but I have a few questions whether this process can be
improved. I'm using an Nvidia ConnectX6 NIC (mlx5 driver) in case that
matters.


   - Are there public APIs I can call instead of
   rte_eth_dev_attach_secondary and  rte_eth_dev_probing_finish?
   - Is it possible to perform rte_eth_dev_configure, tx/rx queue setup and
   rte_eth_dev_start from the secondary process? These functions return
   various error codes when I try this.
   - When I call rte_eth_dev_start on the primary, it's sending a message
   to the secondary that goes unhandled. Is this a problem, and possibly the
   reason for the unpopulated rte_eth_fp_ops on the secondary? This is the
   error message I see from the primary:

EAL: Fail to recv reply for request
> /var/run/dpdk/mpdemo/mp_socket_1094695_b7080eff23acd4:common_mlx5_mp
> mlx5_net: port 2 failed to request stop/start Rx/Tx (5)


 Thanks for all your help,
Ashish

[-- Attachment #2: Type: text/html, Size: 3911 bytes --]