Linux / MLX4 / ConnectX-3 / secondary process support

[Michael Mason <michael.l.mason@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]

We’re running the configuration in the subject line but we need secondary process support.  however, runtime console messages and the mlx4 docs indicate that mlx4 does not explicitly support this approach.  (Even though I was able to run the multi_process/simple_mp example successfully earlier today so that’s confusing …)

Our options appear to be: (1) use a different NIC (2) change sw architecture (move away from using the primary/secondary process approach)

There is a description of how to get around this in one of the commit messages for mlx4, and I’ve pasted it below:

“ External memory unregistered in EAL memseg list cannot be used for DMA unless such memory has been registered by mlx4_mr_update_ext_mp() in primary process and remapped to the same virtual address in secondary process. If the external memory is registered by primary process but has different virtual address in secondary process, unexpected error may happen”


I think I understand most of this, but does someone have an example for how to do this?  Or could someone add some more details to this paragraph?   Wrt remapping the address, is that just as simple as using the DPDK IPC to send a message from primary->secondary containing the correct virtual address?  Lastly, how would this additional process be created?  Using something outside of the DPDK environment?

Thanks everyone!

[-- Attachment #2: Type: text/html, Size: 3077 bytes --]