[dpdk-dev] Legacy virtio vs. multi-process mode

[dpdk-dev] Legacy virtio vs. multi-process mode

[Juho Snellman]

I've been having trouble getting virtio (in legacy mode) working with the
multi-process model. The latest issue is as follows:

- The primary process calls vtpci_init() with a struct
rte_pci_device that's only valid in that process
- For legacy mode vtpci_init() calls rte_eal_pci_ioport() with that pci
device and a the struct rte_pci_ioport that's in shared memory
- The secondary processes will of course not call vtpci_init()
- When a secondary process tries to notify the vqueue in either the the RX
or TX path, it will do so via the shared memory ioport.
- When rte_eal_pci_ioport_write() tries to dispatch the ioport to the right
place, it'll need to access the PCI device. But that's a garbage pointer in
the context of this process. (Usually it's still pointing to some allocated
memory, so you don't even get a segfault but the ioport write is just
silently aborted).
- This causes the queues to stall, unless the primary process happens to
notify them as well for some reason.

Is there a clean solution? My current workaround is to allocate
rte_pci_device structs in shared memory rather than the heap. (Though all
processes still have their own copy). But that requires rearranging the
initialization order in rte_eal_init() to delay the PCI init, and I'm
guessing there's a good reason for the current order.

-- 
Juho Snellman