Conceptually, for two containers on the same host, how would exchanging traffic work under-the-hood? Specifically, how is the physical NIC involved, if at all?

For example, on a physical host: for TX, a userspace application writes a packet to host memory and pushes its physical address/metadata to the appropriate NIC's TX queue. The NIC uses the physical address + DMA to avoid a copy when serializing/sending. (Similar for RX in the other direction, where the NIC writes to memory.) 

I'm not sure how this would translate to a containerized case on a single host, since traffic shouldn't need to exit and the container network has its own namespace. Say that two different containers have NICs mapped to different PCI addresses (split device). If container A appends to its TX queue, what happens next? 

Thanks again for your help. 

On Tue, Nov 19, 2024 at 2:14 PM Thomas Monjalon <thomas@monjalon.net> wrote:
19/11/2024 22:39, Thea Corinne Rossman:
> This is SO helpful -- thank you so much.
>
> One follow-up question regarding NICs: can multiple containers on the same
> host share the same PCI device? If I have a host NIC with (say) VFIO driver
> binding, do I have to split it with some kind of SR-IOV so that each
> container has its own "NIC" binding? Or, when running DPDK's "devbind"
> script, can I set up each one with the same PCI address?

You need to split devices.
SR-IOV VF may help, or you can use SF which was designed exactly for this,
with the help of the Linux auxiliary bus.