Re: RFC - Tap io_uring PMD

[Stephen Hemminger <stephen@networkplumber.org>]

On Wed, 6 Nov 2024 08:46:55 +0100
Maxime Coquelin <maxime.coquelin@redhat.com> wrote:

> Hi Stephen,
> 
> On 10/30/24 22:56, Stephen Hemminger wrote:
> > The current tap device is slow both due to architectural choices and the
> > overhead of Linux system calls. I am exploring a how to fix that but some
> > of the choices require some tradeoffs. Which leads to some open questions:
> > 
> > 1. DPDK tap also support tunnel (TUN) mode where there is no Ethernet header
> >     only L3. Does anyone actually use this? It is different than what every other
> >     PMD expects.
> > 
> > 2. The fastest way to use kernel TAP device would be to use io_uring.
> >     But this was added in 5.1 kernel (2019). Rather than having conditional or
> >     dual mode in DPDK tap device, perhaps there should just be a new PMD tap_uring?
> > 
> > 3. Current TAP device provides hooks for several rte_flow types by playing
> >     games with kernel qdisc. Does anyone really use this? Propose just not doing
> >     this in new tap_uring.
> > 
> > 4. What other features of TAP device beyond basic send/receive make sense?
> >     It looks like new device could support better statistics.
> > 
> > 5. What about Rx interrupt support?
> > 
> > Probably the hardest part of using io_uring is figuring out how to collect
> > completions. The simplest way would be to handle all completions rx and tx
> > in the rx_burst function.
> >   
> 
> Why not just use Virtio-user PMD with Vhost-kernel backend [0]?
> Are there any missing features that io_uring can address?
> 
> Regards,
> Maxime
> 
> [0]: http://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html
> 

Yes, I looked at that but:
  - virtio user ends up with a busy kernel thread which is not acceptable
    in SOC environment where all resources are locked down. In the SOC I was working
    on DPDK was limited to 4 polling isolated CPU's and 1 sleeping main thread.
    The rest of the CPU resources were hard constrained by cgroups. The virtio user
    thread was a problem.

  - virtio user device is not persistent. If DPDK is being used a dataplane, need to
    be able to quickly restart and not disturb applications and routing in the kernel
    while the tap device is unavailable.  I.e having device present but in no carrier
    state is better than having to teach applications about hot plug or play around
    with multiple addresses on loopback device (which is what Cisco routers do).