Re: [dpdk-dev] [PATCH 0/8] eBPF arm64 JIT support

["Ananyev, Konstantin" <konstantin.ananyev@intel.com>]

Hi everyone,

 
> > >>
> > >> This might be one avenue if all kernel JIT contributors would be on board.
> > >> Another option I'm wondering could be to extend the bpf() syscall in order
> > >> to pass down a description of context and helper mappings e.g. via BTF and
> > >> let everything go through the verifier in the kernel the usual way (I presume
> > >> one goal might be that you want to assure that the generated BPF code passes
> > >> the safety checks before running the prog), then have it JITed and extract
> > >> the generated image in order to use it from user space. Kernel would have
> > >> to make sure it never actually allows attaching this program in the kernel.
> > >> Generated opcodes can already be retrieved today (see below). Such infra
> > >> could potentially help bpf-gcc folks as well as they expressed desire to
> > >> have some sort of a simulator for their gcc BPF test suite.. and it would
> > >> allow for consistent behavior of the BPF runtime. Just a thought.
> > >
> > > This idea looks good. This can remove the verifier code also from DPDK.

Yes, from one side that idea looks very tempting,
As I understand in that case we wouldn't need to worry about licensing,
plus we getting JIT, verifier and might be even cBPF support for free...
As the downside, as Jerin also outlined below - no support for other OSes,
plus in future to get new feature users might need to upgrade to latest kernel.

Just exploring the alternate approach - if we put away for now this licensing hussle,
how difficult you think it would be to make current bpf kernel code sort of
platform independent entity that could be build both as part of kernel and as
standalone user-space lib? 
Again would it be useful for current eBPF users to have an ability to build/run verifier/jit
in user-space too (might be easier to debug, faster prototyping, etc.),
or just extra pain for maintainers? 

Konstantin

> >
> > Right, definitely makes sense to have consolidation also on this one as well
> > aside from the JITs, and pushing to the kernel and receiving back the JITed
> > image seems quite nice and would be generic to open up many other use cases
> > outside of networking. From app pov, it's just an implementation detail where
> > to get that BPF opcode image from anyway.
> >
> > >   A couple of downsides I can think of,
> > >
> > > # We may need to extend the kernel verifier to understand the user-space address
> > > and its symbols for CALL and MEM access operations.
> >
> > Yep, that part would be needed, potentially BTF could be of help here as well
> > for the description of the user space runtime environment like context, helpers
> > etc, so that JIT knows how to handle this.
> 
> Though, We can not conclude the following non-technical aspects in
> this forum like
>  # Dual license Linux Kernel BPF code as GPL-BSD as a separate library.
> #  DPDK BPF support for FreeBSD and Windows OS, Treatment for Other OS?
> # Need a different treatment for old Linux kernels.
> This would call for immediate DPDK release to follow the existing semantics.
> 
> Yes, For the long term, Using the Kernel JITed EBPF program for
> userspace will be helpful. At least DPDK can use it in the future.
> A couple of other things to consider when someone does this
> # https://github.com/iovisor/ubpf can also benefit from this.
> # We need to think about how to support tail call in userspace
> # It is possible to have burst mode support in userspace as an
> improvement (dealing with 4 packets at the time with SIMD).
> Dealing SIMD in kernel space will be an issue or dealing with such
> improvement in general.
> # Kernel verifier and dealing with address has a lot of security
> requirements that may not apply for userspace, and therefore some of
> the
> optimization specific to userspace needs to consider some way
> # Based on my understanding the Linux and DPDK JITed code, following
> optimizations may need/have a different path
> 
> a) Userspace JIT has to deal with a 64bit address space. Kernel BPF
> code can assume the Kernel virtual address space range and optimize.
> b) I see, Kernel JIT always makes stack size as non zero even though
> BPF applications are not using the stack. I am not sure why it
> is that(Could be some security issue). There could be some
> optimization not to push the stack pointer in the prologue if EBPF
> program does
> not use stack
> c) In DPDK JIT compiler, In this first pass, we are checking the
> actual registers really in use and based on that we are crafting
> prologue and
> epilogue at runtime to improve the performance.  It could be just
> implementation detail, but not sure why Linux kernel is not doing such
> optimization.
> 
> Konstantin is the author DPDK EBPF support, I just added arm64 JIT
> support. Maybe he has more data for this direction.
> 
> >
> > > # DPDK supports FreeBSD and Windows OS as well
> >
> > I'm not too familiar with the state on BPF for the latter two, but afaik FreeBSD
> > at least had some effort to implement a BPF runtime into their kernel as well,
> > so similar interface could be provided, but presumably as starting point vast
> > majority of DPDK users are running Linux underneath anyway?
> >
> > > # Need a different treatment for old Linux kernels.
> >
> > Maybe, though I have little insight from DPDK angle here. Wrt BPF and kernel from
> > what we see major cloud providers usually offer quite recent kernels as well as
> > most mainstream distros that are run there, but again, environments where DPDK is
> > typically deployed may differ (?), so cannot really comment.
> 
> 
> 
> 
> 
> 
> 
> >
> > Thanks,
> > Daniel