Re: Failed to load eBPF byte-code on TAP device

[Stephen Hemminger <stephen@networkplumber.org>]

On Thu, 16 Nov 2023 13:41:27 +0530
madhukar mythri <madhukar.mythri@gmail.com> wrote:

> Hi Stephen,
> 
> I had added some logs in the BPF verifier of Kernel code, to print the
> number of instructions processed and error-code returned as follows:
> ============================
> logs # dmesg |tail -n 20
> [   76.318101] #### do_check: instructions Processed 999989 insn
> [   76.318102] #### do_check: instructions Processed 999990 insn
> [   76.318103] #### do_check: instructions Processed 999991 insn
> [   76.318104] #### do_check: instructions Processed 999992 insn
> [   76.318105] #### do_check: instructions Processed 999993 insn
> [   76.318106] #### do_check: instructions Processed 999994 insn
> [   76.318107] #### do_check: instructions Processed 999995 insn
> [   76.318108] #### do_check: instructions Processed 999996 insn
> [   76.318109] #### do_check: instructions Processed 999997 insn
> [   76.318110] #### do_check: instructions Processed 999998 insn
> [   76.318111] #### do_check: instructions Processed 999999 insn
> [   76.318112] #### do_check: instructions Processed 1000000 insn
> [   76.318113] BPF program is too large. Processed 1000001 insn
> [   76.318209] ########## bpf_check:  do_check_main done..: ret: -7
> [   76.318210] ########## bpf_check:  bpf_prog_offload_finalize done..:
> ret: -7
> [   76.318212] ########## bpf_check:  check_max_stack_depth done..: ret: -7
> [   76.318212] ########## bpf_check:  fixup_call_args done..: ret: -7
> [   76.318224] ########## bpf_check:  end..: ret: -7
> [   76.318224] ##########  BPF  bpf_check return err: -7..:
> =========================
> 
> Only these logs which I add in the Kernel-code were printed and do not see
> any other Kernel-logs.
> 
> Thanks,
> Madhuker.
> 
> On Wed, Nov 15, 2023 at 8:49 PM Stephen Hemminger <
> stephen@networkplumber.org> wrote:  
> 
> > On Wed, 15 Nov 2023 15:38:55 +0530
> > madhukar mythri <madhukar.mythri@gmail.com> wrote:
> >  
> > > Hi all,
> > >
> > > On the RHEL9.2 with DPDK 22.11.1 version, DPDK primary application failed
> > > to add RSS flow on TAP sub-device, when loading the TAP BPF byte-code
> > > instructions.
> > >
> > > This "struct bpf_insn l3_l4_hash_insns[]" array(from file:
> > > drivers/net/tap/tap_bpf_insns.h) is in eBPF bytecode instructions format,
> > > this eBPF failed to load on TAP PMD with the following error:
> > >
> > > =====================
> > > rss_add_actions(): Failed to load BPF section 'l3_l4' (7): Argument list
> > > too long.
> > > net_failsafe: Failed to create a flow on sub_device 1."
> > > =====================
> > > On Kernel-version:  5.15.0 #9 SMP PREEMPT
> > > Arch: x86_64 GNU/Linux
> > >
> > > When added some debug logs on Kernel BPF verifier code, we could see that
> > > instruction processed were reached to 1 Million.
> > > But, the Byte code has only 1698 instructions only. Why the Kernel BPF
> > > verifier is processing beyond 1,698 instructions ?
> > >
> > > The same byte-code(with DPDK-22.11.1) worked well with RHEL8.x and not
> > > working in RHEL-9.x version.
> > >
> > > Does anybody faced such issues ?
> > > Please let me know how to debug such issues on Byte-code.
> > >
> > > Thanks,
> > > Madhukar.  
> >
> > Is there anything in the kernel log?

I suspect a kernel bug.
The kernel BPF API is not stable, and RHEL can and does modify the kernel.
Likely a Redhat bug.
Try with recent TAP fixes (in 23.11-rc3).