From: "Loftus, Ciara" <ciara.loftus@intel.com>
To: "Yigit, Ferruh" <ferruh.yigit@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "stephen@networkplumber.org" <stephen@networkplumber.org>,
"Burakov, Anatoly" <anatoly.burakov@intel.com>,
"Singh, Aman Deep" <aman.deep.singh@intel.com>
Subject: RE: [PATCH v4] net/af_xdp: re-enable secondary process support
Date: Fri, 11 Feb 2022 13:01:47 +0000 [thread overview]
Message-ID: <PH0PR11MB4791A2372AF933C5828F86788E309@PH0PR11MB4791.namprd11.prod.outlook.com> (raw)
In-Reply-To: <452f86bf-826a-20c8-93dc-1b8dfee1febe@intel.com>
>
> On 2/11/2022 9:26 AM, Loftus, Ciara wrote:
> >>>
> >>> On 2/10/2022 5:47 PM, Loftus, Ciara wrote:
> >>>>> Subject: Re: [PATCH v4] net/af_xdp: re-enable secondary process
> >> support
> >>>>>
> >>>>> On 2/10/2022 3:40 PM, Loftus, Ciara wrote:
> >>>>>>> Subject: Re: [PATCH v4] net/af_xdp: re-enable secondary process
> >>> support
> >>>>>>>
> >>>>>>> On 2/9/2022 9:48 AM, Ciara Loftus wrote:
> >>>>>>>> Secondary process support had been disabled for the AF_XDP
> PMD
> >>>>>>> because
> >>>>>>>> there was no logic in place to share the AF_XDP socket file
> >> descriptors
> >>>>>>>> between the processes. This commit introduces this logic using
> the
> >>> IPC
> >>>>>>>> APIs.
> >>>>>>>>
> >>>>>>>> Rx and Tx are disabled in the secondary process due to memory
> >>> mapping
> >>>>> of
> >>>>>>>> the AF_XDP rings being assigned by the kernel in the primary
> >> process
> >>>>> only.
> >>>>>>>> However other operations including retrieval of stats are
> permitted.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
> >>>>>>>>
> >>>>>>>
> >>>>>>> Hi Ciara,
> >>>>>>>
> >>>>>>> When I tried to test the patch getting following error [1], it doesn't
> >> look
> >>>>>>> related to this patch but can you help to fix the issue, thanks.
> >>>>>>>
> >>>>>>> [1]
> >>>>>>> libxdp: Couldn't find a BPF file with name xsk_def_xdp_prog.o
> >>>>>>> xsk_configure(): Failed to create xsk socket.
> >>>>>>> eth_rx_queue_setup(): Failed to configure xdp socket
> >>>>>>> Fail to configure port 2 rx queues
> >>>>>>> EAL: Error - exiting with code: 1
> >>>>>>
> >>>>>>
> >>>>>> Hi Ferruh,
> >>>>>>
> >>>>>> This file should be generated when libxdp is compiled.
> >>>>>> Mine is located @ /usr/local/lib/bpf/xsk_def_xdp_prog.o
> >>>>>> Can you check if that file is there for you? It could be in
> >>>>> /usr/local/lib64/bpf/ on your machine.
> >>>>>> What kernel are you running on?
> >>>>>>
> >>>>>
> >>>>> It is in: /usr/local/lib64/bpf/xsk_def_xdp_prog.o
> >>>>>
> >>>>> I had to compile libxdp from source because OS package version was
> old
> >>>>> to work with af_xdp.
> >>>>> Is something required to point location of this file to af_xdp PMD?
> >>>>>
> >>>>> I run kernel:
> >>>>> 5.15.16-200.fc35.x86_64
> >>>>
> >>>> I read through the libxdp code to figure out what happens when
> >> searching
> >>> for the file:
> >>>> https://github.com/xdp-project/xdp-
> >>> tools/blob/v1.2.2/lib/libxdp/libxdp.c#L1055
> >>>>
> >>>> secure_getenv(XDP_OBJECT_ENVVAR) is called which according to the
> >>> README "defaults to /usr/lib/bpf (or /usr/lib64/bpf on systems using a
> split
> >>> library path)".
> >>>> If that fails, BPF_OBJECT_PATH will be searched, which points to
> >>> /usr/lib/bpf
> >>>>
> >>>> I discovered that on my system the getenv() call fails, but the file is
> >>> eventually found because luckily BPF_OBJECT_PATH points to the
> >>> appropriate place for me (lib):
> >>>> https://github.com/xdp-project/xdp-
> tools/blob/v1.2.2/lib/util/util.h#L24
> >>>> I suspect the same failure is happening for you, but since
> >>> BPF_OBJECT_PATH points to lib and not lib64, the file is not found.
> >>>> As a temporary measure can you create a symlink in /usr/local/lib/bpf/
> to
> >>> point to /usr/local/lib/bpf/xsk_def_xdp_prog.o
> >>>> I will investigate the libxdp issue further. Maybe a change is needed in
> >> the
> >>> library. If a change or setup recommendation is needed in DPDK I will
> >> create a
> >>> patch.
> >>>>
> >>>
> >>>
> >>> I don't have XDP_OBJECT_ENVVAR or BPF_OBJECT_PATH environment
> >>> variables set,
> >>> if they should be we should document them.
> >>>
> >>> When I created '/usr/local/lib/bpf/' link, the BPF file found.
> >>> This should be clarified/documented for users.
> >>
> >> Ok. Ideally we shouldn't have to create the symlink. I will look for a better
> >> solution and submit a patch.
> >> The symlink might be a temporary solution if another solution is not
> found.
> >
> > Can you please try setting the environment variable
> LIBXDP_OBJECT_PATH=/usr/local/lib64/bpf/
> > And see if your test works without the symlink?
> > This worked for me and the getenv succeeded.
> > If it works for you too, I'll create a patch for the docs instructing users to do
> the same.
> >
>
> I confirm it works, and +1 to document it.
>
>
> btw, when this environment variable is not set (and no symlink), af_xdp fails
> and testpmd crashes. I think af_xdp failure shouldn't cause a crash in
> testpmd,
> most probably some error checks are needed in the af_xdp driver.
When I trigger the error case in my environment I get a graceful exit:
libxdp: Couldn't find a BPF file with name xsk_def_xdp_prog.o
xsk_configure(): Failed to create xsk socket.
eth_rx_queue_setup(): Failed to configure xdp socket
Fail to configure port 0 rx queues
EAL: Error - exiting with code: 1
Cause: Start ports failed
Can you please provide more info on your crash?
>
> > Thanks,
> > Ciara
> >
> >>
> >>>
> >>>
> >>> And still observing following two:
> >>>
> >>> 1) I don't know what following log means:
> >>> Configuring Port 2 (socket 0)
> >>> libbpf: elf: skipping unrecognized data section(7) .xdp_run_config
> >>> libbpf: elf: skipping unrecognized data section(8) xdp_metadata
> >>> libxdp: XDP flag not supported by libxdp.
> >>> libbpf: elf: skipping unrecognized data section(8) xdp_metadata
> >>> libbpf: elf: skipping unrecognized data section(8) xdp_metadata
> >>
> >> I reported this and a patch was submitted to libbpf to demote those logs:
> >> https://www.spinics.net/lists/bpf/msg49140.html
> >> It looks like the patch never made it. I'll chase it up.
> >> Anyway, the logs can be ignored as they are not errors.
> >>
> >>>
> >>> 2) When I try to create two af_xdp interface, I only got one:
> >>> "--vdev net_af_xdp,iface=enp24s0f1 --vdev
> net_af_xdp,iface=enp24s0f0"
> >>
> >> This is also expected as you haven't given each vdev a unique name. Try:
> >> "--vdev net_af_xdp0,iface=enp24s0f1 --vdev
> net_af_xdp1,iface=enp24s0f0"
> >>
> >> Thank you for the testing.
> >>
> >> Ciara
> >>
> >>>
> >>>
> >>> Thanks,
> >>> ferruh
next prev parent reply other threads:[~2022-02-11 13:02 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-10 15:32 [RFC PATCH] net/af_xdp: reenable " Ciara Loftus
2021-12-11 21:49 ` Stephen Hemminger
2022-01-12 7:54 ` [PATCH] net/af_xdp: re-enable " Ciara Loftus
2022-02-04 12:54 ` [PATCH v2] " Ciara Loftus
2022-02-04 14:18 ` Ferruh Yigit
2022-02-07 7:49 ` Loftus, Ciara
2022-02-07 10:27 ` Ferruh Yigit
2022-02-07 11:39 ` Loftus, Ciara
2022-02-08 10:58 ` Ferruh Yigit
2022-02-08 13:48 ` [PATCH v3] " Ciara Loftus
2022-02-08 17:45 ` Stephen Hemminger
2022-02-08 18:00 ` Ferruh Yigit
2022-02-08 18:42 ` Stephen Hemminger
2022-02-08 18:56 ` Ferruh Yigit
2022-02-09 7:41 ` Loftus, Ciara
2022-02-09 9:48 ` [PATCH v4] " Ciara Loftus
2022-02-09 15:29 ` Stephen Hemminger
2022-02-11 13:32 ` Ferruh Yigit
2022-02-09 17:55 ` Ferruh Yigit
2022-02-10 15:08 ` Ferruh Yigit
2022-02-10 15:19 ` Ferruh Yigit
2022-02-10 15:40 ` Loftus, Ciara
2022-02-10 16:06 ` Ferruh Yigit
2022-02-10 17:47 ` Loftus, Ciara
2022-02-10 20:12 ` Ferruh Yigit
2022-02-11 7:28 ` Loftus, Ciara
2022-02-11 9:26 ` Loftus, Ciara
2022-02-11 12:29 ` Ferruh Yigit
2022-02-11 13:01 ` Loftus, Ciara [this message]
2022-02-11 13:07 ` Ferruh Yigit
2022-02-11 15:32 ` Loftus, Ciara
2022-02-16 11:23 ` Loftus, Ciara
2022-02-11 11:31 ` Ferruh Yigit
2022-02-17 12:44 ` David Marchand
2022-02-17 12:47 ` Ferruh Yigit
2022-02-17 12:53 ` David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=PH0PR11MB4791A2372AF933C5828F86788E309@PH0PR11MB4791.namprd11.prod.outlook.com \
--to=ciara.loftus@intel.com \
--cc=aman.deep.singh@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).