From: "Loftus, Ciara" <ciara.loftus@intel.com>
To: "Yigit, Ferruh" <ferruh.yigit@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "stephen@networkplumber.org" <stephen@networkplumber.org>,
"Burakov, Anatoly" <anatoly.burakov@intel.com>
Subject: RE: [PATCH v2] net/af_xdp: re-enable secondary process support
Date: Mon, 7 Feb 2022 11:39:58 +0000 [thread overview]
Message-ID: <CO1PR11MB4785373353C24F22563F4C108E2C9@CO1PR11MB4785.namprd11.prod.outlook.com> (raw)
In-Reply-To: <0fa260e7-d5c5-7b92-abd1-b125eb62c6f9@intel.com>
> >>
> >> On 2/4/2022 12:54 PM, Ciara Loftus wrote:
> >>> Secondary process support had been disabled for the AF_XDP PMD
> >>> because there was no logic in place to share the AF_XDP socket
> >>> file descriptors between the processes. This commit introduces
> >>> this logic using the IPC APIs.
> >>>
> >>> Since AF_XDP rings are single-producer single-consumer, rx/tx
> >>> in the secondary process is disabled. However other operations
> >>> including retrieval of stats are permitted.
> >>>
> >>> Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
> >>>
> >>> ---
> >>> v1 -> v2:
> >>> * Rebase to next-net
> >>>
> >>> RFC -> v1:
> >>> * Added newline to af_xdp.rst
> >>> * Fixed spelling errors
> >>> * Fixed potential NULL dereference in init_internals
> >>> * Fixed potential free of address-of expression in
> afxdp_mp_request_fds
> >>> ---
> >>> doc/guides/nics/af_xdp.rst | 9 ++
> >>> doc/guides/nics/features/af_xdp.ini | 1 +
> >>> doc/guides/rel_notes/release_22_03.rst | 1 +
> >>> drivers/net/af_xdp/rte_eth_af_xdp.c | 210
> >> +++++++++++++++++++++++--
> >>> 4 files changed, 207 insertions(+), 14 deletions(-)
> >>>
> >>> diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst
> >>> index db02ea1984..eb4eab28a8 100644
> >>> --- a/doc/guides/nics/af_xdp.rst
> >>> +++ b/doc/guides/nics/af_xdp.rst
> >>> @@ -141,4 +141,13 @@ Limitations
> >>> NAPI context from a watchdog timer instead of from softirqs. More
> >> information
> >>> on this feature can be found at [1].
> >>>
> >>> +- **Secondary Processes**
> >>> +
> >>> + Rx and Tx are not supported for secondary processes due to the
> single-
> >> producer
> >>> + single-consumer nature of the AF_XDP rings. However other
> operations
> >> including
> >>> + statistics retrieval are permitted.
> >>
> >> Hi Ciara,
> >>
> >> Isn't this limitation same for all PMDs, like not both primary & secondary
> can
> >> Rx/Tx
> >> from same queue at the same time.
> >> But primary can initiallize the PMD and secondary can do the datapath,
> >> or isn't af_xdp supports multiple queue, if so some queues can be used
> by
> >> primary and some by secondary for datapath.
> >>
> >> Is there anyhing special for af_xdp that prevents it?
> >
> > Hi Ferruh,
> >
> > Thanks for the review.
> > Each queue of the PMD corresponds to a new AF_XDP socket.
> > Each socket has an RX and TX ring that is mmapped from the kernel to
> userspace and this mapping is only valid for the primary process.
> > I did not figure out a way to share that mapping with the secondary process
> successfully. Can you think of anything that might work?
> >
>
> Does the application knows the buffer address for the Rx/Tx, or is
> abstracted to the 'fd'?
The application knows the buffer address of the Rx/Tx rings.
We pass a pointer to these rings to the libbpf xsk_socket__create API, which sets up the mappings:
http://code.dpdk.org/dpdk/v21.11/source/drivers/net/af_xdp/rte_eth_af_xdp.c#L1291
Then later on in the datapath we operate directly on those rings:
http://code.dpdk.org/dpdk/v21.11/source/drivers/net/af_xdp/rte_eth_af_xdp.c#L268
The fd is used in the datapath, but just for the syscalls (recvfrom/poll/send).
> If only 'fd' is used, this patch already converts 'fd' between
> processes.
> cc'ed Anatoly, but what I understand is after MP fd conversion:
> Primary process: FD=x
> Secondary process: FD=y
> And both x & y points to exact same socket in the kernel side.
>
> At least this is how it works for the 'tap' interface, and that is
> why 'fs' are in the process_private area and converted between primary
> and secondary, I thought it will be same for the xdp socket.
>
> Did you test the secondary Rx/Tx in the secondary after this patch?
next prev parent reply other threads:[~2022-02-07 11:40 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-10 15:32 [RFC PATCH] net/af_xdp: reenable " Ciara Loftus
2021-12-11 21:49 ` Stephen Hemminger
2022-01-12 7:54 ` [PATCH] net/af_xdp: re-enable " Ciara Loftus
2022-02-04 12:54 ` [PATCH v2] " Ciara Loftus
2022-02-04 14:18 ` Ferruh Yigit
2022-02-07 7:49 ` Loftus, Ciara
2022-02-07 10:27 ` Ferruh Yigit
2022-02-07 11:39 ` Loftus, Ciara [this message]
2022-02-08 10:58 ` Ferruh Yigit
2022-02-08 13:48 ` [PATCH v3] " Ciara Loftus
2022-02-08 17:45 ` Stephen Hemminger
2022-02-08 18:00 ` Ferruh Yigit
2022-02-08 18:42 ` Stephen Hemminger
2022-02-08 18:56 ` Ferruh Yigit
2022-02-09 7:41 ` Loftus, Ciara
2022-02-09 9:48 ` [PATCH v4] " Ciara Loftus
2022-02-09 15:29 ` Stephen Hemminger
2022-02-11 13:32 ` Ferruh Yigit
2022-02-09 17:55 ` Ferruh Yigit
2022-02-10 15:08 ` Ferruh Yigit
2022-02-10 15:19 ` Ferruh Yigit
2022-02-10 15:40 ` Loftus, Ciara
2022-02-10 16:06 ` Ferruh Yigit
2022-02-10 17:47 ` Loftus, Ciara
2022-02-10 20:12 ` Ferruh Yigit
2022-02-11 7:28 ` Loftus, Ciara
2022-02-11 9:26 ` Loftus, Ciara
2022-02-11 12:29 ` Ferruh Yigit
2022-02-11 13:01 ` Loftus, Ciara
2022-02-11 13:07 ` Ferruh Yigit
2022-02-11 15:32 ` Loftus, Ciara
2022-02-16 11:23 ` Loftus, Ciara
2022-02-11 11:31 ` Ferruh Yigit
2022-02-17 12:44 ` David Marchand
2022-02-17 12:47 ` Ferruh Yigit
2022-02-17 12:53 ` David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CO1PR11MB4785373353C24F22563F4C108E2C9@CO1PR11MB4785.namprd11.prod.outlook.com \
--to=ciara.loftus@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).