From: Bruce Richardson <bruce.richardson@intel.com>
To: "Zhang, Qi Z" <qi.z.zhang@intel.com>
Cc: "Koikkara Reeny, Shibin" <shibin.koikkara.reeny@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Burakov, Anatoly" <anatoly.burakov@intel.com>,
"Loftus, Ciara" <ciara.loftus@intel.com>
Subject: Re: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration
Date: Thu, 2 Feb 2023 15:18:36 +0000 [thread overview]
Message-ID: <Y9vUTPCMyDIrqN+Y@bricha3-MOBL.ger.corp.intel.com> (raw)
In-Reply-To: <DM4PR11MB59949BAF28DDD53EA310F7B2D7D69@DM4PR11MB5994.namprd11.prod.outlook.com>
On Thu, Feb 02, 2023 at 02:49:52PM +0000, Zhang, Qi Z wrote:
>
>
> > -----Original Message-----
> > From: Koikkara Reeny, Shibin <shibin.koikkara.reeny@intel.com>
> > Sent: Thursday, January 19, 2023 11:10 PM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>; dev@dpdk.org; Burakov, Anatoly
> > <anatoly.burakov@intel.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>
> > Cc: Loftus, Ciara <ciara.loftus@intel.com>
> > Subject: RE: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration
> >
> >
> > > -----Original Message-----
> > > From: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > Sent: Wednesday, January 18, 2023 12:10 PM
> > > To: Koikkara Reeny, Shibin <shibin.koikkara.reeny@intel.com>;
> > > dev@dpdk.org; Burakov, Anatoly <anatoly.burakov@intel.com>;
> > > Richardson, Bruce <bruce.richardson@intel.com>
> > > Cc: Loftus, Ciara <ciara.loftus@intel.com>
> > > Subject: RE: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Koikkara Reeny, Shibin <shibin.koikkara.reeny@intel.com>
> > > > Sent: Wednesday, December 14, 2022 11:41 PM
> > > > To: dev@dpdk.org; Burakov, Anatoly <anatoly.burakov@intel.com>;
> > > > Richardson, Bruce <bruce.richardson@intel.com>
> > > > Cc: Loftus, Ciara <ciara.loftus@intel.com>; Zhang, Qi Z
> > > > <qi.z.zhang@intel.com>; Koikkara Reeny, Shibin
> > > > <shibin.koikkara.reeny@intel.com>
> > > > Subject: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration
> > > >
> > > > Integrate support for the AF_XDP CNI and device plugin [1] so that
> > > > the DPDK AF_XDP PMD can work in an unprivileged container
> > environment.
> > > > Part of the AF_XDP PMD initialization process involves loading an
> > > > eBPF program onto the given netdev. This operation requires
> > > > privileges, which prevents the PMD from being able to work in an
> > > > unprivileged container (without root access). The plugin CNI handles
> > > > the program loading. CNI open Unix Domain Socket (UDS) and waits
> > > > listening for a client to make requests over that UDS. The
> > > > client(DPDK) connects and a "handshake" occurs, then the File
> > > > Descriptor which points to the XSKMAP associated with the loaded
> > > > eBPF program is handed over to the client. The client can then
> > > > proceed with creating an AF_XDP socket and inserting the socket into
> > > > the XSKMAP pointed to by the FD received on the
> > > UDS.
> > > >
> > > > A new vdev arg "use_cni" is created to indicate user wishes to run
> > > > the PMD in unprivileged mode and to receive the XSKMAP FD from the
> > CNI.
> > > > When this flag is set, the XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD
> > > > libbpf flag should be used when creating the socket, which tells
> > > > libbpf not to load the default libbpf program on the netdev. We tell
> > > > libbpf not to do this because the loading is handled by the CNI in this
> > scenario.
> > > >
> > > > [1]: https://github.com/intel/afxdp-plugins-for-kubernetes
> > > >
> > > > Signed-off-by: Shibin Koikkara Reeny
> > > > <shibin.koikkara.reeny@intel.com>
> > > > ---
> > > > drivers/net/af_xdp/rte_eth_af_xdp.c | 337
> > > > +++++++++++++++++++++++++++-
> > > > 1 file changed, 325 insertions(+), 12 deletions(-)
> > > >
> > > > diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c
> > > > b/drivers/net/af_xdp/rte_eth_af_xdp.c
> > > > index b6ec9bf490..196d98ad97 100644
> > > > --- a/drivers/net/af_xdp/rte_eth_af_xdp.c
> > > > +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
> > > > @@ -7,6 +7,7 @@
> > > > #include <string.h>
> > > > #include <netinet/in.h>
> > > > #include <net/if.h>
> > > > +#include <sys/un.h>
> > > > #include <sys/socket.h>
> > > > #include <sys/ioctl.h>
> > > > #include <linux/if_ether.h>
> > > > @@ -81,6 +82,24 @@ RTE_LOG_REGISTER_DEFAULT(af_xdp_logtype,
> > > > NOTICE);
> > > >
> > > > #define ETH_AF_XDP_MP_KEY "afxdp_mp_send_fds"
> > > >
> > > > +#define MAX_LONG_OPT_SZ 64
> > > > +#define UDS_MAX_FD_NUM 2
> > > > +#define UDS_MAX_CMD_LEN 64
> > > > +#define UDS_MAX_CMD_RESP 128
> > > > +#define UDS_XSK_MAP_FD_MSG "/xsk_map_fd"
> > > > +#define UDS_SOCK "/tmp/afxdp.sock"
> > > > +#define UDS_CONNECT_MSG "/connect"
> > > > +#define UDS_HOST_OK_MSG "/host_ok"
> > > > +#define UDS_HOST_NAK_MSG "/host_nak"
> > > > +#define UDS_VERSION_MSG "/version"
> > > > +#define UDS_XSK_MAP_FD_MSG "/xsk_map_fd"
> > > > +#define UDS_XSK_SOCKET_MSG "/xsk_socket"
> > > > +#define UDS_FD_ACK_MSG "/fd_ack"
> > > > +#define UDS_FD_NAK_MSG "/fd_nak"
> > > > +#define UDS_FIN_MSG "/fin"
> > > > +#define UDS_FIN_ACK_MSG "/fin_ack"
> > > > +
> > > > +
> > > > static int afxdp_dev_count;
> > > >
> > > > /* Message header to synchronize fds via IPC */ @@ -151,6 +170,7 @@
> > > > struct pmd_internals {
> > > > char prog_path[PATH_MAX];
> > > > bool custom_prog_configured;
> > > > bool force_copy;
> > > > + bool use_cni;
> > > > struct bpf_map *map;
> > > >
> > > > struct rte_ether_addr eth_addr;
> > > > @@ -170,6 +190,7 @@ struct pmd_process_private {
> > > > #define ETH_AF_XDP_PROG_ARG "xdp_prog"
> > > > #define ETH_AF_XDP_BUDGET_ARG
> > > "busy_budget"
> > > > #define ETH_AF_XDP_FORCE_COPY_ARG "force_copy"
> > > > +#define ETH_AF_XDP_USE_CNI_ARG "use_cni"
> > > >
> > > > static const char * const valid_arguments[] = {
> > > > ETH_AF_XDP_IFACE_ARG,
> > > > @@ -179,8 +200,8 @@ static const char * const valid_arguments[] = {
> > > > ETH_AF_XDP_PROG_ARG,
> > > > ETH_AF_XDP_BUDGET_ARG,
> > > > ETH_AF_XDP_FORCE_COPY_ARG,
> > > > - NULL
> > > > -};
> > > > + ETH_AF_XDP_USE_CNI_ARG,
> > > > + NULL};
> > > >
> > > > static const struct rte_eth_link pmd_link = {
> > > > .link_speed = RTE_ETH_SPEED_NUM_10G, @@ -1129,7 +1150,8
> > @@
> > > > xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals,
> > > > ret = xsk_umem__create(&umem->umem, base_addr,
> > > umem_size,
> > > > &rxq->fq, &rxq->cq, &usr_config);
> > > > if (ret) {
> > > > - AF_XDP_LOG(ERR, "Failed to create umem\n");
> > > > + AF_XDP_LOG(ERR, "Failed to create umem [%d]:
> > > > [%s]\n",
> > > > + errno, strerror(errno));
> > > > goto err;
> > > > }
> > > > umem->buffer = base_addr;
> > > > @@ -1314,6 +1336,245 @@ configure_preferred_busy_poll(struct
> > > > pkt_rx_queue *rxq)
> > > > return 0;
> > > > }
> > > >
> > > > +static int
> > > > +init_uds_sock(struct sockaddr_un *server) {
> > > > + int sock;
> > > > +
> > > > + sock = socket(AF_UNIX, SOCK_SEQPACKET, 0);
> > > > + if (sock < 0) {
> > > > + AF_XDP_LOG(ERR, "Failed to opening stream socket\n");
> > > > + return -1;
> > > > + }
> > > > +
> > > > + server->sun_family = AF_UNIX;
> > > > + strlcpy(server->sun_path, UDS_SOCK, sizeof(server->sun_path));
> > > > +
> > > > + if (connect(sock, (struct sockaddr *)server, sizeof(struct
> > > > sockaddr_un)) < 0) {
> > >
> > > seems the server address is hard coded as "/tmp/afxdp.sock", is any
> > > spec we follows, or should we parse this as a devargs?
> > > better add some comment or external link that help to explain this
> >
> > It was already hardcoded in the afxdp-plugins
> > https://github.com/intel/afxdp-plugins-for-
> > kubernetes/blob/main/constants/constants.go .
>
> OK, I saw this has been explained in your new doc patch.
>
> Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>
>
I would suggest the doc patch should be merged into this patch - code and
doc should go together. Please do a v3 with both patches together, thanks.
Assuming Qi is ok with it, you can probably keep his ack on the new
version.
/Bruce
next prev parent reply other threads:[~2023-02-02 15:18 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <to=20221213104746.805221-1-shibin.koikkara.reeny@intel.com>
2022-12-14 15:41 ` Shibin Koikkara Reeny
2023-01-18 12:10 ` Zhang, Qi Z
2023-01-19 15:10 ` Koikkara Reeny, Shibin
2023-02-02 14:49 ` Zhang, Qi Z
2023-02-02 15:18 ` Bruce Richardson [this message]
2023-02-02 17:02 ` Koikkara Reeny, Shibin
2023-01-25 16:43 ` Burakov, Anatoly
2022-12-14 15:34 Shibin Koikkara Reeny
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y9vUTPCMyDIrqN+Y@bricha3-MOBL.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=ciara.loftus@intel.com \
--cc=dev@dpdk.org \
--cc=qi.z.zhang@intel.com \
--cc=shibin.koikkara.reeny@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).