From: "Zhang, Qi Z" <qi.z.zhang@intel.com>
To: "Koikkara Reeny, Shibin" <shibin.koikkara.reeny@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Burakov, Anatoly" <anatoly.burakov@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>
Cc: "Loftus, Ciara" <ciara.loftus@intel.com>
Subject: RE: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration
Date: Wed, 18 Jan 2023 12:10:29 +0000 [thread overview]
Message-ID: <DM4PR11MB59940603BC7E8C2FA5F17A8DD7C79@DM4PR11MB5994.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20221214154102.1521489-1-shibin.koikkara.reeny@intel.com>
> -----Original Message-----
> From: Koikkara Reeny, Shibin <shibin.koikkara.reeny@intel.com>
> Sent: Wednesday, December 14, 2022 11:41 PM
> To: dev@dpdk.org; Burakov, Anatoly <anatoly.burakov@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>
> Cc: Loftus, Ciara <ciara.loftus@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Koikkara Reeny, Shibin
> <shibin.koikkara.reeny@intel.com>
> Subject: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration
>
> Integrate support for the AF_XDP CNI and device plugin [1] so that the DPDK
> AF_XDP PMD can work in an unprivileged container environment.
> Part of the AF_XDP PMD initialization process involves loading an eBPF
> program onto the given netdev. This operation requires privileges, which
> prevents the PMD from being able to work in an unprivileged container
> (without root access). The plugin CNI handles the program loading. CNI open
> Unix Domain Socket (UDS) and waits listening for a client to make requests
> over that UDS. The client(DPDK) connects and a "handshake" occurs, then
> the File Descriptor which points to the XSKMAP associated with the loaded
> eBPF program is handed over to the client. The client can then proceed with
> creating an AF_XDP socket and inserting the socket into the XSKMAP pointed
> to by the FD received on the UDS.
>
> A new vdev arg "use_cni" is created to indicate user wishes to run the PMD in
> unprivileged mode and to receive the XSKMAP FD from the CNI.
> When this flag is set, the XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD libbpf
> flag should be used when creating the socket, which tells libbpf not to load
> the default libbpf program on the netdev. We tell libbpf not to do this
> because the loading is handled by the CNI in this scenario.
>
> [1]: https://github.com/intel/afxdp-plugins-for-kubernetes
>
> Signed-off-by: Shibin Koikkara Reeny <shibin.koikkara.reeny@intel.com>
> ---
> drivers/net/af_xdp/rte_eth_af_xdp.c | 337
> +++++++++++++++++++++++++++-
> 1 file changed, 325 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c
> b/drivers/net/af_xdp/rte_eth_af_xdp.c
> index b6ec9bf490..196d98ad97 100644
> --- a/drivers/net/af_xdp/rte_eth_af_xdp.c
> +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
> @@ -7,6 +7,7 @@
> #include <string.h>
> #include <netinet/in.h>
> #include <net/if.h>
> +#include <sys/un.h>
> #include <sys/socket.h>
> #include <sys/ioctl.h>
> #include <linux/if_ether.h>
> @@ -81,6 +82,24 @@ RTE_LOG_REGISTER_DEFAULT(af_xdp_logtype,
> NOTICE);
>
> #define ETH_AF_XDP_MP_KEY "afxdp_mp_send_fds"
>
> +#define MAX_LONG_OPT_SZ 64
> +#define UDS_MAX_FD_NUM 2
> +#define UDS_MAX_CMD_LEN 64
> +#define UDS_MAX_CMD_RESP 128
> +#define UDS_XSK_MAP_FD_MSG "/xsk_map_fd"
> +#define UDS_SOCK "/tmp/afxdp.sock"
> +#define UDS_CONNECT_MSG "/connect"
> +#define UDS_HOST_OK_MSG "/host_ok"
> +#define UDS_HOST_NAK_MSG "/host_nak"
> +#define UDS_VERSION_MSG "/version"
> +#define UDS_XSK_MAP_FD_MSG "/xsk_map_fd"
> +#define UDS_XSK_SOCKET_MSG "/xsk_socket"
> +#define UDS_FD_ACK_MSG "/fd_ack"
> +#define UDS_FD_NAK_MSG "/fd_nak"
> +#define UDS_FIN_MSG "/fin"
> +#define UDS_FIN_ACK_MSG "/fin_ack"
> +
> +
> static int afxdp_dev_count;
>
> /* Message header to synchronize fds via IPC */ @@ -151,6 +170,7 @@
> struct pmd_internals {
> char prog_path[PATH_MAX];
> bool custom_prog_configured;
> bool force_copy;
> + bool use_cni;
> struct bpf_map *map;
>
> struct rte_ether_addr eth_addr;
> @@ -170,6 +190,7 @@ struct pmd_process_private {
> #define ETH_AF_XDP_PROG_ARG "xdp_prog"
> #define ETH_AF_XDP_BUDGET_ARG "busy_budget"
> #define ETH_AF_XDP_FORCE_COPY_ARG "force_copy"
> +#define ETH_AF_XDP_USE_CNI_ARG "use_cni"
>
> static const char * const valid_arguments[] = {
> ETH_AF_XDP_IFACE_ARG,
> @@ -179,8 +200,8 @@ static const char * const valid_arguments[] = {
> ETH_AF_XDP_PROG_ARG,
> ETH_AF_XDP_BUDGET_ARG,
> ETH_AF_XDP_FORCE_COPY_ARG,
> - NULL
> -};
> + ETH_AF_XDP_USE_CNI_ARG,
> + NULL};
>
> static const struct rte_eth_link pmd_link = {
> .link_speed = RTE_ETH_SPEED_NUM_10G,
> @@ -1129,7 +1150,8 @@ xsk_umem_info *xdp_umem_configure(struct
> pmd_internals *internals,
> ret = xsk_umem__create(&umem->umem, base_addr,
> umem_size,
> &rxq->fq, &rxq->cq, &usr_config);
> if (ret) {
> - AF_XDP_LOG(ERR, "Failed to create umem\n");
> + AF_XDP_LOG(ERR, "Failed to create umem [%d]:
> [%s]\n",
> + errno, strerror(errno));
> goto err;
> }
> umem->buffer = base_addr;
> @@ -1314,6 +1336,245 @@ configure_preferred_busy_poll(struct
> pkt_rx_queue *rxq)
> return 0;
> }
>
> +static int
> +init_uds_sock(struct sockaddr_un *server) {
> + int sock;
> +
> + sock = socket(AF_UNIX, SOCK_SEQPACKET, 0);
> + if (sock < 0) {
> + AF_XDP_LOG(ERR, "Failed to opening stream socket\n");
> + return -1;
> + }
> +
> + server->sun_family = AF_UNIX;
> + strlcpy(server->sun_path, UDS_SOCK, sizeof(server->sun_path));
> +
> + if (connect(sock, (struct sockaddr *)server, sizeof(struct
> sockaddr_un)) < 0) {
seems the server address is hard coded as "/tmp/afxdp.sock", is any spec we follows, or should we parse this as a devargs?
better add some comment or external link that help to explain this
next prev parent reply other threads:[~2023-01-18 12:10 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <to=20221213104746.805221-1-shibin.koikkara.reeny@intel.com>
2022-12-14 15:41 ` Shibin Koikkara Reeny
2023-01-18 12:10 ` Zhang, Qi Z [this message]
2023-01-19 15:10 ` Koikkara Reeny, Shibin
2023-02-02 14:49 ` Zhang, Qi Z
2023-02-02 15:18 ` Bruce Richardson
2023-02-02 17:02 ` Koikkara Reeny, Shibin
2023-01-25 16:43 ` Burakov, Anatoly
2022-12-14 15:34 Shibin Koikkara Reeny
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM4PR11MB59940603BC7E8C2FA5F17A8DD7C79@DM4PR11MB5994.namprd11.prod.outlook.com \
--to=qi.z.zhang@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=bruce.richardson@intel.com \
--cc=ciara.loftus@intel.com \
--cc=dev@dpdk.org \
--cc=shibin.koikkara.reeny@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).