From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 191A94241A; Thu, 19 Jan 2023 16:17:15 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AE6D74067E; Thu, 19 Jan 2023 16:17:14 +0100 (CET) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 9C9EB40223 for ; Thu, 19 Jan 2023 16:17:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1674141432; x=1705677432; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=RkAEC0tou/furVCzEBstm2cwBSs2KBzAzFTC85oSaL4=; b=N2g02BMLbJKlXxdHG+SVmkxcJtC7LNYekZEqTFyXl+r+MLpLs4vCjjwN 0FbNQ5TjvMiieV2SRxygJZgnI75YfPigBgU0RRXWH73M/s0tz2jTXjBwO pzBorUnbSTwW1Se6trku0NfyQQWyRhO2UiYnc26+2ZZBNdaL8jzlgVW0P wfmVQftBc7rv48dLkn6SiM6tZJVsw2kb4hNl3k9e2fiJGgl3y6FoFX2Zs 4fy9esf55J51d1oV60rx3PC7XqWl5gPIFjxPRetg3z5ZI6x3ZKRE7CYnD 4avu+xZldkpK1/CltWfZjMm5MfDG59BkP2cvxqidzKYje6RcR2EL+pqyM A==; X-IronPort-AV: E=McAfee;i="6500,9779,10595"; a="322996838" X-IronPort-AV: E=Sophos;i="5.97,229,1669104000"; d="scan'208";a="322996838" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Jan 2023 07:10:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10595"; a="833998665" X-IronPort-AV: E=Sophos;i="5.97,229,1669104000"; d="scan'208";a="833998665" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orsmga005.jf.intel.com with ESMTP; 19 Jan 2023 07:10:06 -0800 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Thu, 19 Jan 2023 07:10:06 -0800 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16 via Frontend Transport; Thu, 19 Jan 2023 07:10:06 -0800 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.106) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.16; Thu, 19 Jan 2023 07:10:06 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VB3pGRit6+vJzaTETZmIsu5vNs0IQiV4mgtxdj/QYgbfJqhwOL6QoQb0+aD8vv9EHMReedUm+jgCrnmzVwfEr+huL6xuzv7zeiZunkt9xvU3+O0T1JmbdFYaPVzEgXOxQUvPgJiUijPqVX4Nqgks06gISUgZHxH+LGGRUBmKjcn2IF2B0XEL7oCyqnCYSci7SjdSl4xWZPOBMfxS17t9C0376GlSsZQ0zdNyYhotPso1/spFZeRIQovvrsLquzm1QrpqMpDlZw2nouWdhjuV6itwyIBD+m1Ut4HIZXInDYpfDQNut7J9hCepvY9k0ZBY4RnVo3iGgKjs85kqGI8OBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Is+R6m5NOqK5UOXEBO/hf/bGs+8FsvNOqWS8VfppyTQ=; b=EjjR+sRzwoT/RvQ7nLtnLeEYjYhTVUYzJfw8lqMQMxipsvFJemZkOx3sgp7qJ8m5DV2r/A42C5RxeoXpLin7dgzYg0R8+qxfWB/MLn/r8eHqZa7xbTzqeGJTYEowPDm3AARJo0M3mhkl8aYGIrnaf/u3P1rXLbp8LmKdjEYqwTrXQqaa/ceEWCpBsB6G+y2wudDVcZiDu7zx8zI3qxV+LvlYjrX/s4ejaO6RxXVjzPNRgR+26+KGCKub4NrEeNfEatD5eb33qxI3S//e5/UknTgIQeCTPPkpnTrYWaotbMavqHMZ99pQLkDP1mT53u/8P0/8ks6GzdLkEVjW6cdZAQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM6PR11MB3995.namprd11.prod.outlook.com (2603:10b6:5:6::12) by IA1PR11MB7271.namprd11.prod.outlook.com (2603:10b6:208:429::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13; Thu, 19 Jan 2023 15:10:04 +0000 Received: from DM6PR11MB3995.namprd11.prod.outlook.com ([fe80::a995:2442:6c1d:5cbb]) by DM6PR11MB3995.namprd11.prod.outlook.com ([fe80::a995:2442:6c1d:5cbb%4]) with mapi id 15.20.5986.023; Thu, 19 Jan 2023 15:10:04 +0000 From: "Koikkara Reeny, Shibin" To: "Zhang, Qi Z" , "dev@dpdk.org" , "Burakov, Anatoly" , "Richardson, Bruce" CC: "Loftus, Ciara" Subject: RE: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration Thread-Topic: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration Thread-Index: AQHZD9KLSiNeohomUU6tmdIpuzHyRa6kTBCAgAG80FA= Date: Thu, 19 Jan 2023 15:10:03 +0000 Message-ID: References: <20221214154102.1521489-1-shibin.koikkara.reeny@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM6PR11MB3995:EE_|IA1PR11MB7271:EE_ x-ms-office365-filtering-correlation-id: 77d74a4a-5dbe-4988-7ebe-08dafa2f3e1c x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: D65a4PpQFlnsRe8NUauj5x0SuZANTPwxTUE4GJj948uuK1+2QrGXshaurqafNRlzN2A6YoFFfg0EQWUQx/hYyNrDJrKAv8Np4XLwd+nHiCRDMPNxosYGmNDl6z5OSSObTtLnChzq3t7oRLyKp91l1HGo1nP2PXsTOzfrmnfv+ApkUozn2+Na5i3EoHOgEgGCwBo28WUx8xwZ9UodJk74aV23LpoCp41hSK9EWXIwYOTMqP6xjsa3BSbfGwEdqFRxCVZMSRCpvGJif5LR1n51SQrKlFUxU1onVi9N2DbadxoMhdB+N6yGV8Y8hfNoHyFbV3oY9zk+p+4a1tzAgGI1VkLqAZoWB8g0n9jjiIQLuDWtwLKlcVKGOBnjHoxGNoMwo82iG0eW07UMwGfD1DVA10UYTsFlHFBTHdUo9n9+Io7sIX9Q7jBFzrI+bVlJYQA8nQIxke9kv1prIYvAJ/CDQvtcNbqcczCt3xASEZ1AOdgLosXGXF6st/hX/ntmeh8RgW4d+gM3Gb7ePTthhtgn2jT8qXsmyh56F6Ha54+EZcDk8dNK1HYKXagS9L6i4wxgsLIuunWsrIBpM+uHIYKu8qSySJwhHJzQmVO4HL/+QL4cgZm1vuqZJyKhQuYYcM4X7qUbx3p7J0gaCQSxGrTlzaITRxlDQ9+mKg0pbOqOBcA99llDxmH5wp8/vRJCXkwb+hAidVxfddnqMIutmIhZPOjBVWazu/XbgIDE543ikBInwKHyGkgVli0JkoZnMuJTDVceWFp3YADrXGsN2qev9A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB3995.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(346002)(39860400002)(396003)(366004)(376002)(136003)(451199015)(82960400001)(38100700002)(6506007)(122000001)(5660300002)(86362001)(966005)(2906002)(8936002)(33656002)(76116006)(52536014)(66946007)(64756008)(4326008)(55016003)(41300700001)(66476007)(71200400001)(26005)(186003)(66556008)(66446008)(9686003)(83380400001)(7696005)(316002)(107886003)(6636002)(110136005)(38070700005)(8676002)(53546011)(478600001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?ft4uYJ9fleaRrjNJS3nCWkR58x/KTtVxMKOxucuRk9k+PWsSZ/vN2mFxwz0Z?= =?us-ascii?Q?Z+oN83kJC6BezRIU8owBM8TbqcJ/VIRwlSPjyrutFRo7roY7nmABLcuI1+UL?= =?us-ascii?Q?VYinqA1rz2kGikNEu/gAOCucnkyAia7DlKQ3+623i6/oAmDKFWyUkkDNvUnk?= =?us-ascii?Q?BRz4HRHPNi8FJJ0Xrm4y1s9zOCb6ip9JWvzhYQE4IJE2oRlLvV3HlnR9kvaH?= =?us-ascii?Q?l4jMto8bvTCI7GvYQsc8EkPEKvD/fyW9DIbvKxd171nzi5IulNL159RdFo0V?= =?us-ascii?Q?f7PNNCIAW4+FRSK9hQ2bAIhMd184Zeq3h7jWJtL/frz1dxHXTTHVZKKn21Zf?= =?us-ascii?Q?ni500/MiVIsx4HAKV94CnDlr06cbs4wt87JIx2g/gFDh/tXUCGRnMCjBv3M+?= =?us-ascii?Q?mO13a9JsCPKTajWb1HSIwjik/18+yNsoatzIILFWScQkyUOd/h9A6hXU1Euk?= =?us-ascii?Q?Gj5NXC9c+LlKMLZ5OW+VDCsr17ylcUet7hYASaVT76L1D2dSfu/zBv/QltHL?= =?us-ascii?Q?+ADpe1NfxWX70ZqYeUco/ioDPv+LjQKpMnFA291Svhc0poOoVjaMcY30Ek0/?= =?us-ascii?Q?YCVBe2Mtp99K/kM8bdVrCLKsBng6lUDPUDL2N4+ydE5BaW+6P7HbqdIUXsrH?= =?us-ascii?Q?+aMDm9wBGj1/vnAMv4+3Unm36WdF4E2+HJ6UVXRvQ2wbovtJYr4NiymZCDZd?= =?us-ascii?Q?tBTTmFV4KNPFghzIaFL6H/lAVl8HT/uu/QOUHmdyAxsIyIAP1odufQzMJQYI?= =?us-ascii?Q?bYD0g6i66RN5svEWJR4c/kBeYHmLrxJjAZXDtGvgThB2zJX/qTIvxB+P8ao+?= =?us-ascii?Q?+GTHNt1Ue7bXobWQnxHZwZvOF92w5fbI0kcfQtzKSM6Vvk912STYEwWhqSX7?= =?us-ascii?Q?0+o0MtfgLB2/8orEnxnQC+IqywReDGVWg7bDq9PHfJ6JryhJRqDoAxZv/m67?= =?us-ascii?Q?dSagrOUnK5/M6XCCAfawKBQuvCyNabmyZtSULAUD31flt27L2RSZM+Gi0AXQ?= =?us-ascii?Q?aur8d2zI3W3+1HtpI5BNbyJVR+TdtgtLBwRJb+oR/zNjThfiZdOVAI5tgCId?= =?us-ascii?Q?VR7f+8PumaqaIg6yYPZXio4zB7+6g5Aft/xErz9rImRWupnHpaiNVm3DspXo?= =?us-ascii?Q?YbmA/GJtMioJMXFFBp3MikDsdn5Vki1w7RH1gVFaZioBynSnorn6iUKNjuBv?= =?us-ascii?Q?/gd+A+nJ3apQ4Lk0nOU6fX5opFv94ABwLPFFiTT39Bf66nTneFaw/sPWn61i?= =?us-ascii?Q?rw2g6o+fzaon2a5gErkkzl24bWOc15kDHl+LY7J6oJP+KJ4Rg7wqFLnwPsBC?= =?us-ascii?Q?OpPiFH4M+ajOAtCFNZJbutbIh82VFWndDnyXsoqEJZ35KuuIc+6nXNzN7Gg/?= =?us-ascii?Q?FRk4n+VLgj1hZpnCmm0OfROcos5wMMMPe/0OPBScKgyOWWTyBobL6/nXWZ+1?= =?us-ascii?Q?sEXl1DzedNag0eIPDgaI4Z0O1zLK4Kj1EO58VWclCQ67+xDBG4c2hj6gr9h0?= =?us-ascii?Q?GOza6mEX1gPlJWMdmjXMPCrcJf8PwUerpb0FX+uUGW24wbsrY9maXuB560qQ?= =?us-ascii?Q?3sT5KWIil4PBgV8nFQjdG1Fga46J7cETs93vTYJ5FBmTV1dzQ92o0WFRZ6pm?= =?us-ascii?Q?ZIf4c/PTWIiUI+ueCamc1Ts=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3995.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 77d74a4a-5dbe-4988-7ebe-08dafa2f3e1c X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jan 2023 15:10:03.8183 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JyHwdE701B7dlHTX3FEyJc//XY/pxb54SCh/+H793JtykqioAqwHcQ2WJVkGco8sPW9gNdVxjGArXeJhv6tQihFofb2fLpTnyClSeUY39kM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB7271 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Zhang, Qi Z > Sent: Wednesday, January 18, 2023 12:10 PM > To: Koikkara Reeny, Shibin ; > dev@dpdk.org; Burakov, Anatoly ; Richardson, > Bruce > Cc: Loftus, Ciara > Subject: RE: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration >=20 >=20 >=20 > > -----Original Message----- > > From: Koikkara Reeny, Shibin > > Sent: Wednesday, December 14, 2022 11:41 PM > > To: dev@dpdk.org; Burakov, Anatoly ; > > Richardson, Bruce > > Cc: Loftus, Ciara ; Zhang, Qi Z > > ; Koikkara Reeny, Shibin > > > > Subject: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration > > > > Integrate support for the AF_XDP CNI and device plugin [1] so that the > > DPDK AF_XDP PMD can work in an unprivileged container environment. > > Part of the AF_XDP PMD initialization process involves loading an eBPF > > program onto the given netdev. This operation requires privileges, > > which prevents the PMD from being able to work in an unprivileged > > container (without root access). The plugin CNI handles the program > > loading. CNI open Unix Domain Socket (UDS) and waits listening for a > > client to make requests over that UDS. The client(DPDK) connects and a > > "handshake" occurs, then the File Descriptor which points to the > > XSKMAP associated with the loaded eBPF program is handed over to the > > client. The client can then proceed with creating an AF_XDP socket and > > inserting the socket into the XSKMAP pointed to by the FD received on t= he > UDS. > > > > A new vdev arg "use_cni" is created to indicate user wishes to run the > > PMD in unprivileged mode and to receive the XSKMAP FD from the CNI. > > When this flag is set, the XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD libbpf > > flag should be used when creating the socket, which tells libbpf not > > to load the default libbpf program on the netdev. We tell libbpf not > > to do this because the loading is handled by the CNI in this scenario. > > > > [1]: https://github.com/intel/afxdp-plugins-for-kubernetes > > > > Signed-off-by: Shibin Koikkara Reeny > > --- > > drivers/net/af_xdp/rte_eth_af_xdp.c | 337 > > +++++++++++++++++++++++++++- > > 1 file changed, 325 insertions(+), 12 deletions(-) > > > > diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c > > b/drivers/net/af_xdp/rte_eth_af_xdp.c > > index b6ec9bf490..196d98ad97 100644 > > --- a/drivers/net/af_xdp/rte_eth_af_xdp.c > > +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c > > @@ -7,6 +7,7 @@ > > #include > > #include > > #include > > +#include > > #include > > #include > > #include > > @@ -81,6 +82,24 @@ RTE_LOG_REGISTER_DEFAULT(af_xdp_logtype, > > NOTICE); > > > > #define ETH_AF_XDP_MP_KEY "afxdp_mp_send_fds" > > > > +#define MAX_LONG_OPT_SZ 64 > > +#define UDS_MAX_FD_NUM 2 > > +#define UDS_MAX_CMD_LEN 64 > > +#define UDS_MAX_CMD_RESP 128 > > +#define UDS_XSK_MAP_FD_MSG "/xsk_map_fd" > > +#define UDS_SOCK "/tmp/afxdp.sock" > > +#define UDS_CONNECT_MSG "/connect" > > +#define UDS_HOST_OK_MSG "/host_ok" > > +#define UDS_HOST_NAK_MSG "/host_nak" > > +#define UDS_VERSION_MSG "/version" > > +#define UDS_XSK_MAP_FD_MSG "/xsk_map_fd" > > +#define UDS_XSK_SOCKET_MSG "/xsk_socket" > > +#define UDS_FD_ACK_MSG "/fd_ack" > > +#define UDS_FD_NAK_MSG "/fd_nak" > > +#define UDS_FIN_MSG "/fin" > > +#define UDS_FIN_ACK_MSG "/fin_ack" > > + > > + > > static int afxdp_dev_count; > > > > /* Message header to synchronize fds via IPC */ @@ -151,6 +170,7 @@ > > struct pmd_internals { > > char prog_path[PATH_MAX]; > > bool custom_prog_configured; > > bool force_copy; > > + bool use_cni; > > struct bpf_map *map; > > > > struct rte_ether_addr eth_addr; > > @@ -170,6 +190,7 @@ struct pmd_process_private { > > #define ETH_AF_XDP_PROG_ARG "xdp_prog" > > #define ETH_AF_XDP_BUDGET_ARG > "busy_budget" > > #define ETH_AF_XDP_FORCE_COPY_ARG "force_copy" > > +#define ETH_AF_XDP_USE_CNI_ARG "use_cni" > > > > static const char * const valid_arguments[] =3D { > > ETH_AF_XDP_IFACE_ARG, > > @@ -179,8 +200,8 @@ static const char * const valid_arguments[] =3D { > > ETH_AF_XDP_PROG_ARG, > > ETH_AF_XDP_BUDGET_ARG, > > ETH_AF_XDP_FORCE_COPY_ARG, > > - NULL > > -}; > > + ETH_AF_XDP_USE_CNI_ARG, > > + NULL}; > > > > static const struct rte_eth_link pmd_link =3D { > > .link_speed =3D RTE_ETH_SPEED_NUM_10G, @@ -1129,7 +1150,8 @@ > > xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals, > > ret =3D xsk_umem__create(&umem->umem, base_addr, > umem_size, > > &rxq->fq, &rxq->cq, &usr_config); > > if (ret) { > > - AF_XDP_LOG(ERR, "Failed to create umem\n"); > > + AF_XDP_LOG(ERR, "Failed to create umem [%d]: > > [%s]\n", > > + errno, strerror(errno)); > > goto err; > > } > > umem->buffer =3D base_addr; > > @@ -1314,6 +1336,245 @@ configure_preferred_busy_poll(struct > > pkt_rx_queue *rxq) > > return 0; > > } > > > > +static int > > +init_uds_sock(struct sockaddr_un *server) { > > + int sock; > > + > > + sock =3D socket(AF_UNIX, SOCK_SEQPACKET, 0); > > + if (sock < 0) { > > + AF_XDP_LOG(ERR, "Failed to opening stream socket\n"); > > + return -1; > > + } > > + > > + server->sun_family =3D AF_UNIX; > > + strlcpy(server->sun_path, UDS_SOCK, sizeof(server->sun_path)); > > + > > + if (connect(sock, (struct sockaddr *)server, sizeof(struct > > sockaddr_un)) < 0) { >=20 > seems the server address is hard coded as "/tmp/afxdp.sock", is any > spec we follows, or should we parse this as a devargs? > better add some comment or external link that help to explain this It was already hardcoded in the afxdp-plugins https://github.com/intel/afxd= p-plugins-for-kubernetes/blob/main/constants/constants.go . If we want to go with the devargs we will still need to put a default hardc= oded value as there is no documentation showing the where and what is the n= ame of the socket in the repo without going through the code.