From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0468A4240D; Wed, 18 Jan 2023 13:10:36 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D4742400D6; Wed, 18 Jan 2023 13:10:35 +0100 (CET) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 2DB594003F for ; Wed, 18 Jan 2023 13:10:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1674043833; x=1705579833; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=fZw5e5mP3/euDES3d0LkYZlV3w89L1LyEF0xoB/TI1E=; b=jVc40NVWotoCHMKHntCGtYtfPEB0iefkjouyIAYOjLiWlkv6h6GQKb6A jxptczR1mvYAaTtDW+7bpKkJWcBqFcX07PWJmrQLzyCZbtcqu7bjXPstG GzVo0gQ8hxwPJeKEydYTquGJTxhJmfBA+6PerLIB6LsurPrAeX3hop4wy LfR2pZ0wGE7ybU14qWmarK3vQ8mtsht3KAmvszWg5BJlms6pMKo4U0ND6 T3+hp2nLl6hNS/pRdljfDQj8WN4RktbU9MdWgOjbclZYnr9zGULWAIr8y KurAljvO+KJraYGWGhgdLLkSmsSlKGpnJbfV5Kpdtin3yZ7NmMJOHM90O w==; X-IronPort-AV: E=McAfee;i="6500,9779,10593"; a="322656921" X-IronPort-AV: E=Sophos;i="5.97,226,1669104000"; d="scan'208";a="322656921" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jan 2023 04:10:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10593"; a="833556993" X-IronPort-AV: E=Sophos;i="5.97,226,1669104000"; d="scan'208";a="833556993" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga005.jf.intel.com with ESMTP; 18 Jan 2023 04:10:31 -0800 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Wed, 18 Jan 2023 04:10:31 -0800 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Wed, 18 Jan 2023 04:10:31 -0800 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16 via Frontend Transport; Wed, 18 Jan 2023 04:10:31 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.177) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.16; Wed, 18 Jan 2023 04:10:31 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LooC9tF0y1Mi2mmh7SSdc/jtE7SCP8tI+exuHDxcruHvqjSkMSoC7sLpkAa9b6DPzHKSCYwTd/IzDHURyOhGzcOjadC4zLkbvB9dgENd/kycKVO6bWAMTHckYgKDWPlFgZJiX1H4kf462wdhNe5cdpcJ5RdwDYiqEQSlvTaxX8cQosQ7Ga4RbQntSXSP0vczOw32g4UNnvvu0xy0dJwclD9h2JPcmpZiJan59kwsiatX2WIqoGrVa0fHv7+YDQbndrC+WPZFAOcbalk6kgP0CLesMbphtmpXGKIwLFIvKWZ+RodIbPWI1JekPYD6l0K3bttSHUDJCNjiGCM5V4WmEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=n8g/wy3PqjmzHlap8oE2wMSLOUgjxJ3bMaiNZs2QHVc=; b=mRXn9oVJYT3EzcseFnGkJQlcpreMeYU81xovGaiaZBzEF+e634ajL48TYCo2pMz1LpA2d5Ql8i2+LNFyUXoUyZRuZBdcmCPiwvaz0hswfsVn/Xs6KgrqSCEpIeFW3Ko8rWYuydCCMV+7r0kgZGNDbJr0BfpKkHwX9z0sOIiN8wUJT9RrVDZGZPUyobBe7dIVuVEZ8K5/mJXnqo+At6iOMKz+5TddLjzJUPAeyJR2iblZBgbzs1H7pO9JMz7KgiPNXeQAUA86LCjSvCH74qfU1DI6jhGJGvZCFZRM303qSyD/HFX8WxKBq+MgJirbrlqrX/g8/zseN3HpEIIf3QkaYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM4PR11MB5994.namprd11.prod.outlook.com (2603:10b6:8:5d::20) by PH0PR11MB7588.namprd11.prod.outlook.com (2603:10b6:510:28b::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13; Wed, 18 Jan 2023 12:10:29 +0000 Received: from DM4PR11MB5994.namprd11.prod.outlook.com ([fe80::eb70:80e0:1b88:3ca5]) by DM4PR11MB5994.namprd11.prod.outlook.com ([fe80::eb70:80e0:1b88:3ca5%9]) with mapi id 15.20.6002.024; Wed, 18 Jan 2023 12:10:29 +0000 From: "Zhang, Qi Z" To: "Koikkara Reeny, Shibin" , "dev@dpdk.org" , "Burakov, Anatoly" , "Richardson, Bruce" CC: "Loftus, Ciara" Subject: RE: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration Thread-Topic: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration Thread-Index: AQHZD9KLg3yJAyWJXESupanX+QJqc66kSwvg Date: Wed, 18 Jan 2023 12:10:29 +0000 Message-ID: References: <20221214154102.1521489-1-shibin.koikkara.reeny@intel.com> In-Reply-To: <20221214154102.1521489-1-shibin.koikkara.reeny@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM4PR11MB5994:EE_|PH0PR11MB7588:EE_ x-ms-office365-filtering-correlation-id: 2eeca088-3f93-4fee-4fbf-08daf94cfdc0 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: LHCVDPexpXj962d1fFDDDY/FN30xNsPVNYJUVfibYus1T03OxvzK+wNBI5W2Dj6+r6W6ykagEHJcSk6YV9017C2ObGc9U84K7HFas7ToLPWNdKUrixkOhW6ZOWps0q0zAlie8l5by4N/pfxlSObImRUgLd4qhaVLSjP47fIUc/RkjVZeVEJCehHbTYxTxsqJJu0BJtMW3H4ZWA8+xiaOA7EzrAv8LwmYUNbHUKILA6rpnasPP32smYB9QPl/eWS2LsQk5q7WXnX1KADhpfen4GEIjfg9y9Uh8bLrQqbjvL8/iOEXxIesMm70o89meU1qDrl/+QGgdLuvMtAfPM/ZmRyLdj0QTN6MeLq4HObeX+uVh0fAEBoEPIoCPFkXE01YI7lIp4YzhMS+3oTthrHyND0X7shOwkQ7sUid2USHhADCydJb0s38KiIcbyNt80pr6YuNQxJaHh7g6FiW0AZmknkH4rUYORf8CLOb/9tvLpXHYlHuqXKhfX1Xg81MbxxwtMypDhWWUuBHXOnfwQk83WgCBkPQT14lG9nad9oJXmHJxIwlACsP8/karuieHitcOqEcBCWCJeGdmW4JhLy45SBdEzmAjfDFxP11wyg74V8kJqVq6OnyufSmW3n7cFWGBzQJZQlCBAeVI4wDopZ1ZW9c29N1qJ6BQvAEtanQTBlcGr/wIRvaoT4w2rwMXMjn65FK6fY9QdVVWRrMR/+ss+nEVkfCkaOZB0S1CLConuE= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5994.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(346002)(366004)(396003)(136003)(39860400002)(376002)(451199015)(53546011)(4326008)(66446008)(86362001)(76116006)(55016003)(186003)(64756008)(9686003)(8676002)(66556008)(26005)(41300700001)(66476007)(66946007)(71200400001)(316002)(6636002)(110136005)(7696005)(6506007)(107886003)(122000001)(966005)(38070700005)(38100700002)(2906002)(478600001)(82960400001)(5660300002)(83380400001)(52536014)(33656002)(8936002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?YCQsHml+O5r70yhG+ZZv7gApMYaGDLyCAE4CmNPfBuN13+TDDTpxLmQG1LzX?= =?us-ascii?Q?GKooolhSMrp420CALTWWFQcoU9FS5ZZqWRjjfUXLYAiuPetRUfb2uuhHZ8n2?= =?us-ascii?Q?Dl/PE70JeRc8iSL0gg6fEZIrxaEkGud7G2NP3VP1MW5asI4uvtRHwuNI4cRu?= =?us-ascii?Q?VGu6pRPQIKJB52zE/BIP46ync9OOX+TrWVmMMjaDsh+zRgBzEYqDSMIROfh6?= =?us-ascii?Q?MqiRZp6mReY/WfHLDGzOpGiR9OATVCU3Mt2ZNJZOzZSizgu8BfGAIQTWWzbA?= =?us-ascii?Q?/yovh4yVECyE8hP0n4uDyAOdaqOJGGYF1Z67vMEil1MVOUFcGb343LTcrU0H?= =?us-ascii?Q?GaUEYimB39Ju3iLIG9MEo9X4BrAjcFigq1wjOdOdldmqFxjRJIZ7vsrNhDRB?= =?us-ascii?Q?/ZCMvv61rQcQZSs666EU9USNQ6vQVtLPcQ3jVM2INnZYDlCInaWJohK9EVnI?= =?us-ascii?Q?6SaN4U2prMPyCiOpLSa/YZtc29J7J2u/91mePrjl8LgEdMblfvNgR5bCGiNW?= =?us-ascii?Q?xjyFuCEQIgpnbTkxWzajiCkq5lEQuf/QvzioBJ+RRDOIfMk12dkbvlC62EWn?= =?us-ascii?Q?K8rypPAAmNexj3SydEeVowTZ7GagGkElfMz89eu9S4qcgdOVagQR7kdWrL7f?= =?us-ascii?Q?naFsV7bCh1tpj8tlHtwUtJvf6nvtrGH3geirf48WPM+DrZDkpWFIfnX8DK2p?= =?us-ascii?Q?2zykEBX0nqdpn3b8bXjAC+8HFG+0dQnslyofn8uyf4cUeCdsTcUi05rkaKVY?= =?us-ascii?Q?hLlv25K5nuX6h54ychGT6/zj5OT/mq4qYgk6B0IZDUovuhbzVsuutG6id4AM?= =?us-ascii?Q?XtRWretaC8p3pbWtxowBACM29/iswtvAJNAVcS3Xpzrf2cXwoYaNbfIdFc7o?= =?us-ascii?Q?IeKP1DBkM7QCUSsopDsAz43/7DM6Hr6RuLFz9HN2QhxpDC8IZ3Q/y/JrupYb?= =?us-ascii?Q?qzX4f1zMfhgf9SkEEOTrj1r0vghgP+w/nIF+oYyEFRK+VQgiyuA4C32KYwmy?= =?us-ascii?Q?nGa4rMj5b56btxxMSRbzFW8MK9ar44ru5Z9uIZEyHmStHsUZJHnstPigDJ3L?= =?us-ascii?Q?D3FFd4zjs+RxOGbfIgtmhLwNGp4q5KiPlXCAycTeSasvENQPTfWQF+MkiKdi?= =?us-ascii?Q?69O/ebybCRVbFtHFwrh2AcrARMBGKDXRZG2PIvk/V2PgLkF6f2cv0ohWT0hn?= =?us-ascii?Q?mGDiedF+kD98UmotglxsnJzrKQ6oLR/9LDcHuXbzGY7Ixn+TmhC0hOcBXWU8?= =?us-ascii?Q?6cjwBijwIlh1aiXoHbAo8FwHTt5iKKp+O77bmlNNHq4dMnoy13hVCTjNNQLq?= =?us-ascii?Q?UQdkvXmQF6EKCtxV6TWIcsHnf37UnmsrMmqH1YUJgsmij+fv7R8vP/NXJtr0?= =?us-ascii?Q?H3d7BtcxIUnF0aCQCTskW80zeJBrlEu2/rJXRINuXaxYHLcfgUk8+SrcKoJs?= =?us-ascii?Q?BqCfvuLchLByDz/8jqgjA2Gk7WiddVLEfyNrnoT+XxqkqZ8DkmrWCl8VCEig?= =?us-ascii?Q?0NwZDNUX+/kvVS+0vtEt/j9xK2FjXUUJUQHRUPHx+UvpqeV1Jw7eh8G6mCw9?= =?us-ascii?Q?/T+Wp/lcCbxl7WF8XoG64hvs37ZGqO2s1578XInq?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5994.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2eeca088-3f93-4fee-4fbf-08daf94cfdc0 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jan 2023 12:10:29.6103 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: L27qPHARQR4X1kHgdei4bADL0+dEh6bTDvNCjWOQfcnQO8sSlAHTr24jNfoWQ8Azu1720gqL5aGJWKZGNPP9jA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB7588 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Koikkara Reeny, Shibin > Sent: Wednesday, December 14, 2022 11:41 PM > To: dev@dpdk.org; Burakov, Anatoly ; > Richardson, Bruce > Cc: Loftus, Ciara ; Zhang, Qi Z > ; Koikkara Reeny, Shibin > > Subject: [PATCH v2] net/af_xdp: AF_XDP PMD CNI Integration >=20 > Integrate support for the AF_XDP CNI and device plugin [1] so that the DP= DK > AF_XDP PMD can work in an unprivileged container environment. > Part of the AF_XDP PMD initialization process involves loading an eBPF > program onto the given netdev. This operation requires privileges, which > prevents the PMD from being able to work in an unprivileged container > (without root access). The plugin CNI handles the program loading. CNI op= en > Unix Domain Socket (UDS) and waits listening for a client to make request= s > over that UDS. The client(DPDK) connects and a "handshake" occurs, then > the File Descriptor which points to the XSKMAP associated with the loaded > eBPF program is handed over to the client. The client can then proceed wi= th > creating an AF_XDP socket and inserting the socket into the XSKMAP pointe= d > to by the FD received on the UDS. >=20 > A new vdev arg "use_cni" is created to indicate user wishes to run the PM= D in > unprivileged mode and to receive the XSKMAP FD from the CNI. > When this flag is set, the XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD libbpf > flag should be used when creating the socket, which tells libbpf not to l= oad > the default libbpf program on the netdev. We tell libbpf not to do this > because the loading is handled by the CNI in this scenario. >=20 > [1]: https://github.com/intel/afxdp-plugins-for-kubernetes >=20 > Signed-off-by: Shibin Koikkara Reeny > --- > drivers/net/af_xdp/rte_eth_af_xdp.c | 337 > +++++++++++++++++++++++++++- > 1 file changed, 325 insertions(+), 12 deletions(-) >=20 > diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c > b/drivers/net/af_xdp/rte_eth_af_xdp.c > index b6ec9bf490..196d98ad97 100644 > --- a/drivers/net/af_xdp/rte_eth_af_xdp.c > +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c > @@ -7,6 +7,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -81,6 +82,24 @@ RTE_LOG_REGISTER_DEFAULT(af_xdp_logtype, > NOTICE); >=20 > #define ETH_AF_XDP_MP_KEY "afxdp_mp_send_fds" >=20 > +#define MAX_LONG_OPT_SZ 64 > +#define UDS_MAX_FD_NUM 2 > +#define UDS_MAX_CMD_LEN 64 > +#define UDS_MAX_CMD_RESP 128 > +#define UDS_XSK_MAP_FD_MSG "/xsk_map_fd" > +#define UDS_SOCK "/tmp/afxdp.sock" > +#define UDS_CONNECT_MSG "/connect" > +#define UDS_HOST_OK_MSG "/host_ok" > +#define UDS_HOST_NAK_MSG "/host_nak" > +#define UDS_VERSION_MSG "/version" > +#define UDS_XSK_MAP_FD_MSG "/xsk_map_fd" > +#define UDS_XSK_SOCKET_MSG "/xsk_socket" > +#define UDS_FD_ACK_MSG "/fd_ack" > +#define UDS_FD_NAK_MSG "/fd_nak" > +#define UDS_FIN_MSG "/fin" > +#define UDS_FIN_ACK_MSG "/fin_ack" > + > + > static int afxdp_dev_count; >=20 > /* Message header to synchronize fds via IPC */ @@ -151,6 +170,7 @@ > struct pmd_internals { > char prog_path[PATH_MAX]; > bool custom_prog_configured; > bool force_copy; > + bool use_cni; > struct bpf_map *map; >=20 > struct rte_ether_addr eth_addr; > @@ -170,6 +190,7 @@ struct pmd_process_private { > #define ETH_AF_XDP_PROG_ARG "xdp_prog" > #define ETH_AF_XDP_BUDGET_ARG "busy_budget" > #define ETH_AF_XDP_FORCE_COPY_ARG "force_copy" > +#define ETH_AF_XDP_USE_CNI_ARG "use_cni" >=20 > static const char * const valid_arguments[] =3D { > ETH_AF_XDP_IFACE_ARG, > @@ -179,8 +200,8 @@ static const char * const valid_arguments[] =3D { > ETH_AF_XDP_PROG_ARG, > ETH_AF_XDP_BUDGET_ARG, > ETH_AF_XDP_FORCE_COPY_ARG, > - NULL > -}; > + ETH_AF_XDP_USE_CNI_ARG, > + NULL}; >=20 > static const struct rte_eth_link pmd_link =3D { > .link_speed =3D RTE_ETH_SPEED_NUM_10G, > @@ -1129,7 +1150,8 @@ xsk_umem_info *xdp_umem_configure(struct > pmd_internals *internals, > ret =3D xsk_umem__create(&umem->umem, base_addr, > umem_size, > &rxq->fq, &rxq->cq, &usr_config); > if (ret) { > - AF_XDP_LOG(ERR, "Failed to create umem\n"); > + AF_XDP_LOG(ERR, "Failed to create umem [%d]: > [%s]\n", > + errno, strerror(errno)); > goto err; > } > umem->buffer =3D base_addr; > @@ -1314,6 +1336,245 @@ configure_preferred_busy_poll(struct > pkt_rx_queue *rxq) > return 0; > } >=20 > +static int > +init_uds_sock(struct sockaddr_un *server) { > + int sock; > + > + sock =3D socket(AF_UNIX, SOCK_SEQPACKET, 0); > + if (sock < 0) { > + AF_XDP_LOG(ERR, "Failed to opening stream socket\n"); > + return -1; > + } > + > + server->sun_family =3D AF_UNIX; > + strlcpy(server->sun_path, UDS_SOCK, sizeof(server->sun_path)); > + > + if (connect(sock, (struct sockaddr *)server, sizeof(struct > sockaddr_un)) < 0) { seems the server address is hard coded as "/tmp/afxdp.sock", is any spec = we follows, or should we parse this as a devargs? better add some comment or external link that help to explain this