From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BBB5044072; Mon, 20 May 2024 03:28:53 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8F4EF402BA; Mon, 20 May 2024 03:28:53 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by mails.dpdk.org (Postfix) with ESMTP id C51A3400EF for ; Mon, 20 May 2024 03:28:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716168533; x=1747704533; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=VdY5mMJzbjMz61UX8hE11qE7QbXsrnJGihnJXfsLO0M=; b=QzDmWkpR73qyIxXFH0Los9C6hoRQ7gPoqpK5E2FXWNtChngQ/1DsVUAr S0JWF81gnkZQLDFu2VST1nKT3w1hr+zG26faa5OuVziywXeWGAEQ6LbCg ELDmrpU958Kb/par37yG2ToMcfGpGFoIX5nD2LJZvO1Sz9YszsJQuLg9/ YfCzsuuyZvT0bISjGUDtP8BN/FOU9fuxwDh1V9DHu+03gTLVZyVIx5s3p ejkQuwQjyp0Yqo850QRw6B+7LHrFLRvh8GTCq/vfUA/Y+qULWp3k8rctT q/nqBvZnItq92BqnHMf2hyAfKguFZJ7N22woKItNwBS7Nm+unkSz/3JDO w==; X-CSE-ConnectionGUID: SXlLe2eWQzyD2YSh2bgovg== X-CSE-MsgGUID: 8RlVZXUGST6duj8C53p3KA== X-IronPort-AV: E=McAfee;i="6600,9927,11077"; a="34790470" X-IronPort-AV: E=Sophos;i="6.08,174,1712646000"; d="scan'208";a="34790470" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2024 18:28:51 -0700 X-CSE-ConnectionGUID: EOy9lBZuTY+f7Vm1v4mdqA== X-CSE-MsgGUID: mKZmVmM4TzyTCgPOHgbuWw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,174,1712646000"; d="scan'208";a="32792011" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orviesa006.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 19 May 2024 18:28:51 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 19 May 2024 18:28:50 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 19 May 2024 18:28:49 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Sun, 19 May 2024 18:28:49 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.168) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Sun, 19 May 2024 18:28:49 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jsGVJRYiV4QxfHL3gS6uixPW7LjGFiTipLTvHkjfgncp6fqFG8vgtfrFrS/Ck2qpcCT5DmINrUPkUwV70H3pvDlsR8JVwXZjdmhLMHcvPx08QIuRK2ZvcBPdz1+coQgVGe2SC+jAmbk/8Q96FpxJ5ggIx/BRUUFYVkvGvkWhEy3mdnSk46OT2mQ6JzdfL7Pd6Wrg/7PB+tdEN3MIcx63SeUx4VxteOApV0fgRowzyZiHkazX1M52e8tHCHxPBlXUDwK9oDv9tCcBYTgspLU25V9zhC1SuKIUjud7hPBRjc6E3FHWIz+FguhM3IXJsTivDfQltwgIkuyRlwSZ7IjDZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qwGSwM2fM1OpD3c+79vFU4TGzxReBZqRKXEMFgpiGio=; b=XiBHQvJvrqFIGLpLfM9hOLC5SP077SPGySUpIpLVAEK+WxXRC9lDmwzE68drCgkICPgm70QQkVy1a2pn5Pf0cOQm4c8zLVQupauldcTf9CZlDusJUEx+Ofl4VRSrItLP7eJwccs1YF03sRD+7hgLBzTrQWIiAafVmO1t6zfisZNtROqlzz61/UrdbkFH1d0EcDa0rc+FgMWXHVldE3MzF9uU4SgEE7IUh8UYAcFAOC9R49l3VFmaoXTAwg4nCeUT3CEpI94yK5Uhg/jhU0hsOhHv3i/DEFLYqgTl7it0X5K+rLtgJJUCL8act3GgXwGZ2cDGjiQTTtVWWLo9kEppRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from PH0PR11MB4775.namprd11.prod.outlook.com (2603:10b6:510:34::22) by SN7PR11MB6557.namprd11.prod.outlook.com (2603:10b6:806:26f::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Mon, 20 May 2024 01:28:42 +0000 Received: from PH0PR11MB4775.namprd11.prod.outlook.com ([fe80::a8f9:b889:964f:c495]) by PH0PR11MB4775.namprd11.prod.outlook.com ([fe80::a8f9:b889:964f:c495%7]) with mapi id 15.20.7587.030; Mon, 20 May 2024 01:28:42 +0000 From: "Du, Frank" To: "Loftus, Ciara" CC: "dev@dpdk.org" Subject: RE: [PATCH v2] net/af_xdp: fix umem map size for zero copy Thread-Topic: [PATCH v2] net/af_xdp: fix umem map size for zero copy Thread-Index: AQHao2YTmeB1E66cJUCYtzW336D4ZbGbcvuAgAPsJTA= Date: Mon, 20 May 2024 01:28:42 +0000 Message-ID: References: <20240426005128.148730-1-frank.du@intel.com> <20240511052618.1890677-1-frank.du@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH0PR11MB4775:EE_|SN7PR11MB6557:EE_ x-ms-office365-filtering-correlation-id: 3b9a5ef0-ef5d-4da5-a1bf-08dc786c2f1e x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230031|366007|376005|1800799015|38070700009; x-microsoft-antispam-message-info: =?us-ascii?Q?3oP+sRDoPAEIKUjCh7lg3Gkc32U/C6vg+Z9yu6h33B7Eof7cHwRqtFyOuUwW?= =?us-ascii?Q?T4QD3e8A9I1IkYvJqakpl2OCov/Lv8wi0rSaFZZNdBq7X7o9gp5rjDVvOhg7?= =?us-ascii?Q?Nyrmt5/k57cqYjOsTPlChPL6e/h2sWWXa+qC5OoliDENlNVSzGqkNxaWy/hY?= =?us-ascii?Q?pkN4Jp1961JktVoxlv0dgaTVixirpjEeyG3Xd5ISjRac6IRfm2PEimom/efD?= =?us-ascii?Q?eRBqlpAFE8h9H/Aj8hqy75CW2z8AVJIYZne/n+PDVeA8XQmuwe0CLgJlKDxj?= =?us-ascii?Q?RjXncGRMOlYdVQLo4WXskt30ddeXd9biybxyWlPQXds572Oz5HtYngCNtQeh?= =?us-ascii?Q?aHHnedI5iDOQDwMEMBkvMypnwjwgGdhoG1MCNuqEi5UU2c48hThs7T8xD+0W?= =?us-ascii?Q?cNATuXjiI92gYisXp0hk3ljh9EHGNd3cXb8VQntwLersyBMtUFyEbgIgR1Qq?= =?us-ascii?Q?JqdXAgi5C++Qtny1APSPCdupch2QUsLhoZh22JebO39GQAJ6lhsAK3KqoqLY?= =?us-ascii?Q?DMHIetJ1trhYlkLSHfjtHs6C28+nqATnFegoJLICUBQazrJUJZQtlW1RK2+S?= =?us-ascii?Q?UnqusGj+FqA9FcAbYlDRh6DZq9stMHMzl0QiwGKVk2GfUa4xYxiYJbfJUDxq?= =?us-ascii?Q?1vlxilhdk1iCvdbpEHQDTaKCosm+DnY/5wrX4QkJwOWoGDQZsAtHHzOLtJO+?= =?us-ascii?Q?rtrmIvrkgLpw6qGu3owyVXuo+AGHoIsXblrlKp2qeNqCyWumYOCCFYsnSvSF?= =?us-ascii?Q?CDSSNfPFrm6c5ZtF9mMH9AiTKGj2IxyNScgbNksm49fIIzagKfILchBQRtfw?= =?us-ascii?Q?RSITiF8Phr8nK6k0OLNI5zGuIyd7OcjtYy89buxOkwo7hZHyVozsYwTVieVV?= =?us-ascii?Q?h7C/s6wReufo2FcbrGNTMPLN3JFmZWjoIKF+KDCa34r7sb03yA7B6mxEMxnV?= =?us-ascii?Q?gCy8aLQqOvKOiOdQzN50O2kewX+HTHJbd7I8L++ODb2E66CjRitqQLO8/BsH?= =?us-ascii?Q?920taRHpyW0LCGh/paRPYtD26p4iLESmvYcTN0AXKq+AbY4za/qEvPenE0uA?= =?us-ascii?Q?tziKvAy+ema753JEgn8382PsaYffEsp1gSqA/iUF9DCwrEN1iw0ArGnexlwP?= =?us-ascii?Q?Y6P4O+xHyIh4vogBNbENvD2NtZyZjf+g9p/1gwjOHh1ZBq1fQD/bAGyWOw3f?= =?us-ascii?Q?oPX/vg/HTpztPC+QHtTR00OTLRYlUqrQlr6SW2E96PCjEjA8VPwCjNn4Y8Y9?= =?us-ascii?Q?ZqSw2Sx4DBLa0HweSLAi6QTVixpg1iwNoQDJhhf3oXD1Y7s988FPjskDLK9O?= =?us-ascii?Q?iIzi4LJTcvZ2HNWfRsZfDcuhdlXs7sQTUEvwX4VvdHAqEg=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB4775.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366007)(376005)(1800799015)(38070700009); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?nEEeseZVBSk9RoaZMpWkTazNSn+Wm9n8RPNyeoMOGiqfulzEgSYh2rH7buwU?= =?us-ascii?Q?DJLaBojpX2AJk9cqCmDL4E4JWvdtUWjaY4XjttCCctBaLpGCTdjZ5ig9EFec?= =?us-ascii?Q?o1s64hiCET36dK8tPsMvfdHBmzx1cjS5uLDmg1t6Dx9YdWDQV8O0UdsTz0SC?= =?us-ascii?Q?VuktBdv/3qkAJtPaumSvJt4GZew4ggTJ7zS1+Nc+gmk7IBTT6DKEktkqhJwc?= =?us-ascii?Q?lS1H/xnw9lPCLlbZveWse8uXeqOB6YD64GTenlXTIFr9ne7nzK5idZfXxRc6?= =?us-ascii?Q?Wo9+rHBR8Qjw1KlufwW4JKvE9v4MF7QSaDIgVnYbcGygjLR2ROi642nPwKIu?= =?us-ascii?Q?tKLoWGVN9aa3Bxbw6cj4zF0EhQ2BEgT46ulS1suY0dnGGY7oOfUMf1fn4DGJ?= =?us-ascii?Q?Fph2ZLfZFWI2lLkFU7aRe+VpYjBzA73xoFGI/sOIPXIjwE9x4XvK47I2Mrev?= =?us-ascii?Q?u+SQrbD5FP6mwy6bd7H2O/ymfF+Hmd4aiGcPAhDA/yuWpcW83jeuJ8ZSM1Ep?= =?us-ascii?Q?vf2lPitDK5QgEBzkabIYSzhoU6vV/lO6fw17CDtEtzl/yfJdK6SZCsTk9IBS?= =?us-ascii?Q?29QRNrF96iMvZ74W2gufVUmH9pAjrZHkPBw6AvwQYuLMhWbSbYcQq4hOmEhW?= =?us-ascii?Q?uZzERpEsJXETHfxkP8YlRk+LtTcO/Vc+920v+lvDPZVlpVbBjoxFjQNqDvtO?= =?us-ascii?Q?12RC2XQJbQwfxJfvR45RLi8AY4+JgjxeyUYHYYRXcGpvg6sR0+jLLsUrDkR5?= =?us-ascii?Q?DD8w4iN9LoxSusfvDcRIAAu6HciCnDVJdbT9Y2GfHsauvhOegUSF081bPbmx?= =?us-ascii?Q?2Y3Ux6ZoChsz1BZN7WDEtEooMHxBcE+EO+i7nxMvO+sGOhezgYSNfWlVLVpE?= =?us-ascii?Q?aiAGjVIHDv+2KVTNPOJgq4f1FhTV0ZWVqy+yf4BeL6JinsL70pwbtNtA2xKa?= =?us-ascii?Q?ta1OyYlcpce1i4NQTsKO7tpM3yzVGSgUR59nkteDV0VMASpL8fDFLTcJtQMe?= =?us-ascii?Q?hFIdgutuhTO7xXjMtvlxZSqlqAyMFyRIfQV7EXoRxtTnmzC1MPvYhE4C3gUG?= =?us-ascii?Q?9X3Qb1Bq33cU4KdMZFyPsQvfQ/s88NalLH9mKDMdzHlNLSETVPSYStpVmLFn?= =?us-ascii?Q?6yaH3rivJ890rAzERPjuqZY9arJ17+/qtBTyUbLyOPDI3SZ0kO4mCp2a9vjl?= =?us-ascii?Q?XtFnyFwL5ZtZIKGlJKUC0dcYQu/LZz4W8sbWu1AMoZ5iDBospe590q2j7zGI?= =?us-ascii?Q?k4v/vS7W2B7KZzeCD9VfoddOnUQ1Mo3lgQ+fB2HgnWVM9GIkxjPr57MkXEw0?= =?us-ascii?Q?4F0LnpVNZD8PW9ygbj3tEIk+p7c5C0vVB5SXIw47b6GA6hNi+qQLqhbnMy/a?= =?us-ascii?Q?77lTh+sg8saX6vlDuaT90tg6o4gxDHO8SeBKKw3SfxSqv4QATgnH/IssQM3p?= =?us-ascii?Q?nkDwrbuhHnu8kEQP01yjFGE6aOZau3GpCyh6YgdOtvfuostfoFhaZzCPGYYU?= =?us-ascii?Q?FTLGsMMgrGvX4DgcE8hdn2gWxkVCUlZNZ7qZ3zidvnDFoBQsldRENOeh0yuw?= =?us-ascii?Q?gFPprrAt1HKvztiGkrNOBdFrPdMjuPr9vdDkHM9/?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4775.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3b9a5ef0-ef5d-4da5-a1bf-08dc786c2f1e X-MS-Exchange-CrossTenant-originalarrivaltime: 20 May 2024 01:28:42.2001 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: nNfcGjqEhpLeGOTNEA9JyxSfJ3FKAgL7uk20zJflrZqzrIPuDH/CqKC/taRkFgUIdHrae/YRKdy80PIRLcB85g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB6557 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Loftus, Ciara > Sent: Friday, May 17, 2024 9:20 PM > To: Du, Frank > Cc: dev@dpdk.org > Subject: RE: [PATCH v2] net/af_xdp: fix umem map size for zero copy >=20 > > > > The current calculation assumes that the mbufs are contiguous. > > However, this assumption is incorrect when the memory spans across a hu= ge > page. > > Correct to directly read the size from the mempool memory chunks. > > > > Signed-off-by: Frank Du >=20 > Hi Frank, >=20 > Thanks for the patch. >=20 > Before your patch the umem_size was calculated using mb_pool->populated_s= ize > * rte_mempool_calc_obj_size(mb_pool->elt_size, ..) With your patch we sum= up > the lens of all memhdrs in the mempool. >=20 > When debugging I see the new calculation can yield a larger value, but th= e new > logic looks good and more thorough to me so I'm happy to opt with the new > approach. Hi Ciara, Thanks for the review. The reason for a larger value is that, the pool cont= ain some necessary space hole to ensure one single mbuf does not span two v= irtual huge page, so that the PA of each mbuf is secure for the hardware op= eration. >=20 > Acked-by: Ciara Loftus >=20 > Thanks, > Ciara >=20 > > > > --- > > v2: > > * Add virtual contiguous detect for for multiple memhdrs. > > --- > > drivers/net/af_xdp/rte_eth_af_xdp.c | 34 ++++++++++++++++++++++++---- > > - > > 1 file changed, 28 insertions(+), 6 deletions(-) > > > > diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c > > b/drivers/net/af_xdp/rte_eth_af_xdp.c > > index 268a130c49..7456108d6d 100644 > > --- a/drivers/net/af_xdp/rte_eth_af_xdp.c > > +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c > > @@ -1039,16 +1039,35 @@ eth_link_update(struct rte_eth_dev *dev > > __rte_unused, } > > > > #if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG) > > -static inline uintptr_t get_base_addr(struct rte_mempool *mp, > > uint64_t > > *align) > > +static inline uintptr_t get_memhdr_info(struct rte_mempool *mp, > > +uint64_t > > *align, size_t *len) > > { > > - struct rte_mempool_memhdr *memhdr; > > + struct rte_mempool_memhdr *memhdr, *next; > > uintptr_t memhdr_addr, aligned_addr; > > + size_t memhdr_len =3D 0; > > > > + /* get the mempool base addr and align */ > > memhdr =3D STAILQ_FIRST(&mp->mem_list); > > memhdr_addr =3D (uintptr_t)memhdr->addr; > > aligned_addr =3D memhdr_addr & ~(getpagesize() - 1); > > *align =3D memhdr_addr - aligned_addr; > > + memhdr_len +=3D memhdr->len; > > + > > + /* check if virtual contiguous memory for multiple memhdrs */ > > + next =3D STAILQ_NEXT(memhdr, next); > > + while (next !=3D NULL) { > > + if ((uintptr_t)next->addr !=3D (uintptr_t)memhdr->addr + > > memhdr->len) { > > + AF_XDP_LOG(ERR, "memory chunks not virtual > > contiguous, " > > + "next: %p, cur: %p(len: %" PRId64 " > > )\n", > > + next->addr, memhdr->addr, memhdr- > > >len); > > + return 0; > > + } > > + /* virtual contiguous */ > > + memhdr =3D next; > > + memhdr_len +=3D memhdr->len; > > + next =3D STAILQ_NEXT(memhdr, next); > > + } > > > > + *len =3D memhdr_len; > > return aligned_addr; > > } > > > > @@ -1125,6 +1144,7 @@ xsk_umem_info *xdp_umem_configure(struct > > pmd_internals *internals, > > void *base_addr =3D NULL; > > struct rte_mempool *mb_pool =3D rxq->mb_pool; > > uint64_t umem_size, align =3D 0; > > + size_t len =3D 0; > > > > if (internals->shared_umem) { > > if (get_shared_umem(rxq, internals->if_name, &umem) < 0) @@ > > -1156,10 +1176,12 @@ xsk_umem_info *xdp_umem_configure(struct > > pmd_internals *internals, > > } > > > > umem->mb_pool =3D mb_pool; > > - base_addr =3D (void *)get_base_addr(mb_pool, &align); > > - umem_size =3D (uint64_t)mb_pool->populated_size * > > - (uint64_t)usr_config.frame_size + > > - align; > > + base_addr =3D (void *)get_memhdr_info(mb_pool, &align, > > &len); > > + if (!base_addr) { > > + AF_XDP_LOG(ERR, "Failed to parse memhdr info from > > pool\n"); > > + goto err; > > + } > > + umem_size =3D (uint64_t)len + align; > > > > ret =3D xsk_umem__create(&umem->umem, base_addr, > umem_size, > > &rxq->fq, &rxq->cq, &usr_config); > > -- > > 2.34.1