From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D49A34404E; Fri, 17 May 2024 15:19:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5BE124028C; Fri, 17 May 2024 15:19:56 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by mails.dpdk.org (Postfix) with ESMTP id E964540268 for ; Fri, 17 May 2024 15:19:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715951994; x=1747487994; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=nbLe4I9sOdA57FrWFWNa+9HXoVM0FjkThf0fCakagyU=; b=C3GQnQtW+WShJ/BGz38QfOcNGSY5K0U69L2X5nI3+MoiCCoOAE5KrR97 Yqo9pZ/+Cd9baHl62n1eIgraNhxUyONnV33Q4MzcK4Gc9WxbVz8K6WEov Bk7vB67BEI9X7As8QbARXKoo9C42Y55OPqJzK2sg5xQbcTgUTrCIkdpV3 Bo61UsNyrEGIQb9v7zg7yOS3feFuPXSaaPxKSzrXlFDXu0tM6+ovbya3G Cc3ZYUkP5bSx2OYsg+E4Kd4n+kwIl0MRGn6ny8pUBHBRyfQyI784JYWtW H38JQIZcUOuRqc/LXUqqFqSgjz2d89+d7Soi9J+NEihVlk+2X+Y7zOtFO Q==; X-CSE-ConnectionGUID: IPTtKdhvSKaai+7oKgyRXg== X-CSE-MsgGUID: +5f+N+TeQTqwx2sB1Xpt5A== X-IronPort-AV: E=McAfee;i="6600,9927,11074"; a="22705701" X-IronPort-AV: E=Sophos;i="6.08,167,1712646000"; d="scan'208";a="22705701" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 06:19:52 -0700 X-CSE-ConnectionGUID: Smy15Jw7S1GQLM3UBmAh6g== X-CSE-MsgGUID: kOAsNZSaQU2a0HWtGJLRIQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,167,1712646000"; d="scan'208";a="32203434" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orviesa006.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 17 May 2024 06:19:52 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 17 May 2024 06:19:51 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Fri, 17 May 2024 06:19:51 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.100) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Fri, 17 May 2024 06:19:51 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ELDOJlTFIyqY22Y0qvCkNNC+RHukvhtdz1lReOaKsF2fc7ob7hLeczdTBDUicgDGdROirKRDCJ6v/4aODncvlhQiZdYKVnySuFWOitdEBaufgepRA8LC7HFz/t+ojonR3bEz3xboo9AjGx3VjNyVS/WWjbxPFdUiHtlacKpC+Cc9d1Mc/FNi4nVsvnakip+T3ZHAzg2VltdO/7mjPdiDtaDCzlnJv054EP5YgX+dLurSay+etyaqQpoElJj/EplPxWt8uyxrORMbcQpsLzAA7oU2sa4rAzG76g76Wfh+ODCfYxnWqNqq5QovzN2QmgTkTtHggs5BeU+ZH2Y1oN8OTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vVZ0SwGofVQNGaqX7/tsjTmwNuYqzCpPFxraTN+P0VU=; b=AlgSbLCAFg0cppfLen9YRyhfnmuMTl509lYaJYjMJmCD2p2YOf6/3lChhZNGPpPNJ9PLclkrVL/Ed0P4MmxNayh8igH1wGBLUgMFcPqwJjNNqh3QFSAQd0itx9cIs/dnIZNNevcMUpitgKDs+SY5lq1S+iVfqaSkatKvaimJEdAJS9TE1vmITFUiPkEmEU23w9GljBGenslIouZydGC7gWTy0dqGmOzXYbp0mBCv+BQRd7unlTP14UZF1hc4MN1bqZ+MLb+/7kd3X3n03a9yEVmZBlTuRt17joxMGnesOVgAuUWe7DbydCJCiJYP/+bHSBqowg7hHocyfyANLfahQQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from MW4PR11MB5872.namprd11.prod.outlook.com (2603:10b6:303:169::14) by DS7PR11MB7781.namprd11.prod.outlook.com (2603:10b6:8:e1::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Fri, 17 May 2024 13:19:49 +0000 Received: from MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::6444:ca4c:aa3e:f8d2]) by MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::6444:ca4c:aa3e:f8d2%4]) with mapi id 15.20.7587.028; Fri, 17 May 2024 13:19:47 +0000 From: "Loftus, Ciara" To: "Du, Frank" CC: "dev@dpdk.org" Subject: RE: [PATCH v2] net/af_xdp: fix umem map size for zero copy Thread-Topic: [PATCH v2] net/af_xdp: fix umem map size for zero copy Thread-Index: AQHao2YIp6/orB3VBEGMOIe5GUdgmLGbcrFQ Date: Fri, 17 May 2024 13:19:47 +0000 Message-ID: References: <20240426005128.148730-1-frank.du@intel.com> <20240511052618.1890677-1-frank.du@intel.com> In-Reply-To: <20240511052618.1890677-1-frank.du@intel.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: MW4PR11MB5872:EE_|DS7PR11MB7781:EE_ x-ms-office365-filtering-correlation-id: 73188498-f678-40eb-5c58-08dc767406a2 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230031|376005|1800799015|366007|38070700009; x-microsoft-antispam-message-info: =?us-ascii?Q?Wa4i6RrV4du6U2OrAzruCEocpDn7qMhvCEahDbrxNZsH5nuYyQnN9Vrp8kVR?= =?us-ascii?Q?QGcAUxeAWCUyTtfazRNMZyPdV0p/JfQlqkBpPOPz1IFnoSuoQwz56vmsU2Qx?= =?us-ascii?Q?TOv2dTp+u+BgRS5AHJMLLQTVQZg4WG3YLzj1NqQSgJM++LYfA237kDtC2CGx?= =?us-ascii?Q?VIhfSpfhC2dOmDX81yghtBu2A1MDHwGdhM+YexxQhsQDPgdMYSt4U/eRbE6D?= =?us-ascii?Q?Pfj07cMucVdi/HJbweZypK8X84H8Z5okoHvp1PQK9D3ZczNN9UGxFK7f26yF?= =?us-ascii?Q?JpYlQikylD5l16KGPOWAfF/K4tslJPysAqHXew6psPnLXKfddMyPTKEkcQEg?= =?us-ascii?Q?SDNz1/wSyz6qq78+dmGGbReSYF7Kzu7o9lKh+3jj6pUERV46TSJJFASwdSLM?= =?us-ascii?Q?6LthcaQxfhapyzydEmGmEH1yrMqPXbW5NkPfATQxwBu6zGx84AHSorSsTdNb?= =?us-ascii?Q?cm0HwVFWGeShzjPkUQkUpGUgMUX4RspoaAFdFQYFhzT5ApOjrt4VqVdVyaAy?= =?us-ascii?Q?p1fsklZQiIJNURo8VSX9gdG5HZGAPcERGq8hPuKsq7sZ+otTlHsagyai6jnq?= =?us-ascii?Q?bwggBBp3kvJ9gmETOqHm8HEBYk4Yy24ZCI+DKnsVS3kJoUd73UdbmxtobpTc?= =?us-ascii?Q?rdAbTd4vvr5Q586ZwmQc7DvmMlKEbA/0MBccTUFythNwDM1JKQyTj7ymImJc?= =?us-ascii?Q?ua5PJz20fEa5Lbh63z7zLrJNL3Ht3SdVmbergXUAvoNkN+P5ab7Q9fYSnnNw?= =?us-ascii?Q?5E7cb4/Wr55gPtLvxAf5F9GgOhUqGYkPxAQDy1FqVohoF0njHfq4yPBf2uRO?= =?us-ascii?Q?P3V33q9BMpUte3yo5X7zDwB+/5m5VCWB6ptW1B0QrW6hZ1EDl0IMVvip1OHI?= =?us-ascii?Q?j98Wk5tj94pIC+qLto93gMYc8r37yR4ncRiMjSw6L5cCwvJHm0WRKdIU50Rh?= =?us-ascii?Q?+uCHB146Fx9vjzHXtb3efW5N7Rn3vfNCuWnsVF1hgd0jLerbcOs/BN+npbyD?= =?us-ascii?Q?YnkVM6Sa3zSHQajrZYSk2N0JqyeG19ZVkEOmVFc7X8rjYliJ5ZPZZF4LLZ/9?= =?us-ascii?Q?HXlVXQbMULaO5a2K6aDSzMnIUkHQUyqG8LfpNvm8FqsjLqbPz/ZE1Tz/KOoJ?= =?us-ascii?Q?xOS9YECKBOTsFjFgcwiu8RglZKgPjt9o5vyLxgODOJp2AhMV9VWNfRWGTxnA?= =?us-ascii?Q?Wa1cF8H4BbYLfQUqPyjFMkSpVJwnptfWaRg2Y8gaSRJouZuvK1XGb/ENDAqn?= =?us-ascii?Q?isFrzt+dUKuLDvbHj1DF6otZwaZVqIFzdZ29TzJcecfQ3pF2u9EGELTsfoci?= =?us-ascii?Q?NBtZKJjrCj8G8SvWPyy0KoX/7u2zSa7sA2SmFZ1kn5HC8w=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW4PR11MB5872.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376005)(1800799015)(366007)(38070700009); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?hrxqeBzZnZwbYc/V4i7E7eCIUsMf1oLv9WGTdZxcA+5/fPlVcxuvggJMmNp5?= =?us-ascii?Q?Qu6YVThnLekBAQ4Xgba2PAumBgY97YkIzEvaBlc1l6riQbgl/O1lmRMDAbu7?= =?us-ascii?Q?8UoBFB6/uDWj3VFHQ3nK7D+3vO77d+QOSxSli5jRASqrifBDxslrMeFkMglp?= =?us-ascii?Q?HG9dlWNMI0AwJN42K7gnNK2pRiO9m116H6kPLbvJQQXK17iXeraM1s/Hskfi?= =?us-ascii?Q?fuAxmNq63+53WViUkR8hnxf/Fxbw9Zwa1/tU8nD2QROYhuIcxcKw47FgwT8T?= =?us-ascii?Q?fgKu3pby90wti7gz1E4iYEie7ERvsZiGtSKwgaeBxsAMCQByvriMYUoJQRur?= =?us-ascii?Q?EloCAPA+q5sEpWDUO4JV808EU894K1D3qI3T/eqtOZi6TwpRWOpZcmloMCC5?= =?us-ascii?Q?cMYmg/GJNG5X5WqnjoxnxqPjGk9HgXq5mjNXsIfjHQTvwBMFE4vGAirFdluS?= =?us-ascii?Q?hovnxptzY0jOelzo8kcVxFSJc6B/MHF/gRogYGt64yRooLJy7T2VKnfGOaRg?= =?us-ascii?Q?T1OHNjazyfcsWSsbfeBD4NwRlOdaxZIwTj67qgLvHmCq4ZegT4b8zFfYFYSb?= =?us-ascii?Q?KrSD6YEZEflVd36lTW/mHnFlBf9j30fVPDRDptdslU2/r9cMUpnHVmiqcOgm?= =?us-ascii?Q?9CJyxEj+0TGDp9VLD+oRRQA5H+mw/QEMIsBEIRGJku98InPsNIKVJTPzxVAG?= =?us-ascii?Q?JwJnNA+t8IrgmU3ZYBzhd2JuQES38dCpGRsC3tcgVPVFsXEBa+Ch7mAezFDY?= =?us-ascii?Q?u/gmwzcyP6czC8OkAenpPx2RUzZ82/tuOz+C5zYoDSHKZ7TMa0i4Tzb5GGB7?= =?us-ascii?Q?yA9KbPfUA4nhg39nedu8ae5xJEUbx4fQdwFSjdxTGGhpfUvX47U6Tf+0RtmX?= =?us-ascii?Q?d5yr2YeU9MJBE6w/+Kp6BzsKlSOoAuh1yVSa5e9mvB5LOA/1yx0kWRpbfshu?= =?us-ascii?Q?S9YcNFS6l8ghLVf7xRLZeIbQ4FRvRtY+jgvyahAW5EzXfolugPDHCgHTBS9p?= =?us-ascii?Q?7bhjnBg6QEs/tybLlI+oIRdvsTf33MxHIEirEmfhk2SA27tqHR46HvuAVR+K?= =?us-ascii?Q?Us+UDgEoeBC9moOYdwQSjcf0/fbwEf3mGbGHFS8oc49HI+2/Cu/XrHosrmps?= =?us-ascii?Q?STRkatlM4cB6pVzutTFk/hufThflIduaHr10TJI59SLog7jr4eGuHwdCqMzF?= =?us-ascii?Q?eYvAbwnpXd4Y/6H1NqsdI/IOCEY3eyHQT38y1c1/etPawaRlX/zMXLMOKZ89?= =?us-ascii?Q?zHDtfhCVzNFx6pw+BFtc57/6NzOZHSkhHgAQzMrmnGOJr6thDfPh9oBwN4el?= =?us-ascii?Q?Fx5em6BGdITYDqvy0SNsDQ0le49KmYTU57YoKmTf0olxU35uiW/ItN+Rsh6E?= =?us-ascii?Q?OS7qYeHGMZTCssIoFjabbQwGrsRdMi8SmL2T5RyDEscfY4HN5BxdF62k+1Oy?= =?us-ascii?Q?SgBBS1BEv8pnbDGx3Ii36PJavxqHGNHYsy1TlYwkfF2bPCm7k0KkTzPZY4mi?= =?us-ascii?Q?IqyBqCJdXlLgn36y5V0g4AyRhVqdZshFKmQp5WetunOzvgccqdVahPJD6mPs?= =?us-ascii?Q?t5u0Ols/ypRa6kGWb89xPN3AM2gcCm4AhGj25jw1?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB5872.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 73188498-f678-40eb-5c58-08dc767406a2 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2024 13:19:47.9212 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 2JjvTUenuOtFxwbSM7gzifY1PxNZYx8F3VQNeuyJ93qCzqRmBzlJXVDXcFX7ruAvU89+l6EFzwPVyWxXDPqA4A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB7781 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org >=20 > The current calculation assumes that the mbufs are contiguous. However, > this assumption is incorrect when the memory spans across a huge page. > Correct to directly read the size from the mempool memory chunks. >=20 > Signed-off-by: Frank Du Hi Frank, Thanks for the patch. Before your patch the umem_size was calculated using mb_pool->populated_siz= e * rte_mempool_calc_obj_size(mb_pool->elt_size, ..) With your patch we sum up the lens of all memhdrs in the mempool. When debugging I see the new calculation can yield a larger value, but the = new logic looks good and more thorough to me so I'm happy to opt with the n= ew approach. Acked-by: Ciara Loftus Thanks, Ciara >=20 > --- > v2: > * Add virtual contiguous detect for for multiple memhdrs. > --- > drivers/net/af_xdp/rte_eth_af_xdp.c | 34 ++++++++++++++++++++++++---- > - > 1 file changed, 28 insertions(+), 6 deletions(-) >=20 > diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c > b/drivers/net/af_xdp/rte_eth_af_xdp.c > index 268a130c49..7456108d6d 100644 > --- a/drivers/net/af_xdp/rte_eth_af_xdp.c > +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c > @@ -1039,16 +1039,35 @@ eth_link_update(struct rte_eth_dev *dev > __rte_unused, > } >=20 > #if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG) > -static inline uintptr_t get_base_addr(struct rte_mempool *mp, uint64_t > *align) > +static inline uintptr_t get_memhdr_info(struct rte_mempool *mp, uint64_t > *align, size_t *len) > { > - struct rte_mempool_memhdr *memhdr; > + struct rte_mempool_memhdr *memhdr, *next; > uintptr_t memhdr_addr, aligned_addr; > + size_t memhdr_len =3D 0; >=20 > + /* get the mempool base addr and align */ > memhdr =3D STAILQ_FIRST(&mp->mem_list); > memhdr_addr =3D (uintptr_t)memhdr->addr; > aligned_addr =3D memhdr_addr & ~(getpagesize() - 1); > *align =3D memhdr_addr - aligned_addr; > + memhdr_len +=3D memhdr->len; > + > + /* check if virtual contiguous memory for multiple memhdrs */ > + next =3D STAILQ_NEXT(memhdr, next); > + while (next !=3D NULL) { > + if ((uintptr_t)next->addr !=3D (uintptr_t)memhdr->addr + > memhdr->len) { > + AF_XDP_LOG(ERR, "memory chunks not virtual > contiguous, " > + "next: %p, cur: %p(len: %" PRId64 " > )\n", > + next->addr, memhdr->addr, memhdr- > >len); > + return 0; > + } > + /* virtual contiguous */ > + memhdr =3D next; > + memhdr_len +=3D memhdr->len; > + next =3D STAILQ_NEXT(memhdr, next); > + } >=20 > + *len =3D memhdr_len; > return aligned_addr; > } >=20 > @@ -1125,6 +1144,7 @@ xsk_umem_info *xdp_umem_configure(struct > pmd_internals *internals, > void *base_addr =3D NULL; > struct rte_mempool *mb_pool =3D rxq->mb_pool; > uint64_t umem_size, align =3D 0; > + size_t len =3D 0; >=20 > if (internals->shared_umem) { > if (get_shared_umem(rxq, internals->if_name, &umem) < 0) > @@ -1156,10 +1176,12 @@ xsk_umem_info *xdp_umem_configure(struct > pmd_internals *internals, > } >=20 > umem->mb_pool =3D mb_pool; > - base_addr =3D (void *)get_base_addr(mb_pool, &align); > - umem_size =3D (uint64_t)mb_pool->populated_size * > - (uint64_t)usr_config.frame_size + > - align; > + base_addr =3D (void *)get_memhdr_info(mb_pool, &align, > &len); > + if (!base_addr) { > + AF_XDP_LOG(ERR, "Failed to parse memhdr info from > pool\n"); > + goto err; > + } > + umem_size =3D (uint64_t)len + align; >=20 > ret =3D xsk_umem__create(&umem->umem, base_addr, > umem_size, > &rxq->fq, &rxq->cq, &usr_config); > -- > 2.34.1