From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A8B7B42361; Wed, 11 Oct 2023 15:36:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 95F4D402CD; Wed, 11 Oct 2023 15:36:56 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2084.outbound.protection.outlook.com [40.107.20.84]) by mails.dpdk.org (Postfix) with ESMTP id 5D036402AE for ; Wed, 11 Oct 2023 15:36:55 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PW2/5l2M1pF6PA4Gsr87OODGz3RdHB3J9X8ZbOP3Puh3jP+OuDPrfpONPLbnGbdydqjBuf3hIDNYFN1ChdtLnGQ2owhxSta6hQolo0dTWoqNzOFylph8LbxOokWUwCCiSv3U+i25AG2MqmfEeuClI/dn+hu/Zmzu1xy23aC02ZFY8SSOpFQS5+rKqMbWzJd+bHtbb4m1azFBI6B5ZSrBHxm2RMsX9+fcZMYNvY8recQoMuz0NQQL6+t5VnZbA/XvpVbcxv83bZx+xGk81JtSpeobbhmbJFfL4Be/f7573rX43JnY95d6XDRRb8bG1TWRrtMWfuyRbU1YGHC8vHFeOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bGE4/WpUXUfSYnLO7jNe3UcGGSVaTVoIHubHgE9FQdg=; b=jIlXu2OyZ0wucb2EIb8Sj/Yy2cCe02A6W2o+s9YZqjiI7hw7dS1r0DvOSIaKeiQG5QteKb8eal2YQ/JYH+VUxiyt4ynZPgoZv2avvaA0gMgEbvi5kIJSkF2cI5poNUs4YQ0Ynn6pTxzK25cB2lpBRRvf5jl8tiCaaO4TSMKlcodw352pJ7DcRF9zzZ+QzGLe0eCApjcZusOdg3E32XssgvzmR9eoQxMC++gVxlqxEVNNGn3fOpvOyRZm99pmWRgA696kff7CakGcX2KfawdEV7oHqJjaqNXDa6u4LZAbQh9tImAEKbeyV4dFaxJjQ3/6zkPOF9bI1Z0ZfdXBBLYS7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bGE4/WpUXUfSYnLO7jNe3UcGGSVaTVoIHubHgE9FQdg=; b=1K8bsLqt4PewUkwojiU4V012gb8jiV5ofjh4E09cC5s5NUM50Wy6yXDlVD7O1tjMxib7O0LyrzE1P5CtiET0q3j2KqbHQP7mkDnPzPXlWRnbm3jCSL9hHkKFKxTR+aYCkVHg1z2iTigrTurvmx0QihFVjHUSXDknEBLSPBIlYUA= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by DU0PR08MB8373.eurprd08.prod.outlook.com (2603:10a6:10:40a::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.38; Wed, 11 Oct 2023 13:36:53 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::caf4:1831:be7e:e57d]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::caf4:1831:be7e:e57d%6]) with mapi id 15.20.6863.032; Wed, 11 Oct 2023 13:36:52 +0000 From: Honnappa Nagarahalli To: "thomas@monjalon.net" , Paul Szczepanek CC: "dev@dpdk.org" , Kamalakshitha Aligeri , nd Subject: RE: [RFC 1/2] eal: add pointer compression functions Thread-Topic: [RFC 1/2] eal: add pointer compression functions Thread-Index: AQHZ+sjctTxzccyMwE2W35BbnsHj5bBD1e0A Date: Wed, 11 Oct 2023 13:36:52 +0000 Message-ID: References: <20230927150854.3670391-1-paul.szczepanek@arm.com> <20230927150854.3670391-2-paul.szczepanek@arm.com> <22038988.EfDdHjke4D@thomas> In-Reply-To: <22038988.EfDdHjke4D@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: D1AA9C04B654A6419DE5540AEE461282.0 x-checkrecipientchecked: true authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DBAPR08MB5814:EE_|DU0PR08MB8373:EE_ x-ms-office365-filtering-correlation-id: aa3c5fe5-ceca-424e-2073-08dbca5f2120 x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr nodisclaimer: true x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3RaRJlgT8fV/ZQmzds0YF+m0qwc1tJ4KSo9g6Q+i1Ec0jk1U+iI1s5Lv1u/uOjvGrjfADjBnhgmDe8i/DLSRge8NMnji/IEXPl0gZ5okvTvbeh5fANLs36MjKGAxuGgBwRBOxoplnROLR+pCLG2Qkub4oPXF+gjUZTjF5/TYi8bIXEYxK769IBjCcrgTNf53yxeFVtW2CvLmBkMzgF58ZwM2T4oCcoiSxe/8L4nSLnWU3U1V0jrRT6kcr51mcVPGHYgg79kNLswUzSDqqxAuQ7A/ZnaE6NWcDzgoSLedOpP4o8kAYpXgGxDZN7r+ZBXmkdYloQjt2XnVIwlM7O9XJC3N9izXdhWIx0in6uKQ3kSbvjUPlbxYXTl/6SFrswepu4K6yXI7ch/poZHnsj3it3nTr0Te4fksU06ydit8ls8u6LACS5DEaOAeJoM9hJ4ts3dsT7S/rmHM8Kf9gSycrLD6DcZhhoRZ+WEiZvROskXQzGQvJMxaHyxWbQMyzSrgv1JXYkT8aJswz09U6MS5JzwpCPE7S585/P9xs0V0i1YAefqwCozT9kaVPjsD3PaAObCJ/+xB/EbIPDSUsD6yfuanl2zBqeJeuFHgOfTNq6whAWtNhRlCUw0zQlnx/Nm2 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(39860400002)(366004)(376002)(396003)(346002)(136003)(230922051799003)(451199024)(186009)(1800799009)(64100799003)(55016003)(71200400001)(53546011)(83380400001)(7696005)(6506007)(66446008)(9686003)(76116006)(26005)(4326008)(41300700001)(478600001)(8676002)(8936002)(5660300002)(2906002)(110136005)(66556008)(54906003)(316002)(66946007)(6636002)(52536014)(66476007)(64756008)(38070700005)(33656002)(122000001)(86362001)(38100700002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?t4L/esrEckWH681FQBfLVerVujBVfL+igkmMNCMQ15tAyHPKRVRWJloXwWu7?= =?us-ascii?Q?fp7Chf5WP2OXdwrSr+gOLvQrcsTB4DBmtgj103oB1od2vwWVSLw/RgAyeZ6L?= =?us-ascii?Q?1Q0iHy891P7Q+ra/A00T9xWgenl6CBjwbmrCs822p3gqFYhL4J3kao7Livs+?= =?us-ascii?Q?nq8ftEmsaAIxekMvE/Z57WBhu9UdXvCqpPQfyvpXyMk/6DsIRliA5/QJtrKP?= =?us-ascii?Q?tiJlvsjNZ6E8pbl0RnA5ZPTMEoqNGFbL+uN18YDKeqOn8akMOXSl1QAOh3br?= =?us-ascii?Q?3sOcI6uDBalc6UMGjCCJpp5KStbFYBQFnw0i4OsX08vIsj0v6SmeyprOAVUW?= =?us-ascii?Q?9P1/1/8S/ofDLwtqIGJb1SOxJ11wUCDJWv/YcApYdoT5HvyhKXUaV43G/vYF?= =?us-ascii?Q?04FWMi8UvD1Pq9jA6Qa95rGY+gy7KluKb3Yl9PhBw1T4kZYm3Gl64t1o8fLV?= =?us-ascii?Q?veF77W5RRJUrvS/XOR9zaSjKgQdYrN4v/WwZ+KWF+Ltc9DQBfjoMg30Adpxi?= =?us-ascii?Q?EzkFP2lBMJRYoST1fUpDNFw9ymXwdpaG1JjNmTCRT/SBuH+kndSk5EZQAGHE?= =?us-ascii?Q?xs5Zrbb9iGcJwicfcVUHey+Jw59FnGT6MpC7kmMeClkfyxE9wdYNcl/03Lur?= =?us-ascii?Q?9A55sL/eXai0za/8l59lUKpdoNqr0nlNNeE+JpQdtt0qwYUjA2KEsAYrsm6x?= =?us-ascii?Q?GmNzuj/384fR1za6TZ0Rs+4TBxN2NCEzaGaVEuTfK7qFfAB9wtOi1l4zzYBw?= =?us-ascii?Q?1MNtV8lH5rM3bU0dZA4c3iX+UIll80ne3vKyGVTyf34Sk/M7LGTCkiWapF/E?= =?us-ascii?Q?epZKriKOORlnYdAw/O70FH/5anTzscnMidbmuBEnVzaX+rWhHq82IQjfB21u?= =?us-ascii?Q?IcQu1p14kxRyZgPGWzeI4n4BjkZ/jaxrs4dvL73QtxdAn5c0NWG8sPfZLXK8?= =?us-ascii?Q?cSGtvrGjPouxKFNOtURRZZ6Ole1Uh36AMgOhBAtWobzySLl6rbOqIuXCJ59U?= =?us-ascii?Q?wR9eAPHPf9+A+ej53nuF/qkj8GXePEejyOVGK4U3A816pm9QTGSwYCxiB2cx?= =?us-ascii?Q?2xoUiXZmNgS+Yjshy7NiJFGk9H11++yxYVcTzm4r5dLJWhi98wwRcnQQ5WNF?= =?us-ascii?Q?8HWzBRvmvceaBHOKPyYxggMXzhJTvcQbTw8Z5aLv/TasTmPwFnr+0p9s8c5X?= =?us-ascii?Q?tjuyjDROmGXm0ZcJKFkLUWyi/EzWD4VDPDw8JW+GOqqivq68bomYiDcyYTpJ?= =?us-ascii?Q?DOgEj+OmRg2dH4g5nlidvPPMaHp3ImNc4KHAFh4jXheXYtpRrmStz64nG6ZH?= =?us-ascii?Q?ImmeldZaRFfy+/bojDklaQTaXktG0AO2RZ2dab7CaPR9iUUF85RuJS/LzNTd?= =?us-ascii?Q?wXUECOPm9z/rgg5PTcNQGb7skPINGuj23oK7oWW6xLYncrpJiwxXZOHmlfiR?= =?us-ascii?Q?/cyMFZ0NeyHHBfZlwbVU4CU+pL5gg13TvMhIEVbP+7rTz8MHAk1hFQYA0WeH?= =?us-ascii?Q?PVc5dOCP9xDumcId/bthHi3tJiWVT1X/Lkgki7UiUk90oKkaJk64AhRMv12v?= =?us-ascii?Q?0TWQXpkED1no8JzUQQ5Dl1Nkijj2b3ucsvcCzHlLv1xis1Lsi1nwX8TiXyTk?= =?us-ascii?Q?ag=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DBAPR08MB5814.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa3c5fe5-ceca-424e-2073-08dbca5f2120 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Oct 2023 13:36:52.8925 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: yCRFr1IenEWBaGseaX05Y7ZKcCpK++auPzqC41Tx10d//1BpQUPqEevw9sgz4+4YsB/H+oOzqcKiWcRI+EW6znCXxltFg6cGwm7z9sfgIZI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8373 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Thomas Monjalon > Sent: Monday, October 9, 2023 10:54 AM > To: Paul Szczepanek > Cc: dev@dpdk.org; Honnappa Nagarahalli ; > Kamalakshitha Aligeri > Subject: Re: [RFC 1/2] eal: add pointer compression functions >=20 > 27/09/2023 17:08, Paul Szczepanek: > > Add a new utility header for compressing pointers. Pointers are > > compressed by taking advantage of their locality. Instead of storing > > the full address only an offset from a known base is stored. >=20 > You probably need to insert some explanations from the cover letter. >=20 > > The provided functions can store pointers in 32bit offsets. > > > > Suggested-by: Honnappa Nagarahalli > > Signed-off-by: Paul Szczepanek > > Signed-off-by: Kamalakshitha Aligeri > > Reviewed-by: Honnappa Nagarahalli > [...] > > --- a/lib/eal/include/meson.build > > +++ b/lib/eal/include/meson.build > > @@ -35,6 +35,7 @@ headers +=3D files( > > 'rte_pci_dev_feature_defs.h', > > 'rte_pci_dev_features.h', > > 'rte_per_lcore.h', > > + 'rte_ptr_compress.h', > > 'rte_pflock.h', > > 'rte_random.h', > > 'rte_reciprocal.h', >=20 > Did you try to sort alphabetically? failed :) >=20 > > +#ifndef _RTE_PTR_COMPRESS_H_ > > +#define _RTE_PTR_COMPRESS_H_ >=20 > No need extra underscores. >=20 > > + > > +/** > > + * @file > > + * RTE pointer compression and decompression. >=20 > RTE has no mean here I think. >=20 > > + */ > > + > > +#include > > +#include > > + > > +#include > > +#include > > +#include > > +#include > > + > > +#ifdef __cplusplus > > +extern "C" { > > +#endif > > + > > +/** > > + * Compress pointers into 32 bit offsets from base pointer. >=20 > I think it should be "32-bit". >=20 > > + * > > + * @note Offsets from the base pointer must fit within 32bits. > > + Alignment allows > > + * us to drop bits from the offsets - this means that for pointers > > + aligned by > > + * 8 bytes they must be within 32GB of the base pointer. Unaligned > > + pointers > > + * must be within 4GB. >=20 > Not clear what is "alignment". > > + * > > + * @param ptr_base > > + * A pointer used to calculate offsets of pointers in src_table. > > + * @param src_table > > + * A pointer to an array of pointers. > > + * @param dest_table > > + * A pointer to an array of compressed pointers returned by this fun= ction. > > + * @param n > > + * The number of objects to compress, must be strictly positive. > > + * @param bit_shift > > + * Byte alignment of memory pointed to by the pointers allows for > > + * bits to be dropped from the offset and hence widen the memory reg= ion > that > > + * can be covered. This controls how many bits are right shifted. > > + **/ > > +static __rte_always_inline void > > +rte_ptr_compress_32(void *ptr_base, void **src_table, > > + uint32_t *dest_table, unsigned int n, unsigned int bit_shift) { > > + unsigned int i =3D 0; > > +#if defined RTE_HAS_SVE_ACLE > > + svuint64_t v_src_table; > > + svuint64_t v_dest_table; > > + svbool_t pg =3D svwhilelt_b64(i, n); > > + do { > > + v_src_table =3D svld1_u64(pg, (uint64_t *)src_table + i); > > + v_dest_table =3D svsub_x(pg, v_src_table, (uint64_t)ptr_base); > > + v_dest_table =3D svlsr_x(pg, v_dest_table, bit_shift); > > + svst1w(pg, &dest_table[i], v_dest_table); > > + i +=3D svcntd(); > > + pg =3D svwhilelt_b64(i, n); > > + } while (svptest_any(svptrue_b64(), pg)); #elif defined __ARM_NEON > > + uint64_t ptr_diff; > > + uint64x2_t v_src_table; > > + uint64x2_t v_dest_table; > > + /* right shift is done by left shifting by negative int */ > > + int64x2_t v_shift =3D vdupq_n_s64(-bit_shift); > > + uint64x2_t v_ptr_base =3D vdupq_n_u64((uint64_t)ptr_base); > > + for (; i < (n & ~0x1); i +=3D 2) { > > + v_src_table =3D vld1q_u64((const uint64_t *)src_table + i); > > + v_dest_table =3D vsubq_u64(v_src_table, v_ptr_base); > > + v_dest_table =3D vshlq_u64(v_dest_table, v_shift); > > + vst1_u32(dest_table + i, vqmovn_u64(v_dest_table)); > > + } > > + /* process leftover single item in case of odd number of n */ > > + if (unlikely(n & 0x1)) { > > + ptr_diff =3D RTE_PTR_DIFF(src_table[i], ptr_base); > > + dest_table[i] =3D (uint32_t) (ptr_diff >> bit_shift); > > + } > > +#else > > + uint64_t ptr_diff; > > + for (; i < n; i++) { > > + ptr_diff =3D RTE_PTR_DIFF(src_table[i], ptr_base); > > + /* save extra bits that are redundant due to alignment */ > > + ptr_diff =3D ptr_diff >> bit_shift; > > + /* make sure no truncation will happen when casting */ > > + RTE_ASSERT(ptr_diff <=3D UINT32_MAX); > > + dest_table[i] =3D (uint32_t) ptr_diff; > > + } > > +#endif > > +} >=20 > I see it is providing some per-CPU optimizations, so it is in favor of ha= ving it in > DPDK. > Other than that, it looks very generic, so it is questionable to have in = DPDK. We had it done for mbuf pointers. But then, we thought it could be generic. Right now the API results in 32b indices. We could make it generic further = by allowing for 16b indices. 8b indices does not make sense. >=20 >=20