From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 62CEEA0512; Wed, 27 Apr 2022 02:42:29 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 024D140E78; Wed, 27 Apr 2022 02:42:29 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2061.outbound.protection.outlook.com [40.107.21.61]) by mails.dpdk.org (Postfix) with ESMTP id A67D140691 for ; Wed, 27 Apr 2022 02:42:27 +0200 (CEST) ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=DUskEh3nRtxb8GFpJRBOnNGJl6ruSwrx+Ac97ZLNGls/tGKlFA5owFJWeRxj1M9aX7BNiACSY4z2sA7KlrSWJfceRGzTl51kH4OesEe2KudAtAMzL2l6S1XjEEBM6/6p+9JXgyAAqgCuBpg8poUI5141a+wZQPeg3dZUrFxOw2NHorkQ5U9hWP8FGovIAhx5yz8A/txRB3vTz+qgFFhO/FR5sRw5XcRwBtmCJBWhEg9acV9h4ssRDgPfICzQsxOKpFShFIYOPlupUb999ZDjlWKAJbnzfkOOP45e4+xNaMmKMKRxMAdLozGK/fz7tavPL0Ee5267p2FpN4R8XkS4tw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UH/MP21I1ac3tb5PNe9clqJah0DNYfHEUUUZj4gHLdw=; b=hTaEmDV8MRGpq30ZrLzigeonJwRPpxel5rt64VPIlBcYu0e80zMMRVSxo+bCMNaZzju909eJ49BWk9q9Rmn03x4zm14nZCo+TWwICbdRS6hUqZTqcK+IDoGemVGA1aWBkZV62X21DhS50PC8mIAVqElK0o7SpxwugNoZAbGvExPySqeWlt+c0JAnNRzR3YtwFmI6zU8q5ioJaL63K/zRKwhKh76zQUXhqyF6VeKbIVydLsrjzJi7eeky+MqZukmITHRVyFJyNfJJEMsbLry2gL3iL31fUZVFeM7Wz+f/OVdUYS+7luCSC0JO0BtZlC2eY8ZfHM7EYCpXiNKbikUSYQ== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=dpdk.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UH/MP21I1ac3tb5PNe9clqJah0DNYfHEUUUZj4gHLdw=; b=iqhYqaWufBngoGXelf744gSw1CWDi5mhAKtLd9q2mTSXyz/EzvZWaZVpOefYjwean7jCuHxOS8hlgvKE/zaXjJtXAlPE9aVkkSPYirjrX35yCedLfsnrFeryBjihJVYPK8XqKDUUTpKMu6YG9Fr2H1GPzDXQv9zSrGZZEfMZGEs= Received: from DB6PR07CA0009.eurprd07.prod.outlook.com (2603:10a6:6:2d::19) by AM6PR08MB3992.eurprd08.prod.outlook.com (2603:10a6:20b:a4::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Wed, 27 Apr 2022 00:42:21 +0000 Received: from DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com (2603:10a6:6:2d:cafe::fa) by DB6PR07CA0009.outlook.office365.com (2603:10a6:6:2d::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13 via Frontend Transport; Wed, 27 Apr 2022 00:42:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT011.mail.protection.outlook.com (10.152.20.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.12 via Frontend Transport; Wed, 27 Apr 2022 00:42:21 +0000 Received: ("Tessian outbound 62985e3c34b6:v118"); Wed, 27 Apr 2022 00:42:21 +0000 X-CR-MTA-TID: 64aa7808 Received: from 94eb5f9f3739.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id CF292B19-56E3-4406-AFEE-5F53ADB6267A.1; Wed, 27 Apr 2022 00:42:15 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 94eb5f9f3739.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 27 Apr 2022 00:42:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HkNUrCvVjJ27mZEICjBOyVWgp/OvqBcvtEJucxrijxtQK+xMyEa1jDJpj97YtTqgVhOpdMT6voin7lLtvsI/RH7tcF+lrW83dzjDqbGJX1LW88Ovj+0E3+oM/cJ/fqxblJy3gXaHbWSnizpI5bRXuHh6bybUkxem/Nv5ObN3gUgfh5TGIqKIaPvpG4xzlbP4KJtrAg3Kp74LrP4H4j2jRe+hI0HuTkSNQwRKnA0WHnnCyEWDnj/KoLxrwyky2mUBhzYGpHLLnMKISdWzR5GlzkiQ8SbtY7iWwGbdHjnjKFhPVnNTRGE92JMZL+5siG7XShLwQfL4pvIA55YOf6o+vA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UH/MP21I1ac3tb5PNe9clqJah0DNYfHEUUUZj4gHLdw=; b=ZIdwSbdzwyKD6ZvQgI8cVAUhOLe/6sgI5X/F9xSsf/Xq8ITPOvSGNqpk1KHxD1JFmK01sdkzwZZMCk268QtSrYATccZXQ3VebK3nSMMCVe/IjbEFDTG3JNr+KvBRXFpXtsqlCbcHbl8i3BZtM3pthnjYr4cdYWOikjlidu78a3QDW6AlR23d7EIsWB2XBseBJueG0KVSw/PTq3FN6pKRrSdrsc54hDt2sr6yfgcO/9EQe+9LBlCTOV5o3FUNtt683WyYaglDaheI6GbKyY0IzugdYgZD/UY2OrtVy79exvpqyAk12xVtr9EcrKpEn0RxVV3LEGf2ZUYBSNkArzSF4g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UH/MP21I1ac3tb5PNe9clqJah0DNYfHEUUUZj4gHLdw=; b=iqhYqaWufBngoGXelf744gSw1CWDi5mhAKtLd9q2mTSXyz/EzvZWaZVpOefYjwean7jCuHxOS8hlgvKE/zaXjJtXAlPE9aVkkSPYirjrX35yCedLfsnrFeryBjihJVYPK8XqKDUUTpKMu6YG9Fr2H1GPzDXQv9zSrGZZEfMZGEs= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by AS8PR08MB7190.eurprd08.prod.outlook.com (2603:10a6:20b:405::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.15; Wed, 27 Apr 2022 00:42:13 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::7516:547a:f52e:2be7]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::7516:547a:f52e:2be7%3]) with mapi id 15.20.5186.021; Wed, 27 Apr 2022 00:42:13 +0000 From: Honnappa Nagarahalli To: Don Wallwork , "dev@dpdk.org" CC: nd , nd Subject: RE: [RFC] eal: allow worker lcore stacks to be allocated from hugepage memory Thread-Topic: [RFC] eal: allow worker lcore stacks to be allocated from hugepage memory Thread-Index: AQHYWWgKgtypg5UDzUyh2XXS8J9p/K0C6JXw Date: Wed, 27 Apr 2022 00:42:13 +0000 Message-ID: References: <20220426122000.24743-1-donw@xsightlabs.com> In-Reply-To: <20220426122000.24743-1-donw@xsightlabs.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 9BEA03D04EFCF04BAA4C7FB6DB333646.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-MS-Office365-Filtering-Correlation-Id: 70d73c5f-3ce5-40cc-ca32-08da27e6c9f4 x-ms-traffictypediagnostic: AS8PR08MB7190:EE_|DB5EUR03FT011:EE_|AM6PR08MB3992:EE_ X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: PlbMWJmYj8ehjrdFetUh+zyL3zzw0aTf5YJ4PAnbqEgGauaBn7gapwrEFc92zk7Sj2h46qT0CcT5gZ70b9kp2mG3EVl7eglVpzguT9hbX+5T2vCPKReVD/xAR1rmyQYdZz1p0+Xd3cCiSnfR2QYWmn8i6hWNAGO4B2gmrAYVCJOuv+D/G8Brofq1T+6ZnLn0pJFpLZqYV4q1HLE5F7wCTyYNmlBaSImuu3pKILtLMn1CmMNJWtWhf7yn3wIKcAY0Ux6n2wfONi/PzzhrroKOf+aQ4g+1nCbpfmILF71jcnQrTwdNd2l9eyrQ1JKdAStAj6Aj8+PRozihPPCw7EAiJXAx6sdgT2CWeifYYSzOD/FXGmn2bH2frlP6TxHofzU1YJAVsu2b/CUdnUPo9OwVW8ANUjD3QumDOXT3vklQHhcVYkEJJ57BmQ104v+46ksR7OeFpRrzgw4hqd+j2B03xFugIY1/U3FojOYKFz+o/JwPXJhBUqLKhbMvVz4ORNSViYw+p/phaoetnpgo+srjTDi0hN1/XVkSpvBOYzhV5mUTtNTZqHZgMb/ne0G0b38VQ6rmQTwtWDQw/3yk6mhbfzzcguBtbtqfyQASl8GZrr9d/9UO9bgWU3wNDcUtt28ag+JShA0YBqlnQvzSYIZcpSq12fuI7IrCX8w90XKnN9jGbfS4glkOccw+lfm1NtbYzOxDosTCBGRzO9ITrvosxg== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(76116006)(4326008)(64756008)(8676002)(66446008)(110136005)(66946007)(316002)(8936002)(5660300002)(33656002)(6506007)(7696005)(55016003)(54906003)(52536014)(508600001)(122000001)(66556008)(66476007)(9686003)(71200400001)(2906002)(83380400001)(26005)(186003)(86362001)(38100700002)(38070700005); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7190 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 6c7f11d3-2534-425a-7931-08da27e6c581 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: a3VqYHE52xkSTzIDhVlExUrrvNQrc7ntlaLj+xW5HehP1By6iRLZkXE4vbSIA+OuDwq2Ynlh7BvPiKsV95wztym4yOJu3VS3TNIpX6MF7+q5/lWAupkTj3mJpGiO8xUob/Z0w3XnBgrDSL+mFxYo01shxaSz9frDd1RugOhQ/Cj+7RNfiGAmQlFSEuxvKN8hcZfmBk6Er3QlUyYnRs6gu0EauA8Dq42NR9nQ5oTdreSfOXlx1b2AIeoUXpdEaMrf7bTq2WzTl0nTMRqXQm/koRuyS6MPH28yaV6DRUXd6efSI7c9oaf1ZOzjsi19LFMaPVxfIwu/dXSuFiYGMNxhq/VulMPC32h67l36N0KY8FjuaDOHcT4m/BxbtEQB0qTl0ydXVhTRGvm1XAhgdVX7G6I1P/Ew4uYSCDGDqlGmq/+natM1D9vKqOKwfrOztFNIGPGSj+OB+ACUjhNLPKLsZ8Z2oZ2Z7yliCnUshGNg+fR1q/m1bGaxYa+yaj6W4+JJF7vjJ4RY6jF65jM0vhATJxWn6gmWtfU1WIvqco19BExZqZd7DRgRhVJ9gUVxO7yu1IPgQPC+YD/9FFGa3xJdFEbENoJmnpByNBCB1WXOCk2Se4L5t7av+W6H+ScQ6lCK1yaKJl5u60AzJx5P8Z2IVC7wrUtAWLtKje/UpGXWseEZBE1sNEO5ezs3W/gSuGn9 X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(33656002)(110136005)(54906003)(86362001)(47076005)(81166007)(356005)(508600001)(8936002)(40460700003)(52536014)(2906002)(55016003)(83380400001)(36860700001)(5660300002)(82310400005)(336012)(4326008)(70586007)(70206006)(8676002)(186003)(26005)(6506007)(7696005)(316002)(9686003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2022 00:42:21.0570 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 70d73c5f-3ce5-40cc-ca32-08da27e6c9f4 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3992 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org >=20 > Add support for using hugepages for worker lcore stack memory. The inten= t is > to improve performance by reducing stack memory related TLB misses and al= so > by using memory local to the NUMA node of each lcore. This is a good idea. Have you measured any performance differences with thi= s patch? What kind of benefits do you see? >=20 > Platforms desiring to make use of this capability must enable the associa= ted > option flag and stack size settings in platform config files. > --- > lib/eal/linux/eal.c | 39 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 39 insertions(+) >=20 > diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index > 1ef263434a..4e1e5b6915 100644 > --- a/lib/eal/linux/eal.c > +++ b/lib/eal/linux/eal.c > @@ -1143,9 +1143,48 @@ rte_eal_init(int argc, char **argv) >=20 > lcore_config[i].state =3D WAIT; >=20 > +#ifdef RTE_EAL_NUMA_AWARE_LCORE_STACK > + /* Allocate NUMA aware stack memory and set pthread > attributes */ > + pthread_attr_t attr; > + void *stack_ptr =3D > + rte_zmalloc_socket("lcore_stack", > + > RTE_EAL_NUMA_AWARE_LCORE_STACK_SIZE, > + > RTE_EAL_NUMA_AWARE_LCORE_STACK_SIZE, > + rte_lcore_to_socket_id(i)); > + > + if (stack_ptr =3D=3D NULL) { > + rte_eal_init_alert("Cannot allocate stack memory"); May be worth adding more details to the error message, like lcore id. > + rte_errno =3D ENOMEM; > + return -1; > + } > + > + if (pthread_attr_init(&attr) !=3D 0) { > + rte_eal_init_alert("Cannot init pthread attributes"); > + rte_errno =3D EINVAL; EFAULT would be better. > + return -1; > + } > + if (pthread_attr_setstack(&attr, > + stack_ptr, > + > RTE_EAL_NUMA_AWARE_LCORE_STACK_SIZE) !=3D 0) { > + rte_eal_init_alert("Cannot set pthread stack > attributes"); > + rte_errno =3D ENOTSUP; EFAULT would be better. > + return -1; > + } > + > + /* create a thread for each lcore */ > + ret =3D pthread_create(&lcore_config[i].thread_id, &attr, > + eal_thread_loop, (void *)(uintptr_t)i); > + > + if (pthread_attr_destroy(&attr) !=3D 0) { > + rte_eal_init_alert("Cannot destroy pthread > attributes"); > + rte_errno =3D EFAULT; > + return -1; > + } > +#else > /* create a thread for each lcore */ > ret =3D pthread_create(&lcore_config[i].thread_id, NULL, > eal_thread_loop, (void *)(uintptr_t)i); > +#endif > if (ret !=3D 0) > rte_panic("Cannot create thread\n"); >=20 > -- > 2.17.1