From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B32BEA00C4; Tue, 26 Apr 2022 14:20:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5DDDC40E78; Tue, 26 Apr 2022 14:20:27 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2072.outbound.protection.outlook.com [40.107.20.72]) by mails.dpdk.org (Postfix) with ESMTP id 9720540C35 for ; Tue, 26 Apr 2022 14:20:26 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HAatzi/IKH+c5yTgz9Q/r1Pk0NiGNMDjw9IKLvy3y3GJXKbYYzktNaOKv61ihn+QvACIBdK7hbv4YjeaNBwJDXSWvzK7p8YUOr6rY8f3zSAwAWWeR3udzuEkqRoipJCJYG9i+KMV1X22fWiU6e7OcT+eh5cs49sccrK81yMvP/FHSMvuTrJG13R+HCbOf767M+Z69c5eN9MBq7zWOFya5fWGJvteHG0P++023JkVO/TLpHksqykb6MsYFw6nNLMzTNky4HZjjrnHxOk5xjy6smBDDimhBey5m5mIF2DEICF5TgdhEvF8VhokegFhAEU4GHm2Z6dHKbYGPF+kopA4Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2q694+384gxctZJUF8XAG2iIDDoM08XrtClzFd3lNvg=; b=T6WumynJrp/ArEs/IMOWg/ys1clc3/mCJHE/bdkSbqiHXeJU8bllm8Pdmj3e3LQa2hBmMF6hhuzJfB3XezEsMVLi2lsf2jqvHI0iyHIeNv+aLDj9CxeOmyM4cWxLFO9dF1jRmoACbrtGpvYeu/7K4Obf9+dFqFjbmaIczqmrPgPAUGb0OeqncYhOLVe3Ro8jXcvygNwI6hYs97dmWYOfr3akADlih7XcZsX6SMTsjzo861E+0H/QlzS8NHt8U6YE/I6oIRbO8BG9o+PnshyGXG9MG3mGzdPttzJtI7y1MQJH6ZQcV3NKxfumalU4LJFJvadO6IXjyC+2qda3Jv5lBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=xsightlabs.com; dmarc=pass action=none header.from=xsightlabs.com; dkim=pass header.d=xsightlabs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xsightlabs.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2q694+384gxctZJUF8XAG2iIDDoM08XrtClzFd3lNvg=; b=RqAIUrtfQMEEcvnG98p70L3Kk+hk0xh2JqEXvzaVkp9m7I8JVB3dFdNVAv1LO1Z5ZS/vrPXS7ADsqkMLA6vF0A11V6X6xmHNWIjE/XpDVu9vfAs+Y3BjZ0WB4Gk6su3ZEeKlWhqnvp71CW0KJ7y++6+6cNiVMppiJZLJCcIcpN8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=xsightlabs.com; Received: from DB9P193MB1482.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:2a6::7) by DB8P193MB0759.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:144::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.13; Tue, 26 Apr 2022 12:20:24 +0000 Received: from DB9P193MB1482.EURP193.PROD.OUTLOOK.COM ([fe80::281a:860b:67a1:2cd9]) by DB9P193MB1482.EURP193.PROD.OUTLOOK.COM ([fe80::281a:860b:67a1:2cd9%9]) with mapi id 15.20.5186.021; Tue, 26 Apr 2022 12:20:24 +0000 From: Don Wallwork To: dev@dpdk.org Cc: Don Wallwork Subject: [RFC] eal: allow worker lcore stacks to be allocated from hugepage memory Date: Tue, 26 Apr 2022 08:19:59 -0400 Message-Id: <20220426122000.24743-1-donw@xsightlabs.com> X-Mailer: git-send-email 2.17.1 Content-Type: text/plain X-ClientProxiedBy: MN2PR19CA0025.namprd19.prod.outlook.com (2603:10b6:208:178::38) To DB9P193MB1482.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:2a6::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b01095ab-10c6-4e18-d7b4-08da277f2412 X-MS-TrafficTypeDiagnostic: DB8P193MB0759:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: l2MmHs3/iAClR8ExVnS/1jbGl/obOyII7vR586vUvaGffotWSWR2J8aa/EL1mOhrt8mkdlge846gfRh8PKL0Dg51J89++L1UOYAK2SAmNr93KWmdbZStt4tMkP4FUolh7mqdXjDczA7bZ0woUl2dtQz9lFP987hHqgqYEyWimEusqoaS0Q6KJru4gNb5iLr9GBKk8Gn8wvLEHGXw5IDuMGtNCCiZgrK/DUf+Ctf25YQWI5npbBWLyrfPF+/YGyetklhAWrBK9pa10JGqS+4nXgrdqe3MFdQz2rSHpb33br0dV7vyOuNUQQ/qfOndYxd8kE2+/sscotdKKLQYwwAWPlOr/Eu5XlNGbqu8PpQBr5lzefkQNPUCOESVloFtxtb/cszqcxDh7q+mty7Wh/LrI519w5wvUb1F0JG9EMhC2L8KLCtaF4iUdsNg/5K6mt0HYDpp/HyspXFukaWrKfA+ZNoHNxXgr4RHcM4Gzcc9CBdA7+0aQ+gwKq5KGdqTcmaGTAX6d28HZtoO0IQkl1w7Fy49ln7WxzebAzu3ZN0Bnx9E7u3Ke+BAXH3LWbadwJnNQihKC6GgiB61ujIDMNI4kuAqYwC4WUmy/8R0/ZeVSYQG85b7kNHX4D+P7vVoIQ5evitbXgEj5vHC8pA+KQy58INldzRm76eefY+pRVyklSDaWQNTwUDENPks5SxCvqfHcJlE+SL/3ZxrSY2SX8evHQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB9P193MB1482.EURP193.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(83380400001)(38350700002)(38100700002)(186003)(52116002)(6506007)(1076003)(86362001)(508600001)(36756003)(26005)(5660300002)(8936002)(2906002)(66476007)(4326008)(2616005)(8676002)(66556008)(6512007)(66946007)(6916009)(316002)(107886003)(6666004)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?rwY/54jnzl7O8xw4U7a70YgkUZ6ksThpKCQkSyTKxKe3WPJ3Ofeal4LvBctt?= =?us-ascii?Q?dD8PM/E6LIMUehebxAWrXvoxeqHhYz0TriN5PaXY0OZRz9AYmd8wiWlv/xWU?= =?us-ascii?Q?3CgEsf7+8MPUc8NGSy7rwqC+mHmOywZYphxLtQ7Klq/ZvWeLHcZpYdgGEr+U?= =?us-ascii?Q?yKPbuxBEY8KS5BLc1zas6HvqC3V4CA1gSDNEAUrpx8kt22P/R4pAUb3jsHPl?= =?us-ascii?Q?v4CuCzsuoCrpYgm30CFsMTb++voz7wACyiMqgOVCr9f/LtfvmCtMOLXdrSZy?= =?us-ascii?Q?OU7brFl/QsRZOAB+vTxHOmvGnS0Mju8fuRcu7Iz/zpFP1oTMxdW+ThFz20pR?= =?us-ascii?Q?QyzaSkVPPcNyOWC1n7TtKpptGR6IaS53vGuGbNsc6cP60dtlCrVqXR+Gy8Zp?= =?us-ascii?Q?nfly58z5iMP1N5phz2WrTBF0RSyECDo6wvIuGJG9aUeck7Zs1BzyL0iQxpsR?= =?us-ascii?Q?FQ8u8NBjDEQAtM7lQORRD8IYfVb0xG2uU4XgPMx+GGsEdBw0fs9cWL/wLA/v?= =?us-ascii?Q?RO0E7Db5H0bYmJEtWdOIQc4+TU0d7FoE0fvG4lk26lgV1vKM/O1oQRarxipD?= =?us-ascii?Q?scSfJfTOXuIC3DlrVdgOihKxLm8iyHnQScbeIoGjCaPUC+Xfh8na1RI6ImWB?= =?us-ascii?Q?ccjSr++8cso5eRa2PdHB+shcrMB9cc820JHuI22siBdtuP7LuK88BxSlPBfJ?= =?us-ascii?Q?JXoknUfWSB2SMxre49v0thx/ljF6c07N9iF+JeGCAnfZhJklbjlxB2PgRKg7?= =?us-ascii?Q?mqCsw18cWWqGYRwvliFc9CG+d7eJaqYoF+JBusAhjW2MTClNugYRFM12uRYr?= =?us-ascii?Q?A7U0ywxwXUFvG9ktUNnoUIEPKZpr81NDS9Q99z4oWmF5rNBrVwR9YltE12SX?= =?us-ascii?Q?f0RDEcKaerVEbC/Sxz7MNb5/xr5/7N10EkRbnB6Zla9mQvCxyy9hBaEbb22T?= =?us-ascii?Q?8VS+ACbykJMwC/zmNaW5BhvlESoJDjFyF/rmAkTnz9hP+i3cbY28poFBiIlx?= =?us-ascii?Q?PLLAvNjr9IxrU3LdbcE9egGXMKCFcKmsYlhoC3vqR/mhLPnAQbV6QsamTlVX?= =?us-ascii?Q?Bq5uQe18r6aL546v4mpQYsMSvgb2VfJo5faf6zi6iu7Gzvl5fxqWQ2wsXVg9?= =?us-ascii?Q?t3U/z0umPO9NhbV3+83f7lCz1szf0LFgQ2xYYJWH9qZw0ID9Zaw+xcE92V8K?= =?us-ascii?Q?Z2mejuJgIuU3jhoTPKoQ5dQ0OS0MrhpJaA/mmOHTMSp4y7skhf5mB9sUE/nw?= =?us-ascii?Q?MX2FFchyT5VdxNuGqk0TtVFGHejfeBNgPepPgdGsRQHFYmDWLEuAx8G7F4PU?= =?us-ascii?Q?7t+ybryjVC10NKiI3w3KNZROBhwlaTfMSLNTXzAYoBuhR/oCRtjAQiKPQWdV?= =?us-ascii?Q?7eYvTer+J3UI4pjF9FGuqUSJQNiuEBZG4K3LjZ73ifyZDr7+MTp6msndNmiu?= =?us-ascii?Q?riDAvo7Gk45/468njZ+ppXjMAVGxKA722aYumSe3jgwSULpBnSL+McSbBoOe?= =?us-ascii?Q?Zm++xegpdEzi9p8eIoLn2Moaq2wp51djyeCBoE3WyD4miatDvXz/yNvmoSnQ?= =?us-ascii?Q?OPQrfs0qNCVOucEAT0+nTT/woVHbtL00JpRiRPEoIste21EP2sc6sCiIw08H?= =?us-ascii?Q?3wxvHQnkQXmoI5VqKNS+ag9ydzB58PRZEqGAkDQ1eL/36R4OXImMX+PjS6CD?= =?us-ascii?Q?qzgwAMMo9s3FMq1ABfZ7F4UOErQa0sEubEuyQbxuPpb8cqwQvYV6HyO/LxDj?= =?us-ascii?Q?fzKVHwc31g=3D=3D?= X-OriginatorOrg: xsightlabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: b01095ab-10c6-4e18-d7b4-08da277f2412 X-MS-Exchange-CrossTenant-AuthSource: DB9P193MB1482.EURP193.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Apr 2022 12:20:24.6984 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 646a3e34-83ea-4273-9177-ab01923abaa9 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: HIkV6LgTsXafWzboVE+/s6FVmV5Z2ES/BZj4Bqu1i+bQKZNGBY3VDm0zzV7HdG25SQN/is780nO1S4Csl4pTEA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8P193MB0759 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support for using hugepages for worker lcore stack memory. The intent is to improve performance by reducing stack memory related TLB misses and also by using memory local to the NUMA node of each lcore. Platforms desiring to make use of this capability must enable the associated option flag and stack size settings in platform config files. --- lib/eal/linux/eal.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index 1ef263434a..4e1e5b6915 100644 --- a/lib/eal/linux/eal.c +++ b/lib/eal/linux/eal.c @@ -1143,9 +1143,48 @@ rte_eal_init(int argc, char **argv) lcore_config[i].state = WAIT; +#ifdef RTE_EAL_NUMA_AWARE_LCORE_STACK + /* Allocate NUMA aware stack memory and set pthread attributes */ + pthread_attr_t attr; + void *stack_ptr = + rte_zmalloc_socket("lcore_stack", + RTE_EAL_NUMA_AWARE_LCORE_STACK_SIZE, + RTE_EAL_NUMA_AWARE_LCORE_STACK_SIZE, + rte_lcore_to_socket_id(i)); + + if (stack_ptr == NULL) { + rte_eal_init_alert("Cannot allocate stack memory"); + rte_errno = ENOMEM; + return -1; + } + + if (pthread_attr_init(&attr) != 0) { + rte_eal_init_alert("Cannot init pthread attributes"); + rte_errno = EINVAL; + return -1; + } + if (pthread_attr_setstack(&attr, + stack_ptr, + RTE_EAL_NUMA_AWARE_LCORE_STACK_SIZE) != 0) { + rte_eal_init_alert("Cannot set pthread stack attributes"); + rte_errno = ENOTSUP; + return -1; + } + + /* create a thread for each lcore */ + ret = pthread_create(&lcore_config[i].thread_id, &attr, + eal_thread_loop, (void *)(uintptr_t)i); + + if (pthread_attr_destroy(&attr) != 0) { + rte_eal_init_alert("Cannot destroy pthread attributes"); + rte_errno = EFAULT; + return -1; + } +#else /* create a thread for each lcore */ ret = pthread_create(&lcore_config[i].thread_id, NULL, eal_thread_loop, (void *)(uintptr_t)i); +#endif if (ret != 0) rte_panic("Cannot create thread\n"); -- 2.17.1