From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0FDF7A04FF; Tue, 3 May 2022 21:47:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 972A440C35; Tue, 3 May 2022 21:47:03 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60057.outbound.protection.outlook.com [40.107.6.57]) by mails.dpdk.org (Postfix) with ESMTP id 45AA140691 for ; Tue, 3 May 2022 21:47:01 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TjF2a/siCHDtO5OyXWanlmipq1BVF3Z5/ITqSCxFshQk+JO/nHxuqvqY20DrD5IizCvRFixACrnmgVomuxeRRWxBWhmIMdWFMqFUTTAT4lG0YJvLHC06N5HVARBgUQRRtDvGnxXOtPu7+Q4f9sZZLfyLEBKdAE7ffkP8ESXm3SJsDn3lBBrA4u1ic+4pPQpymOXChWlEULSQDsU6J4zGfUfiRHmgoz1s96mPL2SRIJXEMSFgNWJiX3doksOTVSn/H9nuNhJKMZ8C/YKEK7y9DK3xsiqwn9/5/XBp61xZVs55Oz3mSqDGPrvSJdnVc1+Kga2lyKCA7PsbgwsZIRun3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bvz9p9F6JFMwqkVM9XngQiwYoZEe5f3f32HJYhX7QBg=; b=Kcao84JFXomDl6iihY2uMufqOP7M/0gMM2qGgtPV6WEmH9SdUM9JRiOeO+Sj07WIxjMTLoyB2q5wSOjSbgMioA+tmvYvatoxe5WOe02tA75bimAYkEfFSqiS16Bu4e9S1/8q8h0E5IqfxlF7/1pPqrJPLic1p3r5mfHVvSnXMgCb3ZatyfGzPcPjHbNes9OEJIkrWJ5WmRHBTCs1Stag9L5ia72FMHgXxE20d6r5lsb0dmw/J5P0/myDSTUkj65UfTJKTypaFhaHpNutjHTB2go5eb5x/y/oZntqlrb763Xi/khZCK55YMeHbf4RXtabv1Y2O7Pvh6v/U3/tl/6BUQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=xsightlabs.com; dmarc=pass action=none header.from=xsightlabs.com; dkim=pass header.d=xsightlabs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xsightlabs.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bvz9p9F6JFMwqkVM9XngQiwYoZEe5f3f32HJYhX7QBg=; b=OW7Wy8ZBdVS1Zi/iUENg9E1HLOZ3Xirfl473Nc4wCbbDNB34p7qddV03M4iXh3FwSqOnmU/MJoh23RqXWnGNTomneqpleb571/lyz3OR0Th6DVhD6CkYG3XcqfyvYHEGlbHDKTR1hLOSffqCUrx82RYg4aQOVhdtwKGnit/1kfk= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=xsightlabs.com; Received: from DB9P193MB1482.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:2a6::7) by DB9P193MB1913.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:25f::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13; Tue, 3 May 2022 19:46:59 +0000 Received: from DB9P193MB1482.EURP193.PROD.OUTLOOK.COM ([fe80::3c11:328c:a5e5:7253]) by DB9P193MB1482.EURP193.PROD.OUTLOOK.COM ([fe80::3c11:328c:a5e5:7253%5]) with mapi id 15.20.5206.024; Tue, 3 May 2022 19:46:59 +0000 Message-ID: <8f603ca2-e4d9-696e-db2d-6d90d2ccc50f@xsightlabs.com> Date: Tue, 3 May 2022 15:46:53 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: Re: [PATCH] eal: allow worker lcore stacks to be allocated from hugepage memory To: "Wang, Haiyue" , "dev@dpdk.org" Cc: "stephen@networkplumber.org" , "mb@smartsharesystems.com" , "Burakov, Anatoly" , "dmitry.kozliuk@gmail.com" , "Richardson, Bruce" , "Honnappa.Nagarahalli@arm.com" , "nd@arm.com" References: <20220502141058.12707-1-donw@xsightlabs.com> From: Don Wallwork In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: VI1PR09CA0095.eurprd09.prod.outlook.com (2603:10a6:803:78::18) To DB9P193MB1482.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:2a6::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 22c446b5-5078-4ec8-8736-08da2d3dafc8 X-MS-TrafficTypeDiagnostic: DB9P193MB1913:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OMHcmzZ73dJdWL4JNX+Bsfwps9JrY2QPjjNznIXfn434a5BL4zOz4Dnj3lITO1fJhCC6goendodeN97J+9V1pYp1PY8jg6jWSFkZ8yvx+OKa+hKy3k2qVSW5vNKixnNaZDcZb9bpwgB7wqOTBqCNpvkyA26+Brlqg2dk96Rs3I365ovm0Fm+hQhGL/P1SxaSyUZjW2qQ91hBF0G7/TdPJ8zx7jlVgbUE21zLWCjcUzDUiEBxD/ZAXsy3q0MinF5PJfvRdplvVCYsl4TvJPqk92rd5n5+F7J5xTbshHXGhphRWOVrZxFfZE2evLpVQhrBIwbYxs9tn2HmfByxCWap07bBKmLBkBaDBySvQicoOgjgVil3rKuUUJkC+Z/ppSq6Dkv9X75zPtHeRI6pmmPn/mKLqkxYltZfKdjcerp3TxoMRqm8PVxrTHxmAr7d6RjLuALzkI0VlbZhKr2ZengAkL5apJU5uzehULvvU0eONsSIZd6tbuEFaLzdTrlZVJfa6bbQI68HjtWx1LNAtHIIsLEhQ8bFsj/mzjBg5IQD45tOYUiGlNKIX28gI7DClXwhVfIZoSxl8Nt4yTujFYEtpQVXtST21zspoZJEtXDLgcALjaPtl0gegKPLrqZZTpsVfipM/iNVl39/nYeOcwqQA3m6Xjx7200paPJ75t7qGsQj4U3v/PCB0UOQJslr7uA0rbx+Wt7iN6ne34Rr5nduR+GEmAC9+gFfkrPKoX4jaSMqMBjSiLtdvIhRNCGvXmoC48CoycgVfAQguEn5/yiWPA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB9P193MB1482.EURP193.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(6486002)(6506007)(53546011)(2906002)(52116002)(31696002)(86362001)(26005)(6666004)(66556008)(66476007)(66946007)(6512007)(8676002)(4326008)(54906003)(110136005)(186003)(38350700002)(38100700002)(5660300002)(8936002)(316002)(83380400001)(508600001)(31686004)(36756003)(2616005)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?R3dHREltY2ZoZXRpNExzb0d2UjVydHRRRnppdXFOeUJNK3BYcFFLYm5QYXgx?= =?utf-8?B?YjJDY3NCbG45cXFFMllpRTJDWmxOUTVHVVh2WTNweHBBdUNlUzVNOTlGRC8x?= =?utf-8?B?UVRNcjRad3pieGFuTHE1aUlWb1grTWpzVDlvSlZaK3FKclhiWE5kQnlXMGxH?= =?utf-8?B?VWVSV0NwV2hhamJYMVBpdXZmeVo0M1Z3dW5MdG81Wm85YkxpRzlaRjZtaHlM?= =?utf-8?B?U1Z6a2pNTy9tNWNnTTUzUEhSOHpXWlExaXZPZFFNWktGQ29BU0dRWUtwVUgv?= =?utf-8?B?TjZGZGhJeFZpS0FlYUZwN1prUGhaM3lrd0RzUGoybzYwN3VFQXppUXlnWWFT?= =?utf-8?B?ZkVLMkF1WHVURG53RXFwcE1peFgraW1VNUFUQ2J1ZkxhdS9VZDdrZUNDaE1T?= =?utf-8?B?OS9vSXBhSXJpRkpEL0hJTEpkNXBMKzF3NXByRUxEMG5NSXRkeHhnZTJ3SWpT?= =?utf-8?B?ZlhMYlltQTBLeUozL2V1ZVg2Z1E4aGhvVmRhcDF2R0NLWllqdlBjTU9nWU9Q?= =?utf-8?B?ZkVuQ2hXSzEwbkx6ZlQ2TUdBaFR1YXZKcmZlcXpISUwzR0VOSHhFcXpKRUtz?= =?utf-8?B?WmhHbTVYSE5oMEs2ZlR3d2VHZGZvbU9ob29jWUlWOFZnTjBLWnc3V1BobmFH?= =?utf-8?B?VmJzUEduMVM1UjZZaTQ3Zjd4VUF4U3RjalBJeUp1STloU2Mza202amdsMzFo?= =?utf-8?B?ZzRuSkRTQ3lrb1ZxelBjSVNTMHlnUEZGcUlhckVIb0hnTSs4emJNaDVpT1RC?= =?utf-8?B?cE1ZbXN0WWVPamg2VFNwWStPR2xDdTNGQ0R5K21QZExtZk5PZVVENGFUcDVP?= =?utf-8?B?WUhnejlGd3g5OFhCS0VGU3plRldMY3dwblRpamdjM2hQbFE3UnYrY0VhYktn?= =?utf-8?B?ZWsxVFROYlBEWFJadGZGUVFsdVhCcVVSL1FmRThKdXJpQytuaFM2bE5OaC84?= =?utf-8?B?cVJVQWttaDZkRDlBOVZyU25JdkpHaGJGM3hZU01tbTZRMWI0dGNSWTliOU9R?= =?utf-8?B?YXhwWFRmcFhyYitFQ01SN2JnY0YrL2NNendvMjVkbnJDSlJ1QUNMcTB4bG5L?= =?utf-8?B?UW5UbjEvWkEvUjlZZEZOc1FsTHl5V2xyT1lsTmpSc1IwSHNRRm13VUd0YmN6?= =?utf-8?B?RzNoR3psck1WYXN1bzZISjhRbEhuTUNITXFvUVg3NVhhOFBQZkNiWk4zKzBh?= =?utf-8?B?bm9xZ3U3Q1ZWOTVoSE1yWE8rRHpkN1pUVXVpUDhQc2VuRTl0eEtRYnM1anZh?= =?utf-8?B?VmhjUjlja0IyaUtDOGNtZm1MSFBkMnVlbHVGZlY4VHhIM01WUHBJckgvL2RB?= =?utf-8?B?cURUVUhEdEhkb3FJSkZNK2dXQlk5UXEvZmQyYm1jamNkeFdsZ1lyTW5sQUVE?= =?utf-8?B?c09vRktsRGNGbXFHRjhsVXdDZjhacFMzbGVWOTNqQ2Rqdy80dWd4TDUzRkJ1?= =?utf-8?B?SHJyWUp6enl2YmVIdFdsbHlsdWF4ZXpuMkcrSWN0MnlkSko3VXY5NnFCNHNx?= =?utf-8?B?MlIyR0p5TWxYVnZSdG5DOG4zR05iN1ZZZ2pNZkM2aDhIdHByQVdITzlDWktI?= =?utf-8?B?UWdtaEw3c3F3cjMwajlleHVWYmx2a1ZhMlA5UlNVUHQ3b1R3bFdBWmVtZzN4?= =?utf-8?B?eE5xRVVkNUtzM3BicVhxQ2VkQ0xhSzIwd2IwcW1LZSs2LzNVb0k3Z2l3TFZL?= =?utf-8?B?Q3NNMUxuVmIvQ3JXWXpWcnZwamsvVGtEY0tPNkJIYk9DelhUclg2ajE0SEFy?= =?utf-8?B?Q1NFOERtQzNlaFZmYm9jNVZneU5HR0JWVDBSdXNjdmdVdHNxaURDRWNwbEVW?= =?utf-8?B?b3h5Wm5zTjlyNnpoZ1JvUTlIV1dxZnpYM3ByU1VYM01QaDVpQjNFWTRhMDFk?= =?utf-8?B?Zk5YRHozdDBOR2Frc3ZKQ2Z6ZHlQWXJPUXRLR1NvV1Qra3lLNmRyU3ZiZlU0?= =?utf-8?B?ckVzenordDBxSXVCR2NCWXJSaDl1bThQUllVY0tGTG55RExVbEo3RzdBbVlK?= =?utf-8?B?UDVRMENjWEFFOEk0eW1vUWIrK0dNcVM4ay9KaXpSUVNlN3pHcUNKWTI3TlRX?= =?utf-8?B?WUJkMVBzVmZOczJxTDU4MllCTmxMME1rOGM0bVBSL3FUbVdUZ21NbTZldzB6?= =?utf-8?B?VTNXZkR0MkdJb3hnVW9PNUNGbjFXam5oUGh4KzgvRWRMdEJISDFRdFoxdVRo?= =?utf-8?B?MXBpaDVlaTF6UHl4L3RhK3V2ZHV1Z3FHNERLN3Q5aWdzc1U5SW9YN2pPUUdQ?= =?utf-8?B?Z2VLb1FwcUJtRUh0d08vTGZCZlA5MzZGY2hKR2duTnZ6MGxBbHNybW1admMw?= =?utf-8?B?U3VvWkVqVzFsbVBndlZheEx5WWR2OHhOalI0YUhzbEJKM0twaFhWUT09?= X-OriginatorOrg: xsightlabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: 22c446b5-5078-4ec8-8736-08da2d3dafc8 X-MS-Exchange-CrossTenant-AuthSource: DB9P193MB1482.EURP193.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 May 2022 19:46:59.3631 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 646a3e34-83ea-4273-9177-ab01923abaa9 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ns/mU6dM0nex394M/nkf8i4Ri830nv5MdSm4L11fe+FzehUX1qeYYLNR6eLx3bC17J/GFs9pxCweQYOutxaLKw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9P193MB1913 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 5/3/2022 9:08 AM, Wang, Haiyue wrote: >> -----Original Message----- >> From: Don Wallwork >> Sent: Monday, May 2, 2022 22:11 >> To: dev@dpdk.org >> Cc: donw@xsightlabs.com; stephen@networkplumber.org; mb@smartsharesystems.com; Burakov, Anatoly >> ; dmitry.kozliuk@gmail.com; Richardson, Bruce ; >> Honnappa.Nagarahalli@arm.com; nd@arm.com >> Subject: [PATCH] eal: allow worker lcore stacks to be allocated from hugepage memory >> >> Add support for using hugepages for worker lcore stack memory. The >> intent is to improve performance by reducing stack memory related TLB >> misses and also by using memory local to the NUMA node of each lcore. >> >> EAL option '--huge-worker-stack [stack-size-kbytes]' is added to allow >> the feature to be enabled at runtime. If the size is not specified, >> the system pthread stack size will be used. >> >> Signed-off-by: Don Wallwork >> --- >> lib/eal/common/eal_common_options.c | 31 ++++++++++++++ >> lib/eal/common/eal_internal_cfg.h | 4 ++ >> lib/eal/common/eal_options.h | 2 + >> lib/eal/linux/eal.c | 65 ++++++++++++++++++++++++++++- >> 4 files changed, 100 insertions(+), 2 deletions(-) >> >> diff --git a/lib/eal/common/eal_common_options.c b/lib/eal/common/eal_common_options.c >> index f247a42455..be9db9ee37 100644 >> --- a/lib/eal/common/eal_common_options.c >> +++ b/lib/eal/common/eal_common_options.c >> @@ -103,6 +103,7 @@ eal_long_options[] = { >> {OPT_TELEMETRY, 0, NULL, OPT_TELEMETRY_NUM }, >> {OPT_NO_TELEMETRY, 0, NULL, OPT_NO_TELEMETRY_NUM }, >> {OPT_FORCE_MAX_SIMD_BITWIDTH, 1, NULL, OPT_FORCE_MAX_SIMD_BITWIDTH_NUM}, >> + {OPT_HUGE_WORKER_STACK, 2, NULL, OPT_HUGE_WORKER_STACK_NUM }, >> >> {0, 0, NULL, 0 } >> }; >> @@ -1618,6 +1619,22 @@ eal_parse_huge_unlink(const char *arg, struct hugepage_file_discipline *out) >> return -1; >> } >> >> +static int >> +eal_parse_huge_worker_stack(const char *arg, size_t *huge_worker_stack_size) >> +{ >> + size_t worker_stack_size; >> + if (arg == NULL) { >> + *huge_worker_stack_size = USE_OS_STACK_SIZE; >> + return 0; >> + } >> + worker_stack_size = atoi(arg); >> + if (worker_stack_size == 0) >> + return -1; > Should we also to check "worker_stack_size *1024 < PTHREAD_STACK_MIN" ? This may be too restrictive in certain environments.  For example, memory constrained platforms may require a smaller worker stack size than this limit would allow. >> + >> + *huge_worker_stack_size = worker_stack_size * 1024; >> + return 0; >> +} >> + > >> -- >> 2.17.1