From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24836A050D; Tue, 26 Apr 2022 23:26:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BE4F540E78; Tue, 26 Apr 2022 23:26:02 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80040.outbound.protection.outlook.com [40.107.8.40]) by mails.dpdk.org (Postfix) with ESMTP id 3822A40691 for ; Tue, 26 Apr 2022 23:26:02 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T6lNLl3UntwhmmLWyjXgF7XaMq48J1H45l+7LyJK0XIMzL7d16GADpMZ74ycNLsRtJC2pK8VgO2+kN11llHAty4OBOvn/l9ewjS6ND/ilHYyn2Acd3tRJ7cz09GMW83oYZVQI1xJT+igCA1Ae8g42Miy5qJKM81Y4Hlu2xdnBtHdkWgj5OUMeJ9brpeopSXIdLGHKddDTLyCtNObtsXJAU2s/Zrxzgu1ynFGXkP/F66XBBjdJ9QHllw3ryeEeljQaKZe+547ckby4g5owCJu95JYrVsqGSimKxWbLJCCPD25Bzq6XPvQ7VdEa3IzZ1OnrdiV1LsyPj6ErBrLZLU+tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=J5w3eG+/Pe2fkYnK1mCsMYdDYTCTOWy7OjKBdqLs8/s=; b=MmfAyIbp7x+PhDYvvRhaMj9RB7YAa6U4Q+KL7EoiXgGgBRqJbFgJ4isNoffB7Opf9Neob5jJ1sBd7reKBT3mq7MUxdz0I4/jA1rmH9aa8Z/Sp+HhnmvjLNClndIPJraZq/JNKnHjqz0F9OHUj9KGpCdA06W+Dr16zNOxoqYK19QCqOXNEhw1sFS15xR1cuRrAyEY02xW2dwHqVeWLwdMMyo9/GDESzh21fpAKeEIiFmopHOMkm5KNi7jnRuZ4wmD0g4dEV1thUbMg2dREIeRZz0Nn4h3Ff4uGAKaJXc7wGh350oouJxGJrMm78eq0q2xQ9IT4pUbfvojBvs1tUlAgA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=xsightlabs.com; dmarc=pass action=none header.from=xsightlabs.com; dkim=pass header.d=xsightlabs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xsightlabs.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J5w3eG+/Pe2fkYnK1mCsMYdDYTCTOWy7OjKBdqLs8/s=; b=DTb459g12wk1it0fGUrGGXUl15VIxpcNdpFrp0taX+8KqE9D2VxA6vrFe5KfI6TkcL+r89/PlMPQuahIzTY7u6I6kepMxssc82K4NeFPF09EEg5LE0GalgNEkfAnbZzxVh2EThmaEmzTwd6TtHmnX1Ix+0mRmuo5Q2LpPFqb/HI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=xsightlabs.com; Received: from DB9P193MB1482.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:2a6::7) by AM0P193MB0483.EURP193.PROD.OUTLOOK.COM (2603:10a6:20b:165::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Tue, 26 Apr 2022 21:26:00 +0000 Received: from DB9P193MB1482.EURP193.PROD.OUTLOOK.COM ([fe80::281a:860b:67a1:2cd9]) by DB9P193MB1482.EURP193.PROD.OUTLOOK.COM ([fe80::281a:860b:67a1:2cd9%9]) with mapi id 15.20.5186.021; Tue, 26 Apr 2022 21:26:00 +0000 Message-ID: Date: Tue, 26 Apr 2022 17:25:56 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: Re: [RFC] eal: allow worker lcore stacks to be allocated from hugepage memory To: Stephen Hemminger Cc: dev@dpdk.org References: <20220426122000.24743-1-donw@xsightlabs.com> <20220426075858.2c28f427@hermes.local> <53a03de6-fb78-986e-64f6-890b08321343@xsightlabs.com> <20220426142124.524069c5@hermes.local> From: Don Wallwork In-Reply-To: <20220426142124.524069c5@hermes.local> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: VI1P190CA0026.EURP190.PROD.OUTLOOK.COM (2603:10a6:802:2b::39) To DB9P193MB1482.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:2a6::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9b724b87-5d63-4de5-6ee7-08da27cb5c56 X-MS-TrafficTypeDiagnostic: AM0P193MB0483:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WMfMCaNVp6/9HCj5+o78O1a/VfHgSIzfmJxFSoM0aQxQzIYvuHsUN7SNSQyNIxTyVcVl//5nf6cOdgkErsumoyNbBQbOwy/FEzKwioeMl3JrsEqcjpqJ+X6nEquwOxBIGBHtzlu7ScXKrQVmTSoo94qBuu+5sbqj9GSYzo/teLvJ7cq5hQKrkJaqzujyOzi5eo4wedfrBdiGy6s4wLl1exT4VxPI+70y3ngKYiX8jS3BC9Qw+9VnfPFd459YrcWUkGuqhdRBEyyVS/YYt3Az8BS2lO5YvmYBk6/laNNjh2YPyEYSCnqHQwWWIzOmnRApwerAdgJ1DH28dEBVuxsO/18roXoUDyJqSPGhB/nIVOY6zkCUq9eOkg0WsM9P2oKFcOQscjD+MDgL7SbhvNNxEqxKVVv7rQdRJVm+aXvuNRfI+s8CPFSv3azrzGdQw1cIvRLfoBkNTosiO2sgJIT1BtEi2MhpAkp/a3siZssB0n/SSAW71HezUAVJO9kY2yR13ksHbtOUHMYG8VuTP4WQNuVnD4B/dv1ArE3SO86DBgQbZK+d2fDBXfKLfSbmfPCdxxq03nhwVwAtQO3QHOaWY8DO7lcOEFMkPSs69Tgl7T6xFWvUTzV1JYkszrC+xycW9p5gYJLNXvaNKTEGbJGeZleLwePOLq0jwxmw5iPCEcSNDwiRUBL6YkCaGISMziRRYJdlN2KLEeqWKMXFrLMPmgYuW893wJDPpYXpczy+0nzijhJcs9okpQfIKxw5OK9hMKSVISt0dAOdjqwlJaytgA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB9P193MB1482.EURP193.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(38350700002)(508600001)(31696002)(6486002)(38100700002)(2906002)(66476007)(66556008)(8676002)(66946007)(6916009)(4326008)(86362001)(8936002)(5660300002)(26005)(83380400001)(6512007)(31686004)(6506007)(2616005)(53546011)(186003)(36756003)(52116002)(6666004)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WFhkL2NyNlhLTENPb2hQaWMrOWN6cTY3bW1uZmsvMWVjamJqZkE4NmNKNnpW?= =?utf-8?B?Y1F4YUEwR25ySnpFZk93OCtqNDdveEdnQ0l6Qlc3RTNyYnY0b3gzQVZKRGty?= =?utf-8?B?dUd4a3puUVFkcjZjeERMV3lITC9EcWJkbUd5d1FQZE52NUorNmRZaW5iNTdQ?= =?utf-8?B?VWNqbGZHU0pPYzZlUlZYK0xTOVpMT01WelBRR3ZtMGVhWVRkOTZjbnhoOHJ6?= =?utf-8?B?VXNsNExsZzZMS29PVGpOODlCaXNpVy84RTNTdVpqUHh2ZTJKQ3V0WFZoQzdi?= =?utf-8?B?QVplaW9iTHhDM1ZqQWw3Y1NQV2crYjZsRzFVYkl5VHBaNlRsTk94cDZOdm1P?= =?utf-8?B?b3cwdGMxOXE4NDRROUY5V3cxREp2K25XRHNBc1ZkS01JVFVIb2IvTktDVEU3?= =?utf-8?B?MExYUGh1QndITlo3VFBRemV1WENzODZnZGlITTBqSWZDVStRSHBaN0hoaTg3?= =?utf-8?B?M09aaHBvMXVxV1lQQ0tIcXNWL2VyNkxqWEd0Z04wRlgzeTJGajFSS2xsVkdy?= =?utf-8?B?T3NDelRrOHRFa2lzejhKRVFhZ0k4QzBaMEZpdng1UDRHdUNEWXNSZWd6Rk9a?= =?utf-8?B?Q2gzWVpNZ0J0ZGVKY3p6aTdBaXdyYXRJSm4yU1hTMEZIcE9RV1BEcEFZc2Q0?= =?utf-8?B?aWRYRmhNM3pHMDN2dzQ2MTlVNjdiQ1d1L0xKMTlGcXY4OVhWYjJyaURmTktT?= =?utf-8?B?QVI4SVVvZVdYSHFZSUdKQVZpVkxpY1hVSHBRUTlFTzhpVElvNGVndmhDMnpT?= =?utf-8?B?UjN4OVpUcFZOMlRWaTVDNWFnN3c3a2NBVnQxOU5EaHB3MlpxQXpiS3BHSVZS?= =?utf-8?B?ZmlidVdkckY3SldLRjdHRkwrWTJmZDZENElRb2FPdGh6M1ZKc1M5bXIyR0RU?= =?utf-8?B?ZFYzeFI2T213azZYMlh3UnpuNFlVb1dsS2lNb2lKMU9zZmpxM3VGUFk3QnZj?= =?utf-8?B?MzEwZERmTUNYNzZUaHFuT3BGdWE2eG9LTlFmU3RDNmgyN29Kc2Jqd3VOTzBL?= =?utf-8?B?NVo5RHI0K0VhUkNWUW1JcUdoN24rK093ZkhidDdxZ0ZwZ1B0Q3hVVVlJNDhL?= =?utf-8?B?MHpYTmhWMzVpQ0VKRFZyc3pZOFRzeExHUkp2SjMweFJ6Smd0eThobnFOek1J?= =?utf-8?B?SW12cUhySEdJUmxKMmYzTGpDaHVHQy9CZ1NRakxjT3NsQ3AyVTJmWDlkN0NQ?= =?utf-8?B?a3BXSzBVSFRSZDBSb25ocXpCeDIzM1ExRFV4T0MzZllOOEZJLzNZbG5yRFZD?= =?utf-8?B?ZVpreWF5VDBPNGM1ajRPaTRWTzl4UG1scnJhbmgzN0Y4WW1zSGd0eW1QeEpi?= =?utf-8?B?aTdKTzRsWFVXeGVLN0MyL0VrNU9yb1BtSlZrcjFZUVYxSUo3SkdZZEs5UjBT?= =?utf-8?B?YS85dk9ZQ0wveCtsWWtPdThMQ1NvS0lYeFAxdEdmWTB4aTBreFZtZGVmRGZD?= =?utf-8?B?VXFEdDdoaFJWNlV3aGh6WHI1eGkrYWZrdW03QTBFeHh2dkNOWHJmMkk3eWVs?= =?utf-8?B?TC81Vy9DSmd2ckszUGlDaDc4d2Z3bHl3Y0ZRZVhHd0s3RW9pTXZMelNVbDZD?= =?utf-8?B?MHY1NzhmRjBsK0VKOFBaaDRIZEpicCtBajBvWmNFWkM4OVUvekZRa1oxSnp6?= =?utf-8?B?WC8wNk5WVC94MW51Z2hkV3RJQUcvTmROTW8xVCtPeVlSRW9aY3JRZmcxV3N4?= =?utf-8?B?QytpT0J5cjcvN2FiajF4NmJKRm9IemhQdXNpdm1HVzB4MGh5cTJoa3FkblFW?= =?utf-8?B?ZU1aOStsSXFoN1BUVjI3NE55TUlHYUs3VExQRzMzYVNiVDVCOUtLaGd5c21K?= =?utf-8?B?ZUp3b0dNc1o3T1FKRmZhMjlsR2hDWmdXd0JiMU15STk4akd5OUM1KzA0eC8r?= =?utf-8?B?MjF0aVRXT0NDZFpoMllxRWRsTm04QWloQStXQldiYmh3Y01vdlZlYzB0V1lz?= =?utf-8?B?TEJxenNSVlB5MUd0RTVwRVZQRjl2blJMUXI2TVNreVpRQ3B2T2pCQlBTN29Q?= =?utf-8?B?M3I0QWRMWjQ1U0lwcFZ0Ykk0eis1bW5ZR2FCMEZHMHdQeldwS2o5MEFPM0Zr?= =?utf-8?B?S1R3L25qOFhNTUJpa0wzZkFiVENBUW5LQWFwZHFDSk9pUndmZzBJMkxUT0RV?= =?utf-8?B?SmpmZi83U0ltZGxpRitYZlUrSjNQR05FNXFlZTdrRE4xQVhzK1B0dHpYVXhC?= =?utf-8?B?QTNxTk53Mk5GckpSYmNCdWxUaE8zbWxDaTFSV3FJclpPNG5FSVMzREdDSVFW?= =?utf-8?B?Qy9oTEY2S3d2Z1JuL0pPTWJYWW5FU0RZdUFFcUw5ODY1eVcyb05UMXRQOHVT?= =?utf-8?B?ZGRSTkxkZnAweFhKWkxxWFU3eGV1N291U1o0b0p2K0QxbnJMVUh1Zz09?= X-OriginatorOrg: xsightlabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b724b87-5d63-4de5-6ee7-08da27cb5c56 X-MS-Exchange-CrossTenant-AuthSource: DB9P193MB1482.EURP193.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Apr 2022 21:26:00.8608 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 646a3e34-83ea-4273-9177-ab01923abaa9 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1xB/5hT9KcOlUNl7hlFl5MvLc9BcFTcAHxto4eRPCncPZ6TlnPt6J9FF4pzQciepiJqIx2ctC3MhQQemQmNnRQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0P193MB0483 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 4/26/2022 5:21 PM, Stephen Hemminger wrote: > On Tue, 26 Apr 2022 17:01:18 -0400 > Don Wallwork wrote: > >> On 4/26/2022 10:58 AM, Stephen Hemminger wrote: >>> On Tue, 26 Apr 2022 08:19:59 -0400 >>> Don Wallwork wrote: >>> >>>> Add support for using hugepages for worker lcore stack memory. The >>>> intent is to improve performance by reducing stack memory related TLB >>>> misses and also by using memory local to the NUMA node of each lcore. >>>> >>>> Platforms desiring to make use of this capability must enable the >>>> associated option flag and stack size settings in platform config >>>> files. >>>> --- >>>> lib/eal/linux/eal.c | 39 +++++++++++++++++++++++++++++++++++++++ >>>> 1 file changed, 39 insertions(+) >>>> >>> Good idea but having a fixed size stack makes writing complex application >>> more difficult. Plus you lose the safety of guard pages. >> Thanks for the quick reply. >> >> The expectation is that use of this optional feature would be limited to >> cases where >> the performance gains justify the implications of these tradeoffs. For >> example, a specific >> data plane application may be okay with limited stack size and could be >> tested to ensure >> stack usage remains within limits. >> >> Also, since this applies only to worker threads, the main thread would >> not be impacted >> by this change. >> >> > I would prefer it as a runtime, not compile time option. > That way distributions could ship DPDK and application could opt in if it wanted. Good point..  I'll work on a v2 and will post that when it's ready.