From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6E41A0093; Thu, 23 Jun 2022 13:21:52 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8DFB840042; Thu, 23 Jun 2022 13:21:52 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20041.outbound.protection.outlook.com [40.107.2.41]) by mails.dpdk.org (Postfix) with ESMTP id 6D2064003F for ; Thu, 23 Jun 2022 13:21:51 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z8J3EaIXr6mBkjb+xniHDa3O86N5z2nyPAxFldLjTB2Q71HCqdsk9Sr9MuVBQnIj22Npv/0vOaseB9cerFecnI+kaUocy6m4fSQDifA8vrxxOA0QcmjUUGjctjE0bAtOwB4rPF+D24St/IVUSRjwtCwRESvKyKUECVJhoetaYVkUDtd6B2HioDkA7hSqE9j0AvH0KpzWwKH3c1OnctCgOLO6ikJm7wt0OvZub2RY2Pb4JtKK1pB8EJlVTWGjC+Cuk9M0jMtQ1BQTnJmdWOARbvWioRU/X/YJ5y9+PA0EckGVhsfCOLkjBNuQ4M5JqNQLf2yEu+YZ4OwEvkKxZlD1/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TAAnU5Nv6mvINdXwKs3DjkCVPwoghSziUmsEkQJU3FQ=; b=S3WYJ+Wx/Z7qNIQv6ZYr8xFIezGwnC9+UbswaRiXSIvKnE2aV0na1MUhahhb8mUVORljMZayJgUPy6MghzK3YRf4wOXS0w1aef2f3W+A27O2ewmwDjQZ7Y4kTkZ3ItBCdNT+gFr85/T4+o0iE+h8HDKNOeuoV986XJySsbz5+phljK2R6gMjckMd4LHq5I8KsMFdEQVj1LMUEoW9EDTiApfXpaMkiDPgViplpGwGk8oeqqlwG5s/w+QUipU1rlGdwjOz1C8sB/2ZBmSDsXdu94M0wsXnCAh/i6gax2M37yXTZ+4zX7o5tUhKEiWxVY4VrwDbKyhvwp7SzU+eeJF2Zw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=xsightlabs.com; dmarc=pass action=none header.from=xsightlabs.com; dkim=pass header.d=xsightlabs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xsightlabs.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TAAnU5Nv6mvINdXwKs3DjkCVPwoghSziUmsEkQJU3FQ=; b=wJm/VGd0K12uSciCoH5slIsGhon9TmvzFXc0CbdzMS9z6O2Qak6bq8kWFHWZUvYCMX/vogTTRsjUH+RZz2jp8hP8S3CC/2B88vbnk+DwgWteBFQNxcBz4dbFhgh3JzvQN03rHdQaeAWQ2feozDJX1Q2T7mYhSQyQyzhpOe1P7Uo= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=xsightlabs.com; Received: from DB9P193MB1482.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:2a6::7) by GV1P193MB2134.EURP193.PROD.OUTLOOK.COM (2603:10a6:150:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.15; Thu, 23 Jun 2022 11:21:49 +0000 Received: from DB9P193MB1482.EURP193.PROD.OUTLOOK.COM ([fe80::61af:58cb:6533:1aaf]) by DB9P193MB1482.EURP193.PROD.OUTLOOK.COM ([fe80::61af:58cb:6533:1aaf%9]) with mapi id 15.20.5353.022; Thu, 23 Jun 2022 11:21:49 +0000 From: Don Wallwork To: dev@dpdk.org Cc: donw@xsightlabs.com, david.marchand@redhat.com, thomas@monjalon.net, stephen@networkplumber.org, fengchengwen@huawei.com, mb@smartsharesystems.com, anatoly.burakov@intel.com, dmitry.kozliuk@gmail.com, bruce.richardson@intel.com, Honnappa.Nagarahalli@arm.com, nd@arm.com, haiyue.wang@intel.com, Kathleen.Capella@arm.com Subject: [PATCH v7] eal: allow worker lcore stacks to be allocated from hugepage memory Date: Thu, 23 Jun 2022 07:21:27 -0400 Message-Id: <20220623112127.43088-1-donw@xsightlabs.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220502141058.12707-1-donw@xsightlabs.com> References: <20220502141058.12707-1-donw@xsightlabs.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: BLAPR03CA0133.namprd03.prod.outlook.com (2603:10b6:208:32e::18) To DB9P193MB1482.EURP193.PROD.OUTLOOK.COM (2603:10a6:10:2a6::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2ae527b6-aadd-4b67-c0aa-08da550a9058 X-MS-TrafficTypeDiagnostic: GV1P193MB2134:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VoJ6mu+i8W1MG/XF4lbZxMknsxyxSxjqWjUfX9XylhL4kspL4Egf4rTg5DOf0GSlkX0ObpPnVUwVRkLEGRJGLuQlZtSh/SBM5687tjHFKHOcMBon/PTPMkfvuiQJVQ3yRyo1h9LcikpODXnu7SIUtjZcBIHOdmmJ/RckHsk5poqJ7FoFAC4CR0e3njyhP6t/EbNC11zVfJAKrKdU870SjjZFHwa0hU54ZWkhJdYy8RLIRIkGGELFItm9IgfNHshXW4uoHLwTF37bjPJXRCAv2YpKi6yCXiQHXuyNZfqDxcDaLfJeku7VYcYucIRERsgTd3nue2lajVZZbFdNR5c3Sp3j5Ny6IhX7zEXJk2ArBtCS/CyNvBKo4c4NFAIwZgkxQTqiF5ROELFq2qmADrK650TAWL3WKznObgSNvzvGDgynxuOZ+oMcn8UMzmRLVx8Q5n47ZiUHCWIRJt1LAO92JO9apOJ/ERwmxqG6gWtYOwx+78EM/y8o47zM0GkulDEnD9tnNB3kGWplW4BlCuXpc5YaGQbwfGH2Y7DrPDLs7rpbUZlkGm8n8/CrO1tKlxp41mgwXum4W+Dz5F5QX4xmdjR5R/gSACi3lBHjjKKP+JEnVu7nUJB8aeJRx6QYdQtBCL7z/+A0pfw35R1y4aWHALCRWZcu45HrHPvu4IvglQWhjNgJG7BzFacBlnkOs/N6tPw28q7wKIJsW48TjvJxLgFoMLssOkYEUjFdZ8/AmeS26NLV8lYAUm3C/Ej1W7Ka X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB9P193MB1482.EURP193.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230016)(4636009)(136003)(39850400004)(366004)(376002)(396003)(346002)(6506007)(26005)(6666004)(6916009)(478600001)(36756003)(41300700001)(8676002)(6486002)(6512007)(30864003)(316002)(52116002)(66574015)(83380400001)(5660300002)(4326008)(8936002)(86362001)(1076003)(2906002)(2616005)(7416002)(66556008)(186003)(66946007)(66476007)(38350700002)(38100700002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RmFaWDhuOUR3MXdIMFJJU0NyNkQvN1NjMVM0YkZOYnZ4UVl6c081bUhGTEhS?= =?utf-8?B?VFRnTEdXNWZ5eHRndGt3eXlRQ1poRjRoejIxZTNWWTF5VnIrRCtCTjlhVnNB?= =?utf-8?B?czE1cTdIUVBycHI5bDVlNWhxVjBqbEp5Y1BER293akUyQUlGYTUyNjNqM0ow?= =?utf-8?B?eVhicEMrWHdDZG5wWkZNRXgrSkE0NHVOZm5QSW1nUVZsK3hoNFNJbUdseE1k?= =?utf-8?B?TWJ2ZW95Z3lwRDFnOGQwSFE5cDJtME53cUJsQVVNalZGaDg1M1MrUmdwVlM2?= =?utf-8?B?dy9wSzB6UkZhcDRjMEpoQXorL0xXaFhlZ0lyYnRiWEM2K0xNSlN5MzRXOThh?= =?utf-8?B?ZC9DTUkvK2dhcEZQNjNQZ0M5YmUvK3NWNXAvVnkxbTFOTGJTbTFLYlB4UlFS?= =?utf-8?B?NVp6dDlCU3ZNd1FRMmhmejM3R0s3V3FydWZvOTM2QmN3Q1RWSU05TW5tbDFi?= =?utf-8?B?elhDOHlwMTJaTzMyOHlXQlZ0OEhBWFlRVXdQdkt1SloxOUg1K0lIeGVTbTNF?= =?utf-8?B?dktHQWZUR3poNjNRbWNzaHBqNDRPNXdxeER4eDA3aEdYbVUvWWovb2o3U0Q5?= =?utf-8?B?ZER1SGMzZnZMbkJMNFgyamdtRDdzMEJPclQ1UWNoWWJCRE1jTXVrSEpMNmUv?= =?utf-8?B?Uk9WUTRvN0thVG03bVJsSTY5Qlc0TUt6UWhJeWVFR3NDY3FMemxNRzJEQUlO?= =?utf-8?B?Q2U4VlhrUkVUK3JPVDVkalZCQWVJdEpUcGNFS3pLaythTHVKcm1Ea2FHNS9t?= =?utf-8?B?YWtpd0RRU1JVOSttWHp1QU9ZTm4wN1hzdXg2djJSd1l0enA4RHQ2cERyVFNI?= =?utf-8?B?TWJSS0trMXFLRlZxUzk1L3NkOER4YktDVytwaWZMNFdtMG1zaDdzbVU3Smdq?= =?utf-8?B?YktBa3VLeVNUOTdFL0xkek9RVUkzaFBWOTVyM0tya1lBQS81akhkb05qc05q?= =?utf-8?B?V1BNb2I2b3Z4VkhQUUY5SXRabXZNWnNnZ0JLdVB3aXhTbjQ4YXREWGFad1RL?= =?utf-8?B?MG4zNGZraFpQNDdlbmFjeExwa0FZRy9NQWE4NHJzb3lMM0kreDNEYVh5czhD?= =?utf-8?B?eGVTeFBHTWZlb3FFbzhxWmhUekhYa1BCYkoxZnpxWUhpOWg3Tmp6eU9xV25R?= =?utf-8?B?bnA2NllmNHFsNUNOZFgwVHJhbEM0TkNyZ2pNeTl1YitGV1djU1kwamEzNVk4?= =?utf-8?B?eVRSelROZHdlVTl1SVZFOEVlNUdVVEUrcmNWbktzWWhZdU1BSDZjL0pYVlQr?= =?utf-8?B?Lzh3MWpxWjBzOTVIK1NEeHF2TlVXckRjdjNGR2RNRllhTnArbXdmenBCYXpa?= =?utf-8?B?eUdVS1pod3dwOXczQSsrK0p4ZFR2akprdlZqT0NSandocDc1WHIzTEtzTnFW?= =?utf-8?B?RTFIMVI2dS9UMjBiR1I5aHN4VVdKMklnTENEK2xVNXJGaXg0Sjd1RHVoQVRR?= =?utf-8?B?NEhDdVVjOEZBTnJ1a0RESlp2TkJ6dDRXekhsdVFVMXhaN0JPcDlleGd0QUJ1?= =?utf-8?B?ZGtzYmlKcHhGY0pOMmhFaU90bmxCcFJiaTFwUEJaYmtvOW4xZnkra0l6SXZI?= =?utf-8?B?ZlNvL3c4TjJ4MTNhWnc2RlN0a0prajJZcTEvbDZjTHZaRUJ6dXZCcTJUUzRi?= =?utf-8?B?WS91V2tscisrOUc4T1BkYXQwcFJQaWlVZWRxWWN0dnhjeDVuWXRvTkx6Rkd6?= =?utf-8?B?b1h0Wmx5cnZNZGhiamMzL0RZSDZpU2gyUC9EL0IydjFEK3VFRktDaEFHbEZM?= =?utf-8?B?cENHOUFQRnVZWGhtdEFJbUg3S2h0Y3UyOW1DVTl0K1FrekMzbjFZL0VzamtO?= =?utf-8?B?aHZrZ0tJSlpyOHRia0FPa28vdkV4K3oxVFBnck5qRGh5SWVpaFpQNVJJOTRR?= =?utf-8?B?SU5wWHhLUFEvNk9BSGxZam9HbHdEQURZOTNDeEJvOGtRK0EyRVB0cjNidlkr?= =?utf-8?B?QTRYWVBRYmd0WVFkaURqZmpMd3EyTzRIME82ekFSQ29NU2d1Y1R6SWs0MS9x?= =?utf-8?B?UUE5MTBnOXRGenNOSzlNVzJldjAxWHZBRVdvK1Fka28xZ0ZBMDRibTVUUDcz?= =?utf-8?B?K3pCMUhjMi9tQmp0bEhWV3RRMDBzMTJFSG0wU01hb080c2hYcVN4YUdIdmJH?= =?utf-8?B?NU14L0pub3A4cTFZM1hSOHVUWjNuNnZQOEx6UjU4dGF5NWxUaWNyL281SzlV?= =?utf-8?B?ZjM1ZE85aHU1N0xqQjJSRFQvQlNySWxiRk9GdlNrRmg4bGtOSUxwWlNnOXkx?= =?utf-8?B?VXdJYzlQY3hVeVZ1QlpoTngyc2NuWFhsajNzRjg2WjlpcERmNjcrQkpXSVdL?= =?utf-8?B?MHZPdDFpNENKbnVRcU1mZnNHeDBDekJHUDBRKzJZVFFsU1RQS0hzQT09?= X-OriginatorOrg: xsightlabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2ae527b6-aadd-4b67-c0aa-08da550a9058 X-MS-Exchange-CrossTenant-AuthSource: DB9P193MB1482.EURP193.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jun 2022 11:21:48.8303 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 646a3e34-83ea-4273-9177-ab01923abaa9 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zl9qtc1bMAlrv/eVzvKhbiLJOQBB4B44uclidhbNTpEJH3/wiPRkeDNJ5osfj10fP907hNFN/LB5z2/O52eHPA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1P193MB2134 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support for using hugepages for worker lcore stack memory. The intent is to improve performance by reducing stack memory related TLB misses and also by using memory local to the NUMA node of each lcore. EAL option '--huge-worker-stack [stack-size-in-kbytes]' is added to allow the feature to be enabled at runtime. If the size is not specified, the system pthread stack size will be used. Signed-off-by: Don Wallwork Acked-by: Morten Brørup Acked-by: Chengwen Feng --- app/test/test_eal_flags.c | 36 ++++++ doc/guides/linux_gsg/eal_args.include.rst | 6 + .../prog_guide/env_abstraction_layer.rst | 19 ++++ lib/eal/common/eal_common_options.c | 7 ++ lib/eal/common/eal_internal_cfg.h | 1 + lib/eal/common/eal_options.h | 2 + lib/eal/linux/eal.c | 105 +++++++++++++++++- 7 files changed, 174 insertions(+), 2 deletions(-) diff --git a/app/test/test_eal_flags.c b/app/test/test_eal_flags.c index 68deb86aab..624379d49e 100644 --- a/app/test/test_eal_flags.c +++ b/app/test/test_eal_flags.c @@ -859,6 +859,14 @@ test_no_huge_flag(void) /* With --no-huge, -m and --socket-mem */ const char *argv4[] = {prgname, prefix, no_huge, "-m", DEFAULT_MEM_SIZE, "--socket-mem=" DEFAULT_MEM_SIZE}; + + /* With --no-huge and --huge-worker-stack (should fail) */ + const char * const argv5[] = {prgname, prefix, no_huge, + "--huge-worker-stack"}; + /* With --no-huge and --huge-worker-stack=512 (should fail) */ + const char * const argv6[] = {prgname, prefix, no_huge, + "--huge-worker-stack=512"}; + if (launch_proc(argv1) != 0) { printf("Error - process did not run ok with --no-huge flag\n"); return -1; @@ -882,6 +890,16 @@ test_no_huge_flag(void) "--socket-mem flags\n"); return -1; } + if (launch_proc(argv5) == 0) { + printf("Error - process run ok with --no-huge and " + "--huge-worker-stack flags"); + return -1; + } + if (launch_proc(argv6) == 0) { + printf("Error - process run ok with --no-huge and " + "--huge-worker-stack=size flags"); + return -1; + } return 0; } @@ -1031,6 +1049,14 @@ test_misc_flags(void) const char * const argv20[] = {prgname, "--file-prefix=uiodev", "--create-uio-dev"}; + /* Try running with --huge-worker-stack flag */ + const char * const argv21[] = {prgname, prefix, mp_flag, + "--huge-worker-stack"}; + + /* Try running with --huge-worker-stack=512 flag */ + const char * const argv22[] = {prgname, prefix, mp_flag, + "--huge-worker-stack=512"}; + /* run all tests also applicable to FreeBSD first */ if (launch_proc(argv0) == 0) { @@ -1130,6 +1156,16 @@ test_misc_flags(void) "--create-uio-dev parameter\n"); goto fail; } + if (launch_proc(argv21) != 0) { + printf("Error - process did not run ok with " + "--huge-worker-stack parameter\n"); + goto fail; + } + if (launch_proc(argv22) != 0) { + printf("Error - process did not run ok with " + "--huge-worker-stack=size parameter\n"); + goto fail; + } rmdir(hugepath_dir3); rmdir(hugepath_dir2); diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst index 3549a0cf56..9cfbf7de84 100644 --- a/doc/guides/linux_gsg/eal_args.include.rst +++ b/doc/guides/linux_gsg/eal_args.include.rst @@ -116,6 +116,12 @@ Memory-related options Force IOVA mode to a specific value. +* ``--huge-worker-stack[=size]`` + + Allocate worker stack memory from hugepage memory. Stack size defaults + to system pthread stack size unless the optional size (in kbytes) is + specified. + Debugging options ~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 5f0748fba1..42def41e61 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -329,6 +329,25 @@ Another option is to use bigger page sizes. Since fewer pages are required to cover the same memory area, fewer file descriptors will be stored internally by EAL. +Hugepage Worker Stacks +^^^^^^^^^^^^^^^^^^^^^^ + +When the ``--huge-worker-stack[=size]`` EAL option is specified, worker +thread stacks are allocated from hugepage memory local to the NUMA node +of the thread. Worker stack size defaults to system pthread stack size +if the optional size parameter is not specified. + +.. warning:: + Stacks allocated from hugepage memory are not protected by guard + pages. Worker stacks must be sufficiently sized to prevent stack + overflow when this option is used. + + As with normal thread stacks, hugepage worker thread stack size is + fixed and is not dynamically resized. Therefore, an application that + is free of stack page faults under a given load should be safe with + hugepage worker thread stacks given the same thread stack size and + loading conditions. + Support for Externally Allocated Memory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/lib/eal/common/eal_common_options.c b/lib/eal/common/eal_common_options.c index f247a42455..726ee5a5d5 100644 --- a/lib/eal/common/eal_common_options.c +++ b/lib/eal/common/eal_common_options.c @@ -103,6 +103,7 @@ eal_long_options[] = { {OPT_TELEMETRY, 0, NULL, OPT_TELEMETRY_NUM }, {OPT_NO_TELEMETRY, 0, NULL, OPT_NO_TELEMETRY_NUM }, {OPT_FORCE_MAX_SIMD_BITWIDTH, 1, NULL, OPT_FORCE_MAX_SIMD_BITWIDTH_NUM}, + {OPT_HUGE_WORKER_STACK, 2, NULL, OPT_HUGE_WORKER_STACK_NUM }, {0, 0, NULL, 0 } }; @@ -2079,6 +2080,12 @@ eal_check_common_options(struct internal_config *internal_cfg) "be specified together with --"OPT_NO_HUGE"\n"); return -1; } + if (internal_cfg->no_hugetlbfs && + internal_cfg->huge_worker_stack_size != 0) { + RTE_LOG(ERR, EAL, "Option --"OPT_HUGE_WORKER_STACK" cannot " + "be specified together with --"OPT_NO_HUGE"\n"); + return -1; + } if (internal_conf->force_socket_limits && internal_conf->legacy_mem) { RTE_LOG(ERR, EAL, "Option --"OPT_SOCKET_LIMIT " is only supported in non-legacy memory mode\n"); diff --git a/lib/eal/common/eal_internal_cfg.h b/lib/eal/common/eal_internal_cfg.h index b71faadd18..167ec501fa 100644 --- a/lib/eal/common/eal_internal_cfg.h +++ b/lib/eal/common/eal_internal_cfg.h @@ -102,6 +102,7 @@ struct internal_config { unsigned int no_telemetry; /**< true to disable Telemetry */ struct simd_bitwidth max_simd_bitwidth; /**< max simd bitwidth path to use */ + size_t huge_worker_stack_size; /**< worker thread stack size */ }; void eal_reset_internal_config(struct internal_config *internal_cfg); diff --git a/lib/eal/common/eal_options.h b/lib/eal/common/eal_options.h index 8e4f7202a2..3cc9cb6412 100644 --- a/lib/eal/common/eal_options.h +++ b/lib/eal/common/eal_options.h @@ -87,6 +87,8 @@ enum { OPT_NO_TELEMETRY_NUM, #define OPT_FORCE_MAX_SIMD_BITWIDTH "force-max-simd-bitwidth" OPT_FORCE_MAX_SIMD_BITWIDTH_NUM, +#define OPT_HUGE_WORKER_STACK "huge-worker-stack" + OPT_HUGE_WORKER_STACK_NUM, OPT_LONG_MAX_NUM }; diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index 1ef263434a..40e3400d30 100644 --- a/lib/eal/linux/eal.c +++ b/lib/eal/linux/eal.c @@ -451,6 +451,11 @@ eal_usage(const char *prgname) " --"OPT_LEGACY_MEM" Legacy memory mode (no dynamic allocation, contiguous segments)\n" " --"OPT_SINGLE_FILE_SEGMENTS" Put all hugepage memory in single files\n" " --"OPT_MATCH_ALLOCATIONS" Free hugepages exactly as allocated\n" + " --"OPT_HUGE_WORKER_STACK"[=size]\n" + " Allocate worker thread stacks from\n" + " hugepage memory. Size is in units of\n" + " kbytes and defaults to system thread\n" + " stack size if not specified.\n" "\n"); /* Allow the application to print its usage message too if hook is set */ if (hook) { @@ -579,6 +584,47 @@ eal_log_level_parse(int argc, char **argv) optarg = old_optarg; } +static int +eal_parse_huge_worker_stack(const char *arg) +{ + struct internal_config *cfg = eal_get_internal_configuration(); + + if (arg == NULL || arg[0] == '\0') { + pthread_attr_t attr; + int ret; + + if (pthread_attr_init(&attr) != 0) { + RTE_LOG(ERR, EAL, + "Could not retrieve default stack size\n"); + return -1; + } + ret = pthread_attr_getstacksize(&attr, + &cfg->huge_worker_stack_size); + pthread_attr_destroy(&attr); + if (ret != 0) { + RTE_LOG(ERR, EAL, + "Could not retrieve default stack size\n"); + return -1; + } + } else { + unsigned long stack_size; + char *end; + + errno = 0; + stack_size = strtoul(arg, &end, 10); + if (errno || end == NULL || stack_size == 0 || + stack_size >= (size_t)-1 / 1024) + return -1; + + cfg->huge_worker_stack_size = stack_size * 1024; + } + + RTE_LOG(DEBUG, EAL, + "Each worker thread will use %zu kB of DPDK memory as stack\n", + cfg->huge_worker_stack_size / 1024); + return 0; +} + /* Parse the argument given in the command line of the application */ static int eal_parse_args(int argc, char **argv) @@ -716,6 +762,14 @@ eal_parse_args(int argc, char **argv) internal_conf->match_allocations = 1; break; + case OPT_HUGE_WORKER_STACK_NUM: + if (eal_parse_huge_worker_stack(optarg) < 0) { + RTE_LOG(ERR, EAL, "invalid parameter for --" + OPT_HUGE_WORKER_STACK"\n"); + return -1; + } + break; + default: if (opt < OPT_LONG_MIN_NUM && isprint(opt)) { RTE_LOG(ERR, EAL, "Option %c is not supported " @@ -857,6 +911,54 @@ is_iommu_enabled(void) return n > 2; } +static int +eal_worker_thread_create(unsigned int lcore_id) +{ + pthread_attr_t *attrp = NULL; + void *stack_ptr = NULL; + pthread_attr_t attr; + size_t stack_size; + int ret = -1; + + stack_size = eal_get_internal_configuration()->huge_worker_stack_size; + if (stack_size != 0) { + /* Allocate NUMA aware stack memory and set pthread attributes */ + stack_ptr = rte_zmalloc_socket("lcore_stack", + stack_size, + RTE_CACHE_LINE_SIZE, + rte_lcore_to_socket_id(lcore_id)); + if (stack_ptr == NULL) { + rte_eal_init_alert("Cannot allocate worker lcore stack memory"); + rte_errno = ENOMEM; + goto out; + } + + if (pthread_attr_init(&attr) != 0) { + rte_eal_init_alert("Cannot init pthread attributes"); + rte_errno = EFAULT; + goto out; + } + attrp = &attr; + + if (pthread_attr_setstack(attrp, stack_ptr, stack_size) != 0) { + rte_eal_init_alert("Cannot set pthread stack attributes"); + rte_errno = EFAULT; + goto out; + } + } + + if (pthread_create(&lcore_config[lcore_id].thread_id, attrp, + eal_thread_loop, (void *)(uintptr_t)lcore_id) == 0) + ret = 0; + +out: + if (ret != 0) + rte_free(stack_ptr); + if (attrp != NULL) + pthread_attr_destroy(attrp); + return ret; +} + /* Launch threads, called at application init(). */ int rte_eal_init(int argc, char **argv) @@ -1144,8 +1246,7 @@ rte_eal_init(int argc, char **argv) lcore_config[i].state = WAIT; /* create a thread for each lcore */ - ret = pthread_create(&lcore_config[i].thread_id, NULL, - eal_thread_loop, (void *)(uintptr_t)i); + ret = eal_worker_thread_create(i); if (ret != 0) rte_panic("Cannot create thread\n"); -- 2.17.1