From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B167BA0C46; Tue, 14 Sep 2021 12:35:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 773164068F; Tue, 14 Sep 2021 12:35:22 +0200 (CEST) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2087.outbound.protection.outlook.com [40.107.102.87]) by mails.dpdk.org (Postfix) with ESMTP id 73ED54003C for ; Tue, 14 Sep 2021 12:35:21 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZUv4x8+o8DMWErrBmR+GFZDKB604gYkW3OdaZcwzfM/O0pxLdjo8m1Cx8wN4gzbzmaPzPrc+XGB/LXlG3CY/zpe1wvTY41cr91VXP0dVU+TV5r3cHpX3SetPjNxnnhvj1o7R9DED2AAQU0k2KEN2CbsohU2l9sJ8inaoune/FeBs74Gmbr1aApZefUTRpIRZ9FQeLxIQL4MrKIpOd4JXYADTGfSk+xOWhlZ7skpTGMR2liKL2+fmlMIOTczxyoC3QBzlIePWBX6OoWboSpmr6AugCb2HupApABLPd7mcuGDadVRlXV9HcefRZ0LS8xJ9/0ZRPBIc701uRaRf+EPkXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ZqJ8XtEvFaKxJtKsOhE/SE3ZSTpylLnInBFsGUBAiTo=; b=llqL/fRUWpi+8hzG3xlvmtBU6hQkGZLSqVlx36DD5cziXi9R07DZJkQDTKWux2mYOS7rxYt+cwtfZ9McequrfM4VdsakN8oNol1MUqkPs1K7NOg3km5gFI0uNVeXvytXnu9yErJYWMKbrlBSpOPJM5VSa0/BO4tonjzZ63KwYwE+sf7A3JyWfXsLmcx8CuQ8cAL8I93v3Itme+nYw/X3R/CFwFF0o0Eaad0hELuK/tfwmc7rh+jp5BcftqTCgAtEmigPluX1a9yWp0ks+f0jU0HWiFzxSTgCOZ/JIUzchORmOM6ND9qKr/495JMMjbsD0j0bqgxVfLtpSL8jBl0nLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZqJ8XtEvFaKxJtKsOhE/SE3ZSTpylLnInBFsGUBAiTo=; b=J5LjFtuk+6UkTlMQy8DXtQUfiC7dJAsOmr3AKdUL1mbxGIcKk6CfSnOsCTj182vEOE+OGkqe8a1IouLPl8S8st7d4sCGrk0BCK94FIwoFcmVXO/tJ5IXsw/ZYPRG2qJS/HLAqFeV232TEw5XGOMEPhaM+/y/cI5oaAJmPkvHJ4ztVqP5GeDjKD16d2A3QrPqt+l1+QhxDnevNP6e/H0YVgxU6lBVW86cALmimLxdatN9ZY/pCciH5/Bq3LTOxpeLj62hM4wd2AS7TV6BXDy6k10QDOQnGMIyOH4rpD1EvO8UUTN1ZzPZ3nFzPF2STmTCfeymyN7gq8eG51V1Q0YxqQ== Received: from CO2PR06CA0068.namprd06.prod.outlook.com (2603:10b6:104:3::26) by MN2PR12MB3069.namprd12.prod.outlook.com (2603:10b6:208:c4::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Tue, 14 Sep 2021 10:35:20 +0000 Received: from CO1NAM11FT048.eop-nam11.prod.protection.outlook.com (2603:10b6:104:3:cafe::c) by CO2PR06CA0068.outlook.office365.com (2603:10b6:104:3::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15 via Frontend Transport; Tue, 14 Sep 2021 10:35:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT048.mail.protection.outlook.com (10.13.175.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 10:35:19 +0000 Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 14 Sep 2021 10:35:17 +0000 Received: from nvidia.com (172.20.187.5) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 14 Sep 2021 10:35:15 +0000 From: Dmitry Kozlyuk To: CC: Anatoly Burakov Date: Tue, 14 Sep 2021 13:34:53 +0300 Message-ID: <20210914103456.535427-1-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210716110806.2566788-1-dkozlyuk@nvidia.com> References: <20210716110806.2566788-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To DRHQMAIL107.nvidia.com (10.27.9.16) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ad4673ab-b95c-4c85-779b-08d9776b595a X-MS-TrafficTypeDiagnostic: MN2PR12MB3069: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /HyTBjfVkkIZof1uMmk/eT9fRd7+i5FCH5/TIzZjy6Q02lMYLkEfbKTWw4z38RUAwTlj9o/H+UAV7YHKx958l6mnqp7Sxh2csSxvmjzccOSNldVbCifHXRCdSTnNoFvCfnOc2AWY/oYSya1eVFIL89WnY/3Km1YIKZ9hlCB9EaEBTIXtw7cO/YHIZz91lGi7xlsklj6NQ5j79ERS8bmjKRZahRltG878XAOCNgjqWPlVFsLOaB/LYdrDCugU0GvVF4LiijV/BZRKjx/nSyGRrYzgDKFHUyaPI40LtDU9NRGaIUjKt2NY7CFbEy2SVqd2T3i35Fd6ezO7KQq1W9MVrCBGMNh1r45WYUvXCJ8oHbCWLK2LxibVXiVQe7Jm5vgzf08V+cNGrulMGJNyz6FEUllMlnm5lIkZlE+36Cu2mXOdgdhh1MS5NstWVKGh2A3hbRyUP90sZTJ8tMYbakyteCnbp2yNEChv3NvrNhGsqdRqeLJfTnhY/x+LCMOZVSqCNSJlt8zOnzKZZcd1tq/Z1QSBMrjVr+tdZAe8FpJepuzuZ82FHZpe58Ovzpm31pdfTiFPooTluQo/RQ70gJ+tKbPlM74KPzT7lejt1C0sR7oyVyzlCFAn3DejxvU0vxkUJRExNzMcdRDvLeyNDQQVrAks6ezMvFivNDGf1yd5MHOEE+QeLKPj+Q4AMQ+6I2Pc4QA9JB68MENin5+udWAQnw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(7696005)(70206006)(6916009)(70586007)(316002)(1076003)(82310400003)(4326008)(7636003)(83380400001)(8936002)(5660300002)(36906005)(2906002)(47076005)(36756003)(2616005)(16526019)(8676002)(186003)(426003)(336012)(508600001)(6666004)(26005)(36860700001)(356005)(6286002)(55016002)(86362001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 10:35:19.2823 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ad4673ab-b95c-4c85-779b-08d9776b595a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT048.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3069 Subject: [dpdk-dev] [PATCH v3 0/3] eal: add memory pre-allocation from existing files X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hugepage allocation from the system takes time, resulting in slow startup or sporadic delays later. Most of the time spent in kernel is zero-filling memory for security reasons, which may be irrelevant in a controlled environment. The bottleneck is memory access speed, so for speeduup the amount of memory cleared must be reduced. We propose a new EAL option --mem-file FILE1,FILE2,... to quickly allocate dirty pages from existing files and clean it as necessary. A new malloc_perf_autotest is provided to estimate the impact. More details are explained in relevant patches. v3: fix hugepage mount point detection v2: fix CI failures Dmitry Kozlyuk (2): eal/linux: make hugetlbfs analysis reusable app/test: add allocator performance autotest Viacheslav Ovsiienko (1): eal: add memory pre-allocation from existing files app/test/meson.build | 2 + app/test/test_malloc_perf.c | 157 +++++++++ doc/guides/linux_gsg/linux_eal_parameters.rst | 17 + lib/eal/common/eal_common_dynmem.c | 6 + lib/eal/common/eal_common_options.c | 23 ++ lib/eal/common/eal_internal_cfg.h | 4 + lib/eal/common/eal_memalloc.h | 8 +- lib/eal/common/eal_options.h | 2 + lib/eal/common/malloc_elem.c | 5 + lib/eal/common/malloc_heap.h | 8 + lib/eal/common/rte_malloc.c | 16 +- lib/eal/include/rte_memory.h | 4 +- lib/eal/linux/eal.c | 28 ++ lib/eal/linux/eal_hugepage_info.c | 158 ++++++--- lib/eal/linux/eal_hugepage_info.h | 39 +++ lib/eal/linux/eal_memalloc.c | 328 +++++++++++++++++- 16 files changed, 735 insertions(+), 70 deletions(-) create mode 100644 app/test/test_malloc_perf.c create mode 100644 lib/eal/linux/eal_hugepage_info.h -- 2.25.1