From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5C84FA0A0F; Mon, 5 Jul 2021 14:50:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C74804068C; Mon, 5 Jul 2021 14:50:08 +0200 (CEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2070.outbound.protection.outlook.com [40.107.220.70]) by mails.dpdk.org (Postfix) with ESMTP id 1C6054003C for ; Mon, 5 Jul 2021 14:50:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PZjzXB/bfxr1lT1J6BJe9iWUn0DBmPWXR4VLZSWbBNOv26LgWkUXNk4jU9Aw/ppIwoE2Ke3oZbpXQWWpvP4pxJveraETXdxEDW7uRnMUBuGxXRXCe/J8uvKKth1QrXJD60BMRJrNpCTn5y9Lp1NF4cJfN/OKSOGexOE9WxtFRUuS9/AV4MwMow4EVN3sObWtzHWx0Apk/UPNepXdptLJZXqc563pR9618p4/f9qe0lTPi8YGeLukojipvf/6SaLQz8QmRsnwMzQNZDUvTM1s9i4XcpTsGy8b5zecaRZddtcbPoI5r4G7AnecQa5Z0gtB9SESd6Yvgwa8oBJZKLXtqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4vXCiDKaObEoJaaEEqxEho4KcWFh7ef1W6+zfZMtjpo=; b=m7+Jj35+36m1aUOXJpoH7FMR4UxF/EvqL3KsB2F/zeu6seYARwse3FXnkXZvV1uIlfZjpsCkvS3S5DyxZXsb2SakqJl9SfK06/QFsZeZoIqebHXM+z3Q20nToi9BY5zuwo3V4h7UgyyPfpXae070+leDjAiSQOgN5vswHDXrY0iAW5cVw0Mh4tadsM+4dFuAiw054o8rrfyU/bLF0YrT5NNppzp/sES3Q2DamZkhyeOUTLd8S88eQstIcSjjrzUaXUWEHuzrh0DvdxwAFNX5upiL/Ldds5GMy74LCyQnziwUaZkNOAwBiZRHU/rhXSSdH7wO/oeDiu/k9q2UWzIjGw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4vXCiDKaObEoJaaEEqxEho4KcWFh7ef1W6+zfZMtjpo=; b=JpQD2Ymu8fqLLueiJNh2K+y8mBEA5Fr1mKRq5dn08n0JWGOuJKWfAYXqNjkv2iZrPn3SpP5NYOs1egfNCJ0cuk4WT8RUkbJcwtqbjD8kgFhIZpuapmOfbxhYQeiYkyHxYExa2GmAjjLXFotvhP9s39ABswWW8SaH9J5vf9IIkF53ZI+xE1JwVgMLV2zsAHv7G175sXNUrmkVTnErIiqBdkL2PoyLkbOxVm0KO3b5K4SCGoWR8nUWnNrnKFlA0p7jxiTKUWyEO9sPbwpkltB3H8JKUjbUN6Bj8KYQVIqnGQhF1GiuD8PHRW+j/zkk0FbUdNTIWHKy/E/xzd/siXhCMw== Received: from MWHPR15CA0045.namprd15.prod.outlook.com (2603:10b6:300:ad::31) by CH2PR12MB4229.namprd12.prod.outlook.com (2603:10b6:610:a5::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.18; Mon, 5 Jul 2021 12:50:05 +0000 Received: from CO1NAM11FT059.eop-nam11.prod.protection.outlook.com (2603:10b6:300:ad:cafe::9e) by MWHPR15CA0045.outlook.office365.com (2603:10b6:300:ad::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Mon, 5 Jul 2021 12:50:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT059.mail.protection.outlook.com (10.13.174.160) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Mon, 5 Jul 2021 12:50:05 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 5 Jul 2021 12:50:04 +0000 From: Dmitry Kozlyuk To: CC: Anatoly Burakov Date: Mon, 5 Jul 2021 15:49:46 +0300 Message-ID: <20210705124950.902701-1-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6dd7eada-2e40-4e1c-4871-08d93fb369ae X-MS-TrafficTypeDiagnostic: CH2PR12MB4229: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BllKNEJXdL1MZbwZ/xW3u/6O30h1s55tA9l1hVuhK4NiPG2gcJiNJG/CDQxC4jxACDKsII9T+j3CQklDhIlkM9hlUf32WGRwK4oRBGNFLefd5tbwj2/PCDpAwbrmwMhqFDUCLKlr6FqLg+2Z5B8Acli1r76A7OMH7WrmL+FL1IRXE9FKDlpqb0+YB5oSxsPmZX+EE3ZzJYbudKNqUnP4ODpP+omUIJMIAoqUrJXBEdFD9iiJ1Vnei78ELH7miWkiIPfBjoNbVQ6QwiJ649zobBxqd3sdljO1CAyuT3FIi1QpksaknxLcGPc+1ObIRT7dQpxnKqkudOCn/KOdKuBLgxU0OLU/ts4s+sANk44gOoKGw9rpzGwCnwV3PRxzHSb16PZtvKeI86kVhM9PNeHvWJBJg/HecSa/1ETBLYkU+czcfOwi1+6rC55ZbcAgBN7gceRFutMhGWEh1GfHe6iz1kv513LYgjZLDSaM+kUlrJcaRqCbGdFZ//FVInXhOiraWGYllVnQrIHlJ57n5Jo8+kIkIzFd/bzQRlTGUDjkpYGJMGRek1jYJEUgBFcpnLhimHrLwb/1C/pGAuhQk7BAlr/QukIWp/PGH/0neSH7pAdSVUMSChrrjVA3BZPzl26M3RRUoT3GOTKef3yFIiDrKvwuwItfnHHiuz4uEfOr6lo2NK1ahLbB3yubd6yeRpi8JuEZ5K04WEB9BIT+kNz3BA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(376002)(396003)(136003)(346002)(46966006)(36840700001)(36860700001)(7636003)(356005)(82310400003)(70586007)(47076005)(2906002)(86362001)(1076003)(83380400001)(5660300002)(82740400003)(70206006)(6666004)(36756003)(36906005)(6286002)(2616005)(7696005)(8676002)(316002)(186003)(6916009)(426003)(8936002)(16526019)(478600001)(336012)(26005)(55016002)(4326008); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jul 2021 12:50:05.3271 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6dd7eada-2e40-4e1c-4871-08d93fb369ae X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT059.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4229 Subject: [dpdk-dev] [PATCH 21.11 0/3] eal: add memory pre-allocation from existing files X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hugepage allocation from the system takes time, resulting in slow startup or sporadic delays later. Most of the time spent in kernel is zero-filling memory for security reasons, which may be irrelevant in a controlled environment. The bottleneck is memory access speed, so for speeduup the amount of memory cleared must be reduced. We propose a new EAL option --mem-file FILE1,FILE2,... to quickly allocate dirty pages from existing files and clean it as necessary. A new malloc_perf_autotest is provided to estimate the impact. More details are explained in relevant patches. Dmitry Kozlyuk (2): eal/linux: make hugetlbfs analysis reusable app/test: add allocator performance autotest Viacheslav Ovsiienko (1): eal: add memory pre-allocation from existing files app/test/meson.build | 2 + app/test/test_malloc_perf.c | 155 +++++++++ doc/guides/linux_gsg/linux_eal_parameters.rst | 17 + lib/eal/common/eal_common_dynmem.c | 6 + lib/eal/common/eal_common_options.c | 23 ++ lib/eal/common/eal_internal_cfg.h | 4 + lib/eal/common/eal_memalloc.h | 8 +- lib/eal/common/eal_options.h | 2 + lib/eal/common/malloc_elem.c | 5 + lib/eal/common/malloc_heap.h | 8 + lib/eal/common/rte_malloc.c | 16 +- lib/eal/include/rte_memory.h | 4 +- lib/eal/linux/eal.c | 28 ++ lib/eal/linux/eal_hugepage_info.c | 158 ++++++--- lib/eal/linux/eal_hugepage_info.h | 39 +++ lib/eal/linux/eal_memalloc.c | 328 +++++++++++++++++- 16 files changed, 733 insertions(+), 70 deletions(-) create mode 100644 app/test/test_malloc_perf.c create mode 100644 lib/eal/linux/eal_hugepage_info.h -- 2.25.1