From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DB012A00C3; Thu, 3 Feb 2022 19:14:06 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F3111411AB; Thu, 3 Feb 2022 19:13:54 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2087.outbound.protection.outlook.com [40.107.93.87]) by mails.dpdk.org (Postfix) with ESMTP id E9686410D5 for ; Thu, 3 Feb 2022 19:13:52 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KcgDiYdHuqBJ9rs/H1/6VbJ6UdxVAo116sQ2QxDMFo4OH7STbsaGxQYAOmZaHkzXsCOEprSsdn+aTAtxgS74q0Ld0t39SGLdtgbentEj1mdYzr3fIccrQhV2y9nmC8REaytJ6oDJk2f0VDRvCTqnbUV0oGMC6kHvgLf7o9dW4+89yCqd5BGGwoV/QKWGn6MTQPci4PBtb6juZ45bYzY51NaYtjoTR/ZNV5d3bkQmKSNt/dkmiyfARWlFtcapuCc3/jbfU8HUQCII8bPD0/S4ZVM3yShimjpfXLwGCdjTEaWTWfuIcgNyGMKDWJa1W0HaSLpBXfa6M4vmrOGFCeZL3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qOIky0naMhVvK5egzkhb7+9xA4v0418SSbVpGe5lRzY=; b=JL0rryWLwHblcVcSJ3Srm1AKzC/VN19hU8WS/N0l605cicVc1mTauavJk4TBzXeHMG+ppY33U4mKyfwZwuwa+DYLu4rRu412OMqDEiCINuxj90Tdigl+x55498b0h8gHiQUU47rOvYKFMIFEV3QjFnWksTl58d5bUoWHBhmzRqcTAmJJxS2b4DH+ESJ81r/d3omSnwjrRDffOHEAOje5qyfqon+yj/pwO2MiwT1d0yqtiA+qc76HTgvKHrNiU9m48RsVN2TB58FGFKvC3bDrKLz0azYo7c1E+3znk0CmCChK/HjDil6UDN+HdT8dVQsTuVWW6xwCJ+9l+J+BNM/eIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qOIky0naMhVvK5egzkhb7+9xA4v0418SSbVpGe5lRzY=; b=lV6t2YZeCuYyP9iWa/nQhgrmy7hQaqmS8w0CobUpEUhJJgjnWKuFOjHjOzgI5a8NctkreJ0+hr4KlEoyCnkbW4y7bsRNQk85JI7WRqgy2Y7W9pTHbDvHVOQTPk7OT5IUeiEe4cq/4Zow/oNALWTKAKcNginGLijV8LQoH7ok755i0lk6Y6FgrBFF5QghpG9cG1qBz2jq/ovhcn+9H2CvCtE4Czs+Z0dAHRD9VXXMqnpmkPZMTPyiGuTh1vFM55S1BrEkfkLCXyKE8Pgn95NgwTkBYrxHOVOJHEe2XJ/oyOQ0Y5zjaSr6qief72j1I3vnAVSpo5bAcCXiprRLSd+mzg== Received: from DM5PR04CA0065.namprd04.prod.outlook.com (2603:10b6:3:ef::27) by BN7PR12MB2596.namprd12.prod.outlook.com (2603:10b6:408:29::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12; Thu, 3 Feb 2022 18:13:51 +0000 Received: from DM6NAM11FT046.eop-nam11.prod.protection.outlook.com (2603:10b6:3:ef:cafe::f1) by DM5PR04CA0065.outlook.office365.com (2603:10b6:3:ef::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12 via Frontend Transport; Thu, 3 Feb 2022 18:13:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by DM6NAM11FT046.mail.protection.outlook.com (10.13.172.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4951.12 via Frontend Transport; Thu, 3 Feb 2022 18:13:50 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 3 Feb 2022 18:13:45 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Thu, 3 Feb 2022 10:13:45 -0800 Received: from nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Thu, 3 Feb 2022 10:13:44 -0800 From: Dmitry Kozlyuk To: CC: Viacheslav Ovsiienko , Aaron Conole Subject: [PATCH v3 2/6] app/test: add allocator performance benchmark Date: Thu, 3 Feb 2022 20:13:32 +0200 Message-ID: <20220203181337.759161-3-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220203181337.759161-1-dkozlyuk@nvidia.com> References: <20220119210917.765505-1-dkozlyuk@nvidia.com> <20220203181337.759161-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 39370326-203b-4f1f-efa7-08d9e740ee34 X-MS-TrafficTypeDiagnostic: BN7PR12MB2596:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BYiW1aTmNvi38JkeOLY+NCGB+t2sb9Snv+1MZ/ahUdZNO/fDpWGG+wEiHU4fFDsteY6An6eP53+DMNN9VJvERo5gh7y3whJ4IshScxchiMPqICBKIdm5wohz1mMQzGUdWrE3I4Ww268x4SFO2kJqb7LGp2h/B5WChOp5VpUSVjHO7AwwqWkjGPveJY6pxs0awGITV0WV+1kY20hVn4xcqwKpq5cIkwclay6kYJh+GCJFo6NU+AkldoINm61510fLpOr3BHMIOLEbb62KSe2oQAaaGAxmUAKsGkU/qstj8X6GiBYN9+E0MoT5ZSgwU8KNBp5rBMMaX6asfeFFVLg9Dc78/cnXaOOHa56LHaWyXLoIdemjUYbDZGZloc5F+PZQkAT753EmtIeC9fpB2YGCjUiKrxTa00wRuw/IXY9OACZsCMxpoMUC6e3+aoskKjFS6HjmS636O2bDmUx7GgA02UTVA1kNRAxwJ7OvngtjBmoqXsaVmW8yocDgvZXOcwivF4XrElfWpyG+DRobkeUm5gn2FnDMTF3RZ1y0pD5OvVz+ns3McX7F5KyqKqlVzSXZ8G7KXB1HSwNzfsyVOvtBHSHwTLksoQZGxJUwlRDFiRBKBHc18Wg1XI0oYsMUascbdYFohPIr9WHuLf2wobzh17KsRvzDPrDLOUwlDfAm1qW1IyK9CdLSM7TS/SDqoKsoGVOrz+6EBKn5vlmgpCYccA== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(70586007)(8936002)(4326008)(86362001)(8676002)(356005)(82310400004)(81166007)(5660300002)(70206006)(40460700003)(6916009)(47076005)(2906002)(36860700001)(316002)(36756003)(508600001)(6666004)(186003)(54906003)(55016003)(2616005)(426003)(7696005)(336012)(1076003)(26005)(6286002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2022 18:13:50.7974 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 39370326-203b-4f1f-efa7-08d9e740ee34 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT046.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR12MB2596 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Memory allocator performance is crucial to applications that deal with large amount of memory or allocate frequently. DPDK allocator performance is affected by EAL options, API used and, at least, allocation size. New autotest is intended to be run with different EAL options. It measures performance with a range of sizes for dirrerent APIs: rte_malloc, rte_zmalloc, and rte_memzone_reserve. Work distribution between allocation and deallocation depends on EAL options. The test prints both times and total time to ease comparison. Memory can be filled with zeroes at different points of allocation path, but it always takes considerable fraction of overall timing. This is why the test measures filling speed and prints how long clearing takes for each size as a reference (for rte_memzone_reserve estimations are printed). Signed-off-by: Dmitry Kozlyuk Reviewed-by: Viacheslav Ovsiienko Acked-by: Aaron Conole --- app/test/meson.build | 2 + app/test/test_malloc_perf.c | 174 ++++++++++++++++++++++++++++++++++++ 2 files changed, 176 insertions(+) create mode 100644 app/test/test_malloc_perf.c diff --git a/app/test/meson.build b/app/test/meson.build index 5476c180ee..b0486cd129 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -87,6 +87,7 @@ test_sources = files( 'test_lpm6_perf.c', 'test_lpm_perf.c', 'test_malloc.c', + 'test_malloc_perf.c', 'test_mbuf.c', 'test_member.c', 'test_member_perf.c', @@ -256,6 +257,7 @@ extra_test_names = [ perf_test_names = [ 'ring_perf_autotest', + 'malloc_perf_autotest', 'mempool_perf_autotest', 'memcpy_perf_autotest', 'hash_perf_autotest', diff --git a/app/test/test_malloc_perf.c b/app/test/test_malloc_perf.c new file mode 100644 index 0000000000..ccec43ae84 --- /dev/null +++ b/app/test/test_malloc_perf.c @@ -0,0 +1,174 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#include +#include +#include +#include +#include +#include + +#include "test.h" + +#define TEST_LOG(level, ...) RTE_LOG(level, USER1, __VA_ARGS__) + +typedef void * (alloc_t)(const char *name, size_t size, unsigned int align); +typedef void (free_t)(void *addr); +typedef void * (memset_t)(void *addr, int value, size_t size); + +static const uint64_t KB = 1 << 10; +static const uint64_t GB = 1 << 30; + +static double +tsc_to_us(uint64_t tsc, size_t runs) +{ + return (double)tsc / rte_get_tsc_hz() * US_PER_S / runs; +} + +static int +test_memset_perf(double *us_per_gb) +{ + static const size_t RUNS = 20; + + void *ptr; + size_t i; + uint64_t tsc; + + TEST_LOG(INFO, "Reference: memset\n"); + + ptr = rte_malloc(NULL, GB, 0); + if (ptr == NULL) { + TEST_LOG(ERR, "rte_malloc(size=%"PRIx64") failed\n", GB); + return -1; + } + + tsc = rte_rdtsc_precise(); + for (i = 0; i < RUNS; i++) + memset(ptr, 0, GB); + tsc = rte_rdtsc_precise() - tsc; + + *us_per_gb = tsc_to_us(tsc, RUNS); + TEST_LOG(INFO, "Result: %f.3 GiB/s <=> %.2f us/MiB\n", + US_PER_S / *us_per_gb, *us_per_gb / KB); + + rte_free(ptr); + TEST_LOG(INFO, "\n"); + return 0; +} + +static int +test_alloc_perf(const char *name, alloc_t *alloc_fn, free_t *free_fn, + memset_t *memset_fn, double memset_gb_us, size_t max_runs) +{ + static const size_t SIZES[] = { + 1 << 6, 1 << 7, 1 << 10, 1 << 12, 1 << 16, 1 << 20, + 1 << 21, 1 << 22, 1 << 24, 1 << 30 }; + + size_t i, j; + void **ptrs; + + TEST_LOG(INFO, "Performance: %s\n", name); + + ptrs = calloc(max_runs, sizeof(ptrs[0])); + if (ptrs == NULL) { + TEST_LOG(ERR, "Cannot allocate memory for pointers"); + return -1; + } + + TEST_LOG(INFO, "%12s%8s%12s%12s%12s%17s\n", "Size (B)", "Runs", + "Alloc (us)", "Free (us)", "Total (us)", + memset_fn != NULL ? "memset (us)" : "est.memset (us)"); + for (i = 0; i < RTE_DIM(SIZES); i++) { + size_t size = SIZES[i]; + size_t runs_done; + uint64_t tsc_start, tsc_alloc, tsc_memset = 0, tsc_free; + double alloc_time, free_time, memset_time; + + tsc_start = rte_rdtsc_precise(); + for (j = 0; j < max_runs; j++) { + ptrs[j] = alloc_fn(NULL, size, 0); + if (ptrs[j] == NULL) + break; + } + tsc_alloc = rte_rdtsc_precise() - tsc_start; + + if (j == 0) { + TEST_LOG(INFO, "%12zu Interrupted: out of memory.\n", + size); + break; + } + runs_done = j; + + if (memset_fn != NULL) { + tsc_start = rte_rdtsc_precise(); + for (j = 0; j < runs_done && ptrs[j] != NULL; j++) + memset_fn(ptrs[j], 0, size); + tsc_memset = rte_rdtsc_precise() - tsc_start; + } + + tsc_start = rte_rdtsc_precise(); + for (j = 0; j < runs_done && ptrs[j] != NULL; j++) + free_fn(ptrs[j]); + tsc_free = rte_rdtsc_precise() - tsc_start; + + alloc_time = tsc_to_us(tsc_alloc, runs_done); + free_time = tsc_to_us(tsc_free, runs_done); + memset_time = memset_fn != NULL ? + tsc_to_us(tsc_memset, runs_done) : + memset_gb_us * size / GB; + TEST_LOG(INFO, "%12zu%8zu%12.2f%12.2f%12.2f%17.2f\n", + size, runs_done, alloc_time, free_time, + alloc_time + free_time, memset_time); + + memset(ptrs, 0, max_runs * sizeof(ptrs[0])); + } + + free(ptrs); + TEST_LOG(INFO, "\n"); + return 0; +} + +static void * +memzone_alloc(const char *name __rte_unused, size_t size, unsigned int align) +{ + const struct rte_memzone *mz; + char gen_name[RTE_MEMZONE_NAMESIZE]; + + snprintf(gen_name, sizeof(gen_name), "test-mz-%"PRIx64, rte_rdtsc()); + mz = rte_memzone_reserve_aligned(gen_name, size, SOCKET_ID_ANY, + RTE_MEMZONE_1GB | RTE_MEMZONE_SIZE_HINT_ONLY, align); + return (void *)(uintptr_t)mz; +} + +static void +memzone_free(void *addr) +{ + rte_memzone_free((struct rte_memzone *)addr); +} + +static int +test_malloc_perf(void) +{ + static const size_t MAX_RUNS = 10000; + + double memset_us_gb = 0; + + if (test_memset_perf(&memset_us_gb) < 0) + return -1; + + if (test_alloc_perf("rte_malloc", rte_malloc, rte_free, memset, + memset_us_gb, MAX_RUNS) < 0) + return -1; + if (test_alloc_perf("rte_zmalloc", rte_zmalloc, rte_free, memset, + memset_us_gb, MAX_RUNS) < 0) + return -1; + + if (test_alloc_perf("rte_memzone_reserve", memzone_alloc, memzone_free, + NULL, memset_us_gb, RTE_MAX_MEMZONE - 1) < 0) + return -1; + + return 0; +} + +REGISTER_TEST_COMMAND(malloc_perf_autotest, test_malloc_perf); -- 2.25.1