From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 39E19470E5 for ; Thu, 25 Dec 2025 10:22:07 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 33C9D4025E; Thu, 25 Dec 2025 10:22:07 +0100 (CET) Received: from CO1PR03CU002.outbound.protection.outlook.com (mail-westus2azon11010029.outbound.protection.outlook.com [52.101.46.29]) by mails.dpdk.org (Postfix) with ESMTP id CD1F94025E for ; Thu, 25 Dec 2025 10:22:05 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vHzBiQlNYx4DyWaO3w9jqOnf0O3PWZ/urTtNBIcM10uQZ+CwRW3rWHrDh8Cj9NslNKt2/vZJQB6ng/CNrmZFzGTVql2JJlz6Bm1TIwnu9VF7JLiztVcgcDbu5oq3Yu4K1PTN7Xl4T3uOdZkrA6rkUPnd6DxkvvxflYsEapSsyNdIQvQ6ajuH8RPjnGoEpqBBOhAOuR9PbjlX4l57S+HsJ5wIX1Kb1KrN4tIevcoqyubmOmeqDpMaUdrEUiyLAENM+Wtc6Bz/0GWRj5qFEh73H4VGX5dpr9tJDbqmZ125SlEYv8/XmJ3Egh+UIa/boeLc5umsIK6KFoA1PJ/OuEFtsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9lHOMhN7t76l8cHniyAhJhR1i5p3mQBH2+D6bGMBsfA=; b=f3V3WrcQJqcWYb8bhaEY8COO0tuuTLjnDkqs6rP0HBdYuM7ISJJ0Z3BylsFrm6WgxHJ9BDEhH7yGJfuuow4wVFIExFREC/DQGFQolTX4uIOG93pRhVMW6lcOEebjwPOQXje6V5CtIw7mWlcuwjVZoegzagZo7p3VcE+tOlY94jcks2JY7xIAJW4GCrkXZ8tYpPwyRT8BPEeP1EC79UGkxPH1vBF7Gg0PY8Z8bAzsMZcQuev1wMIGDMuXb6mW3ag0iNveJNv/BLQVkyd4W9UeJQn7/+B2k84TgXywtL3V2WNYcDMyMlBDY8wHBgNQCMKVI5EsKaMebeGOPNObS6Lspg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=huawei.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9lHOMhN7t76l8cHniyAhJhR1i5p3mQBH2+D6bGMBsfA=; b=lOhhFTfXYSBI5mgWYQnO8l7VFmarmvuds5ZW90vfHOkY+mpLge3MIZxqlm2wmxW1M1UQxeEFe/dnzE5JnJFiP1g9uRcyCQg/8Vm0aH+K0gLsGg2LpozdLXJK9t+CGjv3mYUko5sPvOL1EofDnRfOpWP2bCjr8lDnBEMc/RynOJQPnynTF/Ta2X7SrOuuObbMCeYKORMs86fgLk1MpTK4araoYlerdPJnTUKPasdgFzhJQcc22S4QuMozYo0M2vC7jY9+saZh6KeKIl7eV+2pZCKNz6Ks3EKGDNXIXwqpFg9FZkdK/IuPSNIoFCYS6imSKoykOmPHSzDgmPyZsBex5A== Received: from CH0PR03CA0392.namprd03.prod.outlook.com (2603:10b6:610:11b::31) by IA0PR12MB8715.namprd12.prod.outlook.com (2603:10b6:208:487::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9456.11; Thu, 25 Dec 2025 09:22:00 +0000 Received: from CH2PEPF00000142.namprd02.prod.outlook.com (2603:10b6:610:11b:cafe::bf) by CH0PR03CA0392.outlook.office365.com (2603:10b6:610:11b::31) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9456.11 via Frontend Transport; Thu, 25 Dec 2025 09:21:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CH2PEPF00000142.mail.protection.outlook.com (10.167.244.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9456.9 via Frontend Transport; Thu, 25 Dec 2025 09:21:59 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 25 Dec 2025 01:21:38 -0800 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 25 Dec 2025 01:21:37 -0800 Received: from nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Thu, 25 Dec 2025 01:21:37 -0800 From: Shani Peretz To: Chengwen Feng CC: dpdk stable Subject: patch 'app/dma-perf: fix on-flight DMA when verifying data' has been queued to stable release 23.11.6 Date: Thu, 25 Dec 2025 11:17:51 +0200 Message-ID: <20251225091938.345892-30-shperetz@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251225091938.345892-1-shperetz@nvidia.com> References: <20251221145746.763179-93-shperetz@nvidia.com> <20251225091938.345892-1-shperetz@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000142:EE_|IA0PR12MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 34278b06-a006-47bb-d5ab-08de43970e4d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|36860700013|1800799024|82310400026|7053199007|13003099007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?LYP2KxciHS0XGIrvEdr+ScI3bqxuguwun9YtdeL2FG7t1diCrba+XxSN0Tw3?= =?us-ascii?Q?zqM+XycX357rqZZoiMR3HWbO3adyLwbHoQUUDfzu+jiDMxzvT+myKcWeuhNd?= =?us-ascii?Q?cg+vPcZuaGr0XKrf3RnnITmjd5lMaUONHg8NdwQ+VUFgH/AcvgBx0BuLCB4d?= =?us-ascii?Q?i7f9PKiIiL2kLn2osC9Ezmv8zV4VbfzScMh5FdZ0t80vuBwMyVOISSfQO147?= =?us-ascii?Q?50GwBVBStbRyL3sa0fg5D/ZYozBCwCf/tTqgxX1GveAr46070/znopzVnx4j?= =?us-ascii?Q?u/ptmX+94KRrqBbMkMshCrc/q2Sz4/xzXVTTYLAsRDyiHpsWd6Ov7Qj5JiZk?= =?us-ascii?Q?726S0vcZ0qRl5I/W2bObq3k8B8Ju04HqRRek5jzlCiTPkQ0wcZ1q11fUCaDc?= =?us-ascii?Q?i9X8/vh9UDV/SeDQih7FUCPA3d9STFQZ3FJNykfq8+o/ul6HUCuH6cVjocOs?= =?us-ascii?Q?fwN5lVZoJGSaWhLdahcijbbUUy1xSTMPG5i23VIbgFo4YEho24oX3lqI8UWz?= =?us-ascii?Q?L3CuzhY9tJvMhydl7fi4l2OmDTU4MRwxpO1i4lMiBoFsU/bZLY1WIhUQS7CS?= =?us-ascii?Q?Opbs9DKBPJuu+ANDlqexe89XX01fz5gHJZjWR1YmRZ4naRhnGn89cQkhsDMl?= =?us-ascii?Q?z4or/BVsdJpFtkDWZspTYAAfLFVq6UVx8i84rqzTD4PnmULNUmsA1NfO9D0s?= =?us-ascii?Q?bqjADmUoHyYx00ukbdFclbsoAblQzqWSGejOw2pujCinJaF84sZC3mnDe7ok?= =?us-ascii?Q?wJPAbdQlVPq9AI1487Hoc0jrIcIO9gf7BlrCTfXC9WYsLqj6ZYahWHwZnQRx?= =?us-ascii?Q?asSnIfp5CdEwZRWtFznnJQ9limUbSa1qJjt/1z8VdsQ5MBchuA28iqdmiGOB?= =?us-ascii?Q?7H8fKox6tvBJZDW8Xxq/BmfT8lVCtMTuZ5MY8yilcGMV9HkKwak97ARnPV51?= =?us-ascii?Q?ktl2SS7o3QZ4+3jakGgfJ1m2dFO7F8EWxF8mEn4ENqAe4fx4Qy2gKMX89zzd?= =?us-ascii?Q?lUlKmiYtWdhNelrwI2QFMqIv69wMx7HhqpELTjAprsBEAUVryZMGiXmzi/Nk?= =?us-ascii?Q?qsHEHw2MiEurzyJKG5v7vp7fwMkgXhcO+ewdx/sdNuhrgB7dVHqVGEiN2lht?= =?us-ascii?Q?+iZ6N77QcWX25G6RkA7SJQss/tSJz7PcmZaBee+xSaFRevAHTqrAHfxUJgUZ?= =?us-ascii?Q?OUb3wfo6djvedXJgEdIhIqe+F23IeUzK8Qs6Ch1TixqdGRCyDEBKYKPvl9pA?= =?us-ascii?Q?yar48KrmIR7lCcRJHShTuWhwCbhrMAMvSqJWX5AJQEWkjo7SgsEXTabOtRq0?= =?us-ascii?Q?OWDdL47eWDe2rdTlgmwfqkVpy+gMdTwCUjiLytvT5JOVuj9pLRPtbOG97c5c?= =?us-ascii?Q?aEI9FcqlakvCNQ0c46+qtBqXM1b20vaIHR+8hgLaWw5DwQ3BH0zRh3e0ggTV?= =?us-ascii?Q?AC7oT+l5RyvWjMMaYz9ja4fA43XjryAEbRNBGWuYukwb1SakLt25ydIreanQ?= =?us-ascii?Q?nFLHrkHN88nqXWrQwjgGfU+gpXSA5V1ujVmg9Mg5czWjt2g7R3PziMeC8WX7?= =?us-ascii?Q?QxNRlmyiCPweEM2s3QZW5YPDpj7Q/jPFftWOlRa8?= X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230040)(376014)(36860700013)(1800799024)(82310400026)(7053199007)(13003099007); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Dec 2025 09:21:59.0708 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 34278b06-a006-47bb-d5ab-08de43970e4d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000142.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8715 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 23.11.6 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 12/30/25. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/shanipr/dpdk-stable This queued commit can be viewed at: https://github.com/shanipr/dpdk-stable/commit/548e62ca1a3df0b3a832f43dd8889cd908a99e27 Thanks. Shani --- >From 548e62ca1a3df0b3a832f43dd8889cd908a99e27 Mon Sep 17 00:00:00 2001 From: Chengwen Feng Date: Tue, 25 Nov 2025 09:51:29 +0800 Subject: [PATCH] app/dma-perf: fix on-flight DMA when verifying data [ upstream commit d1b3b669674a17c58eabf3d631b21aaad7232403 ] There maybe on-flight DMA when verify_data() because the DMA device may still working when worker exit. This commit add wait DMA complete stage before worker exit. Fixes: 623dc9364dc6 ("app/dma-perf: introduce DMA performance test") Cc: stable@dpdk.org Signed-off-by: Chengwen Feng --- app/test-dma-perf/benchmark.c | 45 +++++++++++++++++++++++++++-------- 1 file changed, 35 insertions(+), 10 deletions(-) diff --git a/app/test-dma-perf/benchmark.c b/app/test-dma-perf/benchmark.c index 9b1f58c78c..084c953b02 100644 --- a/app/test-dma-perf/benchmark.c +++ b/app/test-dma-perf/benchmark.c @@ -18,7 +18,6 @@ #define MAX_DMA_CPL_NB 255 #define TEST_WAIT_U_SECOND 10000 -#define POLL_MAX 1000 #define CSV_LINE_DMA_FMT "Scenario %u,%u,%s,%u,%u,%u,%u,%.2lf,%" PRIu64 ",%.3lf,%.3lf\n" #define CSV_LINE_CPU_FMT "Scenario %u,%u,NA,NA,NA,%u,%u,%.2lf,%" PRIu64 ",%.3lf,%.3lf\n" @@ -215,6 +214,40 @@ do_dma_submit_and_poll(uint16_t dev_id, uint64_t *async_cnt, worker_info->total_cpl += nr_cpl; } +static int +do_dma_submit_and_wait_cpl(uint16_t dev_id, uint64_t async_cnt) +{ +#define MAX_WAIT_MSEC 1000 +#define MAX_POLL 1000 +#define DEQ_SZ 64 + enum rte_dma_vchan_status st; + uint32_t poll_cnt = 0; + uint32_t wait_ms = 0; + uint16_t nr_cpl; + + rte_dma_submit(dev_id, 0); + + if (rte_dma_vchan_status(dev_id, 0, &st) < 0) { + rte_delay_ms(MAX_WAIT_MSEC); + goto wait_cpl; + } + + while (st == RTE_DMA_VCHAN_ACTIVE && wait_ms++ < MAX_WAIT_MSEC) { + rte_delay_ms(1); + rte_dma_vchan_status(dev_id, 0, &st); + } + +wait_cpl: + while ((async_cnt > 0) && (poll_cnt++ < MAX_POLL)) { + nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL, NULL); + async_cnt -= nr_cpl; + } + if (async_cnt > 0) + PRINT_ERR("Error: wait DMA %u failed!\n", dev_id); + + return async_cnt == 0 ? 0 : -1; +} + static inline int do_dma_mem_copy(void *p) { @@ -226,10 +259,8 @@ do_dma_mem_copy(void *p) const uint32_t buf_size = para->buf_size; struct rte_mbuf **srcs = para->srcs; struct rte_mbuf **dsts = para->dsts; - uint16_t nr_cpl; uint64_t async_cnt = 0; uint32_t i; - uint32_t poll_cnt = 0; int ret; worker_info->stop_flag = false; @@ -260,13 +291,7 @@ dma_copy: break; } - rte_dma_submit(dev_id, 0); - while ((async_cnt > 0) && (poll_cnt++ < POLL_MAX)) { - nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL, NULL); - async_cnt -= nr_cpl; - } - - return 0; + return do_dma_submit_and_wait_cpl(dev_id, async_cnt); } static inline int -- 2.43.0 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2025-12-25 11:16:37.814178465 +0200 +++ 0030-app-dma-perf-fix-on-flight-DMA-when-verifying-data.patch 2025-12-25 11:16:35.518105000 +0200 @@ -1 +1 @@ -From d1b3b669674a17c58eabf3d631b21aaad7232403 Mon Sep 17 00:00:00 2001 +From 548e62ca1a3df0b3a832f43dd8889cd908a99e27 Mon Sep 17 00:00:00 2001 @@ -3 +3 @@ -Date: Mon, 20 Oct 2025 12:11:03 +0800 +Date: Tue, 25 Nov 2025 09:51:29 +0800 @@ -5,0 +6,2 @@ +[ upstream commit d1b3b669674a17c58eabf3d631b21aaad7232403 ] + @@ -16,2 +18,2 @@ - app/test-dma-perf/benchmark.c | 71 ++++++++++++++++++++++------------- - 1 file changed, 44 insertions(+), 27 deletions(-) + app/test-dma-perf/benchmark.c | 45 +++++++++++++++++++++++++++-------- + 1 file changed, 35 insertions(+), 10 deletions(-) @@ -20 +22 @@ -index 6643ccc95f..4ce95d0f7b 100644 +index 9b1f58c78c..084c953b02 100644 @@ -23 +25 @@ -@@ -19,7 +19,6 @@ +@@ -18,7 +18,6 @@ @@ -31 +33 @@ -@@ -293,6 +292,45 @@ do_dma_submit_and_poll(uint16_t dev_id, uint64_t *async_cnt, +@@ -215,6 +214,40 @@ do_dma_submit_and_poll(uint16_t dev_id, uint64_t *async_cnt, @@ -36 +38 @@ -+do_dma_submit_and_wait_cpl(uint16_t dev_id, uint64_t async_cnt, bool use_ops) ++do_dma_submit_and_wait_cpl(uint16_t dev_id, uint64_t async_cnt) @@ -41 +42,0 @@ -+ struct rte_dma_op *op[DEQ_SZ]; @@ -47,2 +48 @@ -+ if (!use_ops) -+ rte_dma_submit(dev_id, 0); ++ rte_dma_submit(dev_id, 0); @@ -62,4 +62 @@ -+ if (use_ops) -+ nr_cpl = rte_dma_dequeue_ops(dev_id, 0, op, DEQ_SZ); -+ else -+ nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL, NULL); ++ nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL, NULL); @@ -75 +72 @@ - do_dma_plain_mem_copy(void *p) + do_dma_mem_copy(void *p) @@ -77 +74 @@ -@@ -304,10 +342,8 @@ do_dma_plain_mem_copy(void *p) +@@ -226,10 +259,8 @@ do_dma_mem_copy(void *p) @@ -88,25 +85 @@ -@@ -338,13 +374,7 @@ dma_copy: - break; - } - -- rte_dma_submit(dev_id, 0); -- while ((async_cnt > 0) && (poll_cnt++ < POLL_MAX)) { -- nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL, NULL); -- async_cnt -= nr_cpl; -- } -- -- return 0; -+ return do_dma_submit_and_wait_cpl(dev_id, async_cnt, false); - } - - static inline int -@@ -360,8 +390,6 @@ do_dma_sg_mem_copy(void *p) - const uint16_t dev_id = para->dev_id; - uint32_t nr_buf = para->nr_buf; - uint64_t async_cnt = 0; -- uint32_t poll_cnt = 0; -- uint16_t nr_cpl; - uint32_t i, j; - int ret; - -@@ -397,13 +425,7 @@ dma_copy: +@@ -260,13 +291,7 @@ dma_copy: @@ -123,28 +96 @@ -+ return do_dma_submit_and_wait_cpl(dev_id, async_cnt, false); - } - - static inline int -@@ -414,11 +436,11 @@ do_dma_enq_deq_mem_copy(void *p) - volatile struct worker_info *worker_info = &(para->worker_info); - struct rte_dma_op **dma_ops = para->dma_ops; - uint16_t kick_batch = para->kick_batch, sz; -- uint16_t enq, deq, poll_cnt; -- uint64_t tenq, tdeq; - const uint16_t dev_id = para->dev_id; - uint32_t nr_buf = para->nr_buf; - struct rte_dma_op *op[DEQ_SZ]; -+ uint64_t tenq, tdeq; -+ uint16_t enq, deq; - uint32_t i; - - worker_info->stop_flag = false; -@@ -454,11 +476,7 @@ do_dma_enq_deq_mem_copy(void *p) - break; - } - -- poll_cnt = 0; -- while ((tenq != tdeq) && (poll_cnt++ < POLL_MAX)) -- tdeq += rte_dma_dequeue_ops(dev_id, 0, op, DEQ_SZ); -- -- return 0; -+ return do_dma_submit_and_wait_cpl(dev_id, tenq - tdeq, true); ++ return do_dma_submit_and_wait_cpl(dev_id, async_cnt); @@ -154,8 +99,0 @@ -@@ -614,7 +632,6 @@ setup_memory_env(struct test_configure *cfg, uint32_t nr_buf, - } - - if (cfg->use_ops) { -- - nr_buf /= RTE_MAX(nb_src_sges, nb_dst_sges); - *dma_ops = rte_zmalloc(NULL, nr_buf * (sizeof(struct rte_dma_op *)), - RTE_CACHE_LINE_SIZE);