From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5295D45BAE; Wed, 23 Oct 2024 10:03:04 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CFBD941141; Wed, 23 Oct 2024 10:02:23 +0200 (CEST) Received: from EUR02-VI1-obe.outbound.protection.outlook.com (mail-vi1eur02on2050.outbound.protection.outlook.com [40.107.241.50]) by mails.dpdk.org (Postfix) with ESMTP id E8AA14065B for ; Wed, 23 Oct 2024 10:02:11 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mKVjhE7rgBtPcWBNKvP2BKoyY98NM/oafDf4ieteedgD5pUHGkLIz3oUi8kW8ivuREuT7MA+sFHJC3Pjw28BrBC9eaDn3wg0HfUHXfJ9kaM3ZhsipXtmQuOJD1litrBN2RhWtWTz7k11ie+BXhsDnoZEMb0pvizhW+n1a3LSHBZNJ72LLVWRWmf5PUfEnO1cn5TLhWrQnhXoHNvJc5sFytKSZGgYsijfFdcbYuYHijRTmKIqwMzXJN5g+Q9zF59TV9g6fqKzvqmxE2/YKzNSW2t/4mCV5w5AhntTdVGEZCaSeccCo1uwbOaskNxmdxLtJnhBXVhOXMDOQ3EdzREVxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iP2c/4rnAjtikmq4Wf9dMfW4zAfhUVVPRc+9Hw5SRmk=; b=o2upOclsWRpdzl3ZP4EU7wES0/itV0DUGDxtkxoE600qAONHfzQLyK43vtNHmtcAZogl5JUMRsew2xGia6JZl7ivYYXLw+nMsVxV1urvFyCu6c3MwygvwzqgopoXsyEyboZ4et6KRYxQ9xhkr8smInX/40s9hCbRaZNbrloUqY2ElAoj0b5HDLxhjWRGr25w5L4xiIRiYAWq5el4LAgj6eGYxooQVA7C3wc8B97/7Jk0IHdUYYbtM3omvDLNa7m/5TBwoO9mTIFtbS7j1e0HlxyuoeXvdeoWgwbw5ZzDsGRea2gqlcFobAQt+mq0mlfviD4uRVNWIDdHPvp8s/QOTQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iP2c/4rnAjtikmq4Wf9dMfW4zAfhUVVPRc+9Hw5SRmk=; b=Spj4PiuJzqkLy1adwvus1Mn09v2uKt6BR/JoIltD1GMUWhCilNdY6euh18+Q8hP5MMS4SSp3vftvEXRSssLiYJ8bX7xUYKtaZh7MioLbugaLzMQD6Mpd7whecwBYjxBX7x5H2m5siSMCtFMQBmgBDCHi8zlZ/g26jGCKJrUBo3QxbGYtPbDvF+po4XyWeqEiaTiklk+o2jzOmgYyAUshw+iGgCn+2KUYGjj2NAshnYrUthKw135VWm4VkqEFo0ees5L0a0byWNHOZax7A0vhRmOjG7Se6uD/h9UYrfIFp1zPgedkfWsTH2hheT70MsUM7PP4vBFhgUcGrqVanqpu+g== Received: from DUZPR01CA0316.eurprd01.prod.exchangelabs.com (2603:10a6:10:4ba::14) by DUZPR07MB9959.eurprd07.prod.outlook.com (2603:10a6:10:4de::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.23; Wed, 23 Oct 2024 08:02:05 +0000 Received: from DU2PEPF00028D0A.eurprd03.prod.outlook.com (2603:10a6:10:4ba:cafe::3f) by DUZPR01CA0316.outlook.office365.com (2603:10a6:10:4ba::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17 via Frontend Transport; Wed, 23 Oct 2024 08:02:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU2PEPF00028D0A.mail.protection.outlook.com (10.167.242.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.14 via Frontend Transport; Wed, 23 Oct 2024 08:02:04 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.60) with Microsoft SMTP Server id 15.2.1544.11; Wed, 23 Oct 2024 10:02:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 3F11C3800BB; Wed, 23 Oct 2024 10:02:03 +0200 (CEST) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: CC: , =?UTF-8?q?Morten=20Br=C3=B8rup?= , Stephen Hemminger , Konstantin Ananyev , David Marchand , Jerin Jacob , Luka Jankovic , Thomas Monjalon , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= Subject: [PATCH v16 4/8] eal: add lcore variables' programmer's guide Date: Wed, 23 Oct 2024 09:52:58 +0200 Message-ID: <20241023075302.869008-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241023075302.869008-1-mattias.ronnblom@ericsson.com> References: <20241017055743.848292-1-mattias.ronnblom@ericsson.com> <20241023075302.869008-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF00028D0A:EE_|DUZPR07MB9959:EE_ X-MS-Office365-Filtering-Correlation-Id: 03b709c5-656f-42f2-c0b5-08dcf338fbf5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|36860700013|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?UlpGQlRVYnlkY0gvdldYNDhVUU96MGlsRDNSN0NwYk02bGhvUEl0eWYzeHhr?= =?utf-8?B?VDg0VWp3TVNWczdMN3Y0cmZJeklZOGtCMVIzSFdwRUdLRWg4dEsyck5XOC8r?= =?utf-8?B?OVFDOWFmSnNOMzR4cUlvV2tscWlCSW5kaEVLUUlZdzMzQnZmWGZJQlJva1BO?= =?utf-8?B?LzVIVktldUhHTW5sSmd3eGZBUExDdm5hRE9MK0VzOURSRW16ZUIrTFROZFpL?= =?utf-8?B?UkR0V2hqeG1uU202Q3liZ3ByZHlUdGlFYTFTVi9jbUpHY09vRnlNQzRFY01O?= =?utf-8?B?eTlBSGxNTERMa2ppb3Jub3g5ZTlrVjc3cU54MW9KT3dpd0pHT1I1bUV5WUcz?= =?utf-8?B?MGxjeDVTcjFHbXJWWjZ6UWRiSDlrNDNUZFRzazdUTGkvUTVjcVl6UEx4MnRh?= =?utf-8?B?UjJubDhEL1pOUE5OUEsraXpKT05uS1lJK2czT3NpTXR5RzhXcjkrTmlwenR5?= =?utf-8?B?WXhkN2E4UHRFazZHY21YTmVJM3BTNGFRLy9LaXZzZWpPQjFBaS8vaFUvbGw3?= =?utf-8?B?YWlpcHdQdEFKUlJQMkoxdHhwNENzRmw4NGtxK2grRWJFdFp2U2FCSEtqWnlU?= =?utf-8?B?dlpObm0wZjE3WWxvNWZ5elU2UlZnTjlBNDFoYzMycHg1TmV5dElSRjZ6V2wz?= =?utf-8?B?S2NqRGlHbXQ3SFVVRXRBTmVtaHNQZ1pVejZWZ3VVcjFGSHN5Y3gzN000dWtq?= =?utf-8?B?SEZDRG5ORVIraG9kaVd0UkVmeEJHU2ozNkZjRHBuTHk3MVRHcjhONkZzUkRy?= =?utf-8?B?djIyZVYzQk4wRHowSm16MWtzc210Z3RUNUY3aWJERGw1RlFGMXR4OUNKNkl5?= =?utf-8?B?cEdvRW0xelkxb1lrTWhpd3p2b29aWjNjclBPdWxGN3N4MldOOENQRS9UVFln?= =?utf-8?B?VS9CQ0ZGRFdlbXRKZWFZUzNBOFI2T1VVSGN6bXpINzhzNGxjZ0t2REJZTFdF?= =?utf-8?B?cGhHd0M5TlNhRU1nREM1bkNwMzA2Myt2bDVzbXFMTjhWSGpwcUZKWGxaemhO?= =?utf-8?B?TVZWNlhhaGpmaUlhZ0hZZmNIUHVPdzZnVmcxM0ZRSlVwN0tFZUdvQkF1RFYv?= =?utf-8?B?ZVJZQnp6Z1U5OXNYbWtyMHpTRGdXeVB3R3VsT2QwQ2VkL0wvS0pNMlBWYUVR?= =?utf-8?B?T3A4RTRIQjJyd3JUaitRRHJSc09KVVU3T0lyMjZNU2pOeVpXaWR5V2xLeVB4?= =?utf-8?B?S3ZXY1Y3bTVvNEZwSi9aWEVLMVRFUHZvZnNmbitYQ3ExSDM5WUFHWi9nWkZp?= =?utf-8?B?WUhXQzg3SHkvWFJtRkUycDZISS94cmYxcXdMYnhDWncyc1RSaFluMWJDcFJP?= =?utf-8?B?cER1amovaWJZYW1TUE9hUGZ5cDM0UUI4VGhUSW1QeDlkaGNmOWZjczBiWHlk?= =?utf-8?B?MTV2ajZWOThjUkN4U0RMNGJLV2pOMitXNTRjaFZQcFNUU1dpTXcwSUJoQklR?= =?utf-8?B?N0xYWnV0bHZEekdnUldMa1RlYjVmODd3WlFMVEVjc0M5ZjUyOFZtUFJ5dkxk?= =?utf-8?B?VXMwZy95N0FQSFhhYnoreERkNkJZWTN1cm9INjZNQlRyejFSanJQdXFUM1ox?= =?utf-8?B?VWRhN1psR2owR1VFWFVJZFRGeERKMDllalczaGxlRUIyZjZ6ZFkvcGYyUVF5?= =?utf-8?B?aTRnbHJaeklSak9LZjR0cmY2ckRFZjczUVNJOTk2WkJSOVR2REhZNXQwTzls?= =?utf-8?B?Z0REMzVBUkRkZ3BqZVRtVWVRUXdUcjdBci9TR2hkK3RWNzRzZlNaN2pWd1B1?= =?utf-8?B?MlJEaG9tc2lvakJPRUs1TG1MWXZUaTNyL0tVT1ByeDVlS1BzWTZHR2t6cW5n?= =?utf-8?B?bEFFQ0svUWlRVkdaVDIveU4rY0dtbG9UL1JQZXpoNUUydGI4YVkzWWlFMHJY?= =?utf-8?B?SVlNZ1BJNVlINmtwMlNLaElMcjBOTGVHeklxYUtYdjFTMG8wMTk2bFptZXg3?= =?utf-8?Q?t9+nctpyIJg=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(376014)(36860700013)(82310400026)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 08:02:04.7860 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 03b709c5-656f-42f2-c0b5-08dcf338fbf5 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF00028D0A.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DUZPR07MB9959 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add lcore variables programmer's guide. This guide gives both an overview of the API, its implementation, and alternatives to the use of lcore variables for maintaining per-lcore id data. It has pictures, too. Signed-off-by: Mattias Rönnblom --- .../prog_guide/img/lcore_var_mem_layout.svg | 310 ++++++++++ .../img/static_array_mem_layout.svg | 278 +++++++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/lcore_var.rst | 548 ++++++++++++++++++ 4 files changed, 1137 insertions(+) create mode 100644 doc/guides/prog_guide/img/lcore_var_mem_layout.svg create mode 100644 doc/guides/prog_guide/img/static_array_mem_layout.svg create mode 100644 doc/guides/prog_guide/lcore_var.rst diff --git a/doc/guides/prog_guide/img/lcore_var_mem_layout.svg b/doc/guides/prog_guide/img/lcore_var_mem_layout.svg new file mode 100644 index 0000000000..ebb4fa2431 --- /dev/null +++ b/doc/guides/prog_guide/img/lcore_var_mem_layout.svg @@ -0,0 +1,310 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 0 + + 1 + + 2 + + 3 + + 4 + + 5 + + 6 + + 7 + + 0 + + int a + + char b + + <padding> + + 8 + + long c + + 16 + + long d + + 24 + + <unallocated> + + 32 + + 40 + + 48 + + 56 + + 64 + + int a + + char b + + <padding> + + 72 + + long c + + 80 + + long d + + 88 + + <unallocated> + + 96 + + 104 + + 112 + + 120 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + struct x_lcore + + + + + + + + + + + + lcore id 0 + + + + + + + + + + + + struct y_lcore + + + + + + #define RTE_MAX_LCORE 2#define RTE_MAX_LCORE_VAR 64 + + + + + + + + + + + + lcore id 1 + + + + + + + + + + + + struct x_lcore + + + + + + + + + + + + struct y_lcore + + + + + + + + + + + + struct lcore_var_buffer.data + + + + + + Handle pointers:x_lcores y_lcores + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc/guides/prog_guide/img/static_array_mem_layout.svg b/doc/guides/prog_guide/img/static_array_mem_layout.svg new file mode 100644 index 0000000000..ed8bead826 --- /dev/null +++ b/doc/guides/prog_guide/img/static_array_mem_layout.svg @@ -0,0 +1,278 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 0 + + 1 + + 2 + + 3 + + 4 + + 5 + + 6 + + 7 + + 0 + + int a + + char b + + <padding> + + 8 + + __rte_cache_aligned <padding> + + 16 + + 24 + + 32 + + 40 + + 48 + + 56 + + 64 + + RTE_CACHE_GUARD <padding> + + 72 + + 80 + + 88 + + 96 + + 104 + + 112 + + 120 + + 128 + + int a + + char b + + <padding> + + 136 + + __rte_cache_aligned <padding> + + 144 + + 152 + + 160 + + 168 + + 176 + + 184 + + 192 + + RTE_CACHE_GUARD <padding> + + 200 + + 208 + + 216 + + 224 + + 232 + + 240 + + 248 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + struct x_lcorelcore id 0 + + + + + + + + + + + + struct x_lcore x_lcores[RTE_MAX_LCORE] + + + + + + + + + + + + struct x_lcorelcore id 1 + + + + \ No newline at end of file diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 7eb1a98d88..c4432c4b74 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -27,6 +27,7 @@ Memory Management mempool_lib mbuf_lib multi_proc_support + lcore_var CPU Management diff --git a/doc/guides/prog_guide/lcore_var.rst b/doc/guides/prog_guide/lcore_var.rst new file mode 100644 index 0000000000..b647ba7391 --- /dev/null +++ b/doc/guides/prog_guide/lcore_var.rst @@ -0,0 +1,548 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2024 Ericsson AB + +Lcore Variables +=============== + +The ``rte_lcore_var.h`` API provides a mechanism to allocate and +access per-lcore id variables in a space- and cycle-efficient manner. + +Lcore Variables API +------------------- + +A per-lcore id variable (or lcore variable for short) holds a unique +value for each EAL thread and registered non-EAL thread. Thus, there +is one distinct value for each past, current and future lcore +id-equipped thread, with a total of ``RTE_MAX_LCORE`` instances. + +The value of the lcore variable for one lcore id is independent of the +values associated with other lcore ids within the same variable. + +For detailed information on the lcore variables API, please refer to +the ``rte_lcore_var.h`` API documentation. + +Lcore Variable Handle +^^^^^^^^^^^^^^^^^^^^^ + +To allocate and access an lcore variable's values, a *handle* is +used. The handle is represented by an opaque pointer, only to be +dereferenced using the appropriate ```` macros. + +The handle is a pointer to the value's type (e.g., for an ``uint32_t`` +lcore variable, the handle is a ``uint32_t *``). + +The reason the handle is typed (i.e., it's not a void pointer or an +integer) is to enable type checking when accessing values of the lcore +variable. + +A handle may be passed between modules and threads just like any other +pointer. + +A valid (i.e., allocated) handle never has the value NULL. Thus, a +handle set to NULL may be used to signify that allocation has not yet +been done. + +Lcore Variable Allocation +^^^^^^^^^^^^^^^^^^^^^^^^^ + +An lcore variable is created in two steps: + +1. Define an lcore variable handle by using ``RTE_LCORE_VAR_HANDLE``. +2. Allocate lcore variable storage and initialize the handle by using + ``RTE_LCORE_VAR_ALLOC`` or ``RTE_LCORE_VAR_INIT``. Allocation + generally occurs at the time of module initialization, but may be + done at any time. + +The lifetime of an lcore variable is not tied to the thread that +created it. + +Each lcore variable has ``RTE_MAX_LCORE`` values, one for each +possible lcore id. All of an lcore variable's values may be accessed +from the moment the lcore variable is created, throughout the lifetime +of the EAL (i.e., until ``rte_eal_cleanup()``). + +Lcore variables do not need to be freed and cannot be freed. + +Access +^^^^^^ + +The value of any lcore variable for any lcore id may be accessed from +any thread (including unregistered threads), but it should only be +*frequently* read from or written to by the *owner*. A thread is +considered the owner of a particular lcore variable value instance if +it has the lcore id associated with that instance. + +Non-owner accesses results in *false sharing*. As long as non-owner +accesses are rare, they will have only a very slight effect on +performance. This property of lcore variables memory organization is +intentional. See the implementation section for more information. + +Values of the same lcore variable, associated with different lcore ids +may be frequently read or written by their respective owners without +risking false sharing. + +An appropriate synchronization mechanism, such as atomic load and +stores, should be employed to prevent data races between the owning +thread and any other thread accessing the same value instance. + +The value of the lcore variable for a particular lcore id is accessed +via ``RTE_LCORE_VAR_LCORE``. + +A common pattern is for an EAL thread or a registered non-EAL +thread to access its own lcore variable value. For this purpose, a +shorthand exists as ``RTE_LCORE_VAR``. + +The handle, defined by ``RTE_LCORE_VAR_HANDLE``, is a pointer of the +same type as the value, but it must be treated as an opaque identifier +and cannot be directly dereferenced. + +Lcore variable handles and value pointers may be freely passed +between different threads. + +Storage +^^^^^^^ + +An lcore variable's values may be of a primitive type like ``int``, +but is typically a ``struct``. + +The lcore variable handle introduces a per-variable (not +per-value/per-lcore id) overhead of ``sizeof(void *)`` bytes, so there +are some memory footprint gains to be made by organizing all per-lcore +id data for a particular module as one lcore variable (e.g., as a +struct). + +An application may define an lcore variable handle without ever +allocating the lcore variable. + +The size of an lcore variable's value cannot exceed the DPDK +build-time constant ``RTE_MAX_LCORE_VAR``. An lcore variable's size is +the size of one of its value instance, not the aggregate of all its +``RTE_MAX_LCORE`` instances. + +Lcore variables should generally *not* be ``__rte_cache_aligned`` and +need *not* include a ``RTE_CACHE_GUARD`` field, since these constructs +are designed to avoid false sharing. With lcore variables, false +sharing is largely avoided by other means. In the case of an lcore +variable instance, the thread most recently accessing nearby data +structures should almost always be the lcore variable's owner. Adding +padding (e.g., with ``RTE_CACHE_GUARD``) will increase the effective +memory working set size, potentially reducing performance. + +Lcore variable values are initialized to zero by default. + +Lcore variables are not stored in huge page memory. + +Example +^^^^^^^ + +Below is an example of the use of an lcore variable: + +.. code-block:: c + + struct foo_lcore_state { + int a; + long b; + }; + + static RTE_LCORE_VAR_HANDLE(struct foo_lcore_state, lcore_states); + + long foo_get_a_plus_b(void) + { + const struct foo_lcore_state *state = RTE_LCORE_VAR(lcore_states); + + return state->a + state->b; + } + + RTE_INIT(rte_foo_init) + { + RTE_LCORE_VAR_ALLOC(lcore_states); + + unsigned int lcore_id; + struct foo_lcore_state *state; + RTE_LCORE_VAR_FOREACH(lcore_id, state, lcore_states) { + /* initialize state */ + } + + /* other initialization */ + } + + +Implementation +-------------- + +This section gives an overview of the implementation of lcore +variables, and some background to its design. + +Lcore Variable Buffers +^^^^^^^^^^^^^^^^^^^^^^ + +Lcore variable values are kept in a set of ``lcore_var_buffer`` structs. + +.. code-block:: c + + struct lcore_var_buffer { + char data[RTE_MAX_LCORE_VAR * RTE_MAX_LCORE]; + struct lcore_var_buffer *prev; + }; + +An lcore var buffer stores at a minimum one, but usually many, lcore +variables. + +The value instances for all lcore ids are stored in the same +buffer. However, each lcore id has its own slice of the ``data`` +array. Such a slice is ``RTE_MAX_LCORE_VAR`` bytes in size. + +In this way, the values associated with a particular lcore id are +grouped spatially close (in memory). No padding is required to prevent +false sharing. + +.. code-block:: c + + static struct lcore_var_buffer *current_buffer; + + /* initialized to trigger buffer allocation on first allocation */ + static size_t offset = RTE_MAX_LCORE_VAR; + +The implementation maintains a current ``lcore_var_buffer`` and +an ``offset``, where the latter tracks how many bytes of this +current buffer has been allocated. + +The ``offset`` is progressively incremented (by the size of the +just-allocated lcore variable), as lcore variables are being +allocated. + +If the allocation of a variable would result in an ``offset`` larger +than ``RTE_MAX_LCORE_VAR`` (i.e., the slice size), the buffer is +full. In that case, new buffer is allocated off the heap, and the +``offset`` is reset. + +The lcore var buffers are arranged in a link list, to allow freeing +them at the point of ``rte_eal_cleanup()``, thereby avoiding false +positives from tools like valgrind memcheck. + +The lcore variable buffers are allocated off the regular C heap. There +are a number of reasons for not using ```` and huge +pages for lcore variables: + +- The libc heap is available at any time, including early in the + DPDK initialization. +- The amount of data kept in lcore variables is projected to be small, + and thus is unlikely to induce translate lookaside buffer (TLB) + misses. +- The last (and potentially only) lcore buffer in the chain will + likely only partially be in use. Huge pages of the sort used by DPDK + are always resident in memory, and their use would result in a + significant amount of memory going to waste. An example: ~256 kB + worth of lcore variables are allocated by DPDK libraries, PMDs and + the application. ``RTE_MAX_LCORE_VAR`` is set to 1 MB and + ``RTE_MAX_LCORE`` to 128. With 4 kB OS pages, only the first ~64 + pages of each of the 128 per-lcore id slices in the (only) + ``lcore_var_buffer`` will actually be resident (paged in). Here, + demand paging saves ~98 MB of memory. + +Not residing in huge pages, lcore variables cannot be accessed from +secondary processes. + +Heap allocation failures are treated as fatal. The reason for this +unorthodox design is that a majority of the allocations are deemed to +happen at initialization. An early heap allocation failure for a fixed +amount of data is a situation not unlike one where there is not enough +memory available for static variables (i.e., the BSS or data +sections). + +Provided these assumptions hold true, it's deemed acceptable to leave +the application out of handling memory allocation failures. + +The upside of this approach is that no error handling code is required +on the API user side. + +Lcore Variable Handles +^^^^^^^^^^^^^^^^^^^^^^ + +Upon lcore variable allocation, the lcore variables API returns an +opaque *handle* in the form of a pointer. The value of the pointer is +``buffer->data + offset``. + +Translating a handle base pointer to a pointer to a value associated +with a particular lcore id is straightforward: + +.. code-block:: c + + static inline void * + rte_lcore_var_lcore(unsigned int lcore_id, void *handle) + { + return RTE_PTR_ADD(handle, lcore_id * RTE_MAX_LCORE_VAR); + } + +``RTE_MAX_LCORE_VAR`` is a public macro to allow the compiler to +optimize the ``lcore_id * RTE_MAX_LCORE_VAR`` expression, and replace +the multiplication with a less expensive arithmetic operation. + +To maintain type safety, the ``RTE_LCORE_VAR*()`` macros should be +used, instead of directly invoking ``rte_lcore_var_lcore()``. The +macros return a pointer of the same type as the handle (i.e., a +pointer to the value's type). + +Memory Layout +^^^^^^^^^^^^^ + +This section describes how lcore variables are organized in memory. + +As an illustration, two example modules are used, ``rte_x`` and +``rte_y``, both maintaining per-lcore id state as a part of their +implementation. + +Two different methods will be used to maintain such state - lcore +variables and, to serve as a reference, lcore id-indexed static +arrays. + +Certain parameters are scaled down to make graphical depictions more +practical. + +For the purpose of this exercise, a ``RTE_MAX_LCORE`` of 2 is +assumed. In a real-world configuration the maximum number of EAL +threads and registered threads will be much greater (e.g., 128). + +The lcore variables example assumes a ``RTE_MAX_LCORE_VAR`` of 64. In +a real-world configuration (as controlled by ``rte_config.h``) the +value of this compile-time constant will be much greater (e.g., +1048576). + +The per-lcore id state is also smaller than what most real-world +modules would have. + +Lcore Variables Example +""""""""""""""""""""""" + +When lcore variables are used, the parts of ``rte_x`` and ``rte_y`` +that deal with the declaration and allocation of per-lcore id data may +look something like below. + +.. code-block:: c + + /* -- Lcore variables -- */ + + /* rte_x.c */ + + struct x_lcore + { + int a; + char b; + }; + + static RTE_LCORE_VAR_HANDLE(struct x_lcore, x_lcores); + RTE_LCORE_VAR_INIT(x_lcores); + + /../ + + /* rte_y.c */ + + struct y_lcore + { + long c; + long d; + }; + + static RTE_LCORE_VAR_HANDLE(struct y_lcore, y_lcores); + RTE_LCORE_VAR_INIT(y_lcores); + + /../ + +The resulting memory layout will look something like the following: + +.. _figure_lcore_var_mem_layout: + +.. figure:: img/lcore_var_mem_layout.* + +The above figure assumes that ``x_lcores`` is allocated prior to +``y_lcores``. ``RTE_LCORE_VAR_INIT()`` relies constructors, run prior +to ``main()`` in an undefined order. + +The use of lcore variables ensures that per-lcore id data is kept in +close proximity, within a designated region of memory. This proximity +enhances data locality and can improve performance. + +Lcore Id Index Static Array Example +""""""""""""""""""""""""""""""""""" + +Below is an example of the struct declarations, declarations and the +resulting organization in memory in case an lcore id indexed static +array of cache-line aligned, RTE_CACHE_GUARDed structs are used to +maintain per-lcore id state. + +This is a common pattern in DPDK, which lcore variables attempts to +replace. + +.. code-block:: c + + /* -- Cache-aligned static arrays -- */ + + /* rte_x.c */ + + struct x_lcore + { + int a; + char b; + RTE_CACHE_GUARD; + } __rte_cache_aligned; + + static struct x_lcore x_lcores[RTE_MAX_LCORE]; + + /../ + + /* rte_y.c */ + + struct y_lcore + { + long c; + long d; + RTE_CACHE_GUARD; + } __rte_cache_aligned; + + static struct y_lcore y_lcores[RTE_MAX_LCORE]; + + /../ + +In this approach, accessing the state for a particular lcore id is +merely a matter retrieving the lcore id and looking up the correct +struct instance. + +.. code-block:: c + + struct x_lcore *my_lcore_state = &x_lcores[rte_lcore_id()]; + +The address "0" at the top of the left-most column in the figure +represent the base address for the ``x_lcores`` array (in the BSS +segment in memory). + +The figure only includes the memory layout for the ``rte_x`` example +module. ``rte_y`` would look very similar, with ``y_lcores`` being +located at some other address in the BSS section. + +.. _figure_static_array_mem_layout: + +.. figure:: img/static_array_mem_layout.* + +The static array approach results in the per-lcore id being organized +around modules, not lcore ids. To avoid false sharing, an extensive +use of padding is employed, causing cache fragmentation. + +Because the padding is interspersed with the data, demand paging is +unlikely to reduce the actual resident DRAM memory footprint. This is +because the padding is smaller than a typical operating system memory +page (usually 4 kB). + +Performance +^^^^^^^^^^^ + +One of the goals of lcore variables is to improve performance. This is +achieved by packing often-used data in fewer cache lines, and thus +reducing fragmentation in CPU caches and thus somewhat improving the +effective cache size and cache hit rates. + +The application-level gains depends much on how much data is kept in +lcore variables, and how often it is accessed, and how much pressure +the application asserts on the CPU caches (i.e., how much other memory +it accesses). + +The ``lcore_var_perf_autotest`` is an attempt at exploring the +performance benefits (or drawbacks) of lcore variables compared to its +alternatives. Being a micro benchmark, it needs to be taken with a +grain of salt. + +Generally, one shouldn't expect more than some very modest gains in +performance after a switch from lcore id indexed arrays to lcore +variables. + +An additional benefit of the use of lcore variables is that it avoids +certain tricky issues related to CPU core hardware prefetching (e.g., +next-N-lines prefetching) that may cause false sharing even when data +used by two cores do not reside on the same cache line. Hardware +prefetch behavior is generally not publicly documented and varies +across CPU vendors, CPU generations and BIOS (or similar) +configurations. For applications aiming to be portable, this may cause +issues. Often, CPU hardware prefetch-induced issues are non-existent, +except some particular circumstances, where their adverse effects may +be significant. + +Alternatives +------------ + +Lcore Id Indexed Static Arrays +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lcore variables are designed to replace a pattern exemplified below: + +.. code-block:: c + + struct __rte_cache_aligned foo_lcore_state { + int a; + long b; + RTE_CACHE_GUARD; + }; + + static struct foo_lcore_state lcore_states[RTE_MAX_LCORE]; + +This scheme is simple and effective, but has one drawback: the data is +organized so that objects related to all lcores for a particular +module are kept close in memory. At a bare minimum, this requires +sizing data structures (e.g., using ``__rte_cache_aligned``) to an +even number of cache lines and ensuring that allocation of such +objects are cache line aligned to avoid false sharing. With CPU +hardware prefetching and memory loads resulting from speculative +execution (functions which seemingly are getting more eager faster +than they are getting more intelligent), one or more "guard" cache +lines may be required to separate one lcore's data from another's and +prevent false sharing. + +Lcore variables offer the advantage of working with, rather than +against, the CPU's assumptions. A next-line hardware prefetcher, +for example, may function as intended (i.e., to the benefit, not +detriment, of system performance). + +Thread Local Storage +^^^^^^^^^^^^^^^^^^^^ + +An alternative to ``rte_lcore_var.h`` is the ``rte_per_lcore.h`` API, +which makes use of thread-local storage (TLS, e.g., GCC ``__thread`` or +C11 ``_Thread_local``). + +The are a number of differences between using TLS and the use of lcore +variables. + +The lifecycle of a thread-local variable instance is tied to that of +the thread. The data cannot be accessed before the thread has been +created, nor after it has terminated. As a result, thread-local +variables must be initialized in a "lazy" manner (e.g., at the point +of thread creation). Lcore variables may be accessed immediately after +having been allocated (which may occur before any thread beyond the +main thread is running). + +A thread-local variable is duplicated across all threads in the +process, including unregistered non-EAL threads (i.e., "regular" +threads). For DPDK applications heavily relying on multi-threading (in +conjunction to DPDK's "one thread per core" pattern), either by having +many concurrent threads or creating/destroying threads at a high rate, +an excessive use of thread-local variables may cause inefficiencies +(e.g., increased thread creation overhead due to thread-local storage +initialization or increased memory footprint). Lcore variables *only* +exist for threads with an lcore id. + +Whether data in thread-local storage can be shared between threads +(i.e., whether a pointer to a thread-local variable can be passed to +and successfully dereferenced by a non-owning thread) depends on the +specifics of the TLS implementation. With GCC __thread and GCC +_Thread_local, data sharing between threads is supported. In the C11 +standard, accessing another thread's _Thread_local object is +implementation-defined. Lcore variable instances may be accessed +reliably by any thread. + +Lcore variables also relies on TLS to retrieve the thread's +lcore id. However, the rest of the per-thread data is not kept in TLS. + +From a memory layout perspective, TLS is similar to lcore variables, +and thus per-thread data structure need not be padded. + +In case the above-mentioned drawbacks of the use of TLS is of no +significance to a particular application, TLS is a good alternative to +lcore variables. -- 2.43.0