From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D90A445BD4; Fri, 25 Oct 2024 10:51:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D1B3A4065F; Fri, 25 Oct 2024 10:51:10 +0200 (CEST) Received: from EUR02-VI1-obe.outbound.protection.outlook.com (mail-vi1eur02on2070.outbound.protection.outlook.com [40.107.241.70]) by mails.dpdk.org (Postfix) with ESMTP id F23E6402CF for ; Fri, 25 Oct 2024 10:50:55 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lC6Zq9/HeDPtMr2deitc+KIs2tj2Xo7Sj/nNEC4GAOvlnp69s4CbHDva06ud7MSKOUxQFG0cFS51OTKks8/NZ2l209kLyBLj+tYkLl1CKAQdKwOPzj0kQWSoi+JsUBWDzhe+YjaA3FfnYQetOtinR6dly1KKc1lQoutzjnkQjlFC0UBgQ+bqVLoNvKkLxGufil6p3wbGuHJQs4HLHQbMDPxDIqn3wSG3+KDn7iQjCgAvA40ffJ12uPcj1y9ujgbCoKzrnDFBR697M6ntl9OVNcI5/mq7cEsi5+VKxas5rpIYiljc0WvP7Afef7lbIKLFiGstZZULcfw2dAHEJev3yQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dQnD+JpUua1Wt9PmLpBYbHLNuGQIjDuyayp0IQu7ECo=; b=vjAaW8Ae796yO2FiHZVbVLsQNWjOthQN1CIo4w6yoQXPld+G8FVTcH55LAADDFORPRcHWy71MG0SlxOTWEYNzXAP24BB7Ka3H2ozzcGZy9goS0byznKa9cA1Tz7k1zpiodEIOGymzn7IhWcnyXgV9GFpm7q9ZyOINxyZeHCe+8uBtlvOluO31oA7GV5nbuC1DMAHpdga4fIkEfsYmClV6XCVeRokEBsUlMNUDS11yhfl9NFBOZIfgABBp2imvkkB/dIffilaJ0eym9qfvb3IH5DOj2Wjc8CwgTK8F3Bg7kDBurY7f1bf4jGjABgmm24VB/dVa0c2/C0iVJz2QY7zHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dQnD+JpUua1Wt9PmLpBYbHLNuGQIjDuyayp0IQu7ECo=; b=R09DqLFV8EAHe/58AMGOLGIWV/TjFOt7uq0TPKe2rDvsjM2fXflxUnmlRAgyRk1aBgufpJkHmizmdUo+mtB0nVSkhsHdJbFxae9leHbiabJCb2BRp3pCdXUCFGike+WScSRWTavlTNIwIeL0LOqMe942tDwwClwa3TGtBXIO64PLTjczTMYzYq/5P4JqxsvaMiE8cA/l5u/P92X+ZTaNCYSJvQX9MBXE6/b8byJj5eo4YOG4AWiB9PigbxPofq6fmisfON5uliar0YT1PK4KvoBLvL3EeVVGavA0Zaudy0caKQ6B8ja0bR1b35MHuVl8ZkmRoLCMkjNwzAA1rzEFEg== Received: from AM0PR08CA0002.eurprd08.prod.outlook.com (2603:10a6:208:d2::15) by AS2PR07MB9403.eurprd07.prod.outlook.com (2603:10a6:20b:643::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.21; Fri, 25 Oct 2024 08:50:52 +0000 Received: from AM3PEPF0000A795.eurprd04.prod.outlook.com (2603:10a6:208:d2:cafe::dd) by AM0PR08CA0002.outlook.office365.com (2603:10a6:208:d2::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.21 via Frontend Transport; Fri, 25 Oct 2024 08:50:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM3PEPF0000A795.mail.protection.outlook.com (10.167.16.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.14 via Frontend Transport; Fri, 25 Oct 2024 08:50:51 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.64) with Microsoft SMTP Server id 15.2.1544.11; Fri, 25 Oct 2024 10:50:50 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id F19531C00AA; Fri, 25 Oct 2024 10:50:49 +0200 (CEST) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: CC: , =?UTF-8?q?Morten=20Br=C3=B8rup?= , Stephen Hemminger , Konstantin Ananyev , David Marchand , Jerin Jacob , Luka Jankovic , Thomas Monjalon , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= Subject: [PATCH v17 4/8] eal: add lcore variables' programmer's guide Date: Fri, 25 Oct 2024 10:41:45 +0200 Message-ID: <20241025084149.873037-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241025084149.873037-1-mattias.ronnblom@ericsson.com> References: <20241023075302.869008-1-mattias.ronnblom@ericsson.com> <20241025084149.873037-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM3PEPF0000A795:EE_|AS2PR07MB9403:EE_ X-MS-Office365-Filtering-Correlation-Id: 2d2743e8-278b-4aca-17f8-08dcf4d22165 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|36860700013|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?TjJ6NUpRUVhuRU05SFJLbWl2aEVsekU2Y3dBQ1RhOWZJVEF2US94dTdGcXhn?= =?utf-8?B?NDJrTGxTWU10UU1ub2plYlFidG9MT3lNbmdINWI0SWZSaWVFc0ZGUnBkc21l?= =?utf-8?B?OTVWbXVpQVV2RVhzYXZlYmtnZ21IbC9tTTFZWFpXRWdvUkZ2ZU14ZmVZVXpq?= =?utf-8?B?WEpEVDBPN1pYdC9vTkxnMUxyb0JvaW5oSHdFUHBuT215a05ycVZBWjZxQ2NI?= =?utf-8?B?S000WlJCWkNYSW42dTU1Q1NZSmFrY0tIMVNSWVo3bHdHbkxTVjJ0dVdtYVRx?= =?utf-8?B?cCswSm12bkZuelFBa201dHFMUHpHcFhlRzJ4aTJjaHpMVzJpbFhFT1VqTm9q?= =?utf-8?B?d2lhTnZpakJFQjBiSjRxdVJaTnZtcjluYTFjcHpEdDV4eDZOVVdTM3FveEx3?= =?utf-8?B?NVBJTWQ2N1N1UFR3WFNia2RpUDJMblo2QkFqRkpOOEFWaDdvRzhKeTJoVy9u?= =?utf-8?B?dmxDU2JIcldWdEZ0OTlDOGJWY2J1dVdOUkI5YUZreGZtWXI0WEQrVnlsOVd1?= =?utf-8?B?aHk4ZlYvcmVNQXJydmdBUHBpVkoxV00yNXZOODBUUlkzV1ZpWGwweURpU0hq?= =?utf-8?B?ZTJvekhNcEFWODZLYlEyOW1FVU5ORDVMMzNhelNQd0dKQjlmM3FCb2NpYnJt?= =?utf-8?B?SUFXMXlwd1FmMW9uK2Zmb3c3a253NXV6NzVnaEx1YytkLy81ZGJnR2dRYXBO?= =?utf-8?B?Vmw5bG9ZeHFRYUpxLzhXMGdhVWVra2tvZWVzMXhhZktSYXlveXFKbmRzYVJJ?= =?utf-8?B?dVJJcWZOcGN4ZkhaWUtnUTh2SWFzVURaTFdYWVpWY1NQQnpwNys2NmJOWlFJ?= =?utf-8?B?OTEyRDRlODhIMXdFenRpeFZyQ0lpMGh4RkZ0VW5xSitNV3lSSmVSRXRmMFU5?= =?utf-8?B?OTlONStSRmNhSG81dDdIQUlEZnN5NU9PT2g1UEtwREJEU1hqSVJTMnVaa01h?= =?utf-8?B?LzFaaTlzNXFpaUVPSnBMdFdtUFRZWjZiMUdqMDNRbGRpZXVJSGpUcU1LQ1dk?= =?utf-8?B?L0QvMlpKcGVQY2R1ZVVuZnRUSlpnaDg2K2k4WVVJMjZFdkduWm5QbEVPbFdp?= =?utf-8?B?dGU1ZmIrL1daWjFpNTk5WUQyMk10bFZTN1JNVisvQVZaSzdvOVA3RmVEN0tq?= =?utf-8?B?aWNUVDZVaFBqK2oxeDhZcStUSzI0c25HMEo4c1M5OFZRS1BzaWFOK3M5Y203?= =?utf-8?B?N0ZoSkdUVHBJSzFrakdoL2plNzZHL21qamJCeE13eFl5QVNtRzMyVVpVY0xX?= =?utf-8?B?SitiVDVNczFObmRYckJ4TVZDYzUwSDFMenF2aTdJaU5yUGVsN3VtOXRtRGcv?= =?utf-8?B?dnZacmpmWi9hTE1idEorbHVxL1ltWWNjZmx1YnRkaGhXUkJmaFNxRDcza2xk?= =?utf-8?B?aDdESnZ1cmU3RlBMOHAwNERMMEtWUFUxS2NBeWRNZFFVbGRobWxoWEQ5SHdj?= =?utf-8?B?c2hSVXNsZXJISXRtbFNqSk1wc29vOVVGYlRYVTZhV2JFb3R4ZWxRWFhQbkhi?= =?utf-8?B?Rm5BTU1GUXIvSkwwMWRJYlY3UUFjbVhqdkxjTDBIN3RSTHViRXpKeWxjeHR4?= =?utf-8?B?MmRvSURDMUh4RHMxVC9SNmZSd2UwMXRvM0JoYkg2bk5oRU1rWUlHRTkxemU3?= =?utf-8?B?dGk4bmtNOEl3SHhlRHA2U0JOTFRiSmtkb0hTRDNuaGMrMUc5L1lDVUNqcitL?= =?utf-8?B?Wk9JcnQ2N0IzR0hFejdQdTU0Z3REeitrR0JDYU1tZHhETzVUMjVvQzYwT3Zy?= =?utf-8?B?RGpQSy8vMDBwMjMwbXhlS3MwSnVVTUdxMXpvaEs3TExtejhsTXVINHhGT0lR?= =?utf-8?B?cFZ1VTVZSTVFeGJ3b3VwQjcxNi9hMWxEd1ZuMkhHZWxoNDR0T29TMGNmdXRG?= =?utf-8?B?MVAxbVFWaE5FYkFRcFBxMGc1bGxtL0Y2OTJNeXB3TkQwUDFNSC96bjhkbkZh?= =?utf-8?Q?1ckV41aeXhY=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(376014)(36860700013)(82310400026)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2024 08:50:51.9881 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2d2743e8-278b-4aca-17f8-08dcf4d22165 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF0000A795.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR07MB9403 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add lcore variables programmer's guide. This guide gives both an overview of the API, its implementation, and alternatives to the use of lcore variables for maintaining per-lcore id data. It has pictures, too. Signed-off-by: Mattias Rönnblom Reviewed-By: Luka Jankovic -- PATCH v17: * Change the color used for padding in the diagram to improve contrast. (Luka Jankovic) * Mention ``RTE_LCORE_VAR_FOREACH``. (Luka Jankovic) --- .../prog_guide/img/lcore_var_mem_layout.svg | 310 ++++++++++ .../img/static_array_mem_layout.svg | 278 +++++++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/lcore_var.rst | 551 ++++++++++++++++++ 4 files changed, 1140 insertions(+) create mode 100644 doc/guides/prog_guide/img/lcore_var_mem_layout.svg create mode 100644 doc/guides/prog_guide/img/static_array_mem_layout.svg create mode 100644 doc/guides/prog_guide/lcore_var.rst diff --git a/doc/guides/prog_guide/img/lcore_var_mem_layout.svg b/doc/guides/prog_guide/img/lcore_var_mem_layout.svg new file mode 100644 index 0000000000..c4b286316c --- /dev/null +++ b/doc/guides/prog_guide/img/lcore_var_mem_layout.svg @@ -0,0 +1,310 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 0 + + 1 + + 2 + + 3 + + 4 + + 5 + + 6 + + 7 + + 0 + + int a + + char b + + <padding> + + 8 + + long c + + 16 + + long d + + 24 + + <unallocated> + + 32 + + 40 + + 48 + + 56 + + 64 + + int a + + char b + + <padding> + + 72 + + long c + + 80 + + long d + + 88 + + <unallocated> + + 96 + + 104 + + 112 + + 120 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + struct x_lcore + + + + + + + + + + + + lcore id 0 + + + + + + + + + + + + struct y_lcore + + + + + + #define RTE_MAX_LCORE 2#define RTE_MAX_LCORE_VAR 64 + + + + + + + + + + + + lcore id 1 + + + + + + + + + + + + struct x_lcore + + + + + + + + + + + + struct y_lcore + + + + + + + + + + + + struct lcore_var_buffer.data + + + + + + Handle pointers:x_lcores y_lcores + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc/guides/prog_guide/img/static_array_mem_layout.svg b/doc/guides/prog_guide/img/static_array_mem_layout.svg new file mode 100644 index 0000000000..87aa5b26f5 --- /dev/null +++ b/doc/guides/prog_guide/img/static_array_mem_layout.svg @@ -0,0 +1,278 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 0 + + 1 + + 2 + + 3 + + 4 + + 5 + + 6 + + 7 + + 0 + + int a + + char b + + <padding> + + 8 + + __rte_cache_aligned <padding> + + 16 + + 24 + + 32 + + 40 + + 48 + + 56 + + 64 + + RTE_CACHE_GUARD <padding> + + 72 + + 80 + + 88 + + 96 + + 104 + + 112 + + 120 + + 128 + + int a + + char b + + <padding> + + 136 + + __rte_cache_aligned <padding> + + 144 + + 152 + + 160 + + 168 + + 176 + + 184 + + 192 + + RTE_CACHE_GUARD <padding> + + 200 + + 208 + + 216 + + 224 + + 232 + + 240 + + 248 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + struct x_lcorelcore id 0 + + + + + + + + + + + + struct x_lcore x_lcores[RTE_MAX_LCORE] + + + + + + + + + + + + struct x_lcorelcore id 1 + + + + \ No newline at end of file diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 7eb1a98d88..c4432c4b74 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -27,6 +27,7 @@ Memory Management mempool_lib mbuf_lib multi_proc_support + lcore_var CPU Management diff --git a/doc/guides/prog_guide/lcore_var.rst b/doc/guides/prog_guide/lcore_var.rst new file mode 100644 index 0000000000..d0558d23f4 --- /dev/null +++ b/doc/guides/prog_guide/lcore_var.rst @@ -0,0 +1,551 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2024 Ericsson AB + +Lcore Variables +=============== + +The ``rte_lcore_var.h`` API provides a mechanism to allocate and +access per-lcore id variables in a space- and cycle-efficient manner. + +Lcore Variables API +------------------- + +A per-lcore id variable (or lcore variable for short) holds a unique +value for each EAL thread and registered non-EAL thread. Thus, there +is one distinct value for each past, current and future lcore +id-equipped thread, with a total of ``RTE_MAX_LCORE`` instances. + +The value of the lcore variable for one lcore id is independent of the +values associated with other lcore ids within the same variable. + +For detailed information on the lcore variables API, please refer to +the ``rte_lcore_var.h`` API documentation. + +Lcore Variable Handle +^^^^^^^^^^^^^^^^^^^^^ + +To allocate and access an lcore variable's values, a *handle* is +used. The handle is represented by an opaque pointer, only to be +dereferenced using the appropriate ```` macros. + +The handle is a pointer to the value's type (e.g., for an ``uint32_t`` +lcore variable, the handle is a ``uint32_t *``). + +The reason the handle is typed (i.e., it's not a void pointer or an +integer) is to enable type checking when accessing values of the lcore +variable. + +A handle may be passed between modules and threads just like any other +pointer. + +A valid (i.e., allocated) handle never has the value NULL. Thus, a +handle set to NULL may be used to signify that allocation has not yet +been done. + +Lcore Variable Allocation +^^^^^^^^^^^^^^^^^^^^^^^^^ + +An lcore variable is created in two steps: + +1. Define an lcore variable handle by using ``RTE_LCORE_VAR_HANDLE``. +2. Allocate lcore variable storage and initialize the handle by using + ``RTE_LCORE_VAR_ALLOC`` or ``RTE_LCORE_VAR_INIT``. Allocation + generally occurs at the time of module initialization, but may be + done at any time. + +The lifetime of an lcore variable is not tied to the thread that +created it. + +Each lcore variable has ``RTE_MAX_LCORE`` values, one for each +possible lcore id. All of an lcore variable's values may be accessed +from the moment the lcore variable is created, throughout the lifetime +of the EAL (i.e., until ``rte_eal_cleanup()``). + +Lcore variables do not need to be freed and cannot be freed. + +Access +^^^^^^ + +The value of any lcore variable for any lcore id may be accessed from +any thread (including unregistered threads), but it should only be +*frequently* read from or written to by the *owner*. A thread is +considered the owner of a particular lcore variable value instance if +it has the lcore id associated with that instance. + +Non-owner accesses results in *false sharing*. As long as non-owner +accesses are rare, they will have only a very slight effect on +performance. This property of lcore variables memory organization is +intentional. See the implementation section for more information. + +Values of the same lcore variable, associated with different lcore ids +may be frequently read or written by their respective owners without +risking false sharing. + +An appropriate synchronization mechanism, such as atomic load and +stores, should be employed to prevent data races between the owning +thread and any other thread accessing the same value instance. + +The value of the lcore variable for a particular lcore id is accessed +via ``RTE_LCORE_VAR_LCORE``. + +A common pattern is for an EAL thread or a registered non-EAL +thread to access its own lcore variable value. For this purpose, a +shorthand exists as ``RTE_LCORE_VAR``. + +``RTE_LCORE_VAR_FOREACH`` may be used to iterate over all values of a +particular lcore variable. + +The handle, defined by ``RTE_LCORE_VAR_HANDLE``, is a pointer of the +same type as the value, but it must be treated as an opaque identifier +and cannot be directly dereferenced. + +Lcore variable handles and value pointers may be freely passed +between different threads. + +Storage +^^^^^^^ + +An lcore variable's values may be of a primitive type like ``int``, +but is typically a ``struct``. + +The lcore variable handle introduces a per-variable (not +per-value/per-lcore id) overhead of ``sizeof(void *)`` bytes, so there +are some memory footprint gains to be made by organizing all per-lcore +id data for a particular module as one lcore variable (e.g., as a +struct). + +An application may define an lcore variable handle without ever +allocating the lcore variable. + +The size of an lcore variable's value cannot exceed the DPDK +build-time constant ``RTE_MAX_LCORE_VAR``. An lcore variable's size is +the size of one of its value instance, not the aggregate of all its +``RTE_MAX_LCORE`` instances. + +Lcore variables should generally *not* be ``__rte_cache_aligned`` and +need *not* include a ``RTE_CACHE_GUARD`` field, since these constructs +are designed to avoid false sharing. With lcore variables, false +sharing is largely avoided by other means. In the case of an lcore +variable instance, the thread most recently accessing nearby data +structures should almost always be the lcore variable's owner. Adding +padding (e.g., with ``RTE_CACHE_GUARD``) will increase the effective +memory working set size, potentially reducing performance. + +Lcore variable values are initialized to zero by default. + +Lcore variables are not stored in huge page memory. + +Example +^^^^^^^ + +Below is an example of the use of an lcore variable: + +.. code-block:: c + + struct foo_lcore_state { + int a; + long b; + }; + + static RTE_LCORE_VAR_HANDLE(struct foo_lcore_state, lcore_states); + + long foo_get_a_plus_b(void) + { + const struct foo_lcore_state *state = RTE_LCORE_VAR(lcore_states); + + return state->a + state->b; + } + + RTE_INIT(rte_foo_init) + { + RTE_LCORE_VAR_ALLOC(lcore_states); + + unsigned int lcore_id; + struct foo_lcore_state *state; + RTE_LCORE_VAR_FOREACH(lcore_id, state, lcore_states) { + /* initialize state */ + } + + /* other initialization */ + } + + +Implementation +-------------- + +This section gives an overview of the implementation of lcore +variables, and some background to its design. + +Lcore Variable Buffers +^^^^^^^^^^^^^^^^^^^^^^ + +Lcore variable values are kept in a set of ``lcore_var_buffer`` structs. + +.. code-block:: c + + struct lcore_var_buffer { + char data[RTE_MAX_LCORE_VAR * RTE_MAX_LCORE]; + struct lcore_var_buffer *prev; + }; + +An lcore var buffer stores at a minimum one, but usually many, lcore +variables. + +The value instances for all lcore ids are stored in the same +buffer. However, each lcore id has its own slice of the ``data`` +array. Such a slice is ``RTE_MAX_LCORE_VAR`` bytes in size. + +In this way, the values associated with a particular lcore id are +grouped spatially close (in memory). No padding is required to prevent +false sharing. + +.. code-block:: c + + static struct lcore_var_buffer *current_buffer; + + /* initialized to trigger buffer allocation on first allocation */ + static size_t offset = RTE_MAX_LCORE_VAR; + +The implementation maintains a current ``lcore_var_buffer`` and +an ``offset``, where the latter tracks how many bytes of this +current buffer has been allocated. + +The ``offset`` is progressively incremented (by the size of the +just-allocated lcore variable), as lcore variables are being +allocated. + +If the allocation of a variable would result in an ``offset`` larger +than ``RTE_MAX_LCORE_VAR`` (i.e., the slice size), the buffer is +full. In that case, new buffer is allocated off the heap, and the +``offset`` is reset. + +The lcore var buffers are arranged in a link list, to allow freeing +them at the point of ``rte_eal_cleanup()``, thereby avoiding false +positives from tools like valgrind memcheck. + +The lcore variable buffers are allocated off the regular C heap. There +are a number of reasons for not using ```` and huge +pages for lcore variables: + +- The libc heap is available at any time, including early in the + DPDK initialization. +- The amount of data kept in lcore variables is projected to be small, + and thus is unlikely to induce translate lookaside buffer (TLB) + misses. +- The last (and potentially only) lcore buffer in the chain will + likely only partially be in use. Huge pages of the sort used by DPDK + are always resident in memory, and their use would result in a + significant amount of memory going to waste. An example: ~256 kB + worth of lcore variables are allocated by DPDK libraries, PMDs and + the application. ``RTE_MAX_LCORE_VAR`` is set to 1 MB and + ``RTE_MAX_LCORE`` to 128. With 4 kB OS pages, only the first ~64 + pages of each of the 128 per-lcore id slices in the (only) + ``lcore_var_buffer`` will actually be resident (paged in). Here, + demand paging saves ~98 MB of memory. + +Not residing in huge pages, lcore variables cannot be accessed from +secondary processes. + +Heap allocation failures are treated as fatal. The reason for this +unorthodox design is that a majority of the allocations are deemed to +happen at initialization. An early heap allocation failure for a fixed +amount of data is a situation not unlike one where there is not enough +memory available for static variables (i.e., the BSS or data +sections). + +Provided these assumptions hold true, it's deemed acceptable to leave +the application out of handling memory allocation failures. + +The upside of this approach is that no error handling code is required +on the API user side. + +Lcore Variable Handles +^^^^^^^^^^^^^^^^^^^^^^ + +Upon lcore variable allocation, the lcore variables API returns an +opaque *handle* in the form of a pointer. The value of the pointer is +``buffer->data + offset``. + +Translating a handle base pointer to a pointer to a value associated +with a particular lcore id is straightforward: + +.. code-block:: c + + static inline void * + rte_lcore_var_lcore(unsigned int lcore_id, void *handle) + { + return RTE_PTR_ADD(handle, lcore_id * RTE_MAX_LCORE_VAR); + } + +``RTE_MAX_LCORE_VAR`` is a public macro to allow the compiler to +optimize the ``lcore_id * RTE_MAX_LCORE_VAR`` expression, and replace +the multiplication with a less expensive arithmetic operation. + +To maintain type safety, the ``RTE_LCORE_VAR*()`` macros should be +used, instead of directly invoking ``rte_lcore_var_lcore()``. The +macros return a pointer of the same type as the handle (i.e., a +pointer to the value's type). + +Memory Layout +^^^^^^^^^^^^^ + +This section describes how lcore variables are organized in memory. + +As an illustration, two example modules are used, ``rte_x`` and +``rte_y``, both maintaining per-lcore id state as a part of their +implementation. + +Two different methods will be used to maintain such state - lcore +variables and, to serve as a reference, lcore id-indexed static +arrays. + +Certain parameters are scaled down to make graphical depictions more +practical. + +For the purpose of this exercise, a ``RTE_MAX_LCORE`` of 2 is +assumed. In a real-world configuration the maximum number of EAL +threads and registered threads will be much greater (e.g., 128). + +The lcore variables example assumes a ``RTE_MAX_LCORE_VAR`` of 64. In +a real-world configuration (as controlled by ``rte_config.h``) the +value of this compile-time constant will be much greater (e.g., +1048576). + +The per-lcore id state is also smaller than what most real-world +modules would have. + +Lcore Variables Example +""""""""""""""""""""""" + +When lcore variables are used, the parts of ``rte_x`` and ``rte_y`` +that deal with the declaration and allocation of per-lcore id data may +look something like below. + +.. code-block:: c + + /* -- Lcore variables -- */ + + /* rte_x.c */ + + struct x_lcore + { + int a; + char b; + }; + + static RTE_LCORE_VAR_HANDLE(struct x_lcore, x_lcores); + RTE_LCORE_VAR_INIT(x_lcores); + + /../ + + /* rte_y.c */ + + struct y_lcore + { + long c; + long d; + }; + + static RTE_LCORE_VAR_HANDLE(struct y_lcore, y_lcores); + RTE_LCORE_VAR_INIT(y_lcores); + + /../ + +The resulting memory layout will look something like the following: + +.. _figure_lcore_var_mem_layout: + +.. figure:: img/lcore_var_mem_layout.* + +The above figure assumes that ``x_lcores`` is allocated prior to +``y_lcores``. ``RTE_LCORE_VAR_INIT()`` relies constructors, run prior +to ``main()`` in an undefined order. + +The use of lcore variables ensures that per-lcore id data is kept in +close proximity, within a designated region of memory. This proximity +enhances data locality and can improve performance. + +Lcore Id Index Static Array Example +""""""""""""""""""""""""""""""""""" + +Below is an example of the struct declarations, declarations and the +resulting organization in memory in case an lcore id indexed static +array of cache-line aligned, RTE_CACHE_GUARDed structs are used to +maintain per-lcore id state. + +This is a common pattern in DPDK, which lcore variables attempts to +replace. + +.. code-block:: c + + /* -- Cache-aligned static arrays -- */ + + /* rte_x.c */ + + struct x_lcore + { + int a; + char b; + RTE_CACHE_GUARD; + } __rte_cache_aligned; + + static struct x_lcore x_lcores[RTE_MAX_LCORE]; + + /../ + + /* rte_y.c */ + + struct y_lcore + { + long c; + long d; + RTE_CACHE_GUARD; + } __rte_cache_aligned; + + static struct y_lcore y_lcores[RTE_MAX_LCORE]; + + /../ + +In this approach, accessing the state for a particular lcore id is +merely a matter retrieving the lcore id and looking up the correct +struct instance. + +.. code-block:: c + + struct x_lcore *my_lcore_state = &x_lcores[rte_lcore_id()]; + +The address "0" at the top of the left-most column in the figure +represent the base address for the ``x_lcores`` array (in the BSS +segment in memory). + +The figure only includes the memory layout for the ``rte_x`` example +module. ``rte_y`` would look very similar, with ``y_lcores`` being +located at some other address in the BSS section. + +.. _figure_static_array_mem_layout: + +.. figure:: img/static_array_mem_layout.* + +The static array approach results in the per-lcore id being organized +around modules, not lcore ids. To avoid false sharing, an extensive +use of padding is employed, causing cache fragmentation. + +Because the padding is interspersed with the data, demand paging is +unlikely to reduce the actual resident DRAM memory footprint. This is +because the padding is smaller than a typical operating system memory +page (usually 4 kB). + +Performance +^^^^^^^^^^^ + +One of the goals of lcore variables is to improve performance. This is +achieved by packing often-used data in fewer cache lines, and thus +reducing fragmentation in CPU caches and thus somewhat improving the +effective cache size and cache hit rates. + +The application-level gains depends much on how much data is kept in +lcore variables, and how often it is accessed, and how much pressure +the application asserts on the CPU caches (i.e., how much other memory +it accesses). + +The ``lcore_var_perf_autotest`` is an attempt at exploring the +performance benefits (or drawbacks) of lcore variables compared to its +alternatives. Being a micro benchmark, it needs to be taken with a +grain of salt. + +Generally, one shouldn't expect more than some very modest gains in +performance after a switch from lcore id indexed arrays to lcore +variables. + +An additional benefit of the use of lcore variables is that it avoids +certain tricky issues related to CPU core hardware prefetching (e.g., +next-N-lines prefetching) that may cause false sharing even when data +used by two cores do not reside on the same cache line. Hardware +prefetch behavior is generally not publicly documented and varies +across CPU vendors, CPU generations and BIOS (or similar) +configurations. For applications aiming to be portable, this may cause +issues. Often, CPU hardware prefetch-induced issues are non-existent, +except some particular circumstances, where their adverse effects may +be significant. + +Alternatives +------------ + +Lcore Id Indexed Static Arrays +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lcore variables are designed to replace a pattern exemplified below: + +.. code-block:: c + + struct __rte_cache_aligned foo_lcore_state { + int a; + long b; + RTE_CACHE_GUARD; + }; + + static struct foo_lcore_state lcore_states[RTE_MAX_LCORE]; + +This scheme is simple and effective, but has one drawback: the data is +organized so that objects related to all lcores for a particular +module are kept close in memory. At a bare minimum, this requires +sizing data structures (e.g., using ``__rte_cache_aligned``) to an +even number of cache lines and ensuring that allocation of such +objects are cache line aligned to avoid false sharing. With CPU +hardware prefetching and memory loads resulting from speculative +execution (functions which seemingly are getting more eager faster +than they are getting more intelligent), one or more "guard" cache +lines may be required to separate one lcore's data from another's and +prevent false sharing. + +Lcore variables offer the advantage of working with, rather than +against, the CPU's assumptions. A next-line hardware prefetcher, +for example, may function as intended (i.e., to the benefit, not +detriment, of system performance). + +Thread Local Storage +^^^^^^^^^^^^^^^^^^^^ + +An alternative to ``rte_lcore_var.h`` is the ``rte_per_lcore.h`` API, +which makes use of thread-local storage (TLS, e.g., GCC ``__thread`` or +C11 ``_Thread_local``). + +The are a number of differences between using TLS and the use of lcore +variables. + +The lifecycle of a thread-local variable instance is tied to that of +the thread. The data cannot be accessed before the thread has been +created, nor after it has terminated. As a result, thread-local +variables must be initialized in a "lazy" manner (e.g., at the point +of thread creation). Lcore variables may be accessed immediately after +having been allocated (which may occur before any thread beyond the +main thread is running). + +A thread-local variable is duplicated across all threads in the +process, including unregistered non-EAL threads (i.e., "regular" +threads). For DPDK applications heavily relying on multi-threading (in +conjunction to DPDK's "one thread per core" pattern), either by having +many concurrent threads or creating/destroying threads at a high rate, +an excessive use of thread-local variables may cause inefficiencies +(e.g., increased thread creation overhead due to thread-local storage +initialization or increased memory footprint). Lcore variables *only* +exist for threads with an lcore id. + +Whether data in thread-local storage can be shared between threads +(i.e., whether a pointer to a thread-local variable can be passed to +and successfully dereferenced by a non-owning thread) depends on the +specifics of the TLS implementation. With GCC __thread and GCC +_Thread_local, data sharing between threads is supported. In the C11 +standard, accessing another thread's _Thread_local object is +implementation-defined. Lcore variable instances may be accessed +reliably by any thread. + +Lcore variables also relies on TLS to retrieve the thread's +lcore id. However, the rest of the per-thread data is not kept in TLS. + +From a memory layout perspective, TLS is similar to lcore variables, +and thus per-thread data structure need not be padded. + +In case the above-mentioned drawbacks of the use of TLS is of no +significance to a particular application, TLS is a good alternative to +lcore variables. -- 2.43.0