From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-eopbgr30079.outbound.protection.outlook.com [40.107.3.79]) by dpdk.org (Postfix) with ESMTP id C3539288C for ; Fri, 18 Jan 2019 07:48:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SnITcdbVnMBstLh7xg70pSe8i/MLGYmB8Y8Bl+7lhL4=; b=poILzuPECVzfszSgIyLE0FMiKrzOHv6pTBMBspiRtdCGFoeR7ZbmnK9716b7Hf59DZ9LfcuRTzsDOhiiFZesQgOJxAbJ9tY057rVNNV27tSh49hUSXUQbj3VGT/JrMsNXnzCEoYniWZdkvRYmFTyN6y67KhH3xA1Ab9UHoTBQaI= Received: from AM6PR08MB3672.eurprd08.prod.outlook.com (20.177.115.76) by AM6PR08MB4487.eurprd08.prod.outlook.com (20.179.18.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1516.13; Fri, 18 Jan 2019 06:48:47 +0000 Received: from AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::25ec:2db7:d268:2b7b]) by AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::25ec:2db7:d268:2b7b%2]) with mapi id 15.20.1537.018; Fri, 18 Jan 2019 06:48:47 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "dev@dpdk.org" , "stephen@networkplumber.org" , "paulmck@linux.ibm.com" CC: "Gavin Hu (Arm Technology China)" , Dharmik Thakkar , nd , nd Thread-Topic: [RFC v2 1/2] rcu: add RCU library supporting QSBR mechanism Thread-Index: AQHUmZwkA3lN9JDuXEiclvOGJBGOtKWwUixggACXpdCAAUkpAIACgqOw Date: Fri, 18 Jan 2019 06:48:46 +0000 Message-ID: References: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> <20181222021420.5114-1-honnappa.nagarahalli@arm.com> <20181222021420.5114-2-honnappa.nagarahalli@arm.com> <2601191342CEEE43887BDE71AB977258010D904212@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB977258010D904AC7@irsmsx105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB977258010D904AC7@irsmsx105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.103.75] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM6PR08MB4487; 6:wh/dhCZNYfYDigdYGbrNTgbZftjirzeDp12gkoX/1TgMWA0R+8Snit7H4CixDkJ9P9U8n4Obhqn3u8bnwjFa6b4dT3igHRvrSLx3c9Y5Cjm/NaYowvC6Uihstk9qltf/8CixxIcSNwsPRUTRlloXyO+QF1QsQ+xJXa+VniqGwchXIHUkyzR1tEXL7sHS74P8rCvoPor7Tlqb6zu5fal+OXHwFLjMwX7NzeEgTYd+Rd0Mw3PShxxa6nXvaRuLOaSivKTKA8k+40BZmC2+OxMU/UpG5GoM2WwiUI4IFJ+JZErQS+VnnzYVRKvCRrzpryiBBGPm33dhdYE9+c+LABMWJyPWQWOxhQDULRhRBqZ85VHPR/Z98gYcin0K3zogUE2uEP7nuwsBAGyQz7xQZxC8/w1FIdvtUVyZeSR4AQdcypuZfWy9HAqGPMtcGXQhu4E3z3q4EW4JmR4m4jIY4jzcHg==; 5:QzlyhdSM4+15ECJCm8FhlCGi/gsNwGVo09oEdJc7chYkJN0xbV6JziwtKt51/VIDioZ0jPve4P/FQ+Ej3C+W+V0noeDyelSBNqlmRPEp/dfHTxQpWt3eaPtJMDjnrh1JHLHPhv8PogNGdU68WAy2WdHkq06oCm5y2OlXXGANstFNBp+ZSzzQtWKpYtSBlqf5fV3rV40JHO3eUlLot+OCyA==; 7:an23oPDl22D6vTkIVut+2w3z8lHDvC3SElqBDax83S7tYEq7nIu2Dv8C5edoMagD99zLJRNLwtm62WTouVxFjH9T/yeK5cn6HvFNgar5bVfv2NwgpRxFlfroQ6zV8e4SDGI2EkRG6nBPf8VIcwKozQ== x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: 4fc53a29-4186-47c1-d658-08d67d10fefc x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM6PR08MB4487; x-ms-traffictypediagnostic: AM6PR08MB4487: nodisclaimer: True x-microsoft-antispam-prvs: x-forefront-prvs: 0921D55E4F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(39860400002)(396003)(376002)(366004)(136003)(199004)(189003)(76176011)(5660300001)(54906003)(8676002)(71190400001)(81166006)(6116002)(3846002)(316002)(345774005)(110136005)(2201001)(305945005)(478600001)(86362001)(71200400001)(106356001)(105586002)(8936002)(7736002)(81156014)(74316002)(68736007)(256004)(6246003)(55016002)(229853002)(97736004)(186003)(6506007)(93886005)(4326008)(53936002)(33656002)(25786009)(102836004)(446003)(11346002)(486006)(2501003)(26005)(7696005)(72206003)(66066001)(2906002)(99286004)(6436002)(9686003)(476003)(14454004); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR08MB4487; H:AM6PR08MB3672.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: OonDE+PV8KRHYWFURsDG+on+CGgrOsC8VZzQgA8L7NotVTRPJG9NrSbAQhpV2YYg1gU8naDezy6Rs7qtA1vovu5UmCoWKgZBOz1Z3mR7fRAa/0FI4VXRPpGj9yH4QQ7WhkF12Z/xVJOXVVot4S7cyl9qZXQzZcsvFWKuHnXffk0WUT2fKLC8K6tYvOBvisdhEmIHer+mLRYRjmEMsQ+jL/PyfIeHawaOnaaOIVEgRIxIF0GASPYXapvrNXhCmyBrP/xQjp6Qx6qlber+dW+qTy1qfj81UI5A0vXryVPT0IdxEljlMcFdC6xNs27TphMSnoCnGpA41OAGMzOtYb+PpAJL7Yik8//ph2C0QYD3+wlth7yqd1qXyAPdI+vxNT0YdJ3QjGM2JD5BNW+wzU6p8k8tBesUt2b99i9SnTrj5EI= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4fc53a29-4186-47c1-d658-08d67d10fefc X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jan 2019 06:48:47.0035 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4487 Subject: Re: [dpdk-dev] [RFC v2 1/2] rcu: add RCU library supporting QSBR mechanism X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jan 2019 06:48:48 -0000 >=20 > > > ... > > > > > > > diff --git a/lib/librte_rcu/rte_rcu_qsbr.h > > > > b/lib/librte_rcu/rte_rcu_qsbr.h new file mode 100644 index > > > > 000000000..c818e77fd > > > > --- /dev/null > > > > +++ b/lib/librte_rcu/rte_rcu_qsbr.h > > > > @@ -0,0 +1,321 @@ > > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > > + * Copyright (c) 2018 Arm Limited */ > > > > + > > > > +#ifndef _RTE_RCU_QSBR_H_ > > > > +#define _RTE_RCU_QSBR_H_ > > > > + > > > > +/** > > > > + * @file > > > > + * RTE Quiescent State Based Reclamation (QSBR) > > > > + * > > > > + * Quiescent State (QS) is any point in the thread execution > > > > + * where the thread does not hold a reference to shared memory. > > > > + * A critical section for a data structure can be a quiescent > > > > +state for > > > > + * another data structure. > > > > + * > > > > + * This library provides the ability to identify quiescent state. > > > > + */ > > > > + > > > > +#ifdef __cplusplus > > > > +extern "C" { > > > > +#endif > > > > + > > > > +#include > > > > +#include > > > > +#include > > > > +#include > > > > +#include > > > > +#include > > > > +#include > > > > + > > > > +/**< Maximum number of reader threads supported. */ #define > > > > +RTE_RCU_MAX_THREADS 128 > > > > + > > > > +#if !RTE_IS_POWER_OF_2(RTE_RCU_MAX_THREADS) > > > > +#error RTE_RCU_MAX_THREADS must be a power of 2 #endif > > > > + > > > > +/**< Number of array elements required for the bit-map */ #define > > > > +RTE_QSBR_BIT_MAP_ELEMS > (RTE_RCU_MAX_THREADS/(sizeof(uint64_t) > > > * 8)) > > > > + > > > > +/* Thread IDs are stored as a bitmap of 64b element array. Given > > > > +thread id > > > > + * needs to be converted to index into the array and the id > > > > +within > > > > + * the array element. > > > > + */ > > > > +#define RTE_QSBR_THR_INDEX_SHIFT 6 #define > RTE_QSBR_THR_ID_MASK > > > > +0x3f > > > > + > > > > +/* Worker thread counter */ > > > > +struct rte_rcu_qsbr_cnt { > > > > + uint64_t cnt; /**< Quiescent state counter. */ } > > > > +__rte_cache_aligned; > > > > + > > > > +/** > > > > + * RTE thread Quiescent State structure. > > > > + */ > > > > +struct rte_rcu_qsbr { > > > > + uint64_t reg_thread_id[RTE_QSBR_BIT_MAP_ELEMS] > > > __rte_cache_aligned; > > > > + /**< Registered reader thread IDs - reader threads reporting > > > > + * on this QS variable represented in a bit map. > > > > + */ > > > > + > > > > + uint64_t token __rte_cache_aligned; > > > > + /**< Counter to allow for multiple simultaneous QS queries */ > > > > + > > > > + struct rte_rcu_qsbr_cnt w[RTE_RCU_MAX_THREADS] > > > __rte_cache_aligned; > > > > + /**< QS counter for each reader thread, counts upto > > > > + * current value of token. > > > > > > As I understand you decided to stick with neutral thread_id and let > > > user define what exactly thread_id is (lcore, syste, thread id, somet= hing > else)? > > Yes, that is correct. I will reply to the other thread to continue the = discussion. > > > > > If so, can you probably get rid of RTE_RCU_MAX_THREADS limitation? > > I am not seeing this as a limitation. The user can change this if requi= red. May > be I should change it as follows: > > #ifndef RTE_RCU_MAX_THREADS > > #define RTE_RCU_MAX_THREADS 128 > > #endif >=20 > Yep, that's better, though it would still require user to rebuild the cod= e if he > would like to increase total number of threads supported. Agree > Though it seems relatively simply to extend current code to support dynam= ic > max thread num here (2 variable arrays plus shift value plus mask). Agree, supporting dynamic 'max thread num' is simple. But this means memory= needs to be allocated to the arrays. The API 'rte_rcu_qsbr_init' has to ta= ke max thread num as the parameter. We also have to introduce another API t= o free this memory. This will become very similar to alloc/free APIs I had = in the v1. I hope I am following you well, please correct me if not. >=20 > > > > > I.E. struct rte_rcu_qsbr_cnt w[] and allow user at init time to > > > define max number of threads allowed. > > > Or something like: > > > #define RTE_RCU_QSBR_DEF(name, max_thread) struct name { \ > > > uint64_t reg_thread_id[ALIGN_CEIL(max_thread, 64) >> 6]; \ > > > ... > > > struct rte_rcu_qsbr_cnt w[max_thread]; \ } > > I am trying to understand this. I am not following why 'name' is > > required? Would the user call 'RTE_RCU_QSBR_DEF' in the application > header file? >=20 > My thought here was to allow user to define his own structures, depending= on > the number of max threads he needs/wants: > RTE_RCU_QSBR_DEF(rte_rcu_qsbr_128, 128); > RTE_RCU_QSBR_DEF(rte_rcu_qsbr_64, 64); ... Thank you for the clarification, I follow you now. However, it will not sol= ve the problem of dynamic max thread num. Changes to the max number of thre= ads will require recompilation. > Konstantin