From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140077.outbound.protection.outlook.com [40.107.14.77]) by dpdk.org (Postfix) with ESMTP id 929AE343C for ; Thu, 24 Jan 2019 18:15:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TzOt2WsMdEDHqh7ysoVQ6KPJh7V81w+xNH7SYLNlAdg=; b=WfJdKKgirUybmPoTWj+Dh+o5ABjZsl9Nk3OeHA+Xr3ke2fGr8MqIfFB/UWFhj3xdzhImUNvBvZHjlRyKXyZ8Vb/AMrTFnTCgTWxnR52UuykSXmHaXQkQKcUGp+bQER/3gtTVns8jz8SW416koiu0oFe1kkBmStDNsDmolISnsm0= Received: from AM6PR08MB3672.eurprd08.prod.outlook.com (20.177.115.76) by AM6PR08MB3110.eurprd08.prod.outlook.com (52.135.163.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1558.16; Thu, 24 Jan 2019 17:15:24 +0000 Received: from AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::9120:87d6:b17c:fadd]) by AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::9120:87d6:b17c:fadd%3]) with mapi id 15.20.1558.016; Thu, 24 Jan 2019 17:15:24 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "dev@dpdk.org" , "stephen@networkplumber.org" , "paulmck@linux.ibm.com" CC: "Gavin Hu (Arm Technology China)" , Dharmik Thakkar , nd , nd Thread-Topic: [RFC v2 1/2] rcu: add RCU library supporting QSBR mechanism Thread-Index: AQHUmZwkA3lN9JDuXEiclvOGJBGOtKWwUixggACXpdCAAUkpAIACgqOwgABi02CACb/TYA== Date: Thu, 24 Jan 2019 17:15:24 +0000 Message-ID: References: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> <20181222021420.5114-1-honnappa.nagarahalli@arm.com> <20181222021420.5114-2-honnappa.nagarahalli@arm.com> <2601191342CEEE43887BDE71AB977258010D904212@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB977258010D904AC7@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB977258010D9058F6@irsmsx105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB977258010D9058F6@irsmsx105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM6PR08MB3110; 6:fbWUx63pHcZYywcBkuR08lSJ7MjSAMsMiLCpQ57iV7WAQwqoOLxXatuu8dly0fuMufLPy2NeRlvv217rCYsj11EYognid12jrwmlnRhqpD0ClCYIPctLQtSOrBeXRt/VtqbTF4tNf2JA4rzLMad0Gge8IQS0bvdeIAdFSuio0MQGcn1lHuiRFMKJS0xkST2xO3fjYTyKbZv8XZNDR1g9m21IhxIIFvWYvNjIDDMbOy3G+DSk/7Fb5Bca4liNK5KDPkrmg3k29f0sLEJN+O5GwKAtTs/VKi/dc5Aq5pfTK4VFhNKrn1n6/sQ3h9m19U3pSDAgimggVI5C8B4wURs49t7MzI782SeDw+p/bZ6//dD11ax6UPY/0jClUPWoxI3vfmJLWygCOy4M9xjHTnOwgZvASifIZ8F4r00JHwutkiHh0Nn/LeTzvREK7IPB2GOMkOrpdWbjZikCJHCpdYUVcg==; 5:jqM2vwryaoywg2BVpmeElCEkFXLch4/pNcAOnivOXiagF1qwabk5t4ClRY9ruUlAGW65hraOMUWlqLe9ofB7kHsm8GZ9uZsKzWj7JGMDOxcoD9rlFPD4SDaPmwsW78A+fAIRXy/64wuYIGUGh0N+JOmPNKQbPSRx3xfhzUCHwe6m1vUN7/GtlVTMxCM682EGbR8zKhI9QsFyml0La298BA==; 7:xdQRMgEld2e6l4f8foqD4GVVKDymAd5r+zyZa24c7Tt+RgURO3wo+2F5uWwCF0NdYiwXWUh5CE68VcJRLpbJPMmgTv3nAlpP7+iGiO8tztdKQKyfonMkb76YwC/L0E3zXntt7ya9G5S5lA4hI5gmXw== x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: 0bee353e-013f-4508-12ec-08d6821f8722 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600110)(711020)(4605077)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM6PR08MB3110; x-ms-traffictypediagnostic: AM6PR08MB3110: nodisclaimer: True x-microsoft-antispam-prvs: x-forefront-prvs: 0927AA37C7 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(366004)(396003)(376002)(39860400002)(346002)(199004)(189003)(7696005)(74316002)(33656002)(99286004)(6436002)(106356001)(55016002)(9686003)(72206003)(7736002)(305945005)(478600001)(3846002)(6116002)(71200400001)(11346002)(110136005)(26005)(76176011)(476003)(71190400001)(54906003)(186003)(486006)(102836004)(2501003)(316002)(6506007)(345774005)(446003)(93886005)(68736007)(14444005)(105586002)(2201001)(8936002)(66066001)(8676002)(81166006)(256004)(81156014)(6246003)(2906002)(14454004)(53936002)(4326008)(86362001)(25786009)(97736004)(229853002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR08MB3110; H:AM6PR08MB3672.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: VGR/X1Fwws5kmgq1tP4Y3gtywK+aIoSsE6ier7IIXayvchzb9/3JsmMz4rdsuuWtd8crjbJMeudCYaf5Sgy9PmSp2unrJPOsO6mQDg1KQUUulGz0HfJNvrsrfl9vxIwc2mDJZmY/mLr5dqqOzrBY7QG5yuZdKtTnSXiS5iFSegukp1FInC/19ahTucp8CfKsmzEQJVph6JsIpx88a6qxlP2hnSKQgMjsywhhlkv9e3DI3p8GjYC9Rqc1OPNqoaUfw24h4JXmQSMqoGFpZRDXW4uNfLcFoxDzbewIl2jTabkSXoGevkHCK21S0KbYsaqW4ftXehqHGIQz7qcZLf2SLLQrRxlWaQHEj0LJcU8bVE9IGFMbzc5bfbZOrTDD5lgnvZ3qRglY+hBQCBnWoS3wGUpixi4oPYEzvIyJ6epIbuY= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0bee353e-013f-4508-12ec-08d6821f8722 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jan 2019 17:15:24.1476 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3110 Subject: Re: [dpdk-dev] [RFC v2 1/2] rcu: add RCU library supporting QSBR mechanism X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2019 17:15:26 -0000 > > > > > > +/** > > > > > > + * RTE thread Quiescent State structure. > > > > > > + */ > > > > > > +struct rte_rcu_qsbr { > > > > > > + uint64_t reg_thread_id[RTE_QSBR_BIT_MAP_ELEMS] > > > > > __rte_cache_aligned; > > > > > > + /**< Registered reader thread IDs - reader threads reporting > > > > > > + * on this QS variable represented in a bit map. > > > > > > + */ > > > > > > + > > > > > > + uint64_t token __rte_cache_aligned; > > > > > > + /**< Counter to allow for multiple simultaneous QS queries > > > > > > +*/ > > > > > > + > > > > > > + struct rte_rcu_qsbr_cnt w[RTE_RCU_MAX_THREADS] > > > > > __rte_cache_aligned; > > > > > > + /**< QS counter for each reader thread, counts upto > > > > > > + * current value of token. > > > > > > > > > > As I understand you decided to stick with neutral thread_id and > > > > > let user define what exactly thread_id is (lcore, syste, thread > > > > > id, something > > > else)? > > > > Yes, that is correct. I will reply to the other thread to continue = the > discussion. > > > > > > > > > If so, can you probably get rid of RTE_RCU_MAX_THREADS limitation= ? > > > > I am not seeing this as a limitation. The user can change this if > > > > required. May > > > be I should change it as follows: > > > > #ifndef RTE_RCU_MAX_THREADS > > > > #define RTE_RCU_MAX_THREADS 128 > > > > #endif > > > > > > Yep, that's better, though it would still require user to rebuild > > > the code if he would like to increase total number of threads support= ed. > > Agree > > > > > Though it seems relatively simply to extend current code to support > > > dynamic max thread num here (2 variable arrays plus shift value plus > mask). > > Agree, supporting dynamic 'max thread num' is simple. But this means > > memory needs to be allocated to the arrays. The API > > 'rte_rcu_qsbr_init' has to take max thread num as the parameter. We als= o > have to introduce another API to free this memory. This will become very > similar to alloc/free APIs I had in the v1. > > I hope I am following you well, please correct me if not. >=20 > I think we can still leave alloc/free tasks to the user. > We probabply just need extra function rte_rcu_qsbr_size(uint32_ > max_threads) to help user calculate required size. > rte_rcu_qsbr_init() might take as an additional parameter 'size' to make > checks. The size is returned by an API provided by the library. Why does it need to= be validated again? If 'size' is required for rte_rcu_qsbr_init, it could = calculate it again. > Thought about something like that: >=20 > size_t sz =3D rte_rcu_qsbr_size(max_threads); struct rte_rcu_qsbr *qsbr = =3D > alloc_aligned(CACHE_LINE, sz); rte_rcu_qsbr_init(qsbr, max_threads, sz); = ... >=20 Do you see any advantage for allowing the user to allocate the memory? This approach requires the user to call 3 APIs (including memory allocation= ). These 3 can be abstracted in a rte_rcu_qsbr_alloc API, user has to call = just 1 API. > Konstantin >=20 > > > > > > > > > > > > > > I.E. struct rte_rcu_qsbr_cnt w[] and allow user at init time to > > > > > define max number of threads allowed. > > > > > Or something like: > > > > > #define RTE_RCU_QSBR_DEF(name, max_thread) struct name { \ > > > > > uint64_t reg_thread_id[ALIGN_CEIL(max_thread, 64) >> 6]; \ > > > > > ... > > > > > struct rte_rcu_qsbr_cnt w[max_thread]; \ } > > > > I am trying to understand this. I am not following why 'name' is > > > > required? Would the user call 'RTE_RCU_QSBR_DEF' in the > > > > application > > > header file? > > > > > > My thought here was to allow user to define his own structures, > > > depending on the number of max threads he needs/wants: > > > RTE_RCU_QSBR_DEF(rte_rcu_qsbr_128, 128); > > > RTE_RCU_QSBR_DEF(rte_rcu_qsbr_64, 64); ... > > Thank you for the clarification, I follow you now. However, it will > > not solve the problem of dynamic max thread num. Changes to the max > number of threads will require recompilation. > > > > > Konstantin