From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B9C91A2F18 for ; Thu, 3 Oct 2019 08:42:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EA4AB1BFD9; Thu, 3 Oct 2019 08:42:41 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60070.outbound.protection.outlook.com [40.107.6.70]) by dpdk.org (Postfix) with ESMTP id 2B95A1BFD8 for ; Thu, 3 Oct 2019 08:42:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SjlKbi7EaXo3Xd434mcODfg5JLuFukLSM7M3OcDyleU=; b=mJhbBfeZDcSNTgwIc+CCbvl8vERwQgg6BnB1Me/mGUwozGj2Gl7I38IQ3LICzwnWIemebrhoNP2tMmAjOGSqwy4A4C2IdaO+cGAQaq1HWgn9rQG4obygVMD9nC8CsPKuGBe0WB896yB65Hs9mtAAQMp8aKeGNi/+dA882kIJxMc= Received: from DB6PR0802CA0026.eurprd08.prod.outlook.com (2603:10a6:4:a3::12) by AM7PR08MB5399.eurprd08.prod.outlook.com (2603:10a6:20b:109::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.17; Thu, 3 Oct 2019 06:42:38 +0000 Received: from VE1EUR03FT060.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::206) by DB6PR0802CA0026.outlook.office365.com (2603:10a6:4:a3::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2305.20 via Frontend Transport; Thu, 3 Oct 2019 06:42:38 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT060.mail.protection.outlook.com (10.152.19.187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2305.15 via Frontend Transport; Thu, 3 Oct 2019 06:42:36 +0000 Received: ("Tessian outbound 0939a6bab6b1:v33"); Thu, 03 Oct 2019 06:42:29 +0000 X-CR-MTA-TID: 64aa7808 Received: from 8fecdcd4870d.2 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.10.53]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 03A2C456-4E7E-4CF5-8EB7-3560861E8DBA.1; Thu, 03 Oct 2019 06:42:24 +0000 Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-db5eur03lp2053.outbound.protection.outlook.com [104.47.10.53]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8fecdcd4870d.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384); Thu, 03 Oct 2019 06:42:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WMg3evHG/ywNVj7090qiXUmZJopHtV6BybB3MAPqD+1ROjI+2bfs2/iRZmps7fqacSZWdLlj/ZYdzbJESTm+rzDryiLJ7f5YPiLbacS5CDLUg6GdDnSTalrs7N7BDTrTHeK4hYypNa8mxk1Nj88uA4CjhYFleMocYqKSGJuCsvqhFK3wa/wmLvUlXpLXw18jiGv6+m9Xnul2PoRozjs6QiZzV4bZn7BfhuOmvT9zMtPRN7Ar2jC7mXtOcAy4XTWsmWeHOvo3SlCKr5zCjziyW3CBNI4NoC8WV2xs2TDWPshI0IKiKzxYj84PamvVeyJHg4VI2kSviFxJH8h6YI1+hQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SjlKbi7EaXo3Xd434mcODfg5JLuFukLSM7M3OcDyleU=; b=ZJrQDBdNlg43gyEOOCojWx8TfzTz2w4iXqJ7NUqoS8niq8as2LNXHd4ASvmqBG4g9plNEvpfI6An+J/ym4CHpawB4Hnv0hg1Smt73GQjvyS+KmQJkjzwMgooPW8FHQe78T1ZIpGNIWpUGWEuPssAy2cshmCPsdt2wbxrdzZ9iqxMMUr8ojU0N5hfP1Sgyez/al0+saqW455zEfTRncqtQOSYjz/5/a6bDGcPX6/phoH5teNW3kru9A/riw427yHDkEnPihMxoFOLdLCNU62FIjhMH0XUhK8PYm3/PVa8WtMte3uZ7fvyLBrR66HSuCMUM8bcWENdi7IkzJOIdbu2XA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SjlKbi7EaXo3Xd434mcODfg5JLuFukLSM7M3OcDyleU=; b=mJhbBfeZDcSNTgwIc+CCbvl8vERwQgg6BnB1Me/mGUwozGj2Gl7I38IQ3LICzwnWIemebrhoNP2tMmAjOGSqwy4A4C2IdaO+cGAQaq1HWgn9rQG4obygVMD9nC8CsPKuGBe0WB896yB65Hs9mtAAQMp8aKeGNi/+dA882kIJxMc= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1PR08MB4893.eurprd08.prod.outlook.com (10.255.114.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Thu, 3 Oct 2019 06:42:22 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6%7]) with mapi id 15.20.2305.023; Thu, 3 Oct 2019 06:42:22 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "stephen@networkplumber.org" , "paulmck@linux.ibm.com" CC: "Wang, Yipeng1" , "Medvedkin, Vladimir" , "Ruifeng Wang (Arm Technology China)" , Dharmik Thakkar , Honnappa Nagarahalli , "dev@dpdk.org" , nd , nd Thread-Topic: [PATCH v3 2/3] lib/rcu: add resource reclamation APIs Thread-Index: AQHVeVI7HPAEoqcrDkuUAtadkcufoadIeADg Date: Thu, 3 Oct 2019 06:42:22 +0000 Message-ID: References: <20190906094534.36060-1-ruifeng.wang@arm.com> <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20191001062917.35578-3-honnappa.nagarahalli@arm.com> <2601191342CEEE43887BDE71AB9772580191970035@irsmsx105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB9772580191970035@irsmsx105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: c5efd326-511e-44eb-9b18-1efd38b6db50.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 979490cd-8436-4a3c-b84b-08d747cce106 X-MS-Office365-Filtering-HT: Tenant X-MS-TrafficTypeDiagnostic: VE1PR08MB4893:|VE1PR08MB4893:|AM7PR08MB5399: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882; x-forefront-prvs: 01792087B6 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(366004)(396003)(376002)(346002)(136003)(39860400002)(199004)(189003)(5660300002)(102836004)(74316002)(76176011)(54906003)(66066001)(9686003)(110136005)(7736002)(6506007)(305945005)(14454004)(71190400001)(71200400001)(99286004)(11346002)(486006)(26005)(316002)(476003)(33656002)(446003)(7696005)(478600001)(25786009)(186003)(14444005)(256004)(30864003)(2501003)(6246003)(81166006)(8676002)(81156014)(55016002)(229853002)(8936002)(52536014)(2201001)(6436002)(86362001)(4326008)(64756008)(2906002)(66556008)(76116006)(66946007)(66446008)(66476007)(3846002)(6116002); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4893; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: GFc8hMjHAq1rLwEEe9LZCRd1rPk/AAkwrUkshZ2QI1P5hz3TNR+dFZQmbv3uJMFLHBzCCZMYvHG20Gd89XyF1ZeWLjsHCo+5TwLgX/C6Js3B0OGwClvP2V+3UU5d6XpXlxMeQTH/RZq+iEOhe24J2WA5u8sCj+2wubhPCKtG/54AIBrSW/7gwymU9N88A63/siRnahAbfpYFi+2gvjLquZOr6kDpMTtuAPtZjxIl2DcNgxS9NwVMHFdeFi+Y4ULeJ1YGWA1a5/qMBjqjcU2OSpETUjHyZaTkPTxmFOAv2ZT3tKl4OQXCnmal0B52kALvCES2yMn92jdy4httiNOPkFJQnj06OlpnprQwdIy3TacJb6B0VP0R7q0hhQKSY0XPwCjwbv1zbkr0mUFlfJbQqjXxyKy9SPs5KstNxj93rUM= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4893 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT060.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(396003)(346002)(376002)(39860400002)(136003)(199004)(189003)(336012)(74316002)(305945005)(26005)(186003)(476003)(126002)(99286004)(102836004)(7696005)(55016002)(76176011)(486006)(97756001)(63350400001)(9686003)(446003)(11346002)(229853002)(6506007)(7736002)(46406003)(2501003)(70586007)(76130400001)(86362001)(2201001)(22756006)(66066001)(8676002)(47776003)(33656002)(70206006)(50466002)(14444005)(23726003)(26826003)(2906002)(478600001)(4326008)(81166006)(25786009)(6116002)(3846002)(52536014)(30864003)(5660300002)(356004)(316002)(54906003)(110136005)(14454004)(6246003)(36906005)(81156014)(8746002)(8936002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM7PR08MB5399; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 10389d79-709d-4013-a129-08d747ccd85d NoDisclaimer: True X-Forefront-PRVS: 01792087B6 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: M9U3kf1tmEZQAwqzlZkJrdqfd/ygsMSLLvsn64l6GfC4CoeINJnXurqKw1ypxEG9l0XOtDrGvLv+S3USWjOvmdioPzDB3yS5FqilCD+QPdc3kq5wCneZL3V0HHrb+Sd0sukrxBLJ8UPX33bksJ8pOUIvSqRcZ3QIzPzf+PLiYQl8zcU5oRCELjHDEOFQqLDKecUVd2La/wBJz4NudCRiCZTCmlaIXZw883YS0zIEFsTtu2aZ7e+TOWMtqJJgY0/eRL6KV0Z+400qm4lD1bH9lUSER6bqhooQghveAix71qvuRD6ZLRkZczjE04Yj6/r1URMZLVeQiRanJu64bO7hrJQp6WnF3MRB7PJJ2o9zPKCkP+Yg7hS8JF/EISiXs/uMAJZsfS0dibc9biaG0HoNwGNskql+UaAJbQRm9EXPXGM= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Oct 2019 06:42:36.8887 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 979490cd-8436-4a3c-b84b-08d747cce106 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5399 Subject: Re: [dpdk-dev] [PATCH v3 2/3] lib/rcu: add resource reclamation APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > > + > > +/* Reclaim resources from the defer queue. */ int > > +rte_rcu_qsbr_dq_reclaim(struct rte_rcu_qsbr_dq *dq) { > > + uint32_t max_cnt; > > + uint32_t cnt; > > + void *token; > > + uint64_t *tmp; > > + uint32_t i; > > + > > + if (dq =3D=3D NULL) { > > + rte_log(RTE_LOG_ERR, rte_rcu_log_type, > > + "%s(): Invalid input parameter\n", __func__); > > + rte_errno =3D EINVAL; > > + > > + return 1; > > + } > > + > > + /* Anything to reclaim? */ > > + if (rte_ring_count(dq->r) =3D=3D 0) > > + return 0; > > + > > + /* Reclaim at the max 1/16th the total number of entries. */ > > + max_cnt =3D dq->size >> RTE_RCU_QSBR_MAX_RECLAIM_LIMIT; > > + max_cnt =3D (max_cnt =3D=3D 0) ? dq->size : max_cnt; > > + cnt =3D 0; > > + > > + /* Check reader threads quiescent state and reclaim resources */ > > + while ((cnt < max_cnt) && (rte_ring_peek(dq->r, &token) =3D=3D 0) && > > + (rte_rcu_qsbr_check(dq->v, (uint64_t)((uintptr_t)token), false) >=20 > One more thing I forgot to ask - how this construct supposed to work on 3= 2 > bit machines? > peek() will return 32-bit value, while qsbr_check() operates with 64bit > tokens... > As I understand in that case you need to peek() 2 elems. Yes, that is the intention. Ring APIs with desired element size will help a= ddress the 32b machines. > Might work, but still think better to introduce serialize version of > ring_dequeue() See my other mail about re_ring_peek(). >=20 >=20 > > + =3D=3D 1)) { > > + (void)rte_ring_sc_dequeue(dq->r, &token); > > + /* The resource to dequeue needs to be a multiple of 64b > > + * due to the limitation of the rte_ring implementation. > > + */ > > + for (i =3D 0, tmp =3D (uint64_t *)dq->e; i < dq->esize/8; > > + i++, tmp++) > > + (void)rte_ring_sc_dequeue(dq->r, > > + (void *)(uintptr_t)tmp); > > + dq->f(dq->p, dq->e); > > + > > + cnt++; > > + } > > + > > + rte_log(RTE_LOG_INFO, rte_rcu_log_type, > > + "%s(): Reclaimed %u resources\n", __func__, cnt); > > + > > + if (cnt =3D=3D 0) { > > + /* No resources were reclaimed */ > > + rte_errno =3D EAGAIN; > > + return 1; > > + } > > + > > + return 0; > > +} > > + > > +/* Delete a defer queue. */ > > +int > > +rte_rcu_qsbr_dq_delete(struct rte_rcu_qsbr_dq *dq) { > > + if (dq =3D=3D NULL) { > > + rte_log(RTE_LOG_ERR, rte_rcu_log_type, > > + "%s(): Invalid input parameter\n", __func__); > > + rte_errno =3D EINVAL; > > + > > + return 1; > > + } > > + > > + /* Reclaim all the resources */ > > + if (rte_rcu_qsbr_dq_reclaim(dq) !=3D 0) > > + /* Error number is already set by the reclaim API */ > > + return 1; > > + > > + rte_ring_free(dq->r); > > + rte_free(dq); > > + > > + return 0; > > +} > > + > > int rte_rcu_log_type; > > > > RTE_INIT(rte_rcu_register) > > diff --git a/lib/librte_rcu/rte_rcu_qsbr.h > > b/lib/librte_rcu/rte_rcu_qsbr.h index c80f15c00..185d4b50a 100644 > > --- a/lib/librte_rcu/rte_rcu_qsbr.h > > +++ b/lib/librte_rcu/rte_rcu_qsbr.h > > @@ -34,6 +34,7 @@ extern "C" { > > #include > > #include > > #include > > +#include > > > > extern int rte_rcu_log_type; > > > > @@ -109,6 +110,67 @@ struct rte_rcu_qsbr { > > */ > > } __rte_cache_aligned; > > > > +/** > > + * Call back function called to free the resources. > > + * > > + * @param p > > + * Pointer provided while creating the defer queue > > + * @param e > > + * Pointer to the resource data stored on the defer queue > > + * > > + * @return > > + * None > > + */ > > +typedef void (*rte_rcu_qsbr_free_resource)(void *p, void *e); > > + > > +#define RTE_RCU_QSBR_DQ_NAMESIZE RTE_RING_NAMESIZE > > + > > +/** > > + * Trigger automatic reclamation after 1/8th the defer queue is full. > > + */ > > +#define RTE_RCU_QSBR_AUTO_RECLAIM_LIMIT 3 > > + > > +/** > > + * Reclaim at the max 1/16th the total number of resources. > > + */ > > +#define RTE_RCU_QSBR_MAX_RECLAIM_LIMIT 4 > > + > > +/** > > + * Parameters used when creating the defer queue. > > + */ > > +struct rte_rcu_qsbr_dq_parameters { > > + const char *name; > > + /**< Name of the queue. */ > > + uint32_t size; > > + /**< Number of entries in queue. Typically, this will be > > + * the same as the maximum number of entries supported in the > > + * lock free data structure. > > + * Data structures with unbounded number of entries is not > > + * supported currently. > > + */ > > + uint32_t esize; > > + /**< Size (in bytes) of each element in the defer queue. > > + * This has to be multiple of 8B as the rte_ring APIs > > + * support 8B element sizes only. > > + */ > > + rte_rcu_qsbr_free_resource f; > > + /**< Function to call to free the resource. */ > > + void *p; > > + /**< Pointer passed to the free function. Typically, this is the > > + * pointer to the data structure to which the resource to free > > + * belongs. This can be NULL. > > + */ > > + struct rte_rcu_qsbr *v; > > + /**< RCU QSBR variable to use for this defer queue */ }; > > + > > +/* RTE defer queue structure. > > + * This structure holds the defer queue. The defer queue is used to > > + * hold the deleted entries from the data structure that are not > > + * yet freed. > > + */ > > +struct rte_rcu_qsbr_dq; > > + > > /** > > * @warning > > * @b EXPERIMENTAL: this API may change without prior notice @@ > > -648,6 +710,113 @@ __rte_experimental int rte_rcu_qsbr_dump(FILE *f, > > struct rte_rcu_qsbr *v); > > > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * Create a queue used to store the data structure elements that can > > + * be freed later. This queue is referred to as 'defer queue'. > > + * > > + * @param params > > + * Parameters to create a defer queue. > > + * @return > > + * On success - Valid pointer to defer queue > > + * On error - NULL > > + * Possible rte_errno codes are: > > + * - EINVAL - NULL parameters are passed > > + * - ENOMEM - Not enough memory > > + */ > > +__rte_experimental > > +struct rte_rcu_qsbr_dq * > > +rte_rcu_qsbr_dq_create(const struct rte_rcu_qsbr_dq_parameters > > +*params); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * Enqueue one resource to the defer queue and start the grace period. > > + * The resource will be freed later after at least one grace period > > + * is over. > > + * > > + * If the defer queue is full, it will attempt to reclaim resources. > > + * It will also reclaim resources at regular intervals to avoid > > + * the defer queue from growing too big. > > + * > > + * This API is not multi-thread safe. It is expected that the caller > > + * provides multi-thread safety by locking a mutex or some other means= . > > + * > > + * A lock free multi-thread writer algorithm could achieve > > +multi-thread > > + * safety by creating and using one defer queue per thread. > > + * > > + * @param dq > > + * Defer queue to allocate an entry from. > > + * @param e > > + * Pointer to resource data to copy to the defer queue. The size of > > + * the data to copy is equal to the element size provided when the > > + * defer queue was created. > > + * @return > > + * On success - 0 > > + * On error - 1 with rte_errno set to > > + * - EINVAL - NULL parameters are passed > > + * - ENOSPC - Defer queue is full. This condition can not happen > > + * if the defer queue size is equal (or larger) than the > > + * number of elements in the data structure. > > + */ > > +__rte_experimental > > +int > > +rte_rcu_qsbr_dq_enqueue(struct rte_rcu_qsbr_dq *dq, void *e); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * Reclaim resources from the defer queue. > > + * > > + * This API is not multi-thread safe. It is expected that the caller > > + * provides multi-thread safety by locking a mutex or some other means= . > > + * > > + * A lock free multi-thread writer algorithm could achieve > > +multi-thread > > + * safety by creating and using one defer queue per thread. > > + * > > + * @param dq > > + * Defer queue to reclaim an entry from. > > + * @return > > + * On successful reclamation of at least 1 resource - 0 > > + * On error - 1 with rte_errno set to > > + * - EINVAL - NULL parameters are passed > > + * - EAGAIN - None of the resources have completed at least 1 grace > period, > > + * try again. > > + */ > > +__rte_experimental > > +int > > +rte_rcu_qsbr_dq_reclaim(struct rte_rcu_qsbr_dq *dq); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * Delete a defer queue. > > + * > > + * It tries to reclaim all the resources on the defer queue. > > + * If any of the resources have not completed the grace period > > + * the reclamation stops and returns immediately. The rest of > > + * the resources are not reclaimed and the defer queue is not > > + * freed. > > + * > > + * @param dq > > + * Defer queue to delete. > > + * @return > > + * On success - 0 > > + * On error - 1 > > + * Possible rte_errno codes are: > > + * - EINVAL - NULL parameters are passed > > + * - EAGAIN - Some of the resources have not completed at least 1 gr= ace > > + * period, try again. > > + */ > > +__rte_experimental > > +int > > +rte_rcu_qsbr_dq_delete(struct rte_rcu_qsbr_dq *dq); > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/librte_rcu/rte_rcu_qsbr_pvt.h > > b/lib/librte_rcu/rte_rcu_qsbr_pvt.h > > new file mode 100644 > > index 000000000..2122bc36a > > --- /dev/null > > +++ b/lib/librte_rcu/rte_rcu_qsbr_pvt.h > > @@ -0,0 +1,46 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright (c) 2019 Arm Limited > > + */ > > + > > +#ifndef _RTE_RCU_QSBR_PVT_H_ > > +#define _RTE_RCU_QSBR_PVT_H_ > > + > > +/** > > + * This file is private to the RCU library. It should not be included > > + * by the user of this library. > > + */ > > + > > +#ifdef __cplusplus > > +extern "C" { > > +#endif > > + > > +#include "rte_rcu_qsbr.h" > > + > > +/* RTE defer queue structure. > > + * This structure holds the defer queue. The defer queue is used to > > + * hold the deleted entries from the data structure that are not > > + * yet freed. > > + */ > > +struct rte_rcu_qsbr_dq { > > + struct rte_rcu_qsbr *v; /**< RCU QSBR variable used by this queue.*/ > > + struct rte_ring *r; /**< RCU QSBR defer queue. */ > > + uint32_t size; > > + /**< Number of elements in the defer queue */ > > + uint32_t esize; > > + /**< Size (in bytes) of data stored on the defer queue */ > > + rte_rcu_qsbr_free_resource f; > > + /**< Function to call to free the resource. */ > > + void *p; > > + /**< Pointer passed to the free function. Typically, this is the > > + * pointer to the data structure to which the resource to free > > + * belongs. > > + */ > > + char e[0]; > > + /**< Temporary storage to copy the defer queue element. */ }; > > + > > +#ifdef __cplusplus > > +} > > +#endif > > + > > +#endif /* _RTE_RCU_QSBR_PVT_H_ */ > > diff --git a/lib/librte_rcu/rte_rcu_version.map > > b/lib/librte_rcu/rte_rcu_version.map > > index f8b9ef2ab..dfac88a37 100644 > > --- a/lib/librte_rcu/rte_rcu_version.map > > +++ b/lib/librte_rcu/rte_rcu_version.map > > @@ -8,6 +8,10 @@ EXPERIMENTAL { > > rte_rcu_qsbr_synchronize; > > rte_rcu_qsbr_thread_register; > > rte_rcu_qsbr_thread_unregister; > > + rte_rcu_qsbr_dq_create; > > + rte_rcu_qsbr_dq_enqueue; > > + rte_rcu_qsbr_dq_reclaim; > > + rte_rcu_qsbr_dq_delete; > > > > local: *; > > }; > > diff --git a/lib/meson.build b/lib/meson.build index > > e5ff83893..0e1be8407 100644 > > --- a/lib/meson.build > > +++ b/lib/meson.build > > @@ -11,7 +11,9 @@ > > libraries =3D [ > > 'kvargs', # eal depends on kvargs > > 'eal', # everything depends on eal > > - 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core > > + 'ring', > > + 'rcu', # rcu depends on ring > > + 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core > > 'cmdline', > > 'metrics', # bitrate/latency stats depends on this > > 'hash', # efd depends on this > > @@ -22,7 +24,7 @@ libraries =3D [ > > 'gro', 'gso', 'ip_frag', 'jobstats', > > 'kni', 'latencystats', 'lpm', 'member', > > 'power', 'pdump', 'rawdev', > > - 'rcu', 'reorder', 'sched', 'security', 'stack', 'vhost', > > + 'reorder', 'sched', 'security', 'stack', 'vhost', > > # ipsec lib depends on net, crypto and security > > 'ipsec', > > # add pkt framework libs which use other libs from above > > -- > > 2.17.1