From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B764EA2EFC for ; Mon, 14 Oct 2019 06:11:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 198D71C18E; Mon, 14 Oct 2019 06:11:38 +0200 (CEST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50048.outbound.protection.outlook.com [40.107.5.48]) by dpdk.org (Postfix) with ESMTP id 308921C113 for ; Mon, 14 Oct 2019 06:11:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T3v2w3sTAHX8vTETYpFAYQ1Xx7Ps0NzYR2Nk5x0VaXQ=; b=vZ79TKc1b/5inoP2oMi+iGxou4EEDu769qAORpKeROnR0m8YRDGs6mEjtbE0U7x0Zas6aYE3A1VUDYyKUhyxLRSIDQ4l/UGwmuREvEajm6LoyuVsQGpelH+pBUibqR50D5gs288UGnCQH6vq3V6+kQqly/CMqC7shuM26efxUCY= Received: from DB6PR0801CA0054.eurprd08.prod.outlook.com (2603:10a6:4:2b::22) by HE1PR0801MB2121.eurprd08.prod.outlook.com (2603:10a6:3:80::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.21; Mon, 14 Oct 2019 04:11:31 +0000 Received: from AM5EUR03FT051.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e08::202) by DB6PR0801CA0054.outlook.office365.com (2603:10a6:4:2b::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2347.16 via Frontend Transport; Mon, 14 Oct 2019 04:11:31 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT051.mail.protection.outlook.com (10.152.16.246) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2305.15 via Frontend Transport; Mon, 14 Oct 2019 04:11:29 +0000 Received: ("Tessian outbound 081de437afc7:v33"); Mon, 14 Oct 2019 04:11:16 +0000 X-CR-MTA-TID: 64aa7808 Received: from 3650e9ce0646.1 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.8.50]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id F171A6F4-AF8D-4DF4-9B51-FEAE71B2C625.1; Mon, 14 Oct 2019 04:11:11 +0000 Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-am5eur03lp2050.outbound.protection.outlook.com [104.47.8.50]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3650e9ce0646.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384); Mon, 14 Oct 2019 04:11:11 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X3m7vAZ+XMM5tn6Y2FYwDVTMfAqLei1stwieQJuMCtL+lg29Vnedc9Ydx4Kz9wMIvtkz+xkk+OZ8FnjEQpMJifMzHtr1blr0VNDHrN/VFE/ZPAKuj+18CFfJra2HVzIE0RRiccoLVMdoUgZkm/DjMLxg8hmnsgLmx1TVWK2328tOklH3fTCPPaCSUy/W2r+7mNmb/7/TpvW1B7quHPYBhuX3LOce5xuU2YaOs6xn5Bvul/HKqWS5nk1ZA2X/HdHQNf3LnbmwYTKWPwkVe+Ve+xoOP4bofspVpR0/8LL3C44/1UswGLiJb0XPdnpkjath/mFj7vX+QKC0SLNPq69WCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T3v2w3sTAHX8vTETYpFAYQ1Xx7Ps0NzYR2Nk5x0VaXQ=; b=aZ/HeBPxYzacLSo+7BlETEHgHHc6IrwseL4sf4oCZBE1heVxPJFQxMUh+lzYuV35T9e7DGHtNoEgCTvH9VSSaDLoIiiOnuXFP8xZohJ4eZ7MyTK1+MK9+JHd8CFpCXLGJ7mH5MiCoZtD8E9bi/K/1U93aniW7kNGhmJ3/RS1CdMnU5AMICbNA8y4paj/9yZV0641bn3mI86p90F4/ozN3YYNCSRwwLO36PbRr8b3k0alMH7HS2LTnA/tIHdmESuZ9VZLLSXM0tUZ8zNXlRM2sDSLB1Ooo86bvF3iavZhZs1jQCSgnhpv+Rsvr/mciXYARoPYqxzQh2u3r9S0a+K53g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T3v2w3sTAHX8vTETYpFAYQ1Xx7Ps0NzYR2Nk5x0VaXQ=; b=vZ79TKc1b/5inoP2oMi+iGxou4EEDu769qAORpKeROnR0m8YRDGs6mEjtbE0U7x0Zas6aYE3A1VUDYyKUhyxLRSIDQ4l/UGwmuREvEajm6LoyuVsQGpelH+pBUibqR50D5gs288UGnCQH6vq3V6+kQqly/CMqC7shuM26efxUCY= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1PR08MB5055.eurprd08.prod.outlook.com (10.255.159.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.17; Mon, 14 Oct 2019 04:11:09 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6%7]) with mapi id 15.20.2347.021; Mon, 14 Oct 2019 04:11:09 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "stephen@networkplumber.org" , "paulmck@linux.ibm.com" CC: "Wang, Yipeng1" , "Medvedkin, Vladimir" , "Ruifeng Wang (Arm Technology China)" , Dharmik Thakkar , "dev@dpdk.org" , Honnappa Nagarahalli , nd , nd Thread-Topic: [PATCH v3 1/3] lib/ring: add peek API Thread-Index: AQHVeVEbQBVWnKnBNky5vPA5w4G8vadIeNrQgAZwDgCAAs5McIACT7WAgADnlFCAAKKugIAAPHEAgAND4oCAAIUYYA== Date: Mon, 14 Oct 2019 04:11:08 +0000 Message-ID: References: <20190906094534.36060-1-ruifeng.wang@arm.com> <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20191001062917.35578-2-honnappa.nagarahalli@arm.com> <2601191342CEEE43887BDE71AB9772580191970014@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772580191971EBE@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772580191975145@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772580191975AE3@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB97725801A8C65794@IRSMSX104.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB97725801A8C65794@IRSMSX104.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 7df28bdb-aee7-4a56-a424-81693e3d93fa.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 93a8cdfd-ab79-4915-e767-08d7505c96cf X-MS-Office365-Filtering-HT: Tenant X-MS-TrafficTypeDiagnostic: VE1PR08MB5055:|VE1PR08MB5055:|HE1PR0801MB2121: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:1728;OLM:1728; x-forefront-prvs: 01901B3451 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(376002)(346002)(136003)(396003)(39860400002)(366004)(189003)(199004)(14454004)(478600001)(6246003)(4326008)(66066001)(33656002)(99286004)(76176011)(7696005)(25786009)(81156014)(14444005)(8676002)(256004)(81166006)(52536014)(71190400001)(66946007)(76116006)(66446008)(66476007)(64756008)(66556008)(8936002)(71200400001)(6506007)(2501003)(7736002)(74316002)(186003)(26005)(2906002)(305945005)(102836004)(11346002)(476003)(5660300002)(30864003)(446003)(2201001)(86362001)(6436002)(486006)(316002)(3846002)(6116002)(110136005)(54906003)(229853002)(55016002)(9686003); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5055; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: RlUOfndewYzxYk3t0g+ikJku3hdSInzGKyOBTSuwgKj3AVS4Bcm3NTcE7250A/onazEMjNNDNmgkbPDdL0qCpn+ZEghRA0POWUAmE1rfGc9a7gDk86WP7SUHskhLxMtUCjl1kuZG8i7KdBZJhft1IrublNx1DWOoxVY2Ny7C/RXbOEvYAOwj6Y6x3Ju0708ov9l2ebpwzc1kDprAoe5oTltQpLV42Bw8ILVZDje6xmF8QZyt3WYKVS5mL9DiZqKNXQ+riJyDa1ShCCq9s4cgkDK2XSs0Vv9tOrdKNxbysfXfmgC7RR66dmUARQVHSrvAdxynjEjPTN2qNpw05qwrK9HCjD0I5lmE4ZfhlwRch44C/qFSkisO+LLgdxMvq0c1X16Q7TkMUhlTeH4YfVx8Dk03cSiuVU2nM3fUtpuFpX4= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5055 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT051.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(136003)(376002)(396003)(346002)(189003)(199004)(2906002)(70586007)(33656002)(5660300002)(76130400001)(3846002)(70206006)(2201001)(50466002)(6116002)(86362001)(46406003)(23726003)(66066001)(356004)(30864003)(4326008)(76176011)(7696005)(47776003)(102836004)(8936002)(8746002)(26005)(81166006)(81156014)(6506007)(8676002)(52536014)(99286004)(25786009)(336012)(110136005)(316002)(55016002)(186003)(229853002)(54906003)(446003)(2501003)(63350400001)(486006)(11346002)(14444005)(9686003)(6246003)(476003)(126002)(26826003)(478600001)(97756001)(305945005)(7736002)(22756006)(74316002)(14454004)(36906005); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0801MB2121; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; MX:1; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: f4c0c2c4-4b7b-4a65-d816-08d7505c8ab6 NoDisclaimer: True X-Forefront-PRVS: 01901B3451 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F2XWwJUtTdSPzYnAHhTYBzk+bwbclR/E7j/nhSa2D6er7AuU6ArSW8smQEhERMFs9y0HxSSJVM92ewWnhmWGzb2x5dP0dK7/fYUCJNPusWNf/eAPu0UZSL1g1TZEHnpfxig0m2QerLRIBE6IRit25Y1uHE3p4Y5BJZVg+d+5XWW3O9MA89D7QW3gT+cKqja0cBoEkk7KT8kNF8NXowLI0X5kA+6L0LWB5kXQQKwloRDG829wLoYNOlxJ088GiogXdwcc9VNHUmjXg5dSS/bvOz9NyMJHP9i9QLZODt7+9nHQh00R4Pkz4lrjrp3PXlbITQBM81xes8x296I853/RJxUCa1UBiw9tuQRctUZXjCk+Qg17TrnYj1uHaZ21Nd2Ut9YHZ81hfV+7W8oX881I9ng+QL59c05OOHX8w6nvdNo= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Oct 2019 04:11:29.2654 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 93a8cdfd-ab79-4915-e767-08d7505c96cf X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0801MB2121 Subject: Re: [dpdk-dev] [PATCH v3 1/3] lib/ring: add peek API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > > > > > Subject: [PATCH v3 1/3] lib/ring: add peek API > > > > > > > > > > > > > > > > > > > > From: Ruifeng Wang > > > > > > > > > > > > > > > > > > > > The peek API allows fetching the next available object > > > > > > > > > > in the ring without dequeuing it. This helps in > > > > > > > > > > scenarios where dequeuing of objects depend on their va= lue. > > > > > > > > > > > > > > > > > > > > Signed-off-by: Dharmik Thakkar > > > > > > > > > > > > > > > > > > > > Signed-off-by: Ruifeng Wang > > > > > > > > > > Reviewed-by: Honnappa Nagarahalli > > > > > > > > > > > > > > > > > > > > Reviewed-by: Gavin Hu > > > > > > > > > > --- > > > > > > > > > > lib/librte_ring/rte_ring.h | 30 > > > > > > > > > > ++++++++++++++++++++++++++++++ > > > > > > > > > > 1 file changed, 30 insertions(+) > > > > > > > > > > > > > > > > > > > > diff --git a/lib/librte_ring/rte_ring.h > > > > > > > > > > b/lib/librte_ring/rte_ring.h index > > > > > > > > > > 2a9f768a1..d3d0d5e18 > > > > > > > > > > 100644 > > > > > > > > > > --- a/lib/librte_ring/rte_ring.h > > > > > > > > > > +++ b/lib/librte_ring/rte_ring.h > > > > > > > > > > @@ -953,6 +953,36 @@ rte_ring_dequeue_burst(struct > > > > > > > > > > rte_ring *r, void > > > > > > > > > **obj_table, > > > > > > > > > > r->cons.single, available); } > > > > > > > > > > > > > > > > > > > > +/** > > > > > > > > > > + * Peek one object from a ring. > > > > > > > > > > + * > > > > > > > > > > + * The peek API allows fetching the next available > > > > > > > > > > +object in the ring > > > > > > > > > > + * without dequeuing it. This API is not multi-thread > > > > > > > > > > +safe with respect > > > > > > > > > > + * to other consumer threads. > > > > > > > > > > + * > > > > > > > > > > + * @param r > > > > > > > > > > + * A pointer to the ring structure. > > > > > > > > > > + * @param obj_p > > > > > > > > > > + * A pointer to a void * pointer (object) that will = be filled. > > > > > > > > > > + * @return > > > > > > > > > > + * - 0: Success, object available > > > > > > > > > > + * - -ENOENT: Not enough entries in the ring. > > > > > > > > > > + */ > > > > > > > > > > +__rte_experimental > > > > > > > > > > +static __rte_always_inline int rte_ring_peek(struct > > > > > > > > > > +rte_ring *r, void **obj_p) > > > > > > > > > > > > > > > > > > As it is not MT safe, then I think we need _sc_ in the > > > > > > > > > name, to follow other rte_ring functions naming > > > > > > > > > conventions > > > > > > > > > (rte_ring_sc_peek() or so). > > > > > > > > Agree > > > > > > > > > > > > > > > > > > > > > > > > > > As a better alternative what do you think about > > > > > > > > > introducing a serialized versions of DPDK rte_ring dequeu= e > functions? > > > > > > > > > Something like that: > > > > > > > > > > > > > > > > > > /* same as original ring dequeue, but: > > > > > > > > > * 1) move cons.head only if cons.head =3D=3D const.tail > > > > > > > > > * 2) don't update cons.tail > > > > > > > > > */ > > > > > > > > > unsigned int > > > > > > > > > rte_ring_serial_dequeue_bulk(struct rte_ring *r, void > > > > > > > > > **obj_table, unsigned int n, > > > > > > > > > unsigned int *available); > > > > > > > > > > > > > > > > > > /* sets both cons.head and cons.tail to cons.head + num > > > > > > > > > */ void rte_ring_serial_dequeue_finish(struct rte_ring > > > > > > > > > *r, uint32_t num); > > > > > > > > > > > > > > > > > > /* resets cons.head to const.tail value */ void > > > > > > > > > rte_ring_serial_dequeue_abort(struct rte_ring *r); > > > > > > > > > > > > > > > > > > Then your dq_reclaim cycle function will look like that: > > > > > > > > > > > > > > > > > > const uint32_t nb_elt =3D dq->elt_size/8 + 1; uint32_t > > > > > > > > > avl, n; uintptr_t elt[nb_elt]; ... > > > > > > > > > > > > > > > > > > do { > > > > > > > > > > > > > > > > > > /* read next elem from the queue */ > > > > > > > > > n =3D rte_ring_serial_dequeue_bulk(dq->r, elt, nb_elt, = &avl); > > > > > > > > > if (n =3D=3D 0) > > > > > > > > > break; > > > > > > > > > > > > > > > > > > /* wrong period, keep elem in the queue */ if > > > > > > > > > (rte_rcu_qsbr_check(dr->v, > > > > > > > > > elt[0]) !=3D 1) { > > > > > > > > > rte_ring_serial_dequeue_abort(dq->r); > > > > > > > > > break; > > > > > > > > > } > > > > > > > > > > > > > > > > > > /* can reclaim, remove elem from the queue */ > > > > > > > > > rte_ring_serial_dequeue_finish(dr->q, nb_elt); > > > > > > > > > > > > > > > > > > /*call reclaim function */ > > > > > > > > > dr->f(dr->p, elt); > > > > > > > > > > > > > > > > > > } while (avl >=3D nb_elt); > > > > > > > > > > > > > > > > > > That way, I think even rte_rcu_qsbr_dq_reclaim() can be M= T > safe. > > > > > > > > > As long as actual reclamation callback itself is MT safe = of > course. > > > > > > > > > > > > > > > > I think it is a great idea. The other writers would still > > > > > > > > be polling for the current writer to update the tail or > > > > > > > > update the head. This makes it a > > > > > > > blocking solution. > > > > > > > > > > > > > > Yep, it is a blocking one. > > > > > > > > > > > > > > > We can make the other threads not poll i.e. they will quit > > > > > > > > reclaiming if they > > > > > > > see that other writers are dequeuing from the queue. > > > > > > > > > > > > > > Actually didn't think about that possibility, but yes should > > > > > > > be possible to have _try_ semantics too. > > > > > > > > > > > > > > >The other way is to use per thread queues. > > > > > > > > > > > > > > > > The other requirement I see is to support unbounded-size > > > > > > > > data structures where in the data structures do not have a > > > > > > > > pre-determined number of entries. Also, currently the > > > > > > > > defer queue size is equal to the total > > > > > > > number of entries in a given data structure. There are plans > > > > > > > to support dynamically resizable defer queue. This means, > > > > > > > memory allocation which will affect the lock-free-ness of the > solution. > > > > > > > > > > > > > > > > So, IMO: > > > > > > > > 1) The API should provide the capability to support > > > > > > > > different algorithms - > > > > > > > may be through some flags? > > > > > > > > 2) The requirements for the ring are pretty unique to the > > > > > > > > problem we have here (for ex: move the cons-head only if > > > > > > > > cons-tail is also the same, skip > > > > > > > polling). So, we should probably implement a ring with-in > > > > > > > the RCU > > > library? > > > > > > > > > > > > > > Personally, I think such serialization ring API would be > > > > > > > useful for other cases too. > > > > > > > There are few cases when user need to read contents of the > > > > > > > queue without removing elements from it. > > > > > > > Let say we do use similar approach inside TLDK to implement > > > > > > > TCP transmit queue. > > > > > > > If such API would exist in DPDK we can just use it > > > > > > > straightway, without maintaining a separate one. > > > > > > ok > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From the timeline perspective, adding all these > > > > > > > > capabilities would be difficult to get done with in 19.11 > > > > > > > > timeline. What I have here satisfies my current needs. I > > > > > > > > suggest that we make provisions in APIs now to > > > > > > > support all these features, but do the implementation in the > > > > > > > coming > > > > > releases. > > > > > > > Does this sound ok for you? > > > > > > > > > > > > > > Not sure I understand your suggestion here... > > > > > > > Could you explain it a bit more - how new API will look like > > > > > > > and what would be left for the future. > > > > > > For this patch, I suggest we do not add any more complexity. > > > > > > If someone wants a lock-free/block-free mechanism, it is > > > > > > available by creating > > > > > per thread defer queues. > > > > > > > > > > > > We push the following to the future: > > > > > > 1) Dynamically size adjustable defer queue. IMO, with this, > > > > > > the lock-free/block-free reclamation will not be available > > > > > > (memory allocation > > > > > requires locking). The memory for the defer queue will be > > > > > allocated/freed in chunks of 'size' elements as the queue > grows/shrinks. > > > > > > > > > > That one is fine by me. > > > > > In fact I don't know would be there a real use-case for dynamic > > > > > defer queue for rcu var... > > > > > But I suppose that's subject for another discussion. > > > > Currently, the defer queue size is equal to the number of > > > > resources in the data structure. This is unnecessary as the reclama= tion is > done regularly. > > > > If a smaller queue size is used, the queue might get full (even > > > > after > > > reclamation), in which case, the queue size should be increased. > > > > > > I understand the intention. > > > Though I am not very happy with approach where to free one resource > > > we first have to allocate another one. > > > Sounds like a source of deadlocks and for that case probably > > > unnecessary complication. > > It depends on the use case. For some use cases lock-free reader-writer > > concurrency is enough (in which case there is no need to have a queue > > large enough to hold all the resources) and some would require lock-fre= e > reader-writer and writer-writer concurrency (where, theoretically, a queu= e > large enough to hold all the resources would be required). > > > > > But again, as it is not for 19.11 we don't have to discuss it now. > > > > > > > > > > > > > > > > > > > > 2) Constant size defer queue with lock-free and block-free > > > > > > reclamation (single option). The defer queue will be of fixed > > > > > > length 'size'. If the queue gets full an error is returned. > > > > > > The user could provide a 'size' equal > > > > > to the number of elements in a data structure to ensure queue > > > > > never gets > > > full. > > > > > > > > > > Ok so for 19.11 what enqueue/dequeue model do you plan to support= ? > > > > > - MP/MC > > > > > - MP/SC > > > > > - SP/SC > > > > Just SP/SC > > > > > > Ok, just to confirm we are on the same page: > > > there would be a possibility for one thread do dq_enqueue(), second > > > one do > > > dq_reclaim() simultaneously (of course if actual reclamation > > > function is thread safe)? > > Yes, that is allowed. Mutual exclusion is required only around dq_recla= im. This is not completely correct (as you have pointed out below) as dq_enqueu= e, will end up calling do_reclaim >=20 > Ok, and that probably due to nature of ring_sc_peek(), right?. > BuT user can set reclaim threshold higher then number of elems in the def= ere > queue, and that should help to prevent dq_reclaim() from inside > dq_enqueue(), correct? Yes, that is possible. > If so, I have no objections in general to the proposed plan. > Konstantin >=20 > > > > > > > > > > - non MT at all (only same single thread can do enqueue and > > > > > dequeue) > > > > If MT safe is required, one should use 1 defer queue per thread for= now. > > > > > > > > > > > > > > And related question: > > > > > What additional rte_ring API you plan to introduce in that case? > > > > > - None > > > > > - rte_ring_sc_peek() > > > > rte_ring_peek will be changed to rte_ring_sc_peek > > > > > > > > > - rte_ring_serial_dequeue() > > > > > > > > > > > > > > > > > I would add a 'flags' field in rte_rcu_qsbr_dq_parameters and > > > > > > provide > > > > > > 2 #defines, one for dynamically variable size defer queue and > > > > > > the other for > > > > > constant size defer queue. > > > > > > > > > > > > However, IMO, using per thread defer queue is a much simpler > > > > > > way to > > > > > achieve 2. It does not add any significant burden to the user eit= her. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +{ > > > > > > > > > > + uint32_t prod_tail =3D r->prod.tail; > > > > > > > > > > + uint32_t cons_head =3D r->cons.head; > > > > > > > > > > + uint32_t count =3D (prod_tail - cons_head) & r->mask; > > > > > > > > > > + unsigned int n =3D 1; > > > > > > > > > > + if (count) { > > > > > > > > > > + DEQUEUE_PTRS(r, &r[1], cons_head, obj_p, n, > void *); > > > > > > > > > > + return 0; > > > > > > > > > > + } > > > > > > > > > > + return -ENOENT; > > > > > > > > > > +} > > > > > > > > > > + > > > > > > > > > > #ifdef __cplusplus > > > > > > > > > > } > > > > > > > > > > #endif > > > > > > > > > > -- > > > > > > > > > > 2.17.1