From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F39F9A3160 for ; Fri, 11 Oct 2019 20:28:49 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 612441EB4F; Fri, 11 Oct 2019 20:28:49 +0200 (CEST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50050.outbound.protection.outlook.com [40.107.5.50]) by dpdk.org (Postfix) with ESMTP id 95ED51EA4A for ; Fri, 11 Oct 2019 20:28:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RsOhimOUD3AdGcttVGgVVUwkhDnCGSkvR7L2/Vztza4=; b=mgCH6aBw2VEOq37gLW2JEdwK7yB7n9Rat9X206vTMWVCZQo6+Z9VfZWrDv7B8sJOETuOkBrXjmmiMGNDm4h9dIjUA6CGSapcIV0jH21rq5QR1k9WONRV9oZwPVhN8hBtfVvPcZakEJy9MNjuGRGJ4WmwRP6JRM+DQxG9BDj9rKI= Received: from DB6PR0801CA0048.eurprd08.prod.outlook.com (2603:10a6:4:2b::16) by DBBPR08MB4298.eurprd08.prod.outlook.com (2603:10a6:10:cb::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.16; Fri, 11 Oct 2019 18:28:41 +0000 Received: from VE1EUR03FT017.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::207) by DB6PR0801CA0048.outlook.office365.com (2603:10a6:4:2b::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.16 via Frontend Transport; Fri, 11 Oct 2019 18:28:41 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT017.mail.protection.outlook.com (10.152.18.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15 via Frontend Transport; Fri, 11 Oct 2019 18:28:39 +0000 Received: ("Tessian outbound 081de437afc7:v33"); Fri, 11 Oct 2019 18:28:35 +0000 X-CR-MTA-TID: 64aa7808 Received: from 98df8b4397fb.2 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.13.58]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id E369382A-C7BC-4E95-9B47-14EC46151D04.1; Fri, 11 Oct 2019 18:28:30 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04lp2058.outbound.protection.outlook.com [104.47.13.58]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 98df8b4397fb.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 11 Oct 2019 18:28:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fr7zo3ZdJz2Tx4wx4o8qJ+iN12N5JdcT/tk+V0XmgaTkowFihLy5tSsTQRCUqG1Uiyqmbaiq8X9yIfuzkqc5yETDieaIMLbYFnMEV+x1IP6S9q05bTi9yNSwC+d//6TM8yf2Nne54VkwZgN8zBlqH2vpyu6BP/YTH0TCJXEeezluLXg4oVLXSDQHTwpMI1VXexhEGQ65j708E+dDcJoz3zn8ykkjhbvqe3fy+hDbl4/uzcZY43KEHRhw8GlZGjiYDqS7R8nojZ+fBIalYnMoO56Mm3qd6/mrUS0ghNBLiqPwVCd7lKZRbxmFgJ4/E1uP8t5ohs/f2EKyB7mZ0dXiNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RsOhimOUD3AdGcttVGgVVUwkhDnCGSkvR7L2/Vztza4=; b=GhjvjtAHbjwdISV97VAosBGFLMv5Tl0Z4EgNIX8SatPcPOBKaNgbzgLpQ6FdYlIyjY48Xg43foh4SI+iQhSTJ/HFrqkzqe4BaMC31vL0e20R7KFp/pJ7VP62EhNkwfHCAeLMSaWCRJEO5USX/RZEMwBosYs9ykoJJa4zrJEU9Zr/TSpLxPSCn4CLrrPWjdz3mmyfGXSDrxLIs+X6gBKrDPDZpz3zgbeDg1N9aQZySph39Wildv6XZ4fP9Mb/C0f3p7QUGQCPwonEaGRBgigBEAjFWO4G50ipUGmUOqcRndA3HGy3cwybqvBZLg7oKaW2Ixe3j1Jmzq5M+wFkS3SiiA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RsOhimOUD3AdGcttVGgVVUwkhDnCGSkvR7L2/Vztza4=; b=mgCH6aBw2VEOq37gLW2JEdwK7yB7n9Rat9X206vTMWVCZQo6+Z9VfZWrDv7B8sJOETuOkBrXjmmiMGNDm4h9dIjUA6CGSapcIV0jH21rq5QR1k9WONRV9oZwPVhN8hBtfVvPcZakEJy9MNjuGRGJ4WmwRP6JRM+DQxG9BDj9rKI= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1PR08MB4880.eurprd08.prod.outlook.com (10.255.114.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2327.24; Fri, 11 Oct 2019 18:28:27 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6%7]) with mapi id 15.20.2347.016; Fri, 11 Oct 2019 18:28:27 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "stephen@networkplumber.org" , "paulmck@linux.ibm.com" CC: "Wang, Yipeng1" , "Medvedkin, Vladimir" , "Ruifeng Wang (Arm Technology China)" , Dharmik Thakkar , "dev@dpdk.org" , Honnappa Nagarahalli , nd , nd Thread-Topic: [PATCH v3 1/3] lib/ring: add peek API Thread-Index: AQHVeVEbQBVWnKnBNky5vPA5w4G8vadIeNrQgAZwDgCAAs5McIACT7WAgADnlFCAAKKugIAAPHEA Date: Fri, 11 Oct 2019 18:28:27 +0000 Message-ID: References: <20190906094534.36060-1-ruifeng.wang@arm.com> <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20191001062917.35578-2-honnappa.nagarahalli@arm.com> <2601191342CEEE43887BDE71AB9772580191970014@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772580191971EBE@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772580191975145@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772580191975AE3@irsmsx105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB9772580191975AE3@irsmsx105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 2ae1603d-0daa-452f-9d85-2353c9f0e5b6.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 61f94e99-c970-4c61-1e5a-08d74e78d677 X-MS-Office365-Filtering-HT: Tenant X-MS-TrafficTypeDiagnostic: VE1PR08MB4880:|VE1PR08MB4880:|DBBPR08MB4298: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:1728;OLM:1728; x-forefront-prvs: 0187F3EA14 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(136003)(346002)(39860400002)(366004)(376002)(189003)(199004)(33656002)(11346002)(14444005)(256004)(476003)(486006)(25786009)(2201001)(6436002)(229853002)(4326008)(6116002)(446003)(305945005)(2501003)(2906002)(74316002)(86362001)(3846002)(316002)(110136005)(76176011)(7696005)(6506007)(102836004)(186003)(54906003)(7736002)(71190400001)(8676002)(8936002)(52536014)(30864003)(81166006)(66066001)(99286004)(81156014)(71200400001)(76116006)(6246003)(9686003)(66946007)(66446008)(64756008)(66556008)(66476007)(5660300002)(478600001)(55016002)(14454004)(26005); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4880; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 3qkULgYxDl41o/WNFEHWJKtByCmB14H1sPs6/x8jjRgnW4+wq5FXrtGTXm3OFmGhNPF0NnwWaIXgMnWkD0SGZU8t7pTA7dM8XPTxnSPKuO4GQYZ8V85mdjR4bhEEv2ik4528sWupiLFLsfrOyhUH+UbfTOAo3supaR6Z0d1CrP9A/vmyOZC4Th0VSqT26o6tDOM79MJOHa1TX5HcLb0nmf+45zNbFDhgCGsZ0O6AfntpUQ6l17QlPPQ9MMjmpwRGYt2VboocJ7vMAONqAjCtoKjfQgjdH2fbjhbnnh/h/kS0NM61/RZE45QiCC+dUlQ0CNwXzjMmHNHbGqkr8w3M7TvQ7Zrs3DpqUWweU3qqx9pqZqnN/kRSH9mF5Y1X1BUkSXwBiHNMfN7w1xzq6/ymKzuU5DkSljmjaDnVSMyvPgE= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4880 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT017.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(136003)(396003)(346002)(376002)(39860400002)(189003)(199004)(4326008)(26826003)(81156014)(81166006)(14454004)(478600001)(2501003)(8676002)(9686003)(486006)(66066001)(63350400001)(52536014)(356004)(47776003)(446003)(11346002)(476003)(30864003)(50466002)(336012)(25786009)(97756001)(5660300002)(126002)(22756006)(33656002)(229853002)(23726003)(7696005)(76176011)(8936002)(6506007)(102836004)(8746002)(2906002)(26005)(3846002)(6116002)(186003)(6246003)(76130400001)(46406003)(7736002)(55016002)(74316002)(305945005)(70586007)(70206006)(99286004)(54906003)(110136005)(36906005)(316002)(2201001)(86362001)(14444005); DIR:OUT; SFP:1101; SCL:1; SRVR:DBBPR08MB4298; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; MX:1; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: f52202cd-88bf-454f-5e0c-08d74e78cf40 NoDisclaimer: True X-Forefront-PRVS: 0187F3EA14 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6TtqQr544/CTZb4JFDROAUwCrYAqAfeqTNxxKvV2sp8NTiUeodS6iBnQ4iepsnlp++eTYFb9KZ9GSD8V2IPLEmtYrfnp+mUMgNdQhEub6BzW8IYskA7YxQTF3H3X03dnDM2U8k5556N9g+U7chCaSB31JR/L4LK3AkUVq1Er07f5E2tp6/HMPQvTq4xdIs005gzJlTg/Rr+rNgQLj/Ib+pec4BynbfEIj2XerviUH1Qv526H1AJZ9+bwXamOvMGO2L8GJFH6FMv5F9hT9DIeFUPuYAZ7KvS3ybSkSM9v13v9FUaqn1PpCMjzlXwPAJGBCykXzDp0GvTn2kzI9FDsZRh9uANWaj0rB7Yj2E/gL4n4kG/wMAUB8y2/Th43Jq2SKEhP8O66k21ZwAZMo/6JyPzzODekRnZlQiVNdOtwqrc= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Oct 2019 18:28:39.5976 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 61f94e99-c970-4c61-1e5a-08d74e78d677 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4298 Subject: Re: [dpdk-dev] [PATCH v3 1/3] lib/ring: add peek API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > > > > > > > > > > > > > > > > > > Subject: [PATCH v3 1/3] lib/ring: add peek API > > > > > > > > > > > > > > > > From: Ruifeng Wang > > > > > > > > > > > > > > > > The peek API allows fetching the next available object in > > > > > > > > the ring without dequeuing it. This helps in scenarios > > > > > > > > where dequeuing of objects depend on their value. > > > > > > > > > > > > > > > > Signed-off-by: Dharmik Thakkar > > > > > > > > Signed-off-by: Ruifeng Wang > > > > > > > > Reviewed-by: Honnappa Nagarahalli > > > > > > > > > > > > > > > > Reviewed-by: Gavin Hu > > > > > > > > --- > > > > > > > > lib/librte_ring/rte_ring.h | 30 > > > > > > > > ++++++++++++++++++++++++++++++ > > > > > > > > 1 file changed, 30 insertions(+) > > > > > > > > > > > > > > > > diff --git a/lib/librte_ring/rte_ring.h > > > > > > > > b/lib/librte_ring/rte_ring.h index 2a9f768a1..d3d0d5e18 > > > > > > > > 100644 > > > > > > > > --- a/lib/librte_ring/rte_ring.h > > > > > > > > +++ b/lib/librte_ring/rte_ring.h > > > > > > > > @@ -953,6 +953,36 @@ rte_ring_dequeue_burst(struct > > > > > > > > rte_ring *r, void > > > > > > > **obj_table, > > > > > > > > r->cons.single, available); } > > > > > > > > > > > > > > > > +/** > > > > > > > > + * Peek one object from a ring. > > > > > > > > + * > > > > > > > > + * The peek API allows fetching the next available object > > > > > > > > +in the ring > > > > > > > > + * without dequeuing it. This API is not multi-thread > > > > > > > > +safe with respect > > > > > > > > + * to other consumer threads. > > > > > > > > + * > > > > > > > > + * @param r > > > > > > > > + * A pointer to the ring structure. > > > > > > > > + * @param obj_p > > > > > > > > + * A pointer to a void * pointer (object) that will be f= illed. > > > > > > > > + * @return > > > > > > > > + * - 0: Success, object available > > > > > > > > + * - -ENOENT: Not enough entries in the ring. > > > > > > > > + */ > > > > > > > > +__rte_experimental > > > > > > > > +static __rte_always_inline int rte_ring_peek(struct > > > > > > > > +rte_ring *r, void **obj_p) > > > > > > > > > > > > > > As it is not MT safe, then I think we need _sc_ in the name, > > > > > > > to follow other rte_ring functions naming conventions > > > > > > > (rte_ring_sc_peek() or so). > > > > > > Agree > > > > > > > > > > > > > > > > > > > > As a better alternative what do you think about introducing > > > > > > > a serialized versions of DPDK rte_ring dequeue functions? > > > > > > > Something like that: > > > > > > > > > > > > > > /* same as original ring dequeue, but: > > > > > > > * 1) move cons.head only if cons.head =3D=3D const.tail > > > > > > > * 2) don't update cons.tail > > > > > > > */ > > > > > > > unsigned int > > > > > > > rte_ring_serial_dequeue_bulk(struct rte_ring *r, void > > > > > > > **obj_table, unsigned int n, > > > > > > > unsigned int *available); > > > > > > > > > > > > > > /* sets both cons.head and cons.tail to cons.head + num */ > > > > > > > void rte_ring_serial_dequeue_finish(struct rte_ring *r, > > > > > > > uint32_t num); > > > > > > > > > > > > > > /* resets cons.head to const.tail value */ void > > > > > > > rte_ring_serial_dequeue_abort(struct rte_ring *r); > > > > > > > > > > > > > > Then your dq_reclaim cycle function will look like that: > > > > > > > > > > > > > > const uint32_t nb_elt =3D dq->elt_size/8 + 1; uint32_t avl, > > > > > > > n; uintptr_t elt[nb_elt]; ... > > > > > > > > > > > > > > do { > > > > > > > > > > > > > > /* read next elem from the queue */ > > > > > > > n =3D rte_ring_serial_dequeue_bulk(dq->r, elt, nb_elt, &avl= ); > > > > > > > if (n =3D=3D 0) > > > > > > > break; > > > > > > > > > > > > > > /* wrong period, keep elem in the queue */ if > > > > > > > (rte_rcu_qsbr_check(dr->v, > > > > > > > elt[0]) !=3D 1) { > > > > > > > rte_ring_serial_dequeue_abort(dq->r); > > > > > > > break; > > > > > > > } > > > > > > > > > > > > > > /* can reclaim, remove elem from the queue */ > > > > > > > rte_ring_serial_dequeue_finish(dr->q, nb_elt); > > > > > > > > > > > > > > /*call reclaim function */ > > > > > > > dr->f(dr->p, elt); > > > > > > > > > > > > > > } while (avl >=3D nb_elt); > > > > > > > > > > > > > > That way, I think even rte_rcu_qsbr_dq_reclaim() can be MT sa= fe. > > > > > > > As long as actual reclamation callback itself is MT safe of c= ourse. > > > > > > > > > > > > I think it is a great idea. The other writers would still be > > > > > > polling for the current writer to update the tail or update > > > > > > the head. This makes it a > > > > > blocking solution. > > > > > > > > > > Yep, it is a blocking one. > > > > > > > > > > > We can make the other threads not poll i.e. they will quit > > > > > > reclaiming if they > > > > > see that other writers are dequeuing from the queue. > > > > > > > > > > Actually didn't think about that possibility, but yes should be > > > > > possible to have _try_ semantics too. > > > > > > > > > > >The other way is to use per thread queues. > > > > > > > > > > > > The other requirement I see is to support unbounded-size data > > > > > > structures where in the data structures do not have a > > > > > > pre-determined number of entries. Also, currently the defer > > > > > > queue size is equal to the total > > > > > number of entries in a given data structure. There are plans to > > > > > support dynamically resizable defer queue. This means, memory > > > > > allocation which will affect the lock-free-ness of the solution. > > > > > > > > > > > > So, IMO: > > > > > > 1) The API should provide the capability to support different > > > > > > algorithms - > > > > > may be through some flags? > > > > > > 2) The requirements for the ring are pretty unique to the > > > > > > problem we have here (for ex: move the cons-head only if > > > > > > cons-tail is also the same, skip > > > > > polling). So, we should probably implement a ring with-in the RCU > library? > > > > > > > > > > Personally, I think such serialization ring API would be useful > > > > > for other cases too. > > > > > There are few cases when user need to read contents of the queue > > > > > without removing elements from it. > > > > > Let say we do use similar approach inside TLDK to implement TCP > > > > > transmit queue. > > > > > If such API would exist in DPDK we can just use it straightway, > > > > > without maintaining a separate one. > > > > ok > > > > > > > > > > > > > > > > > > > > > From the timeline perspective, adding all these capabilities > > > > > > would be difficult to get done with in 19.11 timeline. What I > > > > > > have here satisfies my current needs. I suggest that we make > > > > > > provisions in APIs now to > > > > > support all these features, but do the implementation in the > > > > > coming > > > releases. > > > > > Does this sound ok for you? > > > > > > > > > > Not sure I understand your suggestion here... > > > > > Could you explain it a bit more - how new API will look like and > > > > > what would be left for the future. > > > > For this patch, I suggest we do not add any more complexity. If > > > > someone wants a lock-free/block-free mechanism, it is available by > > > > creating > > > per thread defer queues. > > > > > > > > We push the following to the future: > > > > 1) Dynamically size adjustable defer queue. IMO, with this, the > > > > lock-free/block-free reclamation will not be available (memory > > > > allocation > > > requires locking). The memory for the defer queue will be > > > allocated/freed in chunks of 'size' elements as the queue grows/shrin= ks. > > > > > > That one is fine by me. > > > In fact I don't know would be there a real use-case for dynamic > > > defer queue for rcu var... > > > But I suppose that's subject for another discussion. > > Currently, the defer queue size is equal to the number of resources in > > the data structure. This is unnecessary as the reclamation is done regu= larly. > > If a smaller queue size is used, the queue might get full (even after > reclamation), in which case, the queue size should be increased. >=20 > I understand the intention. > Though I am not very happy with approach where to free one resource we fi= rst > have to allocate another one. > Sounds like a source of deadlocks and for that case probably unnecessary > complication. It depends on the use case. For some use cases lock-free reader-writer conc= urrency is enough (in which case there is no need to have a queue large eno= ugh to hold all the resources) and some would require lock-free reader-writ= er and writer-writer concurrency (where, theoretically, a queue large enoug= h to hold all the resources would be required). > But again, as it is not for 19.11 we don't have to discuss it now. >=20 > > > > > > > > > > > 2) Constant size defer queue with lock-free and block-free > > > > reclamation (single option). The defer queue will be of fixed > > > > length 'size'. If the queue gets full an error is returned. The > > > > user could provide a 'size' equal > > > to the number of elements in a data structure to ensure queue never g= ets > full. > > > > > > Ok so for 19.11 what enqueue/dequeue model do you plan to support? > > > - MP/MC > > > - MP/SC > > > - SP/SC > > Just SP/SC >=20 > Ok, just to confirm we are on the same page: > there would be a possibility for one thread do dq_enqueue(), second one d= o > dq_reclaim() simultaneously (of course if actual reclamation function is = thread > safe)? Yes, that is allowed. Mutual exclusion is required only around dq_reclaim. >=20 > > > - non MT at all (only same single thread can do enqueue and dequeue) > > If MT safe is required, one should use 1 defer queue per thread for now= . > > > > > > > > And related question: > > > What additional rte_ring API you plan to introduce in that case? > > > - None > > > - rte_ring_sc_peek() > > rte_ring_peek will be changed to rte_ring_sc_peek > > > > > - rte_ring_serial_dequeue() > > > > > > > > > > > I would add a 'flags' field in rte_rcu_qsbr_dq_parameters and > > > > provide > > > > 2 #defines, one for dynamically variable size defer queue and the > > > > other for > > > constant size defer queue. > > > > > > > > However, IMO, using per thread defer queue is a much simpler way > > > > to > > > achieve 2. It does not add any significant burden to the user either. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +{ > > > > > > > > + uint32_t prod_tail =3D r->prod.tail; > > > > > > > > + uint32_t cons_head =3D r->cons.head; > > > > > > > > + uint32_t count =3D (prod_tail - cons_head) & r->mask; > > > > > > > > + unsigned int n =3D 1; > > > > > > > > + if (count) { > > > > > > > > + DEQUEUE_PTRS(r, &r[1], cons_head, obj_p, n, void *); > > > > > > > > + return 0; > > > > > > > > + } > > > > > > > > + return -ENOENT; > > > > > > > > +} > > > > > > > > + > > > > > > > > #ifdef __cplusplus > > > > > > > > } > > > > > > > > #endif > > > > > > > > -- > > > > > > > > 2.17.1