From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0EEE1A2F63 for ; Thu, 3 Oct 2019 21:49:29 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1658E1C18F; Thu, 3 Oct 2019 21:49:28 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60082.outbound.protection.outlook.com [40.107.6.82]) by dpdk.org (Postfix) with ESMTP id 39FFD1C135 for ; Thu, 3 Oct 2019 21:49:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RSNKh48xsRwwZA1FiXGpcnh8AN9M+E4i+jWPR+Ti71o=; b=FjpdyFwpYJBB224GNJec50zs+LDLwpT9ivczsLlbOHKIBfs/qL8vPgOJJ1RpB3NNy0z3yctKP5M1h6GV9KiOinkf0jCXWLXpWTc0rMvLsNX+V7raet8VPed1WIG+8rKHLqQGuw/Vto1zTZUupdKt484EMJPpc3P+s0FYgEmsLSg= Received: from HE1PR08CA0057.eurprd08.prod.outlook.com (2603:10a6:7:2a::28) by AM6PR08MB5239.eurprd08.prod.outlook.com (2603:10a6:20b:e6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Thu, 3 Oct 2019 19:49:23 +0000 Received: from DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e0a::205) by HE1PR08CA0057.outlook.office365.com (2603:10a6:7:2a::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2305.20 via Frontend Transport; Thu, 3 Oct 2019 19:49:23 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT044.mail.protection.outlook.com (10.152.21.167) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2305.15 via Frontend Transport; Thu, 3 Oct 2019 19:49:20 +0000 Received: ("Tessian outbound 6481c7fa5a3c:v33"); Thu, 03 Oct 2019 19:49:16 +0000 X-CR-MTA-TID: 64aa7808 Received: from dcab200148f5.1 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.1.54]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 26F6EB9E-FAB6-41E1-9145-7BEF1394BF9D.1; Thu, 03 Oct 2019 19:49:11 +0000 Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01lp2054.outbound.protection.outlook.com [104.47.1.54]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id dcab200148f5.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384); Thu, 03 Oct 2019 19:49:11 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZQcy4f6AoYW/vP5Y6gq7Cay0hQ1+h999E691ifs0jZEc7DZZRjrOJiclhGFu6/h0/C+SEuusKr3yS3jCRk677z2PUHKL+0qCxYXstc40QS5qQJOWQLvcfRefny7Rw9a+mG2OSPYwk2/R6wld2dmiaVnUbXO7imv4jmtnhPkhgiijdGXigEGjn3VR/6sz/ejw1916rLGTvgDOmAqvKyxgbDtiytf96vrNAVCFKWcPnUrObTedIu18HOuD22y+0AsoabCMR6qqq1k9/FuImakdAQQsBnOqEEWyhbFHg1noss6elz1aYQrLgFoq/xPrLUtLRwFCumUALjj8zjK3HDb23g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RSNKh48xsRwwZA1FiXGpcnh8AN9M+E4i+jWPR+Ti71o=; b=NuRR0Z5s52vZ+NPhMQitXt5tF82CqQXKjJ5N0gUhQnzbLJcWVWqaqxLXwoqFLQ3IjJ+g8gle0BQlPWVQy33pRqJyW5Tue8CV2kswqCQf64+N+fMh6u/oE8udSz2t/KNlJgasOClYWfBK25AZPARHvbKxrmohCs6Ag/FVnNQTESed3c/9YvK/Wo1qfOZNLbT+gbXaF5i4z4L/DaI5Egrgpp1kelwTVicyPoMl+zTqK/XLctMkUZ+LKdafnSVEC5j9BrPwGXylF7Msiy4s+W2zFq52YHdgJbOE4FDZlck5mioL0jnrDnjoUp0fStZ6NTe1yhFypxlTPrjFwDJnMeLm7w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RSNKh48xsRwwZA1FiXGpcnh8AN9M+E4i+jWPR+Ti71o=; b=FjpdyFwpYJBB224GNJec50zs+LDLwpT9ivczsLlbOHKIBfs/qL8vPgOJJ1RpB3NNy0z3yctKP5M1h6GV9KiOinkf0jCXWLXpWTc0rMvLsNX+V7raet8VPed1WIG+8rKHLqQGuw/Vto1zTZUupdKt484EMJPpc3P+s0FYgEmsLSg= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1PR08MB4653.eurprd08.prod.outlook.com (10.255.27.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.17; Thu, 3 Oct 2019 19:49:09 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6%7]) with mapi id 15.20.2305.023; Thu, 3 Oct 2019 19:49:09 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "stephen@networkplumber.org" , "paulmck@linux.ibm.com" CC: "Wang, Yipeng1" , "Medvedkin, Vladimir" , "Ruifeng Wang (Arm Technology China)" , Dharmik Thakkar , "dev@dpdk.org" , nd , nd Thread-Topic: [PATCH v3 1/3] lib/ring: add peek API Thread-Index: AQHVeVEbQBVWnKnBNky5vPA5w4G8vadIeNrQ Date: Thu, 3 Oct 2019 19:49:09 +0000 Message-ID: References: <20190906094534.36060-1-ruifeng.wang@arm.com> <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20191001062917.35578-2-honnappa.nagarahalli@arm.com> <2601191342CEEE43887BDE71AB9772580191970014@irsmsx105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB9772580191970014@irsmsx105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 06c7dcf5-7f5b-42e8-8ab7-d487d85ab11b.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: f6306c3f-ae23-4fdc-4cf9-08d7483ac8a2 X-MS-Office365-Filtering-HT: Tenant X-MS-TrafficTypeDiagnostic: VE1PR08MB4653:|VE1PR08MB4653:|AM6PR08MB5239: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:7691;OLM:7691; x-forefront-prvs: 01792087B6 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(376002)(396003)(366004)(39850400004)(136003)(346002)(189003)(199004)(305945005)(71200400001)(76176011)(26005)(7696005)(74316002)(25786009)(2906002)(14454004)(6506007)(7736002)(5660300002)(186003)(256004)(14444005)(33656002)(64756008)(76116006)(55016002)(110136005)(54906003)(66556008)(66446008)(6246003)(66476007)(6436002)(486006)(86362001)(2201001)(9686003)(2501003)(52536014)(476003)(4326008)(316002)(66946007)(81166006)(8936002)(8676002)(3846002)(71190400001)(102836004)(81156014)(66066001)(478600001)(229853002)(446003)(6116002)(99286004)(11346002); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4653; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: bKcqcnItzFd+Qe8ItaJ3Tmwi4tXAxXlWW7FzZFlB/+MzWPFCgoWr9HWNnZxS+gaaat/SPDFonCWsNpmQ9ROtiChZxUPh9P+lCrO2SH316xfTPOWtEORF8izSvbsRmtOY8ATGjmx6bizX/APwABuKcceu4VysbCcntQIYmwpwqo2eDjxMGrZ3bv8Je0FeqMkGrNNVVrt7tkuaDiEZKfpUn4yzM35qPT97VuGZNO4SYuN60IjKFu022PmI4dOmbl0jXZwcG7H1xUsbKfzJZL4MFdPIG4vw/exXFlR/YMW4wIrOxhcwAv+D18vwp+HyaaTbtPghiSODhcSGu6KxJh+UixE4Bzg7xFuYL9tXxO3s82dVJAClLlU8CV58vJ4+YmyaY9Ptj969J1zqlKRytwxLtvKF39ACjDWMzHVh0uW9h2w= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4653 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(136003)(39860400002)(396003)(346002)(376002)(199004)(189003)(316002)(14454004)(356004)(2501003)(2906002)(66066001)(22756006)(47776003)(14444005)(2201001)(86362001)(25786009)(478600001)(26826003)(110136005)(52536014)(46406003)(33656002)(5660300002)(229853002)(7736002)(186003)(63350400001)(26005)(99286004)(7696005)(6506007)(74316002)(102836004)(11346002)(336012)(76176011)(50466002)(305945005)(9686003)(55016002)(476003)(486006)(126002)(6246003)(4326008)(446003)(70586007)(6116002)(3846002)(54906003)(23726003)(97756001)(81166006)(8936002)(8676002)(81156014)(76130400001)(70206006)(8746002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR08MB5239; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: edd503bc-e6a0-4fa7-612c-08d7483ac1d5 NoDisclaimer: True X-Forefront-PRVS: 01792087B6 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JuYKQsPnE2xXC4pjQJl4bou8b1SqYkE16X1YtJR+4WYgMmb6zWFEh0CIwHwQAd4Xjd8YhkQkP6q6i2p+GAChrcMSTpyY+BE7idB0SZ4S9upqOeICEe82VdKmmcwLW2V6+4MwS3K4PdKzEcdpzJEO0O05Zw+himtJ3YUOzWIOcvm7lk7nDDn5M5i6sRGxhdpXZI/DPPFsVMdjTrocpFMIs8vaYmcBURjT6YG2+tV9+KhGFyzNIo0vXF4C7pdoRhttDZ7mFb0aKzZZ+uBXXb6S9YgHpsegDiA6X6jipXqktDIwH/xH8rtdxjGHtqZv6LT+RTJAnlwmZlMH8MXccGdIbD3l18ZOjUgPJvgV55eH481qND+xp5L7A+T2mnccGTdlMWwcjT/Dffh13x2m0gVAlAim7ELM82ofonFlqHou/SE= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Oct 2019 19:49:20.7006 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f6306c3f-ae23-4fdc-4cf9-08d7483ac8a2 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5239 Subject: Re: [dpdk-dev] [PATCH v3 1/3] lib/ring: add peek API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > Subject: [PATCH v3 1/3] lib/ring: add peek API > > > > From: Ruifeng Wang > > > > The peek API allows fetching the next available object in the ring > > without dequeuing it. This helps in scenarios where dequeuing of > > objects depend on their value. > > > > Signed-off-by: Dharmik Thakkar > > Signed-off-by: Ruifeng Wang > > Reviewed-by: Honnappa Nagarahalli > > Reviewed-by: Gavin Hu > > --- > > lib/librte_ring/rte_ring.h | 30 ++++++++++++++++++++++++++++++ > > 1 file changed, 30 insertions(+) > > > > diff --git a/lib/librte_ring/rte_ring.h b/lib/librte_ring/rte_ring.h > > index 2a9f768a1..d3d0d5e18 100644 > > --- a/lib/librte_ring/rte_ring.h > > +++ b/lib/librte_ring/rte_ring.h > > @@ -953,6 +953,36 @@ rte_ring_dequeue_burst(struct rte_ring *r, void > **obj_table, > > r->cons.single, available); > > } > > > > +/** > > + * Peek one object from a ring. > > + * > > + * The peek API allows fetching the next available object in the ring > > + * without dequeuing it. This API is not multi-thread safe with > > +respect > > + * to other consumer threads. > > + * > > + * @param r > > + * A pointer to the ring structure. > > + * @param obj_p > > + * A pointer to a void * pointer (object) that will be filled. > > + * @return > > + * - 0: Success, object available > > + * - -ENOENT: Not enough entries in the ring. > > + */ > > +__rte_experimental > > +static __rte_always_inline int > > +rte_ring_peek(struct rte_ring *r, void **obj_p) >=20 > As it is not MT safe, then I think we need _sc_ in the name, to follow ot= her > rte_ring functions naming conventions > (rte_ring_sc_peek() or so). Agree >=20 > As a better alternative what do you think about introducing a serialized > versions of DPDK rte_ring dequeue functions? > Something like that: >=20 > /* same as original ring dequeue, but: > * 1) move cons.head only if cons.head =3D=3D const.tail > * 2) don't update cons.tail > */ > unsigned int > rte_ring_serial_dequeue_bulk(struct rte_ring *r, void **obj_table, unsign= ed > int n, > unsigned int *available); >=20 > /* sets both cons.head and cons.tail to cons.head + num */ void > rte_ring_serial_dequeue_finish(struct rte_ring *r, uint32_t num); >=20 > /* resets cons.head to const.tail value */ void > rte_ring_serial_dequeue_abort(struct rte_ring *r); >=20 > Then your dq_reclaim cycle function will look like that: >=20 > const uint32_t nb_elt =3D dq->elt_size/8 + 1; uint32_t avl, n; uintptr_t > elt[nb_elt]; ... >=20 > do { >=20 > /* read next elem from the queue */ > n =3D rte_ring_serial_dequeue_bulk(dq->r, elt, nb_elt, &avl); > if (n =3D=3D 0) > break; >=20 > /* wrong period, keep elem in the queue */ if (rte_rcu_qsbr_check(dr->v= , > elt[0]) !=3D 1) { > rte_ring_serial_dequeue_abort(dq->r); > break; > } >=20 > /* can reclaim, remove elem from the queue */ > rte_ring_serial_dequeue_finish(dr->q, nb_elt); >=20 > /*call reclaim function */ > dr->f(dr->p, elt); >=20 > } while (avl >=3D nb_elt); >=20 > That way, I think even rte_rcu_qsbr_dq_reclaim() can be MT safe. > As long as actual reclamation callback itself is MT safe of course. I think it is a great idea. The other writers would still be polling for th= e current writer to update the tail or update the head. This makes it a blo= cking solution. We can make the other threads not poll i.e. they will quit reclaiming if th= ey see that other writers are dequeuing from the queue. The other way is to= use per thread queues. The other requirement I see is to support unbounded-size data structures wh= ere in the data structures do not have a pre-determined number of entries. = Also, currently the defer queue size is equal to the total number of entrie= s in a given data structure. There are plans to support dynamically resizab= le defer queue. This means, memory allocation which will affect the lock-fr= ee-ness of the solution. So, IMO: 1) The API should provide the capability to support different algorithms - = may be through some flags? 2) The requirements for the ring are pretty unique to the problem we have h= ere (for ex: move the cons-head only if cons-tail is also the same, skip po= lling). So, we should probably implement a ring with-in the RCU library? >From the timeline perspective, adding all these capabilities would be diffi= cult to get done with in 19.11 timeline. What I have here satisfies my curr= ent needs. I suggest that we make provisions in APIs now to support all the= se features, but do the implementation in the coming releases. Does this so= und ok for you? >=20 > > +{ > > + uint32_t prod_tail =3D r->prod.tail; > > + uint32_t cons_head =3D r->cons.head; > > + uint32_t count =3D (prod_tail - cons_head) & r->mask; > > + unsigned int n =3D 1; > > + if (count) { > > + DEQUEUE_PTRS(r, &r[1], cons_head, obj_p, n, void *); > > + return 0; > > + } > > + return -ENOENT; > > +} > > + > > #ifdef __cplusplus > > } > > #endif > > -- > > 2.17.1