From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5D09E457D5; Thu, 15 Aug 2024 15:22:14 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 12EF0427E6; Thu, 15 Aug 2024 15:22:14 +0200 (CEST) Received: from dkmailrelay1.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id AE2884027E for ; Thu, 15 Aug 2024 15:22:12 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesys.local [192.168.4.10]) by dkmailrelay1.smartsharesystems.com (Postfix) with ESMTP id 7FF2D20EE7; Thu, 15 Aug 2024 15:22:12 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: [RFC 3/6] ring/soring: introduce Staged Ordered Ring Date: Thu, 15 Aug 2024 15:22:07 +0200 X-MimeOLE: Produced By Microsoft Exchange V6.5 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35E9F642@smartserver.smartshare.dk> In-Reply-To: <71059883ae384f798713361c3ffaa0f2@huawei.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [RFC 3/6] ring/soring: introduce Staged Ordered Ring Thread-Index: AQHa7vC1mkB/pilhbkGpjKPW9f7ZHLIoCHoAgAA2zGCAAAfEYA== References: <20240815085339.1434-1-konstantin.v.ananyev@yandex.ru><20240815085339.1434-4-konstantin.v.ananyev@yandex.ru> <98CBD80474FA8B44BF855DF32C47DC35E9F641@smartserver.smartshare.dk> <71059883ae384f798713361c3ffaa0f2@huawei.com> From: =?iso-8859-1?Q?Morten_Br=F8rup?= To: "Konstantin Ananyev" , "Konstantin Ananyev" , Cc: , , , , , X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > From: Konstantin Ananyev [mailto:konstantin.ananyev@huawei.com] >=20 > > > From: Konstantin Ananyev > > > > > > Staged-Ordered-Ring (SORING) provides a SW abstraction for = 'ordered' > queues > > > with multiple processing 'stages'. > > > It is based on conventional DPDK rte_ring, re-uses many of its = concepts, > > > and even substantial part of its code. > > > It can be viewed as an 'extension' of rte_ring functionality. > > > In particular, main SORING properties: > > > - circular ring buffer with fixed size objects > > > - producer, consumer plus multiple processing stages in the = middle. > > > - allows to split objects processing into multiple stages. > > > - objects remain in the same ring while moving from one stage to = the > other, > > > initial order is preserved, no extra copying needed. > > > - preserves the ingress order of objects within the queue across = multiple > > > stages, i.e.: > > > at the same stage multiple threads can process objects from the = ring in > > > any order, but for the next stage objects will always appear in = the > > > original order. > > > - each stage (and producer/consumer) can be served by single = and/or > > > multiple threads. > > > - number of stages, size and number of objects in the ring are > > > configurable at ring initialization time. > > > > > > Data-path API provides four main operations: > > > - enqueue/dequeue works in the same manner as for conventional = rte_ring, > > > all rte_ring synchronization types are supported. > > > - acquire/release - for each stage there is an acquire (start) and > > > release (finish) operation. > > > after some objects are 'acquired' - given thread can safely = assume that > > > it has exclusive possession of these objects till 'release' for = them is > > > invoked. > > > Note that right now user has to release exactly the same number = of > > > objects that was acquired before. > > > After 'release', objects can be 'acquired' by next stage and/or = dequeued > > > by the consumer (in case of last stage). > > > > > > Expected use-case: applications that uses pipeline model > > > (probably with multiple stages) for packet processing, when = preserving > > > incoming packet order is important. I.E.: IPsec processing, etc. > > > > > > Signed-off-by: Konstantin Ananyev > > > --- > > > > The existing RING library is for a ring of objects. > > > > It is very confusing that the new SORING library is for a ring of = object > pairs (obj, objst). > > > > The new SORING library should be for a ring of objects, like the = existing > RING library. Please get rid of all the objst stuff. > > > > This might also improve performance when not using the optional = secondary > object. > > > > > > With that in place, you can extend the SORING library with = additional APIs > for object pairs. > > > > I suggest calling the secondary object "metadata" instead of = "status" or > "state" or "ret-value". > > I agree that data passed as {obj[num], meta[num]} is more efficient = than > {obj, meta}[num] in some use cases, which is why your API > > uses two vector pointers instead of one. >=20 > I suppose what you suggest is to have 2 set of functions: one that = takes both > objs[] and meta[] and second that takes just objs[]? > If so, yes I can do that - in fact I was thinking about same thing. Yes, please. Mainly for readability/familiarity; it makes the API much more similar = to the Ring API. > BTW, right now meta[] is an optional one anyway. I noticed that meta[] is optional, but it is confusing that the APIs are = so much different than the Ring APIs. With two sets of functions, the basic set will resemble the Ring APIs = much more. > Also will probably get rid of explicit 'behavior' and will have = '_burst_' and > '_bulk_' versions instead, > same as rte_ring. +1 >=20 > > > > Furthermore, you should consider semi-zero-copy APIs for the > "acquire"/"release" functions: > > > > The "acquire" function can use a concept similar to = rte_pktmbuf_read(), > where a vector is provided for copying (if the ring wraps), and > > the return value either points directly to the objects in the ring = (zero- > copy), or to the vector where the objects were copied to. >=20 > You mean to introduce analog of rte_ring '_zc_' functions? > Yes, I considered that, but decided to leave it for the future. Somewhat similar, but I think the (semi-)zero-copy "acquire"/"release" = APIs will be simpler than the rte_ring's _zc_ functions because we know = that no other thread can dequeue the objects out of the ring before the = processing stage has released them, i.e. no additional locking is = required. Anyway, leave it for the future. I don't think it will require changes to the underlying implementation, = so we don't need to consider it in advance. > First, because we do need a generic and simple function with copying = things > anyway. > Second I am not so convinced that this _zc_ will give much performance = gain, > while it definitely makes API not that straightforward. >=20 > > And the "release" function does not need to copy the object vector = back if > the "acquire" function returned a zero-copy pointer. >=20 > For "release" you don't need to *always* copy objs[] and meta[]. > It is optional and is left for the user to decide based on the = use-case. > If he doesn't need to update objs[] or meta[] he can just pass a NULL = ptr > here. Yes, I noticed. I'm mentioning that zero-copy adds a "release" case where copying back = the modified object vector is not required; when the processing stage = has modified the object vector (e.g. replaced some objects) directly in = the ring, and not in a copy returned by "acquire". >=20 >=20