From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D8FF145965; Thu, 19 Sep 2024 19:03:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6F9084339E; Thu, 19 Sep 2024 19:03:29 +0200 (CEST) Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) by mails.dpdk.org (Postfix) with ESMTP id BE86143344 for ; Thu, 19 Sep 2024 19:03:28 +0200 (CEST) Received: by mail-qt1-f177.google.com with SMTP id d75a77b69052e-45816db2939so7308731cf.3 for ; Thu, 19 Sep 2024 10:03:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726765408; x=1727370208; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=5DwvO1mk/OfJANGeuSnId/tFA2HE9cootZYuI02XMds=; b=bbj0SDqnpER0tstdk7+kOTvVTzJNMr7OC2hANu47qVKb4BvmowJR7QRNetF15v4PAV HHHheW31PJ/BMD/sFnXR9eXXKDw4sVuHib/dogyVW+aLcHEIaaxsJXmg8Cy+/jncvMEL TOZfoh11WsZihI/U1QMREzoc8DWAPG+0ycS+dU+QYhQhK6ThgMrS95oA+52IxgyixgUl 1oWf1g1CfxLXDqTdwrr1OsJoALoOGXT1e2Swcp+jYA+tw73OcYA+fD48JauVpjDp5CB7 rMoF9/5MceZEwjLIblB6Ht8DpdyYS3lg8YAsJbL4ZyYIPW7GlaHRBvKQNz0rq3ukwixb rodg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726765408; x=1727370208; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5DwvO1mk/OfJANGeuSnId/tFA2HE9cootZYuI02XMds=; b=NX4TBuKVgBHrSIxwZdww0IFyG1+9H/ScgE4vt56t09Z+5kslgw0UNzUKz8rVDb/HzM 4qJl8nLOz7q4Mfk/Em/dggvJxKmHtFoQPm+HNmT+xAHIB8l+MyNjZNg0wVaj9mr1UT3y J7KHK6HVc8NlmqaLBXpCY0hlxumRb6L+CQaoMOWVREXYDEQQ3M94+tkSlPJ4fZtTJsOL sXpqQhBoCkL91Lrgy4+etu8kIxX8yNdZKa1tuAP28rDXdfBR4Y2354hQXjJkXcNx5ZmN cEz476DhIQM8pMUrJ5BfSR8DlfQBzBQ9rqtMJxPUFUJ7A7uTW9n8ROYZFT1vws05OcuX yLNQ== X-Gm-Message-State: AOJu0Yx/nmb/jjLrd9B4/hdzvBTQqWtduT05IrfRI5hAHTNCCJM02pmO NV8Kx+G7i/PAHS4hqjbIpfP08+dGIXO08xjcbA8Dia+WJdqOeLu3a22/sPWmxk9MWpsm6k3/LRb U30K5f3dhMoseyMzM7t5RhwGBD3U= X-Google-Smtp-Source: AGHT+IEXyarXMizgt/aYLa33mMHeXboUIoSlxfsZaTbtdMEg8KamTI7mZCGaOgdi4MvWBC6p45vdko5M9xbitQ/Khm8= X-Received: by 2002:ac8:7d0f:0:b0:456:937e:2f76 with SMTP id d75a77b69052e-45b20556ca5mr2693001cf.39.1726765407620; Thu, 19 Sep 2024 10:03:27 -0700 (PDT) MIME-Version: 1.0 References: <20240916123733.994-1-konstantin.v.ananyev@yandex.ru> <20240917120946.1212-1-konstantin.v.ananyev@yandex.ru> <20240917120946.1212-5-konstantin.v.ananyev@yandex.ru> In-Reply-To: <20240917120946.1212-5-konstantin.v.ananyev@yandex.ru> From: Jerin Jacob Date: Thu, 19 Sep 2024 22:33:00 +0530 Message-ID: Subject: Re: [PATCH v4 4/5] ring/soring: introduce Staged Ordered Ring To: Konstantin Ananyev Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, jerinj@marvell.com, hemant.agrawal@nxp.com, bruce.richardson@intel.com, drc@linux.vnet.ibm.com, ruifeng.wang@arm.com, mb@smartsharesystems.com, eimear.morrissey@huawei.com, Konstantin Ananyev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Sep 17, 2024 at 5:49=E2=80=AFPM Konstantin Ananyev wrote: > > From: Konstantin Ananyev > > Staged-Ordered-Ring (SORING) provides a SW abstraction for 'ordered' queu= es > with multiple processing 'stages'. > It is based on conventional DPDK rte_ring, re-uses many of its concepts, > and even substantial part of its code. > It can be viewed as an 'extension' of rte_ring functionality. > In particular, main SORING properties: > - circular ring buffer with fixed size objects > - producer, consumer plus multiple processing stages in the middle. > - allows to split objects processing into multiple stages. > - objects remain in the same ring while moving from one stage to the othe= r, > initial order is preserved, no extra copying needed. > - preserves the ingress order of objects within the queue across multiple > stages, i.e.: > at the same stage multiple threads can process objects from the ring in > any order, but for the next stage objects will always appear in the > original order. > - each stage (and producer/consumer) can be served by single and/or > multiple threads. > - number of stages, size and number of objects in the ring are > configurable at ring initialization time. > > Data-path API provides four main operations: > - enqueue/dequeue works in the same manner as for conventional rte_ring, > all rte_ring synchronization types are supported. > - acquire/release - for each stage there is an acquire (start) and > release (finish) operation. > after some objects are 'acquired' - given thread can safely assume that > it has exclusive possession of these objects till 'release' for them is > invoked. > Note that right now user has to release exactly the same number of > objects that was acquired before. > After 'release', objects can be 'acquired' by next stage and/or dequeue= d > by the consumer (in case of last stage). > > Expected use-case: applications that uses pipeline model > (probably with multiple stages) for packet processing, when preserving > incoming packet order is important. I.E.: IPsec processing, etc. > > Signed-off-by: Eimear Morrissey > Signed-off-by: Konstantin Ananyev > --- > doc/guides/rel_notes/release_24_11.rst | 8 + > lib/ring/meson.build | 4 +- > lib/ring/rte_soring.c | 182 ++++++++ > lib/ring/rte_soring.h | 543 ++++++++++++++++++++++++ > lib/ring/soring.c | 548 +++++++++++++++++++++++++ > lib/ring/soring.h | 124 ++++++ > lib/ring/version.map | 19 + > 7 files changed, 1426 insertions(+), 2 deletions(-) > create mode 100644 lib/ring/rte_soring.c > create mode 100644 lib/ring/rte_soring.h > create mode 100644 lib/ring/soring.c > create mode 100644 lib/ring/soring.h Good feature and makes sense to not make as eventdev driver. # I think, it is worth updating doc/guides/prog_guide/ring_lib.rst for this new type and means to use it. # Missing rte_soring.h update in doc/api/doxy-api-index.md > diff --git a/lib/ring/rte_soring.h b/lib/ring/rte_soring.h > new file mode 100644 > index 0000000000..9c35b4a18c > --- /dev/null > +++ b/lib/ring/rte_soring.h > @@ -0,0 +1,543 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2024 Huawei Technologies Co., Ltd > + */ > + > +#ifndef _RTE_SORING_H_ > +#define _RTE_SORING_H_ > + > +/** > + * @file > + * This file contains definition of RTE soring (Staged Ordered Ring) pub= lic API. > + * Brief description: > + * enqueue/dequeue works the same as for conventional rte_ring: > + * any rte_ring sync types can be used, etc. > + * Plus there could be multiple 'stages'. > + * For each stage there is an acquire (start) and release (finish) opera= tion. > + * after some elems are 'acquired' - user can safely assume that he has > + * exclusive possession of these elems till 'release' for them is done. > + * Note that right now user has to release exactly the same number of el= ems > + * he acquired before. > + * After 'release', elems can be 'acquired' by next stage and/or dequeue= d > + * (in case of last stage). > + */ > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include > + > +/* upper 2 bits are used for status */ Add Doxygen comment for public symbol > +#define RTE_SORING_ST_BIT 30 > + > +/* max possible number of elements in the soring */ Add Doxygen comment for public symbol > +#define RTE_SORING_ELEM_MAX (RTE_BIT32(RTE_SORING_ST_BIT) - 1) > + > +struct rte_soring_param { > + /** expected name of the ring */ > + const char *name; > + /** number of elemnts in the ring */ > + uint32_t elems; > + /** size of elements in the ring, must be a multiple of 4 */ > + uint32_t elem_size; > + /** > + * size of metadata for each elem, must be a multiple of 4. > + * This parameter defines a size of supplementary and optional > + * array of metadata associated with each object in the soring. > + * While element size is configurable (see 'elem_size' parameter = above), > + * so user can specify it big enough to hold both object and its > + * metadata together, for performance reasons it might be plausib= le > + * to access them as separate arrays. > + * Common usage scenario when such separation helps: > + * enqueue() - writes to objects array > + * acquire() - reads from objects array > + * release() - writes to metadata array (as an example: return co= de) > + * dequeue() - reads both objects and metadata array > + */ > + uint32_t meta_size; > + /** number of stages in the ring */ > + uint32_t stages; > + /** sync type for producer */ > + enum rte_ring_sync_type prod_synt; > + /** sync type for consumer */ > + enum rte_ring_sync_type cons_synt; > +}; > + > +struct rte_soring; > + > +/** > + * Calculate the memory size needed for a soring > + * > + * This function returns the number of bytes needed for a ring, given > + * the expected parameters for it. This value is the sum of the size of > + * the internal metadata and the size of the memory needed by the > + * actual ring elements and theri rec-codes. The value is aligned to a c= ache typo - theri > + * line size. > + * > + * @param prm > + * Pointer to the structure that contains soring creation paramers. > + * @return > + * - The memory size needed for the soring on success. > + * - -EINVAL if provided paramer values are invalid. > + */ > +__rte_experimental > +ssize_t > +rte_soring_get_memsize(const struct rte_soring_param *prm); > + > +/** > + * Initialize a soring structure. > + * > + * Initialize a soring structure in memory pointed by "r". > + * The size of the memory area must be large enough to store the soring > + * internal structures plus the objects and ret-code tables. typo- ret-code > + * It is strongly advised to use rte_ring_get_memsize() to get the > + * appropriate size. > + * > + * @param r > + * Pointer to the soring structure. I think, you can also mention, memory allocated with size provided by rte_soring_get_memsize(). Also add @see to cross-reference. > + * @param prm > + * Pointer to the structure that contains soring creation paramers. > + * @return > + * - 0 on success, or a negative error code. > + */ > +__rte_experimental > +int > +rte_soring_init(struct rte_soring *r, const struct rte_soring_param *pr= m); > + > +/** > + * Return the total number of filled entries in a ring. Across the patch, a ring can be replaced by a soring ring > + * > + * @param r > + * A pointer to the soring structure. > + * @return > + * The number of entries in the ring. > + */ > +__rte_experimental > +unsigned int > +rte_soring_count(const struct rte_soring *r); > + > +/** > + * Return the total number of unfilled entries in a ring. > + * > + * @param r > + * A pointer to the soring structure. > + * @return > + * The number of free entries in the ring. > + */ > +__rte_experimental > +unsigned int > +rte_soring_free_count(const struct rte_soring *r); > + > +/** > + * Dump the status of the soring > + * > + * @param f > + * A pointer to a file for output > + * @param r > + * Pointer to the soring structure. > + */ > +__rte_experimental > +void > +rte_soring_dump(FILE *f, const struct rte_soring *r); > + > +/** > + * Enqueue several objects on the ring. > + * > + * @param r > + * A pointer to the soring structure. > + * @param objs > + * A pointer to an array of objects to enqueue. > + * Size of objects to enqueue must be the same value as 'elem_size' pa= rameter > + * used while creating the ring. Otherwise the results are undefined. > + * @param n > + * The number of objects to add in the ring from the 'objs'. > + * @param free_space > + * if non-NULL, returns the amount of space in the ring after the > + * enqueue operation has finished. > + * @return > + * - Actual number of objects enqueued, either 0 or @n. > + */ > +__rte_experimental > +uint32_t > +rte_soring_enqueue_bulk(struct rte_soring *r, const void *objs, > + uint32_t n, uint32_t *free_space); > + > +/** > + * Enqueue several objects plus metadata on the ring. > + * > + * @param r > + * A pointer to the soring structure. > + * @param objs > + * A pointer to an array of objects to enqueue. > + * Size of objects to enqueue must be the same value as 'elem_size' pa= rameter > + * used while creating the ring. Otherwise the results are undefined. > + * @param meta > + * A pointer to an array of metadata values for each object to enqueue= . > + * Note that if user not using object metadata values, then this param= eter > + * can be NULL. > + * Size of elements in this array must be the same value as 'meta_size= ' > + * parameter used while creating the ring. If user created the soring = with > + * 'meta_size' value equals zero, then 'meta' parameter should be NULL= . > + * Otherwise the results are undefined. > + * @param n > + * The number of objects to add in the ring from the 'objs'. > + * @param free_space > + * if non-NULL, returns the amount of space in the ring after the > + * enqueue operation has finished. > + * @return > + * - Actual number of objects enqueued, either 0 or @n. > + */ > +__rte_experimental > +uint32_t > +rte_soring_enqueux_bulk(struct rte_soring *r, const void *objs, > + const void *meta, uint32_t n, uint32_t *free_space); > + > +/** > + * Enqueue several objects on the ring. Across the patch, _burst and _bulk has the same API comment > + * > + * @param r > + * A pointer to the soring structure. > + * @param objs > + * A pointer to an array of objects to enqueue. > + * Size of objects to enqueue must be the same value as 'elem_size' pa= rameter > + * used while creating the ring. Otherwise the results are undefined. > + * @param n > + * The number of objects to add in the ring from the 'objs'. > + * @param free_space > + * if non-NULL, returns the amount of space in the ring after the > + * enqueue operation has finished. > + * @return > + * - Actual number of objects enqueued. > + */ > +__rte_experimental > +uint32_t > +rte_soring_enqueue_burst(struct rte_soring *r, const void *objs, > + uint32_t n, uint32_t *free_space); > + > +/** > + * Acquire several objects from the ring for given stage. > + * > + * @param r > + * A pointer to the soring structure. > + * @param objs > + * A pointer to an array of objects to acquire. > + * Size of objects must be the same value as 'elem_size' parameter > + * used while creating the ring. Otherwise the results are undefined. > + * @param stage > + * Stage to acquire objects for. > + * @param num > + * The number of objects to acquire. > + * @param ftoken > + * Pointer to the opaque 'token' value used by release() op. > + * User has to store this value somewhere, and later provide to the > + * release(). > + * @param available > + * If non-NULL, returns the number of remaining ring entries for given= stage > + * after the acquire has finished. > + * @return > + * - Actual number of objects acquired, either 0 or 'num'. > + */ > +__rte_experimental > +uint32_t > +rte_soring_acquire_bulk(struct rte_soring *r, void *objs, > + uint32_t stage, uint32_t num, uint32_t *ftoken, uint32_t *availab= le); Does stage needs to be uint32_t?