From: Stephen Hemminger <stephen@networkplumber.org>
To: Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>
Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, jerinj@marvell.com,
hemant.agrawal@nxp.com, bruce.richardson@intel.com,
drc@linux.vnet.ibm.com, ruifeng.wang@arm.com,
mb@smartsharesystems.com, eimear.morrissey@huawei.com,
Konstantin Ananyev <konstantin.ananyev@huawei.com>
Subject: Re: [PATCH v5 0/6] Stage-Ordered API and other extensions for ring library
Date: Tue, 15 Oct 2024 08:59:55 -0700 [thread overview]
Message-ID: <20241015085955.16540ecb@hermes.local> (raw)
In-Reply-To: <20241015130111.826-1-konstantin.v.ananyev@yandex.ru>
On Tue, 15 Oct 2024 14:01:05 +0100
Konstantin Ananyev <konstantin.v.ananyev@yandex.ru> wrote:
> From: Konstantin Ananyev <konstantin.ananyev@huawei.com>
>
> NOTE UPFRONT: this version is still not ready for merging.
> Missing items:
> - ARM/PPC tests passing
> - PG update
>
> v4 -> v5
> - fix public API/doc comments from Jerin
> - update devtools/build-dict.sh (Stephen)
> - fix MSVC warnings
> - introduce new test-suite for meson (stress) with
> ring_stress_autotest and soring_stress_autotest in it
> - enhance error report in tests
> - reorder some sync code in soring and add extra checks
> (for better debuggability)
>
> v3 -> v4:
> - fix compilation/doxygen complains (attempt #2)
> - updated release notes
>
> v2 -> v3:
> - fix compilation/doxygen complains
> - dropped patch:
> "examples/l3fwd: make ACL work in pipeline and eventdev modes": [2]
> As was mentioned in the patch desctiption it was way too big,
> controversial and incomplete. If the community is ok to introduce
> pipeline model into the l3fwd, then it is propbably worth to be
> a separate patch series.
>
> v1 -> v2:
> - rename 'elmst/objst' to 'meta' (Morten)
> - introduce new data-path APIs set: one with both meta{} and objs[],
> second with just objs[] (Morten)
> - split data-path APIs into burst/bulk flavours (same as rte_ring)
> - added dump function for te_soring and improved dump() for rte_ring.
> - dropped patch:
> " ring: minimize reads of the counterpart cache-line"
> - no performance gain observed
> - actually it does change behavior of conventional rte_ring
> enqueue/dequeue APIs -
> it could return available/free less then actually exist in the ring.
> As in some other libs we reliy on that information - it will
> introduce problems.
>
> The main aim of these series is to extend ring library with
> new API that allows user to create/use Staged-Ordered-Ring (SORING)
> abstraction. In addition to that there are few other patches that serve
> different purposes:
> - first two patches are just code reordering to de-duplicate
> and generalize existing rte_ring code.
> - patch #3 extends rte_ring_dump() to correctly print head/tail metadata
> for different sync modes.
> - next two patches introduce SORING API into the ring library and
> provide UT for it.
>
> SORING overview
> ===============
> Staged-Ordered-Ring (SORING) provides a SW abstraction for 'ordered' queues
> with multiple processing 'stages'. It is based on conventional DPDK
> rte_ring, re-uses many of its concepts, and even substantial part of
> its code.
> It can be viewed as an 'extension' of rte_ring functionality.
> In particular, main SORING properties:
> - circular ring buffer with fixed size objects
> - producer, consumer plus multiple processing stages in between.
> - allows to split objects processing into multiple stages.
> - objects remain in the same ring while moving from one stage to the other,
> initial order is preserved, no extra copying needed.
> - preserves the ingress order of objects within the queue across multiple
> stages
> - each stage (and producer/consumer) can be served by single and/or
> multiple threads.
>
> - number of stages, size and number of objects in the ring are
> configurable at ring initialization time.
>
> Data-path API provides four main operations:
> - enqueue/dequeue works in the same manner as for conventional rte_ring,
> all rte_ring synchronization types are supported.
> - acquire/release - for each stage there is an acquire (start) and
> release (finish) operation. After some objects are 'acquired' -
> given thread can safely assume that it has exclusive ownership of
> these objects till it will invoke 'release' for them.
> After 'release', objects can be 'acquired' by next stage and/or dequeued
> by the consumer (in case of last stage).
>
> Expected use-case: applications that uses pipeline model
> (probably with multiple stages) for packet processing, when preserving
> incoming packet order is important.
>
> The concept of ‘ring with stages’ is similar to DPDK OPDL eventdev PMD [1],
> but the internals are different.
> In particular, SORING maintains internal array of 'states' for each element
> in the ring that is shared by all threads/processes that access the ring.
> That allows 'release' to avoid excessive waits on the tail value and helps
> to improve performancei and scalability.
> In terms of performance, with our measurements rte_soring and
> conventional rte_ring provide nearly identical numbers.
> As an example, on our SUT: Intel ICX CPU @ 2.00GHz,
> l3fwd (--lookup=acl) in pipeline mode [2] both
> rte_ring and rte_soring reach ~20Mpps for single I/O lcore and same
> number of worker lcores.
>
> [1] https://www.dpdk.org/wp-content/uploads/sites/35/2018/06/DPDK-China2017-Ma-OPDL.pdf
> [2] https://patchwork.dpdk.org/project/dpdk/patch/20240906131348.804-7-konstantin.v.ananyev@yandex.ru/
>
> Eimear Morrissey (1):
> ring: make dump function more verbose
>
> Konstantin Ananyev (5):
> ring: common functions for 'move head' ops
> ring: make copying functions generic
> ring/soring: introduce Staged Ordered Ring
> app/test: add unit tests for soring API
> test: add stress test suite
>
> .mailmap | 1 +
> app/test/meson.build | 3 +
> app/test/suites/meson.build | 10 +
> app/test/test.h | 1 +
> app/test/test_ring_stress.c | 2 +-
> app/test/test_ring_stress_impl.h | 1 +
> app/test/test_soring.c | 442 +++++++++++++
> app/test/test_soring_mt_stress.c | 40 ++
> app/test/test_soring_stress.c | 48 ++
> app/test/test_soring_stress.h | 35 ++
> app/test/test_soring_stress_impl.h | 834 +++++++++++++++++++++++++
> devtools/build-dict.sh | 1 +
> doc/api/doxy-api-index.md | 1 +
> doc/guides/rel_notes/release_24_11.rst | 8 +
> lib/ring/meson.build | 4 +-
> lib/ring/rte_ring.c | 87 ++-
> lib/ring/rte_ring.h | 15 +
> lib/ring/rte_ring_c11_pvt.h | 134 +---
> lib/ring/rte_ring_elem_pvt.h | 181 ++++--
> lib/ring/rte_ring_generic_pvt.h | 121 +---
> lib/ring/rte_ring_hts_elem_pvt.h | 85 +--
> lib/ring/rte_ring_rts_elem_pvt.h | 85 +--
> lib/ring/rte_soring.c | 182 ++++++
> lib/ring/rte_soring.h | 555 ++++++++++++++++
> lib/ring/soring.c | 561 +++++++++++++++++
> lib/ring/soring.h | 124 ++++
> lib/ring/version.map | 26 +
> 27 files changed, 3190 insertions(+), 397 deletions(-)
> create mode 100644 app/test/test_soring.c
> create mode 100644 app/test/test_soring_mt_stress.c
> create mode 100644 app/test/test_soring_stress.c
> create mode 100644 app/test/test_soring_stress.h
> create mode 100644 app/test/test_soring_stress_impl.h
> create mode 100644 lib/ring/rte_soring.c
> create mode 100644 lib/ring/rte_soring.h
> create mode 100644 lib/ring/soring.c
> create mode 100644 lib/ring/soring.h
>
And some build failures
####################################################################################
#### [Begin job log] "ubuntu-22.04-gcc-mini" at step Build and test
####################################################################################
../lib/eal/include/rte_bitops.h:1481:9: note: in expansion of macro ‘__RTE_BIT_OVERLOAD_SZ_4R’
1481 | __RTE_BIT_OVERLOAD_SZ_4R(family, fun, qualifier, 64, ret_type, arg1_type, arg1_name, \
| ^~~~~~~~~~~~~~~~~~~~~~~~
../lib/eal/include/rte_bitops.h:1497:1: note: in expansion of macro ‘__RTE_BIT_OVERLOAD_4R’
1497 | __RTE_BIT_OVERLOAD_4R(atomic_, test_and_assign,, bool, unsigned int, nr, bool, value,
| ^~~~~~~~~~~~~~~~~~~~~
../lib/eal/include/rte_bitops.h:1463:1: note: previous declaration ‘bool rte_bit_atomic_test_and_assign(uint32_t*, unsigned int, bool, int)’
1463 | rte_bit_ ## family ## fun(qualifier uint ## size ## _t *addr, arg1_type arg1_name, \
| ^~~~~~~~
../lib/eal/include/rte_bitops.h:1472:9: note: in expansion of macro ‘__RTE_BIT_OVERLOAD_V_4R’
1472 | __RTE_BIT_OVERLOAD_V_4R(family,, fun, qualifier, size, ret_type, arg1_type, arg1_name, \
| ^~~~~~~~~~~~~~~~~~~~~~~
../lib/eal/include/rte_bitops.h:1479:9: note: in expansion of macro ‘__RTE_BIT_OVERLOAD_SZ_4R’
1479 | __RTE_BIT_OVERLOAD_SZ_4R(family, fun, qualifier, 32, ret_type, arg1_type, arg1_name, \
| ^~~~~~~~~~~~~~~~~~~~~~~~
../lib/eal/include/rte_bitops.h:1497:1: note: in expansion of macro ‘__RTE_BIT_OVERLOAD_4R’
1497 | __RTE_BIT_OVERLOAD_4R(atomic_, test_and_assign,, bool, unsigned int, nr, bool, value,
| ^~~~~~~~~~~~~~~~~~~~~
[847/912] Compiling C++ object 'buildtools/chkincs/fe389a9@@chkincs-cpp at exe/meson-generated_rte_mbuf_dyn.cpp.o'.
[848/912] Compiling C++ object 'buildtools/chkincs/fe389a9@@chkincs-cpp at exe/meson-generated_rte_mempool.cpp.o'.
[849/912] Compiling C++ object 'buildtools/chkincs/fe389a9@@chkincs-cpp at exe/meson-generated_rte_mempool_trace_fp.cpp.o'.
[850/912] Compiling C++ object 'buildtools/chkincs/fe389a9@@chkincs-cpp at exe/meson-generated_rte_mbuf.cpp.o'.
[851/912] Compiling C object 'app/a172ced@@dpdk-test at exe/test_test_memcpy_perf.c.o'.
ninja: build stopped: subcommand failed.
##[error]Process completed with exit code 1.
####################################################################################
#### [End job log] "ubuntu-22.04-gcc-mini" at step Build and test
####################################################################################
next prev parent reply other threads:[~2024-10-15 16:00 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-15 8:53 [RFC " Konstantin Ananyev
2024-08-15 8:53 ` [RFC 1/6] ring: common functions for 'move head' ops Konstantin Ananyev
2024-08-15 8:53 ` [RFC 2/6] ring: make copying functions generic Konstantin Ananyev
2024-08-15 8:53 ` [RFC 3/6] ring/soring: introduce Staged Ordered Ring Konstantin Ananyev
2024-08-15 11:11 ` Morten Brørup
2024-08-15 12:41 ` Konstantin Ananyev
2024-08-15 13:22 ` Morten Brørup
2024-08-26 19:04 ` Mattias Rönnblom
2024-09-03 13:55 ` Konstantin Ananyev
2024-08-15 8:53 ` [RFC 4/6] app/test: add unit tests for soring API Konstantin Ananyev
2024-08-15 8:53 ` [RFC 5/6] examples/l3fwd: make ACL work in pipeline and eventdev modes Konstantin Ananyev
2024-08-15 8:53 ` [RFC 6/6] ring: minimize reads of the counterpart cache-line Konstantin Ananyev
2024-09-06 13:13 ` [RFCv2 0/6] Stage-Ordered API and other extensions for ring library Konstantin Ananyev
2024-09-06 13:13 ` [RFCv2 1/6] ring: common functions for 'move head' ops Konstantin Ananyev
2024-09-06 13:13 ` [RFCv2 2/6] ring: make copying functions generic Konstantin Ananyev
2024-09-06 13:13 ` [RFCv2 3/6] ring: make dump function more verbose Konstantin Ananyev
2024-09-06 13:13 ` [RFCv2 4/6] ring/soring: introduce Staged Ordered Ring Konstantin Ananyev
2024-09-06 13:13 ` [RFCv2 5/6] app/test: add unit tests for soring API Konstantin Ananyev
2024-09-06 13:13 ` [RFCv2 6/6] examples/l3fwd: make ACL work in pipeline and eventdev modes Konstantin Ananyev
2024-09-16 12:37 ` [PATCH v3 0/5] Stage-Ordered API and other extensions for ring library Konstantin Ananyev
2024-09-16 12:37 ` [PATCH v3 1/5] ring: common functions for 'move head' ops Konstantin Ananyev
2024-09-16 12:37 ` [PATCH v3 2/5] ring: make copying functions generic Konstantin Ananyev
2024-09-16 12:37 ` [PATCH v3 3/5] ring: make dump function more verbose Konstantin Ananyev
2024-09-16 12:37 ` [PATCH v3 4/5] ring/soring: introduce Staged Ordered Ring Konstantin Ananyev
2024-09-16 12:37 ` [PATCH v3 5/5] app/test: add unit tests for soring API Konstantin Ananyev
2024-09-17 12:09 ` [PATCH v4 0/5] Stage-Ordered API and other extensions for ring library Konstantin Ananyev
2024-09-17 12:09 ` [PATCH v4 1/5] ring: common functions for 'move head' ops Konstantin Ananyev
2024-09-17 12:09 ` [PATCH v4 2/5] ring: make copying functions generic Konstantin Ananyev
2024-09-17 12:09 ` [PATCH v4 3/5] ring: make dump function more verbose Konstantin Ananyev
2024-09-17 12:09 ` [PATCH v4 4/5] ring/soring: introduce Staged Ordered Ring Konstantin Ananyev
2024-09-19 17:03 ` Jerin Jacob
2024-09-17 12:09 ` [PATCH v4 5/5] app/test: add unit tests for soring API Konstantin Ananyev
2024-10-12 18:09 ` [PATCH v4 0/5] Stage-Ordered API and other extensions for ring library Stephen Hemminger
2024-10-15 13:01 ` [PATCH v5 0/6] " Konstantin Ananyev
2024-10-15 13:01 ` [PATCH v5 1/6] ring: common functions for 'move head' ops Konstantin Ananyev
2024-10-15 15:04 ` Morten Brørup
2024-10-15 13:01 ` [PATCH v5 2/6] ring: make copying functions generic Konstantin Ananyev
2024-10-15 13:01 ` [PATCH v5 3/6] ring: make dump function more verbose Konstantin Ananyev
2024-10-15 13:01 ` [PATCH v5 4/6] ring/soring: introduce Staged Ordered Ring Konstantin Ananyev
2024-10-15 13:01 ` [PATCH v5 5/6] app/test: add unit tests for soring API Konstantin Ananyev
2024-10-15 13:01 ` [PATCH v5 6/6] test: add stress test suite Konstantin Ananyev
2024-10-15 15:59 ` Stephen Hemminger [this message]
2024-10-15 16:02 ` [PATCH v5 0/6] Stage-Ordered API and other extensions for ring library Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241015085955.16540ecb@hermes.local \
--to=stephen@networkplumber.org \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=drc@linux.vnet.ibm.com \
--cc=eimear.morrissey@huawei.com \
--cc=hemant.agrawal@nxp.com \
--cc=honnappa.nagarahalli@arm.com \
--cc=jerinj@marvell.com \
--cc=konstantin.ananyev@huawei.com \
--cc=konstantin.v.ananyev@yandex.ru \
--cc=mb@smartsharesystems.com \
--cc=ruifeng.wang@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).