From: Ola Liljedahl <Ola.Liljedahl@arm.com>
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
"gage.eads@intel.com" <gage.eads@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>
Cc: nd <nd@arm.com>,
"bruce.richardson@intel.com" <bruce.richardson@intel.com>,
"thomas@monjalon.net" <thomas@monjalon.net>,
"konstantin.ananyev@intel.com" <konstantin.ananyev@intel.com>,
"Song Zhu (Arm Technology China)" <Song.Zhu@arm.com>,
"stephen@networkplumber.org" <stephen@networkplumber.org>,
"olivier.matz@6wind.com" <olivier.matz@6wind.com>,
"arybchenko@solarflare.com" <arybchenko@solarflare.com>,
"Gavin Hu (Arm Technology China)" <Gavin.Hu@arm.com>
Subject: Re: [dpdk-dev] [PATCH v3 0/5] Add non-blocking ring
Date: Mon, 28 Jan 2019 10:41:31 +0000 [thread overview]
Message-ID: <1548672102.3076.21.camel@arm.com> (raw)
In-Reply-To: <9184057F7FC11744A2107296B6B8EB1E541CB774@FMSMSX108.amr.corp.intel.com>
On Fri, 2019-01-25 at 17:56 +0000, Eads, Gage wrote:
>
> >
> > -----Original Message-----
> > From: Eads, Gage
> > Sent: Friday, January 25, 2019 11:43 AM
> > To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org
> > Cc: olivier.matz@6wind.com; arybchenko@solarflare.com; Richardson, Bruce
> > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>; stephen@networkplumber.org; nd
> > <nd@arm.com>; thomas@monjalon.net; Ola Liljedahl
> > <Ola.Liljedahl@arm.com>; Gavin Hu (Arm Technology China)
> > <Gavin.Hu@arm.com>; Song Zhu (Arm Technology China)
> > <Song.Zhu@arm.com>; nd <nd@arm.com>
> > Subject: RE: [dpdk-dev] [PATCH v3 0/5] Add non-blocking ring
> >
> > Hi Honnappa,
> >
> > Works for me -- I'm in favor of the best performing implementation, whoever
> > provides it.
> >
> > To allow an apples-to-apples comparison, I suggest Ola's/ARM's
> > implementation
> > be made to fit into the rte_ring API with an associated mempool handler.
> > That'll
> > allow us to use the existing ring and mempool performance tests as well.
> > Feel
> > free to use code from this patchset for the rte_ring integration, if that
> > helps, of
> > course.
> >
> But also, if Ola/ARM's algorithm is sufficiently similar to this one, it's
> probably better to tweak this patchset's enqueue and dequeue functions with
> any improvements you can identify rather than creating an entirely separate
> implementation.
There are strong similarities. But my implementation is separate from rte_ring
(whose code is a mess) which also freed me from any interoperatibility with the
rte_ring code and data structure (with two pairs of head+tail which is
unnecessary for the lock-free ring buffer).
My design and implementation is here:
https://github.com/ARM-software/progress64/blob/master/src/p64_lfring.c
I have a DPDK version in flight. Merging the relevant changes into your patch
makes sense. There are some differences we will have to agree on.
>
> >
> > I expect to have v4 available within the next week.
> >
> > Thanks,
> > Gage
> >
> > >
> > > -----Original Message-----
> > > From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
> > > Sent: Thursday, January 24, 2019 11:21 PM
> > > To: Eads, Gage <gage.eads@intel.com>; dev@dpdk.org
> > > Cc: olivier.matz@6wind.com; arybchenko@solarflare.com; Richardson,
> > > Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> > > <konstantin.ananyev@intel.com>; stephen@networkplumber.org; nd
> > > <nd@arm.com>; thomas@monjalon.net; Ola Liljedahl
> > > <Ola.Liljedahl@arm.com>; Gavin Hu (Arm Technology China)
> > > <Gavin.Hu@arm.com>; Song Zhu (Arm Technology China)
> > > <Song.Zhu@arm.com>; nd <nd@arm.com>
> > > Subject: RE: [dpdk-dev] [PATCH v3 0/5] Add non-blocking ring
> > >
> > > Hi Gage,
> > > Thank you for this patch. Arm (Ola Liljedahl) had worked on a non-
> > > blocking ring algorithm. We were planning to add it to DPDK at some
> > > point this year. I am wondering if you would be open to take a look at
> > > the algorithm and collaborate?
> > >
> > > I am yet to fully understand both the algorithms. But, Ola has
> > > reviewed your patch and can provide a quick overview of the differences
> > > here.
> > >
> > > If you agree, we can send a RFC patch. You can review that and do
> > > performance benchmarking on your platforms. I can also benchmark your
> > > patch (may be once you fix the issue identified in
> > > __rte_ring_do_nb_enqueue_mp function?) on Arm platforms. May be we can
> > end up with a better combined algorithm.
> > >
> > >
> > > Hi Thomas/Bruce,
> > > Please let me know if this is ok and if there is a better way to do
> > > this.
> > >
> > > Thank you,
> > > Honnappa
> > >
> > > >
> > > > -----Original Message-----
> > > > From: dev <dev-bounces@dpdk.org> On Behalf Of Gage Eads
> > > > Sent: Friday, January 18, 2019 9:23 AM
> > > > To: dev@dpdk.org
> > > > Cc: olivier.matz@6wind.com; arybchenko@solarflare.com;
> > > > bruce.richardson@intel.com; konstantin.ananyev@intel.com;
> > > > stephen@networkplumber.org
> > > > Subject: [dpdk-dev] [PATCH v3 0/5] Add non-blocking ring
> > > >
> > > > For some users, the rte ring's "non-preemptive" constraint is not
> > > > acceptable; for example, if the application uses a mixture of pinned
> > > > high-priority threads and multiplexed low-priority threads that
> > > > share a
> > > mempool.
> > > >
> > > >
> > > > This patchset introduces a non-blocking ring, on top of which a
> > > > mempool can run.
> > > > Crucially, the non-blocking algorithm relies on a 128-bit
> > > > compare-and-swap, so it is currently limited to x86_64 machines.
> > > > This is also an experimental API, so RING_F_NB users must build with
> > > > the
> > > ALLOW_EXPERIMENTAL_API flag.
> > > >
> > > >
> > > > The ring uses more compare-and-swap atomic operations than the
> > > > regular rte
> > > > ring:
> > > > With no contention, an enqueue of n pointers uses (1 + 2n) CAS
> > > > operations and a dequeue of n pointers uses 2. This algorithm has
> > > > worse average-case performance than the regular rte ring
> > > > (particularly a highly-contended ring with large bulk accesses),
> > > > however:
> > > > - For applications with preemptible pthreads, the regular rte ring's
> > > > worst-
> > case
> > >
> > > >
> > > > performance (i.e. one thread being preempted in the update_tail()
> > > > critical
> > > > section) is much worse than the non-blocking ring's.
> > > > - Software caching can mitigate the average case performance for ring-
> > based
> > >
> > > >
> > > > algorithms. For example, a non-blocking ring based mempool (a
> > > > likely use case
> > > > for this ring) with per-thread caching.
> > > >
> > > > The non-blocking ring is enabled via a new flag, RING_F_NB. For
> > > > ease-of-use, existing ring enqueue/dequeue functions work with both
> > > > "regular" and non- blocking rings.
> > > >
> > > > This patchset also adds non-blocking versions of ring_autotest and
> > > > ring_perf_autotest, and a non-blocking ring based mempool.
> > > >
> > > > This patchset makes one API change; a deprecation notice will be
> > > > posted in a separate commit.
> > > >
> > > > This patchset depends on the non-blocking stack patchset[1].
> > > >
> > > > [1] http://mails.dpdk.org/archives/dev/2019-January/123653.html
> > > >
> > > > v3:
> > > > - Avoid the ABI break by putting 64-bit head and tail values in the
> > > > same
> > > > cacheline as struct rte_ring's prod and cons members.
> > > > - Don't attempt to compile rte_atomic128_cmpset without
> > > > ALLOW_EXPERIMENTAL_API, as this would break a large number of
> > libraries.
> > >
> > > >
> > > > - Add a helpful warning to __rte_ring_do_nb_enqueue_mp() in case
> > > > someone tries
> > > > to use RING_F_NB without the ALLOW_EXPERIMENTAL_API flag.
> > > > - Update the ring mempool to use experimental APIs
> > > > - Clarify that RINB_F_NB is only limited to x86_64 currently;
> > > > ARMv8.1-A builds
> > > > can eventually support it with the CASP instruction.
> > > >
> > > > v2:
> > > > - Merge separate docs commit into patch #5
> > > > - Convert uintptr_t to size_t
> > > > - Add a compile-time check for the size of size_t
> > > > - Fix a space-after-typecast issue
> > > > - Fix an unnecessary-parentheses checkpatch warning
> > > > - Bump librte_ring's library version
> > > >
> > > > Gage Eads (5):
> > > > ring: add 64-bit headtail structure
> > > > ring: add a non-blocking implementation
> > > > test_ring: add non-blocking ring autotest
> > > > test_ring_perf: add non-blocking ring perf test
> > > > mempool/ring: add non-blocking ring handlers
> > > >
> > > > doc/guides/prog_guide/env_abstraction_layer.rst | 2 +-
> > > > drivers/mempool/ring/Makefile | 1 +
> > > > drivers/mempool/ring/meson.build | 2 +
> > > > drivers/mempool/ring/rte_mempool_ring.c | 58 ++-
> > > > lib/librte_eventdev/rte_event_ring.h | 2 +-
> > > > lib/librte_ring/Makefile | 3 +-
> > > > lib/librte_ring/rte_ring.c | 72 ++-
> > > > lib/librte_ring/rte_ring.h | 574
> > > > ++++++++++++++++++++++--
> > > > lib/librte_ring/rte_ring_generic_64.h | 152 +++++++
> > > > lib/librte_ring/rte_ring_version.map | 7 +
> > > > test/test/test_ring.c | 57 ++-
> > > > test/test/test_ring_perf.c | 19 +-
> > > > 12 files changed, 874 insertions(+), 75 deletions(-) create mode
> > > > 100644 lib/librte_ring/rte_ring_generic_64.h
> > > >
> > > > --
> > > > 2.13.6
--
Ola Liljedahl, Networking System Architect, Arm
Phone +46706866373, Skype ola.liljedahl
next prev parent reply other threads:[~2019-01-28 10:41 UTC|newest]
Thread overview: 123+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-10 21:01 [dpdk-dev] [PATCH 0/6] " Gage Eads
2019-01-10 21:01 ` [dpdk-dev] [PATCH 1/6] ring: change head and tail to pointer-width size Gage Eads
2019-01-11 4:38 ` Stephen Hemminger
2019-01-11 19:07 ` Eads, Gage
2019-01-11 10:25 ` Burakov, Anatoly
2019-01-11 19:12 ` Eads, Gage
2019-01-11 19:55 ` Stephen Hemminger
2019-01-15 15:48 ` Eads, Gage
2019-01-11 10:40 ` Burakov, Anatoly
2019-01-11 10:58 ` Bruce Richardson
2019-01-11 11:30 ` Burakov, Anatoly
[not found] ` <20190111115851.GC3336@bricha3-MOBL.ger.corp.intel.com>
2019-01-11 19:27 ` Eads, Gage
2019-01-21 14:14 ` Burakov, Anatoly
2019-01-22 18:27 ` Eads, Gage
2019-01-10 21:01 ` [dpdk-dev] [PATCH 2/6] ring: add a non-blocking implementation Gage Eads
2019-01-10 21:01 ` [dpdk-dev] [PATCH 3/6] test_ring: add non-blocking ring autotest Gage Eads
2019-01-10 21:01 ` [dpdk-dev] [PATCH 4/6] test_ring_perf: add non-blocking ring perf test Gage Eads
2019-01-10 21:01 ` [dpdk-dev] [PATCH 5/6] mempool/ring: add non-blocking ring handlers Gage Eads
2019-01-13 13:43 ` Andrew Rybchenko
2019-01-10 21:01 ` [dpdk-dev] [PATCH 6/6] doc: add NB ring comment to EAL "known issues" Gage Eads
2019-01-11 2:51 ` Varghese, Vipin
2019-01-11 19:30 ` Eads, Gage
2019-01-14 0:07 ` Varghese, Vipin
2019-01-15 23:52 ` [dpdk-dev] [PATCH v2 0/5] Add non-blocking ring Gage Eads
2019-01-15 23:52 ` [dpdk-dev] [PATCH v2 1/5] ring: change head and tail to pointer-width size Gage Eads
2019-01-15 23:52 ` [dpdk-dev] [PATCH v2 2/5] ring: add a non-blocking implementation Gage Eads
2019-01-15 23:52 ` [dpdk-dev] [PATCH v2 3/5] test_ring: add non-blocking ring autotest Gage Eads
2019-01-15 23:52 ` [dpdk-dev] [PATCH v2 4/5] test_ring_perf: add non-blocking ring perf test Gage Eads
2019-01-15 23:52 ` [dpdk-dev] [PATCH v2 5/5] mempool/ring: add non-blocking ring handlers Gage Eads
2019-01-16 0:26 ` [dpdk-dev] [PATCH v2 0/5] Add non-blocking ring Stephen Hemminger
2019-01-18 15:23 ` [dpdk-dev] [PATCH v3 " Gage Eads
2019-01-18 15:23 ` [dpdk-dev] [PATCH v3 1/5] ring: add 64-bit headtail structure Gage Eads
2019-01-18 15:23 ` [dpdk-dev] [PATCH v3 2/5] ring: add a non-blocking implementation Gage Eads
2019-01-22 10:12 ` Ola Liljedahl
2019-01-22 14:49 ` Ola Liljedahl
2019-01-22 21:31 ` Eads, Gage
2019-01-23 10:16 ` Ola Liljedahl
2019-01-25 17:21 ` Eads, Gage
2019-01-28 10:35 ` Ola Liljedahl
2019-01-28 18:54 ` Eads, Gage
2019-01-28 22:31 ` Ola Liljedahl
2019-01-28 13:34 ` Jerin Jacob Kollanukkaran
2019-01-28 13:43 ` Ola Liljedahl
2019-01-28 14:04 ` Jerin Jacob Kollanukkaran
2019-01-28 14:06 ` Ola Liljedahl
2019-01-28 18:59 ` Eads, Gage
2019-01-18 15:23 ` [dpdk-dev] [PATCH v3 3/5] test_ring: add non-blocking ring autotest Gage Eads
2019-01-18 15:23 ` [dpdk-dev] [PATCH v3 4/5] test_ring_perf: add non-blocking ring perf test Gage Eads
2019-01-18 15:23 ` [dpdk-dev] [PATCH v3 5/5] mempool/ring: add non-blocking ring handlers Gage Eads
2019-01-22 9:27 ` [dpdk-dev] [PATCH v3 0/5] Add non-blocking ring Ola Liljedahl
2019-01-22 10:15 ` Ola Liljedahl
2019-01-22 19:15 ` Eads, Gage
2019-01-23 16:02 ` Jerin Jacob Kollanukkaran
2019-01-23 16:29 ` Ola Liljedahl
2019-01-28 13:10 ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-01-25 5:20 ` [dpdk-dev] " Honnappa Nagarahalli
2019-01-25 17:42 ` Eads, Gage
2019-01-25 17:56 ` Eads, Gage
2019-01-28 10:41 ` Ola Liljedahl [this message]
2019-01-28 18:14 ` [dpdk-dev] [PATCH v4 " Gage Eads
2019-01-28 18:14 ` [dpdk-dev] [PATCH v4 1/5] ring: add 64-bit headtail structure Gage Eads
2019-01-29 12:56 ` Ola Liljedahl
2019-01-30 4:26 ` Eads, Gage
2019-01-28 18:14 ` [dpdk-dev] [PATCH v4 2/5] ring: add a non-blocking implementation Gage Eads
2019-01-28 18:14 ` [dpdk-dev] [PATCH v4 3/5] test_ring: add non-blocking ring autotest Gage Eads
2019-01-28 18:14 ` [dpdk-dev] [PATCH v4 4/5] test_ring_perf: add non-blocking ring perf test Gage Eads
2019-01-28 18:14 ` [dpdk-dev] [PATCH v4 5/5] mempool/ring: add non-blocking ring handlers Gage Eads
2019-03-05 17:40 ` [dpdk-dev] [PATCH v5 0/6] Add lock-free ring and mempool handler Gage Eads
2019-03-05 17:40 ` [dpdk-dev] [PATCH v5 1/6] ring: add a pointer-width headtail structure Gage Eads
2019-03-05 17:40 ` [dpdk-dev] [PATCH v5 2/6] ring: add a ring start marker Gage Eads
2019-03-05 17:40 ` [dpdk-dev] [PATCH v5 3/6] ring: add a lock-free implementation Gage Eads
2019-03-05 17:40 ` [dpdk-dev] [PATCH v5 4/6] test_ring: add lock-free ring autotest Gage Eads
2019-03-05 17:40 ` [dpdk-dev] [PATCH v5 5/6] test_ring_perf: add lock-free ring perf test Gage Eads
2019-03-05 17:40 ` [dpdk-dev] [PATCH v5 6/6] mempool/ring: add lock-free ring handlers Gage Eads
2019-03-06 15:03 ` [dpdk-dev] [PATCH v6 0/6] Add lock-free ring and mempool handler Gage Eads
2019-03-06 15:03 ` [dpdk-dev] [PATCH v6 1/6] ring: add a pointer-width headtail structure Gage Eads
2019-03-06 15:03 ` [dpdk-dev] [PATCH v6 2/6] ring: add a ring start marker Gage Eads
2019-03-06 15:03 ` [dpdk-dev] [PATCH v6 3/6] ring: add a lock-free implementation Gage Eads
2019-03-06 15:03 ` [dpdk-dev] [PATCH v6 4/6] test_ring: add lock-free ring autotest Gage Eads
2019-03-06 15:03 ` [dpdk-dev] [PATCH v6 5/6] test_ring_perf: add lock-free ring perf test Gage Eads
2019-03-06 15:03 ` [dpdk-dev] [PATCH v6 6/6] mempool/ring: add lock-free ring handlers Gage Eads
2019-03-18 21:35 ` [dpdk-dev] [PATCH v7 0/6] Add lock-free ring and mempool handler Gage Eads
2019-03-18 21:35 ` Gage Eads
2019-03-18 21:35 ` [dpdk-dev] [PATCH v7 1/6] ring: add a pointer-width headtail structure Gage Eads
2019-03-18 21:35 ` Gage Eads
2019-03-18 21:35 ` [dpdk-dev] [PATCH v7 2/6] ring: add a ring start marker Gage Eads
2019-03-18 21:35 ` Gage Eads
2019-03-18 21:35 ` [dpdk-dev] [PATCH v7 3/6] ring: add a lock-free implementation Gage Eads
2019-03-18 21:35 ` Gage Eads
2019-03-19 15:50 ` Stephen Hemminger
2019-03-19 15:50 ` Stephen Hemminger
2019-03-18 21:35 ` [dpdk-dev] [PATCH v7 4/6] test_ring: add lock-free ring autotest Gage Eads
2019-03-18 21:35 ` Gage Eads
2019-03-18 21:35 ` [dpdk-dev] [PATCH v7 5/6] test_ring_perf: add lock-free ring perf test Gage Eads
2019-03-18 21:35 ` Gage Eads
2019-03-18 21:35 ` [dpdk-dev] [PATCH v7 6/6] mempool/ring: add lock-free ring handlers Gage Eads
2019-03-18 21:35 ` Gage Eads
2019-03-18 21:49 ` [dpdk-dev] [PATCH v7 0/6] Add lock-free ring and mempool handler Eads, Gage
2019-03-18 21:49 ` Eads, Gage
2019-03-19 15:51 ` Stephen Hemminger
2019-03-19 15:51 ` Stephen Hemminger
2019-04-01 19:23 ` Eads, Gage
2019-04-01 19:23 ` Eads, Gage
2019-04-02 10:16 ` Ola Liljedahl
2019-04-02 10:16 ` Ola Liljedahl
2019-04-04 22:28 ` Eads, Gage
2019-04-04 22:28 ` Eads, Gage
2019-03-19 1:20 ` [dpdk-dev] [PATCH v8 " Gage Eads
2019-03-19 1:20 ` Gage Eads
2019-03-19 1:20 ` [dpdk-dev] [PATCH v8 1/6] ring: add a pointer-width headtail structure Gage Eads
2019-03-19 1:20 ` Gage Eads
2019-03-19 1:20 ` [dpdk-dev] [PATCH v8 2/6] ring: add a ring start marker Gage Eads
2019-03-19 1:20 ` Gage Eads
2019-03-19 1:20 ` [dpdk-dev] [PATCH v8 3/6] ring: add a lock-free implementation Gage Eads
2019-03-19 1:20 ` Gage Eads
2019-03-19 1:20 ` [dpdk-dev] [PATCH v8 4/6] test_ring: add lock-free ring autotest Gage Eads
2019-03-19 1:20 ` Gage Eads
2019-03-19 1:20 ` [dpdk-dev] [PATCH v8 5/6] test_ring_perf: add lock-free ring perf test Gage Eads
2019-03-19 1:20 ` Gage Eads
2019-03-19 1:20 ` [dpdk-dev] [PATCH v8 6/6] mempool/ring: add lock-free ring handlers Gage Eads
2019-03-19 1:20 ` Gage Eads
2019-04-03 16:46 ` [dpdk-dev] [PATCH v8 0/6] Add lock-free ring and mempool handler Thomas Monjalon
2019-04-03 16:46 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1548672102.3076.21.camel@arm.com \
--to=ola.liljedahl@arm.com \
--cc=Gavin.Hu@arm.com \
--cc=Honnappa.Nagarahalli@arm.com \
--cc=Song.Zhu@arm.com \
--cc=arybchenko@solarflare.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=gage.eads@intel.com \
--cc=konstantin.ananyev@intel.com \
--cc=nd@arm.com \
--cc=olivier.matz@6wind.com \
--cc=stephen@networkplumber.org \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).