From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
To: "Wang, Yipeng1" <yipeng1.wang@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>,
"De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"honnappa.nagarahalli@dpdk.org" <honnappa.nagarahalli@dpdk.org>,
"Gavin Hu (Arm Technology China)" <Gavin.Hu@arm.com>,
Steve Capper <Steve.Capper@arm.com>,
Ola Liljedahl <Ola.Liljedahl@arm.com>, nd <nd@arm.com>,
"Gobriel, Sameh" <sameh.gobriel@intel.com>
Subject: Re: [dpdk-dev] [PATCH 0/4] Address reader-writer concurrency in rte_hash
Date: Fri, 28 Sep 2018 21:11:18 +0000 [thread overview]
Message-ID: <DB7PR08MB36741BAF98F4ABFCFE3B036998EC0@DB7PR08MB3674.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <D2C4A16CA39F7F4E8E384D204491D7A6614D7F25@FMSMSX151.amr.corp.intel.com>
>
> Hi Honnappa,
>
> Reply inlined:
Hi Yipeng,
Thank you so much for reviewing.
>
> >-----Original Message-----
> >
> > Currently, reader-writer concurrency problems in rte_hash are
> > addressed using reader-writer locks. Use of reader-writer locks
> > results in following issues:
> >
> > 1) In many of the use cases for the hash table, writer threads
> > are running on control plane. If the writer is preempted while
> > holding the lock, it will block the readers for an extended period
> > resulting in packet drops. This problem seems to apply for platforms
> > with transactional memory support as well because of the algorithm
> > used for rte_rwlock_write_lock_tm:
> >
> > static inline void
> > rte_rwlock_write_lock_tm(rte_rwlock_t *rwl)
> > {
> > if (likely(rte_try_tm(&rwl->cnt)))
> > return;
> > rte_rwlock_write_lock(rwl);
> > }
> >
> > i.e. there is a posibility of using rte_rwlock_write_lock in
> > failure cases.
> [Wang, Yipeng] In our test, TSX failure happens very rarely on a TSX
> platform. But we agree that without TSX, the current rte_rwlock
> implementation may make the writer to hold a lock for a period of time.
>
> > 2) Reader-writer lock based solution does not address the following
> > issue.
> > rte_hash_lookup_xxx APIs return the index of the element in
> > the key store. Application(reader) can use that index to reference
> > other data structures in its scope. Because of this, the
> > index should not be freed till the application completes
> > using the index.
> [Wang, Yipeng] I agree on this use case. But I think we should provide new
> API functions for deletion to users who want this behavior, without
> changing the meaning of current API if that is possible.
In the lock-free algorithm, the rte_hash_delete API will not free the index. The new API rte_hash_free will free the index. The solution for the algorithm with rw locks needs to be thought about.
>
> > Current code:
> > Cores Lookup Lookup
> > with add
> > 2 474 246
> > 4 935 579
> > 6 1387 1048
> > 8 1766 1480
> > 10 2119 1951
> > 12 2546 2441
> >
> > With this patch:
> > Cores Lookup Lookup
> > with add
> > 2 291 211
> > 4 297 196
> > 6 304 198
> > 8 309 202
> > 10 315 205
> > 12 319 209
> >
> [Wang, Yipeng] It would be good if you could provide the platform
> information on these results.
Apologies, I should have done that. The machine I am using is: Intel(R) Xeon(R) CPU E5-2660 v4 @ 2.00GHz, 64G memory. This is a hacked test case which is not upstreamed. In the case of 'Lookup with add' - I had equal number of threads calling 'rte_hash_add' and 'rte_hash_lookup'. In the case of 'Lookup' - a set of entries were added and all the threads called 'rte_hash_lookup'. Note that these are the numbers without htm. We have created another test case which I will upstream as next version of this patch. I will publish the numbers with that test case. So, you should be able to reproduce the numbers with that test case.
>
> Another comment is as you know we also have a new extension to rte_hash
> to enable extendable buckets and partial-key hashing. Thanks for the
> comments btw. Could you check if your lockless scheme also applies to
> those extensions?
Thank you for reminding me on this. I thought I had covered everything. On a relook, I have missed few key issues. I will reply on the other email thread.
next prev parent reply other threads:[~2018-09-28 21:11 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-06 17:12 Honnappa Nagarahalli
2018-09-06 17:12 ` [dpdk-dev] [PATCH 1/4] hash: correct key store element alignment Honnappa Nagarahalli
2018-09-27 23:58 ` Wang, Yipeng1
2018-09-06 17:12 ` [dpdk-dev] [PATCH 2/4] hash: add memory ordering to avoid race conditions Honnappa Nagarahalli
2018-09-28 0:43 ` Wang, Yipeng1
2018-09-30 22:20 ` Honnappa Nagarahalli
2018-10-01 22:41 ` Wang, Yipeng1
2018-10-01 10:42 ` Ola Liljedahl
2018-10-02 1:52 ` Wang, Yipeng1
2018-09-06 17:12 ` [dpdk-dev] [PATCH 3/4] hash: fix rw concurrency while moving keys Honnappa Nagarahalli
2018-09-28 1:00 ` Wang, Yipeng1
2018-09-28 8:26 ` Bruce Richardson
2018-09-28 8:55 ` Van Haaren, Harry
2018-09-30 22:33 ` Honnappa Nagarahalli
2018-10-02 13:17 ` Van Haaren, Harry
2018-10-02 23:58 ` Wang, Yipeng1
2018-10-03 17:32 ` Honnappa Nagarahalli
2018-10-03 17:56 ` Wang, Yipeng1
2018-10-03 23:05 ` Ola Liljedahl
2018-10-04 3:32 ` Honnappa Nagarahalli
2018-10-04 3:54 ` Honnappa Nagarahalli
2018-10-04 19:16 ` Wang, Yipeng1
2018-09-30 23:05 ` Honnappa Nagarahalli
2018-10-01 22:56 ` Wang, Yipeng1
2018-10-03 0:16 ` Wang, Yipeng1
2018-10-03 17:39 ` Honnappa Nagarahalli
2018-09-06 17:12 ` [dpdk-dev] [PATCH 4/4] hash: enable lock-free reader-writer concurrency Honnappa Nagarahalli
2018-09-28 1:33 ` Wang, Yipeng1
2018-10-01 4:11 ` Honnappa Nagarahalli
2018-10-01 23:54 ` Wang, Yipeng1
2018-10-11 5:24 ` Honnappa Nagarahalli
2018-09-14 21:18 ` [dpdk-dev] [PATCH 0/4] Address reader-writer concurrency in rte_hash Honnappa Nagarahalli
2018-09-26 14:36 ` Honnappa Nagarahalli
2018-09-27 23:45 ` Wang, Yipeng1
2018-09-28 21:11 ` Honnappa Nagarahalli [this message]
2018-10-02 0:30 ` Wang, Yipeng1
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DB7PR08MB36741BAF98F4ABFCFE3B036998EC0@DB7PR08MB3674.eurprd08.prod.outlook.com \
--to=honnappa.nagarahalli@arm.com \
--cc=Gavin.Hu@arm.com \
--cc=Ola.Liljedahl@arm.com \
--cc=Steve.Capper@arm.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=honnappa.nagarahalli@dpdk.org \
--cc=nd@arm.com \
--cc=pablo.de.lara.guarch@intel.com \
--cc=sameh.gobriel@intel.com \
--cc=yipeng1.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).