From: "Medvedkin, Vladimir" <vladimir.medvedkin@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: "Wang, Yipeng1" <yipeng1.wang@intel.com>,
"Stephen Hemminger" <stephen@networkplumber.org>,
"dev@dpdk.org" <dev@dpdk.org>,
"Morten Brørup" <mb@smartsharesystems.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
"Gobriel, Sameh" <sameh.gobriel@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>,
"Suanming Mou" <suanmingm@mellanox.com>,
"Olivier Matz" <olivier.matz@6wind.com>,
"Xueming(Steven) Li" <xuemingl@mellanox.com>,
"Andrew Rybchenko" <arybchenko@solarflare.com>,
"Asaf Penso" <asafp@mellanox.com>, "Ori Kam" <orika@mellanox.com>
Subject: Re: [dpdk-dev] [PATCH 0/3] add new Double Word Key hash table
Date: Wed, 1 Apr 2020 18:28:00 +0000 [thread overview]
Message-ID: <52b1d35cb20448b28820e9a510f9ce94@intel.com> (raw)
In-Reply-To: <4749580.haC6HkEk0m@xps>
Hi Thomas,
-----Original Message-----
From: Thomas Monjalon <thomas@monjalon.net>
Sent: Tuesday, March 31, 2020 8:56 PM
To: Medvedkin, Vladimir <vladimir.medvedkin@intel.com>
Cc: Wang, Yipeng1 <yipeng1.wang@intel.com>; Stephen Hemminger <stephen@networkplumber.org>; dev@dpdk.org; Morten Brørup <mb@smartsharesystems.com>; dev@dpdk.org; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>; Suanming Mou <suanmingm@mellanox.com>; Olivier Matz <olivier.matz@6wind.com>; Xueming(Steven) Li <xuemingl@mellanox.com>; Andrew Rybchenko <arybchenko@solarflare.com>; Asaf Penso <asafp@mellanox.com>; Ori Kam <orika@mellanox.com>
Subject: Re: [dpdk-dev] [PATCH 0/3] add new Double Word Key hash table
26/03/2020 18:28, Medvedkin, Vladimir:
> Hi Yipeng, Stephen, all,
>
> On 17/03/2020 19:52, Wang, Yipeng1 wrote:
> > From: Stephen Hemminger <stephen@networkplumber.org>
> >> On Mon, 16 Mar 2020 18:27:40 +0000
> >> "Medvedkin, Vladimir" <vladimir.medvedkin@intel.com> wrote:
> >>
> >>> Hi Morten,
> >>>
> >>>
> >>> On 16/03/2020 14:39, Morten Brørup wrote:
> >>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Vladimir
> >>>>> Medvedkin
> >>>>> Sent: Monday, March 16, 2020 2:38 PM
> >>>>>
> >>>>> Currently DPDK has a special implementation of a hash table for
> >>>>> 4 byte keys which is called FBK hash. Unfortunately its main
> >>>>> drawback is that it only supports 2 byte values.
> >>>>> The new implementation called DWK (double word key) hash
> >>>>> supports 8 byte values, which is enough to store a pointer.
> >>>>>
> >>>>> It would also be nice to get feedback on whether to leave the
> >>>>> old FBK and new DWK implementations, or whether to deprecate the
> >>>>> old
> >> one?
> >>>> <rant on>
> >>>>
> >>>> Who comes up with these names?!?
> >>>>
> >>>> FBK (Four Byte Key) and DWK (Double Word Key) is supposed to mean
> >> the same. Could you use 32 somewhere in the name instead, like in
> >> int32_t, instead of using a growing list of creative synonyms for the same thing?
> >> Pretty please, with a cherry on top!
> >>>
> >>> That's true, at first I named it as fbk2, but then it was decided
> >>> to rename it "dwk", so that there was no confusion with the
> >>> existing FBK library. Naming suggestions are welcome!
> >>>
> >>>> And if the value size is fixed too, perhaps the name should also
> >>>> indicate
> >> the value size.
> >>>> <rant off>
> >>>>
> >>>> It's a shame we don't have C++ class templates available in DPDK...
> >>>>
> >>>> In other news, Mellanox has sent an RFC for an "indexed memory pool"
> >> library [1] to conserve memory by using uintXX_t instead of
> >> pointers, so perhaps a variant of a 32 bit key hash library with 32
> >> bit values (in addition to
> >> 16 bit values in FBK and 64 bit in DWK) would be nice combination
> >> with that library.
> >>>> [1]: http://mails.dpdk.org/archives/dev/2019-October/147513.html
Yes some work is in progress to propose a new memory allocator for small objects of fixed size with small memory overhead.
> >> Why is this different (or better) than existing rte_hash.
> >> Having more flavors is not necessarily a good thing (except in
> >> Gelato)
> > [Wang, Yipeng]
> > Hi, Vladimir,
> > As Stephen mentioned, I think it is good idea to explain the benefit
> > of this new type of hash table more explicitly such as Specific use cases, differences with current rte_hash, and performance numbers, etc.
>
> The main reason for this new hash library is performance. As I
> mentioned earlier, the current rte_fbk implementation is pretty fast
> but it has a number of drawbacks such as 2 byte values and limited
> collision resolving capabilities. On the other hand, rte_hash (cuckoo
> hash) doesn't have this drawbacks but at the cost of lower performance
> comparing to rte_fbk.
>
> If I understand correctly, performance penalty are due to :
>
> 1. Load two buckets
>
> 2. First compare signatures
>
> 3. If signature comparison hits get a key index and find memory
> location with a key itself and get the key
>
> 4. Using indirect call to memcmp() to compare two uint32_t.
>
> The new proposed 4 byte key hash table doesn't have rte_fbk drawbacks
> while offers the same performance as rte_fbk.
>
> Regarding use cases, in rte_ipsec_sad we are using rte_hash with 4
> byte key size. Replacing it with a new implementation gives about 30%
> in performance.
>
> The main disadvantage comparing to rte_hash is some performance
> degradation with high average table utilization due to chain resolving
> for 5th and subsequent collision.
Thanks for explaining.
Please, such information should added in the documentation:
doc/guides/prog_guide/hash_lib.rst
I'm going to submit v2 this week, will add documentation update.
next prev parent reply other threads:[~2020-04-01 18:31 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-16 13:38 Vladimir Medvedkin
2020-03-16 13:38 ` [dpdk-dev] [PATCH 1/3] hash: add dwk hash library Vladimir Medvedkin
2020-03-16 13:38 ` [dpdk-dev] [PATCH 2/3] test: add dwk hash autotests Vladimir Medvedkin
2020-03-16 13:38 ` [dpdk-dev] [PATCH 3/3] test: add dwk perf tests Vladimir Medvedkin
2020-03-16 14:39 ` [dpdk-dev] [PATCH 0/3] add new Double Word Key hash table Morten Brørup
2020-03-16 18:27 ` Medvedkin, Vladimir
2020-03-16 19:32 ` Stephen Hemminger
2020-03-17 19:52 ` Wang, Yipeng1
2020-03-26 17:28 ` Medvedkin, Vladimir
2020-03-31 19:55 ` Thomas Monjalon
2020-03-31 21:17 ` Honnappa Nagarahalli
2020-04-01 18:37 ` Medvedkin, Vladimir
2020-04-01 18:28 ` Medvedkin, Vladimir [this message]
2020-03-16 19:33 ` Morten Brørup
2020-04-08 18:19 ` [dpdk-dev] [PATCH v2 0/4] add new k32v64 " Vladimir Medvedkin
2020-04-15 18:17 ` [dpdk-dev] [PATCH v3 " Vladimir Medvedkin
2020-04-15 18:51 ` Mattias Rönnblom
2020-04-16 10:18 ` Medvedkin, Vladimir
2020-04-16 11:40 ` Mattias Rönnblom
2020-04-17 0:21 ` Wang, Yipeng1
2020-04-23 16:19 ` Ananyev, Konstantin
2020-05-08 20:08 ` Medvedkin, Vladimir
2020-04-16 9:39 ` Thomas Monjalon
2020-04-16 14:02 ` Medvedkin, Vladimir
2020-04-16 14:38 ` Thomas Monjalon
2020-05-08 19:58 ` [dpdk-dev] [PATCH v4 0/4] add new kv " Vladimir Medvedkin
2020-06-16 16:37 ` Thomas Monjalon
2021-03-24 21:28 ` Thomas Monjalon
2021-03-25 12:03 ` Medvedkin, Vladimir
2023-06-12 16:11 ` Stephen Hemminger
2020-05-08 19:58 ` [dpdk-dev] [PATCH v4 1/4] hash: add kv hash library Vladimir Medvedkin
2020-06-23 15:44 ` Ananyev, Konstantin
2020-06-23 23:06 ` Ananyev, Konstantin
2020-06-25 19:56 ` Medvedkin, Vladimir
2020-06-25 19:49 ` Medvedkin, Vladimir
2020-06-24 1:19 ` Wang, Yipeng1
2020-06-25 20:26 ` Medvedkin, Vladimir
2020-06-25 4:27 ` Honnappa Nagarahalli
2020-05-08 19:58 ` [dpdk-dev] [PATCH v4 2/4] hash: add documentation for " Vladimir Medvedkin
2020-05-08 19:58 ` [dpdk-dev] [PATCH v4 3/4] test: add kv hash autotests Vladimir Medvedkin
2020-05-08 19:58 ` [dpdk-dev] [PATCH v4 4/4] test: add kv perf tests Vladimir Medvedkin
2020-04-15 18:17 ` [dpdk-dev] [PATCH v3 1/4] hash: add k32v64 hash library Vladimir Medvedkin
2020-04-15 18:59 ` Mattias Rönnblom
2020-04-16 10:23 ` Medvedkin, Vladimir
2020-04-23 13:31 ` Ananyev, Konstantin
2020-05-08 20:14 ` Medvedkin, Vladimir
2020-04-29 21:29 ` Honnappa Nagarahalli
2020-05-08 20:38 ` Medvedkin, Vladimir
2020-04-15 18:17 ` [dpdk-dev] [PATCH v3 2/4] hash: add documentation for " Vladimir Medvedkin
2020-04-15 18:17 ` [dpdk-dev] [PATCH v3 3/4] test: add k32v64 hash autotests Vladimir Medvedkin
2020-04-15 18:17 ` [dpdk-dev] [PATCH v3 4/4] test: add k32v64 perf tests Vladimir Medvedkin
2020-04-08 18:19 ` [dpdk-dev] [PATCH v2 1/4] hash: add k32v64 hash library Vladimir Medvedkin
2020-04-08 23:23 ` Ananyev, Konstantin
2020-04-08 18:19 ` [dpdk-dev] [PATCH v2 2/4] hash: add documentation for " Vladimir Medvedkin
2020-04-08 18:19 ` [dpdk-dev] [PATCH v2 3/4] test: add k32v64 hash autotests Vladimir Medvedkin
2020-04-08 18:19 ` [dpdk-dev] [PATCH v2 4/4] test: add k32v64 perf tests Vladimir Medvedkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52b1d35cb20448b28820e9a510f9ce94@intel.com \
--to=vladimir.medvedkin@intel.com \
--cc=arybchenko@solarflare.com \
--cc=asafp@mellanox.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=konstantin.ananyev@intel.com \
--cc=mb@smartsharesystems.com \
--cc=olivier.matz@6wind.com \
--cc=orika@mellanox.com \
--cc=sameh.gobriel@intel.com \
--cc=stephen@networkplumber.org \
--cc=suanmingm@mellanox.com \
--cc=thomas@monjalon.net \
--cc=xuemingl@mellanox.com \
--cc=yipeng1.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).