Re: [dpdk-dev] DPDK Hash library

["Abdul, Jaffar" <jabdul@ciena.com>]

HI

Thanks guys for helping me on this one! Your suggestions are very good and it is very helpful!

Thanks
Jaffar


-----Original Message-----
From: Matthew Hall [mailto:mhall@mhcomputing.net] 
Sent: Thursday, July 02, 2015 12:27 PM
To: De Lara Guarch, Pablo
Cc: Abdul, Jaffar; Richardson, Bruce; dev@dpdk.org
Subject: Re: [dpdk-dev] DPDK Hash library

On Thu, Jul 02, 2015 at 05:55:20PM +0000, De Lara Guarch, Pablo wrote:
> You are probably talking about extendable buckets here.
> The downsize of that approach is that you have to allocate memory on 
> the fly, whereas with the cuckoo hash implementation, the entry can be 
> stored in an alternative bucket without having to reserve more memory (which also will take you more time).
> With this approach, hash tables can get a higher utilization, as other 
> less used buckets can be used to store keys from other busier buckets.
> 
> Pablo

Expanding and shrinking buckets constantly can also be concurrency-hostile, and is a lot more complicated to get right than just using a good rehash algorithm and a nice static hunk of memory on contiguous hugepages for minimal TLB / cache pressure.

If you want to do these more complex manipulations uthash is probably a better route. But it will be slower than the DPDK hashes by quite a ways I think. I used DPDK hash for my TCP socket table where everything is a very predictable size, but I had to use uthash for my unpredictably sized byte buffers for security indicators (IP, URL, Domain, Email, File Hash, etc.)

Of course, when you do this kind of stuff in your app it is going to give you scaling problems and you'll have to spend a lot of time tuning it.

Matthew.