From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.mhcomputing.net (master.mhcomputing.net [74.208.46.186]) by dpdk.org (Postfix) with ESMTP id 62D2611F5 for ; Thu, 2 Jul 2015 21:29:26 +0200 (CEST) Received: by mail.mhcomputing.net (Postfix, from userid 1000) id B1D8880B7BF; Thu, 2 Jul 2015 12:26:38 -0700 (PDT) Date: Thu, 2 Jul 2015 12:26:38 -0700 From: Matthew Hall To: "De Lara Guarch, Pablo" Message-ID: <20150702192638.GA4663@mhcomputing.net> References: <23903483D1A9B04AB5D759C6C845237F042C1A2EA7@ONWVEXCHMB05.ciena.com> <20150702093611.GB7688@bricha3-MOBL3> <23903483D1A9B04AB5D759C6C845237F042C49425B@ONWVEXCHMB05.ciena.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] DPDK Hash library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jul 2015 19:29:26 -0000 On Thu, Jul 02, 2015 at 05:55:20PM +0000, De Lara Guarch, Pablo wrote: > You are probably talking about extendable buckets here. > The downsize of that approach is that you have to allocate memory on the fly, > whereas with the cuckoo hash implementation, the entry can be stored in an alternative bucket > without having to reserve more memory (which also will take you more time). > With this approach, hash tables can get a higher utilization, as other less used > buckets can be used to store keys from other busier buckets. > > Pablo Expanding and shrinking buckets constantly can also be concurrency-hostile, and is a lot more complicated to get right than just using a good rehash algorithm and a nice static hunk of memory on contiguous hugepages for minimal TLB / cache pressure. If you want to do these more complex manipulations uthash is probably a better route. But it will be slower than the DPDK hashes by quite a ways I think. I used DPDK hash for my TCP socket table where everything is a very predictable size, but I had to use uthash for my unpredictably sized byte buffers for security indicators (IP, URL, Domain, Email, File Hash, etc.) Of course, when you do this kind of stuff in your app it is going to give you scaling problems and you'll have to spend a lot of time tuning it. Matthew.