From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <prvs=862561c67c=jabdul@ciena.com> Received: from mx0a-00103a01.pphosted.com (mx0a-00103a01.pphosted.com [67.231.144.234]) by dpdk.org (Postfix) with ESMTP id DDBA2FFA for <dev@dpdk.org>; Thu, 2 Jul 2015 23:02:41 +0200 (CEST) Received: from pps.filterd (m0000419.ppops.net [127.0.0.1]) by mx0a-00103a01.pphosted.com (8.14.5/8.14.5) with SMTP id t62KxlCl024626; Thu, 2 Jul 2015 17:02:40 -0400 Received: from vawvcgsie2k1302.ciena.com (LIN1-118-36-36.ciena.com [63.118.36.36]) by mx0a-00103a01.pphosted.com with ESMTP id 1vd5qs1edw-2 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); Thu, 02 Jul 2015 17:02:40 -0400 Received: from ONWVEXCHHT03.ciena.com (10.128.6.43) by VAWVCGSIE2K1302.ciena.com (10.4.62.16) with Microsoft SMTP Server (TLS) id 15.0.847.32; Thu, 2 Jul 2015 17:02:33 -0400 Received: from ONWVEXCHMB05.ciena.com ([::1]) by ONWVEXCHHT03.ciena.com ([::1]) with mapi; Thu, 2 Jul 2015 17:02:33 -0400 From: "Abdul, Jaffar" <jabdul@ciena.com> To: Matthew Hall <mhall@mhcomputing.net>, "De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com> Date: Thu, 2 Jul 2015 17:02:32 -0400 Thread-Topic: [dpdk-dev] DPDK Hash library Thread-Index: AdC0/WlO7SNPnCQzS5GrRshMG/yisAADPEow Message-ID: <23903483D1A9B04AB5D759C6C845237F042C494356@ONWVEXCHMB05.ciena.com> References: <23903483D1A9B04AB5D759C6C845237F042C1A2EA7@ONWVEXCHMB05.ciena.com> <20150702093611.GB7688@bricha3-MOBL3> <23903483D1A9B04AB5D759C6C845237F042C49425B@ONWVEXCHMB05.ciena.com> <E115CCD9D858EF4F90C690B0DCB4D897272D2398@IRSMSX108.ger.corp.intel.com> <20150702192638.GA4663@mhcomputing.net> In-Reply-To: <20150702192638.GA4663@mhcomputing.net> Accept-Language: en-US, en-CA Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-CA x-tm-as-product-ver: SMEX-10.0.0.1412-7.000.1014-21652.002 x-tm-as-result: No--41.677300-8.000000-31 x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151, 1.0.33, 0.0.0000 definitions=2015-07-02_14:2015-07-02,2015-07-02,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1506180000 definitions=main-1507020324 Cc: "dev@dpdk.org" <dev@dpdk.org> Subject: Re: [dpdk-dev] DPDK Hash library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> X-List-Received-Date: Thu, 02 Jul 2015 21:02:42 -0000 HI Thanks guys for helping me on this one! Your suggestions are very good and = it is very helpful! Thanks Jaffar -----Original Message----- From: Matthew Hall [mailto:mhall@mhcomputing.net]=20 Sent: Thursday, July 02, 2015 12:27 PM To: De Lara Guarch, Pablo Cc: Abdul, Jaffar; Richardson, Bruce; dev@dpdk.org Subject: Re: [dpdk-dev] DPDK Hash library On Thu, Jul 02, 2015 at 05:55:20PM +0000, De Lara Guarch, Pablo wrote: > You are probably talking about extendable buckets here. > The downsize of that approach is that you have to allocate memory on=20 > the fly, whereas with the cuckoo hash implementation, the entry can be=20 > stored in an alternative bucket without having to reserve more memory (wh= ich also will take you more time). > With this approach, hash tables can get a higher utilization, as other=20 > less used buckets can be used to store keys from other busier buckets. >=20 > Pablo Expanding and shrinking buckets constantly can also be concurrency-hostile,= and is a lot more complicated to get right than just using a good rehash a= lgorithm and a nice static hunk of memory on contiguous hugepages for minim= al TLB / cache pressure. If you want to do these more complex manipulations uthash is probably a bet= ter route. But it will be slower than the DPDK hashes by quite a ways I thi= nk. I used DPDK hash for my TCP socket table where everything is a very pre= dictable size, but I had to use uthash for my unpredictably sized byte buff= ers for security indicators (IP, URL, Domain, Email, File Hash, etc.) Of course, when you do this kind of stuff in your app it is going to give y= ou scaling problems and you'll have to spend a lot of time tuning it. Matthew.