From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 009B7B393 for ; Tue, 9 Sep 2014 12:41:30 +0200 (CEST) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP; 09 Sep 2014 03:46:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,491,1406617200"; d="scan'208";a="596750641" Received: from irsmsx102.ger.corp.intel.com ([163.33.3.155]) by fmsmga002.fm.intel.com with ESMTP; 09 Sep 2014 03:46:21 -0700 Received: from irsmsx109.ger.corp.intel.com (163.33.3.23) by IRSMSX102.ger.corp.intel.com (163.33.3.155) with Microsoft SMTP Server (TLS) id 14.3.195.1; Tue, 9 Sep 2014 11:45:28 +0100 Received: from irsmsx103.ger.corp.intel.com ([169.254.3.112]) by IRSMSX109.ger.corp.intel.com ([169.254.13.200]) with mapi id 14.03.0195.001; Tue, 9 Sep 2014 11:45:27 +0100 From: "Richardson, Bruce" To: Matthew Hall , "dev@dpdk.org" Thread-Topic: [dpdk-dev] Defaults for rte_hash Thread-Index: AQHPzBljAXvOQCVRlUGWa/P3eDn+npv4nFUQ Date: Tue, 9 Sep 2014 10:45:27 +0000 Message-ID: <59AF69C657FD0841A61C55336867B5B0343EFBBD@IRSMSX103.ger.corp.intel.com> References: <20140909103153.GA7969@mhcomputing.net> In-Reply-To: <20140909103153.GA7969@mhcomputing.net> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] Defaults for rte_hash X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 10:41:31 -0000 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Matthew Hall > Sent: Tuesday, September 09, 2014 11:32 AM > To: dev@dpdk.org > Subject: [dpdk-dev] Defaults for rte_hash >=20 > Hello, >=20 > I was looking at the code which inits rte_hash objects in examples/l3fwd.= It's > using approx. 1M to 4M hash 'entries' depending on 32-bit vs 64-bit, but = it's > setting the 'bucket_entries' to just 4. >=20 > Normally I'm used to using somewhat deeper hash buckets than that... it s= eems > like having a zillion little tiny hash buckets would cause more TLB press= ure > and memory overhead... or does 4 get shifted / exponentiated into 2**4 ? >=20 > The documentation in > http://dpdk.org/doc/api/structrte__hash__parameters.html > and http://dpdk.org/doc/api/rte__hash_8h.html isn't that clear... is ther= e a > better place to look for this? >=20 > In my case I'm looking to create a table of 4M or 8M entries, containing > tables of security threat IPs / domains, to be detected in the traffic. S= o it > would be good to have some understanding how not to waste a ton of memory > on a > table this huge without making it run super slow either. >=20 > Did anybody have some experience with how to get this right? It might be worth looking too at the hash table structures in the librte_ta= ble directory for packet framework. These should give better scalability ac= ross millions of flows than the existing rte_hash implementation. [We're lo= oking here to provide in the future a similar, more scalable, hash table im= plementation with an API like that of rte_hash, but that is still under dev= elopment here at the moment.] >=20 > Another thing... the LPM table uses 16-bit Hop IDs. But I would probably = have > more than 64K CIDR blocks of badness on the Internet available to me for > analysis. How would I cope with this, besides just letting some attackers > escape unnoticed? ;) Actually, I think the next hop field in the lpm implementation is only 8-bi= ts, not 16 :-). Each lpm entry is only 16-bits in total. >=20 > Have we got some kind of structure which allows a greater number of CIDRs > even > if it's not quite as fast? >=20 > Thanks, > Matthew.