From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f49.google.com (mail-wm0-f49.google.com [74.125.82.49]) by dpdk.org (Postfix) with ESMTP id D9EE92C5 for ; Mon, 17 Apr 2017 22:06:23 +0200 (CEST) Received: by mail-wm0-f49.google.com with SMTP id w64so41097315wma.0 for ; Mon, 17 Apr 2017 13:06:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=L1rx0Y/AllAT8rKnpppVXsnwmCTIoiKyWCLpWPNT65M=; b=k6C94v/axONiHIxAs1kggYWlFCAD3WlScg+DDhfwPttlfQ0lDsgfv3Svs0b5bPRBhw ds4ZtUGMtqBQ+1MxnPmFj6+AfP+yQXtfwEtBWCWAmhjtXjv8e9S0ag+2rG4pCVdmibf8 g26/uJEjFgxBoDyX0CzdPnQyOki2b5FvYnJGDferPUmO7F8lp+naZnWo0hnwt0/hUQNa nfOpDKMiAdKwI9Z2dTsDMzRXHfw46XESctJ24oxH6MGeATBhi3LxAe0u3d7pGPMcx+Mo tGP+UCj3NdCRJTx769PNCvpE/VDKFSoN9Q7xPHNIVqvo1o+otCrbsH92pSgfoJhBDLjA BVqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=L1rx0Y/AllAT8rKnpppVXsnwmCTIoiKyWCLpWPNT65M=; b=RDuZGh6PBL0no9ybOLsTZpNrSVMDveFUF0dsFHVSdeNJooQGU0Kd6ow9q4IipfiLku EVzdYvVl8ONmsoT2VU8eiHyHgfYIKzZoQnFQMNS9Od8kFu+UG8klGRTo5lQjBsqZNZLs QJCFoe83iEKZ34O35ouFdfPUJD2NxpnWk8Dakk6d+ksP6ZW8DhcNhly0G6aSNy6W+Pba 3s6efRKdziFFLJA7by6bVGUmrU5+H6H0FDgZK6wYP2w9ZbL3InMIAyUjUXFS3rwzmsrd uPYUXqkePhtUbpVK9RcDwNrLHti3qbtqzfb/M9ltNMiCr9Vq+NIlZiM6n6GtZwhOz6wT bnqA== X-Gm-Message-State: AN3rC/6zT2Lacrexy+zHcuhOycB77oLOxQPVdbdPKt8Ko5T7SXmERh2D BSNpzCg9iJuaGUTRnbc8fV75KWNMFg== X-Received: by 10.28.87.16 with SMTP id l16mr9745285wmb.114.1492459583212; Mon, 17 Apr 2017 13:06:23 -0700 (PDT) MIME-Version: 1.0 References: <070B2881-A8CF-4106-9921-41E4AEB288A4@bu.edu> <20170417095430.52cd5fdf@xeon-e3> <37C3AD58-419D-4CE9-AA23-A743B8032ECF@bu.edu> In-Reply-To: <37C3AD58-419D-4CE9-AA23-A743B8032ECF@bu.edu> From: Alex Kiselev Date: Mon, 17 Apr 2017 20:06:12 +0000 Message-ID: To: "Fu, Qiaobin" Cc: Stephen Hemminger , "users@dpdk.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] DPDK hash table dynamic growth X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Apr 2017 20:06:24 -0000 If you need only a single reader and a single writer solution you can use any existing hash table implementation and build on top of it your own resizable version. You can try a straightforward approach. Create a new bigger hash table when the old is full and just copy key/values into it. A new reader/lookup function should take into account that during resizing there can be more then one hash table and check all of them. And I wouldn't copy all keys at once, but instead I'd copy N keys at one iteration of the lcore main loop so the resizing process don't block other operations. =D0=BF=D0=BD, 17 =D0=B0=D0=BF=D1=80. 2017 =D0=B3. =D0=B2 20:44, Fu, Qiaobin= : > Thanks @Alex and @Stephen for pointing these valuable materials to me! > > Maybe it=E2=80=99s better that I can clarify my scenarios more. Actually,= we are > using RSS (source + destination IP) to distribute packets, so we are > maintaining one hash table per lcore (thread), and there is no need for > synchronization issue in our case. If there are more solutions, please le= t > me know. I will compare the solutions, and pick up the best suitable one. > > Best, > Qiaobin > > On Apr 17, 2017, at 1:30 PM, Alex Kiselev wrote: > > I would take a look at: > > 1) http://preshing.com/20160201/new-concurrent-hash-maps-for-cpp/ > 2) https://github.com/efficient/libcuckoo > 3) > > http://high-scale-lib.cvs.sourceforge.net/viewvc/high-scale-lib/high-scal= e-lib/org/cliffc/high_scale_lib/NonBlockingHashMap.java?view=3Dmarkup > https://www.youtube.com/watch?v=3DHJ-719EGIts > https://www.youtube.com/watch?v=3DWYXgtXWejRM > > But there is a catch, none of them is written in C. > > 2017-04-17 19:54 GMT+03:00 Stephen Hemminger = : > > On Mon, 17 Apr 2017 15:47:59 +0000 > "Fu, Qiaobin" wrote: > > Hello, > > Currently, I am using the hash library to handle network flows defined as > source and destination IP addresses. I need to find a way to alleviate > memory pressure when the table is full. However, after some research, I > didn=E2=80=99t find any hints on the dynamic growth in the hash library. = Could > anyone point me some hints on this? Thanks. > > Best, > Qiaobin > > > If you need growing hash table, I recommend the lock-free hash table in > the Linux userspace RCU library; > rather than the more limited DPDK one. > > > > > -- > -- > Kiselev Alexander > > > -- -- Kiselev Alexander