From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f176.google.com (mail-qk0-f176.google.com [209.85.220.176]) by dpdk.org (Postfix) with ESMTP id F06ECDE0 for ; Tue, 18 Apr 2017 07:08:29 +0200 (CEST) Received: by mail-qk0-f176.google.com with SMTP id f133so121175623qke.2 for ; Mon, 17 Apr 2017 22:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ESANZyaK7zMarUtY7ky41qrYFOnxLeehaBf5LLCio1s=; b=lr1jyK9ONjB15wbwQUUeXcaPCzri7MKeDHx8WB/l9bUNTf+JltYC61OjTOsy1zz5Q2 yT0GgKnjQOpquHb4OTtVs9dgyao2QgU9YXTdYmjEx8KBrY4vGFo9ATBw3t7SBFp1xn+c LOUGLt+/+Foouz8W8EIqL6PACDvhYa6b4AUHKiYcSfim+3LDJqLEjZaLFmMCALNbLDES 5s1/0e5ZeKMXBRQM/gh6Oi1g6TjzUNymxOVAfGQoXsMYyPOFQMervleRjZ6xxF1P2lCE xdO61lBe/BAxy6+JlnpfIsdyt6UDtsT6/ApMKl+uk/8JRA+qIKkLpjLO4fdGv1C3pYlu DzeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ESANZyaK7zMarUtY7ky41qrYFOnxLeehaBf5LLCio1s=; b=MnbR/kevDyr8qLNSsTUS7643VpCey2UbfJjMmKNU6PKaK9aIFn15NehXWmHrg37gY3 k2DI8D/WmcHwsoqFRagzFQHt1KvETSz9fhMqmX2kGU5zH/aFyzQc7O2gm60iP9IluYvv D4EAI9Fo+lnhrBRF0XpWx8zBWndahx9sLsNwavCfmGf23pnr28zcYWLGs2FtFXL4xMXB zw0UPgMowZ/1v2gkeqCs6UvPG82hTx/ALB4UCiKerIhUsg2bpTZk1gLt1Yv4s0RRtJPj 6Y4+SGsykIO9EXJH+6Im47SMEY+ptGJhxVaoIfmnrtHmYEJL5skhqnjjyQizLkoUglaT fG1A== X-Gm-Message-State: AN3rC/5rGLI4WHHB6/vAq2vhKYvs6MxecAEzvFEmd1KmPmE2EARq1ams vXd16/z8xHOyMcc+fk4zQ6mXjyvSUw== X-Received: by 10.55.24.219 with SMTP id 88mr11657339qky.249.1492492109316; Mon, 17 Apr 2017 22:08:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.12.179.11 with HTTP; Mon, 17 Apr 2017 22:08:28 -0700 (PDT) In-Reply-To: References: <070B2881-A8CF-4106-9921-41E4AEB288A4@bu.edu> <20170417095430.52cd5fdf@xeon-e3> <37C3AD58-419D-4CE9-AA23-A743B8032ECF@bu.edu> From: Shyam Shrivastav Date: Tue, 18 Apr 2017 10:38:28 +0530 Message-ID: To: Alex Kiselev Cc: "Fu, Qiaobin" , Stephen Hemminger , "users@dpdk.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] DPDK hash table dynamic growth X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Apr 2017 05:08:30 -0000 For theory on growing/shrinking hash tables useful MIT class https://www.youtube.com/watch?v=3DBRO7mVIFt08 On Tue, Apr 18, 2017 at 1:36 AM, Alex Kiselev wrote: > If you need only a single reader and a single writer solution you can use > any existing hash table implementation and build on top of it your own > resizable version. You can try a straightforward approach. Create a new > bigger hash table when the old is full and just copy key/values into it. = A > new reader/lookup function should take into account that during resizing > there can be more then one hash table and check all of them. And I wouldn= 't > copy all keys at once, but instead I'd copy N keys at one iteration of th= e > lcore main loop so the resizing process don't block other operations. > > =D0=BF=D0=BD, 17 =D0=B0=D0=BF=D1=80. 2017 =D0=B3. =D0=B2 20:44, Fu, Qiaob= in : > > > Thanks @Alex and @Stephen for pointing these valuable materials to me! > > > > Maybe it=E2=80=99s better that I can clarify my scenarios more. Actuall= y, we are > > using RSS (source + destination IP) to distribute packets, so we are > > maintaining one hash table per lcore (thread), and there is no need for > > synchronization issue in our case. If there are more solutions, please > let > > me know. I will compare the solutions, and pick up the best suitable on= e. > > > > Best, > > Qiaobin > > > > On Apr 17, 2017, at 1:30 PM, Alex Kiselev wrote: > > > > I would take a look at: > > > > 1) http://preshing.com/20160201/new-concurrent-hash-maps-for-cpp/ > > 2) https://github.com/efficient/libcuckoo > > 3) > > > > http://high-scale-lib.cvs.sourceforge.net/viewvc/high- > scale-lib/high-scale-lib/org/cliffc/high_scale_lib/ > NonBlockingHashMap.java?view=3Dmarkup > > https://www.youtube.com/watch?v=3DHJ-719EGIts > > https://www.youtube.com/watch?v=3DWYXgtXWejRM > > > > But there is a catch, none of them is written in C. > > > > 2017-04-17 19:54 GMT+03:00 Stephen Hemminger >: > > > > On Mon, 17 Apr 2017 15:47:59 +0000 > > "Fu, Qiaobin" wrote: > > > > Hello, > > > > Currently, I am using the hash library to handle network flows defined = as > > source and destination IP addresses. I need to find a way to alleviate > > memory pressure when the table is full. However, after some research, I > > didn=E2=80=99t find any hints on the dynamic growth in the hash library= . Could > > anyone point me some hints on this? Thanks. > > > > Best, > > Qiaobin > > > > > > If you need growing hash table, I recommend the lock-free hash table in > > the Linux userspace RCU library; > > rather than the more limited DPDK one. > > > > > > > > > > -- > > -- > > Kiselev Alexander > > > > > > -- > -- > Kiselev Alexander >