DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Lilijun (Jerry)" <jerry.lilijun@huawei.com>
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
	"'dev@dpdk.org'" <dev@dpdk.org>
Cc: wangyunjian <wangyunjian@huawei.com>,
	xudingke <xudingke@huawei.com>,
	"'stable@dpdk.org'" <stable@dpdk.org>, nd <nd@arm.com>,
	"yipeng1.wang@intel.com" <yipeng1.wang@intel.com>,
	nd <nd@arm.com>
Subject: [dpdk-dev] 答复:  [PATCH] lib/librte_hash: add rte_hash_del_key_fixed without compact
Date: Thu, 14 May 2020 01:51:58 +0000	[thread overview]
Message-ID: <40280F65B1B0B44E8089ED31C01616EBA49C810B@dggeml529-mbx.china.huawei.com> (raw)
In-Reply-To: <DBBPR08MB46464C25121DA5ADB768233198BC0@DBBPR08MB4646.eurprd08.prod.outlook.com>



> -----邮件原件-----
> 发件人: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
> 发送时间: 2020年5月14日 9:22
> 收件人: Lilijun (Jerry) <jerry.lilijun@huawei.com>; 'dev@dpdk.org'
> <dev@dpdk.org>
> 抄送: wangyunjian <wangyunjian@huawei.com>; xudingke
> <xudingke@huawei.com>; 'stable@dpdk.org' <stable@dpdk.org>; nd
> <nd@arm.com>; yipeng1.wang@intel.com; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> 主题: RE: [dpdk-dev] [PATCH] lib/librte_hash: add rte_hash_del_key_fixed
> without compact
> 
> <snip>
> 
> > > > >
> > > > > Adding Yipeng, maintainer for hash library
> > > > >
> > > > > >
> > > > > > Thanks for your reply.
> > > > > >
> > > > > > Using rte_hash iterate and delete keys is to free the related
> > > > > > data's
> > > > memory.
> > > > > > There are two reasons why rte_hash_reset() is not properly:
> > > > > > 1)  the reset function just clear all keys, the key's related
> > > > > > data are
> > > leaked.
> > > > > That is a good point. I think this should be documented in the API.
> > > > >
> > > > > > 2)  In some cases, I don't need delete all keys. Just some
> > > > > > selected keys and data are deleted and released.
> > > > > I understand the problem you have pointed out and understand how
> > > > > to reproduce it. But, the use case is not clear to me. Can you
> > > > > please explain the use case?
> > > > [Lilijun (Jerry)]
> > > >
> > > > As you know, the dpdk rte_hash use a fixed size table to store all
> > > keys/datas.
> > > > The memory used by hash table is related with this fixed size.
> > > > In my case, normally the count of keys is about 100,000 but
> > > > sometimes the count may burst up to 30,000,000.
> > > > In order to save memory usage, I create a small hash table with
> > > > 100,000 size and replace to a bigger one with 30,000,000 size when
> > > > there are more keys to be stored. Also when the key's count
> > > > reduced to less than 100,000, I replace the hash table with a
> > > > small one to save the
> > > memory.
> > > Thank you for explaining this. What happens to the reader when you
> > > are deleting from old table and inserting in the new one? Which
> > > table does the reader lookup from?
> > [Lilijun (Jerry)]
> > Lookup functions works well at any time. The problem is in
> > rte_hash_iterate() functions. Some example codes like this:
> > *next = 0;
> > //If rh has 10000 entries at first.
> > while ((idx = rte_hash_iterate(rh, key, data, next)) >= 0) {
> >               rte_hash_del_key(rh, key);  //BUT HERE maybe only delete
> > 9990 keys !!!
> >               free(*data);
> > }
> > //There are still 10 key/datas not freed and will be leaked.
> > rte_hash_free(rh);
> I understand this problem.
> I am trying to understand if there are other problems in the process you are
> following.
> For ex: when you are transferring an entry from the old table, if the reader is
> looking up from the old table, the entry will not be found, even though the
> entry is available in the new table. Can this happen?
> 
[Lilijun (Jerry)] 
Your example may be happen if read and resize are in two thread context and it's like a RCU cases. 
But I think it can be fixed for reader can retry the lookup or use a read-lock while resize using a write-lock.

> <snip the diff>

      reply	other threads:[~2020-05-14  1:52 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-27  2:28 [dpdk-dev] " Lilijun (Jerry)
2020-04-28 20:46 ` Honnappa Nagarahalli
2020-04-29  1:07   ` [dpdk-dev] 答复: " Lilijun (Jerry)
2020-05-05 23:17     ` [dpdk-dev] " Honnappa Nagarahalli
2020-05-06  1:09       ` [dpdk-dev] 答复: " Lilijun (Jerry)
2020-05-12 23:41         ` [dpdk-dev] " Wang, Yipeng1
2020-05-13  1:28           ` [dpdk-dev] 答复: " Lilijun (Jerry)
2020-05-14 17:44             ` [dpdk-dev] " Wang, Yipeng1
2021-03-24 21:33             ` [dpdk-dev] [dpdk-stable] 答复: " Thomas Monjalon
2021-03-24 23:25               ` Wang, Yipeng1
2021-03-25  8:04                 ` Thomas Monjalon
2020-05-13 19:27         ` [dpdk-dev] " Honnappa Nagarahalli
2020-05-14  0:55           ` [dpdk-dev] 答复: " Lilijun (Jerry)
2020-05-14  1:21             ` [dpdk-dev] " Honnappa Nagarahalli
2020-05-14  1:51               ` Lilijun (Jerry) [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=40280F65B1B0B44E8089ED31C01616EBA49C810B@dggeml529-mbx.china.huawei.com \
    --to=jerry.lilijun@huawei.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=dev@dpdk.org \
    --cc=nd@arm.com \
    --cc=stable@dpdk.org \
    --cc=wangyunjian@huawei.com \
    --cc=xudingke@huawei.com \
    --cc=yipeng1.wang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).