Re: [dpdk-dev] [PATCH] lib/librte_hash: add rte_hash_del_key_fixed without compact

[Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>]

<snip>

> > > >
> > > > Adding Yipeng, maintainer for hash library
> > > >
> > > > >
> > > > > Thanks for your reply.
> > > > >
> > > > > Using rte_hash iterate and delete keys is to free the related
> > > > > data's
> > > memory.
> > > > > There are two reasons why rte_hash_reset() is not properly:
> > > > > 1)  the reset function just clear all keys, the key's related
> > > > > data are
> > leaked.
> > > > That is a good point. I think this should be documented in the API.
> > > >
> > > > > 2)  In some cases, I don't need delete all keys. Just some
> > > > > selected keys and data are deleted and released.
> > > > I understand the problem you have pointed out and understand how
> > > > to reproduce it. But, the use case is not clear to me. Can you
> > > > please explain the use case?
> > > [Lilijun (Jerry)]
> > >
> > > As you know, the dpdk rte_hash use a fixed size table to store all
> > keys/datas.
> > > The memory used by hash table is related with this fixed size.
> > > In my case, normally the count of keys is about 100,000 but
> > > sometimes the count may burst up to 30,000,000.
> > > In order to save memory usage, I create a small hash table with
> > > 100,000 size and replace to a bigger one with 30,000,000 size when
> > > there are more keys to be stored. Also when the key's count reduced
> > > to less than 100,000, I replace the hash table with a small one to
> > > save the
> > memory.
> > Thank you for explaining this. What happens to the reader when you are
> > deleting from old table and inserting in the new one? Which table does
> > the reader lookup from?
> [Lilijun (Jerry)]
> Lookup functions works well at any time. The problem is in rte_hash_iterate()
> functions. Some example codes like this:
> *next = 0;
> //If rh has 10000 entries at first.
> while ((idx = rte_hash_iterate(rh, key, data, next)) >= 0) {
>               rte_hash_del_key(rh, key);  //BUT HERE maybe only delete 9990
> keys !!!
>               free(*data);
> }
> //There are still 10 key/datas not freed and will be leaked.
> rte_hash_free(rh);
I understand this problem.
I am trying to understand if there are other problems in the process you are following.
For ex: when you are transferring an entry from the old table, if the reader is looking up from the old table, the entry will not be found, even though the entry is available in the new table. Can this happen?

<snip the diff>