Re: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value

Re: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value

[Guvva, Vijaya]

Hi Yipeng,
Sure, It will be great help if you can include these changes in the patches you are proposing with the necessary locks.

Thanks,
Vijay 

-----Original Message-----
From: Wang, Yipeng1 <yipeng1.wang@intel.com> 
Sent: Thursday, May 24, 2018 10:47 AM
To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Guvva, Vijaya <Vijaya.Guvva@cavium.com>; Richardson, Bruce <bruce.richardson@intel.com>
Cc: dev@dpdk.org
Subject: RE: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value

Hi, Vijaya, 

Thanks for contributing the new API.

We actually have a patch to support read-write concurrency for rte_hash coming in a couple of weeks.  The new del function you proposed may need to be protected under the new concurrency scheme as well. If you like, we could collaborate together to fit your del function under the new concurrency support. 

Thanks
Yipeng

>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara Guarch, 
>Pablo
>Sent: Thursday, May 17, 2018 1:06 AM
>To: Vijaya Mohan Guvva <vguvva@caviumnetworks.com>; Richardson, Bruce 
><bruce.richardson@intel.com>
>Cc: dev@dpdk.org
>Subject: Re: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to 
>return stored value
>
>Hi Vijaya,
>
>> -----Original Message-----
>> From: Vijaya Mohan Guvva [mailto:vguvva@caviumnetworks.com]
>> Sent: Thursday, May 17, 2018 2:27 AM
>> To: Richardson, Bruce <bruce.richardson@intel.com>; De Lara Guarch, 
>> Pablo <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; Vijaya Mohan Guvva <vguvva@caviumnetworks.com>
>> Subject: [PATCH V2] librte_hash: new hash del abi to return stored 
>> value
>>
>
>You are actually adding new API, not ABI, so I would reword the commit message.
>
>"hash: add API to return stored value at deletion" maybe?
>
>I would add some information in the commit message and move the 
>changelog (V1, V2 notes) after the three dashes.
>
>> V2:
>> Adding another new interface rte_hash_del_key_data to delete key from 
>> hash table and return stored data.
>>
>> V1:
>> Add a new key delete interface rte_hash_del_key_with_hash_data to 
>> delete the key from hash and return the value stored. This is useful 
>> for hash users to free the data stored in the table after key delete 
>> and to avoid maintaining a user data array in the dpdk application.
>>
>> Signed-off-by: Vijaya Mohan Guvva <vguvva@caviumnetworks.com>
>> ---
>>  lib/librte_hash/rte_cuckoo_hash.c | 30 +++++++++++++++++++++++---
>>  lib/librte_hash/rte_hash.h        | 45
>> +++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 72 insertions(+), 3 deletions(-)
>
>...
>
>> +int32_t
>> +rte_hash_del_key_with_hash_data(const struct rte_hash *h, const void *key,
>> +				hash_sig_t sig, void **data);
>> +
>> +/**
>>   * Remove a key from an existing hash table.
>>   * This operation is not multi-thread safe
>>   * and should only be called from one thread.
>> --
>> 1.8.3.1
>
>You need to update the version.map file to add the two new functions 
>(you might need to wait until 18.05 is released, so you can use the 18.08 tag).

Re: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value

[Honnappa Nagarahalli]

Hi Yipeng,
	Can you please elaborate on the read-write concurrency changes you are making? I am looking at few changes myself, want to see how we can align.

Issues that I am looking at are:
1) The delete APIs are not multithread safe - from writers perspective
2) Memory ordering between writer and reader while adding the keys
3) Additional APIs to support RCU while deleting the entries

Thank you,
Honnappa

-----Original Message-----
From: dev <dev-bounces@dpdk.org> On Behalf Of Guvva, Vijaya
Sent: Thursday, May 24, 2018 1:35 PM
To: Wang, Yipeng1 <yipeng1.wang@intel.com>; De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value

Hi Yipeng,
Sure, It will be great help if you can include these changes in the patches you are proposing with the necessary locks.

Thanks,
Vijay 

-----Original Message-----
From: Wang, Yipeng1 <yipeng1.wang@intel.com>
Sent: Thursday, May 24, 2018 10:47 AM
To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Guvva, Vijaya <Vijaya.Guvva@cavium.com>; Richardson, Bruce <bruce.richardson@intel.com>
Cc: dev@dpdk.org
Subject: RE: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value

Hi, Vijaya, 

Thanks for contributing the new API.

We actually have a patch to support read-write concurrency for rte_hash coming in a couple of weeks.  The new del function you proposed may need to be protected under the new concurrency scheme as well. If you like, we could collaborate together to fit your del function under the new concurrency support. 

Thanks
Yipeng

>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara Guarch, 
>Pablo
>Sent: Thursday, May 17, 2018 1:06 AM
>To: Vijaya Mohan Guvva <vguvva@caviumnetworks.com>; Richardson, Bruce 
><bruce.richardson@intel.com>
>Cc: dev@dpdk.org
>Subject: Re: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to 
>return stored value
>
>Hi Vijaya,
>
>> -----Original Message-----
>> From: Vijaya Mohan Guvva [mailto:vguvva@caviumnetworks.com]
>> Sent: Thursday, May 17, 2018 2:27 AM
>> To: Richardson, Bruce <bruce.richardson@intel.com>; De Lara Guarch, 
>> Pablo <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; Vijaya Mohan Guvva <vguvva@caviumnetworks.com>
>> Subject: [PATCH V2] librte_hash: new hash del abi to return stored 
>> value
>>
>
>You are actually adding new API, not ABI, so I would reword the commit message.
>
>"hash: add API to return stored value at deletion" maybe?
>
>I would add some information in the commit message and move the 
>changelog (V1, V2 notes) after the three dashes.
>
>> V2:
>> Adding another new interface rte_hash_del_key_data to delete key from 
>> hash table and return stored data.
>>
>> V1:
>> Add a new key delete interface rte_hash_del_key_with_hash_data to 
>> delete the key from hash and return the value stored. This is useful 
>> for hash users to free the data stored in the table after key delete 
>> and to avoid maintaining a user data array in the dpdk application.
>>
>> Signed-off-by: Vijaya Mohan Guvva <vguvva@caviumnetworks.com>
>> ---
>>  lib/librte_hash/rte_cuckoo_hash.c | 30 +++++++++++++++++++++++---
>>  lib/librte_hash/rte_hash.h        | 45
>> +++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 72 insertions(+), 3 deletions(-)
>
>...
>
>> +int32_t
>> +rte_hash_del_key_with_hash_data(const struct rte_hash *h, const void *key,
>> +				hash_sig_t sig, void **data);
>> +
>> +/**
>>   * Remove a key from an existing hash table.
>>   * This operation is not multi-thread safe
>>   * and should only be called from one thread.
>> --
>> 1.8.3.1
>
>You need to update the version.map file to add the two new functions 
>(you might need to wait until 18.05 is released, so you can use the 18.08 tag).

Re: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value

[Wang, Yipeng1]

Hi, Vijaya, 

Thanks for contributing the new API.

We actually have a patch to support read-write concurrency for rte_hash coming in a couple of weeks.  The new del function you proposed may need to be protected under the new concurrency scheme as well. If you like, we could collaborate together to fit your del function under the new concurrency support. 

Thanks
Yipeng

>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara Guarch, Pablo
>Sent: Thursday, May 17, 2018 1:06 AM
>To: Vijaya Mohan Guvva <vguvva@caviumnetworks.com>; Richardson, Bruce <bruce.richardson@intel.com>
>Cc: dev@dpdk.org
>Subject: Re: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value
>
>Hi Vijaya,
>
>> -----Original Message-----
>> From: Vijaya Mohan Guvva [mailto:vguvva@caviumnetworks.com]
>> Sent: Thursday, May 17, 2018 2:27 AM
>> To: Richardson, Bruce <bruce.richardson@intel.com>; De Lara Guarch, Pablo
>> <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; Vijaya Mohan Guvva <vguvva@caviumnetworks.com>
>> Subject: [PATCH V2] librte_hash: new hash del abi to return stored value
>>
>
>You are actually adding new API, not ABI, so I would reword the commit message.
>
>"hash: add API to return stored value at deletion" maybe?
>
>I would add some information in the commit message and move the changelog (V1, V2 notes)
>after the three dashes.
>
>> V2:
>> Adding another new interface rte_hash_del_key_data to delete key from hash
>> table and return stored data.
>>
>> V1:
>> Add a new key delete interface rte_hash_del_key_with_hash_data to delete the
>> key from hash and return the value stored. This is useful for hash users to free
>> the data stored in the table after key delete and to avoid maintaining a user data
>> array in the dpdk application.
>>
>> Signed-off-by: Vijaya Mohan Guvva <vguvva@caviumnetworks.com>
>> ---
>>  lib/librte_hash/rte_cuckoo_hash.c | 30 +++++++++++++++++++++++---
>>  lib/librte_hash/rte_hash.h        | 45
>> +++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 72 insertions(+), 3 deletions(-)
>
>...
>
>> +int32_t
>> +rte_hash_del_key_with_hash_data(const struct rte_hash *h, const void *key,
>> +				hash_sig_t sig, void **data);
>> +
>> +/**
>>   * Remove a key from an existing hash table.
>>   * This operation is not multi-thread safe
>>   * and should only be called from one thread.
>> --
>> 1.8.3.1
>
>You need to update the version.map file to add the two new functions
>(you might need to wait until 18.05 is released, so you can use the 18.08 tag).

Re: [dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value

[De Lara Guarch, Pablo]

Hi Vijaya,

> -----Original Message-----
> From: Vijaya Mohan Guvva [mailto:vguvva@caviumnetworks.com]
> Sent: Thursday, May 17, 2018 2:27 AM
> To: Richardson, Bruce <bruce.richardson@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; Vijaya Mohan Guvva <vguvva@caviumnetworks.com>
> Subject: [PATCH V2] librte_hash: new hash del abi to return stored value
> 

You are actually adding new API, not ABI, so I would reword the commit message.

"hash: add API to return stored value at deletion" maybe?

I would add some information in the commit message and move the changelog (V1, V2 notes)
after the three dashes.
 
> V2:
> Adding another new interface rte_hash_del_key_data to delete key from hash
> table and return stored data.
> 
> V1:
> Add a new key delete interface rte_hash_del_key_with_hash_data to delete the
> key from hash and return the value stored. This is useful for hash users to free
> the data stored in the table after key delete and to avoid maintaining a user data
> array in the dpdk application.
> 
> Signed-off-by: Vijaya Mohan Guvva <vguvva@caviumnetworks.com>
> ---
>  lib/librte_hash/rte_cuckoo_hash.c | 30 +++++++++++++++++++++++---
>  lib/librte_hash/rte_hash.h        | 45
> +++++++++++++++++++++++++++++++++++++++
>  2 files changed, 72 insertions(+), 3 deletions(-)

...

> +int32_t
> +rte_hash_del_key_with_hash_data(const struct rte_hash *h, const void *key,
> +				hash_sig_t sig, void **data);
> +
> +/**
>   * Remove a key from an existing hash table.
>   * This operation is not multi-thread safe
>   * and should only be called from one thread.
> --
> 1.8.3.1

You need to update the version.map file to add the two new functions
(you might need to wait until 18.05 is released, so you can use the 18.08 tag).

[dpdk-dev] [PATCH V2] librte_hash: new hash del abi to return stored value

[Vijaya Mohan Guvva]

V2:
Adding another new interface rte_hash_del_key_data to delete key
from hash table and return stored data.

V1:
Add a new key delete interface rte_hash_del_key_with_hash_data to
delete the key from hash and return the value stored. This is useful
for hash users to free the data stored in the table after key delete
and to avoid maintaining a user data array in the dpdk application.

Signed-off-by: Vijaya Mohan Guvva <vguvva@caviumnetworks.com>
---
 lib/librte_hash/rte_cuckoo_hash.c | 30 +++++++++++++++++++++++---
 lib/librte_hash/rte_hash.h        | 45 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 72 insertions(+), 3 deletions(-)

diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c
index a07543a..6ea0ef0 100644
--- a/lib/librte_hash/rte_cuckoo_hash.c
+++ b/lib/librte_hash/rte_cuckoo_hash.c
@@ -808,7 +808,7 @@ struct rte_hash *
 
 static inline int32_t
 __rte_hash_del_key_with_hash(const struct rte_hash *h, const void *key,
-						hash_sig_t sig)
+						hash_sig_t sig, void **data)
 {
 	uint32_t bucket_idx;
 	hash_sig_t alt_hash;
@@ -827,6 +827,8 @@ struct rte_hash *
 			k = (struct rte_hash_key *) ((char *)keys +
 					bkt->key_idx[i] * h->key_entry_size);
 			if (rte_hash_cmp_eq(key, k->key, h) == 0) {
+				if (data != NULL)
+					*data = k->pdata;
 				remove_entry(h, bkt, i);
 
 				/*
@@ -852,6 +854,8 @@ struct rte_hash *
 			k = (struct rte_hash_key *) ((char *)keys +
 					bkt->key_idx[i] * h->key_entry_size);
 			if (rte_hash_cmp_eq(key, k->key, h) == 0) {
+				if (data != NULL)
+					*data = k->pdata;
 				remove_entry(h, bkt, i);
 
 				/*
@@ -869,18 +873,38 @@ struct rte_hash *
 }
 
 int32_t
+rte_hash_del_key_with_hash_data(const struct rte_hash *h,
+			const void *key, hash_sig_t sig, void **data)
+{
+	RETURN_IF_TRUE(((h == NULL) || (key == NULL) ||
+			(data == NULL)), -EINVAL);
+	return __rte_hash_del_key_with_hash(h, key, sig, data);
+}
+
+int32_t
 rte_hash_del_key_with_hash(const struct rte_hash *h,
 			const void *key, hash_sig_t sig)
 {
 	RETURN_IF_TRUE(((h == NULL) || (key == NULL)), -EINVAL);
-	return __rte_hash_del_key_with_hash(h, key, sig);
+	return __rte_hash_del_key_with_hash(h, key, sig, NULL);
+}
+
+int32_t
+rte_hash_del_key_data(const struct rte_hash *h, const void *key,
+		      void **data)
+{
+	RETURN_IF_TRUE(((h == NULL) || (key == NULL) ||
+			(data == NULL)), -EINVAL);
+	return __rte_hash_del_key_with_hash(h, key,
+					    rte_hash_hash(h, key), data);
 }
 
 int32_t
 rte_hash_del_key(const struct rte_hash *h, const void *key)
 {
 	RETURN_IF_TRUE(((h == NULL) || (key == NULL)), -EINVAL);
-	return __rte_hash_del_key_with_hash(h, key, rte_hash_hash(h, key));
+	return __rte_hash_del_key_with_hash(h, key,
+					    rte_hash_hash(h, key), NULL);
 }
 
 int
diff --git a/lib/librte_hash/rte_hash.h b/lib/librte_hash/rte_hash.h
index f71ca9f..e0c08e3 100644
--- a/lib/librte_hash/rte_hash.h
+++ b/lib/librte_hash/rte_hash.h
@@ -222,6 +222,51 @@ struct rte_hash *
 rte_hash_del_key(const struct rte_hash *h, const void *key);
 
 /**
+ * Remove a key from an existing hash table and return data stored.
+ * This operation is not multi-thread safe
+ * and should only be called from one thread.
+ *
+ * @param h
+ *   Hash table to remove the key from.
+ * @param key
+ *   Key to remove from the hash table.
+ * @param data
+ *   Output with pointer to data returned from the hash table.
+ * @return
+ *   - -EINVAL if the parameters are invalid.
+ *   - -ENOENT if the key is not found.
+ *   - A positive value that can be used by the caller as an offset into an
+ *     array of user data. This value is unique for this key, and is the same
+ *     value that was returned when the key was added.
+ */
+int32_t
+rte_hash_del_key_data(const struct rte_hash *h, const void *key, void **data);
+
+/**
+ * Remove a key from an existing hash table and return data stored.
+ * This operation is not multi-thread safe
+ * and should only be called from one thread.
+ *
+ * @param h
+ *   Hash table to remove the key from.
+ * @param key
+ *   Key to remove from the hash table.
+ * @param sig
+ *   Precomputed hash value for 'key'.
+ * @param data
+ *   Output with pointer to data returned from the hash table.
+ * @return
+ *   - -EINVAL if the parameters are invalid.
+ *   - -ENOENT if the key is not found.
+ *   - A positive value that can be used by the caller as an offset into an
+ *     array of user data. This value is unique for this key, and is the same
+ *     value that was returned when the key was added.
+ */
+int32_t
+rte_hash_del_key_with_hash_data(const struct rte_hash *h, const void *key,
+				hash_sig_t sig, void **data);
+
+/**
  * Remove a key from an existing hash table.
  * This operation is not multi-thread safe
  * and should only be called from one thread.
-- 
1.8.3.1