[dpdk-dev] [PATCH 1/2] doc: add multithread description to hash library

[dpdk-dev] [PATCH 1/2] doc: add multithread description to hash library

[Yipeng Wang]

Added multithreading related description into programmer
guide of hash library.

Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
 doc/guides/prog_guide/hash_lib.rst | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/doc/guides/prog_guide/hash_lib.rst b/doc/guides/prog_guide/hash_lib.rst
index 180c776..76a1f32 100644
--- a/doc/guides/prog_guide/hash_lib.rst
+++ b/doc/guides/prog_guide/hash_lib.rst
@@ -19,6 +19,8 @@ The main configuration parameters for the hash are:
 
 *   Size of the key in bytes
 
+*   An extra flag used to describe additional settings, for example the multithreading mode of operation (as will be described later)
+
 The hash also allows the configuration of some low-level implementation related parameters such as:
 
 *   Hash function to translate the key into a bucket index
@@ -49,8 +51,7 @@ Apart from these method explained above, the API allows the user three more opti
 Also, the API contains a method to allow the user to look up entries in bursts, achieving higher performance
 than looking up individual entries, as the function prefetches next entries at the time it is operating
 with the first ones, which reduces significantly the impact of the necessary memory accesses.
-Notice that this method uses a pipeline of 8 entries (4 stages of 2 entries), so it is highly recommended
-to use at least 8 entries per burst.
+
 
 The actual data associated with each key can be either managed by the user using a separate table that
 mirrors the hash in terms of number of entries and position of each entry,
@@ -63,11 +64,33 @@ However, this table could also be used for more sophisticated features and provi
 Multi-process support
 ---------------------
 
-The hash library can be used in a multi-process environment, minding that only lookups are thread-safe.
+The hash library can be used in a multi-process environment.
 The only function that can only be used in single-process mode is rte_hash_set_cmp_func(), which sets up
 a custom compare function, which is assigned to a function pointer (therefore, it is not supported in
 multi-process mode).
 
+
+Multi-thread support
+---------------------
+
+The hash library supports multithreading, and the user specifies the needed mode of operation at the creation time of the hash table
+by appropriately setting the flag. In all modes of operation lookups are thread-safe meaning lookups can be called from multiple
+threads concurrently.
+
+For concurrent writes, and concurrent reads and writes the following flag values define the corresponding modes of operation:
+
+*  If the multi-writer flag (RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) is set, multiple threads writing to the table is allowed.
+   Key add, delete, and table reset are protected from other writer threads. With only this flag set, readers are not protected from ongoing writes.
+
+*  If the read/write concurrency (RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY) is set, multithread read/write operation is safe
+   (i.e., no need to stop the readers from accessing the hash table until writers finish their updates. Reads and writes can operate table concurrently).
+
+*  In addition to these two flag values, if the transactional memory flag (RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT) is also set,
+   hardware transactional memory will be used to guarantee the thread safety as long as it is supported by the hardware (for example the Intel® TSX support).
+
+If the platform supports Intel® TSX, it is advised to set the transactional memory flag, as this will speed up concurrent table operations.
+Otherwise concurrent operations will be slower because of the overhead associated with the software locking mechanisms.
+
 Implementation Details
 ----------------------
 
-- 
2.7.4

[dpdk-dev] [PATCH 2/2] hash: add more accurate thread-safety comments

[Yipeng Wang]

Describing the thread-safety support more accurately for
API documentation.

Fixes: f2e3001b53ec ("hash: support read/write concurrency")

Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
 lib/librte_hash/rte_hash.h | 52 ++++++++++++++++++++++++++++++++++------------
 1 file changed, 39 insertions(+), 13 deletions(-)

diff --git a/lib/librte_hash/rte_hash.h b/lib/librte_hash/rte_hash.h
index 1f1a276..748d5cf 100644
--- a/lib/librte_hash/rte_hash.h
+++ b/lib/librte_hash/rte_hash.h
@@ -140,7 +140,9 @@ rte_hash_count(const struct rte_hash *h);
 /**
  * Add a key-value pair to an existing hash table.
  * This operation is not multi-thread safe
- * and should only be called from one thread.
+ * and should only be called from one thread by default.
+ * Thread safety can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to add the key to.
@@ -160,7 +162,9 @@ rte_hash_add_key_data(const struct rte_hash *h, const void *key, void *data);
  * Add a key-value pair with a pre-computed hash value
  * to an existing hash table.
  * This operation is not multi-thread safe
- * and should only be called from one thread.
+ * and should only be called from one thread by default.
+ * Thread safety can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to add the key to.
@@ -181,7 +185,9 @@ rte_hash_add_key_with_hash_data(const struct rte_hash *h, const void *key,
 
 /**
  * Add a key to an existing hash table. This operation is not multi-thread safe
- * and should only be called from one thread.
+ * and should only be called from one thread by default.
+ * Thread safety can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to add the key to.
@@ -199,7 +205,9 @@ rte_hash_add_key(const struct rte_hash *h, const void *key);
 /**
  * Add a key to an existing hash table.
  * This operation is not multi-thread safe
- * and should only be called from one thread.
+ * and should only be called from one thread by default.
+ * Thread safety can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to add the key to.
@@ -219,7 +227,9 @@ rte_hash_add_key_with_hash(const struct rte_hash *h, const void *key, hash_sig_t
 /**
  * Remove a key from an existing hash table.
  * This operation is not multi-thread safe
- * and should only be called from one thread.
+ * and should only be called from one thread by default.
+ * Thread safety can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to remove the key from.
@@ -238,7 +248,9 @@ rte_hash_del_key(const struct rte_hash *h, const void *key);
 /**
  * Remove a key from an existing hash table.
  * This operation is not multi-thread safe
- * and should only be called from one thread.
+ * and should only be called from one thread by default.
+ * Thread safety can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to remove the key from.
@@ -258,7 +270,9 @@ rte_hash_del_key_with_hash(const struct rte_hash *h, const void *key, hash_sig_t
 
 /**
  * Find a key in the hash table given the position.
- * This operation is multi-thread safe.
+ * This operation is multi-thread safe with regarding to other lookup threads.
+ * Read-write concurrency can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to get the key from.
@@ -277,7 +291,9 @@ rte_hash_get_key_with_position(const struct rte_hash *h, const int32_t position,
 
 /**
  * Find a key-value pair in the hash table.
- * This operation is multi-thread safe.
+ * This operation is multi-thread safe with regarding to other lookup threads.
+ * Read-write concurrency can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to look in.
@@ -296,7 +312,9 @@ rte_hash_lookup_data(const struct rte_hash *h, const void *key, void **data);
 /**
  * Find a key-value pair with a pre-computed hash value
  * to an existing hash table.
- * This operation is multi-thread safe.
+ * This operation is multi-thread safe with regarding to other lookup threads.
+ * Read-write concurrency can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to look in.
@@ -317,7 +335,9 @@ rte_hash_lookup_with_hash_data(const struct rte_hash *h, const void *key,
 
 /**
  * Find a key in the hash table.
- * This operation is multi-thread safe.
+ * This operation is multi-thread safe with regarding to other lookup threads.
+ * Read-write concurrency can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to look in.
@@ -335,7 +355,9 @@ rte_hash_lookup(const struct rte_hash *h, const void *key);
 
 /**
  * Find a key in the hash table.
- * This operation is multi-thread safe.
+ * This operation is multi-thread safe with regarding to other lookup threads.
+ * Read-write concurrency can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to look in.
@@ -370,7 +392,9 @@ rte_hash_hash(const struct rte_hash *h, const void *key);
 
 /**
  * Find multiple keys in the hash table.
- * This operation is multi-thread safe.
+ * This operation is multi-thread safe with regarding to other lookup threads.
+ * Read-write concurrency can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to look in.
@@ -391,7 +415,9 @@ rte_hash_lookup_bulk_data(const struct rte_hash *h, const void **keys,
 
 /**
  * Find multiple keys in the hash table.
- * This operation is multi-thread safe.
+ * This operation is multi-thread safe with regarding to other lookup threads.
+ * Read-write concurrency can be enabled by setting flag during
+ * table creation.
  *
  * @param h
  *   Hash table to look in.
-- 
2.7.4

Re: [dpdk-dev] [PATCH 1/2] doc: add multithread description to hash library

[Thomas Monjalon]

Pablo, Marko, review please?

26/07/2018 19:56, Yipeng Wang:
> Added multithreading related description into programmer
> guide of hash library.
> 
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>

Re: [dpdk-dev] [PATCH 1/2] doc: add multithread description to hash library

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> Sent: Sunday, August 5, 2018 9:29 PM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Kovacevic, Marko <marko.kovacevic@intel.com>
> Cc: dev@dpdk.org; Wang, Yipeng1 <yipeng1.wang@intel.com>; Richardson,
> Bruce <bruce.richardson@intel.com>
> Subject: Re: [dpdk-dev] [PATCH 1/2] doc: add multithread description to hash
> library
> 
> Pablo, Marko, review please?

Sorry Thomas, I missed this.

> 
> 26/07/2018 19:56, Yipeng Wang:
> > Added multithreading related description into programmer guide of hash
> > library.
> >
> > Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>

Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
> 
> 
> 
> 

Re: [dpdk-dev] [PATCH 2/2] hash: add more accurate thread-safety comments

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Thursday, July 26, 2018 6:56 PM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Kovacevic, Marko <marko.kovacevic@intel.com>
> Cc: dev@dpdk.org; Wang, Yipeng1 <yipeng1.wang@intel.com>; Richardson,
> Bruce <bruce.richardson@intel.com>
> Subject: [PATCH 2/2] hash: add more accurate thread-safety comments
> 
> Describing the thread-safety support more accurately for API documentation.
> 
> Fixes: f2e3001b53ec ("hash: support read/write concurrency")
> 
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>

Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

Re: [dpdk-dev] [PATCH 1/2] doc: add multithread description to hash library

[Thomas Monjalon]

> > Pablo, Marko, review please?
> 
> Sorry Thomas, I missed this.
> 
> > 
> > 26/07/2018 19:56, Yipeng Wang:
> > > Added multithreading related description into programmer guide of hash
> > > library.
> > >
> > > Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> 
> Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

Series applied, thanks