* [dpdk-dev] [PATCH 0/5] table: add key mask for hash tables
@ 2017-08-24 13:53 Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 1/5] " Cristian Dumitrescu
` (4 more replies)
0 siblings, 5 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-08-24 13:53 UTC (permalink / raw)
To: dev
Main changes:
1. The key_mask parameter is added to all the hash tables that were
previously missing it, as well to the hash compute function. This was
first started in DPDK 2.0, but was only implemented for a couple of
hash tables. The benefit of this approach is that it allows for better
performance for large keys (bigger than 16 bytes), while it preserves
the same performance for small keys [Q&A1].
2. The precomputed key signature (i.e. non-"do-sig") versions have been
removed for all the hash tables, so now the key signature is always
computed on every lookup. Note that this approach also allows for the
precomputed key signature scheme [Q&A2].
3. API cleanup: single parameter structure common for all hash tables.
Q&A:
Q1: How is better lookup performance achieved by using key mask approach
for hash tables?
A1: This approach eliminates the need to consolidate the lookup key in a
single contiguous buffer where the relevant packet fields are written
one by one, which is a very costly operation that also hash strong data
dependencies.
Q2: How can the pre-computed key signature scheme be implemented with
current approach?
A2: The application can implement a straightforward custom hash function
that simply reads the pre-computed key signature from a given offset
in the input key buffer where it has been stored prior to the lookup
operation.
Cristian Dumitrescu (5):
table: add key mask for hash tables
test: update due to api changes in librte_table
test-pipeline: update due to api changes in librte_table
ip_pipeline: update due to api changes in librte_table
deprecation: removed the librte_table notice
doc/guides/rel_notes/deprecation.rst | 8 -
.../pipeline/pipeline_flow_classification.c | 9 +-
.../pipeline/pipeline_flow_classification_be.c | 54 +-
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 18 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 18 +-
lib/librte_table/rte_table_hash.h | 302 +--------
lib/librte_table/rte_table_hash_cuckoo.c | 200 +++---
lib/librte_table/rte_table_hash_ext.c | 407 ++++--------
lib/librte_table/rte_table_hash_key16.c | 691 +++++----------------
lib/librte_table/rte_table_hash_key32.c | 381 +++++++-----
lib/librte_table/rte_table_hash_key8.c | 654 +++++--------------
lib/librte_table/rte_table_hash_lru.c | 506 ++++++---------
test/test-pipeline/main.h | 5 +-
test/test-pipeline/pipeline_hash.c | 107 +---
test/test/test_table.c | 1 +
test/test/test_table.h | 3 +-
test/test/test_table_combined.c | 140 ++---
test/test/test_table_tables.c | 148 ++---
18 files changed, 1160 insertions(+), 2492 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH 1/5] table: add key mask for hash tables
2017-08-24 13:53 [dpdk-dev] [PATCH 0/5] table: add key mask for hash tables Cristian Dumitrescu
@ 2017-08-24 13:53 ` Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 0/5] " Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 2/5] test: update due to api changes in librte_table Cristian Dumitrescu
` (3 subsequent siblings)
4 siblings, 1 reply; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-08-24 13:53 UTC (permalink / raw)
To: dev
Main changes:
1. The key_mask parameter is added to all the hash tables that were
previously missing it, as well to the hash compute function. This was
first started in DPDK 2.0, but was only implemented for a couple of
hash tables. The benefit of this approach is that it allows for better
performance for large keys (bigger than 16 bytes), while it preserves
the same performance for small keys [Q&A1].
2. The precomputed key signature (i.e. non-"do-sig") versions have been
removed for all the hash tables, so now the key signature is always
computed on every lookup. Note that this approach also allows for the
precomputed key signature scheme [Q&A2].
3. API cleanup: single parameter structure common for all hash tables.
Q&A:
Q1: How is better lookup performance achieved by using key mask approach
for hash tables?
A1: This approach eliminates the need to consolidate the lookup key in a
single contiguous buffer where the relevant packet fields are written
one by one, which is a very costly operation that also hash strong data
dependencies.
Q2: How can the pre-computed key signature scheme be implemented with
current approach?
A2: The application can implement a straightforward custom hash function
that simply reads the pre-computed key signature from a given offset
in the input key buffer where it has been stored prior to the lookup
operation.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 302 +-------------
lib/librte_table/rte_table_hash_cuckoo.c | 200 ++++-----
lib/librte_table/rte_table_hash_ext.c | 407 ++++++------------
lib/librte_table/rte_table_hash_key16.c | 691 ++++++++-----------------------
lib/librte_table/rte_table_hash_key32.c | 381 +++++++++--------
lib/librte_table/rte_table_hash_key8.c | 654 +++++++----------------------
lib/librte_table/rte_table_hash_lru.c | 506 +++++++++-------------
7 files changed, 981 insertions(+), 2160 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 57505a6..978ae98 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -74,19 +74,7 @@ extern "C" {
* 4 keys, potentially until all keys in this bucket are examined. The
* extendible bucket logic requires maintaining specific data structures
* per table and per each bucket.
- * 2. Key signature computation:
- * a. Pre-computed key signature: The key lookup operation is split between
- * two CPU cores. The first CPU core (typically the CPU core performing
- * packet RX) extracts the key from the input packet, computes the key
- * signature and saves both the key and the key signature in the packet
- * buffer as packet meta-data. The second CPU core reads both the key and
- * the key signature from the packet meta-data and performs the bucket
- * search step of the key lookup operation.
- * b. Key signature computed on lookup (do-sig): The same CPU core reads
- * the key from the packet meta-data, uses it to compute the key
- * signature and also performs the bucket search step of the key lookup
- * operation.
- * 3. Key size:
+ * 2. Key size:
* a. Configurable key size
* b. Single key size (8-byte, 16-byte or 32-byte key size)
*
@@ -98,59 +86,28 @@ extern "C" {
/** Hash function */
typedef uint64_t (*rte_table_hash_op_hash)(
void *key,
+ void *key_mask,
uint32_t key_size,
uint64_t seed);
-/**
- * Hash tables with configurable key size
- *
- */
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_ext_params {
+/** Hash table parameters */
+struct rte_table_hash_params {
+ /** Name */
+ const char *name;
+
/** Key size (number of bytes) */
uint32_t key_size;
- /** Maximum number of keys */
- uint32_t n_keys;
-
- /** Number of hash table buckets. Each bucket stores up to 4 keys. */
- uint32_t n_buckets;
-
- /** Number of hash table bucket extensions. Each bucket extension has
- space for 4 keys and each bucket can have 0, 1 or more extensions. */
- uint32_t n_buckets_ext;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed value for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
/** Byte offset within packet meta-data where the key is located */
uint32_t key_offset;
-};
-/** Extendible bucket hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_ext_ops;
-
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_ext_dosig_ops;
-
-/** LRU hash table parameters */
-struct rte_table_hash_lru_params {
- /** Key size (number of bytes) */
- uint32_t key_size;
+ /** Key mask */
+ uint8_t *key_mask;
- /** Maximum number of keys */
+ /** Number of keys */
uint32_t n_keys;
- /** Number of hash table buckets. Each bucket stores up to 4 keys. */
+ /** Number of buckets */
uint32_t n_buckets;
/** Hash function */
@@ -158,239 +115,22 @@ struct rte_table_hash_lru_params {
/** Seed value for the hash function */
uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
-/** LRU hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_lru_ops;
-
-/** LRU hash table operations for key signature computed on lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_lru_dosig_ops;
-
-/**
- * 8-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key8_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
};
-/** LRU hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_key8_lru_ops;
-
-/** LRU hash table operations for key signature computed on lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key8_lru_dosig_ops;
-
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key8_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
-/** Extendible bucket hash table operations for pre-computed key signature */
+/** Extendible bucket hash table operations */
+extern struct rte_table_ops rte_table_hash_ext_ops;
extern struct rte_table_ops rte_table_hash_key8_ext_ops;
-
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key8_ext_dosig_ops;
-
-/**
- * 16-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key16_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
-/** LRU hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_key16_lru_ops;
-
-/** LRU hash table operations for key signature computed on lookup
- ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key16_lru_dosig_ops;
-
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key16_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
-/** Extendible bucket operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key16_ext_ops;
-
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key16_ext_dosig_ops;
-
-/**
- * 32-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key32_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
-/** LRU hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_key32_lru_ops;
-
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key32_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
-/** Extendible bucket hash table operations */
extern struct rte_table_ops rte_table_hash_key32_ext_ops;
-/** Cuckoo hash table parameters */
-struct rte_table_hash_cuckoo_params {
- /** Key size (number of bytes */
- uint32_t key_size;
-
- /** Maximum number of hash table entries */
- uint32_t n_keys;
-
- /** Hash function used to calculate hash */
- rte_table_hash_op_hash f_hash;
-
- /** Seed value or Init value used by f_hash */
- uint32_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Hash table name */
- const char *name;
-};
+/** LRU hash table operations */
+extern struct rte_table_ops rte_table_hash_lru_ops;
+extern struct rte_table_ops rte_table_hash_key8_lru_ops;
+extern struct rte_table_ops rte_table_hash_key16_lru_ops;
+extern struct rte_table_ops rte_table_hash_key32_lru_ops;
-/** Cuckoo hash table operations */
-extern struct rte_table_ops rte_table_hash_cuckoo_dosig_ops;
+/* Cuckoo hash table operations */
+extern struct rte_table_ops rte_table_hash_cuckoo_ops;
#ifdef __cplusplus
}
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
index da1597f..b3e8eb9 100644
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2016 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
@@ -66,25 +66,28 @@ struct rte_table_hash {
uint32_t n_keys;
rte_table_hash_op_hash f_hash;
uint32_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
- const char *name;
/* cuckoo hash table object */
struct rte_hash *h_table;
/* Lookup table */
- uint8_t memory[0] __rte_cache_aligned; };
+ uint8_t memory[0] __rte_cache_aligned;
+};
static int
-check_params_create_hash_cuckoo(const struct
-rte_table_hash_cuckoo_params *params) {
- /* Check for valid parameters */
+check_params_create_hash_cuckoo(struct rte_table_hash_params *params)
+{
if (params == NULL) {
RTE_LOG(ERR, TABLE, "NULL Input Parameters.\n");
return -EINVAL;
}
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "Table name is NULL.\n");
+ return -EINVAL;
+ }
+
if (params->key_size == 0) {
RTE_LOG(ERR, TABLE, "Invalid key_size.\n");
return -EINVAL;
@@ -100,11 +103,6 @@ rte_table_hash_cuckoo_params *params) {
return -EINVAL;
}
- if (params->name == NULL) {
- RTE_LOG(ERR, TABLE, "Table name is NULL.\n");
- return -EINVAL;
- }
-
return 0;
}
@@ -113,34 +111,25 @@ rte_table_hash_cuckoo_create(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_hash *rte_hash_handle;
+ struct rte_table_hash_params *p = params;
+ struct rte_hash *h_table;
struct rte_table_hash *t;
uint32_t total_size, total_cl_size;
/* Check input parameters */
- struct rte_table_hash_cuckoo_params *p =
- (struct rte_table_hash_cuckoo_params *) params;
-
if (check_params_create_hash_cuckoo(params))
return NULL;
/* Memory allocation */
- total_cl_size =
- (sizeof(struct rte_table_hash) +
- RTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;
- total_cl_size += (p->n_keys * entry_size +
- RTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;
+ total_cl_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));
+ total_cl_size += RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
total_size = total_cl_size * RTE_CACHE_LINE_SIZE;
- t = rte_zmalloc_socket("TABLE",
- total_size,
- RTE_CACHE_LINE_SIZE,
- socket_id);
+ t = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (t == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for Cuckoo hash table\n",
- __func__,
- (uint32_t)sizeof(struct rte_table_hash));
+ "%s: Cannot allocate %u bytes for cuckoo hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
@@ -154,13 +143,13 @@ rte_table_hash_cuckoo_create(void *params,
.name = p->name
};
- rte_hash_handle = rte_hash_find_existing(p->name);
- if (rte_hash_handle == NULL) {
- rte_hash_handle = rte_hash_create(&hash_cuckoo_params);
- if (NULL == rte_hash_handle) {
+ h_table = rte_hash_find_existing(p->name);
+ if (h_table == NULL) {
+ h_table = rte_hash_create(&hash_cuckoo_params);
+ if (h_table == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: failed to create cuckoo hash table. keysize: %u",
- __func__, hash_cuckoo_params.key_len);
+ "%s: failed to create cuckoo hash table %s\n",
+ __func__, p->name);
rte_free(t);
return NULL;
}
@@ -172,26 +161,22 @@ rte_table_hash_cuckoo_create(void *params,
t->n_keys = p->n_keys;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
- t->name = p->name;
- t->h_table = rte_hash_handle;
+ t->h_table = h_table;
RTE_LOG(INFO, TABLE,
- "%s: Cuckoo Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ "%s: Cuckoo hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
return t;
}
static int
rte_table_hash_cuckoo_free(void *table) {
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
struct rte_table_hash *t = table;
+ if (table == NULL)
+ return -EINVAL;
+
rte_hash_free(t->h_table);
rte_free(t);
@@ -200,25 +185,18 @@ rte_table_hash_cuckoo_free(void *table) {
static int
rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
- int *key_found, void **entry_ptr) {
+ int *key_found, void **entry_ptr)
+{
+ struct rte_table_hash *t = table;
int pos = 0;
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (key == NULL) {
- RTE_LOG(ERR, TABLE, "%s: key parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (entry == NULL) {
- RTE_LOG(ERR, TABLE, "%s: entry parameter is NULL\n", __func__);
+ /* Check input parameters */
+ if ((table == NULL) ||
+ (key == NULL) ||
+ (entry == NULL) ||
+ (key_found == NULL) ||
+ (entry_ptr == NULL))
return -EINVAL;
- }
-
- struct rte_table_hash *t = table;
/* Find Existing entries */
pos = rte_hash_lookup(t->h_table, key);
@@ -231,17 +209,15 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
*entry_ptr = existing_entry;
return 0;
-} else if (pos == -ENOENT) {
- /* Entry not found. Adding new entry */
+ }
+
+ if (pos == -ENOENT) {
+ /* Entry not found. Adding new entry */
uint8_t *new_entry;
pos = rte_hash_add_key(t->h_table, key);
- if (pos < 0) {
- RTE_LOG(ERR, TABLE,
- "%s: Entry not added, status : %u\n",
- __func__, pos);
+ if (pos < 0)
return pos;
- }
new_entry = &t->memory[pos * t->entry_size];
memcpy(new_entry, entry, t->entry_size);
@@ -250,25 +226,22 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
*entry_ptr = new_entry;
return 0;
}
+
return pos;
}
static int
rte_table_hash_cuckoo_entry_delete(void *table, void *key,
- int *key_found, __rte_unused void *entry) {
+ int *key_found, void *entry)
+{
+ struct rte_table_hash *t = table;
int pos = 0;
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (key == NULL) {
- RTE_LOG(ERR, TABLE, "%s: key parameter is NULL\n", __func__);
+ /* Check input parameters */
+ if ((table == NULL) ||
+ (key == NULL) ||
+ (key_found == NULL))
return -EINVAL;
- }
-
- struct rte_table_hash *t = table;
pos = rte_hash_del_key(t->h_table, key);
if (pos >= 0) {
@@ -279,20 +252,21 @@ rte_table_hash_cuckoo_entry_delete(void *table, void *key,
memcpy(entry, entry_ptr, t->entry_size);
memset(&t->memory[pos * t->entry_size], 0, t->entry_size);
+ return 0;
}
+ *key_found = 0;
return pos;
}
-
static int
-rte_table_hash_cuckoo_lookup_dosig(void *table,
+rte_table_hash_cuckoo_lookup(void *table,
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
void **entries)
{
- struct rte_table_hash *t = (struct rte_table_hash *)table;
+ struct rte_table_hash *t = table;
uint64_t pkts_mask_out = 0;
uint32_t i;
@@ -301,20 +275,19 @@ rte_table_hash_cuckoo_lookup_dosig(void *table,
RTE_TABLE_HASH_CUCKOO_STATS_PKTS_IN_ADD(t, n_pkts_in);
if ((pkts_mask & (pkts_mask + 1)) == 0) {
- const uint8_t *keys[64];
- int32_t positions[64], status;
+ const uint8_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
+ int32_t positions[RTE_PORT_IN_BURST_SIZE_MAX], status;
/* Keys for bulk lookup */
for (i = 0; i < n_pkts_in; i++)
keys[i] = RTE_MBUF_METADATA_UINT8_PTR(pkts[i],
- t->key_offset);
+ t->key_offset);
/* Bulk Lookup */
status = rte_hash_lookup_bulk(t->h_table,
(const void **) keys,
n_pkts_in,
positions);
-
if (status == 0) {
for (i = 0; i < n_pkts_in; i++) {
if (likely(positions[i] >= 0)) {
@@ -326,7 +299,7 @@ rte_table_hash_cuckoo_lookup_dosig(void *table,
}
}
}
- } else {
+ } else
for (i = 0; i < (uint32_t)(RTE_PORT_IN_BURST_SIZE_MAX
- __builtin_clzll(pkts_mask)); i++) {
uint64_t pkt_mask = 1LLU << i;
@@ -345,7 +318,6 @@ rte_table_hash_cuckoo_lookup_dosig(void *table,
}
}
}
- }
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_CUCKOO_STATS_PKTS_LOOKUP_MISS(t,
@@ -370,13 +342,13 @@ rte_table_hash_cuckoo_stats_read(void *table, struct rte_table_stats *stats,
return 0;
}
-struct rte_table_ops rte_table_hash_cuckoo_dosig_ops = {
+struct rte_table_ops rte_table_hash_cuckoo_ops = {
.f_create = rte_table_hash_cuckoo_create,
.f_free = rte_table_hash_cuckoo_free,
.f_add = rte_table_hash_cuckoo_entry_add,
.f_delete = rte_table_hash_cuckoo_entry_delete,
.f_add_bulk = NULL,
.f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_cuckoo_lookup_dosig,
+ .f_lookup = rte_table_hash_cuckoo_lookup,
.f_stats = rte_table_hash_cuckoo_stats_read,
};
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
index e718102..45e3ce6 100644
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
@@ -106,7 +106,6 @@ struct rte_table_hash {
uint32_t n_buckets_ext;
rte_table_hash_op_hash f_hash;
uint64_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
/* Internal */
@@ -120,6 +119,7 @@ struct rte_table_hash {
struct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];
/* Tables */
+ uint64_t *key_mask;
struct bucket *buckets;
struct bucket *buckets_ext;
uint8_t *key_mem;
@@ -132,12 +132,39 @@ struct rte_table_hash {
};
static int
-check_params_create(struct rte_table_hash_ext_params *params)
+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)
{
- uint32_t n_buckets_min;
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ if (a64[i] != (b64[i] & b_mask64[i]))
+ return 1;
+
+ return 0;
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ dst64[i] = src64[i] & src_mask64[i];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
/* key_size */
- if ((params->key_size == 0) ||
+ if ((params->key_size < sizeof(uint64_t)) ||
(!rte_is_power_of_2(params->key_size))) {
RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
return -EINVAL;
@@ -151,10 +178,8 @@ check_params_create(struct rte_table_hash_ext_params *params)
}
/* n_buckets */
- n_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;
if ((params->n_buckets == 0) ||
- (!rte_is_power_of_2(params->n_keys)) ||
- (params->n_buckets < n_buckets_min)) {
+ (!rte_is_power_of_2(params->n_buckets))) {
RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -171,15 +196,14 @@ check_params_create(struct rte_table_hash_ext_params *params)
static void *
rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_ext_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *t;
uint32_t total_size, table_meta_sz;
- uint32_t bucket_sz, bucket_ext_sz, key_sz;
+ uint32_t key_mask_sz, bucket_sz, bucket_ext_sz, key_sz;
uint32_t key_stack_sz, bkt_ext_stack_sz, data_sz;
- uint32_t bucket_offset, bucket_ext_offset, key_offset;
+ uint32_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset;
uint32_t key_stack_offset, bkt_ext_stack_offset, data_offset;
- uint32_t i;
+ uint32_t n_keys, n_buckets_ext, i;
/* Check input parameters */
if ((check_params_create(p) != 0) ||
@@ -188,38 +212,58 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
(sizeof(struct bucket) != (RTE_CACHE_LINE_SIZE / 2)))
return NULL;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_keys = p->n_keys;
+ if (n_keys < KEYS_PER_BUCKET)
+ n_keys = KEYS_PER_BUCKET;
+ n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
+
/* Memory allocation */
table_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));
+ key_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);
bucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));
bucket_ext_sz =
- RTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(struct bucket));
- key_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);
- key_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));
+ RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(struct bucket));
+ key_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * p->key_size);
+ key_stack_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * sizeof(uint32_t));
bkt_ext_stack_sz =
- RTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(uint32_t));
- data_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
- total_size = table_meta_sz + bucket_sz + bucket_ext_sz + key_sz +
- key_stack_sz + bkt_ext_stack_sz + data_sz;
+ RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
+ data_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * entry_size);
+ total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz +
+ key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz;
- t = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ t = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (t == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %u bytes for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table memory footprint is "
- "%u bytes\n", __func__, p->key_size, total_size);
+ RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table %s memory footprint is "
+ "%u bytes\n", __func__, p->key_size, p->name, total_size);
/* Memory initialization */
t->key_size = p->key_size;
t->entry_size = entry_size;
- t->n_keys = p->n_keys;
+ t->n_keys = n_keys;
t->n_buckets = p->n_buckets;
- t->n_buckets_ext = p->n_buckets_ext;
+ t->n_buckets_ext = n_buckets_ext;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
/* Internal */
@@ -228,13 +272,15 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
t->data_size_shl = __builtin_ctzl(entry_size);
/* Tables */
- bucket_offset = 0;
+ key_mask_offset = 0;
+ bucket_offset = key_mask_offset + key_mask_sz;
bucket_ext_offset = bucket_offset + bucket_sz;
key_offset = bucket_ext_offset + bucket_ext_sz;
key_stack_offset = key_offset + key_sz;
bkt_ext_stack_offset = key_stack_offset + key_stack_sz;
data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz;
+ t->key_mask = (uint64_t *) &t->memory[key_mask_offset];
t->buckets = (struct bucket *) &t->memory[bucket_offset];
t->buckets_ext = (struct bucket *) &t->memory[bucket_ext_offset];
t->key_mem = &t->memory[key_offset];
@@ -242,6 +288,12 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
t->bkt_ext_stack = (uint32_t *) &t->memory[bkt_ext_stack_offset];
t->data_mem = &t->memory[data_offset];
+ /* Key mask */
+ if (p->key_mask == NULL)
+ memset(t->key_mask, 0xFF, p->key_size);
+ else
+ memcpy(t->key_mask, p->key_mask, p->key_size);
+
/* Key stack */
for (i = 0; i < t->n_keys; i++)
t->key_stack[i] = t->n_keys - 1 - i;
@@ -277,7 +329,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -290,7 +342,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
uint8_t *bkt_key =
&t->key_mem[bkt_key_index << t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -327,7 +379,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
bkt->sig[i] = (uint16_t) sig;
bkt->key_pos[i] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
*key_found = 0;
@@ -358,7 +410,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
/* Install new key into bucket */
bkt->sig[0] = (uint16_t) sig;
bkt->key_pos[0] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
*key_found = 0;
@@ -378,7 +430,7 @@ void *entry)
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -392,7 +444,7 @@ void *entry)
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -437,8 +489,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
- void **entries,
- int dosig)
+ void **entries)
{
struct rte_table_hash *t = (struct rte_table_hash *) table;
uint64_t pkts_mask_out = 0;
@@ -458,11 +509,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- if (dosig)
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
- else
- sig = RTE_MBUF_METADATA_UINT32(pkt,
- t->signature_offset);
+ sig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
@@ -476,8 +523,8 @@ static int rte_table_hash_ext_lookup_unoptimized(
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
- t->key_size) == 0)) {
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key,
+ t->key_mask, t->key_size) == 0)) {
uint8_t *data = &t->data_mem[
bkt_key_index << t->data_size_shl];
@@ -576,11 +623,12 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\
uint64_t *bkt_key = (uint64_t *) key; \
+ uint64_t *key_mask = f->key_mask; \
\
switch (f->key_size) { \
case 8: \
{ \
- uint64_t xor = pkt_key[0] ^ bkt_key[0]; \
+ uint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
match_key = 0; \
if (xor == 0) \
match_key = 1; \
@@ -591,8 +639,8 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[2], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
or = xor[0] | xor[1]; \
match_key = 0; \
if (or == 0) \
@@ -604,10 +652,10 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[4], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
or = xor[0] | xor[1] | xor[2] | xor[3]; \
match_key = 0; \
if (or == 0) \
@@ -619,14 +667,14 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[8], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- xor[4] = pkt_key[4] ^ bkt_key[4]; \
- xor[5] = pkt_key[5] ^ bkt_key[5]; \
- xor[6] = pkt_key[6] ^ bkt_key[6]; \
- xor[7] = pkt_key[7] ^ bkt_key[7]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ xor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4]; \
+ xor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5]; \
+ xor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6]; \
+ xor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7]; \
or = xor[0] | xor[1] | xor[2] | xor[3] | \
xor[4] | xor[5] | xor[6] | xor[7]; \
match_key = 0; \
@@ -637,7 +685,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
\
default: \
match_key = 0; \
- if (memcmp(pkt_key, bkt_key, f->key_size) == 0) \
+ if (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0) \
match_key = 1; \
} \
}
@@ -685,38 +733,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
-{ \
- struct grinder *g10, *g11; \
- uint64_t sig10, sig11, bkt10_index, bkt11_index; \
- struct rte_mbuf *mbuf10, *mbuf11; \
- struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
- uint64_t bucket_mask = t->bucket_mask; \
- uint32_t signature_offset = t->signature_offset; \
- \
- mbuf10 = pkts[pkt10_index]; \
- sig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\
- bkt10_index = sig10 & bucket_mask; \
- bkt10 = &buckets[bkt10_index]; \
- \
- mbuf11 = pkts[pkt11_index]; \
- sig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\
- bkt11_index = sig11 & bucket_mask; \
- bkt11 = &buckets[bkt11_index]; \
- \
- rte_prefetch0(bkt10); \
- rte_prefetch0(bkt11); \
- \
- g10 = &g[pkt10_index]; \
- g10->sig = sig10; \
- g10->bkt = bkt10; \
- \
- g11 = &g[pkt11_index]; \
- g11->sig = sig11; \
- g11->bkt = bkt11; \
-}
-
-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index) \
+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
{ \
struct grinder *g10, *g11; \
uint64_t sig10, sig11, bkt10_index, bkt11_index; \
@@ -731,13 +748,13 @@ static int rte_table_hash_ext_lookup_unoptimized(
\
mbuf10 = pkts[pkt10_index]; \
key10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset); \
- sig10 = (uint64_t) f_hash(key10, key_size, seed); \
+ sig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed); \
bkt10_index = sig10 & bucket_mask; \
bkt10 = &buckets[bkt10_index]; \
\
mbuf11 = pkts[pkt11_index]; \
key11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset); \
- sig11 = (uint64_t) f_hash(key11, key_size, seed); \
+ sig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed); \
bkt11_index = sig11 & bucket_mask; \
bkt11 = &buckets[bkt11_index]; \
\
@@ -874,7 +891,7 @@ static int rte_table_hash_ext_lookup(
/* Cannot run the pipeline with less than 7 packets */
if (__builtin_popcountll(pkts_mask) < 7) {
status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 0);
+ pkts_mask, lookup_hit_mask, entries);
RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -
__builtin_popcountll(*lookup_hit_mask));
return status;
@@ -982,144 +999,7 @@ static int rte_table_hash_ext_lookup(
uint64_t pkts_mask_out_slow = 0;
status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 0);
- pkts_mask_out |= pkts_mask_out_slow;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return status;
-}
-
-static int rte_table_hash_ext_lookup_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *t = (struct rte_table_hash *) table;
- struct grinder *g = t->grinders;
- uint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;
- uint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;
- uint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;
- int status = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_EXT_STATS_PKTS_IN_ADD(t, n_pkts_in);
-
- /* Cannot run the pipeline with less than 7 packets */
- if (__builtin_popcountll(pkts_mask) < 7) {
- status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 1);
- RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -
- __builtin_popcountll(*lookup_hit_mask));
- return status;
- }
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline feed */
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline feed */
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,
- pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index,
- pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,
- pkts_mask_out, entries);
- }
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Slow path */
- pkts_mask_match_many &= ~pkts_mask_out;
- if (pkts_mask_match_many) {
- uint64_t pkts_mask_out_slow = 0;
-
- status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 1);
+ pkts_mask_match_many, &pkts_mask_out_slow, entries);
pkts_mask_out |= pkts_mask_out_slow;
}
@@ -1152,14 +1032,3 @@ struct rte_table_ops rte_table_hash_ext_ops = {
.f_lookup = rte_table_hash_ext_lookup,
.f_stats = rte_table_hash_ext_stats_read,
};
-
-struct rte_table_ops rte_table_hash_ext_dosig_ops = {
- .f_create = rte_table_hash_ext_create,
- .f_free = rte_table_hash_ext_free,
- .f_add = rte_table_hash_ext_entry_add,
- .f_delete = rte_table_hash_ext_entry_delete,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_ext_lookup_dosig,
- .f_stats = rte_table_hash_ext_stats_read,
-};
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
index ce057b7..f6cf6c6 100644
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 16
+#define KEY_SIZE 16
+
+#define KEYS_PER_BUCKET 4
#define RTE_BUCKET_ENTRY_VALID 0x1LLU
@@ -79,11 +81,9 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask[2];
rte_table_hash_op_hash f_hash;
@@ -99,17 +99,47 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key16_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return (a64[0] != (b64[0] & b_mask64[0])) ||
+ (a64[1] != (b64[1] & b_mask64[1]));
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+ dst64[1] = src64[1] & src_mask64[1];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params) {
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
return -EINVAL;
}
/* f_hash */
if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: f_hash function pointer is NULL\n", __func__);
+ RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
+ __func__);
return -EINVAL;
}
@@ -121,46 +151,45 @@ rte_table_hash_create_key16_lru(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key16_lru_params *p =
- (struct rte_table_hash_key16_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket,
- key_size, bucket_size_cl, total_size, i;
+ uint32_t n_buckets, bucket_size_cl;
+ uint32_t total_size, n_keys, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_16) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 16;
+
+ n_keys = p->n_keys;
+ if (n_keys < KEYS_PER_BUCKET)
+ n_keys = KEYS_PER_BUCKET;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
+ n_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /
+ KEYS_PER_BUCKET);
+ bucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +
+ KEYS_PER_BUCKET * entry_size);
total_size = sizeof(struct rte_table_hash) + n_buckets *
bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (f == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %u bytes for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ "%s: Hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -212,19 +241,19 @@ rte_table_hash_entry_add_key16_lru(
uint64_t signature, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -238,13 +267,13 @@ rte_table_hash_entry_add_key16_lru(
/* Key is not present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -257,7 +286,7 @@ rte_table_hash_entry_add_key16_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
bucket->signature[pos] = signature;
- memcpy(bucket->key[pos], key, f->key_size);
+ keycpy(&bucket->key[pos], key, f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
*key_found = 0;
@@ -278,19 +307,19 @@ rte_table_hash_entry_delete_key16_lru(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = 0;
@@ -306,81 +335,68 @@ rte_table_hash_entry_delete_key16_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key16_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: f_hash function pointer is NULL\n", __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key16_ext(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key16_ext_params *p =
- (struct rte_table_hash_key16_ext_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size,
- bucket_size_cl, stack_size_cl, total_size, i;
+ uint32_t n_keys, n_buckets, n_buckets_ext;
+ uint32_t bucket_size_cl, stack_size_cl, total_size, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_16) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 16;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_keys = p->n_keys;
+ if (n_keys < KEYS_PER_BUCKET)
+ n_keys = KEYS_PER_BUCKET;
+ n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
+ n_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /
+ KEYS_PER_BUCKET);
+ bucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size_cl = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
total_size = sizeof(struct rte_table_hash) +
((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *
RTE_CACHE_LINE_SIZE;
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (f == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %u bytes for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ "%s: Hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -390,9 +406,6 @@ rte_table_hash_create_key16_ext(void *params,
f->stack = (uint32_t *)
&f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
- for (i = 0; i < n_buckets_ext; i++)
- f->stack[i] = i;
-
if (p->key_mask != NULL) {
f->key_mask[0] = (((uint64_t *)p->key_mask)[0]);
f->key_mask[1] = (((uint64_t *)p->key_mask)[1]);
@@ -401,6 +414,9 @@ rte_table_hash_create_key16_ext(void *params,
f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
}
+ for (i = 0; i < n_buckets_ext; i++)
+ f->stack[i] = i;
+
return f;
}
@@ -432,20 +448,20 @@ rte_table_hash_entry_add_key16_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (bucket = bucket0; bucket != NULL; bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -458,17 +474,17 @@ rte_table_hash_entry_add_key16_ext(
/* Key is not present in the bucket */
for (bucket_prev = NULL, bucket = bucket0; bucket != NULL;
- bucket_prev = bucket, bucket = bucket->next)
+ bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -487,7 +503,7 @@ rte_table_hash_entry_add_key16_ext(
bucket_prev->next_valid = 1;
bucket->signature[0] = signature;
- memcpy(bucket->key[0], key, f->key_size);
+ keycpy(&bucket->key[0], key, f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -509,7 +525,7 @@ rte_table_hash_entry_delete_key16_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_16 *)
&f->memory[bucket_index * f->bucket_size];
@@ -520,18 +536,17 @@ rte_table_hash_entry_delete_key16_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = 0;
*key_found = 1;
if (entry)
- memcpy(entry, bucket_data,
- f->entry_size);
+ memcpy(entry, bucket_data, f->entry_size);
if ((bucket->signature[0] == 0) &&
(bucket->signature[1] == 0) &&
@@ -558,26 +573,28 @@ rte_table_hash_entry_delete_key16_ext(
return 0;
}
-#define lookup_key16_cmp(key_in, bucket, pos) \
+#define lookup_key16_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4][2], or[4], signature[4]; \
+ uint64_t xor[4][2], or[4], signature[4], k[2]; \
\
+ k[0] = key_in[0] & f->key_mask[0]; \
+ k[1] = key_in[1] & f->key_mask[1]; \
signature[0] = (~bucket->signature[0]) & 1; \
signature[1] = (~bucket->signature[1]) & 1; \
signature[2] = (~bucket->signature[2]) & 1; \
signature[3] = (~bucket->signature[3]) & 1; \
\
- xor[0][0] = key_in[0] ^ bucket->key[0][0]; \
- xor[0][1] = key_in[1] ^ bucket->key[0][1]; \
+ xor[0][0] = k[0] ^ bucket->key[0][0]; \
+ xor[0][1] = k[1] ^ bucket->key[0][1]; \
\
- xor[1][0] = key_in[0] ^ bucket->key[1][0]; \
- xor[1][1] = key_in[1] ^ bucket->key[1][1]; \
+ xor[1][0] = k[0] ^ bucket->key[1][0]; \
+ xor[1][1] = k[1] ^ bucket->key[1][1]; \
\
- xor[2][0] = key_in[0] ^ bucket->key[2][0]; \
- xor[2][1] = key_in[1] ^ bucket->key[2][1]; \
+ xor[2][0] = k[0] ^ bucket->key[2][0]; \
+ xor[2][1] = k[1] ^ bucket->key[2][1]; \
\
- xor[3][0] = key_in[0] ^ bucket->key[3][0]; \
- xor[3][1] = key_in[1] ^ bucket->key[3][1]; \
+ xor[3][0] = k[0] ^ bucket->key[3][0]; \
+ xor[3][1] = k[1] ^ bucket->key[3][1]; \
\
or[0] = xor[0][0] | xor[0][1] | signature[0]; \
or[1] = xor[1][0] | xor[1][1] | signature[1]; \
@@ -610,30 +627,12 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
- uint64_t signature; \
- uint32_t bucket_index; \
- \
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
- bucket_index = signature & (f->n_buckets - 1); \
- bucket1 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket_index * f->bucket_size]; \
- rte_prefetch0(bucket1); \
- rte_prefetch0((void *)(((uintptr_t) bucket1) + RTE_CACHE_LINE_SIZE));\
-}
-
-#define lookup1_stage1_dosig(mbuf1, bucket1, f) \
-{ \
uint64_t *key; \
uint64_t signature = 0; \
uint32_t bucket_index; \
- uint64_t hash_key_buffer[2]; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\
- \
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- signature = f->f_hash(hash_key_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed); \
+ signature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed); \
\
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_16 *) \
@@ -648,14 +647,10 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket2, pos); \
+ lookup_key16_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -673,14 +668,10 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket2, pos); \
+ lookup_key16_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -703,15 +694,11 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket, pos); \
+ lookup_key16_cmp(key, bucket, pos, f); \
\
pkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -775,36 +762,12 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
- \
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
- bucket10 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
- rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
- \
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
- bucket11 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
- rte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\
-}
-
-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f) \
-{ \
uint64_t *key10, *key11; \
- uint64_t hash_offset_buffer[2]; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
\
key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset);\
- hash_offset_buffer[0] = key10[0] & f->key_mask[0]; \
- hash_offset_buffer[1] = key10[1] & f->key_mask[1]; \
- signature10 = f->f_hash(hash_offset_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed);\
+ signature10 = f->f_hash(key10, f->key_mask, KEY_SIZE, f->seed);\
bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_16 *) \
&f->memory[bucket10_index * f->bucket_size]; \
@@ -812,10 +775,7 @@ rte_table_hash_entry_delete_key16_ext(
rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
\
key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset);\
- hash_offset_buffer[0] = key11[0] & f->key_mask[0]; \
- hash_offset_buffer[1] = key11[1] & f->key_mask[1]; \
- signature11 = f->f_hash(hash_offset_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed);\
+ signature11 = f->f_hash(key11, f->key_mask, KEY_SIZE, f->seed);\
bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_16 *) \
&f->memory[bucket11_index * f->bucket_size]; \
@@ -829,19 +789,13 @@ rte_table_hash_entry_delete_key16_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask; \
uint64_t *key20, *key21; \
- uint64_t hash_key_buffer20[2]; \
- uint64_t hash_key_buffer21[2]; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_key_buffer20[0] = key20[0] & f->key_mask[0]; \
- hash_key_buffer20[1] = key20[1] & f->key_mask[1]; \
- hash_key_buffer21[0] = key21[0] & f->key_mask[0]; \
- hash_key_buffer21[1] = key21[1] & f->key_mask[1]; \
\
- lookup_key16_cmp(hash_key_buffer20, bucket20, pos20); \
- lookup_key16_cmp(hash_key_buffer21, bucket21, pos21); \
+ lookup_key16_cmp(key20, bucket20, pos20, f); \
+ lookup_key16_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -864,19 +818,13 @@ rte_table_hash_entry_delete_key16_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\
uint64_t *key20, *key21; \
- uint64_t hash_key_buffer20[2]; \
- uint64_t hash_key_buffer21[2]; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_key_buffer20[0] = key20[0] & f->key_mask[0]; \
- hash_key_buffer20[1] = key20[1] & f->key_mask[1]; \
- hash_key_buffer21[0] = key21[0] & f->key_mask[0]; \
- hash_key_buffer21[1] = key21[1] & f->key_mask[1]; \
\
- lookup_key16_cmp(hash_key_buffer20, bucket20, pos20); \
- lookup_key16_cmp(hash_key_buffer21, bucket21, pos21); \
+ lookup_key16_cmp(key20, bucket20, pos20, f); \
+ lookup_key16_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -916,6 +864,7 @@ rte_table_hash_lookup_key16_lru(
uint64_t pkts_mask_out = 0;
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
+
RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
/* Cannot run the pipeline with less than 5 packets */
@@ -932,8 +881,8 @@ rte_table_hash_lookup_key16_lru(
}
*lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f,
- n_pkts_in - __builtin_popcountll(pkts_mask_out));
+ RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
+ __builtin_popcountll(pkts_mask_out));
return 0;
}
@@ -1026,136 +975,7 @@ rte_table_hash_lookup_key16_lru(
RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
__builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key16_lru() */
-
-static int
-rte_table_hash_lookup_key16_lru_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
-
- RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_16 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key16_lru_dosig() */
+} /* lookup LRU */
static int
rte_table_hash_lookup_key16_ext(
@@ -1175,6 +995,7 @@ rte_table_hash_lookup_key16_ext(
uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
+
RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
/* Cannot run the pipeline with less than 5 packets */
@@ -1306,159 +1127,7 @@ rte_table_hash_lookup_key16_ext(
RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
__builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key16_ext() */
-
-static int
-rte_table_hash_lookup_key16_ext_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0, buckets_mask = 0;
- struct rte_bucket_4_16 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];
- uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
-
- RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_16 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask,
- buckets, keys, f);
- }
-
- goto grind_next_buckets;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
-grind_next_buckets:
- /* Grind next buckets */
- for ( ; buckets_mask; ) {
- uint64_t buckets_mask_next = 0;
-
- for ( ; buckets_mask; ) {
- uint64_t pkt_mask;
- uint32_t pkt_index;
-
- pkt_index = __builtin_ctzll(buckets_mask);
- pkt_mask = 1LLU << pkt_index;
- buckets_mask &= ~pkt_mask;
-
- lookup_grinder(pkt_index, buckets, keys, pkts_mask_out,
- entries, buckets_mask_next, f);
- }
-
- buckets_mask = buckets_mask_next;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key16_ext_dosig() */
+} /* lookup EXT */
static int
rte_table_hash_key16_stats_read(void *table, struct rte_table_stats *stats, int clear)
@@ -1485,15 +1154,6 @@ struct rte_table_ops rte_table_hash_key16_lru_ops = {
.f_stats = rte_table_hash_key16_stats_read,
};
-struct rte_table_ops rte_table_hash_key16_lru_dosig_ops = {
- .f_create = rte_table_hash_create_key16_lru,
- .f_free = rte_table_hash_free_key16_lru,
- .f_add = rte_table_hash_entry_add_key16_lru,
- .f_delete = rte_table_hash_entry_delete_key16_lru,
- .f_lookup = rte_table_hash_lookup_key16_lru_dosig,
- .f_stats = rte_table_hash_key16_stats_read,
-};
-
struct rte_table_ops rte_table_hash_key16_ext_ops = {
.f_create = rte_table_hash_create_key16_ext,
.f_free = rte_table_hash_free_key16_ext,
@@ -1504,12 +1164,3 @@ struct rte_table_ops rte_table_hash_key16_ext_ops = {
.f_lookup = rte_table_hash_lookup_key16_ext,
.f_stats = rte_table_hash_key16_stats_read,
};
-
-struct rte_table_ops rte_table_hash_key16_ext_dosig_ops = {
- .f_create = rte_table_hash_create_key16_ext,
- .f_free = rte_table_hash_free_key16_ext,
- .f_add = rte_table_hash_entry_add_key16_ext,
- .f_delete = rte_table_hash_entry_delete_key16_ext,
- .f_lookup = rte_table_hash_lookup_key16_ext_dosig,
- .f_stats = rte_table_hash_key16_stats_read,
-};
diff --git a/lib/librte_table/rte_table_hash_key32.c b/lib/librte_table/rte_table_hash_key32.c
index 31fe6fd..5038669 100644
--- a/lib/librte_table/rte_table_hash_key32.c
+++ b/lib/librte_table/rte_table_hash_key32.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 32
+#define KEY_SIZE 32
+
+#define KEYS_PER_BUCKET 4
#define RTE_BUCKET_ENTRY_VALID 0x1LLU
@@ -79,12 +81,11 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
+ uint64_t key_mask[4];
rte_table_hash_op_hash f_hash;
uint64_t seed;
@@ -98,10 +99,44 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key32_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return (a64[0] != (b64[0] & b_mask64[0])) ||
+ (a64[1] != (b64[1] & b_mask64[1])) ||
+ (a64[2] != (b64[2] & b_mask64[2])) ||
+ (a64[3] != (b64[3] & b_mask64[3]));
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+ dst64[1] = src64[1] & src_mask64[1];
+ dst64[2] = src64[2] & src_mask64[2];
+ dst64[3] = src64[3] & src_mask64[3];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params) {
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
return -EINVAL;
}
@@ -120,51 +155,61 @@ rte_table_hash_create_key32_lru(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key32_lru_params *p =
- (struct rte_table_hash_key32_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;
- uint32_t total_size, i;
+ uint32_t n_buckets, bucket_size_cl;
+ uint32_t total_size, n_keys, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
- ((sizeof(struct rte_bucket_4_32) % 64) != 0)) {
+ ((sizeof(struct rte_bucket_4_32) % 64) != 0))
return NULL;
- }
- n_entries_per_bucket = 4;
- key_size = 32;
+
+ n_keys = p->n_keys;
+ if (n_keys < KEYS_PER_BUCKET)
+ n_keys = KEYS_PER_BUCKET;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
+ n_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /
+ KEYS_PER_BUCKET);
+ bucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +
+ KEYS_PER_BUCKET * entry_size);
total_size = sizeof(struct rte_table_hash) + n_buckets *
bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (f == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %u bytes for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n", __func__,
- total_size);
+ "%s: Hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
+ if (p->key_mask != NULL) {
+ f->key_mask[0] = ((uint64_t *)p->key_mask)[0];
+ f->key_mask[1] = ((uint64_t *)p->key_mask)[1];
+ f->key_mask[2] = ((uint64_t *)p->key_mask)[2];
+ f->key_mask[3] = ((uint64_t *)p->key_mask)[3];
+ } else {
+ f->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;
+ }
+
for (i = 0; i < n_buckets; i++) {
struct rte_bucket_4_32 *bucket;
@@ -204,7 +249,7 @@ rte_table_hash_entry_add_key32_lru(
uint64_t signature, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -213,10 +258,10 @@ rte_table_hash_entry_add_key32_lru(
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -230,13 +275,13 @@ rte_table_hash_entry_add_key32_lru(
/* Key is not present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -249,10 +294,10 @@ rte_table_hash_entry_add_key32_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
bucket->signature[pos] = signature;
- memcpy(bucket->key[pos], key, f->key_size);
+ keycpy(&bucket->key[pos], key, f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
- *key_found = 0;
+ *key_found = 0;
*entry_ptr = (void *) &bucket->data[pos * f->entry_size];
return 0;
@@ -270,7 +315,7 @@ rte_table_hash_entry_delete_key32_lru(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -279,10 +324,10 @@ rte_table_hash_entry_delete_key32_lru(
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = 0;
@@ -299,81 +344,69 @@ rte_table_hash_entry_delete_key32_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key32_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
- __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key32_ext(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key32_ext_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket;
- uint32_t key_size, bucket_size_cl, stack_size_cl, total_size, i;
+ uint32_t n_keys, n_buckets, n_buckets_ext;
+ uint32_t bucket_size_cl, stack_size_cl, total_size, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_32) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 32;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_keys = p->n_keys;
+ if (n_keys < KEYS_PER_BUCKET)
+ n_keys = KEYS_PER_BUCKET;
+ n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
+ n_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /
+ KEYS_PER_BUCKET);
+ bucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size_cl = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
+
total_size = sizeof(struct rte_table_hash) +
((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *
RTE_CACHE_LINE_SIZE;
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (f == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %u bytes for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n", __func__,
- total_size);
+ "%s: Hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -383,6 +416,18 @@ rte_table_hash_create_key32_ext(void *params,
f->stack = (uint32_t *)
&f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
+ if (p->key_mask != NULL) {
+ f->key_mask[0] = (((uint64_t *)p->key_mask)[0]);
+ f->key_mask[1] = (((uint64_t *)p->key_mask)[1]);
+ f->key_mask[2] = (((uint64_t *)p->key_mask)[2]);
+ f->key_mask[3] = (((uint64_t *)p->key_mask)[3]);
+ } else {
+ f->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;
+ }
+
for (i = 0; i < n_buckets_ext; i++)
f->stack[i] = i;
@@ -417,7 +462,7 @@ rte_table_hash_entry_add_key32_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -427,10 +472,10 @@ rte_table_hash_entry_add_key32_ext(
for (bucket = bucket0; bucket != NULL; bucket = bucket->next) {
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -448,14 +493,14 @@ rte_table_hash_entry_add_key32_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -475,7 +520,7 @@ rte_table_hash_entry_add_key32_ext(
bucket_prev->next_valid = 1;
bucket->signature[0] = signature;
- memcpy(bucket->key[0], key, f->key_size);
+ keycpy(&bucket->key[0], key, f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -497,7 +542,7 @@ rte_table_hash_entry_delete_key32_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -508,24 +553,23 @@ rte_table_hash_entry_delete_key32_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = 0;
*key_found = 1;
if (entry)
- memcpy(entry, bucket_data,
- f->entry_size);
+ memcpy(entry, bucket_data, f->entry_size);
if ((bucket->signature[0] == 0) &&
- (bucket->signature[1] == 0) &&
- (bucket->signature[2] == 0) &&
- (bucket->signature[3] == 0) &&
- (bucket_prev != NULL)) {
+ (bucket->signature[1] == 0) &&
+ (bucket->signature[2] == 0) &&
+ (bucket->signature[3] == 0) &&
+ (bucket_prev != NULL)) {
bucket_prev->next = bucket->next;
bucket_prev->next_valid =
bucket->next_valid;
@@ -546,34 +590,39 @@ rte_table_hash_entry_delete_key32_ext(
return 0;
}
-#define lookup_key32_cmp(key_in, bucket, pos) \
+#define lookup_key32_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4][4], or[4], signature[4]; \
+ uint64_t xor[4][4], or[4], signature[4], k[4]; \
+ \
+ k[0] = key_in[0] & f->key_mask[0]; \
+ k[1] = key_in[1] & f->key_mask[1]; \
+ k[2] = key_in[2] & f->key_mask[2]; \
+ k[3] = key_in[3] & f->key_mask[3]; \
\
signature[0] = ((~bucket->signature[0]) & 1); \
signature[1] = ((~bucket->signature[1]) & 1); \
signature[2] = ((~bucket->signature[2]) & 1); \
signature[3] = ((~bucket->signature[3]) & 1); \
\
- xor[0][0] = key_in[0] ^ bucket->key[0][0]; \
- xor[0][1] = key_in[1] ^ bucket->key[0][1]; \
- xor[0][2] = key_in[2] ^ bucket->key[0][2]; \
- xor[0][3] = key_in[3] ^ bucket->key[0][3]; \
+ xor[0][0] = k[0] ^ bucket->key[0][0]; \
+ xor[0][1] = k[1] ^ bucket->key[0][1]; \
+ xor[0][2] = k[2] ^ bucket->key[0][2]; \
+ xor[0][3] = k[3] ^ bucket->key[0][3]; \
\
- xor[1][0] = key_in[0] ^ bucket->key[1][0]; \
- xor[1][1] = key_in[1] ^ bucket->key[1][1]; \
- xor[1][2] = key_in[2] ^ bucket->key[1][2]; \
- xor[1][3] = key_in[3] ^ bucket->key[1][3]; \
+ xor[1][0] = k[0] ^ bucket->key[1][0]; \
+ xor[1][1] = k[1] ^ bucket->key[1][1]; \
+ xor[1][2] = k[2] ^ bucket->key[1][2]; \
+ xor[1][3] = k[3] ^ bucket->key[1][3]; \
\
- xor[2][0] = key_in[0] ^ bucket->key[2][0]; \
- xor[2][1] = key_in[1] ^ bucket->key[2][1]; \
- xor[2][2] = key_in[2] ^ bucket->key[2][2]; \
- xor[2][3] = key_in[3] ^ bucket->key[2][3]; \
+ xor[2][0] = k[0] ^ bucket->key[2][0]; \
+ xor[2][1] = k[1] ^ bucket->key[2][1]; \
+ xor[2][2] = k[2] ^ bucket->key[2][2]; \
+ xor[2][3] = k[3] ^ bucket->key[2][3]; \
\
- xor[3][0] = key_in[0] ^ bucket->key[3][0]; \
- xor[3][1] = key_in[1] ^ bucket->key[3][1]; \
- xor[3][2] = key_in[2] ^ bucket->key[3][2]; \
- xor[3][3] = key_in[3] ^ bucket->key[3][3]; \
+ xor[3][0] = k[0] ^ bucket->key[3][0]; \
+ xor[3][1] = k[1] ^ bucket->key[3][1]; \
+ xor[3][2] = k[2] ^ bucket->key[3][2]; \
+ xor[3][3] = k[3] ^ bucket->key[3][3]; \
\
or[0] = xor[0][0] | xor[0][1] | xor[0][2] | xor[0][3] | signature[0];\
or[1] = xor[1][0] | xor[1][1] | xor[1][2] | xor[1][3] | signature[1];\
@@ -604,12 +653,15 @@ rte_table_hash_entry_delete_key32_ext(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf0, key_offset));\
}
-#define lookup1_stage1(mbuf1, bucket1, f) \
+#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
+ uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
\
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
+ key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset); \
+ signature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed); \
+ \
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_32 *) \
&f->memory[bucket_index * f->bucket_size]; \
@@ -627,8 +679,7 @@ rte_table_hash_entry_delete_key32_ext(
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- \
- lookup_key32_cmp(key, bucket2, pos); \
+ lookup_key32_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -649,8 +700,7 @@ rte_table_hash_entry_delete_key32_ext(
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- \
- lookup_key32_cmp(key, bucket2, pos); \
+ lookup_key32_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -678,7 +728,7 @@ rte_table_hash_entry_delete_key32_ext(
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
\
- lookup_key32_cmp(key, bucket, pos); \
+ lookup_key32_cmp(key, bucket, pos, f); \
\
pkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -745,22 +795,27 @@ rte_table_hash_entry_delete_key32_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
+ uint64_t *key10, *key11; \
+ uint64_t signature10, signature11; \
+ uint32_t bucket10_index, bucket11_index; \
+ \
+ key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset); \
+ signature10 = f->f_hash(key10, f->key_mask, KEY_SIZE, f->seed); \
\
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
+ bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_32 *) \
&f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
+ rte_prefetch0(bucket10); \
rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
rte_prefetch0((void *)(((uintptr_t) bucket10) + 2 * RTE_CACHE_LINE_SIZE));\
\
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
+ key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset); \
+ signature11 = f->f_hash(key11, f->key_mask, KEY_SIZE, f->seed);\
+ \
+ bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_32 *) \
&f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
+ rte_prefetch0(bucket11); \
rte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\
rte_prefetch0((void *)(((uintptr_t) bucket11) + 2 * RTE_CACHE_LINE_SIZE));\
}
@@ -776,8 +831,8 @@ rte_table_hash_entry_delete_key32_ext(
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
\
- lookup_key32_cmp(key20, bucket20, pos20); \
- lookup_key32_cmp(key21, bucket21, pos21); \
+ lookup_key32_cmp(key20, bucket20, pos20, f); \
+ lookup_key32_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -805,8 +860,8 @@ rte_table_hash_entry_delete_key32_ext(
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
\
- lookup_key32_cmp(key20, bucket20, pos20); \
- lookup_key32_cmp(key21, bucket21, pos21); \
+ lookup_key32_cmp(key20, bucket20, pos20, f); \
+ lookup_key32_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
index 5f0c656..39d0458 100644
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 8
+#define KEY_SIZE 8
+
+#define KEYS_PER_BUCKET 4
#ifdef RTE_TABLE_STATS_COLLECT
@@ -76,11 +78,9 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask;
rte_table_hash_op_hash f_hash;
@@ -96,10 +96,38 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return a64[0] != (b64[0] & b_mask64[0]);
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params) {
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
return -EINVAL;
}
@@ -116,47 +144,45 @@ check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {
static void *
rte_table_hash_create_key8_lru(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_key8_lru_params *p =
- (struct rte_table_hash_key8_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;
- uint32_t total_size, i;
+ uint32_t n_buckets, bucket_size_cl;
+ uint32_t total_size, n_keys, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
- ((sizeof(struct rte_bucket_4_8) % 64) != 0)) {
+ ((sizeof(struct rte_bucket_4_8) % 64) != 0))
return NULL;
- }
- n_entries_per_bucket = 4;
- key_size = 8;
+
+ n_keys = p->n_keys;
+ if (n_keys < KEYS_PER_BUCKET)
+ n_keys = KEYS_PER_BUCKET;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *
- entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
+ n_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /
+ KEYS_PER_BUCKET);
+ bucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +
+ KEYS_PER_BUCKET * entry_size);
total_size = sizeof(struct rte_table_hash) + n_buckets *
bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (f == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %u bytes for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ "%s: Hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -205,7 +231,7 @@ rte_table_hash_entry_add_key8_lru(
uint64_t signature, mask, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -213,10 +239,10 @@ rte_table_hash_entry_add_key8_lru(
/* Key is present in the bucket */
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -235,7 +261,7 @@ rte_table_hash_entry_add_key8_lru(
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature |= mask;
- bucket->key[i] = *((uint64_t *) key);
+ keycpy(&bucket->key[i], key, &f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -247,10 +273,10 @@ rte_table_hash_entry_add_key8_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
- bucket->key[pos] = *((uint64_t *) key);
+ keycpy(&bucket->key[pos], key, &f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
- *key_found = 0;
+ *key_found = 0;
*entry_ptr = (void *) &bucket->data[pos * f->entry_size];
return 0;
@@ -268,7 +294,7 @@ rte_table_hash_entry_delete_key8_lru(
uint64_t signature, mask;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -276,10 +302,10 @@ rte_table_hash_entry_delete_key8_lru(
/* Key is present in the bucket */
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature &= ~mask;
@@ -296,79 +322,66 @@ rte_table_hash_entry_delete_key8_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key8_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
- __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key8_ext(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_key8_ext_params *p =
- (struct rte_table_hash_key8_ext_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size;
+ uint32_t n_keys, n_buckets, n_buckets_ext;
uint32_t bucket_size_cl, stack_size_cl, total_size, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_8) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 8;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_keys = p->n_keys;
+ if (n_keys < KEYS_PER_BUCKET)
+ n_keys = KEYS_PER_BUCKET;
+ n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *
- entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + ((n_buckets +
- n_buckets_ext) * bucket_size_cl + stack_size_cl) *
+ n_buckets = rte_align32pow2((n_keys + KEYS_PER_BUCKET - 1) /
+ KEYS_PER_BUCKET);
+ bucket_size_cl = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size_cl = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
+ total_size = sizeof(struct rte_table_hash) +
+ ((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *
RTE_CACHE_LINE_SIZE;
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (f == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %u bytes for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ "%s: Hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -417,7 +430,7 @@ rte_table_hash_entry_add_key8_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -428,10 +441,10 @@ rte_table_hash_entry_add_key8_ext(
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -456,7 +469,7 @@ rte_table_hash_entry_add_key8_ext(
f->entry_size];
bucket->signature |= mask;
- bucket->key[i] = *((uint64_t *) key);
+ keycpy(&bucket->key[i], key, &f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -476,7 +489,7 @@ rte_table_hash_entry_add_key8_ext(
bucket_prev->next_valid = 1;
bucket->signature = 1;
- bucket->key[0] = *((uint64_t *) key);
+ keycpy(&bucket->key[0], key, &f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -498,7 +511,7 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -510,10 +523,10 @@ rte_table_hash_entry_delete_key8_ext(
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -546,16 +559,17 @@ rte_table_hash_entry_delete_key8_ext(
return 0;
}
-#define lookup_key8_cmp(key_in, bucket, pos) \
+#define lookup_key8_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4], signature; \
+ uint64_t xor[4], signature, k; \
\
signature = ~bucket->signature; \
\
- xor[0] = (key_in[0] ^ bucket->key[0]) | (signature & 1);\
- xor[1] = (key_in[0] ^ bucket->key[1]) | (signature & 2);\
- xor[2] = (key_in[0] ^ bucket->key[2]) | (signature & 4);\
- xor[3] = (key_in[0] ^ bucket->key[3]) | (signature & 8);\
+ k = key_in[0] & f->key_mask; \
+ xor[0] = (k ^ bucket->key[0]) | (signature & 1); \
+ xor[1] = (k ^ bucket->key[1]) | (signature & 2); \
+ xor[2] = (k ^ bucket->key[2]) | (signature & 4); \
+ xor[3] = (k ^ bucket->key[3]) | (signature & 8); \
\
pos = 4; \
if (xor[0] == 0) \
@@ -583,27 +597,12 @@ rte_table_hash_entry_delete_key8_ext(
#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
- uint64_t signature; \
- uint32_t bucket_index; \
- \
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
- bucket_index = signature & (f->n_buckets - 1); \
- bucket1 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket_index * f->bucket_size]; \
- rte_prefetch0(bucket1); \
-}
-
-#define lookup1_stage1_dosig(mbuf1, bucket1, f) \
-{ \
uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\
- hash_key_buffer = *key & f->key_mask; \
- signature = f->f_hash(&hash_key_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed); \
+ signature = f->f_hash(key, &f->key_mask, KEY_SIZE, f->seed); \
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_8 *) \
&f->memory[bucket_index * f->bucket_size]; \
@@ -617,12 +616,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer = key[0] & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket2, pos); \
+ lookup_key8_cmp(key, bucket2, pos, f); \
\
pkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -641,12 +637,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer = *key & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket2, pos); \
+ lookup_key8_cmp(key, bucket2, pos, f); \
\
pkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -670,13 +663,10 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
- hash_key_buffer = (*key) & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket, pos); \
+ lookup_key8_cmp(key, bucket, pos, f); \
\
pkt_mask = ((bucket->signature >> pos) & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -738,29 +728,9 @@ rte_table_hash_entry_delete_key8_ext(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
-{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
- \
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
- bucket10 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
- \
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
- bucket11 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
-}
-
-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f)\
+#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\
{ \
uint64_t *key10, *key11; \
- uint64_t hash_offset_buffer10; \
- uint64_t hash_offset_buffer11; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
rte_table_hash_op_hash f_hash = f->f_hash; \
@@ -769,18 +739,14 @@ rte_table_hash_entry_delete_key8_ext(
\
key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, key_offset);\
key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, key_offset);\
- hash_offset_buffer10 = *key10 & f->key_mask; \
- hash_offset_buffer11 = *key11 & f->key_mask; \
\
- signature10 = f_hash(&hash_offset_buffer10, \
- RTE_TABLE_HASH_KEY_SIZE, seed); \
+ signature10 = f_hash(key10, &f->key_mask, KEY_SIZE, seed); \
bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_8 *) \
&f->memory[bucket10_index * f->bucket_size]; \
rte_prefetch0(bucket10); \
\
- signature11 = f_hash(&hash_offset_buffer11, \
- RTE_TABLE_HASH_KEY_SIZE, seed); \
+ signature11 = f_hash(key11, &f->key_mask,KEY_SIZE, seed); \
bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_8 *) \
&f->memory[bucket11_index * f->bucket_size]; \
@@ -793,17 +759,13 @@ rte_table_hash_entry_delete_key8_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask; \
uint64_t *key20, *key21; \
- uint64_t hash_offset_buffer20; \
- uint64_t hash_offset_buffer21; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_offset_buffer20 = *key20 & f->key_mask; \
- hash_offset_buffer21 = *key21 & f->key_mask; \
\
- lookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\
- lookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\
+ lookup_key8_cmp(key20, bucket20, pos20, f); \
+ lookup_key8_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\
pkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\
@@ -826,17 +788,13 @@ rte_table_hash_entry_delete_key8_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\
uint64_t *key20, *key21; \
- uint64_t hash_offset_buffer20; \
- uint64_t hash_offset_buffer21; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_offset_buffer20 = *key20 & f->key_mask; \
- hash_offset_buffer21 = *key21 & f->key_mask; \
\
- lookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\
- lookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\
+ lookup_key8_cmp(key20, bucket20, pos20, f); \
+ lookup_key8_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\
pkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\
@@ -871,8 +829,8 @@ rte_table_hash_lookup_key8_lru(
struct rte_table_hash *f = (struct rte_table_hash *) table;
struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index,
- pkt11_index, pkt20_index, pkt21_index;
+ uint32_t pkt00_index, pkt01_index, pkt10_index;
+ uint32_t pkt11_index, pkt20_index, pkt21_index;
uint64_t pkts_mask_out = 0;
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
@@ -888,7 +846,7 @@ rte_table_hash_lookup_key8_lru(
lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
lookup1_stage1(mbuf, bucket, f);
lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
+ pkts_mask_out, entries, f);
}
*lookup_hit_mask = pkts_mask_out;
@@ -984,133 +942,7 @@ rte_table_hash_lookup_key8_lru(
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key8_lru() */
-
-static int
-rte_table_hash_lookup_key8_lru_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_8 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key8_lru_dosig() */
+} /* lookup LRU */
static int
rte_table_hash_lookup_key8_ext(
@@ -1142,8 +974,8 @@ rte_table_hash_lookup_key8_ext(
lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
lookup1_stage1(mbuf, bucket, f);
lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask, buckets,
- keys, f);
+ pkts_mask_out, entries, buckets_mask,
+ buckets, keys, f);
}
goto grind_next_buckets;
@@ -1260,157 +1092,7 @@ rte_table_hash_lookup_key8_ext(
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key8_ext() */
-
-static int
-rte_table_hash_lookup_key8_ext_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0, buckets_mask = 0;
- struct rte_bucket_4_8 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];
- uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_8 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask,
- buckets, keys, f);
- }
-
- goto grind_next_buckets;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
-grind_next_buckets:
- /* Grind next buckets */
- for ( ; buckets_mask; ) {
- uint64_t buckets_mask_next = 0;
-
- for ( ; buckets_mask; ) {
- uint64_t pkt_mask;
- uint32_t pkt_index;
-
- pkt_index = __builtin_ctzll(buckets_mask);
- pkt_mask = 1LLU << pkt_index;
- buckets_mask &= ~pkt_mask;
-
- lookup_grinder(pkt_index, buckets, keys, pkts_mask_out,
- entries, buckets_mask_next, f);
- }
-
- buckets_mask = buckets_mask_next;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key8_dosig_ext() */
+} /* lookup EXT */
static int
rte_table_hash_key8_stats_read(void *table, struct rte_table_stats *stats, int clear)
@@ -1437,17 +1119,6 @@ struct rte_table_ops rte_table_hash_key8_lru_ops = {
.f_stats = rte_table_hash_key8_stats_read,
};
-struct rte_table_ops rte_table_hash_key8_lru_dosig_ops = {
- .f_create = rte_table_hash_create_key8_lru,
- .f_free = rte_table_hash_free_key8_lru,
- .f_add = rte_table_hash_entry_add_key8_lru,
- .f_delete = rte_table_hash_entry_delete_key8_lru,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lookup_key8_lru_dosig,
- .f_stats = rte_table_hash_key8_stats_read,
-};
-
struct rte_table_ops rte_table_hash_key8_ext_ops = {
.f_create = rte_table_hash_create_key8_ext,
.f_free = rte_table_hash_free_key8_ext,
@@ -1458,14 +1129,3 @@ struct rte_table_ops rte_table_hash_key8_ext_ops = {
.f_lookup = rte_table_hash_lookup_key8_ext,
.f_stats = rte_table_hash_key8_stats_read,
};
-
-struct rte_table_ops rte_table_hash_key8_ext_dosig_ops = {
- .f_create = rte_table_hash_create_key8_ext,
- .f_free = rte_table_hash_free_key8_ext,
- .f_add = rte_table_hash_entry_add_key8_ext,
- .f_delete = rte_table_hash_entry_delete_key8_ext,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lookup_key8_ext_dosig,
- .f_stats = rte_table_hash_key8_stats_read,
-};
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
index 5a4864e..9fbc2c2 100644
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
@@ -86,7 +86,6 @@ struct rte_table_hash {
uint32_t n_buckets;
rte_table_hash_op_hash f_hash;
uint64_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
/* Internal */
@@ -99,6 +98,7 @@ struct rte_table_hash {
struct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];
/* Tables */
+ uint64_t *key_mask;
struct bucket *buckets;
uint8_t *key_mem;
uint8_t *data_mem;
@@ -109,12 +109,39 @@ struct rte_table_hash {
};
static int
-check_params_create(struct rte_table_hash_lru_params *params)
+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)
{
- uint32_t n_buckets_min;
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ if (a64[i] != (b64[i] & b_mask64[i]))
+ return 1;
+
+ return 0;
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ dst64[i] = src64[i] & src_mask64[i];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
/* key_size */
- if ((params->key_size == 0) ||
+ if ((params->key_size < sizeof(uint64_t)) ||
(!rte_is_power_of_2(params->key_size))) {
RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
return -EINVAL;
@@ -128,10 +155,8 @@ check_params_create(struct rte_table_hash_lru_params *params)
}
/* n_buckets */
- n_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;
if ((params->n_buckets == 0) ||
- (!rte_is_power_of_2(params->n_keys)) ||
- (params->n_buckets < n_buckets_min)) {
+ (!rte_is_power_of_2(params->n_keys))) {
RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -148,13 +173,13 @@ check_params_create(struct rte_table_hash_lru_params *params)
static void *
rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_lru_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *t;
uint32_t total_size, table_meta_sz;
- uint32_t bucket_sz, key_sz, key_stack_sz, data_sz;
- uint32_t bucket_offset, key_offset, key_stack_offset, data_offset;
- uint32_t i;
+ uint32_t key_mask_sz, bucket_sz, key_sz, key_stack_sz, data_sz;
+ uint32_t key_mask_offset, bucket_offset, key_offset;
+ uint32_t key_stack_offset, data_offset;
+ uint32_t n_keys, n_buckets, i;
/* Check input parameters */
if ((check_params_create(p) != 0) ||
@@ -164,33 +189,53 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
return NULL;
}
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = n_keys / KEYS_PER_BUCKET
+ */
+ n_keys = p->n_keys;
+ if (n_keys < KEYS_PER_BUCKET)
+ n_keys = KEYS_PER_BUCKET;
+ n_buckets = RTE_MAX(p->n_buckets, n_keys / KEYS_PER_BUCKET);
+
/* Memory allocation */
table_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));
- bucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));
- key_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);
- key_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));
- data_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
- total_size = table_meta_sz + bucket_sz + key_sz + key_stack_sz +
- data_sz;
-
- t = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ key_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);
+ bucket_sz = RTE_CACHE_LINE_ROUNDUP(n_buckets * sizeof(struct bucket));
+ key_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * p->key_size);
+ key_stack_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * sizeof(uint32_t));
+ data_sz = RTE_CACHE_LINE_ROUNDUP(n_keys * entry_size);
+ total_size = table_meta_sz + key_mask_sz + bucket_sz + key_sz +
+ key_stack_sz + data_sz;
+
+ t = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (t == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %u bytes for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table memory footprint is "
- "%u bytes\n", __func__, p->key_size, total_size);
+ RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table %s memory footprint is "
+ "%u bytes\n", __func__, p->key_size, p->name, total_size);
/* Memory initialization */
t->key_size = p->key_size;
t->entry_size = entry_size;
- t->n_keys = p->n_keys;
- t->n_buckets = p->n_buckets;
+ t->n_keys = n_keys;
+ t->n_buckets = n_buckets;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
/* Internal */
@@ -199,16 +244,24 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
t->data_size_shl = __builtin_ctzl(entry_size);
/* Tables */
- bucket_offset = 0;
+ key_mask_offset = 0;
+ bucket_offset = key_mask_offset + key_mask_sz;
key_offset = bucket_offset + bucket_sz;
key_stack_offset = key_offset + key_sz;
data_offset = key_stack_offset + key_stack_sz;
+ t->key_mask = (uint64_t *) &t->memory[key_mask_offset];
t->buckets = (struct bucket *) &t->memory[bucket_offset];
t->key_mem = &t->memory[key_offset];
t->key_stack = (uint32_t *) &t->memory[key_stack_offset];
t->data_mem = &t->memory[data_offset];
+ /* Key mask */
+ if (p->key_mask == NULL)
+ memset(t->key_mask, 0xFF, p->key_size);
+ else
+ memcpy(t->key_mask, p->key_mask, p->key_size);
+
/* Key stack */
for (i = 0; i < t->n_keys; i++)
t->key_stack[i] = t->n_keys - 1 - i;
@@ -246,7 +299,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -258,8 +311,8 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key, t->key_size)
- == 0)) {
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
+ t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -292,7 +345,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
bkt->sig[i] = (uint16_t) sig;
bkt->key_pos[i] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
lru_update(bkt, i);
@@ -311,7 +364,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint8_t *data = &t->data_mem[bkt_key_index << t->data_size_shl];
bkt->sig[pos] = (uint16_t) sig;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
lru_update(bkt, pos);
@@ -330,7 +383,7 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -343,7 +396,7 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,
t->key_size_shl];
if ((sig == bkt_sig) &&
- (memcmp(key, bkt_key, t->key_size) == 0)) {
+ (keycmp(bkt_key, key, t->key_mask, t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -365,8 +418,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
- void **entries,
- int dosig)
+ void **entries)
{
struct rte_table_hash *t = (struct rte_table_hash *) table;
uint64_t pkts_mask_out = 0;
@@ -387,11 +439,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- if (dosig)
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
- else
- sig = RTE_MBUF_METADATA_UINT32(pkt,
- t->signature_offset);
+ sig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
@@ -404,7 +452,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -502,74 +550,75 @@ static int rte_table_hash_lru_lookup_unoptimized(
match_pos = (LUT_MATCH_POS >> (mask_all << 1)) & 3; \
}
-#define lookup_cmp_key(mbuf, key, match_key, f) \
-{ \
+#define lookup_cmp_key(mbuf, key, match_key, f) \
+{ \
uint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\
- uint64_t *bkt_key = (uint64_t *) key; \
- \
- switch (f->key_size) { \
- case 8: \
- { \
- uint64_t xor = pkt_key[0] ^ bkt_key[0]; \
- match_key = 0; \
- if (xor == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 16: \
- { \
- uint64_t xor[2], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- or = xor[0] | xor[1]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 32: \
- { \
- uint64_t xor[4], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- or = xor[0] | xor[1] | xor[2] | xor[3]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 64: \
- { \
- uint64_t xor[8], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- xor[4] = pkt_key[4] ^ bkt_key[4]; \
- xor[5] = pkt_key[5] ^ bkt_key[5]; \
- xor[6] = pkt_key[6] ^ bkt_key[6]; \
- xor[7] = pkt_key[7] ^ bkt_key[7]; \
- or = xor[0] | xor[1] | xor[2] | xor[3] | \
- xor[4] | xor[5] | xor[6] | xor[7]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- default: \
- match_key = 0; \
- if (memcmp(pkt_key, bkt_key, f->key_size) == 0) \
- match_key = 1; \
- } \
+ uint64_t *bkt_key = (uint64_t *) key; \
+ uint64_t *key_mask = f->key_mask; \
+ \
+ switch (f->key_size) { \
+ case 8: \
+ { \
+ uint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ match_key = 0; \
+ if (xor == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 16: \
+ { \
+ uint64_t xor[2], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ or = xor[0] | xor[1]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 32: \
+ { \
+ uint64_t xor[4], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ or = xor[0] | xor[1] | xor[2] | xor[3]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 64: \
+ { \
+ uint64_t xor[8], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ xor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4]; \
+ xor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5]; \
+ xor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6]; \
+ xor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7]; \
+ or = xor[0] | xor[1] | xor[2] | xor[3] | \
+ xor[4] | xor[5] | xor[6] | xor[7]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ default: \
+ match_key = 0; \
+ if (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0) \
+ match_key = 1; \
+ } \
}
#define lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index)\
@@ -616,38 +665,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
-{ \
- struct grinder *g10, *g11; \
- uint64_t sig10, sig11, bkt10_index, bkt11_index; \
- struct rte_mbuf *mbuf10, *mbuf11; \
- struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
- uint64_t bucket_mask = t->bucket_mask; \
- uint32_t signature_offset = t->signature_offset; \
- \
- mbuf10 = pkts[pkt10_index]; \
- sig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\
- bkt10_index = sig10 & bucket_mask; \
- bkt10 = &buckets[bkt10_index]; \
- \
- mbuf11 = pkts[pkt11_index]; \
- sig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\
- bkt11_index = sig11 & bucket_mask; \
- bkt11 = &buckets[bkt11_index]; \
- \
- rte_prefetch0(bkt10); \
- rte_prefetch0(bkt11); \
- \
- g10 = &g[pkt10_index]; \
- g10->sig = sig10; \
- g10->bkt = bkt10; \
- \
- g11 = &g[pkt11_index]; \
- g11->sig = sig11; \
- g11->bkt = bkt11; \
-}
-
-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index)\
+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index)\
{ \
struct grinder *g10, *g11; \
uint64_t sig10, sig11, bkt10_index, bkt11_index; \
@@ -662,13 +680,13 @@ static int rte_table_hash_lru_lookup_unoptimized(
\
mbuf10 = pkts[pkt10_index]; \
key10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset);\
- sig10 = (uint64_t) f_hash(key10, key_size, seed); \
+ sig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed);\
bkt10_index = sig10 & bucket_mask; \
bkt10 = &buckets[bkt10_index]; \
\
mbuf11 = pkts[pkt11_index]; \
key11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset);\
- sig11 = (uint64_t) f_hash(key11, key_size, seed); \
+ sig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed);\
bkt11_index = sig11 & bucket_mask; \
bkt11 = &buckets[bkt11_index]; \
\
@@ -819,7 +837,7 @@ static int rte_table_hash_lru_lookup(
/* Cannot run the pipeline with less than 7 packets */
if (__builtin_popcountll(pkts_mask) < 7)
return rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 0);
+ pkts_mask, lookup_hit_mask, entries);
/* Pipeline stage 0 */
lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
@@ -923,140 +941,7 @@ static int rte_table_hash_lru_lookup(
uint64_t pkts_mask_out_slow = 0;
status = rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 0);
- pkts_mask_out |= pkts_mask_out_slow;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_LRU_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return status;
-}
-
-static int rte_table_hash_lru_lookup_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *t = (struct rte_table_hash *) table;
- struct grinder *g = t->grinders;
- uint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;
- uint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;
- uint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;
- int status = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_LRU_STATS_PKTS_IN_ADD(t, n_pkts_in);
-
- /* Cannot run the pipeline with less than 7 packets */
- if (__builtin_popcountll(pkts_mask) < 7)
- return rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 1);
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline feed */
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline feed */
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,
- pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index,
- pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,
- pkts_mask_out, entries);
- }
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Slow path */
- pkts_mask_match_many &= ~pkts_mask_out;
- if (pkts_mask_match_many) {
- uint64_t pkts_mask_out_slow = 0;
-
- status = rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 1);
+ pkts_mask_match_many, &pkts_mask_out_slow, entries);
pkts_mask_out |= pkts_mask_out_slow;
}
@@ -1089,14 +974,3 @@ struct rte_table_ops rte_table_hash_lru_ops = {
.f_lookup = rte_table_hash_lru_lookup,
.f_stats = rte_table_hash_lru_stats_read,
};
-
-struct rte_table_ops rte_table_hash_lru_dosig_ops = {
- .f_create = rte_table_hash_lru_create,
- .f_free = rte_table_hash_lru_free,
- .f_add = rte_table_hash_lru_entry_add,
- .f_delete = rte_table_hash_lru_entry_delete,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lru_lookup_dosig,
- .f_stats = rte_table_hash_lru_stats_read,
-};
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH 2/5] test: update due to api changes in librte_table
2017-08-24 13:53 [dpdk-dev] [PATCH 0/5] table: add key mask for hash tables Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 1/5] " Cristian Dumitrescu
@ 2017-08-24 13:53 ` Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 3/5] test-pipeline: " Cristian Dumitrescu
` (2 subsequent siblings)
4 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-08-24 13:53 UTC (permalink / raw)
To: dev
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
test/test/test_table.c | 1 +
test/test/test_table.h | 3 +-
test/test/test_table_combined.c | 140 +++++++++++++++++--------------------
test/test/test_table_tables.c | 148 +++++++++++++++++-----------------------
4 files changed, 128 insertions(+), 164 deletions(-)
diff --git a/test/test/test_table.c b/test/test/test_table.c
index 9e9eed8..db7d4e6 100644
--- a/test/test/test_table.c
+++ b/test/test/test_table.c
@@ -72,6 +72,7 @@ static void app_init_rings(void);
static void app_init_mbuf_pools(void);
uint64_t pipeline_test_hash(void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed)
{
diff --git a/test/test/test_table.h b/test/test/test_table.h
index 84d1845..8c1df33 100644
--- a/test/test/test_table.h
+++ b/test/test/test_table.h
@@ -94,7 +94,7 @@
APP_METADATA_OFFSET(32)); \
k32 = (uint32_t *) key; \
k32[0] = (value); \
- *signature = pipeline_test_hash(key, 0, 0); \
+ *signature = pipeline_test_hash(key, NULL, 0, 0); \
rte_ring_enqueue((ring), m); \
} while (0)
@@ -131,6 +131,7 @@
/* Function definitions */
uint64_t pipeline_test_hash(
void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed);
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index a2d19a1..417bc42 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -200,8 +200,6 @@ test_table_type(struct rte_table_ops *table_ops, void *table_args,
return -CHECK_TABLE_CONSISTENCY;
}
-
-
/* Flow test - All hits */
if (table_packets->n_hit_packets) {
for (i = 0; i < table_packets->n_hit_packets; i++)
@@ -248,7 +246,6 @@ test_table_type(struct rte_table_ops *table_ops, void *table_args,
VERIFY_TRAFFIC(RING_TX, table_packets->n_miss_packets, 0);
}
-
/* Change table entry action */
printf("Change entry action\n");
table_entry.table_id = ring_out_2_id;
@@ -441,12 +438,15 @@ test_table_hash8lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key8_lru_params key8lru_params = {
- .n_entries = 1<<24,
- .f_hash = pipeline_test_hash,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key8lru_params = {
+ .name = "TABLE",
+ .key_size = 8,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
uint8_t key8lru[8];
@@ -475,14 +475,14 @@ test_table_hash8lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key8lru_params.n_entries = 0;
+ key8lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8lru_params.n_entries = 1<<16;
+ key8lru_params.n_keys = 1<<16;
key8lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key8_lru_ops,
@@ -499,13 +499,15 @@ test_table_hash16lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key16_lru_params key16lru_params = {
- .n_entries = 1<<16,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key16lru_params = {
+ .name = "TABLE",
+ .key_size = 16,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
uint8_t key16lru[16];
@@ -534,14 +536,14 @@ test_table_hash16lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key16lru_params.n_entries = 0;
+ key16lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16lru_params.n_entries = 1<<16;
+ key16lru_params.n_keys = 1<<16;
key16lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key16_lru_ops,
@@ -558,12 +560,15 @@ test_table_hash32lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key32_lru_params key32lru_params = {
- .n_entries = 1<<16,
+ struct rte_table_hash_params key32lru_params = {
+ .name = "TABLE",
+ .key_size = 32,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
.f_hash = pipeline_test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
};
uint8_t key32lru[32];
@@ -592,14 +597,14 @@ test_table_hash32lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key32lru_params.n_entries = 0;
+ key32lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key32_lru_ops,
(void *)&key32lru_params, (void *)key32lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32lru_params.n_entries = 1<<16;
+ key32lru_params.n_keys = 1<<16;
key32lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key32_lru_ops,
@@ -616,14 +621,15 @@ test_table_hash8ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key8_ext_params key8ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key8ext_params = {
+ .name = "TABLE",
+ .key_size = 8,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
uint8_t key8ext[8];
@@ -652,14 +658,14 @@ test_table_hash8ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key8ext_params.n_entries = 0;
+ key8ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8ext_params.n_entries = 1<<16;
+ key8ext_params.n_keys = 1<<16;
key8ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key8_ext_ops,
@@ -667,13 +673,6 @@ test_table_hash8ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8ext_params.f_hash = pipeline_test_hash;
- key8ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key8_ext_ops,
- (void *)&key8ext_params, (void *)key8ext, &table_packets, NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
@@ -683,14 +682,15 @@ test_table_hash16ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key16_ext_params key16ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key16ext_params = {
+ .name = "TABLE",
+ .key_size = 16,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
uint8_t key16ext[16];
@@ -719,14 +719,14 @@ test_table_hash16ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key16ext_params.n_entries = 0;
+ key16ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16ext_params.n_entries = 1<<16;
+ key16ext_params.n_keys = 1<<16;
key16ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key16_ext_ops,
@@ -734,13 +734,6 @@ test_table_hash16ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16ext_params.f_hash = pipeline_test_hash;
- key16ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key16_ext_ops,
- (void *)&key16ext_params, (void *)key16ext, &table_packets, NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
@@ -750,13 +743,15 @@ test_table_hash32ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key32_ext_params key32ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
+ struct rte_table_hash_params key32ext_params = {
+ .name = "TABLE",
+ .key_size = 32,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
.f_hash = pipeline_test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
};
uint8_t key32ext[32];
@@ -785,14 +780,14 @@ test_table_hash32ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key32ext_params.n_entries = 0;
+ key32ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key32_ext_ops,
(void *)&key32ext_params, (void *)key32ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32ext_params.n_entries = 1<<16;
+ key32ext_params.n_keys = 1<<16;
key32ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key32_ext_ops,
@@ -800,14 +795,6 @@ test_table_hash32ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32ext_params.f_hash = pipeline_test_hash;
- key32ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key32_ext_ops,
- (void *)&key32ext_params, (void *)key32ext, &table_packets,
- NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
@@ -817,14 +804,15 @@ test_table_hash_cuckoo_combined(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_cuckoo_params cuckoo_params = {
+ struct rte_table_hash_params cuckoo_params = {
+ .name = "TABLE",
.key_size = 32,
- .n_keys = 1<<16,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
.f_hash = pipeline_test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .name = "CUCKOO_HASH",
};
uint8_t key_cuckoo[32];
@@ -847,7 +835,7 @@ test_table_hash_cuckoo_combined(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -855,7 +843,7 @@ test_table_hash_cuckoo_combined(void)
/* Invalid parameters */
cuckoo_params.key_size = 0;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -863,7 +851,7 @@ test_table_hash_cuckoo_combined(void)
cuckoo_params.key_size = 32;
cuckoo_params.n_keys = 0;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -871,7 +859,7 @@ test_table_hash_cuckoo_combined(void)
cuckoo_params.n_keys = 1<<16;
cuckoo_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index d835eb9..187a867 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -60,16 +60,16 @@ table_test table_tests[] = {
memset(key, 0, 32); \
k32 = (uint32_t *) key; \
k32[0] = (value); \
- *signature = pipeline_test_hash(key, 0, 0); \
+ *signature = pipeline_test_hash(key, NULL, 0, 0); \
} while (0)
unsigned n_table_tests = RTE_DIM(table_tests);
/* Function prototypes */
static int
-test_table_hash_lru_generic(struct rte_table_ops *ops);
+test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size);
static int
-test_table_hash_ext_generic(struct rte_table_ops *ops);
+test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size);
struct rte_bucket_4_8 {
/* Cache line 0 */
@@ -655,7 +655,7 @@ test_table_lpm_ipv6(void)
}
static int
-test_table_hash_lru_generic(struct rte_table_ops *ops)
+test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size)
{
int status, i;
uint64_t expected_mask = 0, result_mask;
@@ -667,36 +667,24 @@ test_table_hash_lru_generic(struct rte_table_ops *ops)
int key_found;
/* Initialize params and create tables */
- struct rte_table_hash_key8_lru_params hash_params = {
- .n_entries = 1 << 10,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(1),
+ struct rte_table_hash_params hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 10,
+ .n_buckets = 1 << 10,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
- hash_params.n_entries = 0;
+ hash_params.n_keys = 0;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -1;
- hash_params.n_entries = 1 << 10;
- hash_params.signature_offset = APP_METADATA_OFFSET(1);
-
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -2;
-
- hash_params.signature_offset = APP_METADATA_OFFSET(0);
- hash_params.key_offset = APP_METADATA_OFFSET(1);
-
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -3;
-
- hash_params.key_offset = APP_METADATA_OFFSET(32);
+ hash_params.n_keys = 1 << 10;
hash_params.f_hash = NULL;
table = ops->f_create(&hash_params, 0, 1);
@@ -770,7 +758,7 @@ test_table_hash_lru_generic(struct rte_table_ops *ops)
}
static int
-test_table_hash_ext_generic(struct rte_table_ops *ops)
+test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size)
{
int status, i;
uint64_t expected_mask = 0, result_mask;
@@ -782,35 +770,24 @@ test_table_hash_ext_generic(struct rte_table_ops *ops)
void *entry_ptr;
/* Initialize params and create tables */
- struct rte_table_hash_key8_ext_params hash_params = {
- .n_entries = 1 << 10,
- .n_entries_ext = 1 << 4,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(1),
+ struct rte_table_hash_params hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 10,
+ .n_buckets = 1 << 10,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
- hash_params.n_entries = 0;
+ hash_params.n_keys = 0;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -1;
- hash_params.n_entries = 1 << 10;
- hash_params.n_entries_ext = 0;
- table = ops->f_create(&hash_params, 0, 1);
- if (table != NULL)
- return -2;
-
- hash_params.n_entries_ext = 1 << 4;
- hash_params.signature_offset = APP_METADATA_OFFSET(1);
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -2;
-
- hash_params.signature_offset = APP_METADATA_OFFSET(0);
+ hash_params.n_keys = 1 << 10;
hash_params.key_offset = APP_METADATA_OFFSET(1);
table = ops->f_create(&hash_params, 0, 1);
@@ -895,20 +872,21 @@ test_table_hash_lru(void)
{
int status;
- status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_ops);
- if (status < 0)
- return status;
-
status = test_table_hash_lru_generic(
- &rte_table_hash_key8_lru_dosig_ops);
+ &rte_table_hash_key8_lru_ops,
+ 8);
if (status < 0)
return status;
- status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_ops);
+ status = test_table_hash_lru_generic(
+ &rte_table_hash_key16_lru_ops,
+ 16);
if (status < 0)
return status;
- status = test_table_hash_lru_generic(&rte_table_hash_key32_lru_ops);
+ status = test_table_hash_lru_generic(
+ &rte_table_hash_key32_lru_ops,
+ 32);
if (status < 0)
return status;
@@ -924,20 +902,15 @@ test_table_hash_ext(void)
{
int status;
- status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops, 8);
if (status < 0)
return status;
- status = test_table_hash_ext_generic(
- &rte_table_hash_key8_ext_dosig_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops, 16);
if (status < 0)
return status;
- status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_ext_generic(&rte_table_hash_key32_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key32_ext_ops, 32);
if (status < 0)
return status;
@@ -959,23 +932,24 @@ test_table_hash_cuckoo(void)
uint32_t entry_size = 1;
/* Initialize params and create tables */
- struct rte_table_hash_cuckoo_params cuckoo_params = {
+ struct rte_table_hash_params cuckoo_params = {
+ .name = "TABLE",
.key_size = 32,
- .n_keys = 1 << 24,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
.f_hash = pipeline_test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .name = "CUCKOO",
};
- table = rte_table_hash_cuckoo_dosig_ops.f_create(NULL, 0, entry_size);
+ table = rte_table_hash_cuckoo_ops.f_create(NULL, 0, entry_size);
if (table != NULL)
return -1;
cuckoo_params.key_size = 0;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -2;
@@ -983,7 +957,7 @@ test_table_hash_cuckoo(void)
cuckoo_params.key_size = 32;
cuckoo_params.n_keys = 0;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -3;
@@ -991,7 +965,7 @@ test_table_hash_cuckoo(void)
cuckoo_params.n_keys = 1 << 24;
cuckoo_params.f_hash = NULL;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -4;
@@ -999,24 +973,24 @@ test_table_hash_cuckoo(void)
cuckoo_params.f_hash = pipeline_test_hash;
cuckoo_params.name = NULL;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -5;
cuckoo_params.name = "CUCKOO";
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table == NULL)
return -6;
/* Free */
- status = rte_table_hash_cuckoo_dosig_ops.f_free(table);
+ status = rte_table_hash_cuckoo_ops.f_free(table);
if (status < 0)
return -7;
- status = rte_table_hash_cuckoo_dosig_ops.f_free(NULL);
+ status = rte_table_hash_cuckoo_ops.f_free(NULL);
if (status == 0)
return -8;
@@ -1027,60 +1001,60 @@ test_table_hash_cuckoo(void)
memset(key_cuckoo, 0, 32);
kcuckoo[0] = rte_be_to_cpu_32(0xadadadad);
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params, 0, 1);
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params, 0, 1);
if (table == NULL)
return -9;
entry = 'A';
- status = rte_table_hash_cuckoo_dosig_ops.f_add(NULL, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(NULL, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status == 0)
return -10;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, NULL, &entry,
+ status = rte_table_hash_cuckoo_ops.f_add(table, NULL, &entry,
&key_found, &entry_ptr);
if (status == 0)
return -11;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
NULL, &key_found, &entry_ptr);
if (status == 0)
return -12;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status != 0)
return -13;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status != 0)
return -14;
/* Delete */
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(NULL, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(NULL, &key_cuckoo,
&key_found, NULL);
if (status == 0)
return -15;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, NULL,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, NULL,
&key_found, NULL);
if (status == 0)
return -16;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, &key_cuckoo,
&key_found, NULL);
if (status != 0)
return -17;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, &key_cuckoo,
&key_found, NULL);
if (status != -ENOENT)
return -18;
/* Traffic flow */
entry = 'A';
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found,
&entry_ptr);
if (status < 0)
@@ -1093,7 +1067,7 @@ test_table_hash_cuckoo(void)
} else
PREPARE_PACKET(mbufs[i], 0xadadadab);
- rte_table_hash_cuckoo_dosig_ops.f_lookup(table, mbufs, -1,
+ rte_table_hash_cuckoo_ops.f_lookup(table, mbufs, -1,
&result_mask, (void **)entries);
if (result_mask != expected_mask)
return -20;
@@ -1102,7 +1076,7 @@ test_table_hash_cuckoo(void)
for (i = 0; i < RTE_PORT_IN_BURST_SIZE_MAX; i++)
rte_pktmbuf_free(mbufs[i]);
- status = rte_table_hash_cuckoo_dosig_ops.f_free(table);
+ status = rte_table_hash_cuckoo_ops.f_free(table);
return 0;
}
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH 3/5] test-pipeline: update due to api changes in librte_table
2017-08-24 13:53 [dpdk-dev] [PATCH 0/5] table: add key mask for hash tables Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 1/5] " Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 2/5] test: update due to api changes in librte_table Cristian Dumitrescu
@ 2017-08-24 13:53 ` Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 4/5] ip_pipeline: " Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 5/5] deprecation: removed the librte_table notice Cristian Dumitrescu
4 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-08-24 13:53 UTC (permalink / raw)
To: dev
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
test/test-pipeline/main.h | 5 +-
test/test-pipeline/pipeline_hash.c | 107 +++++--------------------------------
2 files changed, 18 insertions(+), 94 deletions(-)
diff --git a/test/test-pipeline/main.h b/test/test-pipeline/main.h
index 3685849..26395a3 100644
--- a/test/test-pipeline/main.h
+++ b/test/test-pipeline/main.h
@@ -131,7 +131,10 @@ enum {
void app_main_loop_rx(void);
void app_main_loop_rx_metadata(void);
-uint64_t test_hash(void *key, uint32_t key_size, uint64_t seed);
+uint64_t test_hash(void *key,
+ void *key_mask,
+ uint32_t key_size,
+ uint64_t seed);
void app_main_loop_worker(void);
void app_main_loop_worker_pipeline_stub(void);
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 991e381..edc1663 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -169,23 +169,23 @@ app_main_loop_worker_pipeline_hash(void) {
"ring %d\n", i);
}
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = test_hash,
+ .seed = 0,
+ };
+
/* Table configuration */
switch (app.pipeline_type) {
case e_APP_PIPELINE_HASH_KEY8_EXT:
case e_APP_PIPELINE_HASH_KEY16_EXT:
case e_APP_PIPELINE_HASH_KEY32_EXT:
{
- struct rte_table_hash_ext_params table_hash_params = {
- .key_size = key_size,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .n_buckets_ext = 1 << 21,
- .f_hash = test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_ext_ops,
.arg_create = &table_hash_params,
@@ -204,16 +204,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_KEY16_LRU:
case e_APP_PIPELINE_HASH_KEY32_LRU:
{
- struct rte_table_hash_lru_params table_hash_params = {
- .key_size = key_size,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_lru_ops,
.arg_create = &table_hash_params,
@@ -230,16 +220,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_EXT:
{
- struct rte_table_hash_key8_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .f_hash = test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_hash_params,
@@ -256,15 +236,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_LRU:
{
- struct rte_table_hash_key8_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .f_hash = test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key8_lru_ops,
.arg_create = &table_hash_params,
@@ -281,16 +252,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_EXT:
{
- struct rte_table_hash_key16_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
- .key_mask = NULL,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key16_ext_ops,
.arg_create = &table_hash_params,
@@ -307,15 +268,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_LRU:
{
- struct rte_table_hash_key16_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
- .key_mask = NULL,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key16_lru_ops,
.arg_create = &table_hash_params,
@@ -332,15 +284,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_EXT:
{
- struct rte_table_hash_key32_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key32_ext_ops,
.arg_create = &table_hash_params,
@@ -358,14 +301,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_LRU:
{
- struct rte_table_hash_key32_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key32_lru_ops,
.arg_create = &table_hash_params,
@@ -390,23 +325,8 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_CUCKOO_KEY112:
case e_APP_PIPELINE_HASH_CUCKOO_KEY128:
{
- char hash_name[RTE_HASH_NAMESIZE];
-
- snprintf(hash_name, sizeof(hash_name), "RTE_TH_CUCKOO_%d",
- app.pipeline_type);
-
- struct rte_table_hash_cuckoo_params table_hash_params = {
- .key_size = key_size,
- .n_keys = (1 << 24) + 1,
- .f_hash = test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .name = hash_name,
- };
-
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_cuckoo_dosig_ops,
+ .ops = &rte_table_hash_cuckoo_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -477,6 +397,7 @@ app_main_loop_worker_pipeline_hash(void) {
uint64_t test_hash(
void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed)
{
@@ -539,7 +460,7 @@ app_main_loop_rx_metadata(void) {
} else
continue;
- *signature = test_hash(key, 0, 0);
+ *signature = test_hash(key, NULL, 0, 0);
}
do {
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH 4/5] ip_pipeline: update due to api changes in librte_table
2017-08-24 13:53 [dpdk-dev] [PATCH 0/5] table: add key mask for hash tables Cristian Dumitrescu
` (2 preceding siblings ...)
2017-08-24 13:53 ` [dpdk-dev] [PATCH 3/5] test-pipeline: " Cristian Dumitrescu
@ 2017-08-24 13:53 ` Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 5/5] deprecation: removed the librte_table notice Cristian Dumitrescu
4 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-08-24 13:53 UTC (permalink / raw)
To: dev
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification.c | 9 ++--
.../pipeline/pipeline_flow_classification_be.c | 54 ++++------------------
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 18 ++++----
.../ip_pipeline/pipeline/pipeline_routing_be.c | 18 ++++----
4 files changed, 33 insertions(+), 66 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
index 9ef50cc..a8fae1e 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
@@ -88,8 +88,11 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
uint32_t *signature)
{
uint8_t buffer[PIPELINE_FC_FLOW_KEY_MAX_SIZE];
+ uint8_t m[PIPELINE_FC_FLOW_KEY_MAX_SIZE]; /* key mask */
void *key_buffer = (key_out) ? key_out : buffer;
+ memset(m, 0xFF, sizeof(m));
+
switch (key_in->type) {
case FLOW_KEY_QINQ:
{
@@ -101,7 +104,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
qinq->cvlan = rte_cpu_to_be_16(key_in->key.qinq.cvlan);
if (signature)
- *signature = (uint32_t) hash_default_key8(qinq, 8, 0);
+ *signature = (uint32_t) hash_default_key8(qinq, m, 8, 0);
return 0;
}
@@ -118,7 +121,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
ipv4->port_dst = rte_cpu_to_be_16(key_in->key.ipv4_5tuple.port_dst);
if (signature)
- *signature = (uint32_t) hash_default_key16(ipv4, 16, 0);
+ *signature = (uint32_t) hash_default_key16(ipv4, m, 16, 0);
return 0;
}
@@ -136,7 +139,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
ipv6->port_dst = rte_cpu_to_be_16(key_in->key.ipv6_5tuple.port_dst);
if (signature)
- *signature = (uint32_t) hash_default_key64(ipv6, 64, 0);
+ *signature = (uint32_t) hash_default_key64(ipv6, m, 64, 0);
return 0;
}
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 026f00c..e489957 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -492,40 +492,15 @@ static void *pipeline_fc_init(struct pipeline_params *params,
/* Tables */
p->n_tables = 1;
{
- struct rte_table_hash_key8_ext_params
- table_hash_key8_params = {
- .n_entries = p_fc->n_flows,
- .n_entries_ext = p_fc->n_flows,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
- .key_mask = (p_fc->key_mask_present) ?
- p_fc->key_mask : NULL,
- .seed = 0,
- };
-
- struct rte_table_hash_key16_ext_params
- table_hash_key16_params = {
- .n_entries = p_fc->n_flows,
- .n_entries_ext = p_fc->n_flows,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
- .key_mask = (p_fc->key_mask_present) ?
- p_fc->key_mask : NULL,
- .seed = 0,
- };
-
- struct rte_table_hash_ext_params
- table_hash_params = {
+ struct rte_table_hash_params table_hash_params = {
+ .name = p->name,
.key_size = p_fc->key_size,
+ .key_offset = p_fc->key_offset,
+ .key_mask = (p_fc->key_mask_present)? p_fc->key_mask : NULL,
.n_keys = p_fc->n_flows,
.n_buckets = p_fc->n_flows / 4,
- .n_buckets_ext = p_fc->n_flows / 4,
.f_hash = hash_func[(p_fc->key_size / 8) - 1],
.seed = 0,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
};
struct rte_pipeline_table_params table_params = {
@@ -542,32 +517,19 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
- if (p_fc->hash_offset != 0) {
- table_params.ops =
- &rte_table_hash_key8_ext_ops;
- } else {
- table_params.ops =
- &rte_table_hash_key8_ext_dosig_ops;
- }
- table_params.arg_create = &table_hash_key8_params;
+ table_params.ops = &rte_table_hash_key8_ext_ops;
break;
case 16:
- if (p_fc->hash_offset != 0) {
- table_params.ops =
- &rte_table_hash_key16_ext_ops;
- } else {
- table_params.ops =
- &rte_table_hash_key16_ext_dosig_ops;
- }
- table_params.arg_create = &table_hash_key16_params;
+ table_params.ops = &rte_table_hash_key16_ext_ops;
break;
default:
table_params.ops = &rte_table_hash_ext_ops;
- table_params.arg_create = &table_hash_params;
}
+ table_params.arg_create = &table_hash_params;
+
status = rte_pipeline_table_create(p->p,
&table_params,
&p->table_id[0]);
diff --git a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
index 8cb2f0c..2500332 100644
--- a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
@@ -76,7 +76,7 @@ static pipeline_msg_req_handler handlers[] = {
pipeline_msg_req_invalid_handler,
};
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt_work_dma(
struct rte_mbuf *pkt,
void *arg,
@@ -102,7 +102,7 @@ pkt_work_dma(
/* Read (dma_dst), compute (hash), write (hash) */
if (hash_enabled) {
- uint32_t hash = p->f_hash(dma_dst, dma_size, 0);
+ uint32_t hash = p->f_hash(dma_src, dma_mask, dma_size, 0);
*dma_hash = hash;
if (lb_hash) {
@@ -121,7 +121,7 @@ pkt_work_dma(
}
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt4_work_dma(
struct rte_mbuf **pkts,
void *arg,
@@ -173,10 +173,10 @@ pkt4_work_dma(
/* Read (dma_dst), compute (hash), write (hash) */
if (hash_enabled) {
- uint32_t hash0 = p->f_hash(dma_dst0, dma_size, 0);
- uint32_t hash1 = p->f_hash(dma_dst1, dma_size, 0);
- uint32_t hash2 = p->f_hash(dma_dst2, dma_size, 0);
- uint32_t hash3 = p->f_hash(dma_dst3, dma_size, 0);
+ uint32_t hash0 = p->f_hash(dma_src0, dma_mask, dma_size, 0);
+ uint32_t hash1 = p->f_hash(dma_src1, dma_mask, dma_size, 0);
+ uint32_t hash2 = p->f_hash(dma_src2, dma_mask, dma_size, 0);
+ uint32_t hash3 = p->f_hash(dma_src3, dma_mask, dma_size, 0);
*dma_hash0 = hash0;
*dma_hash1 = hash1;
@@ -217,7 +217,7 @@ pkt4_work_dma(
}
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt_work_swap(
struct rte_mbuf *pkt,
void *arg)
@@ -241,7 +241,7 @@ pkt_work_swap(
}
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt4_work_swap(
struct rte_mbuf **pkts,
void *arg)
diff --git a/examples/ip_pipeline/pipeline/pipeline_routing_be.c b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
index 7831716..d281e65 100644
--- a/examples/ip_pipeline/pipeline/pipeline_routing_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
@@ -191,7 +191,7 @@ struct layout {
dst->c = src->c; \
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt_work_routing(
struct rte_mbuf *pkt,
struct rte_pipeline_table_entry *table_entry,
@@ -317,7 +317,7 @@ pkt_work_routing(
}
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt4_work_routing(
struct rte_mbuf **pkts,
struct rte_pipeline_table_entry **table_entries,
@@ -1349,17 +1349,19 @@ pipeline_routing_init(struct pipeline_params *params,
/* ARP table configuration */
if (p_rt->params.n_arp_entries) {
- struct rte_table_hash_key8_ext_params table_arp_params = {
- .n_entries = p_rt->params.n_arp_entries,
- .n_entries_ext = p_rt->params.n_arp_entries,
+ struct rte_table_hash_params table_arp_params = {
+ .name = p->name,
+ .key_size = 8,
+ .key_offset = p_rt->params.arp_key_offset,
+ .key_mask = NULL,
+ .n_keys = p_rt->params.n_arp_entries,
+ .n_buckets = p_rt->params.n_arp_entries / 4,
.f_hash = hash_default_key8,
.seed = 0,
- .signature_offset = 0, /* Unused */
- .key_offset = p_rt->params.arp_key_offset,
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_ext_dosig_ops,
+ .ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_arp_params,
.f_action_hit = get_arp_table_ah_hit(p_rt),
.f_action_miss = NULL,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH 5/5] deprecation: removed the librte_table notice
2017-08-24 13:53 [dpdk-dev] [PATCH 0/5] table: add key mask for hash tables Cristian Dumitrescu
` (3 preceding siblings ...)
2017-08-24 13:53 ` [dpdk-dev] [PATCH 4/5] ip_pipeline: " Cristian Dumitrescu
@ 2017-08-24 13:53 ` Cristian Dumitrescu
4 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-08-24 13:53 UTC (permalink / raw)
To: dev
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
doc/guides/rel_notes/deprecation.rst | 8 --------
1 file changed, 8 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 3362f33..e181a61 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -112,11 +112,3 @@ Deprecation Notices
- ``rte_cryptodev_allocate_driver``
-* librte_table: The ``key_mask`` parameter will be added to all the hash tables
- that currently do not have it, as well as to the hash compute function prototype.
- The non-"do-sig" versions of the hash tables will be removed
- (including the ``signature_offset`` parameter)
- and the "do-sig" versions renamed accordingly.
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH V2 0/5] table: add key mask for hash tables
2017-08-24 13:53 ` [dpdk-dev] [PATCH 1/5] " Cristian Dumitrescu
@ 2017-10-10 11:18 ` Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 1/5] " Cristian Dumitrescu
` (5 more replies)
0 siblings, 6 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-10 11:18 UTC (permalink / raw)
To: dev
Main changes:
1. The key_mask parameter is added to all the hash tables that were
previously missing it, as well to the hash compute function. This was
first started in DPDK 2.0, but was only implemented for a couple of
hash tables. The benefit of this approach is that it allows for better
performance for large keys (bigger than 16 bytes), while it preserves
the same performance for small keys [Q&A1].
2. The precomputed key signature (i.e. non-"do-sig") versions have been
removed for all the hash tables, so now the key signature is always
computed on every lookup. Note that this approach also allows for the
precomputed key signature scheme [Q&A2].
3. API cleanup: single parameter structure common for all hash tables.
Q&A:
Q1: How is better lookup performance achieved by using key mask approach
for hash tables?
A1: This approach eliminates the need to consolidate the lookup key in a
single contiguous buffer where the relevant packet fields are written
one by one, which is a very costly operation that also hash strong data
dependencies.
Q2: How can the pre-computed key signature scheme be implemented with
current approach?
A2: The application can implement a straightforward custom hash function
that simply reads the pre-computed key signature from a given offset
in the input key buffer where it has been stored prior to the lookup
operation.
Cristian Dumitrescu (5):
table: add key mask for hash tables
test: update due to api changes in librte_table
test-pipeline: update due to api changes in librte_table
ip_pipeline: update due to api changes in librte_table
deprecation: removed the librte_table notice
doc/guides/rel_notes/deprecation.rst | 6 -
examples/ip_pipeline/pipeline/hash_func.h | 178 +++--
.../pipeline/pipeline_flow_classification.c | 12 +-
.../pipeline/pipeline_flow_classification_be.c | 51 +-
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 18 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 18 +-
lib/librte_table/rte_table_hash.h | 306 +--------
lib/librte_table/rte_table_hash_cuckoo.c | 205 +++---
lib/librte_table/rte_table_hash_ext.c | 417 ++++--------
lib/librte_table/rte_table_hash_key16.c | 749 ++++++---------------
lib/librte_table/rte_table_hash_key32.c | 435 +++++++-----
lib/librte_table/rte_table_hash_key8.c | 715 ++++++--------------
lib/librte_table/rte_table_hash_lru.c | 513 ++++++--------
lib/librte_table/rte_table_version.map | 23 +-
test/test-pipeline/main.h | 5 +-
test/test-pipeline/pipeline_hash.c | 107 +--
test/test/test_table.c | 1 +
test/test/test_table.h | 3 +-
test/test/test_table_combined.c | 140 ++--
test/test/test_table_tables.c | 148 ++--
20 files changed, 1444 insertions(+), 2606 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH V2 1/5] table: add key mask for hash tables
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 0/5] " Cristian Dumitrescu
@ 2017-10-10 11:18 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 2/5] test: update due to api changes in librte_table Cristian Dumitrescu
` (4 subsequent siblings)
5 siblings, 1 reply; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-10 11:18 UTC (permalink / raw)
To: dev
Main changes:
1. The key_mask parameter is added to all the hash tables that were
previously missing it, as well to the hash compute function. This was
first started in DPDK 2.0, but was only implemented for a couple of
hash tables. The benefit of this approach is that it allows for better
performance for large keys (bigger than 16 bytes), while it preserves
the same performance for small keys [Q&A1].
2. The precomputed key signature (i.e. non-"do-sig") versions have been
removed for all the hash tables, so now the key signature is always
computed on every lookup. Note that this approach also allows for the
precomputed key signature scheme [Q&A2].
3. API cleanup: single parameter structure common for all hash tables.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
Changes in V2:
- Removed unnecessary constraints and improved checks for input params
- Fixed potential overflow issues for large tables
- Fixed map file
- Fixed minor style issues
lib/librte_table/rte_table_hash.h | 306 +------------
lib/librte_table/rte_table_hash_cuckoo.c | 205 ++++-----
lib/librte_table/rte_table_hash_ext.c | 417 ++++++-----------
lib/librte_table/rte_table_hash_key16.c | 749 +++++++++----------------------
lib/librte_table/rte_table_hash_key32.c | 435 +++++++++++-------
lib/librte_table/rte_table_hash_key8.c | 715 +++++++++--------------------
lib/librte_table/rte_table_hash_lru.c | 513 +++++++++------------
lib/librte_table/rte_table_version.map | 23 +-
8 files changed, 1160 insertions(+), 2203 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 57505a6..4ee6984 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -45,8 +45,6 @@ extern "C" {
* These tables use the exact match criterion to uniquely associate data to
* lookup keys.
*
- * Use-cases: Flow classification table, Address Resolution Protocol (ARP) table
- *
* Hash table types:
* 1. Entry add strategy on bucket full:
* a. Least Recently Used (LRU): One of the existing keys in the bucket is
@@ -59,7 +57,7 @@ extern "C" {
* to the bucket, it also becomes the new MRU key. When a key needs to
* be picked and dropped, the most likely candidate for drop, i.e. the
* current LRU key, is always picked. The LRU logic requires maintaining
- * specific data structures per each bucket.
+ * specific data structures per each bucket. Use-cases: flow cache, etc.
* b. Extendible bucket (ext): The bucket is extended with space for 4 more
* keys. This is done by allocating additional memory at table init time,
* which is used to create a pool of free keys (the size of this pool is
@@ -73,20 +71,8 @@ extern "C" {
* first group of 4 keys, the search continues beyond the first group of
* 4 keys, potentially until all keys in this bucket are examined. The
* extendible bucket logic requires maintaining specific data structures
- * per table and per each bucket.
- * 2. Key signature computation:
- * a. Pre-computed key signature: The key lookup operation is split between
- * two CPU cores. The first CPU core (typically the CPU core performing
- * packet RX) extracts the key from the input packet, computes the key
- * signature and saves both the key and the key signature in the packet
- * buffer as packet meta-data. The second CPU core reads both the key and
- * the key signature from the packet meta-data and performs the bucket
- * search step of the key lookup operation.
- * b. Key signature computed on lookup (do-sig): The same CPU core reads
- * the key from the packet meta-data, uses it to compute the key
- * signature and also performs the bucket search step of the key lookup
- * operation.
- * 3. Key size:
+ * per table and per each bucket. Use-cases: flow table, etc.
+ * 2. Key size:
* a. Configurable key size
* b. Single key size (8-byte, 16-byte or 32-byte key size)
*
@@ -98,59 +84,28 @@ extern "C" {
/** Hash function */
typedef uint64_t (*rte_table_hash_op_hash)(
void *key,
+ void *key_mask,
uint32_t key_size,
uint64_t seed);
-/**
- * Hash tables with configurable key size
- *
- */
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_ext_params {
+/** Hash table parameters */
+struct rte_table_hash_params {
+ /** Name */
+ const char *name;
+
/** Key size (number of bytes) */
uint32_t key_size;
- /** Maximum number of keys */
- uint32_t n_keys;
-
- /** Number of hash table buckets. Each bucket stores up to 4 keys. */
- uint32_t n_buckets;
-
- /** Number of hash table bucket extensions. Each bucket extension has
- space for 4 keys and each bucket can have 0, 1 or more extensions. */
- uint32_t n_buckets_ext;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed value for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
/** Byte offset within packet meta-data where the key is located */
uint32_t key_offset;
-};
-/** Extendible bucket hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_ext_ops;
-
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_ext_dosig_ops;
-
-/** LRU hash table parameters */
-struct rte_table_hash_lru_params {
- /** Key size (number of bytes) */
- uint32_t key_size;
+ /** Key mask */
+ uint8_t *key_mask;
- /** Maximum number of keys */
+ /** Number of keys */
uint32_t n_keys;
- /** Number of hash table buckets. Each bucket stores up to 4 keys. */
+ /** Number of buckets */
uint32_t n_buckets;
/** Hash function */
@@ -158,239 +113,22 @@ struct rte_table_hash_lru_params {
/** Seed value for the hash function */
uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
};
-/** LRU hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_lru_ops;
-
-/** LRU hash table operations for key signature computed on lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_lru_dosig_ops;
-
-/**
- * 8-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key8_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
-/** LRU hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_key8_lru_ops;
-
-/** LRU hash table operations for key signature computed on lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key8_lru_dosig_ops;
-
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key8_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
-/** Extendible bucket hash table operations for pre-computed key signature */
+/** Extendible bucket hash table operations */
+extern struct rte_table_ops rte_table_hash_ext_ops;
extern struct rte_table_ops rte_table_hash_key8_ext_ops;
-
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key8_ext_dosig_ops;
-
-/**
- * 16-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key16_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
-/** LRU hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_key16_lru_ops;
-
-/** LRU hash table operations for key signature computed on lookup
- ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key16_lru_dosig_ops;
-
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key16_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
-/** Extendible bucket operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key16_ext_ops;
-
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key16_ext_dosig_ops;
-
-/**
- * 32-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key32_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
-/** LRU hash table operations for pre-computed key signature */
-extern struct rte_table_ops rte_table_hash_key32_lru_ops;
-
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key32_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
-/** Extendible bucket hash table operations */
extern struct rte_table_ops rte_table_hash_key32_ext_ops;
-/** Cuckoo hash table parameters */
-struct rte_table_hash_cuckoo_params {
- /** Key size (number of bytes */
- uint32_t key_size;
-
- /** Maximum number of hash table entries */
- uint32_t n_keys;
-
- /** Hash function used to calculate hash */
- rte_table_hash_op_hash f_hash;
-
- /** Seed value or Init value used by f_hash */
- uint32_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Hash table name */
- const char *name;
-};
+/** LRU hash table operations */
+extern struct rte_table_ops rte_table_hash_lru_ops;
+extern struct rte_table_ops rte_table_hash_key8_lru_ops;
+extern struct rte_table_ops rte_table_hash_key16_lru_ops;
+extern struct rte_table_ops rte_table_hash_key32_lru_ops;
/** Cuckoo hash table operations */
-extern struct rte_table_ops rte_table_hash_cuckoo_dosig_ops;
+extern struct rte_table_ops rte_table_hash_cuckoo_ops;
#ifdef __cplusplus
}
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
index da1597f..f3845c7 100644
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2016 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
@@ -66,25 +66,28 @@ struct rte_table_hash {
uint32_t n_keys;
rte_table_hash_op_hash f_hash;
uint32_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
- const char *name;
/* cuckoo hash table object */
struct rte_hash *h_table;
/* Lookup table */
- uint8_t memory[0] __rte_cache_aligned; };
+ uint8_t memory[0] __rte_cache_aligned;
+};
static int
-check_params_create_hash_cuckoo(const struct
-rte_table_hash_cuckoo_params *params) {
- /* Check for valid parameters */
+check_params_create_hash_cuckoo(struct rte_table_hash_params *params)
+{
if (params == NULL) {
RTE_LOG(ERR, TABLE, "NULL Input Parameters.\n");
return -EINVAL;
}
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "Table name is NULL.\n");
+ return -EINVAL;
+ }
+
if (params->key_size == 0) {
RTE_LOG(ERR, TABLE, "Invalid key_size.\n");
return -EINVAL;
@@ -100,11 +103,6 @@ rte_table_hash_cuckoo_params *params) {
return -EINVAL;
}
- if (params->name == NULL) {
- RTE_LOG(ERR, TABLE, "Table name is NULL.\n");
- return -EINVAL;
- }
-
return 0;
}
@@ -113,34 +111,24 @@ rte_table_hash_cuckoo_create(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_hash *rte_hash_handle;
+ struct rte_table_hash_params *p = params;
+ struct rte_hash *h_table;
struct rte_table_hash *t;
- uint32_t total_size, total_cl_size;
+ uint32_t total_size;
/* Check input parameters */
- struct rte_table_hash_cuckoo_params *p =
- (struct rte_table_hash_cuckoo_params *) params;
-
if (check_params_create_hash_cuckoo(params))
return NULL;
/* Memory allocation */
- total_cl_size =
- (sizeof(struct rte_table_hash) +
- RTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;
- total_cl_size += (p->n_keys * entry_size +
- RTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;
- total_size = total_cl_size * RTE_CACHE_LINE_SIZE;
-
- t = rte_zmalloc_socket("TABLE",
- total_size,
- RTE_CACHE_LINE_SIZE,
- socket_id);
+ total_size = sizeof(struct rte_table_hash) +
+ RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
+
+ t = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (t == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for Cuckoo hash table\n",
- __func__,
- (uint32_t)sizeof(struct rte_table_hash));
+ "%s: Cannot allocate %u bytes for cuckoo hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
@@ -154,13 +142,13 @@ rte_table_hash_cuckoo_create(void *params,
.name = p->name
};
- rte_hash_handle = rte_hash_find_existing(p->name);
- if (rte_hash_handle == NULL) {
- rte_hash_handle = rte_hash_create(&hash_cuckoo_params);
- if (NULL == rte_hash_handle) {
+ h_table = rte_hash_find_existing(p->name);
+ if (h_table == NULL) {
+ h_table = rte_hash_create(&hash_cuckoo_params);
+ if (h_table == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: failed to create cuckoo hash table. keysize: %u",
- __func__, hash_cuckoo_params.key_len);
+ "%s: failed to create cuckoo hash table %s\n",
+ __func__, p->name);
rte_free(t);
return NULL;
}
@@ -172,26 +160,22 @@ rte_table_hash_cuckoo_create(void *params,
t->n_keys = p->n_keys;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
- t->name = p->name;
- t->h_table = rte_hash_handle;
+ t->h_table = h_table;
RTE_LOG(INFO, TABLE,
- "%s: Cuckoo Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ "%s: Cuckoo hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
return t;
}
static int
rte_table_hash_cuckoo_free(void *table) {
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
struct rte_table_hash *t = table;
+ if (table == NULL)
+ return -EINVAL;
+
rte_hash_free(t->h_table);
rte_free(t);
@@ -200,25 +184,18 @@ rte_table_hash_cuckoo_free(void *table) {
static int
rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
- int *key_found, void **entry_ptr) {
+ int *key_found, void **entry_ptr)
+{
+ struct rte_table_hash *t = table;
int pos = 0;
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (key == NULL) {
- RTE_LOG(ERR, TABLE, "%s: key parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (entry == NULL) {
- RTE_LOG(ERR, TABLE, "%s: entry parameter is NULL\n", __func__);
+ /* Check input parameters */
+ if ((table == NULL) ||
+ (key == NULL) ||
+ (entry == NULL) ||
+ (key_found == NULL) ||
+ (entry_ptr == NULL))
return -EINVAL;
- }
-
- struct rte_table_hash *t = table;
/* Find Existing entries */
pos = rte_hash_lookup(t->h_table, key);
@@ -231,17 +208,15 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
*entry_ptr = existing_entry;
return 0;
-} else if (pos == -ENOENT) {
- /* Entry not found. Adding new entry */
+ }
+
+ if (pos == -ENOENT) {
+ /* Entry not found. Adding new entry */
uint8_t *new_entry;
pos = rte_hash_add_key(t->h_table, key);
- if (pos < 0) {
- RTE_LOG(ERR, TABLE,
- "%s: Entry not added, status : %u\n",
- __func__, pos);
+ if (pos < 0)
return pos;
- }
new_entry = &t->memory[pos * t->entry_size];
memcpy(new_entry, entry, t->entry_size);
@@ -250,25 +225,22 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
*entry_ptr = new_entry;
return 0;
}
+
return pos;
}
static int
rte_table_hash_cuckoo_entry_delete(void *table, void *key,
- int *key_found, __rte_unused void *entry) {
+ int *key_found, void *entry)
+{
+ struct rte_table_hash *t = table;
int pos = 0;
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (key == NULL) {
- RTE_LOG(ERR, TABLE, "%s: key parameter is NULL\n", __func__);
+ /* Check input parameters */
+ if ((table == NULL) ||
+ (key == NULL) ||
+ (key_found == NULL))
return -EINVAL;
- }
-
- struct rte_table_hash *t = table;
pos = rte_hash_del_key(t->h_table, key);
if (pos >= 0) {
@@ -279,20 +251,21 @@ rte_table_hash_cuckoo_entry_delete(void *table, void *key,
memcpy(entry, entry_ptr, t->entry_size);
memset(&t->memory[pos * t->entry_size], 0, t->entry_size);
+ return 0;
}
+ *key_found = 0;
return pos;
}
-
static int
-rte_table_hash_cuckoo_lookup_dosig(void *table,
+rte_table_hash_cuckoo_lookup(void *table,
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
void **entries)
{
- struct rte_table_hash *t = (struct rte_table_hash *)table;
+ struct rte_table_hash *t = table;
uint64_t pkts_mask_out = 0;
uint32_t i;
@@ -301,20 +274,19 @@ rte_table_hash_cuckoo_lookup_dosig(void *table,
RTE_TABLE_HASH_CUCKOO_STATS_PKTS_IN_ADD(t, n_pkts_in);
if ((pkts_mask & (pkts_mask + 1)) == 0) {
- const uint8_t *keys[64];
- int32_t positions[64], status;
+ const uint8_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
+ int32_t positions[RTE_PORT_IN_BURST_SIZE_MAX], status;
/* Keys for bulk lookup */
for (i = 0; i < n_pkts_in; i++)
keys[i] = RTE_MBUF_METADATA_UINT8_PTR(pkts[i],
- t->key_offset);
+ t->key_offset);
/* Bulk Lookup */
status = rte_hash_lookup_bulk(t->h_table,
(const void **) keys,
n_pkts_in,
positions);
-
if (status == 0) {
for (i = 0; i < n_pkts_in; i++) {
if (likely(positions[i] >= 0)) {
@@ -326,7 +298,7 @@ rte_table_hash_cuckoo_lookup_dosig(void *table,
}
}
}
- } else {
+ } else
for (i = 0; i < (uint32_t)(RTE_PORT_IN_BURST_SIZE_MAX
- __builtin_clzll(pkts_mask)); i++) {
uint64_t pkt_mask = 1LLU << i;
@@ -345,7 +317,6 @@ rte_table_hash_cuckoo_lookup_dosig(void *table,
}
}
}
- }
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_CUCKOO_STATS_PKTS_LOOKUP_MISS(t,
@@ -370,13 +341,13 @@ rte_table_hash_cuckoo_stats_read(void *table, struct rte_table_stats *stats,
return 0;
}
-struct rte_table_ops rte_table_hash_cuckoo_dosig_ops = {
+struct rte_table_ops rte_table_hash_cuckoo_ops = {
.f_create = rte_table_hash_cuckoo_create,
.f_free = rte_table_hash_cuckoo_free,
.f_add = rte_table_hash_cuckoo_entry_add,
.f_delete = rte_table_hash_cuckoo_entry_delete,
.f_add_bulk = NULL,
.f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_cuckoo_lookup_dosig,
+ .f_lookup = rte_table_hash_cuckoo_lookup,
.f_stats = rte_table_hash_cuckoo_stats_read,
};
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
index e718102..3af1bca 100644
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
@@ -106,7 +106,6 @@ struct rte_table_hash {
uint32_t n_buckets_ext;
rte_table_hash_op_hash f_hash;
uint64_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
/* Internal */
@@ -120,6 +119,7 @@ struct rte_table_hash {
struct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];
/* Tables */
+ uint64_t *key_mask;
struct bucket *buckets;
struct bucket *buckets_ext;
uint8_t *key_mem;
@@ -132,29 +132,53 @@ struct rte_table_hash {
};
static int
-check_params_create(struct rte_table_hash_ext_params *params)
+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)
{
- uint32_t n_buckets_min;
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ if (a64[i] != (b64[i] & b_mask64[i]))
+ return 1;
+
+ return 0;
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ dst64[i] = src64[i] & src_mask64[i];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
/* key_size */
- if ((params->key_size == 0) ||
+ if ((params->key_size < sizeof(uint64_t)) ||
(!rte_is_power_of_2(params->key_size))) {
RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
return -EINVAL;
}
/* n_keys */
- if ((params->n_keys == 0) ||
- (!rte_is_power_of_2(params->n_keys))) {
+ if (params->n_keys == 0) {
RTE_LOG(ERR, TABLE, "%s: n_keys invalid value\n", __func__);
return -EINVAL;
}
/* n_buckets */
- n_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;
if ((params->n_buckets == 0) ||
- (!rte_is_power_of_2(params->n_keys)) ||
- (params->n_buckets < n_buckets_min)) {
+ (!rte_is_power_of_2(params->n_buckets))) {
RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -171,15 +195,13 @@ check_params_create(struct rte_table_hash_ext_params *params)
static void *
rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_ext_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *t;
- uint32_t total_size, table_meta_sz;
- uint32_t bucket_sz, bucket_ext_sz, key_sz;
- uint32_t key_stack_sz, bkt_ext_stack_sz, data_sz;
- uint32_t bucket_offset, bucket_ext_offset, key_offset;
- uint32_t key_stack_offset, bkt_ext_stack_offset, data_offset;
- uint32_t i;
+ uint64_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz;
+ uint64_t key_stack_sz, bkt_ext_stack_sz, data_sz, total_size;
+ uint64_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset;
+ uint64_t key_stack_offset, bkt_ext_stack_offset, data_offset;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
if ((check_params_create(p) != 0) ||
@@ -188,38 +210,66 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
(sizeof(struct bucket) != (RTE_CACHE_LINE_SIZE / 2)))
return NULL;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
+
/* Memory allocation */
table_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));
+ key_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);
bucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));
bucket_ext_sz =
- RTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(struct bucket));
+ RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(struct bucket));
key_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);
key_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));
bkt_ext_stack_sz =
- RTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(uint32_t));
+ RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
data_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
- total_size = table_meta_sz + bucket_sz + bucket_ext_sz + key_sz +
- key_stack_sz + bkt_ext_stack_sz + data_sz;
+ total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz +
+ key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz;
- t = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ t = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (t == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table memory footprint is "
- "%u bytes\n", __func__, p->key_size, total_size);
+ RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table %s memory "
+ "footprint is %" PRIu64 " bytes\n",
+ __func__, p->key_size, p->name, total_size);
/* Memory initialization */
t->key_size = p->key_size;
t->entry_size = entry_size;
t->n_keys = p->n_keys;
t->n_buckets = p->n_buckets;
- t->n_buckets_ext = p->n_buckets_ext;
+ t->n_buckets_ext = n_buckets_ext;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
/* Internal */
@@ -228,13 +278,15 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
t->data_size_shl = __builtin_ctzl(entry_size);
/* Tables */
- bucket_offset = 0;
+ key_mask_offset = 0;
+ bucket_offset = key_mask_offset + key_mask_sz;
bucket_ext_offset = bucket_offset + bucket_sz;
key_offset = bucket_ext_offset + bucket_ext_sz;
key_stack_offset = key_offset + key_sz;
bkt_ext_stack_offset = key_stack_offset + key_stack_sz;
data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz;
+ t->key_mask = (uint64_t *) &t->memory[key_mask_offset];
t->buckets = (struct bucket *) &t->memory[bucket_offset];
t->buckets_ext = (struct bucket *) &t->memory[bucket_ext_offset];
t->key_mem = &t->memory[key_offset];
@@ -242,6 +294,12 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
t->bkt_ext_stack = (uint32_t *) &t->memory[bkt_ext_stack_offset];
t->data_mem = &t->memory[data_offset];
+ /* Key mask */
+ if (p->key_mask == NULL)
+ memset(t->key_mask, 0xFF, p->key_size);
+ else
+ memcpy(t->key_mask, p->key_mask, p->key_size);
+
/* Key stack */
for (i = 0; i < t->n_keys; i++)
t->key_stack[i] = t->n_keys - 1 - i;
@@ -277,7 +335,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -290,7 +348,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
uint8_t *bkt_key =
&t->key_mem[bkt_key_index << t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -327,7 +385,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
bkt->sig[i] = (uint16_t) sig;
bkt->key_pos[i] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
*key_found = 0;
@@ -358,7 +416,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
/* Install new key into bucket */
bkt->sig[0] = (uint16_t) sig;
bkt->key_pos[0] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
*key_found = 0;
@@ -378,7 +436,7 @@ void *entry)
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -392,7 +450,7 @@ void *entry)
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -437,8 +495,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
- void **entries,
- int dosig)
+ void **entries)
{
struct rte_table_hash *t = (struct rte_table_hash *) table;
uint64_t pkts_mask_out = 0;
@@ -458,11 +515,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- if (dosig)
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
- else
- sig = RTE_MBUF_METADATA_UINT32(pkt,
- t->signature_offset);
+ sig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
@@ -476,8 +529,8 @@ static int rte_table_hash_ext_lookup_unoptimized(
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
- t->key_size) == 0)) {
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key,
+ t->key_mask, t->key_size) == 0)) {
uint8_t *data = &t->data_mem[
bkt_key_index << t->data_size_shl];
@@ -576,11 +629,12 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\
uint64_t *bkt_key = (uint64_t *) key; \
+ uint64_t *key_mask = f->key_mask; \
\
switch (f->key_size) { \
case 8: \
{ \
- uint64_t xor = pkt_key[0] ^ bkt_key[0]; \
+ uint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
match_key = 0; \
if (xor == 0) \
match_key = 1; \
@@ -591,8 +645,8 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[2], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
or = xor[0] | xor[1]; \
match_key = 0; \
if (or == 0) \
@@ -604,10 +658,10 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[4], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
or = xor[0] | xor[1] | xor[2] | xor[3]; \
match_key = 0; \
if (or == 0) \
@@ -619,14 +673,14 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[8], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- xor[4] = pkt_key[4] ^ bkt_key[4]; \
- xor[5] = pkt_key[5] ^ bkt_key[5]; \
- xor[6] = pkt_key[6] ^ bkt_key[6]; \
- xor[7] = pkt_key[7] ^ bkt_key[7]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ xor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4]; \
+ xor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5]; \
+ xor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6]; \
+ xor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7]; \
or = xor[0] | xor[1] | xor[2] | xor[3] | \
xor[4] | xor[5] | xor[6] | xor[7]; \
match_key = 0; \
@@ -637,7 +691,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
\
default: \
match_key = 0; \
- if (memcmp(pkt_key, bkt_key, f->key_size) == 0) \
+ if (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0) \
match_key = 1; \
} \
}
@@ -685,38 +739,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
-{ \
- struct grinder *g10, *g11; \
- uint64_t sig10, sig11, bkt10_index, bkt11_index; \
- struct rte_mbuf *mbuf10, *mbuf11; \
- struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
- uint64_t bucket_mask = t->bucket_mask; \
- uint32_t signature_offset = t->signature_offset; \
- \
- mbuf10 = pkts[pkt10_index]; \
- sig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\
- bkt10_index = sig10 & bucket_mask; \
- bkt10 = &buckets[bkt10_index]; \
- \
- mbuf11 = pkts[pkt11_index]; \
- sig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\
- bkt11_index = sig11 & bucket_mask; \
- bkt11 = &buckets[bkt11_index]; \
- \
- rte_prefetch0(bkt10); \
- rte_prefetch0(bkt11); \
- \
- g10 = &g[pkt10_index]; \
- g10->sig = sig10; \
- g10->bkt = bkt10; \
- \
- g11 = &g[pkt11_index]; \
- g11->sig = sig11; \
- g11->bkt = bkt11; \
-}
-
-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index) \
+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
{ \
struct grinder *g10, *g11; \
uint64_t sig10, sig11, bkt10_index, bkt11_index; \
@@ -731,13 +754,13 @@ static int rte_table_hash_ext_lookup_unoptimized(
\
mbuf10 = pkts[pkt10_index]; \
key10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset); \
- sig10 = (uint64_t) f_hash(key10, key_size, seed); \
+ sig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed); \
bkt10_index = sig10 & bucket_mask; \
bkt10 = &buckets[bkt10_index]; \
\
mbuf11 = pkts[pkt11_index]; \
key11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset); \
- sig11 = (uint64_t) f_hash(key11, key_size, seed); \
+ sig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed); \
bkt11_index = sig11 & bucket_mask; \
bkt11 = &buckets[bkt11_index]; \
\
@@ -874,7 +897,7 @@ static int rte_table_hash_ext_lookup(
/* Cannot run the pipeline with less than 7 packets */
if (__builtin_popcountll(pkts_mask) < 7) {
status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 0);
+ pkts_mask, lookup_hit_mask, entries);
RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -
__builtin_popcountll(*lookup_hit_mask));
return status;
@@ -982,144 +1005,7 @@ static int rte_table_hash_ext_lookup(
uint64_t pkts_mask_out_slow = 0;
status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 0);
- pkts_mask_out |= pkts_mask_out_slow;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return status;
-}
-
-static int rte_table_hash_ext_lookup_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *t = (struct rte_table_hash *) table;
- struct grinder *g = t->grinders;
- uint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;
- uint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;
- uint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;
- int status = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_EXT_STATS_PKTS_IN_ADD(t, n_pkts_in);
-
- /* Cannot run the pipeline with less than 7 packets */
- if (__builtin_popcountll(pkts_mask) < 7) {
- status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 1);
- RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -
- __builtin_popcountll(*lookup_hit_mask));
- return status;
- }
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline feed */
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline feed */
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,
- pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index,
- pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,
- pkts_mask_out, entries);
- }
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Slow path */
- pkts_mask_match_many &= ~pkts_mask_out;
- if (pkts_mask_match_many) {
- uint64_t pkts_mask_out_slow = 0;
-
- status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 1);
+ pkts_mask_match_many, &pkts_mask_out_slow, entries);
pkts_mask_out |= pkts_mask_out_slow;
}
@@ -1152,14 +1038,3 @@ struct rte_table_ops rte_table_hash_ext_ops = {
.f_lookup = rte_table_hash_ext_lookup,
.f_stats = rte_table_hash_ext_stats_read,
};
-
-struct rte_table_ops rte_table_hash_ext_dosig_ops = {
- .f_create = rte_table_hash_ext_create,
- .f_free = rte_table_hash_ext_free,
- .f_add = rte_table_hash_ext_entry_add,
- .f_delete = rte_table_hash_ext_entry_delete,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_ext_lookup_dosig,
- .f_stats = rte_table_hash_ext_stats_read,
-};
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
index ce057b7..e6e7d98 100644
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 16
+#define KEY_SIZE 16
+
+#define KEYS_PER_BUCKET 4
#define RTE_BUCKET_ENTRY_VALID 0x1LLU
@@ -79,11 +81,9 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask[2];
rte_table_hash_op_hash f_hash;
@@ -99,17 +99,54 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key16_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return (a64[0] != (b64[0] & b_mask64[0])) ||
+ (a64[1] != (b64[1] & b_mask64[1]));
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+ dst64[1] = src64[1] & src_mask64[1];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params) {
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_buckets */
+ if ((params->n_buckets == 0) ||
+ (!rte_is_power_of_2(params->n_buckets))) {
+ RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
/* f_hash */
if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: f_hash function pointer is NULL\n", __func__);
+ RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
+ __func__);
return -EINVAL;
}
@@ -121,46 +158,67 @@ rte_table_hash_create_key16_lru(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key16_lru_params *p =
- (struct rte_table_hash_key16_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket,
- key_size, bucket_size_cl, total_size, i;
+ uint64_t bucket_size, total_size;
+ uint32_t n_buckets, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_16) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 16;
+
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + n_buckets *
- bucket_size_cl * RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +
+ KEYS_PER_BUCKET * entry_size);
+ total_size = sizeof(struct rte_table_hash) + n_buckets * bucket_size;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -212,19 +270,19 @@ rte_table_hash_entry_add_key16_lru(
uint64_t signature, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -238,13 +296,13 @@ rte_table_hash_entry_add_key16_lru(
/* Key is not present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -257,7 +315,7 @@ rte_table_hash_entry_add_key16_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
bucket->signature[pos] = signature;
- memcpy(bucket->key[pos], key, f->key_size);
+ keycpy(&bucket->key[pos], key, f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
*key_found = 0;
@@ -278,19 +336,19 @@ rte_table_hash_entry_delete_key16_lru(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = 0;
@@ -306,81 +364,71 @@ rte_table_hash_entry_delete_key16_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key16_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: f_hash function pointer is NULL\n", __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key16_ext(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key16_ext_params *p =
- (struct rte_table_hash_key16_ext_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size,
- bucket_size_cl, stack_size_cl, total_size, i;
+ uint64_t bucket_size, stack_size, total_size;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_16) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 16;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
total_size = sizeof(struct rte_table_hash) +
- ((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *
- RTE_CACHE_LINE_SIZE;
+ (p->n_buckets + n_buckets_ext) * bucket_size + stack_size;
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
- f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->n_buckets = p->n_buckets;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -388,10 +436,7 @@ rte_table_hash_create_key16_ext(void *params,
f->n_buckets_ext = n_buckets_ext;
f->stack_pos = n_buckets_ext;
f->stack = (uint32_t *)
- &f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
-
- for (i = 0; i < n_buckets_ext; i++)
- f->stack[i] = i;
+ &f->memory[(p->n_buckets + n_buckets_ext) * f->bucket_size];
if (p->key_mask != NULL) {
f->key_mask[0] = (((uint64_t *)p->key_mask)[0]);
@@ -401,6 +446,9 @@ rte_table_hash_create_key16_ext(void *params,
f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
}
+ for (i = 0; i < n_buckets_ext; i++)
+ f->stack[i] = i;
+
return f;
}
@@ -432,20 +480,20 @@ rte_table_hash_entry_add_key16_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (bucket = bucket0; bucket != NULL; bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -458,17 +506,17 @@ rte_table_hash_entry_add_key16_ext(
/* Key is not present in the bucket */
for (bucket_prev = NULL, bucket = bucket0; bucket != NULL;
- bucket_prev = bucket, bucket = bucket->next)
+ bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -487,7 +535,7 @@ rte_table_hash_entry_add_key16_ext(
bucket_prev->next_valid = 1;
bucket->signature[0] = signature;
- memcpy(bucket->key[0], key, f->key_size);
+ keycpy(&bucket->key[0], key, f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -509,7 +557,7 @@ rte_table_hash_entry_delete_key16_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_16 *)
&f->memory[bucket_index * f->bucket_size];
@@ -520,18 +568,17 @@ rte_table_hash_entry_delete_key16_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = 0;
*key_found = 1;
if (entry)
- memcpy(entry, bucket_data,
- f->entry_size);
+ memcpy(entry, bucket_data, f->entry_size);
if ((bucket->signature[0] == 0) &&
(bucket->signature[1] == 0) &&
@@ -558,26 +605,28 @@ rte_table_hash_entry_delete_key16_ext(
return 0;
}
-#define lookup_key16_cmp(key_in, bucket, pos) \
+#define lookup_key16_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4][2], or[4], signature[4]; \
+ uint64_t xor[4][2], or[4], signature[4], k[2]; \
\
+ k[0] = key_in[0] & f->key_mask[0]; \
+ k[1] = key_in[1] & f->key_mask[1]; \
signature[0] = (~bucket->signature[0]) & 1; \
signature[1] = (~bucket->signature[1]) & 1; \
signature[2] = (~bucket->signature[2]) & 1; \
signature[3] = (~bucket->signature[3]) & 1; \
\
- xor[0][0] = key_in[0] ^ bucket->key[0][0]; \
- xor[0][1] = key_in[1] ^ bucket->key[0][1]; \
+ xor[0][0] = k[0] ^ bucket->key[0][0]; \
+ xor[0][1] = k[1] ^ bucket->key[0][1]; \
\
- xor[1][0] = key_in[0] ^ bucket->key[1][0]; \
- xor[1][1] = key_in[1] ^ bucket->key[1][1]; \
+ xor[1][0] = k[0] ^ bucket->key[1][0]; \
+ xor[1][1] = k[1] ^ bucket->key[1][1]; \
\
- xor[2][0] = key_in[0] ^ bucket->key[2][0]; \
- xor[2][1] = key_in[1] ^ bucket->key[2][1]; \
+ xor[2][0] = k[0] ^ bucket->key[2][0]; \
+ xor[2][1] = k[1] ^ bucket->key[2][1]; \
\
- xor[3][0] = key_in[0] ^ bucket->key[3][0]; \
- xor[3][1] = key_in[1] ^ bucket->key[3][1]; \
+ xor[3][0] = k[0] ^ bucket->key[3][0]; \
+ xor[3][1] = k[1] ^ bucket->key[3][1]; \
\
or[0] = xor[0][0] | xor[0][1] | signature[0]; \
or[1] = xor[1][0] | xor[1][1] | signature[1]; \
@@ -610,30 +659,12 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
- uint64_t signature; \
- uint32_t bucket_index; \
- \
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
- bucket_index = signature & (f->n_buckets - 1); \
- bucket1 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket_index * f->bucket_size]; \
- rte_prefetch0(bucket1); \
- rte_prefetch0((void *)(((uintptr_t) bucket1) + RTE_CACHE_LINE_SIZE));\
-}
-
-#define lookup1_stage1_dosig(mbuf1, bucket1, f) \
-{ \
uint64_t *key; \
uint64_t signature = 0; \
uint32_t bucket_index; \
- uint64_t hash_key_buffer[2]; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\
- \
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- signature = f->f_hash(hash_key_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed); \
+ signature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed); \
\
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_16 *) \
@@ -648,14 +679,10 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket2, pos); \
+ lookup_key16_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -673,14 +700,10 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket2, pos); \
+ lookup_key16_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -703,15 +726,11 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket, pos); \
+ lookup_key16_cmp(key, bucket, pos, f); \
\
pkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -775,36 +794,12 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
- \
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
- bucket10 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
- rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
- \
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
- bucket11 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
- rte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\
-}
-
-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f) \
-{ \
uint64_t *key10, *key11; \
- uint64_t hash_offset_buffer[2]; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
\
key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset);\
- hash_offset_buffer[0] = key10[0] & f->key_mask[0]; \
- hash_offset_buffer[1] = key10[1] & f->key_mask[1]; \
- signature10 = f->f_hash(hash_offset_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed);\
+ signature10 = f->f_hash(key10, f->key_mask, KEY_SIZE, f->seed);\
bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_16 *) \
&f->memory[bucket10_index * f->bucket_size]; \
@@ -812,10 +807,7 @@ rte_table_hash_entry_delete_key16_ext(
rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
\
key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset);\
- hash_offset_buffer[0] = key11[0] & f->key_mask[0]; \
- hash_offset_buffer[1] = key11[1] & f->key_mask[1]; \
- signature11 = f->f_hash(hash_offset_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed);\
+ signature11 = f->f_hash(key11, f->key_mask, KEY_SIZE, f->seed);\
bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_16 *) \
&f->memory[bucket11_index * f->bucket_size]; \
@@ -829,19 +821,13 @@ rte_table_hash_entry_delete_key16_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask; \
uint64_t *key20, *key21; \
- uint64_t hash_key_buffer20[2]; \
- uint64_t hash_key_buffer21[2]; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_key_buffer20[0] = key20[0] & f->key_mask[0]; \
- hash_key_buffer20[1] = key20[1] & f->key_mask[1]; \
- hash_key_buffer21[0] = key21[0] & f->key_mask[0]; \
- hash_key_buffer21[1] = key21[1] & f->key_mask[1]; \
\
- lookup_key16_cmp(hash_key_buffer20, bucket20, pos20); \
- lookup_key16_cmp(hash_key_buffer21, bucket21, pos21); \
+ lookup_key16_cmp(key20, bucket20, pos20, f); \
+ lookup_key16_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -864,19 +850,13 @@ rte_table_hash_entry_delete_key16_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\
uint64_t *key20, *key21; \
- uint64_t hash_key_buffer20[2]; \
- uint64_t hash_key_buffer21[2]; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_key_buffer20[0] = key20[0] & f->key_mask[0]; \
- hash_key_buffer20[1] = key20[1] & f->key_mask[1]; \
- hash_key_buffer21[0] = key21[0] & f->key_mask[0]; \
- hash_key_buffer21[1] = key21[1] & f->key_mask[1]; \
\
- lookup_key16_cmp(hash_key_buffer20, bucket20, pos20); \
- lookup_key16_cmp(hash_key_buffer21, bucket21, pos21); \
+ lookup_key16_cmp(key20, bucket20, pos20, f); \
+ lookup_key16_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -916,6 +896,7 @@ rte_table_hash_lookup_key16_lru(
uint64_t pkts_mask_out = 0;
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
+
RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
/* Cannot run the pipeline with less than 5 packets */
@@ -932,8 +913,8 @@ rte_table_hash_lookup_key16_lru(
}
*lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f,
- n_pkts_in - __builtin_popcountll(pkts_mask_out));
+ RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
+ __builtin_popcountll(pkts_mask_out));
return 0;
}
@@ -1026,136 +1007,7 @@ rte_table_hash_lookup_key16_lru(
RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
__builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key16_lru() */
-
-static int
-rte_table_hash_lookup_key16_lru_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
-
- RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_16 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key16_lru_dosig() */
+} /* lookup LRU */
static int
rte_table_hash_lookup_key16_ext(
@@ -1175,6 +1027,7 @@ rte_table_hash_lookup_key16_ext(
uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
+
RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
/* Cannot run the pipeline with less than 5 packets */
@@ -1306,159 +1159,7 @@ rte_table_hash_lookup_key16_ext(
RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
__builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key16_ext() */
-
-static int
-rte_table_hash_lookup_key16_ext_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0, buckets_mask = 0;
- struct rte_bucket_4_16 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];
- uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
-
- RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_16 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask,
- buckets, keys, f);
- }
-
- goto grind_next_buckets;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
-grind_next_buckets:
- /* Grind next buckets */
- for ( ; buckets_mask; ) {
- uint64_t buckets_mask_next = 0;
-
- for ( ; buckets_mask; ) {
- uint64_t pkt_mask;
- uint32_t pkt_index;
-
- pkt_index = __builtin_ctzll(buckets_mask);
- pkt_mask = 1LLU << pkt_index;
- buckets_mask &= ~pkt_mask;
-
- lookup_grinder(pkt_index, buckets, keys, pkts_mask_out,
- entries, buckets_mask_next, f);
- }
-
- buckets_mask = buckets_mask_next;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key16_ext_dosig() */
+} /* lookup EXT */
static int
rte_table_hash_key16_stats_read(void *table, struct rte_table_stats *stats, int clear)
@@ -1485,15 +1186,6 @@ struct rte_table_ops rte_table_hash_key16_lru_ops = {
.f_stats = rte_table_hash_key16_stats_read,
};
-struct rte_table_ops rte_table_hash_key16_lru_dosig_ops = {
- .f_create = rte_table_hash_create_key16_lru,
- .f_free = rte_table_hash_free_key16_lru,
- .f_add = rte_table_hash_entry_add_key16_lru,
- .f_delete = rte_table_hash_entry_delete_key16_lru,
- .f_lookup = rte_table_hash_lookup_key16_lru_dosig,
- .f_stats = rte_table_hash_key16_stats_read,
-};
-
struct rte_table_ops rte_table_hash_key16_ext_ops = {
.f_create = rte_table_hash_create_key16_ext,
.f_free = rte_table_hash_free_key16_ext,
@@ -1504,12 +1196,3 @@ struct rte_table_ops rte_table_hash_key16_ext_ops = {
.f_lookup = rte_table_hash_lookup_key16_ext,
.f_stats = rte_table_hash_key16_stats_read,
};
-
-struct rte_table_ops rte_table_hash_key16_ext_dosig_ops = {
- .f_create = rte_table_hash_create_key16_ext,
- .f_free = rte_table_hash_free_key16_ext,
- .f_add = rte_table_hash_entry_add_key16_ext,
- .f_delete = rte_table_hash_entry_delete_key16_ext,
- .f_lookup = rte_table_hash_lookup_key16_ext_dosig,
- .f_stats = rte_table_hash_key16_stats_read,
-};
diff --git a/lib/librte_table/rte_table_hash_key32.c b/lib/librte_table/rte_table_hash_key32.c
index 31fe6fd..0664aa4 100644
--- a/lib/librte_table/rte_table_hash_key32.c
+++ b/lib/librte_table/rte_table_hash_key32.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 32
+#define KEY_SIZE 32
+
+#define KEYS_PER_BUCKET 4
#define RTE_BUCKET_ENTRY_VALID 0x1LLU
@@ -79,12 +81,11 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
+ uint64_t key_mask[4];
rte_table_hash_op_hash f_hash;
uint64_t seed;
@@ -98,10 +99,51 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key32_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return (a64[0] != (b64[0] & b_mask64[0])) ||
+ (a64[1] != (b64[1] & b_mask64[1])) ||
+ (a64[2] != (b64[2] & b_mask64[2])) ||
+ (a64[3] != (b64[3] & b_mask64[3]));
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+ dst64[1] = src64[1] & src_mask64[1];
+ dst64[2] = src64[2] & src_mask64[2];
+ dst64[3] = src64[3] & src_mask64[3];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params) {
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_buckets */
+ if ((params->n_buckets == 0) ||
+ (!rte_is_power_of_2(params->n_buckets))) {
+ RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -120,51 +162,83 @@ rte_table_hash_create_key32_lru(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key32_lru_params *p =
- (struct rte_table_hash_key32_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;
- uint32_t total_size, i;
+ uint64_t bucket_size, total_size;
+ uint32_t n_buckets, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
- ((sizeof(struct rte_bucket_4_32) % 64) != 0)) {
+ ((sizeof(struct rte_bucket_4_32) % 64) != 0))
return NULL;
- }
- n_entries_per_bucket = 4;
- key_size = 32;
+
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + n_buckets *
- bucket_size_cl * RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +
+ KEYS_PER_BUCKET * entry_size);
+ total_size = sizeof(struct rte_table_hash) + n_buckets * bucket_size;
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n", __func__,
- total_size);
+ "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
+ if (p->key_mask != NULL) {
+ f->key_mask[0] = ((uint64_t *)p->key_mask)[0];
+ f->key_mask[1] = ((uint64_t *)p->key_mask)[1];
+ f->key_mask[2] = ((uint64_t *)p->key_mask)[2];
+ f->key_mask[3] = ((uint64_t *)p->key_mask)[3];
+ } else {
+ f->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;
+ }
+
for (i = 0; i < n_buckets; i++) {
struct rte_bucket_4_32 *bucket;
@@ -204,7 +278,7 @@ rte_table_hash_entry_add_key32_lru(
uint64_t signature, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -213,10 +287,10 @@ rte_table_hash_entry_add_key32_lru(
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -230,13 +304,13 @@ rte_table_hash_entry_add_key32_lru(
/* Key is not present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -249,10 +323,10 @@ rte_table_hash_entry_add_key32_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
bucket->signature[pos] = signature;
- memcpy(bucket->key[pos], key, f->key_size);
+ keycpy(&bucket->key[pos], key, f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
- *key_found = 0;
+ *key_found = 0;
*entry_ptr = (void *) &bucket->data[pos * f->entry_size];
return 0;
@@ -270,7 +344,7 @@ rte_table_hash_entry_delete_key32_lru(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -279,10 +353,10 @@ rte_table_hash_entry_delete_key32_lru(
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = 0;
@@ -299,81 +373,72 @@ rte_table_hash_entry_delete_key32_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key32_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
- __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key32_ext(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key32_ext_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket;
- uint32_t key_size, bucket_size_cl, stack_size_cl, total_size, i;
+ uint64_t bucket_size, stack_size, total_size;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_32) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 32;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
total_size = sizeof(struct rte_table_hash) +
- ((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *
- RTE_CACHE_LINE_SIZE;
+ (p->n_buckets + n_buckets_ext) * bucket_size + stack_size;
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n", __func__,
- total_size);
+ "%s: Hash table %s memory footprint "
+ "is %" PRIu64" bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
- f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->n_buckets = p->n_buckets;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -381,7 +446,19 @@ rte_table_hash_create_key32_ext(void *params,
f->n_buckets_ext = n_buckets_ext;
f->stack_pos = n_buckets_ext;
f->stack = (uint32_t *)
- &f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
+ &f->memory[(p->n_buckets + n_buckets_ext) * f->bucket_size];
+
+ if (p->key_mask != NULL) {
+ f->key_mask[0] = (((uint64_t *)p->key_mask)[0]);
+ f->key_mask[1] = (((uint64_t *)p->key_mask)[1]);
+ f->key_mask[2] = (((uint64_t *)p->key_mask)[2]);
+ f->key_mask[3] = (((uint64_t *)p->key_mask)[3]);
+ } else {
+ f->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;
+ }
for (i = 0; i < n_buckets_ext; i++)
f->stack[i] = i;
@@ -417,7 +494,7 @@ rte_table_hash_entry_add_key32_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -427,10 +504,10 @@ rte_table_hash_entry_add_key32_ext(
for (bucket = bucket0; bucket != NULL; bucket = bucket->next) {
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -448,14 +525,14 @@ rte_table_hash_entry_add_key32_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -475,7 +552,7 @@ rte_table_hash_entry_add_key32_ext(
bucket_prev->next_valid = 1;
bucket->signature[0] = signature;
- memcpy(bucket->key[0], key, f->key_size);
+ keycpy(&bucket->key[0], key, f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -497,7 +574,7 @@ rte_table_hash_entry_delete_key32_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -508,24 +585,23 @@ rte_table_hash_entry_delete_key32_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = 0;
*key_found = 1;
if (entry)
- memcpy(entry, bucket_data,
- f->entry_size);
+ memcpy(entry, bucket_data, f->entry_size);
if ((bucket->signature[0] == 0) &&
- (bucket->signature[1] == 0) &&
- (bucket->signature[2] == 0) &&
- (bucket->signature[3] == 0) &&
- (bucket_prev != NULL)) {
+ (bucket->signature[1] == 0) &&
+ (bucket->signature[2] == 0) &&
+ (bucket->signature[3] == 0) &&
+ (bucket_prev != NULL)) {
bucket_prev->next = bucket->next;
bucket_prev->next_valid =
bucket->next_valid;
@@ -546,34 +622,39 @@ rte_table_hash_entry_delete_key32_ext(
return 0;
}
-#define lookup_key32_cmp(key_in, bucket, pos) \
+#define lookup_key32_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4][4], or[4], signature[4]; \
+ uint64_t xor[4][4], or[4], signature[4], k[4]; \
+ \
+ k[0] = key_in[0] & f->key_mask[0]; \
+ k[1] = key_in[1] & f->key_mask[1]; \
+ k[2] = key_in[2] & f->key_mask[2]; \
+ k[3] = key_in[3] & f->key_mask[3]; \
\
signature[0] = ((~bucket->signature[0]) & 1); \
signature[1] = ((~bucket->signature[1]) & 1); \
signature[2] = ((~bucket->signature[2]) & 1); \
signature[3] = ((~bucket->signature[3]) & 1); \
\
- xor[0][0] = key_in[0] ^ bucket->key[0][0]; \
- xor[0][1] = key_in[1] ^ bucket->key[0][1]; \
- xor[0][2] = key_in[2] ^ bucket->key[0][2]; \
- xor[0][3] = key_in[3] ^ bucket->key[0][3]; \
+ xor[0][0] = k[0] ^ bucket->key[0][0]; \
+ xor[0][1] = k[1] ^ bucket->key[0][1]; \
+ xor[0][2] = k[2] ^ bucket->key[0][2]; \
+ xor[0][3] = k[3] ^ bucket->key[0][3]; \
\
- xor[1][0] = key_in[0] ^ bucket->key[1][0]; \
- xor[1][1] = key_in[1] ^ bucket->key[1][1]; \
- xor[1][2] = key_in[2] ^ bucket->key[1][2]; \
- xor[1][3] = key_in[3] ^ bucket->key[1][3]; \
+ xor[1][0] = k[0] ^ bucket->key[1][0]; \
+ xor[1][1] = k[1] ^ bucket->key[1][1]; \
+ xor[1][2] = k[2] ^ bucket->key[1][2]; \
+ xor[1][3] = k[3] ^ bucket->key[1][3]; \
\
- xor[2][0] = key_in[0] ^ bucket->key[2][0]; \
- xor[2][1] = key_in[1] ^ bucket->key[2][1]; \
- xor[2][2] = key_in[2] ^ bucket->key[2][2]; \
- xor[2][3] = key_in[3] ^ bucket->key[2][3]; \
+ xor[2][0] = k[0] ^ bucket->key[2][0]; \
+ xor[2][1] = k[1] ^ bucket->key[2][1]; \
+ xor[2][2] = k[2] ^ bucket->key[2][2]; \
+ xor[2][3] = k[3] ^ bucket->key[2][3]; \
\
- xor[3][0] = key_in[0] ^ bucket->key[3][0]; \
- xor[3][1] = key_in[1] ^ bucket->key[3][1]; \
- xor[3][2] = key_in[2] ^ bucket->key[3][2]; \
- xor[3][3] = key_in[3] ^ bucket->key[3][3]; \
+ xor[3][0] = k[0] ^ bucket->key[3][0]; \
+ xor[3][1] = k[1] ^ bucket->key[3][1]; \
+ xor[3][2] = k[2] ^ bucket->key[3][2]; \
+ xor[3][3] = k[3] ^ bucket->key[3][3]; \
\
or[0] = xor[0][0] | xor[0][1] | xor[0][2] | xor[0][3] | signature[0];\
or[1] = xor[1][0] | xor[1][1] | xor[1][2] | xor[1][3] | signature[1];\
@@ -604,12 +685,15 @@ rte_table_hash_entry_delete_key32_ext(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf0, key_offset));\
}
-#define lookup1_stage1(mbuf1, bucket1, f) \
+#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
+ uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
\
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
+ key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset); \
+ signature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed); \
+ \
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_32 *) \
&f->memory[bucket_index * f->bucket_size]; \
@@ -627,8 +711,7 @@ rte_table_hash_entry_delete_key32_ext(
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- \
- lookup_key32_cmp(key, bucket2, pos); \
+ lookup_key32_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -649,8 +732,7 @@ rte_table_hash_entry_delete_key32_ext(
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- \
- lookup_key32_cmp(key, bucket2, pos); \
+ lookup_key32_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -678,7 +760,7 @@ rte_table_hash_entry_delete_key32_ext(
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
\
- lookup_key32_cmp(key, bucket, pos); \
+ lookup_key32_cmp(key, bucket, pos, f); \
\
pkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -745,22 +827,27 @@ rte_table_hash_entry_delete_key32_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
+ uint64_t *key10, *key11; \
+ uint64_t signature10, signature11; \
+ uint32_t bucket10_index, bucket11_index; \
\
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
+ key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset); \
+ signature10 = f->f_hash(key10, f->key_mask, KEY_SIZE, f->seed); \
+ \
+ bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_32 *) \
&f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
+ rte_prefetch0(bucket10); \
rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
rte_prefetch0((void *)(((uintptr_t) bucket10) + 2 * RTE_CACHE_LINE_SIZE));\
\
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
+ key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset); \
+ signature11 = f->f_hash(key11, f->key_mask, KEY_SIZE, f->seed);\
+ \
+ bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_32 *) \
&f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
+ rte_prefetch0(bucket11); \
rte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\
rte_prefetch0((void *)(((uintptr_t) bucket11) + 2 * RTE_CACHE_LINE_SIZE));\
}
@@ -776,8 +863,8 @@ rte_table_hash_entry_delete_key32_ext(
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
\
- lookup_key32_cmp(key20, bucket20, pos20); \
- lookup_key32_cmp(key21, bucket21, pos21); \
+ lookup_key32_cmp(key20, bucket20, pos20, f); \
+ lookup_key32_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -805,8 +892,8 @@ rte_table_hash_entry_delete_key32_ext(
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
\
- lookup_key32_cmp(key20, bucket20, pos20); \
- lookup_key32_cmp(key21, bucket21, pos21); \
+ lookup_key32_cmp(key20, bucket20, pos20, f); \
+ lookup_key32_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
index 5f0c656..68a9a95 100644
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 8
+#define KEY_SIZE 8
+
+#define KEYS_PER_BUCKET 4
#ifdef RTE_TABLE_STATS_COLLECT
@@ -76,11 +78,9 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask;
rte_table_hash_op_hash f_hash;
@@ -96,10 +96,45 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return a64[0] != (b64[0] & b_mask64[0]);
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params) {
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_buckets */
+ if ((params->n_buckets == 0) ||
+ (!rte_is_power_of_2(params->n_buckets))) {
+ RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -116,47 +151,68 @@ check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {
static void *
rte_table_hash_create_key8_lru(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_key8_lru_params *p =
- (struct rte_table_hash_key8_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;
- uint32_t total_size, i;
+ uint64_t bucket_size, total_size;
+ uint32_t n_buckets, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
- ((sizeof(struct rte_bucket_4_8) % 64) != 0)) {
+ ((sizeof(struct rte_bucket_4_8) % 64) != 0))
return NULL;
- }
- n_entries_per_bucket = 4;
- key_size = 8;
+
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *
- entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + n_buckets *
- bucket_size_cl * RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +
+ KEYS_PER_BUCKET * entry_size);
+ total_size = sizeof(struct rte_table_hash) + n_buckets * bucket_size;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -205,7 +261,7 @@ rte_table_hash_entry_add_key8_lru(
uint64_t signature, mask, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -213,10 +269,10 @@ rte_table_hash_entry_add_key8_lru(
/* Key is present in the bucket */
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -235,7 +291,7 @@ rte_table_hash_entry_add_key8_lru(
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature |= mask;
- bucket->key[i] = *((uint64_t *) key);
+ keycpy(&bucket->key[i], key, &f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -247,10 +303,10 @@ rte_table_hash_entry_add_key8_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
- bucket->key[pos] = *((uint64_t *) key);
+ keycpy(&bucket->key[pos], key, &f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
- *key_found = 0;
+ *key_found = 0;
*entry_ptr = (void *) &bucket->data[pos * f->entry_size];
return 0;
@@ -268,7 +324,7 @@ rte_table_hash_entry_delete_key8_lru(
uint64_t signature, mask;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -276,10 +332,10 @@ rte_table_hash_entry_delete_key8_lru(
/* Key is present in the bucket */
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature &= ~mask;
@@ -296,79 +352,71 @@ rte_table_hash_entry_delete_key8_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key8_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
- __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key8_ext(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_key8_ext_params *p =
- (struct rte_table_hash_key8_ext_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size;
- uint32_t bucket_size_cl, stack_size_cl, total_size, i;
+ uint64_t bucket_size, stack_size, total_size;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_8) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 8;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *
- entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + ((n_buckets +
- n_buckets_ext) * bucket_size_cl + stack_size_cl) *
- RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
+ total_size = sizeof(struct rte_table_hash) +
+ (p->n_buckets + n_buckets_ext) * bucket_size + stack_size;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
- f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->n_buckets = p->n_buckets;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -376,7 +424,7 @@ rte_table_hash_create_key8_ext(void *params, int socket_id, uint32_t entry_size)
f->n_buckets_ext = n_buckets_ext;
f->stack_pos = n_buckets_ext;
f->stack = (uint32_t *)
- &f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
+ &f->memory[(p->n_buckets + n_buckets_ext) * f->bucket_size];
if (p->key_mask != NULL)
f->key_mask = ((uint64_t *)p->key_mask)[0];
@@ -417,7 +465,7 @@ rte_table_hash_entry_add_key8_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -428,10 +476,10 @@ rte_table_hash_entry_add_key8_ext(
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -456,7 +504,7 @@ rte_table_hash_entry_add_key8_ext(
f->entry_size];
bucket->signature |= mask;
- bucket->key[i] = *((uint64_t *) key);
+ keycpy(&bucket->key[i], key, &f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -476,7 +524,7 @@ rte_table_hash_entry_add_key8_ext(
bucket_prev->next_valid = 1;
bucket->signature = 1;
- bucket->key[0] = *((uint64_t *) key);
+ keycpy(&bucket->key[0], key, &f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -498,7 +546,7 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -510,10 +558,10 @@ rte_table_hash_entry_delete_key8_ext(
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -546,16 +594,17 @@ rte_table_hash_entry_delete_key8_ext(
return 0;
}
-#define lookup_key8_cmp(key_in, bucket, pos) \
+#define lookup_key8_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4], signature; \
+ uint64_t xor[4], signature, k; \
\
signature = ~bucket->signature; \
\
- xor[0] = (key_in[0] ^ bucket->key[0]) | (signature & 1);\
- xor[1] = (key_in[0] ^ bucket->key[1]) | (signature & 2);\
- xor[2] = (key_in[0] ^ bucket->key[2]) | (signature & 4);\
- xor[3] = (key_in[0] ^ bucket->key[3]) | (signature & 8);\
+ k = key_in[0] & f->key_mask; \
+ xor[0] = (k ^ bucket->key[0]) | (signature & 1); \
+ xor[1] = (k ^ bucket->key[1]) | (signature & 2); \
+ xor[2] = (k ^ bucket->key[2]) | (signature & 4); \
+ xor[3] = (k ^ bucket->key[3]) | (signature & 8); \
\
pos = 4; \
if (xor[0] == 0) \
@@ -583,27 +632,12 @@ rte_table_hash_entry_delete_key8_ext(
#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
- uint64_t signature; \
- uint32_t bucket_index; \
- \
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
- bucket_index = signature & (f->n_buckets - 1); \
- bucket1 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket_index * f->bucket_size]; \
- rte_prefetch0(bucket1); \
-}
-
-#define lookup1_stage1_dosig(mbuf1, bucket1, f) \
-{ \
uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\
- hash_key_buffer = *key & f->key_mask; \
- signature = f->f_hash(&hash_key_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed); \
+ signature = f->f_hash(key, &f->key_mask, KEY_SIZE, f->seed); \
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_8 *) \
&f->memory[bucket_index * f->bucket_size]; \
@@ -617,12 +651,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer = key[0] & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket2, pos); \
+ lookup_key8_cmp(key, bucket2, pos, f); \
\
pkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -641,12 +672,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer = *key & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket2, pos); \
+ lookup_key8_cmp(key, bucket2, pos, f); \
\
pkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -670,13 +698,10 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
- hash_key_buffer = (*key) & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket, pos); \
+ lookup_key8_cmp(key, bucket, pos, f); \
\
pkt_mask = ((bucket->signature >> pos) & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -738,29 +763,9 @@ rte_table_hash_entry_delete_key8_ext(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
-{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
- \
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
- bucket10 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
- \
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
- bucket11 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
-}
-
-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f)\
+#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\
{ \
uint64_t *key10, *key11; \
- uint64_t hash_offset_buffer10; \
- uint64_t hash_offset_buffer11; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
rte_table_hash_op_hash f_hash = f->f_hash; \
@@ -769,18 +774,14 @@ rte_table_hash_entry_delete_key8_ext(
\
key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, key_offset);\
key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, key_offset);\
- hash_offset_buffer10 = *key10 & f->key_mask; \
- hash_offset_buffer11 = *key11 & f->key_mask; \
\
- signature10 = f_hash(&hash_offset_buffer10, \
- RTE_TABLE_HASH_KEY_SIZE, seed); \
+ signature10 = f_hash(key10, &f->key_mask, KEY_SIZE, seed); \
bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_8 *) \
&f->memory[bucket10_index * f->bucket_size]; \
rte_prefetch0(bucket10); \
\
- signature11 = f_hash(&hash_offset_buffer11, \
- RTE_TABLE_HASH_KEY_SIZE, seed); \
+ signature11 = f_hash(key11, &f->key_mask, KEY_SIZE, seed); \
bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_8 *) \
&f->memory[bucket11_index * f->bucket_size]; \
@@ -793,17 +794,13 @@ rte_table_hash_entry_delete_key8_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask; \
uint64_t *key20, *key21; \
- uint64_t hash_offset_buffer20; \
- uint64_t hash_offset_buffer21; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_offset_buffer20 = *key20 & f->key_mask; \
- hash_offset_buffer21 = *key21 & f->key_mask; \
\
- lookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\
- lookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\
+ lookup_key8_cmp(key20, bucket20, pos20, f); \
+ lookup_key8_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\
pkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\
@@ -826,17 +823,13 @@ rte_table_hash_entry_delete_key8_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\
uint64_t *key20, *key21; \
- uint64_t hash_offset_buffer20; \
- uint64_t hash_offset_buffer21; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_offset_buffer20 = *key20 & f->key_mask; \
- hash_offset_buffer21 = *key21 & f->key_mask; \
\
- lookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\
- lookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\
+ lookup_key8_cmp(key20, bucket20, pos20, f); \
+ lookup_key8_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\
pkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\
@@ -871,8 +864,8 @@ rte_table_hash_lookup_key8_lru(
struct rte_table_hash *f = (struct rte_table_hash *) table;
struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index,
- pkt11_index, pkt20_index, pkt21_index;
+ uint32_t pkt00_index, pkt01_index, pkt10_index;
+ uint32_t pkt11_index, pkt20_index, pkt21_index;
uint64_t pkts_mask_out = 0;
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
@@ -888,7 +881,7 @@ rte_table_hash_lookup_key8_lru(
lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
lookup1_stage1(mbuf, bucket, f);
lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
+ pkts_mask_out, entries, f);
}
*lookup_hit_mask = pkts_mask_out;
@@ -984,133 +977,7 @@ rte_table_hash_lookup_key8_lru(
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key8_lru() */
-
-static int
-rte_table_hash_lookup_key8_lru_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_8 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key8_lru_dosig() */
+} /* lookup LRU */
static int
rte_table_hash_lookup_key8_ext(
@@ -1142,8 +1009,8 @@ rte_table_hash_lookup_key8_ext(
lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
lookup1_stage1(mbuf, bucket, f);
lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask, buckets,
- keys, f);
+ pkts_mask_out, entries, buckets_mask,
+ buckets, keys, f);
}
goto grind_next_buckets;
@@ -1260,157 +1127,7 @@ rte_table_hash_lookup_key8_ext(
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key8_ext() */
-
-static int
-rte_table_hash_lookup_key8_ext_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0, buckets_mask = 0;
- struct rte_bucket_4_8 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];
- uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_8 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask,
- buckets, keys, f);
- }
-
- goto grind_next_buckets;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
-grind_next_buckets:
- /* Grind next buckets */
- for ( ; buckets_mask; ) {
- uint64_t buckets_mask_next = 0;
-
- for ( ; buckets_mask; ) {
- uint64_t pkt_mask;
- uint32_t pkt_index;
-
- pkt_index = __builtin_ctzll(buckets_mask);
- pkt_mask = 1LLU << pkt_index;
- buckets_mask &= ~pkt_mask;
-
- lookup_grinder(pkt_index, buckets, keys, pkts_mask_out,
- entries, buckets_mask_next, f);
- }
-
- buckets_mask = buckets_mask_next;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key8_dosig_ext() */
+} /* lookup EXT */
static int
rte_table_hash_key8_stats_read(void *table, struct rte_table_stats *stats, int clear)
@@ -1437,17 +1154,6 @@ struct rte_table_ops rte_table_hash_key8_lru_ops = {
.f_stats = rte_table_hash_key8_stats_read,
};
-struct rte_table_ops rte_table_hash_key8_lru_dosig_ops = {
- .f_create = rte_table_hash_create_key8_lru,
- .f_free = rte_table_hash_free_key8_lru,
- .f_add = rte_table_hash_entry_add_key8_lru,
- .f_delete = rte_table_hash_entry_delete_key8_lru,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lookup_key8_lru_dosig,
- .f_stats = rte_table_hash_key8_stats_read,
-};
-
struct rte_table_ops rte_table_hash_key8_ext_ops = {
.f_create = rte_table_hash_create_key8_ext,
.f_free = rte_table_hash_free_key8_ext,
@@ -1458,14 +1164,3 @@ struct rte_table_ops rte_table_hash_key8_ext_ops = {
.f_lookup = rte_table_hash_lookup_key8_ext,
.f_stats = rte_table_hash_key8_stats_read,
};
-
-struct rte_table_ops rte_table_hash_key8_ext_dosig_ops = {
- .f_create = rte_table_hash_create_key8_ext,
- .f_free = rte_table_hash_free_key8_ext,
- .f_add = rte_table_hash_entry_add_key8_ext,
- .f_delete = rte_table_hash_entry_delete_key8_ext,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lookup_key8_ext_dosig,
- .f_stats = rte_table_hash_key8_stats_read,
-};
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
index 5a4864e..139b8fb 100644
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
@@ -86,7 +86,6 @@ struct rte_table_hash {
uint32_t n_buckets;
rte_table_hash_op_hash f_hash;
uint64_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
/* Internal */
@@ -99,6 +98,7 @@ struct rte_table_hash {
struct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];
/* Tables */
+ uint64_t *key_mask;
struct bucket *buckets;
uint8_t *key_mem;
uint8_t *data_mem;
@@ -109,12 +109,39 @@ struct rte_table_hash {
};
static int
-check_params_create(struct rte_table_hash_lru_params *params)
+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)
{
- uint32_t n_buckets_min;
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ if (a64[i] != (b64[i] & b_mask64[i]))
+ return 1;
+
+ return 0;
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ dst64[i] = src64[i] & src_mask64[i];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
/* key_size */
- if ((params->key_size == 0) ||
+ if ((params->key_size < sizeof(uint64_t)) ||
(!rte_is_power_of_2(params->key_size))) {
RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
return -EINVAL;
@@ -128,10 +155,8 @@ check_params_create(struct rte_table_hash_lru_params *params)
}
/* n_buckets */
- n_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;
if ((params->n_buckets == 0) ||
- (!rte_is_power_of_2(params->n_keys)) ||
- (params->n_buckets < n_buckets_min)) {
+ (!rte_is_power_of_2(params->n_keys))) {
RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -148,13 +173,13 @@ check_params_create(struct rte_table_hash_lru_params *params)
static void *
rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_lru_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *t;
- uint32_t total_size, table_meta_sz;
- uint32_t bucket_sz, key_sz, key_stack_sz, data_sz;
- uint32_t bucket_offset, key_offset, key_stack_offset, data_offset;
- uint32_t i;
+ uint64_t table_meta_sz, key_mask_sz, bucket_sz, key_sz, key_stack_sz;
+ uint64_t data_sz, total_size;
+ uint64_t key_mask_offset, bucket_offset, key_offset, key_stack_offset;
+ uint64_t data_offset;
+ uint32_t n_buckets, i;
/* Check input parameters */
if ((check_params_create(p) != 0) ||
@@ -164,33 +189,65 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
return NULL;
}
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
+
/* Memory allocation */
table_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));
- bucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));
+ key_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);
+ bucket_sz = RTE_CACHE_LINE_ROUNDUP(n_buckets * sizeof(struct bucket));
key_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);
key_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));
data_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
- total_size = table_meta_sz + bucket_sz + key_sz + key_stack_sz +
- data_sz;
+ total_size = table_meta_sz + key_mask_sz + bucket_sz + key_sz +
+ key_stack_sz + data_sz;
- t = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE,
+ "%s: Cannot allocate %" PRIu64 " bytes for hash "
+ "table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ t = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (t == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %" PRIu64 " bytes for hash "
+ "table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table memory footprint is "
- "%u bytes\n", __func__, p->key_size, total_size);
+ RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table %s memory footprint"
+ " is %" PRIu64 " bytes\n",
+ __func__, p->key_size, p->name, total_size);
/* Memory initialization */
t->key_size = p->key_size;
t->entry_size = entry_size;
t->n_keys = p->n_keys;
- t->n_buckets = p->n_buckets;
+ t->n_buckets = n_buckets;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
/* Internal */
@@ -199,16 +256,24 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
t->data_size_shl = __builtin_ctzl(entry_size);
/* Tables */
- bucket_offset = 0;
+ key_mask_offset = 0;
+ bucket_offset = key_mask_offset + key_mask_sz;
key_offset = bucket_offset + bucket_sz;
key_stack_offset = key_offset + key_sz;
data_offset = key_stack_offset + key_stack_sz;
+ t->key_mask = (uint64_t *) &t->memory[key_mask_offset];
t->buckets = (struct bucket *) &t->memory[bucket_offset];
t->key_mem = &t->memory[key_offset];
t->key_stack = (uint32_t *) &t->memory[key_stack_offset];
t->data_mem = &t->memory[data_offset];
+ /* Key mask */
+ if (p->key_mask == NULL)
+ memset(t->key_mask, 0xFF, p->key_size);
+ else
+ memcpy(t->key_mask, p->key_mask, p->key_size);
+
/* Key stack */
for (i = 0; i < t->n_keys; i++)
t->key_stack[i] = t->n_keys - 1 - i;
@@ -246,7 +311,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -258,8 +323,8 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key, t->key_size)
- == 0)) {
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
+ t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -292,7 +357,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
bkt->sig[i] = (uint16_t) sig;
bkt->key_pos[i] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
lru_update(bkt, i);
@@ -311,7 +376,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint8_t *data = &t->data_mem[bkt_key_index << t->data_size_shl];
bkt->sig[pos] = (uint16_t) sig;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
lru_update(bkt, pos);
@@ -330,7 +395,7 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -343,14 +408,15 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,
t->key_size_shl];
if ((sig == bkt_sig) &&
- (memcmp(key, bkt_key, t->key_size) == 0)) {
+ (keycmp(bkt_key, key, t->key_mask, t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
bkt->sig[i] = 0;
t->key_stack[t->key_stack_tos++] = bkt_key_index;
*key_found = 1;
- memcpy(entry, data, t->entry_size);
+ if (entry)
+ memcpy(entry, data, t->entry_size);
return 0;
}
}
@@ -365,8 +431,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
- void **entries,
- int dosig)
+ void **entries)
{
struct rte_table_hash *t = (struct rte_table_hash *) table;
uint64_t pkts_mask_out = 0;
@@ -387,11 +452,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- if (dosig)
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
- else
- sig = RTE_MBUF_METADATA_UINT32(pkt,
- t->signature_offset);
+ sig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
@@ -404,7 +465,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -502,74 +563,75 @@ static int rte_table_hash_lru_lookup_unoptimized(
match_pos = (LUT_MATCH_POS >> (mask_all << 1)) & 3; \
}
-#define lookup_cmp_key(mbuf, key, match_key, f) \
-{ \
+#define lookup_cmp_key(mbuf, key, match_key, f) \
+{ \
uint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\
- uint64_t *bkt_key = (uint64_t *) key; \
- \
- switch (f->key_size) { \
- case 8: \
- { \
- uint64_t xor = pkt_key[0] ^ bkt_key[0]; \
- match_key = 0; \
- if (xor == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 16: \
- { \
- uint64_t xor[2], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- or = xor[0] | xor[1]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 32: \
- { \
- uint64_t xor[4], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- or = xor[0] | xor[1] | xor[2] | xor[3]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 64: \
- { \
- uint64_t xor[8], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- xor[4] = pkt_key[4] ^ bkt_key[4]; \
- xor[5] = pkt_key[5] ^ bkt_key[5]; \
- xor[6] = pkt_key[6] ^ bkt_key[6]; \
- xor[7] = pkt_key[7] ^ bkt_key[7]; \
- or = xor[0] | xor[1] | xor[2] | xor[3] | \
- xor[4] | xor[5] | xor[6] | xor[7]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- default: \
- match_key = 0; \
- if (memcmp(pkt_key, bkt_key, f->key_size) == 0) \
- match_key = 1; \
- } \
+ uint64_t *bkt_key = (uint64_t *) key; \
+ uint64_t *key_mask = f->key_mask; \
+ \
+ switch (f->key_size) { \
+ case 8: \
+ { \
+ uint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ match_key = 0; \
+ if (xor == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 16: \
+ { \
+ uint64_t xor[2], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ or = xor[0] | xor[1]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 32: \
+ { \
+ uint64_t xor[4], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ or = xor[0] | xor[1] | xor[2] | xor[3]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 64: \
+ { \
+ uint64_t xor[8], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ xor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4]; \
+ xor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5]; \
+ xor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6]; \
+ xor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7]; \
+ or = xor[0] | xor[1] | xor[2] | xor[3] | \
+ xor[4] | xor[5] | xor[6] | xor[7]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ default: \
+ match_key = 0; \
+ if (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0) \
+ match_key = 1; \
+ } \
}
#define lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index)\
@@ -616,38 +678,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
-{ \
- struct grinder *g10, *g11; \
- uint64_t sig10, sig11, bkt10_index, bkt11_index; \
- struct rte_mbuf *mbuf10, *mbuf11; \
- struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
- uint64_t bucket_mask = t->bucket_mask; \
- uint32_t signature_offset = t->signature_offset; \
- \
- mbuf10 = pkts[pkt10_index]; \
- sig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\
- bkt10_index = sig10 & bucket_mask; \
- bkt10 = &buckets[bkt10_index]; \
- \
- mbuf11 = pkts[pkt11_index]; \
- sig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\
- bkt11_index = sig11 & bucket_mask; \
- bkt11 = &buckets[bkt11_index]; \
- \
- rte_prefetch0(bkt10); \
- rte_prefetch0(bkt11); \
- \
- g10 = &g[pkt10_index]; \
- g10->sig = sig10; \
- g10->bkt = bkt10; \
- \
- g11 = &g[pkt11_index]; \
- g11->sig = sig11; \
- g11->bkt = bkt11; \
-}
-
-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index)\
+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index)\
{ \
struct grinder *g10, *g11; \
uint64_t sig10, sig11, bkt10_index, bkt11_index; \
@@ -662,13 +693,13 @@ static int rte_table_hash_lru_lookup_unoptimized(
\
mbuf10 = pkts[pkt10_index]; \
key10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset);\
- sig10 = (uint64_t) f_hash(key10, key_size, seed); \
+ sig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed);\
bkt10_index = sig10 & bucket_mask; \
bkt10 = &buckets[bkt10_index]; \
\
mbuf11 = pkts[pkt11_index]; \
key11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset);\
- sig11 = (uint64_t) f_hash(key11, key_size, seed); \
+ sig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed);\
bkt11_index = sig11 & bucket_mask; \
bkt11 = &buckets[bkt11_index]; \
\
@@ -819,7 +850,7 @@ static int rte_table_hash_lru_lookup(
/* Cannot run the pipeline with less than 7 packets */
if (__builtin_popcountll(pkts_mask) < 7)
return rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 0);
+ pkts_mask, lookup_hit_mask, entries);
/* Pipeline stage 0 */
lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
@@ -923,140 +954,7 @@ static int rte_table_hash_lru_lookup(
uint64_t pkts_mask_out_slow = 0;
status = rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 0);
- pkts_mask_out |= pkts_mask_out_slow;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_LRU_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return status;
-}
-
-static int rte_table_hash_lru_lookup_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *t = (struct rte_table_hash *) table;
- struct grinder *g = t->grinders;
- uint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;
- uint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;
- uint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;
- int status = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_LRU_STATS_PKTS_IN_ADD(t, n_pkts_in);
-
- /* Cannot run the pipeline with less than 7 packets */
- if (__builtin_popcountll(pkts_mask) < 7)
- return rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 1);
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline feed */
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline feed */
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,
- pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index,
- pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,
- pkts_mask_out, entries);
- }
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Slow path */
- pkts_mask_match_many &= ~pkts_mask_out;
- if (pkts_mask_match_many) {
- uint64_t pkts_mask_out_slow = 0;
-
- status = rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 1);
+ pkts_mask_match_many, &pkts_mask_out_slow, entries);
pkts_mask_out |= pkts_mask_out_slow;
}
@@ -1089,14 +987,3 @@ struct rte_table_ops rte_table_hash_lru_ops = {
.f_lookup = rte_table_hash_lru_lookup,
.f_stats = rte_table_hash_lru_stats_read,
};
-
-struct rte_table_ops rte_table_hash_lru_dosig_ops = {
- .f_create = rte_table_hash_lru_create,
- .f_free = rte_table_hash_lru_free,
- .f_add = rte_table_hash_lru_entry_add,
- .f_delete = rte_table_hash_lru_entry_delete,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lru_lookup_dosig,
- .f_stats = rte_table_hash_lru_stats_read,
-};
diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map
index e1eaa27..f03fc3f 100644
--- a/lib/librte_table/rte_table_version.map
+++ b/lib/librte_table/rte_table_version.map
@@ -28,7 +28,7 @@ DPDK_2.2 {
rte_table_hash_key16_ext_dosig_ops;
rte_table_hash_key16_lru_dosig_ops;
-};
+} DPDK_2.0;
DPDK_16.07 {
global:
@@ -36,3 +36,24 @@ DPDK_16.07 {
rte_table_hash_cuckoo_dosig_ops;
} DPDK_2.0;
+
+DPDK_17.11 {
+ global:
+
+ rte_table_acl_ops;
+ rte_table_array_ops;
+ rte_table_hash_ext_ops;
+ rte_table_hash_key8_ext_ops;
+ rte_table_hash_key16_ext_ops;
+ rte_table_hash_key32_ext_ops;
+ rte_table_hash_lru_ops;
+ rte_table_hash_key8_lru_ops;
+ rte_table_hash_key16_lru_ops;
+ rte_table_hash_key32_lru_ops;
+ rte_table_hash_cuckoo_ops;
+ rte_table_lpm_ipv6_ops;
+ rte_table_lpm_ops;
+ rte_table_stub_ops;
+
+ local: *;
+};
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH V2 2/5] test: update due to api changes in librte_table
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 0/5] " Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 1/5] " Cristian Dumitrescu
@ 2017-10-10 11:18 ` Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 3/5] test-pipeline: " Cristian Dumitrescu
` (3 subsequent siblings)
5 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-10 11:18 UTC (permalink / raw)
To: dev
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
Changes in V2:
- None
test/test/test_table.c | 1 +
test/test/test_table.h | 3 +-
test/test/test_table_combined.c | 140 +++++++++++++++++--------------------
test/test/test_table_tables.c | 148 +++++++++++++++++-----------------------
4 files changed, 128 insertions(+), 164 deletions(-)
diff --git a/test/test/test_table.c b/test/test/test_table.c
index 9e9eed8..db7d4e6 100644
--- a/test/test/test_table.c
+++ b/test/test/test_table.c
@@ -72,6 +72,7 @@ static void app_init_rings(void);
static void app_init_mbuf_pools(void);
uint64_t pipeline_test_hash(void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed)
{
diff --git a/test/test/test_table.h b/test/test/test_table.h
index 84d1845..8c1df33 100644
--- a/test/test/test_table.h
+++ b/test/test/test_table.h
@@ -94,7 +94,7 @@
APP_METADATA_OFFSET(32)); \
k32 = (uint32_t *) key; \
k32[0] = (value); \
- *signature = pipeline_test_hash(key, 0, 0); \
+ *signature = pipeline_test_hash(key, NULL, 0, 0); \
rte_ring_enqueue((ring), m); \
} while (0)
@@ -131,6 +131,7 @@
/* Function definitions */
uint64_t pipeline_test_hash(
void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed);
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index a2d19a1..417bc42 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -200,8 +200,6 @@ test_table_type(struct rte_table_ops *table_ops, void *table_args,
return -CHECK_TABLE_CONSISTENCY;
}
-
-
/* Flow test - All hits */
if (table_packets->n_hit_packets) {
for (i = 0; i < table_packets->n_hit_packets; i++)
@@ -248,7 +246,6 @@ test_table_type(struct rte_table_ops *table_ops, void *table_args,
VERIFY_TRAFFIC(RING_TX, table_packets->n_miss_packets, 0);
}
-
/* Change table entry action */
printf("Change entry action\n");
table_entry.table_id = ring_out_2_id;
@@ -441,12 +438,15 @@ test_table_hash8lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key8_lru_params key8lru_params = {
- .n_entries = 1<<24,
- .f_hash = pipeline_test_hash,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key8lru_params = {
+ .name = "TABLE",
+ .key_size = 8,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
uint8_t key8lru[8];
@@ -475,14 +475,14 @@ test_table_hash8lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key8lru_params.n_entries = 0;
+ key8lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8lru_params.n_entries = 1<<16;
+ key8lru_params.n_keys = 1<<16;
key8lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key8_lru_ops,
@@ -499,13 +499,15 @@ test_table_hash16lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key16_lru_params key16lru_params = {
- .n_entries = 1<<16,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key16lru_params = {
+ .name = "TABLE",
+ .key_size = 16,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
uint8_t key16lru[16];
@@ -534,14 +536,14 @@ test_table_hash16lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key16lru_params.n_entries = 0;
+ key16lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16lru_params.n_entries = 1<<16;
+ key16lru_params.n_keys = 1<<16;
key16lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key16_lru_ops,
@@ -558,12 +560,15 @@ test_table_hash32lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key32_lru_params key32lru_params = {
- .n_entries = 1<<16,
+ struct rte_table_hash_params key32lru_params = {
+ .name = "TABLE",
+ .key_size = 32,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
.f_hash = pipeline_test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
};
uint8_t key32lru[32];
@@ -592,14 +597,14 @@ test_table_hash32lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key32lru_params.n_entries = 0;
+ key32lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key32_lru_ops,
(void *)&key32lru_params, (void *)key32lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32lru_params.n_entries = 1<<16;
+ key32lru_params.n_keys = 1<<16;
key32lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key32_lru_ops,
@@ -616,14 +621,15 @@ test_table_hash8ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key8_ext_params key8ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key8ext_params = {
+ .name = "TABLE",
+ .key_size = 8,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
uint8_t key8ext[8];
@@ -652,14 +658,14 @@ test_table_hash8ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key8ext_params.n_entries = 0;
+ key8ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8ext_params.n_entries = 1<<16;
+ key8ext_params.n_keys = 1<<16;
key8ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key8_ext_ops,
@@ -667,13 +673,6 @@ test_table_hash8ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8ext_params.f_hash = pipeline_test_hash;
- key8ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key8_ext_ops,
- (void *)&key8ext_params, (void *)key8ext, &table_packets, NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
@@ -683,14 +682,15 @@ test_table_hash16ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key16_ext_params key16ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key16ext_params = {
+ .name = "TABLE",
+ .key_size = 16,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
uint8_t key16ext[16];
@@ -719,14 +719,14 @@ test_table_hash16ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key16ext_params.n_entries = 0;
+ key16ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16ext_params.n_entries = 1<<16;
+ key16ext_params.n_keys = 1<<16;
key16ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key16_ext_ops,
@@ -734,13 +734,6 @@ test_table_hash16ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16ext_params.f_hash = pipeline_test_hash;
- key16ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key16_ext_ops,
- (void *)&key16ext_params, (void *)key16ext, &table_packets, NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
@@ -750,13 +743,15 @@ test_table_hash32ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key32_ext_params key32ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
+ struct rte_table_hash_params key32ext_params = {
+ .name = "TABLE",
+ .key_size = 32,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
.f_hash = pipeline_test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
};
uint8_t key32ext[32];
@@ -785,14 +780,14 @@ test_table_hash32ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key32ext_params.n_entries = 0;
+ key32ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key32_ext_ops,
(void *)&key32ext_params, (void *)key32ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32ext_params.n_entries = 1<<16;
+ key32ext_params.n_keys = 1<<16;
key32ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key32_ext_ops,
@@ -800,14 +795,6 @@ test_table_hash32ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32ext_params.f_hash = pipeline_test_hash;
- key32ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key32_ext_ops,
- (void *)&key32ext_params, (void *)key32ext, &table_packets,
- NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
@@ -817,14 +804,15 @@ test_table_hash_cuckoo_combined(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_cuckoo_params cuckoo_params = {
+ struct rte_table_hash_params cuckoo_params = {
+ .name = "TABLE",
.key_size = 32,
- .n_keys = 1<<16,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
.f_hash = pipeline_test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .name = "CUCKOO_HASH",
};
uint8_t key_cuckoo[32];
@@ -847,7 +835,7 @@ test_table_hash_cuckoo_combined(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -855,7 +843,7 @@ test_table_hash_cuckoo_combined(void)
/* Invalid parameters */
cuckoo_params.key_size = 0;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -863,7 +851,7 @@ test_table_hash_cuckoo_combined(void)
cuckoo_params.key_size = 32;
cuckoo_params.n_keys = 0;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -871,7 +859,7 @@ test_table_hash_cuckoo_combined(void)
cuckoo_params.n_keys = 1<<16;
cuckoo_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index d835eb9..187a867 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -60,16 +60,16 @@ table_test table_tests[] = {
memset(key, 0, 32); \
k32 = (uint32_t *) key; \
k32[0] = (value); \
- *signature = pipeline_test_hash(key, 0, 0); \
+ *signature = pipeline_test_hash(key, NULL, 0, 0); \
} while (0)
unsigned n_table_tests = RTE_DIM(table_tests);
/* Function prototypes */
static int
-test_table_hash_lru_generic(struct rte_table_ops *ops);
+test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size);
static int
-test_table_hash_ext_generic(struct rte_table_ops *ops);
+test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size);
struct rte_bucket_4_8 {
/* Cache line 0 */
@@ -655,7 +655,7 @@ test_table_lpm_ipv6(void)
}
static int
-test_table_hash_lru_generic(struct rte_table_ops *ops)
+test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size)
{
int status, i;
uint64_t expected_mask = 0, result_mask;
@@ -667,36 +667,24 @@ test_table_hash_lru_generic(struct rte_table_ops *ops)
int key_found;
/* Initialize params and create tables */
- struct rte_table_hash_key8_lru_params hash_params = {
- .n_entries = 1 << 10,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(1),
+ struct rte_table_hash_params hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 10,
+ .n_buckets = 1 << 10,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
- hash_params.n_entries = 0;
+ hash_params.n_keys = 0;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -1;
- hash_params.n_entries = 1 << 10;
- hash_params.signature_offset = APP_METADATA_OFFSET(1);
-
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -2;
-
- hash_params.signature_offset = APP_METADATA_OFFSET(0);
- hash_params.key_offset = APP_METADATA_OFFSET(1);
-
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -3;
-
- hash_params.key_offset = APP_METADATA_OFFSET(32);
+ hash_params.n_keys = 1 << 10;
hash_params.f_hash = NULL;
table = ops->f_create(&hash_params, 0, 1);
@@ -770,7 +758,7 @@ test_table_hash_lru_generic(struct rte_table_ops *ops)
}
static int
-test_table_hash_ext_generic(struct rte_table_ops *ops)
+test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size)
{
int status, i;
uint64_t expected_mask = 0, result_mask;
@@ -782,35 +770,24 @@ test_table_hash_ext_generic(struct rte_table_ops *ops)
void *entry_ptr;
/* Initialize params and create tables */
- struct rte_table_hash_key8_ext_params hash_params = {
- .n_entries = 1 << 10,
- .n_entries_ext = 1 << 4,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(1),
+ struct rte_table_hash_params hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 10,
+ .n_buckets = 1 << 10,
+ .f_hash = pipeline_test_hash,
+ .seed = 0,
};
- hash_params.n_entries = 0;
+ hash_params.n_keys = 0;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -1;
- hash_params.n_entries = 1 << 10;
- hash_params.n_entries_ext = 0;
- table = ops->f_create(&hash_params, 0, 1);
- if (table != NULL)
- return -2;
-
- hash_params.n_entries_ext = 1 << 4;
- hash_params.signature_offset = APP_METADATA_OFFSET(1);
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -2;
-
- hash_params.signature_offset = APP_METADATA_OFFSET(0);
+ hash_params.n_keys = 1 << 10;
hash_params.key_offset = APP_METADATA_OFFSET(1);
table = ops->f_create(&hash_params, 0, 1);
@@ -895,20 +872,21 @@ test_table_hash_lru(void)
{
int status;
- status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_ops);
- if (status < 0)
- return status;
-
status = test_table_hash_lru_generic(
- &rte_table_hash_key8_lru_dosig_ops);
+ &rte_table_hash_key8_lru_ops,
+ 8);
if (status < 0)
return status;
- status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_ops);
+ status = test_table_hash_lru_generic(
+ &rte_table_hash_key16_lru_ops,
+ 16);
if (status < 0)
return status;
- status = test_table_hash_lru_generic(&rte_table_hash_key32_lru_ops);
+ status = test_table_hash_lru_generic(
+ &rte_table_hash_key32_lru_ops,
+ 32);
if (status < 0)
return status;
@@ -924,20 +902,15 @@ test_table_hash_ext(void)
{
int status;
- status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops, 8);
if (status < 0)
return status;
- status = test_table_hash_ext_generic(
- &rte_table_hash_key8_ext_dosig_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops, 16);
if (status < 0)
return status;
- status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_ext_generic(&rte_table_hash_key32_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key32_ext_ops, 32);
if (status < 0)
return status;
@@ -959,23 +932,24 @@ test_table_hash_cuckoo(void)
uint32_t entry_size = 1;
/* Initialize params and create tables */
- struct rte_table_hash_cuckoo_params cuckoo_params = {
+ struct rte_table_hash_params cuckoo_params = {
+ .name = "TABLE",
.key_size = 32,
- .n_keys = 1 << 24,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
.f_hash = pipeline_test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .name = "CUCKOO",
};
- table = rte_table_hash_cuckoo_dosig_ops.f_create(NULL, 0, entry_size);
+ table = rte_table_hash_cuckoo_ops.f_create(NULL, 0, entry_size);
if (table != NULL)
return -1;
cuckoo_params.key_size = 0;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -2;
@@ -983,7 +957,7 @@ test_table_hash_cuckoo(void)
cuckoo_params.key_size = 32;
cuckoo_params.n_keys = 0;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -3;
@@ -991,7 +965,7 @@ test_table_hash_cuckoo(void)
cuckoo_params.n_keys = 1 << 24;
cuckoo_params.f_hash = NULL;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -4;
@@ -999,24 +973,24 @@ test_table_hash_cuckoo(void)
cuckoo_params.f_hash = pipeline_test_hash;
cuckoo_params.name = NULL;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -5;
cuckoo_params.name = "CUCKOO";
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table == NULL)
return -6;
/* Free */
- status = rte_table_hash_cuckoo_dosig_ops.f_free(table);
+ status = rte_table_hash_cuckoo_ops.f_free(table);
if (status < 0)
return -7;
- status = rte_table_hash_cuckoo_dosig_ops.f_free(NULL);
+ status = rte_table_hash_cuckoo_ops.f_free(NULL);
if (status == 0)
return -8;
@@ -1027,60 +1001,60 @@ test_table_hash_cuckoo(void)
memset(key_cuckoo, 0, 32);
kcuckoo[0] = rte_be_to_cpu_32(0xadadadad);
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params, 0, 1);
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params, 0, 1);
if (table == NULL)
return -9;
entry = 'A';
- status = rte_table_hash_cuckoo_dosig_ops.f_add(NULL, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(NULL, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status == 0)
return -10;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, NULL, &entry,
+ status = rte_table_hash_cuckoo_ops.f_add(table, NULL, &entry,
&key_found, &entry_ptr);
if (status == 0)
return -11;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
NULL, &key_found, &entry_ptr);
if (status == 0)
return -12;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status != 0)
return -13;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status != 0)
return -14;
/* Delete */
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(NULL, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(NULL, &key_cuckoo,
&key_found, NULL);
if (status == 0)
return -15;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, NULL,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, NULL,
&key_found, NULL);
if (status == 0)
return -16;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, &key_cuckoo,
&key_found, NULL);
if (status != 0)
return -17;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, &key_cuckoo,
&key_found, NULL);
if (status != -ENOENT)
return -18;
/* Traffic flow */
entry = 'A';
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found,
&entry_ptr);
if (status < 0)
@@ -1093,7 +1067,7 @@ test_table_hash_cuckoo(void)
} else
PREPARE_PACKET(mbufs[i], 0xadadadab);
- rte_table_hash_cuckoo_dosig_ops.f_lookup(table, mbufs, -1,
+ rte_table_hash_cuckoo_ops.f_lookup(table, mbufs, -1,
&result_mask, (void **)entries);
if (result_mask != expected_mask)
return -20;
@@ -1102,7 +1076,7 @@ test_table_hash_cuckoo(void)
for (i = 0; i < RTE_PORT_IN_BURST_SIZE_MAX; i++)
rte_pktmbuf_free(mbufs[i]);
- status = rte_table_hash_cuckoo_dosig_ops.f_free(table);
+ status = rte_table_hash_cuckoo_ops.f_free(table);
return 0;
}
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH V2 3/5] test-pipeline: update due to api changes in librte_table
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 0/5] " Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 1/5] " Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 2/5] test: update due to api changes in librte_table Cristian Dumitrescu
@ 2017-10-10 11:18 ` Cristian Dumitrescu
2017-10-10 11:19 ` [dpdk-dev] [PATCH V2 4/5] ip_pipeline: " Cristian Dumitrescu
` (2 subsequent siblings)
5 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-10 11:18 UTC (permalink / raw)
To: dev
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
Changes in V2:
- None
test/test-pipeline/main.h | 5 +-
test/test-pipeline/pipeline_hash.c | 107 +++++--------------------------------
2 files changed, 18 insertions(+), 94 deletions(-)
diff --git a/test/test-pipeline/main.h b/test/test-pipeline/main.h
index 3685849..26395a3 100644
--- a/test/test-pipeline/main.h
+++ b/test/test-pipeline/main.h
@@ -131,7 +131,10 @@ enum {
void app_main_loop_rx(void);
void app_main_loop_rx_metadata(void);
-uint64_t test_hash(void *key, uint32_t key_size, uint64_t seed);
+uint64_t test_hash(void *key,
+ void *key_mask,
+ uint32_t key_size,
+ uint64_t seed);
void app_main_loop_worker(void);
void app_main_loop_worker_pipeline_stub(void);
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 991e381..edc1663 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -169,23 +169,23 @@ app_main_loop_worker_pipeline_hash(void) {
"ring %d\n", i);
}
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = test_hash,
+ .seed = 0,
+ };
+
/* Table configuration */
switch (app.pipeline_type) {
case e_APP_PIPELINE_HASH_KEY8_EXT:
case e_APP_PIPELINE_HASH_KEY16_EXT:
case e_APP_PIPELINE_HASH_KEY32_EXT:
{
- struct rte_table_hash_ext_params table_hash_params = {
- .key_size = key_size,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .n_buckets_ext = 1 << 21,
- .f_hash = test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_ext_ops,
.arg_create = &table_hash_params,
@@ -204,16 +204,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_KEY16_LRU:
case e_APP_PIPELINE_HASH_KEY32_LRU:
{
- struct rte_table_hash_lru_params table_hash_params = {
- .key_size = key_size,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_lru_ops,
.arg_create = &table_hash_params,
@@ -230,16 +220,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_EXT:
{
- struct rte_table_hash_key8_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .f_hash = test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_hash_params,
@@ -256,15 +236,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_LRU:
{
- struct rte_table_hash_key8_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .f_hash = test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key8_lru_ops,
.arg_create = &table_hash_params,
@@ -281,16 +252,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_EXT:
{
- struct rte_table_hash_key16_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
- .key_mask = NULL,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key16_ext_ops,
.arg_create = &table_hash_params,
@@ -307,15 +268,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_LRU:
{
- struct rte_table_hash_key16_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
- .key_mask = NULL,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key16_lru_ops,
.arg_create = &table_hash_params,
@@ -332,15 +284,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_EXT:
{
- struct rte_table_hash_key32_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key32_ext_ops,
.arg_create = &table_hash_params,
@@ -358,14 +301,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_LRU:
{
- struct rte_table_hash_key32_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key32_lru_ops,
.arg_create = &table_hash_params,
@@ -390,23 +325,8 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_CUCKOO_KEY112:
case e_APP_PIPELINE_HASH_CUCKOO_KEY128:
{
- char hash_name[RTE_HASH_NAMESIZE];
-
- snprintf(hash_name, sizeof(hash_name), "RTE_TH_CUCKOO_%d",
- app.pipeline_type);
-
- struct rte_table_hash_cuckoo_params table_hash_params = {
- .key_size = key_size,
- .n_keys = (1 << 24) + 1,
- .f_hash = test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
- .name = hash_name,
- };
-
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_cuckoo_dosig_ops,
+ .ops = &rte_table_hash_cuckoo_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -477,6 +397,7 @@ app_main_loop_worker_pipeline_hash(void) {
uint64_t test_hash(
void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed)
{
@@ -539,7 +460,7 @@ app_main_loop_rx_metadata(void) {
} else
continue;
- *signature = test_hash(key, 0, 0);
+ *signature = test_hash(key, NULL, 0, 0);
}
do {
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH V2 4/5] ip_pipeline: update due to api changes in librte_table
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 0/5] " Cristian Dumitrescu
` (2 preceding siblings ...)
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 3/5] test-pipeline: " Cristian Dumitrescu
@ 2017-10-10 11:19 ` Cristian Dumitrescu
2017-10-10 11:19 ` [dpdk-dev] [PATCH V2 5/5] deprecation: removed the librte_table notice Cristian Dumitrescu
2017-10-11 14:22 ` [dpdk-dev] [PATCH V2 0/5] table: add key mask for hash tables Dumitrescu, Cristian
5 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-10 11:19 UTC (permalink / raw)
To: dev
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
Changes in V2:
- Added one file that was not picked up in V2 by mistake (hash_func.h)
- Fixed minor style issue
examples/ip_pipeline/pipeline/hash_func.h | 178 ++++++++++++---------
.../pipeline/pipeline_flow_classification.c | 12 +-
.../pipeline/pipeline_flow_classification_be.c | 51 +-----
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 18 +--
.../ip_pipeline/pipeline/pipeline_routing_be.c | 18 ++-
5 files changed, 138 insertions(+), 139 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/hash_func.h b/examples/ip_pipeline/pipeline/hash_func.h
index b112369..49fb44b 100644
--- a/examples/ip_pipeline/pipeline/hash_func.h
+++ b/examples/ip_pipeline/pipeline/hash_func.h
@@ -34,48 +34,56 @@
#define __INCLUDE_HASH_FUNC_H__
static inline uint64_t
-hash_xor_key8(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key8(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0;
- xor0 = seed ^ k[0];
+ xor0 = seed ^ (k[0] & m[0]);
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key16(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key16(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0;
- xor0 = (k[0] ^ seed) ^ k[1];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key24(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key24(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0;
- xor0 = (k[0] ^ seed) ^ k[1];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
- xor0 ^= k[2];
+ xor0 ^= k[2] & m[2];
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key32(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
xor0 ^= xor1;
@@ -83,30 +91,34 @@ hash_xor_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_xor_key40(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key40(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
xor0 ^= xor1;
- xor0 ^= k[4];
+ xor0 ^= k[4] & m[4];
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key48(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1, xor2;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
- xor2 = k[4] ^ k[5];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
+ xor2 = (k[4] & m[4]) ^ (k[5] & m[5]);
xor0 ^= xor1;
@@ -116,17 +128,19 @@ hash_xor_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_xor_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key56(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1, xor2;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
- xor2 = k[4] ^ k[5];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
+ xor2 = (k[4] & m[4]) ^ (k[5] & m[5]);
xor0 ^= xor1;
- xor2 ^= k[6];
+ xor2 ^= k[6] & m[6];
xor0 ^= xor2;
@@ -134,15 +148,17 @@ hash_xor_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_xor_key64(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key64(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1, xor2, xor3;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
- xor2 = k[4] ^ k[5];
- xor3 = k[6] ^ k[7];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
+ xor2 = (k[4] & m[4]) ^ (k[5] & m[5]);
+ xor3 = (k[6] & m[6]) ^ (k[7] & m[7]);
xor0 ^= xor1;
xor2 ^= xor3;
@@ -152,31 +168,35 @@ hash_xor_key64(void *key, __rte_unused uint32_t key_size, uint64_t seed)
return (xor0 >> 32) ^ xor0;
}
-#if defined(RTE_ARCH_X86_64)
+#if defined(RTE_ARCH_X86_64) && defined(RTE_MACHINE_CPUFLAG_SSE4_2)
#include <x86intrin.h>
static inline uint64_t
-hash_crc_key8(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key8(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t crc0;
- crc0 = _mm_crc32_u64(seed, k[0]);
+ crc0 = _mm_crc32_u64(seed, k[0] & m[0]);
return crc0;
}
static inline uint64_t
-hash_crc_key16(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key16(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, crc0, crc1;
- k0 = k[0];
+ k0 = k[0] & m[0];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
crc0 ^= crc1;
@@ -184,16 +204,18 @@ hash_crc_key16(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key24(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key24(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, crc0, crc1;
- k0 = k[0];
- k2 = k[2];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
crc0 = _mm_crc32_u64(crc0, k2);
@@ -203,18 +225,20 @@ hash_crc_key24(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key32(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, crc0, crc1, crc2, crc3;
- k0 = k[0];
- k2 = k[2];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
crc3 = k2 >> 32;
crc0 = _mm_crc32_u64(crc0, crc1);
@@ -226,19 +250,21 @@ hash_crc_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key40(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key40(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, crc0, crc1, crc2, crc3;
- k0 = k[0];
- k2 = k[2];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
crc0 = _mm_crc32_u64(crc0, crc1);
crc1 = _mm_crc32_u64(crc2, crc3);
@@ -249,20 +275,22 @@ hash_crc_key40(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key48(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, k5, crc0, crc1, crc2, crc3;
- k0 = k[0];
- k2 = k[2];
- k5 = k[5];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
+ k5 = k[5] & m[5];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
crc0 = _mm_crc32_u64(crc0, (crc1 << 32) ^ crc2);
crc1 = _mm_crc32_u64(crc3, k5);
@@ -273,22 +301,24 @@ hash_crc_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key56(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5;
- k0 = k[0];
- k2 = k[2];
- k5 = k[5];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
+ k5 = k[5] & m[5];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
- crc4 = _mm_crc32_u64(k5, k[6]);
+ crc4 = _mm_crc32_u64(k5, k[6] & m[6]);
crc5 = k5 >> 32;
crc0 = _mm_crc32_u64(crc0, (crc1 << 32) ^ crc2);
@@ -300,23 +330,25 @@ hash_crc_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key64(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key64(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5;
- k0 = k[0];
- k2 = k[2];
- k5 = k[5];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
+ k5 = k[5] & m[5];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
- crc4 = _mm_crc32_u64(k5, k[6]);
- crc5 = _mm_crc32_u64(k5 >> 32, k[7]);
+ crc4 = _mm_crc32_u64(k5, k[6] & m[6]);
+ crc5 = _mm_crc32_u64(k5 >> 32, k[7] & m[7]);
crc0 = _mm_crc32_u64(crc0, (crc1 << 32) ^ crc2);
crc1 = _mm_crc32_u64(crc3, (crc4 << 32) ^ crc5);
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
index 9ef50cc..3ee7266 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
@@ -88,8 +88,11 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
uint32_t *signature)
{
uint8_t buffer[PIPELINE_FC_FLOW_KEY_MAX_SIZE];
+ uint8_t m[PIPELINE_FC_FLOW_KEY_MAX_SIZE]; /* key mask */
void *key_buffer = (key_out) ? key_out : buffer;
+ memset(m, 0xFF, sizeof(m));
+
switch (key_in->type) {
case FLOW_KEY_QINQ:
{
@@ -101,7 +104,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
qinq->cvlan = rte_cpu_to_be_16(key_in->key.qinq.cvlan);
if (signature)
- *signature = (uint32_t) hash_default_key8(qinq, 8, 0);
+ *signature = (uint32_t) hash_default_key8(qinq, m, 8, 0);
return 0;
}
@@ -118,7 +121,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
ipv4->port_dst = rte_cpu_to_be_16(key_in->key.ipv4_5tuple.port_dst);
if (signature)
- *signature = (uint32_t) hash_default_key16(ipv4, 16, 0);
+ *signature = (uint32_t) hash_default_key16(ipv4, m, 16, 0);
return 0;
}
@@ -136,7 +139,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
ipv6->port_dst = rte_cpu_to_be_16(key_in->key.ipv6_5tuple.port_dst);
if (signature)
- *signature = (uint32_t) hash_default_key64(ipv6, 64, 0);
+ *signature = (uint32_t) hash_default_key64(ipv6, m, 64, 0);
return 0;
}
@@ -832,12 +835,11 @@ app_pipeline_fc_add_bulk(struct app_params *app,
}
/* Free resources */
- app_msg_free(app, rsp);
-
for (i = rsp->n_keys; i < n_keys; i++)
if (new_flow[i])
rte_free(flow[i]);
+ app_msg_free(app, rsp);
rte_free(flow_rsp);
rte_free(flow_req);
rte_free(new_flow);
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 026f00c..9846777 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -492,40 +492,16 @@ static void *pipeline_fc_init(struct pipeline_params *params,
/* Tables */
p->n_tables = 1;
{
- struct rte_table_hash_key8_ext_params
- table_hash_key8_params = {
- .n_entries = p_fc->n_flows,
- .n_entries_ext = p_fc->n_flows,
- .signature_offset = p_fc->hash_offset,
+ struct rte_table_hash_params table_hash_params = {
+ .name = p->name,
+ .key_size = p_fc->key_size,
.key_offset = p_fc->key_offset,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
.key_mask = (p_fc->key_mask_present) ?
p_fc->key_mask : NULL,
- .seed = 0,
- };
-
- struct rte_table_hash_key16_ext_params
- table_hash_key16_params = {
- .n_entries = p_fc->n_flows,
- .n_entries_ext = p_fc->n_flows,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
- .key_mask = (p_fc->key_mask_present) ?
- p_fc->key_mask : NULL,
- .seed = 0,
- };
-
- struct rte_table_hash_ext_params
- table_hash_params = {
- .key_size = p_fc->key_size,
.n_keys = p_fc->n_flows,
.n_buckets = p_fc->n_flows / 4,
- .n_buckets_ext = p_fc->n_flows / 4,
.f_hash = hash_func[(p_fc->key_size / 8) - 1],
.seed = 0,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
};
struct rte_pipeline_table_params table_params = {
@@ -542,32 +518,19 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
- if (p_fc->hash_offset != 0) {
- table_params.ops =
- &rte_table_hash_key8_ext_ops;
- } else {
- table_params.ops =
- &rte_table_hash_key8_ext_dosig_ops;
- }
- table_params.arg_create = &table_hash_key8_params;
+ table_params.ops = &rte_table_hash_key8_ext_ops;
break;
case 16:
- if (p_fc->hash_offset != 0) {
- table_params.ops =
- &rte_table_hash_key16_ext_ops;
- } else {
- table_params.ops =
- &rte_table_hash_key16_ext_dosig_ops;
- }
- table_params.arg_create = &table_hash_key16_params;
+ table_params.ops = &rte_table_hash_key16_ext_ops;
break;
default:
table_params.ops = &rte_table_hash_ext_ops;
- table_params.arg_create = &table_hash_params;
}
+ table_params.arg_create = &table_hash_params;
+
status = rte_pipeline_table_create(p->p,
&table_params,
&p->table_id[0]);
diff --git a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
index 8cb2f0c..2500332 100644
--- a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
@@ -76,7 +76,7 @@ static pipeline_msg_req_handler handlers[] = {
pipeline_msg_req_invalid_handler,
};
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt_work_dma(
struct rte_mbuf *pkt,
void *arg,
@@ -102,7 +102,7 @@ pkt_work_dma(
/* Read (dma_dst), compute (hash), write (hash) */
if (hash_enabled) {
- uint32_t hash = p->f_hash(dma_dst, dma_size, 0);
+ uint32_t hash = p->f_hash(dma_src, dma_mask, dma_size, 0);
*dma_hash = hash;
if (lb_hash) {
@@ -121,7 +121,7 @@ pkt_work_dma(
}
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt4_work_dma(
struct rte_mbuf **pkts,
void *arg,
@@ -173,10 +173,10 @@ pkt4_work_dma(
/* Read (dma_dst), compute (hash), write (hash) */
if (hash_enabled) {
- uint32_t hash0 = p->f_hash(dma_dst0, dma_size, 0);
- uint32_t hash1 = p->f_hash(dma_dst1, dma_size, 0);
- uint32_t hash2 = p->f_hash(dma_dst2, dma_size, 0);
- uint32_t hash3 = p->f_hash(dma_dst3, dma_size, 0);
+ uint32_t hash0 = p->f_hash(dma_src0, dma_mask, dma_size, 0);
+ uint32_t hash1 = p->f_hash(dma_src1, dma_mask, dma_size, 0);
+ uint32_t hash2 = p->f_hash(dma_src2, dma_mask, dma_size, 0);
+ uint32_t hash3 = p->f_hash(dma_src3, dma_mask, dma_size, 0);
*dma_hash0 = hash0;
*dma_hash1 = hash1;
@@ -217,7 +217,7 @@ pkt4_work_dma(
}
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt_work_swap(
struct rte_mbuf *pkt,
void *arg)
@@ -241,7 +241,7 @@ pkt_work_swap(
}
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt4_work_swap(
struct rte_mbuf **pkts,
void *arg)
diff --git a/examples/ip_pipeline/pipeline/pipeline_routing_be.c b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
index 7831716..d281e65 100644
--- a/examples/ip_pipeline/pipeline/pipeline_routing_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
@@ -191,7 +191,7 @@ struct layout {
dst->c = src->c; \
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt_work_routing(
struct rte_mbuf *pkt,
struct rte_pipeline_table_entry *table_entry,
@@ -317,7 +317,7 @@ pkt_work_routing(
}
}
-static __rte_always_inline void
+static inline __attribute__((always_inline)) void
pkt4_work_routing(
struct rte_mbuf **pkts,
struct rte_pipeline_table_entry **table_entries,
@@ -1349,17 +1349,19 @@ pipeline_routing_init(struct pipeline_params *params,
/* ARP table configuration */
if (p_rt->params.n_arp_entries) {
- struct rte_table_hash_key8_ext_params table_arp_params = {
- .n_entries = p_rt->params.n_arp_entries,
- .n_entries_ext = p_rt->params.n_arp_entries,
+ struct rte_table_hash_params table_arp_params = {
+ .name = p->name,
+ .key_size = 8,
+ .key_offset = p_rt->params.arp_key_offset,
+ .key_mask = NULL,
+ .n_keys = p_rt->params.n_arp_entries,
+ .n_buckets = p_rt->params.n_arp_entries / 4,
.f_hash = hash_default_key8,
.seed = 0,
- .signature_offset = 0, /* Unused */
- .key_offset = p_rt->params.arp_key_offset,
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_ext_dosig_ops,
+ .ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_arp_params,
.f_action_hit = get_arp_table_ah_hit(p_rt),
.f_action_miss = NULL,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH V2 5/5] deprecation: removed the librte_table notice
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 0/5] " Cristian Dumitrescu
` (3 preceding siblings ...)
2017-10-10 11:19 ` [dpdk-dev] [PATCH V2 4/5] ip_pipeline: " Cristian Dumitrescu
@ 2017-10-10 11:19 ` Cristian Dumitrescu
2017-10-11 14:22 ` [dpdk-dev] [PATCH V2 0/5] table: add key mask for hash tables Dumitrescu, Cristian
5 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-10 11:19 UTC (permalink / raw)
To: dev
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
doc/guides/rel_notes/deprecation.rst | 6 ------
1 file changed, 6 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index f4269f0..dd0213b 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -96,9 +96,3 @@ Deprecation Notices
* librte_meter: The API will change to accommodate configuration profiles.
Most of the API functions will have an additional opaque parameter.
-
-* librte_table: The ``key_mask`` parameter will be added to all the hash tables
- that currently do not have it, as well as to the hash compute function prototype.
- The non-"do-sig" versions of the hash tables will be removed
- (including the ``signature_offset`` parameter)
- and the "do-sig" versions renamed accordingly.
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [dpdk-dev] [PATCH V2 0/5] table: add key mask for hash tables
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 0/5] " Cristian Dumitrescu
` (4 preceding siblings ...)
2017-10-10 11:19 ` [dpdk-dev] [PATCH V2 5/5] deprecation: removed the librte_table notice Cristian Dumitrescu
@ 2017-10-11 14:22 ` Dumitrescu, Cristian
5 siblings, 0 replies; 53+ messages in thread
From: Dumitrescu, Cristian @ 2017-10-11 14:22 UTC (permalink / raw)
To: dev
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Cristian Dumitrescu
> Sent: Tuesday, October 10, 2017 12:19 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH V2 0/5] table: add key mask for hash tables
>
> Main changes:
>
> 1. The key_mask parameter is added to all the hash tables that were
> previously missing it, as well to the hash compute function. This was
> first started in DPDK 2.0, but was only implemented for a couple of
> hash tables. The benefit of this approach is that it allows for better
> performance for large keys (bigger than 16 bytes), while it preserves
> the same performance for small keys [Q&A1].
>
> 2. The precomputed key signature (i.e. non-"do-sig") versions have been
> removed for all the hash tables, so now the key signature is always
> computed on every lookup. Note that this approach also allows for the
> precomputed key signature scheme [Q&A2].
>
> 3. API cleanup: single parameter structure common for all hash tables.
>
> Q&A:
>
Applied on next-pipeline tree.
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 00/18] librte_table: add key mask for hash tables
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 1/5] " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 01/18] test: removing calls to deprecated " Cristian Dumitrescu
` (36 more replies)
0 siblings, 37 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Main changes:
1. The key_mask parameter is added to all the hash tables that were
previously missing it, as well to the hash compute function. This was
first started in DPDK 2.0, but was only implemented for a couple of
hash tables. The benefit of this approach is that it allows for better
performance for large keys (bigger than 16 bytes), while it preserves
the same performance for small keys [Q&A1].
2. The precomputed key signature (i.e. non-"do-sig") versions have been
removed for all the hash tables, so now the key signature is always
computed on every lookup. Note that this approach also allows for the
precomputed key signature scheme [Q&A2].
3. API cleanup: single parameter structure common for all hash tables.
Q&A:
Q1: How is better lookup performance achieved by using key mask approach
for hash tables?
A1: This approach eliminates the need to consolidate the lookup key in a
single contiguous buffer where the relevant packet fields are written
one by one, which is a very costly operation that also hash strong data
dependencies.
Q2: How can the pre-computed key signature scheme be implemented with
current approach?
A2: The application can implement a straightforward custom hash function
that simply reads the pre-computed key signature from a given offset
in the input key buffer where it has been stored prior to the lookup
operation.
Cristian Dumitrescu (18):
test: removing calls to deprecated hash tables
librte_table: remove deprecated 8-byte key hash tables
librte_table: remove deprecated 16-byte key hash tables
librte_table: remove deprecated variable size key ext hash tables
librte_table: remove deprecated variable size key lru hash tables
librte_table: rename cuckoo hash table ops
librte_table: add unified params structure and mask-based hash func
librte_table: rework variable size key ext hash tables
librte_table: rework variable size key lru hash table
librte_table: rework 8-byte key hash tables
librte_table: rework 16-byte key hash tables
librte_table: rework 32-byte key hash tables
librte_table: rework cuckoo hash table
test: add mask-based hash functions
librte_table: cosmetic enhancements in api file
librte_table: copyright cosmetic updates
librte_table: map file updates
doc: remove deprecation notice for librte_table
doc/guides/rel_notes/deprecation.rst | 6 -
examples/ip_pipeline/pipeline/hash_func.h | 176 +++--
.../pipeline/pipeline_flow_classification.c | 10 +-
.../pipeline/pipeline_flow_classification_be.c | 51 +-
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 10 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 14 +-
lib/librte_table/rte_table_hash.h | 305 +--------
lib/librte_table/rte_table_hash_cuckoo.c | 205 +++---
lib/librte_table/rte_table_hash_ext.c | 417 ++++--------
lib/librte_table/rte_table_hash_key16.c | 750 ++++++---------------
lib/librte_table/rte_table_hash_key32.c | 436 +++++++-----
lib/librte_table/rte_table_hash_key8.c | 716 ++++++--------------
lib/librte_table/rte_table_hash_lru.c | 513 ++++++--------
lib/librte_table/rte_table_version.map | 23 +-
test/test-pipeline/main.h | 5 +-
test/test-pipeline/pipeline_hash.c | 107 +--
test/test/test_table.c | 1 +
test/test/test_table.h | 3 +-
test/test/test_table_combined.c | 137 ++--
test/test/test_table_tables.c | 150 ++---
20 files changed, 1440 insertions(+), 2595 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 01/18] test: removing calls to deprecated hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 02/18] librte_table: remove deprecated 8-byte key " Cristian Dumitrescu
` (35 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Removing calls to hash tables that are going to be removed later.
The calls are removed from test/test, test/test-pipeline,
examples/ip_pipeline.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 18 +++-----------
test/test-pipeline/pipeline_hash.c | 12 +++++-----
test/test/test_table_combined.c | 28 +++++++++++-----------
test/test/test_table_tables.c | 8 +++----
4 files changed, 27 insertions(+), 39 deletions(-)
mode change 100644 => 100755 examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
old mode 100644
new mode 100755
index 026f00c..5e993c3
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -542,29 +542,17 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
- if (p_fc->hash_offset != 0) {
- table_params.ops =
- &rte_table_hash_key8_ext_ops;
- } else {
- table_params.ops =
- &rte_table_hash_key8_ext_dosig_ops;
- }
+ table_params.ops = &rte_table_hash_key8_ext_dosig_ops;
table_params.arg_create = &table_hash_key8_params;
break;
case 16:
- if (p_fc->hash_offset != 0) {
- table_params.ops =
- &rte_table_hash_key16_ext_ops;
- } else {
- table_params.ops =
- &rte_table_hash_key16_ext_dosig_ops;
- }
+ table_params.ops = &rte_table_hash_key16_ext_dosig_ops;
table_params.arg_create = &table_hash_key16_params;
break;
default:
- table_params.ops = &rte_table_hash_ext_ops;
+ table_params.ops = &rte_table_hash_ext_dosig_ops;
table_params.arg_create = &table_hash_params;
}
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 991e381..6d27f33 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -187,7 +187,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_ext_ops,
+ .ops = &rte_table_hash_ext_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -215,7 +215,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_lru_ops,
+ .ops = &rte_table_hash_lru_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -241,7 +241,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_ext_ops,
+ .ops = &rte_table_hash_key8_ext_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -266,7 +266,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_lru_ops,
+ .ops = &rte_table_hash_key8_lru_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -292,7 +292,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key16_ext_ops,
+ .ops = &rte_table_hash_key16_ext_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -317,7 +317,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key16_lru_ops,
+ .ops = &rte_table_hash_key16_lru_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index a2d19a1..a337a59 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -469,7 +469,7 @@ test_table_hash8lru(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key8_lru_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -477,7 +477,7 @@ test_table_hash8lru(void)
/* Invalid parameters */
key8lru_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key8_lru_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -485,7 +485,7 @@ test_table_hash8lru(void)
key8lru_params.n_entries = 1<<16;
key8lru_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key8_lru_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -528,7 +528,7 @@ test_table_hash16lru(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key16_lru_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -536,7 +536,7 @@ test_table_hash16lru(void)
/* Invalid parameters */
key16lru_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key16_lru_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -544,7 +544,7 @@ test_table_hash16lru(void)
key16lru_params.n_entries = 1<<16;
key16lru_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key16_lru_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -646,7 +646,7 @@ test_table_hash8ext(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key8_ext_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -654,7 +654,7 @@ test_table_hash8ext(void)
/* Invalid parameters */
key8ext_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key8_ext_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -662,7 +662,7 @@ test_table_hash8ext(void)
key8ext_params.n_entries = 1<<16;
key8ext_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key8_ext_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -670,7 +670,7 @@ test_table_hash8ext(void)
key8ext_params.f_hash = pipeline_test_hash;
key8ext_params.n_entries_ext = 0;
- status = test_table_type(&rte_table_hash_key8_ext_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets, NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -713,7 +713,7 @@ test_table_hash16ext(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key16_ext_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -721,7 +721,7 @@ test_table_hash16ext(void)
/* Invalid parameters */
key16ext_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key16_ext_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -729,7 +729,7 @@ test_table_hash16ext(void)
key16ext_params.n_entries = 1<<16;
key16ext_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key16_ext_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -737,7 +737,7 @@ test_table_hash16ext(void)
key16ext_params.f_hash = pipeline_test_hash;
key16ext_params.n_entries_ext = 0;
- status = test_table_type(&rte_table_hash_key16_ext_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets, NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index d835eb9..90d2f02 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -895,7 +895,7 @@ test_table_hash_lru(void)
{
int status;
- status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_ops);
+ status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_dosig_ops);
if (status < 0)
return status;
@@ -904,7 +904,7 @@ test_table_hash_lru(void)
if (status < 0)
return status;
- status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_ops);
+ status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_dosig_ops);
if (status < 0)
return status;
@@ -924,7 +924,7 @@ test_table_hash_ext(void)
{
int status;
- status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_dosig_ops);
if (status < 0)
return status;
@@ -933,7 +933,7 @@ test_table_hash_ext(void)
if (status < 0)
return status;
- status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_dosig_ops);
if (status < 0)
return status;
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 02/18] librte_table: remove deprecated 8-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 01/18] test: removing calls to deprecated " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 03/18] librte_table: remove deprecated 16-byte " Cristian Dumitrescu
` (34 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The non-dosig version of the 8-byte key hash tables (both extendable
bucket and LRU) are removed. The remaining hash tables are renamed to
eliminate the dosig particle from their name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 2 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 2 +-
lib/librte_table/rte_table_hash.h | 9 -
lib/librte_table/rte_table_hash_key8.c | 340 +--------------------
test/test-pipeline/pipeline_hash.c | 4 +-
test/test/test_table_combined.c | 14 +-
test/test/test_table_tables.c | 8 +-
7 files changed, 21 insertions(+), 358 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 5e993c3..485ebcb 100755
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -542,7 +542,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
- table_params.ops = &rte_table_hash_key8_ext_dosig_ops;
+ table_params.ops = &rte_table_hash_key8_ext_ops;
table_params.arg_create = &table_hash_key8_params;
break;
diff --git a/examples/ip_pipeline/pipeline/pipeline_routing_be.c b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
index 7831716..aa7f05e 100644
--- a/examples/ip_pipeline/pipeline/pipeline_routing_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
@@ -1359,7 +1359,7 @@ pipeline_routing_init(struct pipeline_params *params,
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_ext_dosig_ops,
+ .ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_arp_params,
.f_action_hit = get_arp_table_ah_hit(p_rt),
.f_action_miss = NULL,
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 57505a6..8ad812c 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -201,12 +201,8 @@ struct rte_table_hash_key8_lru_params {
uint8_t *key_mask;
};
-/** LRU hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key8_lru_ops;
-/** LRU hash table operations for key signature computed on lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key8_lru_dosig_ops;
-
/** Extendible bucket hash table parameters */
struct rte_table_hash_key8_ext_params {
/** Maximum number of entries (and keys) in the table */
@@ -234,13 +230,8 @@ struct rte_table_hash_key8_ext_params {
uint8_t *key_mask;
};
-/** Extendible bucket hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key8_ext_ops;
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key8_ext_dosig_ops;
-
/**
* 16-byte key hash tables
*
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
index 5f0c656..f2b285d 100644
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -583,18 +583,6 @@ rte_table_hash_entry_delete_key8_ext(
#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
- uint64_t signature; \
- uint32_t bucket_index; \
- \
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
- bucket_index = signature & (f->n_buckets - 1); \
- bucket1 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket_index * f->bucket_size]; \
- rte_prefetch0(bucket1); \
-}
-
-#define lookup1_stage1_dosig(mbuf1, bucket1, f) \
-{ \
uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
@@ -738,25 +726,7 @@ rte_table_hash_entry_delete_key8_ext(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
-{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
- \
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
- bucket10 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
- \
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
- bucket11 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
-}
-
-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f)\
+#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\
{ \
uint64_t *key10, *key11; \
uint64_t hash_offset_buffer10; \
@@ -871,8 +841,8 @@ rte_table_hash_lookup_key8_lru(
struct rte_table_hash *f = (struct rte_table_hash *) table;
struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index,
- pkt11_index, pkt20_index, pkt21_index;
+ uint32_t pkt00_index, pkt01_index, pkt10_index;
+ uint32_t pkt11_index, pkt20_index, pkt21_index;
uint64_t pkts_mask_out = 0;
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
@@ -888,7 +858,7 @@ rte_table_hash_lookup_key8_lru(
lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
lookup1_stage1(mbuf, bucket, f);
lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
+ pkts_mask_out, entries, f);
}
*lookup_hit_mask = pkts_mask_out;
@@ -987,132 +957,6 @@ rte_table_hash_lookup_key8_lru(
} /* rte_table_hash_lookup_key8_lru() */
static int
-rte_table_hash_lookup_key8_lru_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_8 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key8_lru_dosig() */
-
-static int
rte_table_hash_lookup_key8_ext(
void *table,
struct rte_mbuf **pkts,
@@ -1142,8 +986,8 @@ rte_table_hash_lookup_key8_ext(
lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
lookup1_stage1(mbuf, bucket, f);
lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask, buckets,
- keys, f);
+ pkts_mask_out, entries, buckets_mask,
+ buckets, keys, f);
}
goto grind_next_buckets;
@@ -1263,156 +1107,6 @@ rte_table_hash_lookup_key8_ext(
} /* rte_table_hash_lookup_key8_ext() */
static int
-rte_table_hash_lookup_key8_ext_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0, buckets_mask = 0;
- struct rte_bucket_4_8 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];
- uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_8 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask,
- buckets, keys, f);
- }
-
- goto grind_next_buckets;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
-grind_next_buckets:
- /* Grind next buckets */
- for ( ; buckets_mask; ) {
- uint64_t buckets_mask_next = 0;
-
- for ( ; buckets_mask; ) {
- uint64_t pkt_mask;
- uint32_t pkt_index;
-
- pkt_index = __builtin_ctzll(buckets_mask);
- pkt_mask = 1LLU << pkt_index;
- buckets_mask &= ~pkt_mask;
-
- lookup_grinder(pkt_index, buckets, keys, pkts_mask_out,
- entries, buckets_mask_next, f);
- }
-
- buckets_mask = buckets_mask_next;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key8_dosig_ext() */
-
-static int
rte_table_hash_key8_stats_read(void *table, struct rte_table_stats *stats, int clear)
{
struct rte_table_hash *t = table;
@@ -1437,17 +1131,6 @@ struct rte_table_ops rte_table_hash_key8_lru_ops = {
.f_stats = rte_table_hash_key8_stats_read,
};
-struct rte_table_ops rte_table_hash_key8_lru_dosig_ops = {
- .f_create = rte_table_hash_create_key8_lru,
- .f_free = rte_table_hash_free_key8_lru,
- .f_add = rte_table_hash_entry_add_key8_lru,
- .f_delete = rte_table_hash_entry_delete_key8_lru,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lookup_key8_lru_dosig,
- .f_stats = rte_table_hash_key8_stats_read,
-};
-
struct rte_table_ops rte_table_hash_key8_ext_ops = {
.f_create = rte_table_hash_create_key8_ext,
.f_free = rte_table_hash_free_key8_ext,
@@ -1458,14 +1141,3 @@ struct rte_table_ops rte_table_hash_key8_ext_ops = {
.f_lookup = rte_table_hash_lookup_key8_ext,
.f_stats = rte_table_hash_key8_stats_read,
};
-
-struct rte_table_ops rte_table_hash_key8_ext_dosig_ops = {
- .f_create = rte_table_hash_create_key8_ext,
- .f_free = rte_table_hash_free_key8_ext,
- .f_add = rte_table_hash_entry_add_key8_ext,
- .f_delete = rte_table_hash_entry_delete_key8_ext,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lookup_key8_ext_dosig,
- .f_stats = rte_table_hash_key8_stats_read,
-};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 6d27f33..8f384fa 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -241,7 +241,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_ext_dosig_ops,
+ .ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -266,7 +266,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_lru_dosig_ops,
+ .ops = &rte_table_hash_key8_lru_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index a337a59..680454f 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -469,7 +469,7 @@ test_table_hash8lru(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -477,7 +477,7 @@ test_table_hash8lru(void)
/* Invalid parameters */
key8lru_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -485,7 +485,7 @@ test_table_hash8lru(void)
key8lru_params.n_entries = 1<<16;
key8lru_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -646,7 +646,7 @@ test_table_hash8ext(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -654,7 +654,7 @@ test_table_hash8ext(void)
/* Invalid parameters */
key8ext_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -662,7 +662,7 @@ test_table_hash8ext(void)
key8ext_params.n_entries = 1<<16;
key8ext_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -670,7 +670,7 @@ test_table_hash8ext(void)
key8ext_params.f_hash = pipeline_test_hash;
key8ext_params.n_entries_ext = 0;
- status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets, NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 90d2f02..43280b0 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -895,12 +895,12 @@ test_table_hash_lru(void)
{
int status;
- status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_dosig_ops);
+ status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_ops);
if (status < 0)
return status;
status = test_table_hash_lru_generic(
- &rte_table_hash_key8_lru_dosig_ops);
+ &rte_table_hash_key8_lru_ops);
if (status < 0)
return status;
@@ -924,12 +924,12 @@ test_table_hash_ext(void)
{
int status;
- status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_dosig_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops);
if (status < 0)
return status;
status = test_table_hash_ext_generic(
- &rte_table_hash_key8_ext_dosig_ops);
+ &rte_table_hash_key8_ext_ops);
if (status < 0)
return status;
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 03/18] librte_table: remove deprecated 16-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 01/18] test: removing calls to deprecated " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 02/18] librte_table: remove deprecated 8-byte key " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 04/18] librte_table: remove deprecated variable size key ext " Cristian Dumitrescu
` (33 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The non-dosig version of the 16-byte key hash tables (both extendable
bucket and LRU) are removed. The remaining hash tables are renamed to
eliminate the dosig particle from their name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 2 +-
lib/librte_table/rte_table_hash.h | 10 -
lib/librte_table/rte_table_hash_key16.c | 342 +--------------------
test/test-pipeline/pipeline_hash.c | 4 +-
test/test/test_table_combined.c | 14 +-
test/test/test_table_tables.c | 4 +-
6 files changed, 16 insertions(+), 360 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 485ebcb..0f667e6 100755
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -547,7 +547,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
break;
case 16:
- table_params.ops = &rte_table_hash_key16_ext_dosig_ops;
+ table_params.ops = &rte_table_hash_key16_ext_ops;
table_params.arg_create = &table_hash_key16_params;
break;
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 8ad812c..57c6073 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -259,13 +259,8 @@ struct rte_table_hash_key16_lru_params {
uint8_t *key_mask;
};
-/** LRU hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key16_lru_ops;
-/** LRU hash table operations for key signature computed on lookup
- ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key16_lru_dosig_ops;
-
/** Extendible bucket hash table parameters */
struct rte_table_hash_key16_ext_params {
/** Maximum number of entries (and keys) in the table */
@@ -293,13 +288,8 @@ struct rte_table_hash_key16_ext_params {
uint8_t *key_mask;
};
-/** Extendible bucket operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key16_ext_ops;
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key16_ext_dosig_ops;
-
/**
* 32-byte key hash tables
*
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
index ce057b7..4ed5c78 100644
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -610,19 +610,6 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
- uint64_t signature; \
- uint32_t bucket_index; \
- \
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
- bucket_index = signature & (f->n_buckets - 1); \
- bucket1 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket_index * f->bucket_size]; \
- rte_prefetch0(bucket1); \
- rte_prefetch0((void *)(((uintptr_t) bucket1) + RTE_CACHE_LINE_SIZE));\
-}
-
-#define lookup1_stage1_dosig(mbuf1, bucket1, f) \
-{ \
uint64_t *key; \
uint64_t signature = 0; \
uint32_t bucket_index; \
@@ -775,26 +762,6 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
- \
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
- bucket10 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
- rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
- \
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
- bucket11 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
- rte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\
-}
-
-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f) \
-{ \
uint64_t *key10, *key11; \
uint64_t hash_offset_buffer[2]; \
uint64_t signature10, signature11; \
@@ -916,6 +883,7 @@ rte_table_hash_lookup_key16_lru(
uint64_t pkts_mask_out = 0;
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
+
RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
/* Cannot run the pipeline with less than 5 packets */
@@ -932,8 +900,8 @@ rte_table_hash_lookup_key16_lru(
}
*lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f,
- n_pkts_in - __builtin_popcountll(pkts_mask_out));
+ RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
+ __builtin_popcountll(pkts_mask_out));
return 0;
}
@@ -1029,135 +997,6 @@ rte_table_hash_lookup_key16_lru(
} /* rte_table_hash_lookup_key16_lru() */
static int
-rte_table_hash_lookup_key16_lru_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
-
- RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_16 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key16_lru_dosig() */
-
-static int
rte_table_hash_lookup_key16_ext(
void *table,
struct rte_mbuf **pkts,
@@ -1175,6 +1014,7 @@ rte_table_hash_lookup_key16_ext(
uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
+
RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
/* Cannot run the pipeline with less than 5 packets */
@@ -1309,158 +1149,6 @@ rte_table_hash_lookup_key16_ext(
} /* rte_table_hash_lookup_key16_ext() */
static int
-rte_table_hash_lookup_key16_ext_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0, buckets_mask = 0;
- struct rte_bucket_4_16 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];
- uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
-
- RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_16 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask,
- buckets, keys, f);
- }
-
- goto grind_next_buckets;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
-grind_next_buckets:
- /* Grind next buckets */
- for ( ; buckets_mask; ) {
- uint64_t buckets_mask_next = 0;
-
- for ( ; buckets_mask; ) {
- uint64_t pkt_mask;
- uint32_t pkt_index;
-
- pkt_index = __builtin_ctzll(buckets_mask);
- pkt_mask = 1LLU << pkt_index;
- buckets_mask &= ~pkt_mask;
-
- lookup_grinder(pkt_index, buckets, keys, pkts_mask_out,
- entries, buckets_mask_next, f);
- }
-
- buckets_mask = buckets_mask_next;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key16_ext_dosig() */
-
-static int
rte_table_hash_key16_stats_read(void *table, struct rte_table_stats *stats, int clear)
{
struct rte_table_hash *t = table;
@@ -1479,37 +1167,15 @@ struct rte_table_ops rte_table_hash_key16_lru_ops = {
.f_free = rte_table_hash_free_key16_lru,
.f_add = rte_table_hash_entry_add_key16_lru,
.f_delete = rte_table_hash_entry_delete_key16_lru,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
.f_lookup = rte_table_hash_lookup_key16_lru,
.f_stats = rte_table_hash_key16_stats_read,
};
-struct rte_table_ops rte_table_hash_key16_lru_dosig_ops = {
- .f_create = rte_table_hash_create_key16_lru,
- .f_free = rte_table_hash_free_key16_lru,
- .f_add = rte_table_hash_entry_add_key16_lru,
- .f_delete = rte_table_hash_entry_delete_key16_lru,
- .f_lookup = rte_table_hash_lookup_key16_lru_dosig,
- .f_stats = rte_table_hash_key16_stats_read,
-};
-
struct rte_table_ops rte_table_hash_key16_ext_ops = {
.f_create = rte_table_hash_create_key16_ext,
.f_free = rte_table_hash_free_key16_ext,
.f_add = rte_table_hash_entry_add_key16_ext,
.f_delete = rte_table_hash_entry_delete_key16_ext,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
.f_lookup = rte_table_hash_lookup_key16_ext,
.f_stats = rte_table_hash_key16_stats_read,
};
-
-struct rte_table_ops rte_table_hash_key16_ext_dosig_ops = {
- .f_create = rte_table_hash_create_key16_ext,
- .f_free = rte_table_hash_free_key16_ext,
- .f_add = rte_table_hash_entry_add_key16_ext,
- .f_delete = rte_table_hash_entry_delete_key16_ext,
- .f_lookup = rte_table_hash_lookup_key16_ext_dosig,
- .f_stats = rte_table_hash_key16_stats_read,
-};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 8f384fa..b4e4e08 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -292,7 +292,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key16_ext_dosig_ops,
+ .ops = &rte_table_hash_key16_ext_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -317,7 +317,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key16_lru_dosig_ops,
+ .ops = &rte_table_hash_key16_lru_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index 680454f..a2d19a1 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -528,7 +528,7 @@ test_table_hash16lru(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -536,7 +536,7 @@ test_table_hash16lru(void)
/* Invalid parameters */
key16lru_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -544,7 +544,7 @@ test_table_hash16lru(void)
key16lru_params.n_entries = 1<<16;
key16lru_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -713,7 +713,7 @@ test_table_hash16ext(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -721,7 +721,7 @@ test_table_hash16ext(void)
/* Invalid parameters */
key16ext_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -729,7 +729,7 @@ test_table_hash16ext(void)
key16ext_params.n_entries = 1<<16;
key16ext_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -737,7 +737,7 @@ test_table_hash16ext(void)
key16ext_params.f_hash = pipeline_test_hash;
key16ext_params.n_entries_ext = 0;
- status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets, NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 43280b0..19aa5a4 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -904,7 +904,7 @@ test_table_hash_lru(void)
if (status < 0)
return status;
- status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_dosig_ops);
+ status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_ops);
if (status < 0)
return status;
@@ -933,7 +933,7 @@ test_table_hash_ext(void)
if (status < 0)
return status;
- status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_dosig_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops);
if (status < 0)
return status;
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 04/18] librte_table: remove deprecated variable size key ext hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (2 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 03/18] librte_table: remove deprecated 16-byte " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 05/18] librte_table: remove deprecated variable size key lru " Cristian Dumitrescu
` (32 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The non-dosig version of the variable size key extendible bucket
hash tables are removed. The remaining hash tables are renamed to
eliminate the dosig particle from their name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 2 +-
lib/librte_table/rte_table_hash.h | 5 -
lib/librte_table/rte_table_hash_ext.c | 196 +--------------------
test/test-pipeline/pipeline_hash.c | 2 +-
4 files changed, 8 insertions(+), 197 deletions(-)
mode change 100644 => 100755 lib/librte_table/rte_table_hash.h
mode change 100644 => 100755 lib/librte_table/rte_table_hash_ext.c
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 0f667e6..191cb15 100755
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -552,7 +552,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
break;
default:
- table_params.ops = &rte_table_hash_ext_dosig_ops;
+ table_params.ops = &rte_table_hash_ext_ops;
table_params.arg_create = &table_hash_params;
}
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
old mode 100644
new mode 100755
index 57c6073..0eb23a7
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -135,13 +135,8 @@ struct rte_table_hash_ext_params {
uint32_t key_offset;
};
-/** Extendible bucket hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_ext_ops;
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_ext_dosig_ops;
-
/** LRU hash table parameters */
struct rte_table_hash_lru_params {
/** Key size (number of bytes) */
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
old mode 100644
new mode 100755
index e718102..c4824c3
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -437,8 +437,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
- void **entries,
- int dosig)
+ void **entries)
{
struct rte_table_hash *t = (struct rte_table_hash *) table;
uint64_t pkts_mask_out = 0;
@@ -458,11 +457,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- if (dosig)
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
- else
- sig = RTE_MBUF_METADATA_UINT32(pkt,
- t->signature_offset);
+ sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
@@ -685,38 +680,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
-{ \
- struct grinder *g10, *g11; \
- uint64_t sig10, sig11, bkt10_index, bkt11_index; \
- struct rte_mbuf *mbuf10, *mbuf11; \
- struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
- uint64_t bucket_mask = t->bucket_mask; \
- uint32_t signature_offset = t->signature_offset; \
- \
- mbuf10 = pkts[pkt10_index]; \
- sig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\
- bkt10_index = sig10 & bucket_mask; \
- bkt10 = &buckets[bkt10_index]; \
- \
- mbuf11 = pkts[pkt11_index]; \
- sig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\
- bkt11_index = sig11 & bucket_mask; \
- bkt11 = &buckets[bkt11_index]; \
- \
- rte_prefetch0(bkt10); \
- rte_prefetch0(bkt11); \
- \
- g10 = &g[pkt10_index]; \
- g10->sig = sig10; \
- g10->bkt = bkt10; \
- \
- g11 = &g[pkt11_index]; \
- g11->sig = sig11; \
- g11->bkt = bkt11; \
-}
-
-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index) \
+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
{ \
struct grinder *g10, *g11; \
uint64_t sig10, sig11, bkt10_index, bkt11_index; \
@@ -874,7 +838,7 @@ static int rte_table_hash_ext_lookup(
/* Cannot run the pipeline with less than 7 packets */
if (__builtin_popcountll(pkts_mask) < 7) {
status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 0);
+ pkts_mask, lookup_hit_mask, entries);
RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -
__builtin_popcountll(*lookup_hit_mask));
return status;
@@ -982,144 +946,7 @@ static int rte_table_hash_ext_lookup(
uint64_t pkts_mask_out_slow = 0;
status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 0);
- pkts_mask_out |= pkts_mask_out_slow;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return status;
-}
-
-static int rte_table_hash_ext_lookup_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *t = (struct rte_table_hash *) table;
- struct grinder *g = t->grinders;
- uint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;
- uint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;
- uint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;
- int status = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_EXT_STATS_PKTS_IN_ADD(t, n_pkts_in);
-
- /* Cannot run the pipeline with less than 7 packets */
- if (__builtin_popcountll(pkts_mask) < 7) {
- status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 1);
- RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -
- __builtin_popcountll(*lookup_hit_mask));
- return status;
- }
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline feed */
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline feed */
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,
- pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index,
- pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,
- pkts_mask_out, entries);
- }
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Slow path */
- pkts_mask_match_many &= ~pkts_mask_out;
- if (pkts_mask_match_many) {
- uint64_t pkts_mask_out_slow = 0;
-
- status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 1);
+ pkts_mask_match_many, &pkts_mask_out_slow, entries);
pkts_mask_out |= pkts_mask_out_slow;
}
@@ -1142,7 +969,7 @@ rte_table_hash_ext_stats_read(void *table, struct rte_table_stats *stats, int cl
return 0;
}
-struct rte_table_ops rte_table_hash_ext_ops = {
+struct rte_table_ops rte_table_hash_ext_ops = {
.f_create = rte_table_hash_ext_create,
.f_free = rte_table_hash_ext_free,
.f_add = rte_table_hash_ext_entry_add,
@@ -1152,14 +979,3 @@ struct rte_table_ops rte_table_hash_ext_ops = {
.f_lookup = rte_table_hash_ext_lookup,
.f_stats = rte_table_hash_ext_stats_read,
};
-
-struct rte_table_ops rte_table_hash_ext_dosig_ops = {
- .f_create = rte_table_hash_ext_create,
- .f_free = rte_table_hash_ext_free,
- .f_add = rte_table_hash_ext_entry_add,
- .f_delete = rte_table_hash_ext_entry_delete,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_ext_lookup_dosig,
- .f_stats = rte_table_hash_ext_stats_read,
-};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index b4e4e08..e473c42 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -187,7 +187,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_ext_dosig_ops,
+ .ops = &rte_table_hash_ext_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 05/18] librte_table: remove deprecated variable size key lru hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (3 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 04/18] librte_table: remove deprecated variable size key ext " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 06/18] librte_table: rename cuckoo hash table ops Cristian Dumitrescu
` (31 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The non-dosig version of the variable size key Least Recently Used
(LRU) hash tables are removed. The remaining hash tables are renamed
to eliminate the dosig particle from their name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 4 -
lib/librte_table/rte_table_hash_lru.c | 190 +---------------------------------
test/test-pipeline/pipeline_hash.c | 2 +-
3 files changed, 6 insertions(+), 190 deletions(-)
mode change 100644 => 100755 lib/librte_table/rte_table_hash_lru.c
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 0eb23a7..f983518 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -163,12 +163,8 @@ struct rte_table_hash_lru_params {
uint32_t key_offset;
};
-/** LRU hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_lru_ops;
-/** LRU hash table operations for key signature computed on lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_lru_dosig_ops;
-
/**
* 8-byte key hash tables
*
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
old mode 100644
new mode 100755
index 5a4864e..0a85123
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -365,8 +365,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
- void **entries,
- int dosig)
+ void **entries)
{
struct rte_table_hash *t = (struct rte_table_hash *) table;
uint64_t pkts_mask_out = 0;
@@ -387,11 +386,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- if (dosig)
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
- else
- sig = RTE_MBUF_METADATA_UINT32(pkt,
- t->signature_offset);
+ sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
@@ -616,38 +611,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
-{ \
- struct grinder *g10, *g11; \
- uint64_t sig10, sig11, bkt10_index, bkt11_index; \
- struct rte_mbuf *mbuf10, *mbuf11; \
- struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
- uint64_t bucket_mask = t->bucket_mask; \
- uint32_t signature_offset = t->signature_offset; \
- \
- mbuf10 = pkts[pkt10_index]; \
- sig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\
- bkt10_index = sig10 & bucket_mask; \
- bkt10 = &buckets[bkt10_index]; \
- \
- mbuf11 = pkts[pkt11_index]; \
- sig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\
- bkt11_index = sig11 & bucket_mask; \
- bkt11 = &buckets[bkt11_index]; \
- \
- rte_prefetch0(bkt10); \
- rte_prefetch0(bkt11); \
- \
- g10 = &g[pkt10_index]; \
- g10->sig = sig10; \
- g10->bkt = bkt10; \
- \
- g11 = &g[pkt11_index]; \
- g11->sig = sig11; \
- g11->bkt = bkt11; \
-}
-
-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index)\
+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index)\
{ \
struct grinder *g10, *g11; \
uint64_t sig10, sig11, bkt10_index, bkt11_index; \
@@ -819,7 +783,7 @@ static int rte_table_hash_lru_lookup(
/* Cannot run the pipeline with less than 7 packets */
if (__builtin_popcountll(pkts_mask) < 7)
return rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 0);
+ pkts_mask, lookup_hit_mask, entries);
/* Pipeline stage 0 */
lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
@@ -923,140 +887,7 @@ static int rte_table_hash_lru_lookup(
uint64_t pkts_mask_out_slow = 0;
status = rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 0);
- pkts_mask_out |= pkts_mask_out_slow;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_LRU_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return status;
-}
-
-static int rte_table_hash_lru_lookup_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *t = (struct rte_table_hash *) table;
- struct grinder *g = t->grinders;
- uint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;
- uint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;
- uint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;
- int status = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_LRU_STATS_PKTS_IN_ADD(t, n_pkts_in);
-
- /* Cannot run the pipeline with less than 7 packets */
- if (__builtin_popcountll(pkts_mask) < 7)
- return rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 1);
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline feed */
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline feed */
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,
- pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index,
- pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,
- pkts_mask_out, entries);
- }
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Slow path */
- pkts_mask_match_many &= ~pkts_mask_out;
- if (pkts_mask_match_many) {
- uint64_t pkts_mask_out_slow = 0;
-
- status = rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 1);
+ pkts_mask_match_many, &pkts_mask_out_slow, entries);
pkts_mask_out |= pkts_mask_out_slow;
}
@@ -1089,14 +920,3 @@ struct rte_table_ops rte_table_hash_lru_ops = {
.f_lookup = rte_table_hash_lru_lookup,
.f_stats = rte_table_hash_lru_stats_read,
};
-
-struct rte_table_ops rte_table_hash_lru_dosig_ops = {
- .f_create = rte_table_hash_lru_create,
- .f_free = rte_table_hash_lru_free,
- .f_add = rte_table_hash_lru_entry_add,
- .f_delete = rte_table_hash_lru_entry_delete,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lru_lookup_dosig,
- .f_stats = rte_table_hash_lru_stats_read,
-};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index e473c42..991e381 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -215,7 +215,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_lru_dosig_ops,
+ .ops = &rte_table_hash_lru_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 06/18] librte_table: rename cuckoo hash table ops
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (4 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 05/18] librte_table: remove deprecated variable size key lru " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 07/18] librte_table: add unified params structure and mask-based hash func Cristian Dumitrescu
` (30 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The dosig particle is eliminated from the cuckoo hash ops name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 3 +--
lib/librte_table/rte_table_hash_cuckoo.c | 6 ++---
test/test-pipeline/pipeline_hash.c | 2 +-
test/test/test_table_combined.c | 8 +++---
test/test/test_table_tables.c | 42 ++++++++++++++++----------------
5 files changed, 30 insertions(+), 31 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index f983518..5f655ee 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -361,8 +361,7 @@ struct rte_table_hash_cuckoo_params {
const char *name;
};
-/** Cuckoo hash table operations */
-extern struct rte_table_ops rte_table_hash_cuckoo_dosig_ops;
+extern struct rte_table_ops rte_table_hash_cuckoo_ops;
#ifdef __cplusplus
}
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
index da1597f..beb45c5 100644
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -286,7 +286,7 @@ rte_table_hash_cuckoo_entry_delete(void *table, void *key,
static int
-rte_table_hash_cuckoo_lookup_dosig(void *table,
+rte_table_hash_cuckoo_lookup(void *table,
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
@@ -370,13 +370,13 @@ rte_table_hash_cuckoo_stats_read(void *table, struct rte_table_stats *stats,
return 0;
}
-struct rte_table_ops rte_table_hash_cuckoo_dosig_ops = {
+struct rte_table_ops rte_table_hash_cuckoo_ops = {
.f_create = rte_table_hash_cuckoo_create,
.f_free = rte_table_hash_cuckoo_free,
.f_add = rte_table_hash_cuckoo_entry_add,
.f_delete = rte_table_hash_cuckoo_entry_delete,
.f_add_bulk = NULL,
.f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_cuckoo_lookup_dosig,
+ .f_lookup = rte_table_hash_cuckoo_lookup,
.f_stats = rte_table_hash_cuckoo_stats_read,
};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 991e381..2f8c625 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -406,7 +406,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_cuckoo_dosig_ops,
+ .ops = &rte_table_hash_cuckoo_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index a2d19a1..f65d34a 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -847,7 +847,7 @@ test_table_hash_cuckoo_combined(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -855,7 +855,7 @@ test_table_hash_cuckoo_combined(void)
/* Invalid parameters */
cuckoo_params.key_size = 0;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -863,7 +863,7 @@ test_table_hash_cuckoo_combined(void)
cuckoo_params.key_size = 32;
cuckoo_params.n_keys = 0;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -871,7 +871,7 @@ test_table_hash_cuckoo_combined(void)
cuckoo_params.n_keys = 1<<16;
cuckoo_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 19aa5a4..efbd0c6 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -969,13 +969,13 @@ test_table_hash_cuckoo(void)
.name = "CUCKOO",
};
- table = rte_table_hash_cuckoo_dosig_ops.f_create(NULL, 0, entry_size);
+ table = rte_table_hash_cuckoo_ops.f_create(NULL, 0, entry_size);
if (table != NULL)
return -1;
cuckoo_params.key_size = 0;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -2;
@@ -983,7 +983,7 @@ test_table_hash_cuckoo(void)
cuckoo_params.key_size = 32;
cuckoo_params.n_keys = 0;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -3;
@@ -991,7 +991,7 @@ test_table_hash_cuckoo(void)
cuckoo_params.n_keys = 1 << 24;
cuckoo_params.f_hash = NULL;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -4;
@@ -999,24 +999,24 @@ test_table_hash_cuckoo(void)
cuckoo_params.f_hash = pipeline_test_hash;
cuckoo_params.name = NULL;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -5;
cuckoo_params.name = "CUCKOO";
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table == NULL)
return -6;
/* Free */
- status = rte_table_hash_cuckoo_dosig_ops.f_free(table);
+ status = rte_table_hash_cuckoo_ops.f_free(table);
if (status < 0)
return -7;
- status = rte_table_hash_cuckoo_dosig_ops.f_free(NULL);
+ status = rte_table_hash_cuckoo_ops.f_free(NULL);
if (status == 0)
return -8;
@@ -1027,60 +1027,60 @@ test_table_hash_cuckoo(void)
memset(key_cuckoo, 0, 32);
kcuckoo[0] = rte_be_to_cpu_32(0xadadadad);
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params, 0, 1);
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params, 0, 1);
if (table == NULL)
return -9;
entry = 'A';
- status = rte_table_hash_cuckoo_dosig_ops.f_add(NULL, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(NULL, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status == 0)
return -10;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, NULL, &entry,
+ status = rte_table_hash_cuckoo_ops.f_add(table, NULL, &entry,
&key_found, &entry_ptr);
if (status == 0)
return -11;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
NULL, &key_found, &entry_ptr);
if (status == 0)
return -12;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status != 0)
return -13;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status != 0)
return -14;
/* Delete */
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(NULL, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(NULL, &key_cuckoo,
&key_found, NULL);
if (status == 0)
return -15;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, NULL,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, NULL,
&key_found, NULL);
if (status == 0)
return -16;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, &key_cuckoo,
&key_found, NULL);
if (status != 0)
return -17;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, &key_cuckoo,
&key_found, NULL);
if (status != -ENOENT)
return -18;
/* Traffic flow */
entry = 'A';
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found,
&entry_ptr);
if (status < 0)
@@ -1093,7 +1093,7 @@ test_table_hash_cuckoo(void)
} else
PREPARE_PACKET(mbufs[i], 0xadadadab);
- rte_table_hash_cuckoo_dosig_ops.f_lookup(table, mbufs, -1,
+ rte_table_hash_cuckoo_ops.f_lookup(table, mbufs, -1,
&result_mask, (void **)entries);
if (result_mask != expected_mask)
return -20;
@@ -1102,7 +1102,7 @@ test_table_hash_cuckoo(void)
for (i = 0; i < RTE_PORT_IN_BURST_SIZE_MAX; i++)
rte_pktmbuf_free(mbufs[i]);
- status = rte_table_hash_cuckoo_dosig_ops.f_free(table);
+ status = rte_table_hash_cuckoo_ops.f_free(table);
return 0;
}
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 07/18] librte_table: add unified params structure and mask-based hash func
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (5 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 06/18] librte_table: rename cuckoo hash table ops Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 08/18] librte_table: rework variable size key ext hash tables Cristian Dumitrescu
` (29 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Add unified parameter structure for all hash tables in librte_table.
Add mask-based hash function prototype, which is input parameter for
all hash tables.
Renamed the non-mask-based hash function prototype and all the calls
to it (to be removed later).
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 2 +-
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 4 +-
lib/librte_table/rte_table_hash.h | 52 ++++++++++++++++++----
lib/librte_table/rte_table_hash_cuckoo.c | 2 +-
lib/librte_table/rte_table_hash_ext.c | 4 +-
lib/librte_table/rte_table_hash_key16.c | 2 +-
lib/librte_table/rte_table_hash_key32.c | 2 +-
lib/librte_table/rte_table_hash_key8.c | 4 +-
lib/librte_table/rte_table_hash_lru.c | 4 +-
9 files changed, 55 insertions(+), 21 deletions(-)
mode change 100644 => 100755 examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 191cb15..e131a5b 100755
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -118,7 +118,7 @@ struct flow_table_entry {
uint32_t pad;
};
-rte_table_hash_op_hash hash_func[] = {
+rte_table_hash_op_hash_nomask hash_func[] = {
hash_default_key8,
hash_default_key16,
hash_default_key24,
diff --git a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
old mode 100644
new mode 100755
index 8cb2f0c..6b57f83
--- a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
@@ -52,7 +52,7 @@
struct pipeline_passthrough {
struct pipeline p;
struct pipeline_passthrough_params params;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint32_t swap_field0_offset[SWAP_DIM];
uint32_t swap_field1_offset[SWAP_DIM];
uint64_t swap_field_mask[SWAP_DIM];
@@ -677,7 +677,7 @@ pipeline_passthrough_parse_args(struct pipeline_passthrough_params *p,
return 0;
}
-static rte_table_hash_op_hash
+static rte_table_hash_op_hash_nomask
get_hash_function(struct pipeline_passthrough *p)
{
switch (p->params.dma_size) {
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 5f655ee..bb5b83d 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -98,6 +98,40 @@ extern "C" {
/** Hash function */
typedef uint64_t (*rte_table_hash_op_hash)(
void *key,
+ void *key_mask,
+ uint32_t key_size,
+ uint64_t seed);
+
+/** Hash table parameters */
+struct rte_table_hash_params {
+ /** Name */
+ const char *name;
+
+ /** Key size (number of bytes) */
+ uint32_t key_size;
+
+ /** Byte offset within packet meta-data where the key is located */
+ uint32_t key_offset;
+
+ /** Key mask */
+ uint8_t *key_mask;
+
+ /** Number of keys */
+ uint32_t n_keys;
+
+ /** Number of buckets */
+ uint32_t n_buckets;
+
+ /** Hash function */
+ rte_table_hash_op_hash f_hash;
+
+ /** Seed value for the hash function */
+ uint64_t seed;
+};
+
+/** Hash function */
+typedef uint64_t (*rte_table_hash_op_hash_nomask)(
+ void *key,
uint32_t key_size,
uint64_t seed);
@@ -121,7 +155,7 @@ struct rte_table_hash_ext_params {
uint32_t n_buckets_ext;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed value for the hash function */
uint64_t seed;
@@ -149,7 +183,7 @@ struct rte_table_hash_lru_params {
uint32_t n_buckets;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed value for the hash function */
uint64_t seed;
@@ -175,7 +209,7 @@ struct rte_table_hash_key8_lru_params {
uint32_t n_entries;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -204,7 +238,7 @@ struct rte_table_hash_key8_ext_params {
uint32_t n_entries_ext;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -233,7 +267,7 @@ struct rte_table_hash_key16_lru_params {
uint32_t n_entries;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -262,7 +296,7 @@ struct rte_table_hash_key16_ext_params {
uint32_t n_entries_ext;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -291,7 +325,7 @@ struct rte_table_hash_key32_lru_params {
uint32_t n_entries;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -318,7 +352,7 @@ struct rte_table_hash_key32_ext_params {
uint32_t n_entries_ext;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -344,7 +378,7 @@ struct rte_table_hash_cuckoo_params {
uint32_t n_keys;
/** Hash function used to calculate hash */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed value or Init value used by f_hash */
uint32_t seed;
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
index beb45c5..9b42423 100644
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -64,7 +64,7 @@ struct rte_table_hash {
uint32_t key_size;
uint32_t entry_size;
uint32_t n_keys;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint32_t seed;
uint32_t signature_offset;
uint32_t key_offset;
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
index c4824c3..72802b8 100755
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -104,7 +104,7 @@ struct rte_table_hash {
uint32_t n_keys;
uint32_t n_buckets;
uint32_t n_buckets_ext;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
uint32_t signature_offset;
uint32_t key_offset;
@@ -688,7 +688,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
uint8_t *key10, *key11; \
uint64_t bucket_mask = t->bucket_mask; \
- rte_table_hash_op_hash f_hash = t->f_hash; \
+ rte_table_hash_op_hash_nomask f_hash = t->f_hash; \
uint64_t seed = t->seed; \
uint32_t key_size = t->key_size; \
uint32_t key_offset = t->key_offset; \
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
index 4ed5c78..7f6651c 100644
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -86,7 +86,7 @@ struct rte_table_hash {
uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask[2];
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
/* Extendible buckets */
diff --git a/lib/librte_table/rte_table_hash_key32.c b/lib/librte_table/rte_table_hash_key32.c
index 31fe6fd..f8f6662 100644
--- a/lib/librte_table/rte_table_hash_key32.c
+++ b/lib/librte_table/rte_table_hash_key32.c
@@ -85,7 +85,7 @@ struct rte_table_hash {
uint32_t bucket_size;
uint32_t signature_offset;
uint32_t key_offset;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
/* Extendible buckets */
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
index f2b285d..4fbb02e 100644
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -83,7 +83,7 @@ struct rte_table_hash {
uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
/* Extendible buckets */
@@ -733,7 +733,7 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t hash_offset_buffer11; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
- rte_table_hash_op_hash f_hash = f->f_hash; \
+ rte_table_hash_op_hash_nomask f_hash = f->f_hash; \
uint64_t seed = f->seed; \
uint32_t key_offset = f->key_offset; \
\
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
index 0a85123..61050f3 100755
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -84,7 +84,7 @@ struct rte_table_hash {
uint32_t entry_size;
uint32_t n_keys;
uint32_t n_buckets;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
uint32_t signature_offset;
uint32_t key_offset;
@@ -619,7 +619,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
uint8_t *key10, *key11; \
uint64_t bucket_mask = t->bucket_mask; \
- rte_table_hash_op_hash f_hash = t->f_hash; \
+ rte_table_hash_op_hash_nomask f_hash = t->f_hash; \
uint64_t seed = t->seed; \
uint32_t key_size = t->key_size; \
uint32_t key_offset = t->key_offset; \
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 08/18] librte_table: rework variable size key ext hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (6 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 07/18] librte_table: add unified params structure and mask-based hash func Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 09/18] librte_table: rework variable size key lru hash table Cristian Dumitrescu
` (28 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the variable size key extendible bucket (EXT) hash
table to use the mask-based hash function and the unified
parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 12 +-
lib/librte_table/rte_table_hash.h | 34 ----
lib/librte_table/rte_table_hash_ext.c | 179 ++++++++++++++-------
test/test-pipeline/pipeline_hash.c | 10 +-
4 files changed, 130 insertions(+), 105 deletions(-)
mode change 100644 => 100755 test/test-pipeline/pipeline_hash.c
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index e131a5b..4a4007c 100755
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -516,16 +516,16 @@ static void *pipeline_fc_init(struct pipeline_params *params,
.seed = 0,
};
- struct rte_table_hash_ext_params
- table_hash_params = {
+ struct rte_table_hash_params table_hash_params = {
+ .name = p->name,
.key_size = p_fc->key_size,
+ .key_offset = p_fc->key_offset,
+ .key_mask = (p_fc->key_mask_present) ?
+ p_fc->key_mask : NULL,
.n_keys = p_fc->n_flows,
.n_buckets = p_fc->n_flows / 4,
- .n_buckets_ext = p_fc->n_flows / 4,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
+ .f_hash = (rte_table_hash_op_hash)hash_func[(p_fc->key_size / 8) - 1],
.seed = 0,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
};
struct rte_pipeline_table_params table_params = {
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index bb5b83d..0024b99 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -135,40 +135,6 @@ typedef uint64_t (*rte_table_hash_op_hash_nomask)(
uint32_t key_size,
uint64_t seed);
-/**
- * Hash tables with configurable key size
- *
- */
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_ext_params {
- /** Key size (number of bytes) */
- uint32_t key_size;
-
- /** Maximum number of keys */
- uint32_t n_keys;
-
- /** Number of hash table buckets. Each bucket stores up to 4 keys. */
- uint32_t n_buckets;
-
- /** Number of hash table bucket extensions. Each bucket extension has
- space for 4 keys and each bucket can have 0, 1 or more extensions. */
- uint32_t n_buckets_ext;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed value for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
extern struct rte_table_ops rte_table_hash_ext_ops;
/** LRU hash table parameters */
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
index 72802b8..0a743ad 100755
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -104,9 +104,8 @@ struct rte_table_hash {
uint32_t n_keys;
uint32_t n_buckets;
uint32_t n_buckets_ext;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
/* Internal */
@@ -120,6 +119,7 @@ struct rte_table_hash {
struct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];
/* Tables */
+ uint64_t *key_mask;
struct bucket *buckets;
struct bucket *buckets_ext;
uint8_t *key_mem;
@@ -132,29 +132,53 @@ struct rte_table_hash {
};
static int
-check_params_create(struct rte_table_hash_ext_params *params)
+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)
{
- uint32_t n_buckets_min;
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ if (a64[i] != (b64[i] & b_mask64[i]))
+ return 1;
+
+ return 0;
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ dst64[i] = src64[i] & src_mask64[i];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
/* key_size */
- if ((params->key_size == 0) ||
+ if ((params->key_size < sizeof(uint64_t)) ||
(!rte_is_power_of_2(params->key_size))) {
RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
return -EINVAL;
}
/* n_keys */
- if ((params->n_keys == 0) ||
- (!rte_is_power_of_2(params->n_keys))) {
+ if (params->n_keys == 0) {
RTE_LOG(ERR, TABLE, "%s: n_keys invalid value\n", __func__);
return -EINVAL;
}
/* n_buckets */
- n_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;
if ((params->n_buckets == 0) ||
- (!rte_is_power_of_2(params->n_keys)) ||
- (params->n_buckets < n_buckets_min)) {
+ (!rte_is_power_of_2(params->n_buckets))) {
RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -171,15 +195,13 @@ check_params_create(struct rte_table_hash_ext_params *params)
static void *
rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_ext_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *t;
- uint32_t total_size, table_meta_sz;
- uint32_t bucket_sz, bucket_ext_sz, key_sz;
- uint32_t key_stack_sz, bkt_ext_stack_sz, data_sz;
- uint32_t bucket_offset, bucket_ext_offset, key_offset;
- uint32_t key_stack_offset, bkt_ext_stack_offset, data_offset;
- uint32_t i;
+ uint64_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz;
+ uint64_t key_stack_sz, bkt_ext_stack_sz, data_sz, total_size;
+ uint64_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset;
+ uint64_t key_stack_offset, bkt_ext_stack_offset, data_offset;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
if ((check_params_create(p) != 0) ||
@@ -188,38 +210,66 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
(sizeof(struct bucket) != (RTE_CACHE_LINE_SIZE / 2)))
return NULL;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
+
/* Memory allocation */
table_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));
+ key_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);
bucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));
bucket_ext_sz =
- RTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(struct bucket));
+ RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(struct bucket));
key_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);
key_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));
bkt_ext_stack_sz =
- RTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(uint32_t));
+ RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
data_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
- total_size = table_meta_sz + bucket_sz + bucket_ext_sz + key_sz +
- key_stack_sz + bkt_ext_stack_sz + data_sz;
+ total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz +
+ key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz;
- t = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ t = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (t == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table memory footprint is "
- "%u bytes\n", __func__, p->key_size, total_size);
+ RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table %s memory "
+ "footprint is %" PRIu64 " bytes\n",
+ __func__, p->key_size, p->name, total_size);
/* Memory initialization */
t->key_size = p->key_size;
t->entry_size = entry_size;
t->n_keys = p->n_keys;
t->n_buckets = p->n_buckets;
- t->n_buckets_ext = p->n_buckets_ext;
+ t->n_buckets_ext = n_buckets_ext;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
/* Internal */
@@ -228,13 +278,15 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
t->data_size_shl = __builtin_ctzl(entry_size);
/* Tables */
- bucket_offset = 0;
+ key_mask_offset = 0;
+ bucket_offset = key_mask_offset + key_mask_sz;
bucket_ext_offset = bucket_offset + bucket_sz;
key_offset = bucket_ext_offset + bucket_ext_sz;
key_stack_offset = key_offset + key_sz;
bkt_ext_stack_offset = key_stack_offset + key_stack_sz;
data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz;
+ t->key_mask = (uint64_t *) &t->memory[key_mask_offset];
t->buckets = (struct bucket *) &t->memory[bucket_offset];
t->buckets_ext = (struct bucket *) &t->memory[bucket_ext_offset];
t->key_mem = &t->memory[key_offset];
@@ -242,6 +294,12 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
t->bkt_ext_stack = (uint32_t *) &t->memory[bkt_ext_stack_offset];
t->data_mem = &t->memory[data_offset];
+ /* Key mask */
+ if (p->key_mask == NULL)
+ memset(t->key_mask, 0xFF, p->key_size);
+ else
+ memcpy(t->key_mask, p->key_mask, p->key_size);
+
/* Key stack */
for (i = 0; i < t->n_keys; i++)
t->key_stack[i] = t->n_keys - 1 - i;
@@ -277,7 +335,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -290,7 +348,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
uint8_t *bkt_key =
&t->key_mem[bkt_key_index << t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -327,7 +385,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
bkt->sig[i] = (uint16_t) sig;
bkt->key_pos[i] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
*key_found = 0;
@@ -358,7 +416,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
/* Install new key into bucket */
bkt->sig[0] = (uint16_t) sig;
bkt->key_pos[0] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
*key_found = 0;
@@ -378,7 +436,7 @@ void *entry)
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -392,7 +450,7 @@ void *entry)
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -457,7 +515,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
+ sig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
@@ -471,8 +529,8 @@ static int rte_table_hash_ext_lookup_unoptimized(
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
- t->key_size) == 0)) {
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key,
+ t->key_mask, t->key_size) == 0)) {
uint8_t *data = &t->data_mem[
bkt_key_index << t->data_size_shl];
@@ -571,11 +629,12 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\
uint64_t *bkt_key = (uint64_t *) key; \
+ uint64_t *key_mask = f->key_mask; \
\
switch (f->key_size) { \
case 8: \
{ \
- uint64_t xor = pkt_key[0] ^ bkt_key[0]; \
+ uint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
match_key = 0; \
if (xor == 0) \
match_key = 1; \
@@ -586,8 +645,8 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[2], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
or = xor[0] | xor[1]; \
match_key = 0; \
if (or == 0) \
@@ -599,10 +658,10 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[4], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
or = xor[0] | xor[1] | xor[2] | xor[3]; \
match_key = 0; \
if (or == 0) \
@@ -614,14 +673,14 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[8], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- xor[4] = pkt_key[4] ^ bkt_key[4]; \
- xor[5] = pkt_key[5] ^ bkt_key[5]; \
- xor[6] = pkt_key[6] ^ bkt_key[6]; \
- xor[7] = pkt_key[7] ^ bkt_key[7]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ xor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4]; \
+ xor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5]; \
+ xor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6]; \
+ xor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7]; \
or = xor[0] | xor[1] | xor[2] | xor[3] | \
xor[4] | xor[5] | xor[6] | xor[7]; \
match_key = 0; \
@@ -632,7 +691,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
\
default: \
match_key = 0; \
- if (memcmp(pkt_key, bkt_key, f->key_size) == 0) \
+ if (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0) \
match_key = 1; \
} \
}
@@ -688,20 +747,20 @@ static int rte_table_hash_ext_lookup_unoptimized(
struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
uint8_t *key10, *key11; \
uint64_t bucket_mask = t->bucket_mask; \
- rte_table_hash_op_hash_nomask f_hash = t->f_hash; \
+ rte_table_hash_op_hash f_hash = t->f_hash; \
uint64_t seed = t->seed; \
uint32_t key_size = t->key_size; \
uint32_t key_offset = t->key_offset; \
\
mbuf10 = pkts[pkt10_index]; \
key10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset); \
- sig10 = (uint64_t) f_hash(key10, key_size, seed); \
+ sig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed); \
bkt10_index = sig10 & bucket_mask; \
bkt10 = &buckets[bkt10_index]; \
\
mbuf11 = pkts[pkt11_index]; \
key11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset); \
- sig11 = (uint64_t) f_hash(key11, key_size, seed); \
+ sig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed); \
bkt11_index = sig11 & bucket_mask; \
bkt11 = &buckets[bkt11_index]; \
\
@@ -969,7 +1028,7 @@ rte_table_hash_ext_stats_read(void *table, struct rte_table_stats *stats, int cl
return 0;
}
-struct rte_table_ops rte_table_hash_ext_ops = {
+struct rte_table_ops rte_table_hash_ext_ops = {
.f_create = rte_table_hash_ext_create,
.f_free = rte_table_hash_ext_free,
.f_add = rte_table_hash_ext_entry_add,
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
old mode 100644
new mode 100755
index 2f8c625..13d309d
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -175,15 +175,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_KEY16_EXT:
case e_APP_PIPELINE_HASH_KEY32_EXT:
{
- struct rte_table_hash_ext_params table_hash_params = {
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
.key_size = key_size,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
.n_keys = 1 << 24,
.n_buckets = 1 << 22,
- .n_buckets_ext = 1 << 21,
- .f_hash = test_hash,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
};
struct rte_pipeline_table_params table_params = {
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 09/18] librte_table: rework variable size key lru hash table
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (7 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 08/18] librte_table: rework variable size key ext hash tables Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 10/18] librte_table: rework 8-byte key hash tables Cristian Dumitrescu
` (27 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the variable size key LRU hash table to use the
mask-based hash function and the unified parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 26 ----
lib/librte_table/rte_table_hash_lru.c | 277 +++++++++++++++++++++-------------
test/test-pipeline/pipeline_hash.c | 9 +-
3 files changed, 177 insertions(+), 135 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 0024b99..fb5f4d7 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -137,32 +137,6 @@ typedef uint64_t (*rte_table_hash_op_hash_nomask)(
extern struct rte_table_ops rte_table_hash_ext_ops;
-/** LRU hash table parameters */
-struct rte_table_hash_lru_params {
- /** Key size (number of bytes) */
- uint32_t key_size;
-
- /** Maximum number of keys */
- uint32_t n_keys;
-
- /** Number of hash table buckets. Each bucket stores up to 4 keys. */
- uint32_t n_buckets;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed value for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
extern struct rte_table_ops rte_table_hash_lru_ops;
/**
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
index 61050f3..eebc1cd 100755
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -84,9 +84,8 @@ struct rte_table_hash {
uint32_t entry_size;
uint32_t n_keys;
uint32_t n_buckets;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
/* Internal */
@@ -99,6 +98,7 @@ struct rte_table_hash {
struct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];
/* Tables */
+ uint64_t *key_mask;
struct bucket *buckets;
uint8_t *key_mem;
uint8_t *data_mem;
@@ -109,12 +109,39 @@ struct rte_table_hash {
};
static int
-check_params_create(struct rte_table_hash_lru_params *params)
+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)
{
- uint32_t n_buckets_min;
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ if (a64[i] != (b64[i] & b_mask64[i]))
+ return 1;
+
+ return 0;
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ dst64[i] = src64[i] & src_mask64[i];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
/* key_size */
- if ((params->key_size == 0) ||
+ if ((params->key_size < sizeof(uint64_t)) ||
(!rte_is_power_of_2(params->key_size))) {
RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
return -EINVAL;
@@ -128,10 +155,8 @@ check_params_create(struct rte_table_hash_lru_params *params)
}
/* n_buckets */
- n_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;
if ((params->n_buckets == 0) ||
- (!rte_is_power_of_2(params->n_keys)) ||
- (params->n_buckets < n_buckets_min)) {
+ (!rte_is_power_of_2(params->n_keys))) {
RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -148,13 +173,13 @@ check_params_create(struct rte_table_hash_lru_params *params)
static void *
rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_lru_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *t;
- uint32_t total_size, table_meta_sz;
- uint32_t bucket_sz, key_sz, key_stack_sz, data_sz;
- uint32_t bucket_offset, key_offset, key_stack_offset, data_offset;
- uint32_t i;
+ uint64_t table_meta_sz, key_mask_sz, bucket_sz, key_sz, key_stack_sz;
+ uint64_t data_sz, total_size;
+ uint64_t key_mask_offset, bucket_offset, key_offset, key_stack_offset;
+ uint64_t data_offset;
+ uint32_t n_buckets, i;
/* Check input parameters */
if ((check_params_create(p) != 0) ||
@@ -164,33 +189,65 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
return NULL;
}
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
+
/* Memory allocation */
table_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));
- bucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));
+ key_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);
+ bucket_sz = RTE_CACHE_LINE_ROUNDUP(n_buckets * sizeof(struct bucket));
key_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);
key_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));
data_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
- total_size = table_meta_sz + bucket_sz + key_sz + key_stack_sz +
- data_sz;
+ total_size = table_meta_sz + key_mask_sz + bucket_sz + key_sz +
+ key_stack_sz + data_sz;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE,
+ "%s: Cannot allocate %" PRIu64 " bytes for hash "
+ "table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
- t = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ t = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (t == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %" PRIu64 " bytes for hash "
+ "table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table memory footprint is "
- "%u bytes\n", __func__, p->key_size, total_size);
+ RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table %s memory footprint"
+ " is %" PRIu64 " bytes\n",
+ __func__, p->key_size, p->name, total_size);
/* Memory initialization */
t->key_size = p->key_size;
t->entry_size = entry_size;
t->n_keys = p->n_keys;
- t->n_buckets = p->n_buckets;
+ t->n_buckets = n_buckets;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
/* Internal */
@@ -199,16 +256,24 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
t->data_size_shl = __builtin_ctzl(entry_size);
/* Tables */
- bucket_offset = 0;
+ key_mask_offset = 0;
+ bucket_offset = key_mask_offset + key_mask_sz;
key_offset = bucket_offset + bucket_sz;
key_stack_offset = key_offset + key_sz;
data_offset = key_stack_offset + key_stack_sz;
+ t->key_mask = (uint64_t *) &t->memory[key_mask_offset];
t->buckets = (struct bucket *) &t->memory[bucket_offset];
t->key_mem = &t->memory[key_offset];
t->key_stack = (uint32_t *) &t->memory[key_stack_offset];
t->data_mem = &t->memory[data_offset];
+ /* Key mask */
+ if (p->key_mask == NULL)
+ memset(t->key_mask, 0xFF, p->key_size);
+ else
+ memcpy(t->key_mask, p->key_mask, p->key_size);
+
/* Key stack */
for (i = 0; i < t->n_keys; i++)
t->key_stack[i] = t->n_keys - 1 - i;
@@ -246,7 +311,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -258,8 +323,8 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key, t->key_size)
- == 0)) {
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
+ t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -292,7 +357,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
bkt->sig[i] = (uint16_t) sig;
bkt->key_pos[i] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
lru_update(bkt, i);
@@ -311,7 +376,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint8_t *data = &t->data_mem[bkt_key_index << t->data_size_shl];
bkt->sig[pos] = (uint16_t) sig;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
lru_update(bkt, pos);
@@ -330,7 +395,7 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -343,14 +408,15 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,
t->key_size_shl];
if ((sig == bkt_sig) &&
- (memcmp(key, bkt_key, t->key_size) == 0)) {
+ (keycmp(bkt_key, key, t->key_mask, t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
bkt->sig[i] = 0;
t->key_stack[t->key_stack_tos++] = bkt_key_index;
*key_found = 1;
- memcpy(entry, data, t->entry_size);
+ if (entry)
+ memcpy(entry, data, t->entry_size);
return 0;
}
}
@@ -386,7 +452,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
+ sig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
@@ -399,7 +465,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -497,74 +563,75 @@ static int rte_table_hash_lru_lookup_unoptimized(
match_pos = (LUT_MATCH_POS >> (mask_all << 1)) & 3; \
}
-#define lookup_cmp_key(mbuf, key, match_key, f) \
-{ \
+#define lookup_cmp_key(mbuf, key, match_key, f) \
+{ \
uint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\
- uint64_t *bkt_key = (uint64_t *) key; \
- \
- switch (f->key_size) { \
- case 8: \
- { \
- uint64_t xor = pkt_key[0] ^ bkt_key[0]; \
- match_key = 0; \
- if (xor == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 16: \
- { \
- uint64_t xor[2], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- or = xor[0] | xor[1]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 32: \
- { \
- uint64_t xor[4], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- or = xor[0] | xor[1] | xor[2] | xor[3]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 64: \
- { \
- uint64_t xor[8], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- xor[4] = pkt_key[4] ^ bkt_key[4]; \
- xor[5] = pkt_key[5] ^ bkt_key[5]; \
- xor[6] = pkt_key[6] ^ bkt_key[6]; \
- xor[7] = pkt_key[7] ^ bkt_key[7]; \
- or = xor[0] | xor[1] | xor[2] | xor[3] | \
- xor[4] | xor[5] | xor[6] | xor[7]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- default: \
- match_key = 0; \
- if (memcmp(pkt_key, bkt_key, f->key_size) == 0) \
- match_key = 1; \
- } \
+ uint64_t *bkt_key = (uint64_t *) key; \
+ uint64_t *key_mask = f->key_mask; \
+ \
+ switch (f->key_size) { \
+ case 8: \
+ { \
+ uint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ match_key = 0; \
+ if (xor == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 16: \
+ { \
+ uint64_t xor[2], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ or = xor[0] | xor[1]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 32: \
+ { \
+ uint64_t xor[4], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ or = xor[0] | xor[1] | xor[2] | xor[3]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 64: \
+ { \
+ uint64_t xor[8], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ xor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4]; \
+ xor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5]; \
+ xor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6]; \
+ xor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7]; \
+ or = xor[0] | xor[1] | xor[2] | xor[3] | \
+ xor[4] | xor[5] | xor[6] | xor[7]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ default: \
+ match_key = 0; \
+ if (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0) \
+ match_key = 1; \
+ } \
}
#define lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index)\
@@ -619,20 +686,20 @@ static int rte_table_hash_lru_lookup_unoptimized(
struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
uint8_t *key10, *key11; \
uint64_t bucket_mask = t->bucket_mask; \
- rte_table_hash_op_hash_nomask f_hash = t->f_hash; \
+ rte_table_hash_op_hash f_hash = t->f_hash; \
uint64_t seed = t->seed; \
uint32_t key_size = t->key_size; \
uint32_t key_offset = t->key_offset; \
\
mbuf10 = pkts[pkt10_index]; \
key10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset);\
- sig10 = (uint64_t) f_hash(key10, key_size, seed); \
+ sig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed);\
bkt10_index = sig10 & bucket_mask; \
bkt10 = &buckets[bkt10_index]; \
\
mbuf11 = pkts[pkt11_index]; \
key11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset);\
- sig11 = (uint64_t) f_hash(key11, key_size, seed); \
+ sig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed);\
bkt11_index = sig11 & bucket_mask; \
bkt11 = &buckets[bkt11_index]; \
\
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 13d309d..3c48640 100755
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -204,14 +204,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_KEY16_LRU:
case e_APP_PIPELINE_HASH_KEY32_LRU:
{
- struct rte_table_hash_lru_params table_hash_params = {
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
.key_size = key_size,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
.n_keys = 1 << 24,
.n_buckets = 1 << 22,
- .f_hash = test_hash,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
};
struct rte_pipeline_table_params table_params = {
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 10/18] librte_table: rework 8-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (8 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 09/18] librte_table: rework variable size key lru hash table Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 11/18] librte_table: rework 16-byte " Cristian Dumitrescu
` (26 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the 8-byte key hash tables (both the extendible
bucket and LRU)to use the mask-based hash function and the
unified parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 14 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 14 +-
lib/librte_table/rte_table_hash.h | 54 ----
lib/librte_table/rte_table_hash_key8.c | 328 +++++++++++----------
test/test-pipeline/pipeline_hash.c | 21 +-
test/test/test_table_combined.c | 39 ++-
test/test/test_table_tables.c | 101 ++-----
7 files changed, 242 insertions(+), 329 deletions(-)
mode change 100644 => 100755 examples/ip_pipeline/pipeline/pipeline_routing_be.c
mode change 100644 => 100755 test/test/test_table_combined.c
mode change 100644 => 100755 test/test/test_table_tables.c
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 4a4007c..47e6d35 100755
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -492,18 +492,6 @@ static void *pipeline_fc_init(struct pipeline_params *params,
/* Tables */
p->n_tables = 1;
{
- struct rte_table_hash_key8_ext_params
- table_hash_key8_params = {
- .n_entries = p_fc->n_flows,
- .n_entries_ext = p_fc->n_flows,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
- .key_mask = (p_fc->key_mask_present) ?
- p_fc->key_mask : NULL,
- .seed = 0,
- };
-
struct rte_table_hash_key16_ext_params
table_hash_key16_params = {
.n_entries = p_fc->n_flows,
@@ -543,7 +531,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
table_params.ops = &rte_table_hash_key8_ext_ops;
- table_params.arg_create = &table_hash_key8_params;
+ table_params.arg_create = &table_hash_params;
break;
case 16:
diff --git a/examples/ip_pipeline/pipeline/pipeline_routing_be.c b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
old mode 100644
new mode 100755
index aa7f05e..796cde7
--- a/examples/ip_pipeline/pipeline/pipeline_routing_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
@@ -1349,13 +1349,15 @@ pipeline_routing_init(struct pipeline_params *params,
/* ARP table configuration */
if (p_rt->params.n_arp_entries) {
- struct rte_table_hash_key8_ext_params table_arp_params = {
- .n_entries = p_rt->params.n_arp_entries,
- .n_entries_ext = p_rt->params.n_arp_entries,
- .f_hash = hash_default_key8,
- .seed = 0,
- .signature_offset = 0, /* Unused */
+ struct rte_table_hash_params table_arp_params = {
+ .name = p->name,
+ .key_size = 8,
.key_offset = p_rt->params.arp_key_offset,
+ .key_mask = NULL,
+ .n_keys = p_rt->params.n_arp_entries,
+ .n_buckets = p_rt->params.n_arp_entries / 4,
+ .f_hash = (rte_table_hash_op_hash)hash_default_key8,
+ .seed = 0,
};
struct rte_pipeline_table_params table_params = {
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index fb5f4d7..6e25054 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -139,62 +139,8 @@ extern struct rte_table_ops rte_table_hash_ext_ops;
extern struct rte_table_ops rte_table_hash_lru_ops;
-/**
- * 8-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key8_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
extern struct rte_table_ops rte_table_hash_key8_lru_ops;
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key8_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
extern struct rte_table_ops rte_table_hash_key8_ext_ops;
/**
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
index 4fbb02e..a44f75c 100644
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 8
+#define KEY_SIZE 8
+
+#define KEYS_PER_BUCKET 4
#ifdef RTE_TABLE_STATS_COLLECT
@@ -76,14 +78,12 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
/* Extendible buckets */
@@ -96,10 +96,46 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return a64[0] != (b64[0] & b_mask64[0]);
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_buckets */
+ if ((params->n_buckets == 0) ||
+ (!rte_is_power_of_2(params->n_buckets))) {
+ RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -116,47 +152,68 @@ check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {
static void *
rte_table_hash_create_key8_lru(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_key8_lru_params *p =
- (struct rte_table_hash_key8_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;
- uint32_t total_size, i;
+ uint64_t bucket_size, total_size;
+ uint32_t n_buckets, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
- ((sizeof(struct rte_bucket_4_8) % 64) != 0)) {
+ ((sizeof(struct rte_bucket_4_8) % 64) != 0))
return NULL;
- }
- n_entries_per_bucket = 4;
- key_size = 8;
+
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *
- entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + n_buckets *
- bucket_size_cl * RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +
+ KEYS_PER_BUCKET * entry_size);
+ total_size = sizeof(struct rte_table_hash) + n_buckets * bucket_size;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -205,7 +262,7 @@ rte_table_hash_entry_add_key8_lru(
uint64_t signature, mask, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -213,10 +270,10 @@ rte_table_hash_entry_add_key8_lru(
/* Key is present in the bucket */
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -235,7 +292,7 @@ rte_table_hash_entry_add_key8_lru(
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature |= mask;
- bucket->key[i] = *((uint64_t *) key);
+ keycpy(&bucket->key[i], key, &f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -247,10 +304,10 @@ rte_table_hash_entry_add_key8_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
- bucket->key[pos] = *((uint64_t *) key);
+ keycpy(&bucket->key[pos], key, &f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
- *key_found = 0;
+ *key_found = 0;
*entry_ptr = (void *) &bucket->data[pos * f->entry_size];
return 0;
@@ -268,7 +325,7 @@ rte_table_hash_entry_delete_key8_lru(
uint64_t signature, mask;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -276,10 +333,10 @@ rte_table_hash_entry_delete_key8_lru(
/* Key is present in the bucket */
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature &= ~mask;
@@ -296,79 +353,71 @@ rte_table_hash_entry_delete_key8_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key8_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
- __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key8_ext(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_key8_ext_params *p =
- (struct rte_table_hash_key8_ext_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size;
- uint32_t bucket_size_cl, stack_size_cl, total_size, i;
+ uint64_t bucket_size, stack_size, total_size;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_8) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 8;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *
- entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + ((n_buckets +
- n_buckets_ext) * bucket_size_cl + stack_size_cl) *
- RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
+ total_size = sizeof(struct rte_table_hash) +
+ (p->n_buckets + n_buckets_ext) * bucket_size + stack_size;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
- f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->n_buckets = p->n_buckets;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -376,7 +425,7 @@ rte_table_hash_create_key8_ext(void *params, int socket_id, uint32_t entry_size)
f->n_buckets_ext = n_buckets_ext;
f->stack_pos = n_buckets_ext;
f->stack = (uint32_t *)
- &f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
+ &f->memory[(p->n_buckets + n_buckets_ext) * f->bucket_size];
if (p->key_mask != NULL)
f->key_mask = ((uint64_t *)p->key_mask)[0];
@@ -417,7 +466,7 @@ rte_table_hash_entry_add_key8_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -428,10 +477,10 @@ rte_table_hash_entry_add_key8_ext(
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -456,7 +505,7 @@ rte_table_hash_entry_add_key8_ext(
f->entry_size];
bucket->signature |= mask;
- bucket->key[i] = *((uint64_t *) key);
+ keycpy(&bucket->key[i], key, &f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -476,7 +525,7 @@ rte_table_hash_entry_add_key8_ext(
bucket_prev->next_valid = 1;
bucket->signature = 1;
- bucket->key[0] = *((uint64_t *) key);
+ keycpy(&bucket->key[0], key, &f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -498,7 +547,7 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -510,10 +559,10 @@ rte_table_hash_entry_delete_key8_ext(
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -546,16 +595,17 @@ rte_table_hash_entry_delete_key8_ext(
return 0;
}
-#define lookup_key8_cmp(key_in, bucket, pos) \
+#define lookup_key8_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4], signature; \
+ uint64_t xor[4], signature, k; \
\
signature = ~bucket->signature; \
\
- xor[0] = (key_in[0] ^ bucket->key[0]) | (signature & 1);\
- xor[1] = (key_in[0] ^ bucket->key[1]) | (signature & 2);\
- xor[2] = (key_in[0] ^ bucket->key[2]) | (signature & 4);\
- xor[3] = (key_in[0] ^ bucket->key[3]) | (signature & 8);\
+ k = key_in[0] & f->key_mask; \
+ xor[0] = (k ^ bucket->key[0]) | (signature & 1); \
+ xor[1] = (k ^ bucket->key[1]) | (signature & 2); \
+ xor[2] = (k ^ bucket->key[2]) | (signature & 4); \
+ xor[3] = (k ^ bucket->key[3]) | (signature & 8); \
\
pos = 4; \
if (xor[0] == 0) \
@@ -586,12 +636,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\
- hash_key_buffer = *key & f->key_mask; \
- signature = f->f_hash(&hash_key_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed); \
+ signature = f->f_hash(key, &f->key_mask, KEY_SIZE, f->seed); \
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_8 *) \
&f->memory[bucket_index * f->bucket_size]; \
@@ -605,12 +652,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer = key[0] & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket2, pos); \
+ lookup_key8_cmp(key, bucket2, pos, f); \
\
pkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -629,12 +673,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer = *key & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket2, pos); \
+ lookup_key8_cmp(key, bucket2, pos, f); \
\
pkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -658,13 +699,10 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
- hash_key_buffer = (*key) & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket, pos); \
+ lookup_key8_cmp(key, bucket, pos, f); \
\
pkt_mask = ((bucket->signature >> pos) & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -729,28 +767,22 @@ rte_table_hash_entry_delete_key8_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\
{ \
uint64_t *key10, *key11; \
- uint64_t hash_offset_buffer10; \
- uint64_t hash_offset_buffer11; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
- rte_table_hash_op_hash_nomask f_hash = f->f_hash; \
+ rte_table_hash_op_hash f_hash = f->f_hash; \
uint64_t seed = f->seed; \
uint32_t key_offset = f->key_offset; \
\
key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, key_offset);\
key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, key_offset);\
- hash_offset_buffer10 = *key10 & f->key_mask; \
- hash_offset_buffer11 = *key11 & f->key_mask; \
\
- signature10 = f_hash(&hash_offset_buffer10, \
- RTE_TABLE_HASH_KEY_SIZE, seed); \
+ signature10 = f_hash(key10, &f->key_mask, KEY_SIZE, seed); \
bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_8 *) \
&f->memory[bucket10_index * f->bucket_size]; \
rte_prefetch0(bucket10); \
\
- signature11 = f_hash(&hash_offset_buffer11, \
- RTE_TABLE_HASH_KEY_SIZE, seed); \
+ signature11 = f_hash(key11, &f->key_mask, KEY_SIZE, seed); \
bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_8 *) \
&f->memory[bucket11_index * f->bucket_size]; \
@@ -763,17 +795,13 @@ rte_table_hash_entry_delete_key8_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask; \
uint64_t *key20, *key21; \
- uint64_t hash_offset_buffer20; \
- uint64_t hash_offset_buffer21; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_offset_buffer20 = *key20 & f->key_mask; \
- hash_offset_buffer21 = *key21 & f->key_mask; \
\
- lookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\
- lookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\
+ lookup_key8_cmp(key20, bucket20, pos20, f); \
+ lookup_key8_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\
pkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\
@@ -796,17 +824,13 @@ rte_table_hash_entry_delete_key8_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\
uint64_t *key20, *key21; \
- uint64_t hash_offset_buffer20; \
- uint64_t hash_offset_buffer21; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_offset_buffer20 = *key20 & f->key_mask; \
- hash_offset_buffer21 = *key21 & f->key_mask; \
\
- lookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\
- lookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\
+ lookup_key8_cmp(key20, bucket20, pos20, f); \
+ lookup_key8_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\
pkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\
@@ -954,7 +978,7 @@ rte_table_hash_lookup_key8_lru(
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key8_lru() */
+} /* lookup LRU */
static int
rte_table_hash_lookup_key8_ext(
@@ -1104,7 +1128,7 @@ rte_table_hash_lookup_key8_ext(
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key8_ext() */
+} /* lookup EXT */
static int
rte_table_hash_key8_stats_read(void *table, struct rte_table_stats *stats, int clear)
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 3c48640..f542506 100755
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -231,13 +231,14 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_EXT:
{
- struct rte_table_hash_key8_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
- .f_hash = test_hash,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
};
@@ -257,12 +258,14 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_LRU:
{
- struct rte_table_hash_key8_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
- .f_hash = test_hash,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
};
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
old mode 100644
new mode 100755
index f65d34a..ddcd14b
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -441,12 +441,15 @@ test_table_hash8lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key8_lru_params key8lru_params = {
- .n_entries = 1<<24,
- .f_hash = pipeline_test_hash,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key8lru_params = {
+ .name = "TABLE",
+ .key_size = 8,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key8lru[8];
@@ -475,14 +478,14 @@ test_table_hash8lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key8lru_params.n_entries = 0;
+ key8lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8lru_params.n_entries = 1<<16;
+ key8lru_params.n_keys = 1<<16;
key8lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key8_lru_ops,
@@ -616,14 +619,15 @@ test_table_hash8ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key8_ext_params key8ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key8ext_params = {
+ .name = "TABLE",
+ .key_size = 8,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key8ext[8];
@@ -652,14 +656,14 @@ test_table_hash8ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key8ext_params.n_entries = 0;
+ key8ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8ext_params.n_entries = 1<<16;
+ key8ext_params.n_keys = 1<<16;
key8ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key8_ext_ops,
@@ -667,13 +671,6 @@ test_table_hash8ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8ext_params.f_hash = pipeline_test_hash;
- key8ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key8_ext_ops,
- (void *)&key8ext_params, (void *)key8ext, &table_packets, NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
old mode 100644
new mode 100755
index efbd0c6..7f85587
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -67,9 +67,9 @@ unsigned n_table_tests = RTE_DIM(table_tests);
/* Function prototypes */
static int
-test_table_hash_lru_generic(struct rte_table_ops *ops);
+test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size);
static int
-test_table_hash_ext_generic(struct rte_table_ops *ops);
+test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size);
struct rte_bucket_4_8 {
/* Cache line 0 */
@@ -655,7 +655,7 @@ test_table_lpm_ipv6(void)
}
static int
-test_table_hash_lru_generic(struct rte_table_ops *ops)
+test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size)
{
int status, i;
uint64_t expected_mask = 0, result_mask;
@@ -667,43 +667,31 @@ test_table_hash_lru_generic(struct rte_table_ops *ops)
int key_found;
/* Initialize params and create tables */
- struct rte_table_hash_key8_lru_params hash_params = {
- .n_entries = 1 << 10,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(1),
+ struct rte_table_hash_params hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 10,
+ .n_buckets = 1 << 10,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
- hash_params.n_entries = 0;
+ hash_params.n_keys = 0;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -1;
- hash_params.n_entries = 1 << 10;
- hash_params.signature_offset = APP_METADATA_OFFSET(1);
-
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -2;
-
- hash_params.signature_offset = APP_METADATA_OFFSET(0);
- hash_params.key_offset = APP_METADATA_OFFSET(1);
-
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -3;
-
- hash_params.key_offset = APP_METADATA_OFFSET(32);
+ hash_params.n_keys = 1 << 10;
hash_params.f_hash = NULL;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -4;
- hash_params.f_hash = pipeline_test_hash;
+ hash_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
table = ops->f_create(&hash_params, 0, 1);
if (table == NULL)
@@ -770,7 +758,7 @@ test_table_hash_lru_generic(struct rte_table_ops *ops)
}
static int
-test_table_hash_ext_generic(struct rte_table_ops *ops)
+test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size)
{
int status, i;
uint64_t expected_mask = 0, result_mask;
@@ -782,35 +770,24 @@ test_table_hash_ext_generic(struct rte_table_ops *ops)
void *entry_ptr;
/* Initialize params and create tables */
- struct rte_table_hash_key8_ext_params hash_params = {
- .n_entries = 1 << 10,
- .n_entries_ext = 1 << 4,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(1),
+ struct rte_table_hash_params hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 10,
+ .n_buckets = 1 << 10,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
- hash_params.n_entries = 0;
+ hash_params.n_keys = 0;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -1;
- hash_params.n_entries = 1 << 10;
- hash_params.n_entries_ext = 0;
- table = ops->f_create(&hash_params, 0, 1);
- if (table != NULL)
- return -2;
-
- hash_params.n_entries_ext = 1 << 4;
- hash_params.signature_offset = APP_METADATA_OFFSET(1);
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -2;
-
- hash_params.signature_offset = APP_METADATA_OFFSET(0);
+ hash_params.n_keys = 1 << 10;
hash_params.key_offset = APP_METADATA_OFFSET(1);
table = ops->f_create(&hash_params, 0, 1);
@@ -824,7 +801,7 @@ test_table_hash_ext_generic(struct rte_table_ops *ops)
if (table != NULL)
return -4;
- hash_params.f_hash = pipeline_test_hash;
+ hash_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
table = ops->f_create(&hash_params, 0, 1);
if (table == NULL)
@@ -895,20 +872,9 @@ test_table_hash_lru(void)
{
int status;
- status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_ops);
- if (status < 0)
- return status;
-
status = test_table_hash_lru_generic(
- &rte_table_hash_key8_lru_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_lru_generic(&rte_table_hash_key32_lru_ops);
+ &rte_table_hash_key8_lru_ops,
+ 8);
if (status < 0)
return status;
@@ -924,20 +890,7 @@ test_table_hash_ext(void)
{
int status;
- status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_ext_generic(
- &rte_table_hash_key8_ext_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_ext_generic(&rte_table_hash_key32_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops, 8);
if (status < 0)
return status;
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 11/18] librte_table: rework 16-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (9 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 10/18] librte_table: rework 8-byte key hash tables Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 12/18] librte_table: rework 32-byte " Cristian Dumitrescu
` (25 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the 16-byte key hash tables (both the extendible
bucket and LRU)to use the mask-based hash function and the
unified parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 14 +-
lib/librte_table/rte_table_hash.h | 54 ---
lib/librte_table/rte_table_hash_key16.c | 366 +++++++++++----------
test/test-pipeline/pipeline_hash.c | 25 +-
test/test/test_table_combined.c | 40 +--
test/test/test_table_tables.c | 8 +
6 files changed, 233 insertions(+), 274 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 47e6d35..e4a18e4 100755
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -492,18 +492,6 @@ static void *pipeline_fc_init(struct pipeline_params *params,
/* Tables */
p->n_tables = 1;
{
- struct rte_table_hash_key16_ext_params
- table_hash_key16_params = {
- .n_entries = p_fc->n_flows,
- .n_entries_ext = p_fc->n_flows,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
- .key_mask = (p_fc->key_mask_present) ?
- p_fc->key_mask : NULL,
- .seed = 0,
- };
-
struct rte_table_hash_params table_hash_params = {
.name = p->name,
.key_size = p_fc->key_size,
@@ -536,7 +524,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
case 16:
table_params.ops = &rte_table_hash_key16_ext_ops;
- table_params.arg_create = &table_hash_key16_params;
+ table_params.arg_create = &table_hash_params;
break;
default:
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 6e25054..079dc93 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -143,62 +143,8 @@ extern struct rte_table_ops rte_table_hash_key8_lru_ops;
extern struct rte_table_ops rte_table_hash_key8_ext_ops;
-/**
- * 16-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key16_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
extern struct rte_table_ops rte_table_hash_key16_lru_ops;
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key16_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
extern struct rte_table_ops rte_table_hash_key16_ext_ops;
/**
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
index 7f6651c..84b59cc 100644
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 16
+#define KEY_SIZE 16
+
+#define KEYS_PER_BUCKET 4
#define RTE_BUCKET_ENTRY_VALID 0x1LLU
@@ -79,14 +81,12 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask[2];
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
/* Extendible buckets */
@@ -99,17 +99,55 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key16_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return (a64[0] != (b64[0] & b_mask64[0])) ||
+ (a64[1] != (b64[1] & b_mask64[1]));
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+ dst64[1] = src64[1] & src_mask64[1];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_buckets */
+ if ((params->n_buckets == 0) ||
+ (!rte_is_power_of_2(params->n_buckets))) {
+ RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
/* f_hash */
if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: f_hash function pointer is NULL\n", __func__);
+ RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
+ __func__);
return -EINVAL;
}
@@ -121,46 +159,67 @@ rte_table_hash_create_key16_lru(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key16_lru_params *p =
- (struct rte_table_hash_key16_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket,
- key_size, bucket_size_cl, total_size, i;
+ uint64_t bucket_size, total_size;
+ uint32_t n_buckets, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_16) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 16;
+
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + n_buckets *
- bucket_size_cl * RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +
+ KEYS_PER_BUCKET * entry_size);
+ total_size = sizeof(struct rte_table_hash) + n_buckets * bucket_size;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -212,19 +271,19 @@ rte_table_hash_entry_add_key16_lru(
uint64_t signature, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -238,13 +297,13 @@ rte_table_hash_entry_add_key16_lru(
/* Key is not present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -257,7 +316,7 @@ rte_table_hash_entry_add_key16_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
bucket->signature[pos] = signature;
- memcpy(bucket->key[pos], key, f->key_size);
+ keycpy(&bucket->key[pos], key, f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
*key_found = 0;
@@ -278,19 +337,19 @@ rte_table_hash_entry_delete_key16_lru(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = 0;
@@ -306,81 +365,71 @@ rte_table_hash_entry_delete_key16_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key16_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: f_hash function pointer is NULL\n", __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key16_ext(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key16_ext_params *p =
- (struct rte_table_hash_key16_ext_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size,
- bucket_size_cl, stack_size_cl, total_size, i;
+ uint64_t bucket_size, stack_size, total_size;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_16) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 16;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
total_size = sizeof(struct rte_table_hash) +
- ((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *
- RTE_CACHE_LINE_SIZE;
+ (p->n_buckets + n_buckets_ext) * bucket_size + stack_size;
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
- f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->n_buckets = p->n_buckets;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -388,10 +437,7 @@ rte_table_hash_create_key16_ext(void *params,
f->n_buckets_ext = n_buckets_ext;
f->stack_pos = n_buckets_ext;
f->stack = (uint32_t *)
- &f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
-
- for (i = 0; i < n_buckets_ext; i++)
- f->stack[i] = i;
+ &f->memory[(p->n_buckets + n_buckets_ext) * f->bucket_size];
if (p->key_mask != NULL) {
f->key_mask[0] = (((uint64_t *)p->key_mask)[0]);
@@ -401,6 +447,9 @@ rte_table_hash_create_key16_ext(void *params,
f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
}
+ for (i = 0; i < n_buckets_ext; i++)
+ f->stack[i] = i;
+
return f;
}
@@ -432,20 +481,20 @@ rte_table_hash_entry_add_key16_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (bucket = bucket0; bucket != NULL; bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -458,17 +507,17 @@ rte_table_hash_entry_add_key16_ext(
/* Key is not present in the bucket */
for (bucket_prev = NULL, bucket = bucket0; bucket != NULL;
- bucket_prev = bucket, bucket = bucket->next)
+ bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -487,7 +536,7 @@ rte_table_hash_entry_add_key16_ext(
bucket_prev->next_valid = 1;
bucket->signature[0] = signature;
- memcpy(bucket->key[0], key, f->key_size);
+ keycpy(&bucket->key[0], key, f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -509,7 +558,7 @@ rte_table_hash_entry_delete_key16_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_16 *)
&f->memory[bucket_index * f->bucket_size];
@@ -520,18 +569,17 @@ rte_table_hash_entry_delete_key16_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = 0;
*key_found = 1;
if (entry)
- memcpy(entry, bucket_data,
- f->entry_size);
+ memcpy(entry, bucket_data, f->entry_size);
if ((bucket->signature[0] == 0) &&
(bucket->signature[1] == 0) &&
@@ -558,26 +606,28 @@ rte_table_hash_entry_delete_key16_ext(
return 0;
}
-#define lookup_key16_cmp(key_in, bucket, pos) \
+#define lookup_key16_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4][2], or[4], signature[4]; \
+ uint64_t xor[4][2], or[4], signature[4], k[2]; \
\
+ k[0] = key_in[0] & f->key_mask[0]; \
+ k[1] = key_in[1] & f->key_mask[1]; \
signature[0] = (~bucket->signature[0]) & 1; \
signature[1] = (~bucket->signature[1]) & 1; \
signature[2] = (~bucket->signature[2]) & 1; \
signature[3] = (~bucket->signature[3]) & 1; \
\
- xor[0][0] = key_in[0] ^ bucket->key[0][0]; \
- xor[0][1] = key_in[1] ^ bucket->key[0][1]; \
+ xor[0][0] = k[0] ^ bucket->key[0][0]; \
+ xor[0][1] = k[1] ^ bucket->key[0][1]; \
\
- xor[1][0] = key_in[0] ^ bucket->key[1][0]; \
- xor[1][1] = key_in[1] ^ bucket->key[1][1]; \
+ xor[1][0] = k[0] ^ bucket->key[1][0]; \
+ xor[1][1] = k[1] ^ bucket->key[1][1]; \
\
- xor[2][0] = key_in[0] ^ bucket->key[2][0]; \
- xor[2][1] = key_in[1] ^ bucket->key[2][1]; \
+ xor[2][0] = k[0] ^ bucket->key[2][0]; \
+ xor[2][1] = k[1] ^ bucket->key[2][1]; \
\
- xor[3][0] = key_in[0] ^ bucket->key[3][0]; \
- xor[3][1] = key_in[1] ^ bucket->key[3][1]; \
+ xor[3][0] = k[0] ^ bucket->key[3][0]; \
+ xor[3][1] = k[1] ^ bucket->key[3][1]; \
\
or[0] = xor[0][0] | xor[0][1] | signature[0]; \
or[1] = xor[1][0] | xor[1][1] | signature[1]; \
@@ -613,14 +663,9 @@ rte_table_hash_entry_delete_key16_ext(
uint64_t *key; \
uint64_t signature = 0; \
uint32_t bucket_index; \
- uint64_t hash_key_buffer[2]; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\
- \
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- signature = f->f_hash(hash_key_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed); \
+ signature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed); \
\
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_16 *) \
@@ -635,14 +680,10 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket2, pos); \
+ lookup_key16_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -660,14 +701,10 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket2, pos); \
+ lookup_key16_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -690,15 +727,11 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket, pos); \
+ lookup_key16_cmp(key, bucket, pos, f); \
\
pkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -763,15 +796,11 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
uint64_t *key10, *key11; \
- uint64_t hash_offset_buffer[2]; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
\
key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset);\
- hash_offset_buffer[0] = key10[0] & f->key_mask[0]; \
- hash_offset_buffer[1] = key10[1] & f->key_mask[1]; \
- signature10 = f->f_hash(hash_offset_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed);\
+ signature10 = f->f_hash(key10, f->key_mask, KEY_SIZE, f->seed);\
bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_16 *) \
&f->memory[bucket10_index * f->bucket_size]; \
@@ -779,10 +808,7 @@ rte_table_hash_entry_delete_key16_ext(
rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
\
key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset);\
- hash_offset_buffer[0] = key11[0] & f->key_mask[0]; \
- hash_offset_buffer[1] = key11[1] & f->key_mask[1]; \
- signature11 = f->f_hash(hash_offset_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed);\
+ signature11 = f->f_hash(key11, f->key_mask, KEY_SIZE, f->seed);\
bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_16 *) \
&f->memory[bucket11_index * f->bucket_size]; \
@@ -796,19 +822,13 @@ rte_table_hash_entry_delete_key16_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask; \
uint64_t *key20, *key21; \
- uint64_t hash_key_buffer20[2]; \
- uint64_t hash_key_buffer21[2]; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_key_buffer20[0] = key20[0] & f->key_mask[0]; \
- hash_key_buffer20[1] = key20[1] & f->key_mask[1]; \
- hash_key_buffer21[0] = key21[0] & f->key_mask[0]; \
- hash_key_buffer21[1] = key21[1] & f->key_mask[1]; \
\
- lookup_key16_cmp(hash_key_buffer20, bucket20, pos20); \
- lookup_key16_cmp(hash_key_buffer21, bucket21, pos21); \
+ lookup_key16_cmp(key20, bucket20, pos20, f); \
+ lookup_key16_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -831,19 +851,13 @@ rte_table_hash_entry_delete_key16_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\
uint64_t *key20, *key21; \
- uint64_t hash_key_buffer20[2]; \
- uint64_t hash_key_buffer21[2]; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_key_buffer20[0] = key20[0] & f->key_mask[0]; \
- hash_key_buffer20[1] = key20[1] & f->key_mask[1]; \
- hash_key_buffer21[0] = key21[0] & f->key_mask[0]; \
- hash_key_buffer21[1] = key21[1] & f->key_mask[1]; \
\
- lookup_key16_cmp(hash_key_buffer20, bucket20, pos20); \
- lookup_key16_cmp(hash_key_buffer21, bucket21, pos21); \
+ lookup_key16_cmp(key20, bucket20, pos20, f); \
+ lookup_key16_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -994,7 +1008,7 @@ rte_table_hash_lookup_key16_lru(
RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
__builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key16_lru() */
+} /* lookup LRU */
static int
rte_table_hash_lookup_key16_ext(
@@ -1146,7 +1160,7 @@ rte_table_hash_lookup_key16_ext(
RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
__builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key16_ext() */
+} /* lookup EXT */
static int
rte_table_hash_key16_stats_read(void *table, struct rte_table_stats *stats, int clear)
@@ -1167,6 +1181,8 @@ struct rte_table_ops rte_table_hash_key16_lru_ops = {
.f_free = rte_table_hash_free_key16_lru,
.f_add = rte_table_hash_entry_add_key16_lru,
.f_delete = rte_table_hash_entry_delete_key16_lru,
+ .f_add_bulk = NULL,
+ .f_delete_bulk = NULL,
.f_lookup = rte_table_hash_lookup_key16_lru,
.f_stats = rte_table_hash_key16_stats_read,
};
@@ -1176,6 +1192,8 @@ struct rte_table_ops rte_table_hash_key16_ext_ops = {
.f_free = rte_table_hash_free_key16_ext,
.f_add = rte_table_hash_entry_add_key16_ext,
.f_delete = rte_table_hash_entry_delete_key16_ext,
+ .f_add_bulk = NULL,
+ .f_delete_bulk = NULL,
.f_lookup = rte_table_hash_lookup_key16_ext,
.f_stats = rte_table_hash_key16_stats_read,
};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index f542506..222c48c 100755
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -285,14 +285,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_EXT:
{
- struct rte_table_hash_key16_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
.key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
+ .seed = 0,
};
struct rte_pipeline_table_params table_params = {
@@ -311,13 +312,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_LRU:
{
- struct rte_table_hash_key16_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
.key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
+ .seed = 0,
};
struct rte_pipeline_table_params table_params = {
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index ddcd14b..4cfd955 100755
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -502,13 +502,15 @@ test_table_hash16lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key16_lru_params key16lru_params = {
- .n_entries = 1<<16,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key16lru_params = {
+ .name = "TABLE",
+ .key_size = 16,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key16lru[16];
@@ -537,14 +539,14 @@ test_table_hash16lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key16lru_params.n_entries = 0;
+ key16lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16lru_params.n_entries = 1<<16;
+ key16lru_params.n_keys = 1<<16;
key16lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key16_lru_ops,
@@ -680,14 +682,15 @@ test_table_hash16ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key16_ext_params key16ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key16ext_params = {
+ .name = "TABLE",
+ .key_size = 16,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key16ext[16];
@@ -716,14 +719,14 @@ test_table_hash16ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key16ext_params.n_entries = 0;
+ key16ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16ext_params.n_entries = 1<<16;
+ key16ext_params.n_keys = 1<<16;
key16ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key16_ext_ops,
@@ -731,13 +734,6 @@ test_table_hash16ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16ext_params.f_hash = pipeline_test_hash;
- key16ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key16_ext_ops,
- (void *)&key16ext_params, (void *)key16ext, &table_packets, NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 7f85587..2f916b7 100755
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -878,6 +878,11 @@ test_table_hash_lru(void)
if (status < 0)
return status;
+ status = test_table_hash_lru_generic(
+ &rte_table_hash_key16_lru_ops,
+ 16);
+ if (status < 0)
+ return status;
status = test_lru_update();
if (status < 0)
return status;
@@ -894,6 +899,9 @@ test_table_hash_ext(void)
if (status < 0)
return status;
+ status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops, 16);
+ if (status < 0)
+ return status;
return 0;
}
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 12/18] librte_table: rework 32-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (10 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 11/18] librte_table: rework 16-byte " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 13/18] librte_table: rework cuckoo hash table Cristian Dumitrescu
` (24 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the 32-byte key hash tables (both the extendible
bucket and LRU)to use the mask-based hash function and the
unified parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 50 -----
lib/librte_table/rte_table_hash_key32.c | 386 ++++++++++++++++++++------------
test/test-pipeline/pipeline_hash.c | 23 +-
test/test/test_table_combined.c | 43 ++--
test/test/test_table_tables.c | 12 +
5 files changed, 283 insertions(+), 231 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 079dc93..16e1dfa 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -147,58 +147,8 @@ extern struct rte_table_ops rte_table_hash_key16_lru_ops;
extern struct rte_table_ops rte_table_hash_key16_ext_ops;
-/**
- * 32-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key32_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
-/** LRU hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key32_lru_ops;
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key32_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
-/** Extendible bucket hash table operations */
extern struct rte_table_ops rte_table_hash_key32_ext_ops;
/** Cuckoo hash table parameters */
diff --git a/lib/librte_table/rte_table_hash_key32.c b/lib/librte_table/rte_table_hash_key32.c
index f8f6662..3eb7e4f 100644
--- a/lib/librte_table/rte_table_hash_key32.c
+++ b/lib/librte_table/rte_table_hash_key32.c
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 32
+#define KEY_SIZE 32
+
+#define KEYS_PER_BUCKET 4
#define RTE_BUCKET_ENTRY_VALID 0x1LLU
@@ -79,13 +81,12 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
- rte_table_hash_op_hash_nomask f_hash;
+ uint64_t key_mask[4];
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
/* Extendible buckets */
@@ -98,10 +99,52 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key32_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return (a64[0] != (b64[0] & b_mask64[0])) ||
+ (a64[1] != (b64[1] & b_mask64[1])) ||
+ (a64[2] != (b64[2] & b_mask64[2])) ||
+ (a64[3] != (b64[3] & b_mask64[3]));
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+ dst64[1] = src64[1] & src_mask64[1];
+ dst64[2] = src64[2] & src_mask64[2];
+ dst64[3] = src64[3] & src_mask64[3];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_buckets */
+ if ((params->n_buckets == 0) ||
+ (!rte_is_power_of_2(params->n_buckets))) {
+ RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -120,51 +163,83 @@ rte_table_hash_create_key32_lru(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key32_lru_params *p =
- (struct rte_table_hash_key32_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;
- uint32_t total_size, i;
+ uint64_t bucket_size, total_size;
+ uint32_t n_buckets, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
- ((sizeof(struct rte_bucket_4_32) % 64) != 0)) {
+ ((sizeof(struct rte_bucket_4_32) % 64) != 0))
return NULL;
- }
- n_entries_per_bucket = 4;
- key_size = 32;
+
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + n_buckets *
- bucket_size_cl * RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +
+ KEYS_PER_BUCKET * entry_size);
+ total_size = sizeof(struct rte_table_hash) + n_buckets * bucket_size;
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n", __func__,
- total_size);
+ "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
+ if (p->key_mask != NULL) {
+ f->key_mask[0] = ((uint64_t *)p->key_mask)[0];
+ f->key_mask[1] = ((uint64_t *)p->key_mask)[1];
+ f->key_mask[2] = ((uint64_t *)p->key_mask)[2];
+ f->key_mask[3] = ((uint64_t *)p->key_mask)[3];
+ } else {
+ f->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;
+ }
+
for (i = 0; i < n_buckets; i++) {
struct rte_bucket_4_32 *bucket;
@@ -204,7 +279,7 @@ rte_table_hash_entry_add_key32_lru(
uint64_t signature, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -213,10 +288,10 @@ rte_table_hash_entry_add_key32_lru(
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -230,13 +305,13 @@ rte_table_hash_entry_add_key32_lru(
/* Key is not present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -249,10 +324,10 @@ rte_table_hash_entry_add_key32_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
bucket->signature[pos] = signature;
- memcpy(bucket->key[pos], key, f->key_size);
+ keycpy(&bucket->key[pos], key, f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
- *key_found = 0;
+ *key_found = 0;
*entry_ptr = (void *) &bucket->data[pos * f->entry_size];
return 0;
@@ -270,7 +345,7 @@ rte_table_hash_entry_delete_key32_lru(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -279,10 +354,10 @@ rte_table_hash_entry_delete_key32_lru(
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = 0;
@@ -299,81 +374,72 @@ rte_table_hash_entry_delete_key32_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key32_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
- __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key32_ext(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key32_ext_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket;
- uint32_t key_size, bucket_size_cl, stack_size_cl, total_size, i;
+ uint64_t bucket_size, stack_size, total_size;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_32) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 32;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
total_size = sizeof(struct rte_table_hash) +
- ((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *
- RTE_CACHE_LINE_SIZE;
+ (p->n_buckets + n_buckets_ext) * bucket_size + stack_size;
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n", __func__,
- total_size);
+ "%s: Hash table %s memory footprint "
+ "is %" PRIu64" bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
- f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->n_buckets = p->n_buckets;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -381,7 +447,19 @@ rte_table_hash_create_key32_ext(void *params,
f->n_buckets_ext = n_buckets_ext;
f->stack_pos = n_buckets_ext;
f->stack = (uint32_t *)
- &f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
+ &f->memory[(p->n_buckets + n_buckets_ext) * f->bucket_size];
+
+ if (p->key_mask != NULL) {
+ f->key_mask[0] = (((uint64_t *)p->key_mask)[0]);
+ f->key_mask[1] = (((uint64_t *)p->key_mask)[1]);
+ f->key_mask[2] = (((uint64_t *)p->key_mask)[2]);
+ f->key_mask[3] = (((uint64_t *)p->key_mask)[3]);
+ } else {
+ f->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;
+ }
for (i = 0; i < n_buckets_ext; i++)
f->stack[i] = i;
@@ -417,7 +495,7 @@ rte_table_hash_entry_add_key32_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -427,10 +505,10 @@ rte_table_hash_entry_add_key32_ext(
for (bucket = bucket0; bucket != NULL; bucket = bucket->next) {
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -448,14 +526,14 @@ rte_table_hash_entry_add_key32_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -475,7 +553,7 @@ rte_table_hash_entry_add_key32_ext(
bucket_prev->next_valid = 1;
bucket->signature[0] = signature;
- memcpy(bucket->key[0], key, f->key_size);
+ keycpy(&bucket->key[0], key, f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -497,7 +575,7 @@ rte_table_hash_entry_delete_key32_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -508,24 +586,23 @@ rte_table_hash_entry_delete_key32_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = 0;
*key_found = 1;
if (entry)
- memcpy(entry, bucket_data,
- f->entry_size);
+ memcpy(entry, bucket_data, f->entry_size);
if ((bucket->signature[0] == 0) &&
- (bucket->signature[1] == 0) &&
- (bucket->signature[2] == 0) &&
- (bucket->signature[3] == 0) &&
- (bucket_prev != NULL)) {
+ (bucket->signature[1] == 0) &&
+ (bucket->signature[2] == 0) &&
+ (bucket->signature[3] == 0) &&
+ (bucket_prev != NULL)) {
bucket_prev->next = bucket->next;
bucket_prev->next_valid =
bucket->next_valid;
@@ -546,34 +623,39 @@ rte_table_hash_entry_delete_key32_ext(
return 0;
}
-#define lookup_key32_cmp(key_in, bucket, pos) \
+#define lookup_key32_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4][4], or[4], signature[4]; \
+ uint64_t xor[4][4], or[4], signature[4], k[4]; \
+ \
+ k[0] = key_in[0] & f->key_mask[0]; \
+ k[1] = key_in[1] & f->key_mask[1]; \
+ k[2] = key_in[2] & f->key_mask[2]; \
+ k[3] = key_in[3] & f->key_mask[3]; \
\
signature[0] = ((~bucket->signature[0]) & 1); \
signature[1] = ((~bucket->signature[1]) & 1); \
signature[2] = ((~bucket->signature[2]) & 1); \
signature[3] = ((~bucket->signature[3]) & 1); \
\
- xor[0][0] = key_in[0] ^ bucket->key[0][0]; \
- xor[0][1] = key_in[1] ^ bucket->key[0][1]; \
- xor[0][2] = key_in[2] ^ bucket->key[0][2]; \
- xor[0][3] = key_in[3] ^ bucket->key[0][3]; \
+ xor[0][0] = k[0] ^ bucket->key[0][0]; \
+ xor[0][1] = k[1] ^ bucket->key[0][1]; \
+ xor[0][2] = k[2] ^ bucket->key[0][2]; \
+ xor[0][3] = k[3] ^ bucket->key[0][3]; \
\
- xor[1][0] = key_in[0] ^ bucket->key[1][0]; \
- xor[1][1] = key_in[1] ^ bucket->key[1][1]; \
- xor[1][2] = key_in[2] ^ bucket->key[1][2]; \
- xor[1][3] = key_in[3] ^ bucket->key[1][3]; \
+ xor[1][0] = k[0] ^ bucket->key[1][0]; \
+ xor[1][1] = k[1] ^ bucket->key[1][1]; \
+ xor[1][2] = k[2] ^ bucket->key[1][2]; \
+ xor[1][3] = k[3] ^ bucket->key[1][3]; \
\
- xor[2][0] = key_in[0] ^ bucket->key[2][0]; \
- xor[2][1] = key_in[1] ^ bucket->key[2][1]; \
- xor[2][2] = key_in[2] ^ bucket->key[2][2]; \
- xor[2][3] = key_in[3] ^ bucket->key[2][3]; \
+ xor[2][0] = k[0] ^ bucket->key[2][0]; \
+ xor[2][1] = k[1] ^ bucket->key[2][1]; \
+ xor[2][2] = k[2] ^ bucket->key[2][2]; \
+ xor[2][3] = k[3] ^ bucket->key[2][3]; \
\
- xor[3][0] = key_in[0] ^ bucket->key[3][0]; \
- xor[3][1] = key_in[1] ^ bucket->key[3][1]; \
- xor[3][2] = key_in[2] ^ bucket->key[3][2]; \
- xor[3][3] = key_in[3] ^ bucket->key[3][3]; \
+ xor[3][0] = k[0] ^ bucket->key[3][0]; \
+ xor[3][1] = k[1] ^ bucket->key[3][1]; \
+ xor[3][2] = k[2] ^ bucket->key[3][2]; \
+ xor[3][3] = k[3] ^ bucket->key[3][3]; \
\
or[0] = xor[0][0] | xor[0][1] | xor[0][2] | xor[0][3] | signature[0];\
or[1] = xor[1][0] | xor[1][1] | xor[1][2] | xor[1][3] | signature[1];\
@@ -604,12 +686,15 @@ rte_table_hash_entry_delete_key32_ext(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf0, key_offset));\
}
-#define lookup1_stage1(mbuf1, bucket1, f) \
+#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
+ uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
\
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
+ key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset); \
+ signature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed); \
+ \
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_32 *) \
&f->memory[bucket_index * f->bucket_size]; \
@@ -627,8 +712,7 @@ rte_table_hash_entry_delete_key32_ext(
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- \
- lookup_key32_cmp(key, bucket2, pos); \
+ lookup_key32_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -649,8 +733,7 @@ rte_table_hash_entry_delete_key32_ext(
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- \
- lookup_key32_cmp(key, bucket2, pos); \
+ lookup_key32_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -678,7 +761,7 @@ rte_table_hash_entry_delete_key32_ext(
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
\
- lookup_key32_cmp(key, bucket, pos); \
+ lookup_key32_cmp(key, bucket, pos, f); \
\
pkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -745,22 +828,27 @@ rte_table_hash_entry_delete_key32_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
+ uint64_t *key10, *key11; \
+ uint64_t signature10, signature11; \
+ uint32_t bucket10_index, bucket11_index; \
\
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
+ key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset); \
+ signature10 = f->f_hash(key10, f->key_mask, KEY_SIZE, f->seed); \
+ \
+ bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_32 *) \
&f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
+ rte_prefetch0(bucket10); \
rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
rte_prefetch0((void *)(((uintptr_t) bucket10) + 2 * RTE_CACHE_LINE_SIZE));\
\
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
+ key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset); \
+ signature11 = f->f_hash(key11, f->key_mask, KEY_SIZE, f->seed);\
+ \
+ bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_32 *) \
&f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
+ rte_prefetch0(bucket11); \
rte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\
rte_prefetch0((void *)(((uintptr_t) bucket11) + 2 * RTE_CACHE_LINE_SIZE));\
}
@@ -776,8 +864,8 @@ rte_table_hash_entry_delete_key32_ext(
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
\
- lookup_key32_cmp(key20, bucket20, pos20); \
- lookup_key32_cmp(key21, bucket21, pos21); \
+ lookup_key32_cmp(key20, bucket20, pos20, f); \
+ lookup_key32_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -805,8 +893,8 @@ rte_table_hash_entry_delete_key32_ext(
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
\
- lookup_key32_cmp(key20, bucket20, pos20); \
- lookup_key32_cmp(key21, bucket21, pos21); \
+ lookup_key32_cmp(key20, bucket20, pos20, f); \
+ lookup_key32_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 222c48c..eda63e8 100755
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -339,12 +339,14 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_EXT:
{
- struct rte_table_hash_key32_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
};
@@ -365,11 +367,14 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_LRU:
{
- struct rte_table_hash_key32_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
};
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index 4cfd955..93603dc 100755
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -563,12 +563,15 @@ test_table_hash32lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key32_lru_params key32lru_params = {
- .n_entries = 1<<16,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key32lru_params = {
+ .name = "TABLE",
+ .key_size = 32,
.key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key32lru[32];
@@ -597,14 +600,14 @@ test_table_hash32lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key32lru_params.n_entries = 0;
+ key32lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key32_lru_ops,
(void *)&key32lru_params, (void *)key32lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32lru_params.n_entries = 1<<16;
+ key32lru_params.n_keys = 1<<16;
key32lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key32_lru_ops,
@@ -743,13 +746,15 @@ test_table_hash32ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key32_ext_params key32ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key32ext_params = {
+ .name = "TABLE",
+ .key_size = 32,
.key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key32ext[32];
@@ -778,14 +783,14 @@ test_table_hash32ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key32ext_params.n_entries = 0;
+ key32ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key32_ext_ops,
(void *)&key32ext_params, (void *)key32ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32ext_params.n_entries = 1<<16;
+ key32ext_params.n_keys = 1<<16;
key32ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key32_ext_ops,
@@ -793,14 +798,6 @@ test_table_hash32ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32ext_params.f_hash = pipeline_test_hash;
- key32ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key32_ext_ops,
- (void *)&key32ext_params, (void *)key32ext, &table_packets,
- NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 2f916b7..03c2723 100755
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -883,6 +883,13 @@ test_table_hash_lru(void)
16);
if (status < 0)
return status;
+
+ status = test_table_hash_lru_generic(
+ &rte_table_hash_key32_lru_ops,
+ 32);
+ if (status < 0)
+ return status;
+
status = test_lru_update();
if (status < 0)
return status;
@@ -902,6 +909,11 @@ test_table_hash_ext(void)
status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops, 16);
if (status < 0)
return status;
+
+ status = test_table_hash_ext_generic(&rte_table_hash_key32_ext_ops, 32);
+ if (status < 0)
+ return status;
+
return 0;
}
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 13/18] librte_table: rework cuckoo hash table
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (11 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 12/18] librte_table: rework 32-byte " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 14/18] test: add mask-based hash functions Cristian Dumitrescu
` (23 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the cuckoo hash table to use the unified parameter
structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 26 ------
lib/librte_table/rte_table_hash_cuckoo.c | 149 +++++++++++++------------------
test/test-pipeline/pipeline_hash.c | 18 ++--
test/test/test_table_combined.c | 13 +--
test/test/test_table_tables.c | 15 ++--
5 files changed, 82 insertions(+), 139 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 16e1dfa..eb18c7a 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -151,32 +151,6 @@ extern struct rte_table_ops rte_table_hash_key32_lru_ops;
extern struct rte_table_ops rte_table_hash_key32_ext_ops;
-/** Cuckoo hash table parameters */
-struct rte_table_hash_cuckoo_params {
- /** Key size (number of bytes */
- uint32_t key_size;
-
- /** Maximum number of hash table entries */
- uint32_t n_keys;
-
- /** Hash function used to calculate hash */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed value or Init value used by f_hash */
- uint32_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Hash table name */
- const char *name;
-};
-
extern struct rte_table_ops rte_table_hash_cuckoo_ops;
#ifdef __cplusplus
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
index 9b42423..8d26597 100644
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -64,27 +64,30 @@ struct rte_table_hash {
uint32_t key_size;
uint32_t entry_size;
uint32_t n_keys;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint32_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
- const char *name;
/* cuckoo hash table object */
struct rte_hash *h_table;
/* Lookup table */
- uint8_t memory[0] __rte_cache_aligned; };
+ uint8_t memory[0] __rte_cache_aligned;
+};
static int
-check_params_create_hash_cuckoo(const struct
-rte_table_hash_cuckoo_params *params) {
- /* Check for valid parameters */
+check_params_create_hash_cuckoo(struct rte_table_hash_params *params)
+{
if (params == NULL) {
RTE_LOG(ERR, TABLE, "NULL Input Parameters.\n");
return -EINVAL;
}
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "Table name is NULL.\n");
+ return -EINVAL;
+ }
+
if (params->key_size == 0) {
RTE_LOG(ERR, TABLE, "Invalid key_size.\n");
return -EINVAL;
@@ -100,11 +103,6 @@ rte_table_hash_cuckoo_params *params) {
return -EINVAL;
}
- if (params->name == NULL) {
- RTE_LOG(ERR, TABLE, "Table name is NULL.\n");
- return -EINVAL;
- }
-
return 0;
}
@@ -113,34 +111,24 @@ rte_table_hash_cuckoo_create(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_hash *rte_hash_handle;
+ struct rte_table_hash_params *p = params;
+ struct rte_hash *h_table;
struct rte_table_hash *t;
- uint32_t total_size, total_cl_size;
+ uint32_t total_size;
/* Check input parameters */
- struct rte_table_hash_cuckoo_params *p =
- (struct rte_table_hash_cuckoo_params *) params;
-
if (check_params_create_hash_cuckoo(params))
return NULL;
/* Memory allocation */
- total_cl_size =
- (sizeof(struct rte_table_hash) +
- RTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;
- total_cl_size += (p->n_keys * entry_size +
- RTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;
- total_size = total_cl_size * RTE_CACHE_LINE_SIZE;
-
- t = rte_zmalloc_socket("TABLE",
- total_size,
- RTE_CACHE_LINE_SIZE,
- socket_id);
+ total_size = sizeof(struct rte_table_hash) +
+ RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
+
+ t = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (t == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for Cuckoo hash table\n",
- __func__,
- (uint32_t)sizeof(struct rte_table_hash));
+ "%s: Cannot allocate %u bytes for cuckoo hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
@@ -154,13 +142,13 @@ rte_table_hash_cuckoo_create(void *params,
.name = p->name
};
- rte_hash_handle = rte_hash_find_existing(p->name);
- if (rte_hash_handle == NULL) {
- rte_hash_handle = rte_hash_create(&hash_cuckoo_params);
- if (NULL == rte_hash_handle) {
+ h_table = rte_hash_find_existing(p->name);
+ if (h_table == NULL) {
+ h_table = rte_hash_create(&hash_cuckoo_params);
+ if (h_table == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: failed to create cuckoo hash table. keysize: %u",
- __func__, hash_cuckoo_params.key_len);
+ "%s: failed to create cuckoo hash table %s\n",
+ __func__, p->name);
rte_free(t);
return NULL;
}
@@ -172,26 +160,22 @@ rte_table_hash_cuckoo_create(void *params,
t->n_keys = p->n_keys;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
- t->name = p->name;
- t->h_table = rte_hash_handle;
+ t->h_table = h_table;
RTE_LOG(INFO, TABLE,
- "%s: Cuckoo Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ "%s: Cuckoo hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
return t;
}
static int
rte_table_hash_cuckoo_free(void *table) {
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
struct rte_table_hash *t = table;
+ if (table == NULL)
+ return -EINVAL;
+
rte_hash_free(t->h_table);
rte_free(t);
@@ -200,25 +184,18 @@ rte_table_hash_cuckoo_free(void *table) {
static int
rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
- int *key_found, void **entry_ptr) {
+ int *key_found, void **entry_ptr)
+{
+ struct rte_table_hash *t = table;
int pos = 0;
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (key == NULL) {
- RTE_LOG(ERR, TABLE, "%s: key parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (entry == NULL) {
- RTE_LOG(ERR, TABLE, "%s: entry parameter is NULL\n", __func__);
+ /* Check input parameters */
+ if ((table == NULL) ||
+ (key == NULL) ||
+ (entry == NULL) ||
+ (key_found == NULL) ||
+ (entry_ptr == NULL))
return -EINVAL;
- }
-
- struct rte_table_hash *t = table;
/* Find Existing entries */
pos = rte_hash_lookup(t->h_table, key);
@@ -231,17 +208,15 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
*entry_ptr = existing_entry;
return 0;
-} else if (pos == -ENOENT) {
- /* Entry not found. Adding new entry */
+ }
+
+ if (pos == -ENOENT) {
+ /* Entry not found. Adding new entry */
uint8_t *new_entry;
pos = rte_hash_add_key(t->h_table, key);
- if (pos < 0) {
- RTE_LOG(ERR, TABLE,
- "%s: Entry not added, status : %u\n",
- __func__, pos);
+ if (pos < 0)
return pos;
- }
new_entry = &t->memory[pos * t->entry_size];
memcpy(new_entry, entry, t->entry_size);
@@ -250,25 +225,22 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
*entry_ptr = new_entry;
return 0;
}
+
return pos;
}
static int
rte_table_hash_cuckoo_entry_delete(void *table, void *key,
- int *key_found, __rte_unused void *entry) {
+ int *key_found, void *entry)
+{
+ struct rte_table_hash *t = table;
int pos = 0;
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (key == NULL) {
- RTE_LOG(ERR, TABLE, "%s: key parameter is NULL\n", __func__);
+ /* Check input parameters */
+ if ((table == NULL) ||
+ (key == NULL) ||
+ (key_found == NULL))
return -EINVAL;
- }
-
- struct rte_table_hash *t = table;
pos = rte_hash_del_key(t->h_table, key);
if (pos >= 0) {
@@ -279,12 +251,13 @@ rte_table_hash_cuckoo_entry_delete(void *table, void *key,
memcpy(entry, entry_ptr, t->entry_size);
memset(&t->memory[pos * t->entry_size], 0, t->entry_size);
+ return 0;
}
+ *key_found = 0;
return pos;
}
-
static int
rte_table_hash_cuckoo_lookup(void *table,
struct rte_mbuf **pkts,
@@ -292,7 +265,7 @@ rte_table_hash_cuckoo_lookup(void *table,
uint64_t *lookup_hit_mask,
void **entries)
{
- struct rte_table_hash *t = (struct rte_table_hash *)table;
+ struct rte_table_hash *t = table;
uint64_t pkts_mask_out = 0;
uint32_t i;
@@ -301,20 +274,19 @@ rte_table_hash_cuckoo_lookup(void *table,
RTE_TABLE_HASH_CUCKOO_STATS_PKTS_IN_ADD(t, n_pkts_in);
if ((pkts_mask & (pkts_mask + 1)) == 0) {
- const uint8_t *keys[64];
- int32_t positions[64], status;
+ const uint8_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
+ int32_t positions[RTE_PORT_IN_BURST_SIZE_MAX], status;
/* Keys for bulk lookup */
for (i = 0; i < n_pkts_in; i++)
keys[i] = RTE_MBUF_METADATA_UINT8_PTR(pkts[i],
- t->key_offset);
+ t->key_offset);
/* Bulk Lookup */
status = rte_hash_lookup_bulk(t->h_table,
(const void **) keys,
n_pkts_in,
positions);
-
if (status == 0) {
for (i = 0; i < n_pkts_in; i++) {
if (likely(positions[i] >= 0)) {
@@ -326,7 +298,7 @@ rte_table_hash_cuckoo_lookup(void *table,
}
}
}
- } else {
+ } else
for (i = 0; i < (uint32_t)(RTE_PORT_IN_BURST_SIZE_MAX
- __builtin_clzll(pkts_mask)); i++) {
uint64_t pkt_mask = 1LLU << i;
@@ -345,7 +317,6 @@ rte_table_hash_cuckoo_lookup(void *table,
}
}
}
- }
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_CUCKOO_STATS_PKTS_LOOKUP_MISS(t,
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index eda63e8..24df81c 100755
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -402,19 +402,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_CUCKOO_KEY112:
case e_APP_PIPELINE_HASH_CUCKOO_KEY128:
{
- char hash_name[RTE_HASH_NAMESIZE];
-
- snprintf(hash_name, sizeof(hash_name), "RTE_TH_CUCKOO_%d",
- app.pipeline_type);
-
- struct rte_table_hash_cuckoo_params table_hash_params = {
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
.key_size = key_size,
- .n_keys = (1 << 24) + 1,
- .f_hash = test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
.key_offset = APP_METADATA_OFFSET(32),
- .name = hash_name,
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
+ .seed = 0,
};
struct rte_pipeline_table_params table_params = {
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index 93603dc..9515dd0 100755
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -807,14 +807,15 @@ test_table_hash_cuckoo_combined(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_cuckoo_params cuckoo_params = {
+ struct rte_table_hash_params cuckoo_params = {
+ .name = "TABLE",
.key_size = 32,
- .n_keys = 1<<16,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
.key_offset = APP_METADATA_OFFSET(32),
- .name = "CUCKOO_HASH",
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key_cuckoo[32];
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 03c2723..7f04212 100755
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -932,14 +932,15 @@ test_table_hash_cuckoo(void)
uint32_t entry_size = 1;
/* Initialize params and create tables */
- struct rte_table_hash_cuckoo_params cuckoo_params = {
+ struct rte_table_hash_params cuckoo_params = {
+ .name = "TABLE",
.key_size = 32,
- .n_keys = 1 << 24,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
.key_offset = APP_METADATA_OFFSET(32),
- .name = "CUCKOO",
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
table = rte_table_hash_cuckoo_ops.f_create(NULL, 0, entry_size);
@@ -969,7 +970,7 @@ test_table_hash_cuckoo(void)
if (table != NULL)
return -4;
- cuckoo_params.f_hash = pipeline_test_hash;
+ cuckoo_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
cuckoo_params.name = NULL;
table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 14/18] test: add mask-based hash functions
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (12 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 13/18] librte_table: rework cuckoo hash table Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 15/18] librte_table: cosmetic enhancements in api file Cristian Dumitrescu
` (22 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Introduce mask-based hash functions in hash_func.h.
Propagate their usage in test/test, test/test-pipeline and
examples/ip_pipeline.
Remove the non-mask-based hash function prototype from API (which
was previously used as build workaround).
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
examples/ip_pipeline/pipeline/hash_func.h | 176 ++++++++++++---------
.../pipeline/pipeline_flow_classification.c | 10 +-
.../pipeline/pipeline_flow_classification_be.c | 9 +-
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 14 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 2 +-
lib/librte_table/rte_table_hash.h | 6 -
test/test-pipeline/main.h | 5 +-
test/test-pipeline/pipeline_hash.c | 113 ++-----------
test/test/test_table.c | 1 +
test/test/test_table.h | 3 +-
test/test/test_table_combined.c | 14 +-
test/test/test_table_tables.c | 12 +-
12 files changed, 155 insertions(+), 210 deletions(-)
mode change 100644 => 100755 examples/ip_pipeline/pipeline/hash_func.h
mode change 100644 => 100755 examples/ip_pipeline/pipeline/pipeline_flow_classification.c
mode change 100644 => 100755 test/test-pipeline/main.h
mode change 100644 => 100755 test/test/test_table.c
mode change 100644 => 100755 test/test/test_table.h
diff --git a/examples/ip_pipeline/pipeline/hash_func.h b/examples/ip_pipeline/pipeline/hash_func.h
old mode 100644
new mode 100755
index b112369..ecd4e05
--- a/examples/ip_pipeline/pipeline/hash_func.h
+++ b/examples/ip_pipeline/pipeline/hash_func.h
@@ -34,48 +34,56 @@
#define __INCLUDE_HASH_FUNC_H__
static inline uint64_t
-hash_xor_key8(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key8(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0;
- xor0 = seed ^ k[0];
+ xor0 = seed ^ (k[0] & m[0]);
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key16(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key16(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0;
- xor0 = (k[0] ^ seed) ^ k[1];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key24(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key24(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0;
- xor0 = (k[0] ^ seed) ^ k[1];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
- xor0 ^= k[2];
+ xor0 ^= k[2] & m[2];
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key32(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
xor0 ^= xor1;
@@ -83,30 +91,34 @@ hash_xor_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_xor_key40(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key40(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
xor0 ^= xor1;
- xor0 ^= k[4];
+ xor0 ^= k[4] & m[4];
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key48(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1, xor2;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
- xor2 = k[4] ^ k[5];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
+ xor2 = (k[4] & m[4]) ^ (k[5] & m[5]);
xor0 ^= xor1;
@@ -116,17 +128,19 @@ hash_xor_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_xor_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key56(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1, xor2;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
- xor2 = k[4] ^ k[5];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
+ xor2 = (k[4] & m[4]) ^ (k[5] & m[5]);
xor0 ^= xor1;
- xor2 ^= k[6];
+ xor2 ^= k[6] & m[6];
xor0 ^= xor2;
@@ -134,15 +148,17 @@ hash_xor_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_xor_key64(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key64(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1, xor2, xor3;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
- xor2 = k[4] ^ k[5];
- xor3 = k[6] ^ k[7];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
+ xor2 = (k[4] & m[4]) ^ (k[5] & m[5]);
+ xor3 = (k[6] & m[6]) ^ (k[7] & m[7]);
xor0 ^= xor1;
xor2 ^= xor3;
@@ -157,26 +173,30 @@ hash_xor_key64(void *key, __rte_unused uint32_t key_size, uint64_t seed)
#include <x86intrin.h>
static inline uint64_t
-hash_crc_key8(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key8(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t crc0;
- crc0 = _mm_crc32_u64(seed, k[0]);
+ crc0 = _mm_crc32_u64(seed, k[0] & m[0]);
return crc0;
}
static inline uint64_t
-hash_crc_key16(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key16(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, crc0, crc1;
- k0 = k[0];
+ k0 = k[0] & m[0];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
crc0 ^= crc1;
@@ -184,16 +204,18 @@ hash_crc_key16(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key24(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key24(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, crc0, crc1;
- k0 = k[0];
- k2 = k[2];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
crc0 = _mm_crc32_u64(crc0, k2);
@@ -203,18 +225,20 @@ hash_crc_key24(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key32(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, crc0, crc1, crc2, crc3;
- k0 = k[0];
- k2 = k[2];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
crc3 = k2 >> 32;
crc0 = _mm_crc32_u64(crc0, crc1);
@@ -226,19 +250,21 @@ hash_crc_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key40(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key40(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, crc0, crc1, crc2, crc3;
- k0 = k[0];
- k2 = k[2];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
crc0 = _mm_crc32_u64(crc0, crc1);
crc1 = _mm_crc32_u64(crc2, crc3);
@@ -249,20 +275,22 @@ hash_crc_key40(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key48(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, k5, crc0, crc1, crc2, crc3;
- k0 = k[0];
- k2 = k[2];
- k5 = k[5];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
+ k5 = k[5] & m[5];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
crc0 = _mm_crc32_u64(crc0, (crc1 << 32) ^ crc2);
crc1 = _mm_crc32_u64(crc3, k5);
@@ -273,22 +301,24 @@ hash_crc_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key56(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5;
- k0 = k[0];
- k2 = k[2];
- k5 = k[5];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
+ k5 = k[5] & m[5];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
- crc4 = _mm_crc32_u64(k5, k[6]);
+ crc4 = _mm_crc32_u64(k5, k[6] & m[6]);
crc5 = k5 >> 32;
crc0 = _mm_crc32_u64(crc0, (crc1 << 32) ^ crc2);
@@ -300,23 +330,25 @@ hash_crc_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key64(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key64(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5;
- k0 = k[0];
- k2 = k[2];
- k5 = k[5];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
+ k5 = k[5] & m[5];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
- crc4 = _mm_crc32_u64(k5, k[6]);
- crc5 = _mm_crc32_u64(k5 >> 32, k[7]);
+ crc4 = _mm_crc32_u64(k5, k[6] & m[6]);
+ crc5 = _mm_crc32_u64(k5 >> 32, k[7] & m[7]);
crc0 = _mm_crc32_u64(crc0, (crc1 << 32) ^ crc2);
crc1 = _mm_crc32_u64(crc3, (crc4 << 32) ^ crc5);
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
old mode 100644
new mode 100755
index 9ef50cc..70b1938
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
@@ -88,8 +88,10 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
uint32_t *signature)
{
uint8_t buffer[PIPELINE_FC_FLOW_KEY_MAX_SIZE];
+ uint8_t m[PIPELINE_FC_FLOW_KEY_MAX_SIZE]; /* key mask */
void *key_buffer = (key_out) ? key_out : buffer;
+ memset(m, 0xFF, sizeof(m));
switch (key_in->type) {
case FLOW_KEY_QINQ:
{
@@ -101,7 +103,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
qinq->cvlan = rte_cpu_to_be_16(key_in->key.qinq.cvlan);
if (signature)
- *signature = (uint32_t) hash_default_key8(qinq, 8, 0);
+ *signature = (uint32_t) hash_default_key8(qinq, m, 8, 0);
return 0;
}
@@ -118,7 +120,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
ipv4->port_dst = rte_cpu_to_be_16(key_in->key.ipv4_5tuple.port_dst);
if (signature)
- *signature = (uint32_t) hash_default_key16(ipv4, 16, 0);
+ *signature = (uint32_t) hash_default_key16(ipv4, m, 16, 0);
return 0;
}
@@ -136,7 +138,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
ipv6->port_dst = rte_cpu_to_be_16(key_in->key.ipv6_5tuple.port_dst);
if (signature)
- *signature = (uint32_t) hash_default_key64(ipv6, 64, 0);
+ *signature = (uint32_t) hash_default_key64(ipv6, m, 64, 0);
return 0;
}
@@ -832,12 +834,12 @@ app_pipeline_fc_add_bulk(struct app_params *app,
}
/* Free resources */
- app_msg_free(app, rsp);
for (i = rsp->n_keys; i < n_keys; i++)
if (new_flow[i])
rte_free(flow[i]);
+ app_msg_free(app, rsp);
rte_free(flow_rsp);
rte_free(flow_req);
rte_free(new_flow);
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index e4a18e4..9846777 100755
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -118,7 +118,7 @@ struct flow_table_entry {
uint32_t pad;
};
-rte_table_hash_op_hash_nomask hash_func[] = {
+rte_table_hash_op_hash hash_func[] = {
hash_default_key8,
hash_default_key16,
hash_default_key24,
@@ -500,7 +500,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
p_fc->key_mask : NULL,
.n_keys = p_fc->n_flows,
.n_buckets = p_fc->n_flows / 4,
- .f_hash = (rte_table_hash_op_hash)hash_func[(p_fc->key_size / 8) - 1],
+ .f_hash = hash_func[(p_fc->key_size / 8) - 1],
.seed = 0,
};
@@ -519,19 +519,18 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
table_params.ops = &rte_table_hash_key8_ext_ops;
- table_params.arg_create = &table_hash_params;
break;
case 16:
table_params.ops = &rte_table_hash_key16_ext_ops;
- table_params.arg_create = &table_hash_params;
break;
default:
table_params.ops = &rte_table_hash_ext_ops;
- table_params.arg_create = &table_hash_params;
}
+ table_params.arg_create = &table_hash_params;
+
status = rte_pipeline_table_create(p->p,
&table_params,
&p->table_id[0]);
diff --git a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
index 6b57f83..93eedbe 100755
--- a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
@@ -52,7 +52,7 @@
struct pipeline_passthrough {
struct pipeline p;
struct pipeline_passthrough_params params;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint32_t swap_field0_offset[SWAP_DIM];
uint32_t swap_field1_offset[SWAP_DIM];
uint64_t swap_field_mask[SWAP_DIM];
@@ -102,7 +102,7 @@ pkt_work_dma(
/* Read (dma_dst), compute (hash), write (hash) */
if (hash_enabled) {
- uint32_t hash = p->f_hash(dma_dst, dma_size, 0);
+ uint32_t hash = p->f_hash(dma_src, dma_mask, dma_size, 0);
*dma_hash = hash;
if (lb_hash) {
@@ -173,10 +173,10 @@ pkt4_work_dma(
/* Read (dma_dst), compute (hash), write (hash) */
if (hash_enabled) {
- uint32_t hash0 = p->f_hash(dma_dst0, dma_size, 0);
- uint32_t hash1 = p->f_hash(dma_dst1, dma_size, 0);
- uint32_t hash2 = p->f_hash(dma_dst2, dma_size, 0);
- uint32_t hash3 = p->f_hash(dma_dst3, dma_size, 0);
+ uint32_t hash0 = p->f_hash(dma_src0, dma_mask, dma_size, 0);
+ uint32_t hash1 = p->f_hash(dma_src1, dma_mask, dma_size, 0);
+ uint32_t hash2 = p->f_hash(dma_src2, dma_mask, dma_size, 0);
+ uint32_t hash3 = p->f_hash(dma_src3, dma_mask, dma_size, 0);
*dma_hash0 = hash0;
*dma_hash1 = hash1;
@@ -677,7 +677,7 @@ pipeline_passthrough_parse_args(struct pipeline_passthrough_params *p,
return 0;
}
-static rte_table_hash_op_hash_nomask
+static rte_table_hash_op_hash
get_hash_function(struct pipeline_passthrough *p)
{
switch (p->params.dma_size) {
diff --git a/examples/ip_pipeline/pipeline/pipeline_routing_be.c b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
index 796cde7..7aaf467 100755
--- a/examples/ip_pipeline/pipeline/pipeline_routing_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
@@ -1356,7 +1356,7 @@ pipeline_routing_init(struct pipeline_params *params,
.key_mask = NULL,
.n_keys = p_rt->params.n_arp_entries,
.n_buckets = p_rt->params.n_arp_entries / 4,
- .f_hash = (rte_table_hash_op_hash)hash_default_key8,
+ .f_hash = hash_default_key8,
.seed = 0,
};
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index eb18c7a..3c53d6a 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -129,12 +129,6 @@ struct rte_table_hash_params {
uint64_t seed;
};
-/** Hash function */
-typedef uint64_t (*rte_table_hash_op_hash_nomask)(
- void *key,
- uint32_t key_size,
- uint64_t seed);
-
extern struct rte_table_ops rte_table_hash_ext_ops;
extern struct rte_table_ops rte_table_hash_lru_ops;
diff --git a/test/test-pipeline/main.h b/test/test-pipeline/main.h
old mode 100644
new mode 100755
index 3685849..26395a3
--- a/test/test-pipeline/main.h
+++ b/test/test-pipeline/main.h
@@ -131,7 +131,10 @@ enum {
void app_main_loop_rx(void);
void app_main_loop_rx_metadata(void);
-uint64_t test_hash(void *key, uint32_t key_size, uint64_t seed);
+uint64_t test_hash(void *key,
+ void *key_mask,
+ uint32_t key_size,
+ uint64_t seed);
void app_main_loop_worker(void);
void app_main_loop_worker_pipeline_stub(void);
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 24df81c..edc1663 100755
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -169,23 +169,23 @@ app_main_loop_worker_pipeline_hash(void) {
"ring %d\n", i);
}
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = test_hash,
+ .seed = 0,
+ };
+
/* Table configuration */
switch (app.pipeline_type) {
case e_APP_PIPELINE_HASH_KEY8_EXT:
case e_APP_PIPELINE_HASH_KEY16_EXT:
case e_APP_PIPELINE_HASH_KEY32_EXT:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_ext_ops,
.arg_create = &table_hash_params,
@@ -204,17 +204,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_KEY16_LRU:
case e_APP_PIPELINE_HASH_KEY32_LRU:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_lru_ops,
.arg_create = &table_hash_params,
@@ -231,17 +220,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_EXT:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_hash_params,
@@ -258,17 +236,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_LRU:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key8_lru_ops,
.arg_create = &table_hash_params,
@@ -285,17 +252,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_EXT:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key16_ext_ops,
.arg_create = &table_hash_params,
@@ -312,17 +268,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_LRU:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key16_lru_ops,
.arg_create = &table_hash_params,
@@ -339,17 +284,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_EXT:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key32_ext_ops,
.arg_create = &table_hash_params,
@@ -367,17 +301,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_LRU:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key32_lru_ops,
.arg_create = &table_hash_params,
@@ -402,17 +325,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_CUCKOO_KEY112:
case e_APP_PIPELINE_HASH_CUCKOO_KEY128:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_cuckoo_ops,
.arg_create = &table_hash_params,
@@ -485,6 +397,7 @@ app_main_loop_worker_pipeline_hash(void) {
uint64_t test_hash(
void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed)
{
@@ -547,7 +460,7 @@ app_main_loop_rx_metadata(void) {
} else
continue;
- *signature = test_hash(key, 0, 0);
+ *signature = test_hash(key, NULL, 0, 0);
}
do {
diff --git a/test/test/test_table.c b/test/test/test_table.c
old mode 100644
new mode 100755
index 9e9eed8..db7d4e6
--- a/test/test/test_table.c
+++ b/test/test/test_table.c
@@ -72,6 +72,7 @@ static void app_init_rings(void);
static void app_init_mbuf_pools(void);
uint64_t pipeline_test_hash(void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed)
{
diff --git a/test/test/test_table.h b/test/test/test_table.h
old mode 100644
new mode 100755
index 84d1845..8c1df33
--- a/test/test/test_table.h
+++ b/test/test/test_table.h
@@ -94,7 +94,7 @@
APP_METADATA_OFFSET(32)); \
k32 = (uint32_t *) key; \
k32[0] = (value); \
- *signature = pipeline_test_hash(key, 0, 0); \
+ *signature = pipeline_test_hash(key, NULL, 0, 0); \
rte_ring_enqueue((ring), m); \
} while (0)
@@ -131,6 +131,7 @@
/* Function definitions */
uint64_t pipeline_test_hash(
void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed);
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index 9515dd0..e863788 100755
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -448,7 +448,7 @@ test_table_hash8lru(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -509,7 +509,7 @@ test_table_hash16lru(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -570,7 +570,7 @@ test_table_hash32lru(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -631,7 +631,7 @@ test_table_hash8ext(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -692,7 +692,7 @@ test_table_hash16ext(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -753,7 +753,7 @@ test_table_hash32ext(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -814,7 +814,7 @@ test_table_hash_cuckoo_combined(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 7f04212..a1b0c58 100755
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -60,7 +60,7 @@ table_test table_tests[] = {
memset(key, 0, 32); \
k32 = (uint32_t *) key; \
k32[0] = (value); \
- *signature = pipeline_test_hash(key, 0, 0); \
+ *signature = pipeline_test_hash(key, NULL, 0, 0); \
} while (0)
unsigned n_table_tests = RTE_DIM(table_tests);
@@ -674,7 +674,7 @@ test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size)
.key_mask = NULL,
.n_keys = 1 << 10,
.n_buckets = 1 << 10,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -691,7 +691,7 @@ test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size)
if (table != NULL)
return -4;
- hash_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
+ hash_params.f_hash = pipeline_test_hash;
table = ops->f_create(&hash_params, 0, 1);
if (table == NULL)
@@ -777,7 +777,7 @@ test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size)
.key_mask = NULL,
.n_keys = 1 << 10,
.n_buckets = 1 << 10,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -801,7 +801,7 @@ test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size)
if (table != NULL)
return -4;
- hash_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
+ hash_params.f_hash = pipeline_test_hash;
table = ops->f_create(&hash_params, 0, 1);
if (table == NULL)
@@ -970,7 +970,7 @@ test_table_hash_cuckoo(void)
if (table != NULL)
return -4;
- cuckoo_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
+ cuckoo_params.f_hash = pipeline_test_hash;
cuckoo_params.name = NULL;
table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 15/18] librte_table: cosmetic enhancements in api file
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (13 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 14/18] test: add mask-based hash functions Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 16/18] librte_table: copyright cosmetic updates Cristian Dumitrescu
` (21 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Consolidated hash tables into functional groups.
Minor changes in comments.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 34 +++++++++-------------------------
1 file changed, 9 insertions(+), 25 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 3c53d6a..081464c 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -45,8 +45,6 @@ extern "C" {
* These tables use the exact match criterion to uniquely associate data to
* lookup keys.
*
- * Use-cases: Flow classification table, Address Resolution Protocol (ARP) table
- *
* Hash table types:
* 1. Entry add strategy on bucket full:
* a. Least Recently Used (LRU): One of the existing keys in the bucket is
@@ -59,7 +57,7 @@ extern "C" {
* to the bucket, it also becomes the new MRU key. When a key needs to
* be picked and dropped, the most likely candidate for drop, i.e. the
* current LRU key, is always picked. The LRU logic requires maintaining
- * specific data structures per each bucket.
+ * specific data structures per each bucket. Use-cases: flow cache, etc.
* b. Extendible bucket (ext): The bucket is extended with space for 4 more
* keys. This is done by allocating additional memory at table init time,
* which is used to create a pool of free keys (the size of this pool is
@@ -73,20 +71,8 @@ extern "C" {
* first group of 4 keys, the search continues beyond the first group of
* 4 keys, potentially until all keys in this bucket are examined. The
* extendible bucket logic requires maintaining specific data structures
- * per table and per each bucket.
- * 2. Key signature computation:
- * a. Pre-computed key signature: The key lookup operation is split between
- * two CPU cores. The first CPU core (typically the CPU core performing
- * packet RX) extracts the key from the input packet, computes the key
- * signature and saves both the key and the key signature in the packet
- * buffer as packet meta-data. The second CPU core reads both the key and
- * the key signature from the packet meta-data and performs the bucket
- * search step of the key lookup operation.
- * b. Key signature computed on lookup (do-sig): The same CPU core reads
- * the key from the packet meta-data, uses it to compute the key
- * signature and also performs the bucket search step of the key lookup
- * operation.
- * 3. Key size:
+ * per table and per each bucket. Use-cases: flow table, etc.
+ * 2. Key size:
* a. Configurable key size
* b. Single key size (8-byte, 16-byte or 32-byte key size)
*
@@ -129,22 +115,20 @@ struct rte_table_hash_params {
uint64_t seed;
};
+/** Extendible bucket hash table operations */
extern struct rte_table_ops rte_table_hash_ext_ops;
+extern struct rte_table_ops rte_table_hash_key8_ext_ops;
+extern struct rte_table_ops rte_table_hash_key16_ext_ops;
+extern struct rte_table_ops rte_table_hash_key32_ext_ops;
+/** LRU hash table operations */
extern struct rte_table_ops rte_table_hash_lru_ops;
extern struct rte_table_ops rte_table_hash_key8_lru_ops;
-
-extern struct rte_table_ops rte_table_hash_key8_ext_ops;
-
extern struct rte_table_ops rte_table_hash_key16_lru_ops;
-
-extern struct rte_table_ops rte_table_hash_key16_ext_ops;
-
extern struct rte_table_ops rte_table_hash_key32_lru_ops;
-extern struct rte_table_ops rte_table_hash_key32_ext_ops;
-
+/** Cuckoo hash table operations */
extern struct rte_table_ops rte_table_hash_cuckoo_ops;
#ifdef __cplusplus
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 16/18] librte_table: copyright cosmetic updates
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (14 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 15/18] librte_table: cosmetic enhancements in api file Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 17/18] librte_table: map file updates Cristian Dumitrescu
` (20 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Removed incorrect white spaces and updated year in copyrigh headers.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 2 +-
lib/librte_table/rte_table_hash_cuckoo.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_ext.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_key16.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_key32.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_key8.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_lru.c | 52 ++++++++++++++++----------------
7 files changed, 157 insertions(+), 157 deletions(-)
mode change 100644 => 100755 lib/librte_table/rte_table_hash_cuckoo.c
mode change 100644 => 100755 lib/librte_table/rte_table_hash_key16.c
mode change 100644 => 100755 lib/librte_table/rte_table_hash_key32.c
mode change 100644 => 100755 lib/librte_table/rte_table_hash_key8.c
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 081464c..15f1902 100755
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
old mode 100644
new mode 100755
index 8d26597..f3845c7
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2016 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
index 0a743ad..3af1bca 100755
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
old mode 100644
new mode 100755
index 84b59cc..b541735
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
diff --git a/lib/librte_table/rte_table_hash_key32.c b/lib/librte_table/rte_table_hash_key32.c
old mode 100644
new mode 100755
index 3eb7e4f..d4364d6
--- a/lib/librte_table/rte_table_hash_key32.c
+++ b/lib/librte_table/rte_table_hash_key32.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
old mode 100644
new mode 100755
index a44f75c..9437304
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
index eebc1cd..139b8fb 100755
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 17/18] librte_table: map file updates
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (15 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 16/18] librte_table: copyright cosmetic updates Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table Cristian Dumitrescu
` (19 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Updated map file to current API. Several API items were removed,
so not using inheritance.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_version.map | 23 ++++++++++++++++++++++-
1 file changed, 22 insertions(+), 1 deletion(-)
mode change 100644 => 100755 lib/librte_table/rte_table_version.map
diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map
old mode 100644
new mode 100755
index e1eaa27..f03fc3f
--- a/lib/librte_table/rte_table_version.map
+++ b/lib/librte_table/rte_table_version.map
@@ -28,7 +28,7 @@ DPDK_2.2 {
rte_table_hash_key16_ext_dosig_ops;
rte_table_hash_key16_lru_dosig_ops;
-};
+} DPDK_2.0;
DPDK_16.07 {
global:
@@ -36,3 +36,24 @@ DPDK_16.07 {
rte_table_hash_cuckoo_dosig_ops;
} DPDK_2.0;
+
+DPDK_17.11 {
+ global:
+
+ rte_table_acl_ops;
+ rte_table_array_ops;
+ rte_table_hash_ext_ops;
+ rte_table_hash_key8_ext_ops;
+ rte_table_hash_key16_ext_ops;
+ rte_table_hash_key32_ext_ops;
+ rte_table_hash_lru_ops;
+ rte_table_hash_key8_lru_ops;
+ rte_table_hash_key16_lru_ops;
+ rte_table_hash_key32_lru_ops;
+ rte_table_hash_cuckoo_ops;
+ rte_table_lpm_ipv6_ops;
+ rte_table_lpm_ops;
+ rte_table_stub_ops;
+
+ local: *;
+};
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (16 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 17/18] librte_table: map file updates Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: add key mask for hash tables Cristian Dumitrescu
` (18 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
doc/guides/rel_notes/deprecation.rst | 6 ------
1 file changed, 6 deletions(-)
mode change 100644 => 100755 doc/guides/rel_notes/deprecation.rst
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
old mode 100644
new mode 100755
index 52058f5..2ea19e7
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -86,9 +86,3 @@ Deprecation Notices
* librte_meter: The API will change to accommodate configuration profiles.
Most of the API functions will have an additional opaque parameter.
-
-* librte_table: The ``key_mask`` parameter will be added to all the hash tables
- that currently do not have it, as well as to the hash compute function prototype.
- The non-"do-sig" versions of the hash tables will be removed
- (including the ``signature_offset`` parameter)
- and the "do-sig" versions renamed accordingly.
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 00/18] librte_table: add key mask for hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (17 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 17:10 ` Dumitrescu, Cristian
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 01/18] test: removing calls to deprecated " Cristian Dumitrescu
` (17 subsequent siblings)
36 siblings, 1 reply; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Main changes:
1. The key_mask parameter is added to all the hash tables that were
previously missing it, as well to the hash compute function. This was
first started in DPDK 2.0, but was only implemented for a couple of
hash tables. The benefit of this approach is that it allows for better
performance for large keys (bigger than 16 bytes), while it preserves
the same performance for small keys [Q&A1].
2. The precomputed key signature (i.e. non-"do-sig") versions have been
removed for all the hash tables, so now the key signature is always
computed on every lookup. Note that this approach also allows for the
precomputed key signature scheme [Q&A2].
3. API cleanup: single parameter structure common for all hash tables.
Q&A:
Q1: How is better lookup performance achieved by using key mask approach
for hash tables?
A1: This approach eliminates the need to consolidate the lookup key in a
single contiguous buffer where the relevant packet fields are written
one by one, which is a very costly operation that also hash strong data
dependencies.
Q2: How can the pre-computed key signature scheme be implemented with
current approach?
A2: The application can implement a straightforward custom hash function
that simply reads the pre-computed key signature from a given offset
in the input key buffer where it has been stored prior to the lookup
operation.
Cristian Dumitrescu (18):
test: removing calls to deprecated hash tables
librte_table: remove deprecated 8-byte key hash tables
librte_table: remove deprecated 16-byte key hash tables
librte_table: remove deprecated variable size key ext hash tables
librte_table: remove deprecated variable size key lru hash tables
librte_table: rename cuckoo hash table ops
librte_table: add unified params structure and mask-based hash func
librte_table: rework variable size key ext hash tables
librte_table: rework variable size key lru hash table
librte_table: rework 8-byte key hash tables
librte_table: rework 16-byte key hash tables
librte_table: rework 32-byte key hash tables
librte_table: rework cuckoo hash table
test: add mask-based hash functions
librte_table: cosmetic enhancements in api file
librte_table: copyright cosmetic updates
librte_table: map file updates
doc: remove deprecation notice for librte_table
doc/guides/rel_notes/deprecation.rst | 6 -
examples/ip_pipeline/pipeline/hash_func.h | 176 +++--
.../pipeline/pipeline_flow_classification.c | 10 +-
.../pipeline/pipeline_flow_classification_be.c | 51 +-
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 10 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 14 +-
lib/librte_table/rte_table_hash.h | 305 +--------
lib/librte_table/rte_table_hash_cuckoo.c | 205 +++---
lib/librte_table/rte_table_hash_ext.c | 417 ++++--------
lib/librte_table/rte_table_hash_key16.c | 750 ++++++---------------
lib/librte_table/rte_table_hash_key32.c | 436 +++++++-----
lib/librte_table/rte_table_hash_key8.c | 716 ++++++--------------
lib/librte_table/rte_table_hash_lru.c | 513 ++++++--------
lib/librte_table/rte_table_version.map | 23 +-
test/test-pipeline/main.h | 5 +-
test/test-pipeline/pipeline_hash.c | 107 +--
test/test/test_table.c | 1 +
test/test/test_table.h | 3 +-
test/test/test_table_combined.c | 137 ++--
test/test/test_table_tables.c | 150 ++---
20 files changed, 1440 insertions(+), 2595 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 01/18] test: removing calls to deprecated hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (18 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: add key mask for hash tables Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 02/18] librte_table: remove deprecated 8-byte key " Cristian Dumitrescu
` (16 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Removing calls to hash tables that are going to be removed later.
The calls are removed from test/test, test/test-pipeline,
examples/ip_pipeline.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 18 +++-----------
test/test-pipeline/pipeline_hash.c | 12 +++++-----
test/test/test_table_combined.c | 28 +++++++++++-----------
test/test/test_table_tables.c | 8 +++----
4 files changed, 27 insertions(+), 39 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 026f00c..5e993c3 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -542,29 +542,17 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
- if (p_fc->hash_offset != 0) {
- table_params.ops =
- &rte_table_hash_key8_ext_ops;
- } else {
- table_params.ops =
- &rte_table_hash_key8_ext_dosig_ops;
- }
+ table_params.ops = &rte_table_hash_key8_ext_dosig_ops;
table_params.arg_create = &table_hash_key8_params;
break;
case 16:
- if (p_fc->hash_offset != 0) {
- table_params.ops =
- &rte_table_hash_key16_ext_ops;
- } else {
- table_params.ops =
- &rte_table_hash_key16_ext_dosig_ops;
- }
+ table_params.ops = &rte_table_hash_key16_ext_dosig_ops;
table_params.arg_create = &table_hash_key16_params;
break;
default:
- table_params.ops = &rte_table_hash_ext_ops;
+ table_params.ops = &rte_table_hash_ext_dosig_ops;
table_params.arg_create = &table_hash_params;
}
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 991e381..6d27f33 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -187,7 +187,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_ext_ops,
+ .ops = &rte_table_hash_ext_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -215,7 +215,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_lru_ops,
+ .ops = &rte_table_hash_lru_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -241,7 +241,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_ext_ops,
+ .ops = &rte_table_hash_key8_ext_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -266,7 +266,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_lru_ops,
+ .ops = &rte_table_hash_key8_lru_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -292,7 +292,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key16_ext_ops,
+ .ops = &rte_table_hash_key16_ext_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -317,7 +317,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key16_lru_ops,
+ .ops = &rte_table_hash_key16_lru_dosig_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index a2d19a1..a337a59 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -469,7 +469,7 @@ test_table_hash8lru(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key8_lru_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -477,7 +477,7 @@ test_table_hash8lru(void)
/* Invalid parameters */
key8lru_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key8_lru_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -485,7 +485,7 @@ test_table_hash8lru(void)
key8lru_params.n_entries = 1<<16;
key8lru_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key8_lru_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -528,7 +528,7 @@ test_table_hash16lru(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key16_lru_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -536,7 +536,7 @@ test_table_hash16lru(void)
/* Invalid parameters */
key16lru_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key16_lru_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -544,7 +544,7 @@ test_table_hash16lru(void)
key16lru_params.n_entries = 1<<16;
key16lru_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key16_lru_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -646,7 +646,7 @@ test_table_hash8ext(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key8_ext_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -654,7 +654,7 @@ test_table_hash8ext(void)
/* Invalid parameters */
key8ext_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key8_ext_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -662,7 +662,7 @@ test_table_hash8ext(void)
key8ext_params.n_entries = 1<<16;
key8ext_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key8_ext_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -670,7 +670,7 @@ test_table_hash8ext(void)
key8ext_params.f_hash = pipeline_test_hash;
key8ext_params.n_entries_ext = 0;
- status = test_table_type(&rte_table_hash_key8_ext_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets, NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -713,7 +713,7 @@ test_table_hash16ext(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key16_ext_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -721,7 +721,7 @@ test_table_hash16ext(void)
/* Invalid parameters */
key16ext_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key16_ext_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -729,7 +729,7 @@ test_table_hash16ext(void)
key16ext_params.n_entries = 1<<16;
key16ext_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key16_ext_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -737,7 +737,7 @@ test_table_hash16ext(void)
key16ext_params.f_hash = pipeline_test_hash;
key16ext_params.n_entries_ext = 0;
- status = test_table_type(&rte_table_hash_key16_ext_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets, NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index d835eb9..90d2f02 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -895,7 +895,7 @@ test_table_hash_lru(void)
{
int status;
- status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_ops);
+ status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_dosig_ops);
if (status < 0)
return status;
@@ -904,7 +904,7 @@ test_table_hash_lru(void)
if (status < 0)
return status;
- status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_ops);
+ status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_dosig_ops);
if (status < 0)
return status;
@@ -924,7 +924,7 @@ test_table_hash_ext(void)
{
int status;
- status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_dosig_ops);
if (status < 0)
return status;
@@ -933,7 +933,7 @@ test_table_hash_ext(void)
if (status < 0)
return status;
- status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_dosig_ops);
if (status < 0)
return status;
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 02/18] librte_table: remove deprecated 8-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (19 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 01/18] test: removing calls to deprecated " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 03/18] librte_table: remove deprecated 16-byte " Cristian Dumitrescu
` (15 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The non-dosig version of the 8-byte key hash tables (both extendable
bucket and LRU) are removed. The remaining hash tables are renamed to
eliminate the dosig particle from their name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 2 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 2 +-
lib/librte_table/rte_table_hash.h | 9 -
lib/librte_table/rte_table_hash_key8.c | 340 +--------------------
test/test-pipeline/pipeline_hash.c | 4 +-
test/test/test_table_combined.c | 14 +-
test/test/test_table_tables.c | 8 +-
7 files changed, 21 insertions(+), 358 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 5e993c3..485ebcb 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -542,7 +542,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
- table_params.ops = &rte_table_hash_key8_ext_dosig_ops;
+ table_params.ops = &rte_table_hash_key8_ext_ops;
table_params.arg_create = &table_hash_key8_params;
break;
diff --git a/examples/ip_pipeline/pipeline/pipeline_routing_be.c b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
index 7831716..aa7f05e 100644
--- a/examples/ip_pipeline/pipeline/pipeline_routing_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
@@ -1359,7 +1359,7 @@ pipeline_routing_init(struct pipeline_params *params,
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_ext_dosig_ops,
+ .ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_arp_params,
.f_action_hit = get_arp_table_ah_hit(p_rt),
.f_action_miss = NULL,
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 57505a6..8ad812c 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -201,12 +201,8 @@ struct rte_table_hash_key8_lru_params {
uint8_t *key_mask;
};
-/** LRU hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key8_lru_ops;
-/** LRU hash table operations for key signature computed on lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key8_lru_dosig_ops;
-
/** Extendible bucket hash table parameters */
struct rte_table_hash_key8_ext_params {
/** Maximum number of entries (and keys) in the table */
@@ -234,13 +230,8 @@ struct rte_table_hash_key8_ext_params {
uint8_t *key_mask;
};
-/** Extendible bucket hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key8_ext_ops;
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key8_ext_dosig_ops;
-
/**
* 16-byte key hash tables
*
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
index 5f0c656..f2b285d 100644
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -583,18 +583,6 @@ rte_table_hash_entry_delete_key8_ext(
#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
- uint64_t signature; \
- uint32_t bucket_index; \
- \
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
- bucket_index = signature & (f->n_buckets - 1); \
- bucket1 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket_index * f->bucket_size]; \
- rte_prefetch0(bucket1); \
-}
-
-#define lookup1_stage1_dosig(mbuf1, bucket1, f) \
-{ \
uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
@@ -738,25 +726,7 @@ rte_table_hash_entry_delete_key8_ext(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
-{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
- \
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
- bucket10 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
- \
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
- bucket11 = (struct rte_bucket_4_8 *) \
- &f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
-}
-
-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f)\
+#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\
{ \
uint64_t *key10, *key11; \
uint64_t hash_offset_buffer10; \
@@ -871,8 +841,8 @@ rte_table_hash_lookup_key8_lru(
struct rte_table_hash *f = (struct rte_table_hash *) table;
struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index,
- pkt11_index, pkt20_index, pkt21_index;
+ uint32_t pkt00_index, pkt01_index, pkt10_index;
+ uint32_t pkt11_index, pkt20_index, pkt21_index;
uint64_t pkts_mask_out = 0;
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
@@ -888,7 +858,7 @@ rte_table_hash_lookup_key8_lru(
lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
lookup1_stage1(mbuf, bucket, f);
lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
+ pkts_mask_out, entries, f);
}
*lookup_hit_mask = pkts_mask_out;
@@ -987,132 +957,6 @@ rte_table_hash_lookup_key8_lru(
} /* rte_table_hash_lookup_key8_lru() */
static int
-rte_table_hash_lookup_key8_lru_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_8 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key8_lru_dosig() */
-
-static int
rte_table_hash_lookup_key8_ext(
void *table,
struct rte_mbuf **pkts,
@@ -1142,8 +986,8 @@ rte_table_hash_lookup_key8_ext(
lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
lookup1_stage1(mbuf, bucket, f);
lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask, buckets,
- keys, f);
+ pkts_mask_out, entries, buckets_mask,
+ buckets, keys, f);
}
goto grind_next_buckets;
@@ -1263,156 +1107,6 @@ rte_table_hash_lookup_key8_ext(
} /* rte_table_hash_lookup_key8_ext() */
static int
-rte_table_hash_lookup_key8_ext_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_8 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0, buckets_mask = 0;
- struct rte_bucket_4_8 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];
- uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_KEY8_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_8 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask,
- buckets, keys, f);
- }
-
- goto grind_next_buckets;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
-grind_next_buckets:
- /* Grind next buckets */
- for ( ; buckets_mask; ) {
- uint64_t buckets_mask_next = 0;
-
- for ( ; buckets_mask; ) {
- uint64_t pkt_mask;
- uint32_t pkt_index;
-
- pkt_index = __builtin_ctzll(buckets_mask);
- pkt_mask = 1LLU << pkt_index;
- buckets_mask &= ~pkt_mask;
-
- lookup_grinder(pkt_index, buckets, keys, pkts_mask_out,
- entries, buckets_mask_next, f);
- }
-
- buckets_mask = buckets_mask_next;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key8_dosig_ext() */
-
-static int
rte_table_hash_key8_stats_read(void *table, struct rte_table_stats *stats, int clear)
{
struct rte_table_hash *t = table;
@@ -1437,17 +1131,6 @@ struct rte_table_ops rte_table_hash_key8_lru_ops = {
.f_stats = rte_table_hash_key8_stats_read,
};
-struct rte_table_ops rte_table_hash_key8_lru_dosig_ops = {
- .f_create = rte_table_hash_create_key8_lru,
- .f_free = rte_table_hash_free_key8_lru,
- .f_add = rte_table_hash_entry_add_key8_lru,
- .f_delete = rte_table_hash_entry_delete_key8_lru,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lookup_key8_lru_dosig,
- .f_stats = rte_table_hash_key8_stats_read,
-};
-
struct rte_table_ops rte_table_hash_key8_ext_ops = {
.f_create = rte_table_hash_create_key8_ext,
.f_free = rte_table_hash_free_key8_ext,
@@ -1458,14 +1141,3 @@ struct rte_table_ops rte_table_hash_key8_ext_ops = {
.f_lookup = rte_table_hash_lookup_key8_ext,
.f_stats = rte_table_hash_key8_stats_read,
};
-
-struct rte_table_ops rte_table_hash_key8_ext_dosig_ops = {
- .f_create = rte_table_hash_create_key8_ext,
- .f_free = rte_table_hash_free_key8_ext,
- .f_add = rte_table_hash_entry_add_key8_ext,
- .f_delete = rte_table_hash_entry_delete_key8_ext,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lookup_key8_ext_dosig,
- .f_stats = rte_table_hash_key8_stats_read,
-};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 6d27f33..8f384fa 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -241,7 +241,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_ext_dosig_ops,
+ .ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -266,7 +266,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key8_lru_dosig_ops,
+ .ops = &rte_table_hash_key8_lru_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index a337a59..680454f 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -469,7 +469,7 @@ test_table_hash8lru(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -477,7 +477,7 @@ test_table_hash8lru(void)
/* Invalid parameters */
key8lru_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -485,7 +485,7 @@ test_table_hash8lru(void)
key8lru_params.n_entries = 1<<16;
key8lru_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key8_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -646,7 +646,7 @@ test_table_hash8ext(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -654,7 +654,7 @@ test_table_hash8ext(void)
/* Invalid parameters */
key8ext_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -662,7 +662,7 @@ test_table_hash8ext(void)
key8ext_params.n_entries = 1<<16;
key8ext_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -670,7 +670,7 @@ test_table_hash8ext(void)
key8ext_params.f_hash = pipeline_test_hash;
key8ext_params.n_entries_ext = 0;
- status = test_table_type(&rte_table_hash_key8_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets, NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 90d2f02..43280b0 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -895,12 +895,12 @@ test_table_hash_lru(void)
{
int status;
- status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_dosig_ops);
+ status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_ops);
if (status < 0)
return status;
status = test_table_hash_lru_generic(
- &rte_table_hash_key8_lru_dosig_ops);
+ &rte_table_hash_key8_lru_ops);
if (status < 0)
return status;
@@ -924,12 +924,12 @@ test_table_hash_ext(void)
{
int status;
- status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_dosig_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops);
if (status < 0)
return status;
status = test_table_hash_ext_generic(
- &rte_table_hash_key8_ext_dosig_ops);
+ &rte_table_hash_key8_ext_ops);
if (status < 0)
return status;
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 03/18] librte_table: remove deprecated 16-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (20 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 02/18] librte_table: remove deprecated 8-byte key " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 04/18] librte_table: remove deprecated variable size key ext " Cristian Dumitrescu
` (14 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The non-dosig version of the 16-byte key hash tables (both extendable
bucket and LRU) are removed. The remaining hash tables are renamed to
eliminate the dosig particle from their name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 2 +-
lib/librte_table/rte_table_hash.h | 10 -
lib/librte_table/rte_table_hash_key16.c | 342 +--------------------
test/test-pipeline/pipeline_hash.c | 4 +-
test/test/test_table_combined.c | 14 +-
test/test/test_table_tables.c | 4 +-
6 files changed, 16 insertions(+), 360 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 485ebcb..0f667e6 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -547,7 +547,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
break;
case 16:
- table_params.ops = &rte_table_hash_key16_ext_dosig_ops;
+ table_params.ops = &rte_table_hash_key16_ext_ops;
table_params.arg_create = &table_hash_key16_params;
break;
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 8ad812c..57c6073 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -259,13 +259,8 @@ struct rte_table_hash_key16_lru_params {
uint8_t *key_mask;
};
-/** LRU hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key16_lru_ops;
-/** LRU hash table operations for key signature computed on lookup
- ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key16_lru_dosig_ops;
-
/** Extendible bucket hash table parameters */
struct rte_table_hash_key16_ext_params {
/** Maximum number of entries (and keys) in the table */
@@ -293,13 +288,8 @@ struct rte_table_hash_key16_ext_params {
uint8_t *key_mask;
};
-/** Extendible bucket operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key16_ext_ops;
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_key16_ext_dosig_ops;
-
/**
* 32-byte key hash tables
*
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
index ce057b7..4ed5c78 100644
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -610,19 +610,6 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
- uint64_t signature; \
- uint32_t bucket_index; \
- \
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
- bucket_index = signature & (f->n_buckets - 1); \
- bucket1 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket_index * f->bucket_size]; \
- rte_prefetch0(bucket1); \
- rte_prefetch0((void *)(((uintptr_t) bucket1) + RTE_CACHE_LINE_SIZE));\
-}
-
-#define lookup1_stage1_dosig(mbuf1, bucket1, f) \
-{ \
uint64_t *key; \
uint64_t signature = 0; \
uint32_t bucket_index; \
@@ -775,26 +762,6 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
- \
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
- bucket10 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
- rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
- \
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
- bucket11 = (struct rte_bucket_4_16 *) \
- &f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
- rte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\
-}
-
-#define lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f) \
-{ \
uint64_t *key10, *key11; \
uint64_t hash_offset_buffer[2]; \
uint64_t signature10, signature11; \
@@ -916,6 +883,7 @@ rte_table_hash_lookup_key16_lru(
uint64_t pkts_mask_out = 0;
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
+
RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
/* Cannot run the pipeline with less than 5 packets */
@@ -932,8 +900,8 @@ rte_table_hash_lookup_key16_lru(
}
*lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f,
- n_pkts_in - __builtin_popcountll(pkts_mask_out));
+ RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
+ __builtin_popcountll(pkts_mask_out));
return 0;
}
@@ -1029,135 +997,6 @@ rte_table_hash_lookup_key16_lru(
} /* rte_table_hash_lookup_key16_lru() */
static int
-rte_table_hash_lookup_key16_lru_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
-
- RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_16 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_lru(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, f);
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_lru(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries, f);
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key16_lru_dosig() */
-
-static int
rte_table_hash_lookup_key16_ext(
void *table,
struct rte_mbuf **pkts,
@@ -1175,6 +1014,7 @@ rte_table_hash_lookup_key16_ext(
uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
__rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
+
RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
/* Cannot run the pipeline with less than 5 packets */
@@ -1309,158 +1149,6 @@ rte_table_hash_lookup_key16_ext(
} /* rte_table_hash_lookup_key16_ext() */
static int
-rte_table_hash_lookup_key16_ext_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *f = (struct rte_table_hash *) table;
- struct rte_bucket_4_16 *bucket10, *bucket11, *bucket20, *bucket21;
- struct rte_mbuf *mbuf00, *mbuf01, *mbuf10, *mbuf11, *mbuf20, *mbuf21;
- uint32_t pkt00_index, pkt01_index, pkt10_index;
- uint32_t pkt11_index, pkt20_index, pkt21_index;
- uint64_t pkts_mask_out = 0, buckets_mask = 0;
- struct rte_bucket_4_16 *buckets[RTE_PORT_IN_BURST_SIZE_MAX];
- uint64_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
-
- RTE_TABLE_HASH_KEY16_STATS_PKTS_IN_ADD(f, n_pkts_in);
-
- /* Cannot run the pipeline with less than 5 packets */
- if (__builtin_popcountll(pkts_mask) < 5) {
- for ( ; pkts_mask; ) {
- struct rte_bucket_4_16 *bucket;
- struct rte_mbuf *mbuf;
- uint32_t pkt_index;
-
- lookup1_stage0(pkt_index, mbuf, pkts, pkts_mask, f);
- lookup1_stage1_dosig(mbuf, bucket, f);
- lookup1_stage2_ext(pkt_index, mbuf, bucket,
- pkts_mask_out, entries, buckets_mask,
- buckets, keys, f);
- }
-
- goto grind_next_buckets;
- }
-
- /*
- * Pipeline fill
- *
- */
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline feed */
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(pkt00_index, pkt01_index, mbuf00, mbuf01, pkts,
- pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(pkt00_index, pkt01_index,
- mbuf00, mbuf01, pkts, pkts_mask, f);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
- }
-
- /*
- * Pipeline flush
- *
- */
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- mbuf10 = mbuf00;
- mbuf11 = mbuf01;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(mbuf10, mbuf11, bucket10, bucket11, f);
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
- /* Pipeline feed */
- bucket20 = bucket10;
- bucket21 = bucket11;
- mbuf20 = mbuf10;
- mbuf21 = mbuf11;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2_ext(pkt20_index, pkt21_index, mbuf20, mbuf21,
- bucket20, bucket21, pkts_mask_out, entries,
- buckets_mask, buckets, keys, f);
-
-grind_next_buckets:
- /* Grind next buckets */
- for ( ; buckets_mask; ) {
- uint64_t buckets_mask_next = 0;
-
- for ( ; buckets_mask; ) {
- uint64_t pkt_mask;
- uint32_t pkt_index;
-
- pkt_index = __builtin_ctzll(buckets_mask);
- pkt_mask = 1LLU << pkt_index;
- buckets_mask &= ~pkt_mask;
-
- lookup_grinder(pkt_index, buckets, keys, pkts_mask_out,
- entries, buckets_mask_next, f);
- }
-
- buckets_mask = buckets_mask_next;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
- __builtin_popcountll(pkts_mask_out));
- return 0;
-} /* rte_table_hash_lookup_key16_ext_dosig() */
-
-static int
rte_table_hash_key16_stats_read(void *table, struct rte_table_stats *stats, int clear)
{
struct rte_table_hash *t = table;
@@ -1479,37 +1167,15 @@ struct rte_table_ops rte_table_hash_key16_lru_ops = {
.f_free = rte_table_hash_free_key16_lru,
.f_add = rte_table_hash_entry_add_key16_lru,
.f_delete = rte_table_hash_entry_delete_key16_lru,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
.f_lookup = rte_table_hash_lookup_key16_lru,
.f_stats = rte_table_hash_key16_stats_read,
};
-struct rte_table_ops rte_table_hash_key16_lru_dosig_ops = {
- .f_create = rte_table_hash_create_key16_lru,
- .f_free = rte_table_hash_free_key16_lru,
- .f_add = rte_table_hash_entry_add_key16_lru,
- .f_delete = rte_table_hash_entry_delete_key16_lru,
- .f_lookup = rte_table_hash_lookup_key16_lru_dosig,
- .f_stats = rte_table_hash_key16_stats_read,
-};
-
struct rte_table_ops rte_table_hash_key16_ext_ops = {
.f_create = rte_table_hash_create_key16_ext,
.f_free = rte_table_hash_free_key16_ext,
.f_add = rte_table_hash_entry_add_key16_ext,
.f_delete = rte_table_hash_entry_delete_key16_ext,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
.f_lookup = rte_table_hash_lookup_key16_ext,
.f_stats = rte_table_hash_key16_stats_read,
};
-
-struct rte_table_ops rte_table_hash_key16_ext_dosig_ops = {
- .f_create = rte_table_hash_create_key16_ext,
- .f_free = rte_table_hash_free_key16_ext,
- .f_add = rte_table_hash_entry_add_key16_ext,
- .f_delete = rte_table_hash_entry_delete_key16_ext,
- .f_lookup = rte_table_hash_lookup_key16_ext_dosig,
- .f_stats = rte_table_hash_key16_stats_read,
-};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 8f384fa..b4e4e08 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -292,7 +292,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key16_ext_dosig_ops,
+ .ops = &rte_table_hash_key16_ext_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
@@ -317,7 +317,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_key16_lru_dosig_ops,
+ .ops = &rte_table_hash_key16_lru_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index 680454f..a2d19a1 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -528,7 +528,7 @@ test_table_hash16lru(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -536,7 +536,7 @@ test_table_hash16lru(void)
/* Invalid parameters */
key16lru_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -544,7 +544,7 @@ test_table_hash16lru(void)
key16lru_params.n_entries = 1<<16;
key16lru_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key16_lru_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -713,7 +713,7 @@ test_table_hash16ext(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -721,7 +721,7 @@ test_table_hash16ext(void)
/* Invalid parameters */
key16ext_params.n_entries = 0;
- status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -729,7 +729,7 @@ test_table_hash16ext(void)
key16ext_params.n_entries = 1<<16;
key16ext_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -737,7 +737,7 @@ test_table_hash16ext(void)
key16ext_params.f_hash = pipeline_test_hash;
key16ext_params.n_entries_ext = 0;
- status = test_table_type(&rte_table_hash_key16_ext_dosig_ops,
+ status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets, NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 43280b0..19aa5a4 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -904,7 +904,7 @@ test_table_hash_lru(void)
if (status < 0)
return status;
- status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_dosig_ops);
+ status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_ops);
if (status < 0)
return status;
@@ -933,7 +933,7 @@ test_table_hash_ext(void)
if (status < 0)
return status;
- status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_dosig_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops);
if (status < 0)
return status;
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 04/18] librte_table: remove deprecated variable size key ext hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (21 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 03/18] librte_table: remove deprecated 16-byte " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 05/18] librte_table: remove deprecated variable size key lru " Cristian Dumitrescu
` (13 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The non-dosig version of the variable size key extendible bucket
hash tables are removed. The remaining hash tables are renamed to
eliminate the dosig particle from their name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 2 +-
lib/librte_table/rte_table_hash.h | 5 -
lib/librte_table/rte_table_hash_ext.c | 196 +--------------------
test/test-pipeline/pipeline_hash.c | 2 +-
4 files changed, 8 insertions(+), 197 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 0f667e6..191cb15 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -552,7 +552,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
break;
default:
- table_params.ops = &rte_table_hash_ext_dosig_ops;
+ table_params.ops = &rte_table_hash_ext_ops;
table_params.arg_create = &table_hash_params;
}
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 57c6073..0eb23a7
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -135,13 +135,8 @@ struct rte_table_hash_ext_params {
uint32_t key_offset;
};
-/** Extendible bucket hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_ext_ops;
-/** Extendible bucket hash table operations for key signature computed on
- lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_ext_dosig_ops;
-
/** LRU hash table parameters */
struct rte_table_hash_lru_params {
/** Key size (number of bytes) */
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
index e718102..c4824c3
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -437,8 +437,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
- void **entries,
- int dosig)
+ void **entries)
{
struct rte_table_hash *t = (struct rte_table_hash *) table;
uint64_t pkts_mask_out = 0;
@@ -458,11 +457,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- if (dosig)
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
- else
- sig = RTE_MBUF_METADATA_UINT32(pkt,
- t->signature_offset);
+ sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
@@ -685,38 +680,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
-{ \
- struct grinder *g10, *g11; \
- uint64_t sig10, sig11, bkt10_index, bkt11_index; \
- struct rte_mbuf *mbuf10, *mbuf11; \
- struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
- uint64_t bucket_mask = t->bucket_mask; \
- uint32_t signature_offset = t->signature_offset; \
- \
- mbuf10 = pkts[pkt10_index]; \
- sig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\
- bkt10_index = sig10 & bucket_mask; \
- bkt10 = &buckets[bkt10_index]; \
- \
- mbuf11 = pkts[pkt11_index]; \
- sig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\
- bkt11_index = sig11 & bucket_mask; \
- bkt11 = &buckets[bkt11_index]; \
- \
- rte_prefetch0(bkt10); \
- rte_prefetch0(bkt11); \
- \
- g10 = &g[pkt10_index]; \
- g10->sig = sig10; \
- g10->bkt = bkt10; \
- \
- g11 = &g[pkt11_index]; \
- g11->sig = sig11; \
- g11->bkt = bkt11; \
-}
-
-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index) \
+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
{ \
struct grinder *g10, *g11; \
uint64_t sig10, sig11, bkt10_index, bkt11_index; \
@@ -874,7 +838,7 @@ static int rte_table_hash_ext_lookup(
/* Cannot run the pipeline with less than 7 packets */
if (__builtin_popcountll(pkts_mask) < 7) {
status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 0);
+ pkts_mask, lookup_hit_mask, entries);
RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -
__builtin_popcountll(*lookup_hit_mask));
return status;
@@ -982,144 +946,7 @@ static int rte_table_hash_ext_lookup(
uint64_t pkts_mask_out_slow = 0;
status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 0);
- pkts_mask_out |= pkts_mask_out_slow;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return status;
-}
-
-static int rte_table_hash_ext_lookup_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *t = (struct rte_table_hash *) table;
- struct grinder *g = t->grinders;
- uint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;
- uint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;
- uint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;
- int status = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_EXT_STATS_PKTS_IN_ADD(t, n_pkts_in);
-
- /* Cannot run the pipeline with less than 7 packets */
- if (__builtin_popcountll(pkts_mask) < 7) {
- status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 1);
- RTE_TABLE_HASH_EXT_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in -
- __builtin_popcountll(*lookup_hit_mask));
- return status;
- }
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline feed */
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline feed */
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,
- pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index,
- pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,
- pkts_mask_out, entries);
- }
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Slow path */
- pkts_mask_match_many &= ~pkts_mask_out;
- if (pkts_mask_match_many) {
- uint64_t pkts_mask_out_slow = 0;
-
- status = rte_table_hash_ext_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 1);
+ pkts_mask_match_many, &pkts_mask_out_slow, entries);
pkts_mask_out |= pkts_mask_out_slow;
}
@@ -1142,7 +969,7 @@ rte_table_hash_ext_stats_read(void *table, struct rte_table_stats *stats, int cl
return 0;
}
-struct rte_table_ops rte_table_hash_ext_ops = {
+struct rte_table_ops rte_table_hash_ext_ops = {
.f_create = rte_table_hash_ext_create,
.f_free = rte_table_hash_ext_free,
.f_add = rte_table_hash_ext_entry_add,
@@ -1152,14 +979,3 @@ struct rte_table_ops rte_table_hash_ext_ops = {
.f_lookup = rte_table_hash_ext_lookup,
.f_stats = rte_table_hash_ext_stats_read,
};
-
-struct rte_table_ops rte_table_hash_ext_dosig_ops = {
- .f_create = rte_table_hash_ext_create,
- .f_free = rte_table_hash_ext_free,
- .f_add = rte_table_hash_ext_entry_add,
- .f_delete = rte_table_hash_ext_entry_delete,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_ext_lookup_dosig,
- .f_stats = rte_table_hash_ext_stats_read,
-};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index b4e4e08..e473c42 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -187,7 +187,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_ext_dosig_ops,
+ .ops = &rte_table_hash_ext_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 05/18] librte_table: remove deprecated variable size key lru hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (22 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 04/18] librte_table: remove deprecated variable size key ext " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 06/18] librte_table: rename cuckoo hash table ops Cristian Dumitrescu
` (12 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The non-dosig version of the variable size key Least Recently Used
(LRU) hash tables are removed. The remaining hash tables are renamed
to eliminate the dosig particle from their name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 4 -
lib/librte_table/rte_table_hash_lru.c | 190 +---------------------------------
test/test-pipeline/pipeline_hash.c | 2 +-
3 files changed, 6 insertions(+), 190 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 0eb23a7..f983518 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -163,12 +163,8 @@ struct rte_table_hash_lru_params {
uint32_t key_offset;
};
-/** LRU hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_lru_ops;
-/** LRU hash table operations for key signature computed on lookup ("do-sig") */
-extern struct rte_table_ops rte_table_hash_lru_dosig_ops;
-
/**
* 8-byte key hash tables
*
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
index 5a4864e..0a85123
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -365,8 +365,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
- void **entries,
- int dosig)
+ void **entries)
{
struct rte_table_hash *t = (struct rte_table_hash *) table;
uint64_t pkts_mask_out = 0;
@@ -387,11 +386,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- if (dosig)
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
- else
- sig = RTE_MBUF_METADATA_UINT32(pkt,
- t->signature_offset);
+ sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
@@ -616,38 +611,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf01, key_offset));\
}
-#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index) \
-{ \
- struct grinder *g10, *g11; \
- uint64_t sig10, sig11, bkt10_index, bkt11_index; \
- struct rte_mbuf *mbuf10, *mbuf11; \
- struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
- uint64_t bucket_mask = t->bucket_mask; \
- uint32_t signature_offset = t->signature_offset; \
- \
- mbuf10 = pkts[pkt10_index]; \
- sig10 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf10, signature_offset);\
- bkt10_index = sig10 & bucket_mask; \
- bkt10 = &buckets[bkt10_index]; \
- \
- mbuf11 = pkts[pkt11_index]; \
- sig11 = (uint64_t) RTE_MBUF_METADATA_UINT32(mbuf11, signature_offset);\
- bkt11_index = sig11 & bucket_mask; \
- bkt11 = &buckets[bkt11_index]; \
- \
- rte_prefetch0(bkt10); \
- rte_prefetch0(bkt11); \
- \
- g10 = &g[pkt10_index]; \
- g10->sig = sig10; \
- g10->bkt = bkt10; \
- \
- g11 = &g[pkt11_index]; \
- g11->sig = sig11; \
- g11->bkt = bkt11; \
-}
-
-#define lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index)\
+#define lookup2_stage1(t, g, pkts, pkt10_index, pkt11_index)\
{ \
struct grinder *g10, *g11; \
uint64_t sig10, sig11, bkt10_index, bkt11_index; \
@@ -819,7 +783,7 @@ static int rte_table_hash_lru_lookup(
/* Cannot run the pipeline with less than 7 packets */
if (__builtin_popcountll(pkts_mask) < 7)
return rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 0);
+ pkts_mask, lookup_hit_mask, entries);
/* Pipeline stage 0 */
lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
@@ -923,140 +887,7 @@ static int rte_table_hash_lru_lookup(
uint64_t pkts_mask_out_slow = 0;
status = rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 0);
- pkts_mask_out |= pkts_mask_out_slow;
- }
-
- *lookup_hit_mask = pkts_mask_out;
- RTE_TABLE_HASH_LRU_STATS_PKTS_LOOKUP_MISS(t, n_pkts_in - __builtin_popcountll(pkts_mask_out));
- return status;
-}
-
-static int rte_table_hash_lru_lookup_dosig(
- void *table,
- struct rte_mbuf **pkts,
- uint64_t pkts_mask,
- uint64_t *lookup_hit_mask,
- void **entries)
-{
- struct rte_table_hash *t = (struct rte_table_hash *) table;
- struct grinder *g = t->grinders;
- uint64_t pkt00_index, pkt01_index, pkt10_index, pkt11_index;
- uint64_t pkt20_index, pkt21_index, pkt30_index, pkt31_index;
- uint64_t pkts_mask_out = 0, pkts_mask_match_many = 0;
- int status = 0;
-
- __rte_unused uint32_t n_pkts_in = __builtin_popcountll(pkts_mask);
- RTE_TABLE_HASH_LRU_STATS_PKTS_IN_ADD(t, n_pkts_in);
-
- /* Cannot run the pipeline with less than 7 packets */
- if (__builtin_popcountll(pkts_mask) < 7)
- return rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask, lookup_hit_mask, entries, 1);
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline feed */
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline feed */
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /*
- * Pipeline run
- *
- */
- for ( ; pkts_mask; ) {
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 0 */
- lookup2_stage0_with_odd_support(t, g, pkts, pkts_mask,
- pkt00_index, pkt01_index);
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index,
- pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index,
- pkts_mask_out, entries);
- }
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
- pkt10_index = pkt00_index;
- pkt11_index = pkt01_index;
-
- /* Pipeline stage 1 */
- lookup2_stage1_dosig(t, g, pkts, pkt10_index, pkt11_index);
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
- pkt20_index = pkt10_index;
- pkt21_index = pkt11_index;
-
- /* Pipeline stage 2 */
- lookup2_stage2(t, g, pkt20_index, pkt21_index, pkts_mask_match_many);
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Pipeline feed */
- pkt30_index = pkt20_index;
- pkt31_index = pkt21_index;
-
- /* Pipeline stage 3 */
- lookup2_stage3(t, g, pkts, pkt30_index, pkt31_index, pkts_mask_out,
- entries);
-
- /* Slow path */
- pkts_mask_match_many &= ~pkts_mask_out;
- if (pkts_mask_match_many) {
- uint64_t pkts_mask_out_slow = 0;
-
- status = rte_table_hash_lru_lookup_unoptimized(table, pkts,
- pkts_mask_match_many, &pkts_mask_out_slow, entries, 1);
+ pkts_mask_match_many, &pkts_mask_out_slow, entries);
pkts_mask_out |= pkts_mask_out_slow;
}
@@ -1089,14 +920,3 @@ struct rte_table_ops rte_table_hash_lru_ops = {
.f_lookup = rte_table_hash_lru_lookup,
.f_stats = rte_table_hash_lru_stats_read,
};
-
-struct rte_table_ops rte_table_hash_lru_dosig_ops = {
- .f_create = rte_table_hash_lru_create,
- .f_free = rte_table_hash_lru_free,
- .f_add = rte_table_hash_lru_entry_add,
- .f_delete = rte_table_hash_lru_entry_delete,
- .f_add_bulk = NULL,
- .f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_lru_lookup_dosig,
- .f_stats = rte_table_hash_lru_stats_read,
-};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index e473c42..991e381 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -215,7 +215,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_lru_dosig_ops,
+ .ops = &rte_table_hash_lru_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 06/18] librte_table: rename cuckoo hash table ops
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (23 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 05/18] librte_table: remove deprecated variable size key lru " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 07/18] librte_table: add unified params structure and mask-based hash func Cristian Dumitrescu
` (11 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
The dosig particle is eliminated from the cuckoo hash ops name.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 3 +--
lib/librte_table/rte_table_hash_cuckoo.c | 6 ++---
test/test-pipeline/pipeline_hash.c | 2 +-
test/test/test_table_combined.c | 8 +++---
test/test/test_table_tables.c | 42 ++++++++++++++++----------------
5 files changed, 30 insertions(+), 31 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index f983518..5f655ee 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -361,8 +361,7 @@ struct rte_table_hash_cuckoo_params {
const char *name;
};
-/** Cuckoo hash table operations */
-extern struct rte_table_ops rte_table_hash_cuckoo_dosig_ops;
+extern struct rte_table_ops rte_table_hash_cuckoo_ops;
#ifdef __cplusplus
}
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
index da1597f..beb45c5 100644
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -286,7 +286,7 @@ rte_table_hash_cuckoo_entry_delete(void *table, void *key,
static int
-rte_table_hash_cuckoo_lookup_dosig(void *table,
+rte_table_hash_cuckoo_lookup(void *table,
struct rte_mbuf **pkts,
uint64_t pkts_mask,
uint64_t *lookup_hit_mask,
@@ -370,13 +370,13 @@ rte_table_hash_cuckoo_stats_read(void *table, struct rte_table_stats *stats,
return 0;
}
-struct rte_table_ops rte_table_hash_cuckoo_dosig_ops = {
+struct rte_table_ops rte_table_hash_cuckoo_ops = {
.f_create = rte_table_hash_cuckoo_create,
.f_free = rte_table_hash_cuckoo_free,
.f_add = rte_table_hash_cuckoo_entry_add,
.f_delete = rte_table_hash_cuckoo_entry_delete,
.f_add_bulk = NULL,
.f_delete_bulk = NULL,
- .f_lookup = rte_table_hash_cuckoo_lookup_dosig,
+ .f_lookup = rte_table_hash_cuckoo_lookup,
.f_stats = rte_table_hash_cuckoo_stats_read,
};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 991e381..2f8c625 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -406,7 +406,7 @@ app_main_loop_worker_pipeline_hash(void) {
};
struct rte_pipeline_table_params table_params = {
- .ops = &rte_table_hash_cuckoo_dosig_ops,
+ .ops = &rte_table_hash_cuckoo_ops,
.arg_create = &table_hash_params,
.f_action_hit = NULL,
.f_action_miss = NULL,
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index a2d19a1..f65d34a 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -847,7 +847,7 @@ test_table_hash_cuckoo_combined(void)
table_packets.n_hit_packets = 50;
table_packets.n_miss_packets = 50;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_OK);
@@ -855,7 +855,7 @@ test_table_hash_cuckoo_combined(void)
/* Invalid parameters */
cuckoo_params.key_size = 0;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -863,7 +863,7 @@ test_table_hash_cuckoo_combined(void)
cuckoo_params.key_size = 32;
cuckoo_params.n_keys = 0;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
@@ -871,7 +871,7 @@ test_table_hash_cuckoo_combined(void)
cuckoo_params.n_keys = 1<<16;
cuckoo_params.f_hash = NULL;
- status = test_table_type(&rte_table_hash_cuckoo_dosig_ops,
+ status = test_table_type(&rte_table_hash_cuckoo_ops,
(void *)&cuckoo_params, (void *)key_cuckoo, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 19aa5a4..efbd0c6 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -969,13 +969,13 @@ test_table_hash_cuckoo(void)
.name = "CUCKOO",
};
- table = rte_table_hash_cuckoo_dosig_ops.f_create(NULL, 0, entry_size);
+ table = rte_table_hash_cuckoo_ops.f_create(NULL, 0, entry_size);
if (table != NULL)
return -1;
cuckoo_params.key_size = 0;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -2;
@@ -983,7 +983,7 @@ test_table_hash_cuckoo(void)
cuckoo_params.key_size = 32;
cuckoo_params.n_keys = 0;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -3;
@@ -991,7 +991,7 @@ test_table_hash_cuckoo(void)
cuckoo_params.n_keys = 1 << 24;
cuckoo_params.f_hash = NULL;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -4;
@@ -999,24 +999,24 @@ test_table_hash_cuckoo(void)
cuckoo_params.f_hash = pipeline_test_hash;
cuckoo_params.name = NULL;
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table != NULL)
return -5;
cuckoo_params.name = "CUCKOO";
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params,
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
0, entry_size);
if (table == NULL)
return -6;
/* Free */
- status = rte_table_hash_cuckoo_dosig_ops.f_free(table);
+ status = rte_table_hash_cuckoo_ops.f_free(table);
if (status < 0)
return -7;
- status = rte_table_hash_cuckoo_dosig_ops.f_free(NULL);
+ status = rte_table_hash_cuckoo_ops.f_free(NULL);
if (status == 0)
return -8;
@@ -1027,60 +1027,60 @@ test_table_hash_cuckoo(void)
memset(key_cuckoo, 0, 32);
kcuckoo[0] = rte_be_to_cpu_32(0xadadadad);
- table = rte_table_hash_cuckoo_dosig_ops.f_create(&cuckoo_params, 0, 1);
+ table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params, 0, 1);
if (table == NULL)
return -9;
entry = 'A';
- status = rte_table_hash_cuckoo_dosig_ops.f_add(NULL, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(NULL, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status == 0)
return -10;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, NULL, &entry,
+ status = rte_table_hash_cuckoo_ops.f_add(table, NULL, &entry,
&key_found, &entry_ptr);
if (status == 0)
return -11;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
NULL, &key_found, &entry_ptr);
if (status == 0)
return -12;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status != 0)
return -13;
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found, &entry_ptr);
if (status != 0)
return -14;
/* Delete */
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(NULL, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(NULL, &key_cuckoo,
&key_found, NULL);
if (status == 0)
return -15;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, NULL,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, NULL,
&key_found, NULL);
if (status == 0)
return -16;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, &key_cuckoo,
&key_found, NULL);
if (status != 0)
return -17;
- status = rte_table_hash_cuckoo_dosig_ops.f_delete(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_delete(table, &key_cuckoo,
&key_found, NULL);
if (status != -ENOENT)
return -18;
/* Traffic flow */
entry = 'A';
- status = rte_table_hash_cuckoo_dosig_ops.f_add(table, &key_cuckoo,
+ status = rte_table_hash_cuckoo_ops.f_add(table, &key_cuckoo,
&entry, &key_found,
&entry_ptr);
if (status < 0)
@@ -1093,7 +1093,7 @@ test_table_hash_cuckoo(void)
} else
PREPARE_PACKET(mbufs[i], 0xadadadab);
- rte_table_hash_cuckoo_dosig_ops.f_lookup(table, mbufs, -1,
+ rte_table_hash_cuckoo_ops.f_lookup(table, mbufs, -1,
&result_mask, (void **)entries);
if (result_mask != expected_mask)
return -20;
@@ -1102,7 +1102,7 @@ test_table_hash_cuckoo(void)
for (i = 0; i < RTE_PORT_IN_BURST_SIZE_MAX; i++)
rte_pktmbuf_free(mbufs[i]);
- status = rte_table_hash_cuckoo_dosig_ops.f_free(table);
+ status = rte_table_hash_cuckoo_ops.f_free(table);
return 0;
}
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 07/18] librte_table: add unified params structure and mask-based hash func
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (24 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 06/18] librte_table: rename cuckoo hash table ops Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 08/18] librte_table: rework variable size key ext hash tables Cristian Dumitrescu
` (10 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Add unified parameter structure for all hash tables in librte_table.
Add mask-based hash function prototype, which is input parameter for
all hash tables.
Renamed the non-mask-based hash function prototype and all the calls
to it (to be removed later).
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 2 +-
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 4 +-
lib/librte_table/rte_table_hash.h | 52 ++++++++++++++++++----
lib/librte_table/rte_table_hash_cuckoo.c | 2 +-
lib/librte_table/rte_table_hash_ext.c | 4 +-
lib/librte_table/rte_table_hash_key16.c | 2 +-
lib/librte_table/rte_table_hash_key32.c | 2 +-
lib/librte_table/rte_table_hash_key8.c | 4 +-
lib/librte_table/rte_table_hash_lru.c | 4 +-
9 files changed, 55 insertions(+), 21 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 191cb15..e131a5b 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -118,7 +118,7 @@ struct flow_table_entry {
uint32_t pad;
};
-rte_table_hash_op_hash hash_func[] = {
+rte_table_hash_op_hash_nomask hash_func[] = {
hash_default_key8,
hash_default_key16,
hash_default_key24,
diff --git a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
index 8cb2f0c..6b57f83
--- a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
@@ -52,7 +52,7 @@
struct pipeline_passthrough {
struct pipeline p;
struct pipeline_passthrough_params params;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint32_t swap_field0_offset[SWAP_DIM];
uint32_t swap_field1_offset[SWAP_DIM];
uint64_t swap_field_mask[SWAP_DIM];
@@ -677,7 +677,7 @@ pipeline_passthrough_parse_args(struct pipeline_passthrough_params *p,
return 0;
}
-static rte_table_hash_op_hash
+static rte_table_hash_op_hash_nomask
get_hash_function(struct pipeline_passthrough *p)
{
switch (p->params.dma_size) {
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 5f655ee..bb5b83d 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -98,6 +98,40 @@ extern "C" {
/** Hash function */
typedef uint64_t (*rte_table_hash_op_hash)(
void *key,
+ void *key_mask,
+ uint32_t key_size,
+ uint64_t seed);
+
+/** Hash table parameters */
+struct rte_table_hash_params {
+ /** Name */
+ const char *name;
+
+ /** Key size (number of bytes) */
+ uint32_t key_size;
+
+ /** Byte offset within packet meta-data where the key is located */
+ uint32_t key_offset;
+
+ /** Key mask */
+ uint8_t *key_mask;
+
+ /** Number of keys */
+ uint32_t n_keys;
+
+ /** Number of buckets */
+ uint32_t n_buckets;
+
+ /** Hash function */
+ rte_table_hash_op_hash f_hash;
+
+ /** Seed value for the hash function */
+ uint64_t seed;
+};
+
+/** Hash function */
+typedef uint64_t (*rte_table_hash_op_hash_nomask)(
+ void *key,
uint32_t key_size,
uint64_t seed);
@@ -121,7 +155,7 @@ struct rte_table_hash_ext_params {
uint32_t n_buckets_ext;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed value for the hash function */
uint64_t seed;
@@ -149,7 +183,7 @@ struct rte_table_hash_lru_params {
uint32_t n_buckets;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed value for the hash function */
uint64_t seed;
@@ -175,7 +209,7 @@ struct rte_table_hash_key8_lru_params {
uint32_t n_entries;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -204,7 +238,7 @@ struct rte_table_hash_key8_ext_params {
uint32_t n_entries_ext;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -233,7 +267,7 @@ struct rte_table_hash_key16_lru_params {
uint32_t n_entries;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -262,7 +296,7 @@ struct rte_table_hash_key16_ext_params {
uint32_t n_entries_ext;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -291,7 +325,7 @@ struct rte_table_hash_key32_lru_params {
uint32_t n_entries;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -318,7 +352,7 @@ struct rte_table_hash_key32_ext_params {
uint32_t n_entries_ext;
/** Hash function */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed for the hash function */
uint64_t seed;
@@ -344,7 +378,7 @@ struct rte_table_hash_cuckoo_params {
uint32_t n_keys;
/** Hash function used to calculate hash */
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
/** Seed value or Init value used by f_hash */
uint32_t seed;
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
index beb45c5..9b42423 100644
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -64,7 +64,7 @@ struct rte_table_hash {
uint32_t key_size;
uint32_t entry_size;
uint32_t n_keys;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint32_t seed;
uint32_t signature_offset;
uint32_t key_offset;
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
index c4824c3..72802b8 100644
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -104,7 +104,7 @@ struct rte_table_hash {
uint32_t n_keys;
uint32_t n_buckets;
uint32_t n_buckets_ext;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
uint32_t signature_offset;
uint32_t key_offset;
@@ -688,7 +688,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
uint8_t *key10, *key11; \
uint64_t bucket_mask = t->bucket_mask; \
- rte_table_hash_op_hash f_hash = t->f_hash; \
+ rte_table_hash_op_hash_nomask f_hash = t->f_hash; \
uint64_t seed = t->seed; \
uint32_t key_size = t->key_size; \
uint32_t key_offset = t->key_offset; \
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
index 4ed5c78..7f6651c 100644
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -86,7 +86,7 @@ struct rte_table_hash {
uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask[2];
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
/* Extendible buckets */
diff --git a/lib/librte_table/rte_table_hash_key32.c b/lib/librte_table/rte_table_hash_key32.c
index 31fe6fd..f8f6662 100644
--- a/lib/librte_table/rte_table_hash_key32.c
+++ b/lib/librte_table/rte_table_hash_key32.c
@@ -85,7 +85,7 @@ struct rte_table_hash {
uint32_t bucket_size;
uint32_t signature_offset;
uint32_t key_offset;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
/* Extendible buckets */
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
index f2b285d..4fbb02e 100644
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -83,7 +83,7 @@ struct rte_table_hash {
uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
/* Extendible buckets */
@@ -733,7 +733,7 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t hash_offset_buffer11; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
- rte_table_hash_op_hash f_hash = f->f_hash; \
+ rte_table_hash_op_hash_nomask f_hash = f->f_hash; \
uint64_t seed = f->seed; \
uint32_t key_offset = f->key_offset; \
\
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
index 0a85123..61050f3 100644
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -84,7 +84,7 @@ struct rte_table_hash {
uint32_t entry_size;
uint32_t n_keys;
uint32_t n_buckets;
- rte_table_hash_op_hash f_hash;
+ rte_table_hash_op_hash_nomask f_hash;
uint64_t seed;
uint32_t signature_offset;
uint32_t key_offset;
@@ -619,7 +619,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
uint8_t *key10, *key11; \
uint64_t bucket_mask = t->bucket_mask; \
- rte_table_hash_op_hash f_hash = t->f_hash; \
+ rte_table_hash_op_hash_nomask f_hash = t->f_hash; \
uint64_t seed = t->seed; \
uint32_t key_size = t->key_size; \
uint32_t key_offset = t->key_offset; \
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 08/18] librte_table: rework variable size key ext hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (25 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 07/18] librte_table: add unified params structure and mask-based hash func Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 09/18] librte_table: rework variable size key lru hash table Cristian Dumitrescu
` (9 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the variable size key extendible bucket (EXT) hash
table to use the mask-based hash function and the unified
parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 12 +-
lib/librte_table/rte_table_hash.h | 34 ----
lib/librte_table/rte_table_hash_ext.c | 179 ++++++++++++++-------
test/test-pipeline/pipeline_hash.c | 10 +-
4 files changed, 130 insertions(+), 105 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index e131a5b..4a4007c 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -516,16 +516,16 @@ static void *pipeline_fc_init(struct pipeline_params *params,
.seed = 0,
};
- struct rte_table_hash_ext_params
- table_hash_params = {
+ struct rte_table_hash_params table_hash_params = {
+ .name = p->name,
.key_size = p_fc->key_size,
+ .key_offset = p_fc->key_offset,
+ .key_mask = (p_fc->key_mask_present) ?
+ p_fc->key_mask : NULL,
.n_keys = p_fc->n_flows,
.n_buckets = p_fc->n_flows / 4,
- .n_buckets_ext = p_fc->n_flows / 4,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
+ .f_hash = (rte_table_hash_op_hash)hash_func[(p_fc->key_size / 8) - 1],
.seed = 0,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
};
struct rte_pipeline_table_params table_params = {
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index bb5b83d..0024b99 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -135,40 +135,6 @@ typedef uint64_t (*rte_table_hash_op_hash_nomask)(
uint32_t key_size,
uint64_t seed);
-/**
- * Hash tables with configurable key size
- *
- */
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_ext_params {
- /** Key size (number of bytes) */
- uint32_t key_size;
-
- /** Maximum number of keys */
- uint32_t n_keys;
-
- /** Number of hash table buckets. Each bucket stores up to 4 keys. */
- uint32_t n_buckets;
-
- /** Number of hash table bucket extensions. Each bucket extension has
- space for 4 keys and each bucket can have 0, 1 or more extensions. */
- uint32_t n_buckets_ext;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed value for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
extern struct rte_table_ops rte_table_hash_ext_ops;
/** LRU hash table parameters */
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
index 72802b8..0a743ad 100644
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -104,9 +104,8 @@ struct rte_table_hash {
uint32_t n_keys;
uint32_t n_buckets;
uint32_t n_buckets_ext;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
/* Internal */
@@ -120,6 +119,7 @@ struct rte_table_hash {
struct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];
/* Tables */
+ uint64_t *key_mask;
struct bucket *buckets;
struct bucket *buckets_ext;
uint8_t *key_mem;
@@ -132,29 +132,53 @@ struct rte_table_hash {
};
static int
-check_params_create(struct rte_table_hash_ext_params *params)
+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)
{
- uint32_t n_buckets_min;
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ if (a64[i] != (b64[i] & b_mask64[i]))
+ return 1;
+
+ return 0;
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ dst64[i] = src64[i] & src_mask64[i];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
/* key_size */
- if ((params->key_size == 0) ||
+ if ((params->key_size < sizeof(uint64_t)) ||
(!rte_is_power_of_2(params->key_size))) {
RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
return -EINVAL;
}
/* n_keys */
- if ((params->n_keys == 0) ||
- (!rte_is_power_of_2(params->n_keys))) {
+ if (params->n_keys == 0) {
RTE_LOG(ERR, TABLE, "%s: n_keys invalid value\n", __func__);
return -EINVAL;
}
/* n_buckets */
- n_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;
if ((params->n_buckets == 0) ||
- (!rte_is_power_of_2(params->n_keys)) ||
- (params->n_buckets < n_buckets_min)) {
+ (!rte_is_power_of_2(params->n_buckets))) {
RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -171,15 +195,13 @@ check_params_create(struct rte_table_hash_ext_params *params)
static void *
rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_ext_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *t;
- uint32_t total_size, table_meta_sz;
- uint32_t bucket_sz, bucket_ext_sz, key_sz;
- uint32_t key_stack_sz, bkt_ext_stack_sz, data_sz;
- uint32_t bucket_offset, bucket_ext_offset, key_offset;
- uint32_t key_stack_offset, bkt_ext_stack_offset, data_offset;
- uint32_t i;
+ uint64_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz;
+ uint64_t key_stack_sz, bkt_ext_stack_sz, data_sz, total_size;
+ uint64_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset;
+ uint64_t key_stack_offset, bkt_ext_stack_offset, data_offset;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
if ((check_params_create(p) != 0) ||
@@ -188,38 +210,66 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
(sizeof(struct bucket) != (RTE_CACHE_LINE_SIZE / 2)))
return NULL;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
+
/* Memory allocation */
table_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));
+ key_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);
bucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));
bucket_ext_sz =
- RTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(struct bucket));
+ RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(struct bucket));
key_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);
key_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));
bkt_ext_stack_sz =
- RTE_CACHE_LINE_ROUNDUP(p->n_buckets_ext * sizeof(uint32_t));
+ RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
data_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
- total_size = table_meta_sz + bucket_sz + bucket_ext_sz + key_sz +
- key_stack_sz + bkt_ext_stack_sz + data_sz;
+ total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz +
+ key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz;
- t = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ t = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (t == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table memory footprint is "
- "%u bytes\n", __func__, p->key_size, total_size);
+ RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table %s memory "
+ "footprint is %" PRIu64 " bytes\n",
+ __func__, p->key_size, p->name, total_size);
/* Memory initialization */
t->key_size = p->key_size;
t->entry_size = entry_size;
t->n_keys = p->n_keys;
t->n_buckets = p->n_buckets;
- t->n_buckets_ext = p->n_buckets_ext;
+ t->n_buckets_ext = n_buckets_ext;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
/* Internal */
@@ -228,13 +278,15 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
t->data_size_shl = __builtin_ctzl(entry_size);
/* Tables */
- bucket_offset = 0;
+ key_mask_offset = 0;
+ bucket_offset = key_mask_offset + key_mask_sz;
bucket_ext_offset = bucket_offset + bucket_sz;
key_offset = bucket_ext_offset + bucket_ext_sz;
key_stack_offset = key_offset + key_sz;
bkt_ext_stack_offset = key_stack_offset + key_stack_sz;
data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz;
+ t->key_mask = (uint64_t *) &t->memory[key_mask_offset];
t->buckets = (struct bucket *) &t->memory[bucket_offset];
t->buckets_ext = (struct bucket *) &t->memory[bucket_ext_offset];
t->key_mem = &t->memory[key_offset];
@@ -242,6 +294,12 @@ rte_table_hash_ext_create(void *params, int socket_id, uint32_t entry_size)
t->bkt_ext_stack = (uint32_t *) &t->memory[bkt_ext_stack_offset];
t->data_mem = &t->memory[data_offset];
+ /* Key mask */
+ if (p->key_mask == NULL)
+ memset(t->key_mask, 0xFF, p->key_size);
+ else
+ memcpy(t->key_mask, p->key_mask, p->key_size);
+
/* Key stack */
for (i = 0; i < t->n_keys; i++)
t->key_stack[i] = t->n_keys - 1 - i;
@@ -277,7 +335,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -290,7 +348,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
uint8_t *bkt_key =
&t->key_mem[bkt_key_index << t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -327,7 +385,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
bkt->sig[i] = (uint16_t) sig;
bkt->key_pos[i] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
*key_found = 0;
@@ -358,7 +416,7 @@ rte_table_hash_ext_entry_add(void *table, void *key, void *entry,
/* Install new key into bucket */
bkt->sig[0] = (uint16_t) sig;
bkt->key_pos[0] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
*key_found = 0;
@@ -378,7 +436,7 @@ void *entry)
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -392,7 +450,7 @@ void *entry)
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -457,7 +515,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
+ sig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt0 = &t->buckets[bkt_index];
@@ -471,8 +529,8 @@ static int rte_table_hash_ext_lookup_unoptimized(
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
- t->key_size) == 0)) {
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key,
+ t->key_mask, t->key_size) == 0)) {
uint8_t *data = &t->data_mem[
bkt_key_index << t->data_size_shl];
@@ -571,11 +629,12 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\
uint64_t *bkt_key = (uint64_t *) key; \
+ uint64_t *key_mask = f->key_mask; \
\
switch (f->key_size) { \
case 8: \
{ \
- uint64_t xor = pkt_key[0] ^ bkt_key[0]; \
+ uint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
match_key = 0; \
if (xor == 0) \
match_key = 1; \
@@ -586,8 +645,8 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[2], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
or = xor[0] | xor[1]; \
match_key = 0; \
if (or == 0) \
@@ -599,10 +658,10 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[4], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
or = xor[0] | xor[1] | xor[2] | xor[3]; \
match_key = 0; \
if (or == 0) \
@@ -614,14 +673,14 @@ static int rte_table_hash_ext_lookup_unoptimized(
{ \
uint64_t xor[8], or; \
\
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- xor[4] = pkt_key[4] ^ bkt_key[4]; \
- xor[5] = pkt_key[5] ^ bkt_key[5]; \
- xor[6] = pkt_key[6] ^ bkt_key[6]; \
- xor[7] = pkt_key[7] ^ bkt_key[7]; \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ xor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4]; \
+ xor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5]; \
+ xor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6]; \
+ xor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7]; \
or = xor[0] | xor[1] | xor[2] | xor[3] | \
xor[4] | xor[5] | xor[6] | xor[7]; \
match_key = 0; \
@@ -632,7 +691,7 @@ static int rte_table_hash_ext_lookup_unoptimized(
\
default: \
match_key = 0; \
- if (memcmp(pkt_key, bkt_key, f->key_size) == 0) \
+ if (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0) \
match_key = 1; \
} \
}
@@ -688,20 +747,20 @@ static int rte_table_hash_ext_lookup_unoptimized(
struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
uint8_t *key10, *key11; \
uint64_t bucket_mask = t->bucket_mask; \
- rte_table_hash_op_hash_nomask f_hash = t->f_hash; \
+ rte_table_hash_op_hash f_hash = t->f_hash; \
uint64_t seed = t->seed; \
uint32_t key_size = t->key_size; \
uint32_t key_offset = t->key_offset; \
\
mbuf10 = pkts[pkt10_index]; \
key10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset); \
- sig10 = (uint64_t) f_hash(key10, key_size, seed); \
+ sig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed); \
bkt10_index = sig10 & bucket_mask; \
bkt10 = &buckets[bkt10_index]; \
\
mbuf11 = pkts[pkt11_index]; \
key11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset); \
- sig11 = (uint64_t) f_hash(key11, key_size, seed); \
+ sig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed); \
bkt11_index = sig11 & bucket_mask; \
bkt11 = &buckets[bkt11_index]; \
\
@@ -969,7 +1028,7 @@ rte_table_hash_ext_stats_read(void *table, struct rte_table_stats *stats, int cl
return 0;
}
-struct rte_table_ops rte_table_hash_ext_ops = {
+struct rte_table_ops rte_table_hash_ext_ops = {
.f_create = rte_table_hash_ext_create,
.f_free = rte_table_hash_ext_free,
.f_add = rte_table_hash_ext_entry_add,
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 2f8c625..13d309d
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -175,15 +175,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_KEY16_EXT:
case e_APP_PIPELINE_HASH_KEY32_EXT:
{
- struct rte_table_hash_ext_params table_hash_params = {
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
.key_size = key_size,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
.n_keys = 1 << 24,
.n_buckets = 1 << 22,
- .n_buckets_ext = 1 << 21,
- .f_hash = test_hash,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
};
struct rte_pipeline_table_params table_params = {
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 09/18] librte_table: rework variable size key lru hash table
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (26 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 08/18] librte_table: rework variable size key ext hash tables Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 10/18] librte_table: rework 8-byte key hash tables Cristian Dumitrescu
` (8 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the variable size key LRU hash table to use the
mask-based hash function and the unified parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 26 ----
lib/librte_table/rte_table_hash_lru.c | 277 +++++++++++++++++++++-------------
test/test-pipeline/pipeline_hash.c | 9 +-
3 files changed, 177 insertions(+), 135 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 0024b99..fb5f4d7 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -137,32 +137,6 @@ typedef uint64_t (*rte_table_hash_op_hash_nomask)(
extern struct rte_table_ops rte_table_hash_ext_ops;
-/** LRU hash table parameters */
-struct rte_table_hash_lru_params {
- /** Key size (number of bytes) */
- uint32_t key_size;
-
- /** Maximum number of keys */
- uint32_t n_keys;
-
- /** Number of hash table buckets. Each bucket stores up to 4 keys. */
- uint32_t n_buckets;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed value for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
extern struct rte_table_ops rte_table_hash_lru_ops;
/**
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
index 61050f3..eebc1cd 100644
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -84,9 +84,8 @@ struct rte_table_hash {
uint32_t entry_size;
uint32_t n_keys;
uint32_t n_buckets;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
/* Internal */
@@ -99,6 +98,7 @@ struct rte_table_hash {
struct grinder grinders[RTE_PORT_IN_BURST_SIZE_MAX];
/* Tables */
+ uint64_t *key_mask;
struct bucket *buckets;
uint8_t *key_mem;
uint8_t *data_mem;
@@ -109,12 +109,39 @@ struct rte_table_hash {
};
static int
-check_params_create(struct rte_table_hash_lru_params *params)
+keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes)
{
- uint32_t n_buckets_min;
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ if (a64[i] != (b64[i] & b_mask64[i]))
+ return 1;
+
+ return 0;
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+ uint32_t i;
+
+ for (i = 0; i < n_bytes / sizeof(uint64_t); i++)
+ dst64[i] = src64[i] & src_mask64[i];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
/* key_size */
- if ((params->key_size == 0) ||
+ if ((params->key_size < sizeof(uint64_t)) ||
(!rte_is_power_of_2(params->key_size))) {
RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
return -EINVAL;
@@ -128,10 +155,8 @@ check_params_create(struct rte_table_hash_lru_params *params)
}
/* n_buckets */
- n_buckets_min = (params->n_keys + KEYS_PER_BUCKET - 1) / params->n_keys;
if ((params->n_buckets == 0) ||
- (!rte_is_power_of_2(params->n_keys)) ||
- (params->n_buckets < n_buckets_min)) {
+ (!rte_is_power_of_2(params->n_keys))) {
RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -148,13 +173,13 @@ check_params_create(struct rte_table_hash_lru_params *params)
static void *
rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_lru_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *t;
- uint32_t total_size, table_meta_sz;
- uint32_t bucket_sz, key_sz, key_stack_sz, data_sz;
- uint32_t bucket_offset, key_offset, key_stack_offset, data_offset;
- uint32_t i;
+ uint64_t table_meta_sz, key_mask_sz, bucket_sz, key_sz, key_stack_sz;
+ uint64_t data_sz, total_size;
+ uint64_t key_mask_offset, bucket_offset, key_offset, key_stack_offset;
+ uint64_t data_offset;
+ uint32_t n_buckets, i;
/* Check input parameters */
if ((check_params_create(p) != 0) ||
@@ -164,33 +189,65 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
return NULL;
}
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
+
/* Memory allocation */
table_meta_sz = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_table_hash));
- bucket_sz = RTE_CACHE_LINE_ROUNDUP(p->n_buckets * sizeof(struct bucket));
+ key_mask_sz = RTE_CACHE_LINE_ROUNDUP(p->key_size);
+ bucket_sz = RTE_CACHE_LINE_ROUNDUP(n_buckets * sizeof(struct bucket));
key_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * p->key_size);
key_stack_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * sizeof(uint32_t));
data_sz = RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
- total_size = table_meta_sz + bucket_sz + key_sz + key_stack_sz +
- data_sz;
+ total_size = table_meta_sz + key_mask_sz + bucket_sz + key_sz +
+ key_stack_sz + data_sz;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE,
+ "%s: Cannot allocate %" PRIu64 " bytes for hash "
+ "table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
- t = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ t = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (t == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %" PRIu64 " bytes for hash "
+ "table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table memory footprint is "
- "%u bytes\n", __func__, p->key_size, total_size);
+ RTE_LOG(INFO, TABLE, "%s (%u-byte key): Hash table %s memory footprint"
+ " is %" PRIu64 " bytes\n",
+ __func__, p->key_size, p->name, total_size);
/* Memory initialization */
t->key_size = p->key_size;
t->entry_size = entry_size;
t->n_keys = p->n_keys;
- t->n_buckets = p->n_buckets;
+ t->n_buckets = n_buckets;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
/* Internal */
@@ -199,16 +256,24 @@ rte_table_hash_lru_create(void *params, int socket_id, uint32_t entry_size)
t->data_size_shl = __builtin_ctzl(entry_size);
/* Tables */
- bucket_offset = 0;
+ key_mask_offset = 0;
+ bucket_offset = key_mask_offset + key_mask_sz;
key_offset = bucket_offset + bucket_sz;
key_stack_offset = key_offset + key_sz;
data_offset = key_stack_offset + key_stack_sz;
+ t->key_mask = (uint64_t *) &t->memory[key_mask_offset];
t->buckets = (struct bucket *) &t->memory[bucket_offset];
t->key_mem = &t->memory[key_offset];
t->key_stack = (uint32_t *) &t->memory[key_stack_offset];
t->data_mem = &t->memory[data_offset];
+ /* Key mask */
+ if (p->key_mask == NULL)
+ memset(t->key_mask, 0xFF, p->key_size);
+ else
+ memcpy(t->key_mask, p->key_mask, p->key_size);
+
/* Key stack */
for (i = 0; i < t->n_keys; i++)
t->key_stack[i] = t->n_keys - 1 - i;
@@ -246,7 +311,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -258,8 +323,8 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key, t->key_size)
- == 0)) {
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
+ t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -292,7 +357,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
bkt->sig[i] = (uint16_t) sig;
bkt->key_pos[i] = bkt_key_index;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
lru_update(bkt, i);
@@ -311,7 +376,7 @@ rte_table_hash_lru_entry_add(void *table, void *key, void *entry,
uint8_t *data = &t->data_mem[bkt_key_index << t->data_size_shl];
bkt->sig[pos] = (uint16_t) sig;
- memcpy(bkt_key, key, t->key_size);
+ keycpy(bkt_key, key, t->key_mask, t->key_size);
memcpy(data, entry, t->entry_size);
lru_update(bkt, pos);
@@ -330,7 +395,7 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,
uint64_t sig;
uint32_t bkt_index, i;
- sig = t->f_hash(key, t->key_size, t->seed);
+ sig = t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
sig = (sig >> 16) | 1LLU;
@@ -343,14 +408,15 @@ rte_table_hash_lru_entry_delete(void *table, void *key, int *key_found,
t->key_size_shl];
if ((sig == bkt_sig) &&
- (memcmp(key, bkt_key, t->key_size) == 0)) {
+ (keycmp(bkt_key, key, t->key_mask, t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
bkt->sig[i] = 0;
t->key_stack[t->key_stack_tos++] = bkt_key_index;
*key_found = 1;
- memcpy(entry, data, t->entry_size);
+ if (entry)
+ memcpy(entry, data, t->entry_size);
return 0;
}
}
@@ -386,7 +452,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
pkt = pkts[pkt_index];
key = RTE_MBUF_METADATA_UINT8_PTR(pkt, t->key_offset);
- sig = (uint64_t) t->f_hash(key, t->key_size, t->seed);
+ sig = (uint64_t) t->f_hash(key, t->key_mask, t->key_size, t->seed);
bkt_index = sig & t->bucket_mask;
bkt = &t->buckets[bkt_index];
@@ -399,7 +465,7 @@ static int rte_table_hash_lru_lookup_unoptimized(
uint8_t *bkt_key = &t->key_mem[bkt_key_index <<
t->key_size_shl];
- if ((sig == bkt_sig) && (memcmp(key, bkt_key,
+ if ((sig == bkt_sig) && (keycmp(bkt_key, key, t->key_mask,
t->key_size) == 0)) {
uint8_t *data = &t->data_mem[bkt_key_index <<
t->data_size_shl];
@@ -497,74 +563,75 @@ static int rte_table_hash_lru_lookup_unoptimized(
match_pos = (LUT_MATCH_POS >> (mask_all << 1)) & 3; \
}
-#define lookup_cmp_key(mbuf, key, match_key, f) \
-{ \
+#define lookup_cmp_key(mbuf, key, match_key, f) \
+{ \
uint64_t *pkt_key = RTE_MBUF_METADATA_UINT64_PTR(mbuf, f->key_offset);\
- uint64_t *bkt_key = (uint64_t *) key; \
- \
- switch (f->key_size) { \
- case 8: \
- { \
- uint64_t xor = pkt_key[0] ^ bkt_key[0]; \
- match_key = 0; \
- if (xor == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 16: \
- { \
- uint64_t xor[2], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- or = xor[0] | xor[1]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 32: \
- { \
- uint64_t xor[4], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- or = xor[0] | xor[1] | xor[2] | xor[3]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- case 64: \
- { \
- uint64_t xor[8], or; \
- \
- xor[0] = pkt_key[0] ^ bkt_key[0]; \
- xor[1] = pkt_key[1] ^ bkt_key[1]; \
- xor[2] = pkt_key[2] ^ bkt_key[2]; \
- xor[3] = pkt_key[3] ^ bkt_key[3]; \
- xor[4] = pkt_key[4] ^ bkt_key[4]; \
- xor[5] = pkt_key[5] ^ bkt_key[5]; \
- xor[6] = pkt_key[6] ^ bkt_key[6]; \
- xor[7] = pkt_key[7] ^ bkt_key[7]; \
- or = xor[0] | xor[1] | xor[2] | xor[3] | \
- xor[4] | xor[5] | xor[6] | xor[7]; \
- match_key = 0; \
- if (or == 0) \
- match_key = 1; \
- } \
- break; \
- \
- default: \
- match_key = 0; \
- if (memcmp(pkt_key, bkt_key, f->key_size) == 0) \
- match_key = 1; \
- } \
+ uint64_t *bkt_key = (uint64_t *) key; \
+ uint64_t *key_mask = f->key_mask; \
+ \
+ switch (f->key_size) { \
+ case 8: \
+ { \
+ uint64_t xor = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ match_key = 0; \
+ if (xor == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 16: \
+ { \
+ uint64_t xor[2], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ or = xor[0] | xor[1]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 32: \
+ { \
+ uint64_t xor[4], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ or = xor[0] | xor[1] | xor[2] | xor[3]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ case 64: \
+ { \
+ uint64_t xor[8], or; \
+ \
+ xor[0] = (pkt_key[0] & key_mask[0]) ^ bkt_key[0]; \
+ xor[1] = (pkt_key[1] & key_mask[1]) ^ bkt_key[1]; \
+ xor[2] = (pkt_key[2] & key_mask[2]) ^ bkt_key[2]; \
+ xor[3] = (pkt_key[3] & key_mask[3]) ^ bkt_key[3]; \
+ xor[4] = (pkt_key[4] & key_mask[4]) ^ bkt_key[4]; \
+ xor[5] = (pkt_key[5] & key_mask[5]) ^ bkt_key[5]; \
+ xor[6] = (pkt_key[6] & key_mask[6]) ^ bkt_key[6]; \
+ xor[7] = (pkt_key[7] & key_mask[7]) ^ bkt_key[7]; \
+ or = xor[0] | xor[1] | xor[2] | xor[3] | \
+ xor[4] | xor[5] | xor[6] | xor[7]; \
+ match_key = 0; \
+ if (or == 0) \
+ match_key = 1; \
+ } \
+ break; \
+ \
+ default: \
+ match_key = 0; \
+ if (keycmp(bkt_key, pkt_key, key_mask, f->key_size) == 0) \
+ match_key = 1; \
+ } \
}
#define lookup2_stage0(t, g, pkts, pkts_mask, pkt00_index, pkt01_index)\
@@ -619,20 +686,20 @@ static int rte_table_hash_lru_lookup_unoptimized(
struct bucket *bkt10, *bkt11, *buckets = t->buckets; \
uint8_t *key10, *key11; \
uint64_t bucket_mask = t->bucket_mask; \
- rte_table_hash_op_hash_nomask f_hash = t->f_hash; \
+ rte_table_hash_op_hash f_hash = t->f_hash; \
uint64_t seed = t->seed; \
uint32_t key_size = t->key_size; \
uint32_t key_offset = t->key_offset; \
\
mbuf10 = pkts[pkt10_index]; \
key10 = RTE_MBUF_METADATA_UINT8_PTR(mbuf10, key_offset);\
- sig10 = (uint64_t) f_hash(key10, key_size, seed); \
+ sig10 = (uint64_t) f_hash(key10, t->key_mask, key_size, seed);\
bkt10_index = sig10 & bucket_mask; \
bkt10 = &buckets[bkt10_index]; \
\
mbuf11 = pkts[pkt11_index]; \
key11 = RTE_MBUF_METADATA_UINT8_PTR(mbuf11, key_offset);\
- sig11 = (uint64_t) f_hash(key11, key_size, seed); \
+ sig11 = (uint64_t) f_hash(key11, t->key_mask, key_size, seed);\
bkt11_index = sig11 & bucket_mask; \
bkt11 = &buckets[bkt11_index]; \
\
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 13d309d..3c48640 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -204,14 +204,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_KEY16_LRU:
case e_APP_PIPELINE_HASH_KEY32_LRU:
{
- struct rte_table_hash_lru_params table_hash_params = {
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
.key_size = key_size,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
.n_keys = 1 << 24,
.n_buckets = 1 << 22,
- .f_hash = test_hash,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
- .key_offset = APP_METADATA_OFFSET(32),
};
struct rte_pipeline_table_params table_params = {
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 10/18] librte_table: rework 8-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (27 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 09/18] librte_table: rework variable size key lru hash table Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 11/18] librte_table: rework 16-byte " Cristian Dumitrescu
` (7 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the 8-byte key hash tables (both the extendible
bucket and LRU)to use the mask-based hash function and the
unified parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 14 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 14 +-
lib/librte_table/rte_table_hash.h | 54 ----
lib/librte_table/rte_table_hash_key8.c | 328 +++++++++++----------
test/test-pipeline/pipeline_hash.c | 21 +-
test/test/test_table_combined.c | 39 ++-
test/test/test_table_tables.c | 101 ++-----
7 files changed, 242 insertions(+), 329 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 4a4007c..47e6d35 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -492,18 +492,6 @@ static void *pipeline_fc_init(struct pipeline_params *params,
/* Tables */
p->n_tables = 1;
{
- struct rte_table_hash_key8_ext_params
- table_hash_key8_params = {
- .n_entries = p_fc->n_flows,
- .n_entries_ext = p_fc->n_flows,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
- .key_mask = (p_fc->key_mask_present) ?
- p_fc->key_mask : NULL,
- .seed = 0,
- };
-
struct rte_table_hash_key16_ext_params
table_hash_key16_params = {
.n_entries = p_fc->n_flows,
@@ -543,7 +531,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
table_params.ops = &rte_table_hash_key8_ext_ops;
- table_params.arg_create = &table_hash_key8_params;
+ table_params.arg_create = &table_hash_params;
break;
case 16:
diff --git a/examples/ip_pipeline/pipeline/pipeline_routing_be.c b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
index aa7f05e..796cde7 100644
--- a/examples/ip_pipeline/pipeline/pipeline_routing_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
@@ -1349,13 +1349,15 @@ pipeline_routing_init(struct pipeline_params *params,
/* ARP table configuration */
if (p_rt->params.n_arp_entries) {
- struct rte_table_hash_key8_ext_params table_arp_params = {
- .n_entries = p_rt->params.n_arp_entries,
- .n_entries_ext = p_rt->params.n_arp_entries,
- .f_hash = hash_default_key8,
- .seed = 0,
- .signature_offset = 0, /* Unused */
+ struct rte_table_hash_params table_arp_params = {
+ .name = p->name,
+ .key_size = 8,
.key_offset = p_rt->params.arp_key_offset,
+ .key_mask = NULL,
+ .n_keys = p_rt->params.n_arp_entries,
+ .n_buckets = p_rt->params.n_arp_entries / 4,
+ .f_hash = (rte_table_hash_op_hash)hash_default_key8,
+ .seed = 0,
};
struct rte_pipeline_table_params table_params = {
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index fb5f4d7..6e25054 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -139,62 +139,8 @@ extern struct rte_table_ops rte_table_hash_ext_ops;
extern struct rte_table_ops rte_table_hash_lru_ops;
-/**
- * 8-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key8_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
extern struct rte_table_ops rte_table_hash_key8_lru_ops;
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key8_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
extern struct rte_table_ops rte_table_hash_key8_ext_ops;
/**
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
index 4fbb02e..a44f75c 100644
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 8
+#define KEY_SIZE 8
+
+#define KEYS_PER_BUCKET 4
#ifdef RTE_TABLE_STATS_COLLECT
@@ -76,14 +78,12 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
/* Extendible buckets */
@@ -96,10 +96,46 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return a64[0] != (b64[0] & b_mask64[0]);
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_buckets */
+ if ((params->n_buckets == 0) ||
+ (!rte_is_power_of_2(params->n_buckets))) {
+ RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -116,47 +152,68 @@ check_params_create_lru(struct rte_table_hash_key8_lru_params *params) {
static void *
rte_table_hash_create_key8_lru(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_key8_lru_params *p =
- (struct rte_table_hash_key8_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;
- uint32_t total_size, i;
+ uint64_t bucket_size, total_size;
+ uint32_t n_buckets, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
- ((sizeof(struct rte_bucket_4_8) % 64) != 0)) {
+ ((sizeof(struct rte_bucket_4_8) % 64) != 0))
return NULL;
- }
- n_entries_per_bucket = 4;
- key_size = 8;
+
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *
- entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + n_buckets *
- bucket_size_cl * RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +
+ KEYS_PER_BUCKET * entry_size);
+ total_size = sizeof(struct rte_table_hash) + n_buckets * bucket_size;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes"
+ " for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -205,7 +262,7 @@ rte_table_hash_entry_add_key8_lru(
uint64_t signature, mask, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -213,10 +270,10 @@ rte_table_hash_entry_add_key8_lru(
/* Key is present in the bucket */
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -235,7 +292,7 @@ rte_table_hash_entry_add_key8_lru(
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature |= mask;
- bucket->key[i] = *((uint64_t *) key);
+ keycpy(&bucket->key[i], key, &f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -247,10 +304,10 @@ rte_table_hash_entry_add_key8_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
- bucket->key[pos] = *((uint64_t *) key);
+ keycpy(&bucket->key[pos], key, &f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
- *key_found = 0;
+ *key_found = 0;
*entry_ptr = (void *) &bucket->data[pos * f->entry_size];
return 0;
@@ -268,7 +325,7 @@ rte_table_hash_entry_delete_key8_lru(
uint64_t signature, mask;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -276,10 +333,10 @@ rte_table_hash_entry_delete_key8_lru(
/* Key is present in the bucket */
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature &= ~mask;
@@ -296,79 +353,71 @@ rte_table_hash_entry_delete_key8_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key8_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
- __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key8_ext(void *params, int socket_id, uint32_t entry_size)
{
- struct rte_table_hash_key8_ext_params *p =
- (struct rte_table_hash_key8_ext_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size;
- uint32_t bucket_size_cl, stack_size_cl, total_size, i;
+ uint64_t bucket_size, stack_size, total_size;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_8) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 8;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_8) + n_entries_per_bucket *
- entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + ((n_buckets +
- n_buckets_ext) * bucket_size_cl + stack_size_cl) *
- RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_8) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
+ total_size = sizeof(struct rte_table_hash) +
+ (p->n_buckets + n_buckets_ext) * bucket_size + stack_size;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
- f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->n_buckets = p->n_buckets;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -376,7 +425,7 @@ rte_table_hash_create_key8_ext(void *params, int socket_id, uint32_t entry_size)
f->n_buckets_ext = n_buckets_ext;
f->stack_pos = n_buckets_ext;
f->stack = (uint32_t *)
- &f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
+ &f->memory[(p->n_buckets + n_buckets_ext) * f->bucket_size];
if (p->key_mask != NULL)
f->key_mask = ((uint64_t *)p->key_mask)[0];
@@ -417,7 +466,7 @@ rte_table_hash_entry_add_key8_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -428,10 +477,10 @@ rte_table_hash_entry_add_key8_ext(
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -456,7 +505,7 @@ rte_table_hash_entry_add_key8_ext(
f->entry_size];
bucket->signature |= mask;
- bucket->key[i] = *((uint64_t *) key);
+ keycpy(&bucket->key[i], key, &f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -476,7 +525,7 @@ rte_table_hash_entry_add_key8_ext(
bucket_prev->next_valid = 1;
bucket->signature = 1;
- bucket->key[0] = *((uint64_t *) key);
+ keycpy(&bucket->key[0], key, &f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -498,7 +547,7 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, &f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_8 *)
&f->memory[bucket_index * f->bucket_size];
@@ -510,10 +559,10 @@ rte_table_hash_entry_delete_key8_ext(
for (i = 0, mask = 1LLU; i < 4; i++, mask <<= 1) {
uint64_t bucket_signature = bucket->signature;
- uint64_t bucket_key = bucket->key[i];
+ uint64_t *bucket_key = &bucket->key[i];
if ((bucket_signature & mask) &&
- (*((uint64_t *) key) == bucket_key)) {
+ (keycmp(bucket_key, key, &f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -546,16 +595,17 @@ rte_table_hash_entry_delete_key8_ext(
return 0;
}
-#define lookup_key8_cmp(key_in, bucket, pos) \
+#define lookup_key8_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4], signature; \
+ uint64_t xor[4], signature, k; \
\
signature = ~bucket->signature; \
\
- xor[0] = (key_in[0] ^ bucket->key[0]) | (signature & 1);\
- xor[1] = (key_in[0] ^ bucket->key[1]) | (signature & 2);\
- xor[2] = (key_in[0] ^ bucket->key[2]) | (signature & 4);\
- xor[3] = (key_in[0] ^ bucket->key[3]) | (signature & 8);\
+ k = key_in[0] & f->key_mask; \
+ xor[0] = (k ^ bucket->key[0]) | (signature & 1); \
+ xor[1] = (k ^ bucket->key[1]) | (signature & 2); \
+ xor[2] = (k ^ bucket->key[2]) | (signature & 4); \
+ xor[3] = (k ^ bucket->key[3]) | (signature & 8); \
\
pos = 4; \
if (xor[0] == 0) \
@@ -586,12 +636,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\
- hash_key_buffer = *key & f->key_mask; \
- signature = f->f_hash(&hash_key_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed); \
+ signature = f->f_hash(key, &f->key_mask, KEY_SIZE, f->seed); \
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_8 *) \
&f->memory[bucket_index * f->bucket_size]; \
@@ -605,12 +652,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer = key[0] & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket2, pos); \
+ lookup_key8_cmp(key, bucket2, pos, f); \
\
pkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -629,12 +673,9 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer = *key & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket2, pos); \
+ lookup_key8_cmp(key, bucket2, pos, f); \
\
pkt_mask = ((bucket2->signature >> pos) & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -658,13 +699,10 @@ rte_table_hash_entry_delete_key8_ext(
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
uint32_t pos; \
- uint64_t hash_key_buffer; \
\
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
- hash_key_buffer = (*key) & f->key_mask; \
- \
- lookup_key8_cmp((&hash_key_buffer), bucket, pos); \
+ lookup_key8_cmp(key, bucket, pos, f); \
\
pkt_mask = ((bucket->signature >> pos) & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -729,28 +767,22 @@ rte_table_hash_entry_delete_key8_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f)\
{ \
uint64_t *key10, *key11; \
- uint64_t hash_offset_buffer10; \
- uint64_t hash_offset_buffer11; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
- rte_table_hash_op_hash_nomask f_hash = f->f_hash; \
+ rte_table_hash_op_hash f_hash = f->f_hash; \
uint64_t seed = f->seed; \
uint32_t key_offset = f->key_offset; \
\
key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, key_offset);\
key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, key_offset);\
- hash_offset_buffer10 = *key10 & f->key_mask; \
- hash_offset_buffer11 = *key11 & f->key_mask; \
\
- signature10 = f_hash(&hash_offset_buffer10, \
- RTE_TABLE_HASH_KEY_SIZE, seed); \
+ signature10 = f_hash(key10, &f->key_mask, KEY_SIZE, seed); \
bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_8 *) \
&f->memory[bucket10_index * f->bucket_size]; \
rte_prefetch0(bucket10); \
\
- signature11 = f_hash(&hash_offset_buffer11, \
- RTE_TABLE_HASH_KEY_SIZE, seed); \
+ signature11 = f_hash(key11, &f->key_mask, KEY_SIZE, seed); \
bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_8 *) \
&f->memory[bucket11_index * f->bucket_size]; \
@@ -763,17 +795,13 @@ rte_table_hash_entry_delete_key8_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask; \
uint64_t *key20, *key21; \
- uint64_t hash_offset_buffer20; \
- uint64_t hash_offset_buffer21; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_offset_buffer20 = *key20 & f->key_mask; \
- hash_offset_buffer21 = *key21 & f->key_mask; \
\
- lookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\
- lookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\
+ lookup_key8_cmp(key20, bucket20, pos20, f); \
+ lookup_key8_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\
pkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\
@@ -796,17 +824,13 @@ rte_table_hash_entry_delete_key8_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\
uint64_t *key20, *key21; \
- uint64_t hash_offset_buffer20; \
- uint64_t hash_offset_buffer21; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_offset_buffer20 = *key20 & f->key_mask; \
- hash_offset_buffer21 = *key21 & f->key_mask; \
\
- lookup_key8_cmp((&hash_offset_buffer20), bucket20, pos20);\
- lookup_key8_cmp((&hash_offset_buffer21), bucket21, pos21);\
+ lookup_key8_cmp(key20, bucket20, pos20, f); \
+ lookup_key8_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = ((bucket20->signature >> pos20) & 1LLU) << pkt20_index;\
pkt21_mask = ((bucket21->signature >> pos21) & 1LLU) << pkt21_index;\
@@ -954,7 +978,7 @@ rte_table_hash_lookup_key8_lru(
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key8_lru() */
+} /* lookup LRU */
static int
rte_table_hash_lookup_key8_ext(
@@ -1104,7 +1128,7 @@ rte_table_hash_lookup_key8_ext(
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_KEY8_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in - __builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key8_ext() */
+} /* lookup EXT */
static int
rte_table_hash_key8_stats_read(void *table, struct rte_table_stats *stats, int clear)
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 3c48640..f542506 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -231,13 +231,14 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_EXT:
{
- struct rte_table_hash_key8_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
- .f_hash = test_hash,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
};
@@ -257,12 +258,14 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_LRU:
{
- struct rte_table_hash_key8_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
- .f_hash = test_hash,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
};
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index f65d34a..ddcd14b 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -441,12 +441,15 @@ test_table_hash8lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key8_lru_params key8lru_params = {
- .n_entries = 1<<24,
- .f_hash = pipeline_test_hash,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key8lru_params = {
+ .name = "TABLE",
+ .key_size = 8,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key8lru[8];
@@ -475,14 +478,14 @@ test_table_hash8lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key8lru_params.n_entries = 0;
+ key8lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key8_lru_ops,
(void *)&key8lru_params, (void *)key8lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8lru_params.n_entries = 1<<16;
+ key8lru_params.n_keys = 1<<16;
key8lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key8_lru_ops,
@@ -616,14 +619,15 @@ test_table_hash8ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key8_ext_params key8ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key8ext_params = {
+ .name = "TABLE",
+ .key_size = 8,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key8ext[8];
@@ -652,14 +656,14 @@ test_table_hash8ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key8ext_params.n_entries = 0;
+ key8ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key8_ext_ops,
(void *)&key8ext_params, (void *)key8ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8ext_params.n_entries = 1<<16;
+ key8ext_params.n_keys = 1<<16;
key8ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key8_ext_ops,
@@ -667,13 +671,6 @@ test_table_hash8ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key8ext_params.f_hash = pipeline_test_hash;
- key8ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key8_ext_ops,
- (void *)&key8ext_params, (void *)key8ext, &table_packets, NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index efbd0c6..7f85587 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -67,9 +67,9 @@ unsigned n_table_tests = RTE_DIM(table_tests);
/* Function prototypes */
static int
-test_table_hash_lru_generic(struct rte_table_ops *ops);
+test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size);
static int
-test_table_hash_ext_generic(struct rte_table_ops *ops);
+test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size);
struct rte_bucket_4_8 {
/* Cache line 0 */
@@ -655,7 +655,7 @@ test_table_lpm_ipv6(void)
}
static int
-test_table_hash_lru_generic(struct rte_table_ops *ops)
+test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size)
{
int status, i;
uint64_t expected_mask = 0, result_mask;
@@ -667,43 +667,31 @@ test_table_hash_lru_generic(struct rte_table_ops *ops)
int key_found;
/* Initialize params and create tables */
- struct rte_table_hash_key8_lru_params hash_params = {
- .n_entries = 1 << 10,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(1),
+ struct rte_table_hash_params hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 10,
+ .n_buckets = 1 << 10,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
- hash_params.n_entries = 0;
+ hash_params.n_keys = 0;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -1;
- hash_params.n_entries = 1 << 10;
- hash_params.signature_offset = APP_METADATA_OFFSET(1);
-
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -2;
-
- hash_params.signature_offset = APP_METADATA_OFFSET(0);
- hash_params.key_offset = APP_METADATA_OFFSET(1);
-
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -3;
-
- hash_params.key_offset = APP_METADATA_OFFSET(32);
+ hash_params.n_keys = 1 << 10;
hash_params.f_hash = NULL;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -4;
- hash_params.f_hash = pipeline_test_hash;
+ hash_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
table = ops->f_create(&hash_params, 0, 1);
if (table == NULL)
@@ -770,7 +758,7 @@ test_table_hash_lru_generic(struct rte_table_ops *ops)
}
static int
-test_table_hash_ext_generic(struct rte_table_ops *ops)
+test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size)
{
int status, i;
uint64_t expected_mask = 0, result_mask;
@@ -782,35 +770,24 @@ test_table_hash_ext_generic(struct rte_table_ops *ops)
void *entry_ptr;
/* Initialize params and create tables */
- struct rte_table_hash_key8_ext_params hash_params = {
- .n_entries = 1 << 10,
- .n_entries_ext = 1 << 4,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(1),
+ struct rte_table_hash_params hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 10,
+ .n_buckets = 1 << 10,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
- hash_params.n_entries = 0;
+ hash_params.n_keys = 0;
table = ops->f_create(&hash_params, 0, 1);
if (table != NULL)
return -1;
- hash_params.n_entries = 1 << 10;
- hash_params.n_entries_ext = 0;
- table = ops->f_create(&hash_params, 0, 1);
- if (table != NULL)
- return -2;
-
- hash_params.n_entries_ext = 1 << 4;
- hash_params.signature_offset = APP_METADATA_OFFSET(1);
- table = ops->f_create(&hash_params, 0, 1);
- if (table == NULL)
- return -2;
-
- hash_params.signature_offset = APP_METADATA_OFFSET(0);
+ hash_params.n_keys = 1 << 10;
hash_params.key_offset = APP_METADATA_OFFSET(1);
table = ops->f_create(&hash_params, 0, 1);
@@ -824,7 +801,7 @@ test_table_hash_ext_generic(struct rte_table_ops *ops)
if (table != NULL)
return -4;
- hash_params.f_hash = pipeline_test_hash;
+ hash_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
table = ops->f_create(&hash_params, 0, 1);
if (table == NULL)
@@ -895,20 +872,9 @@ test_table_hash_lru(void)
{
int status;
- status = test_table_hash_lru_generic(&rte_table_hash_key8_lru_ops);
- if (status < 0)
- return status;
-
status = test_table_hash_lru_generic(
- &rte_table_hash_key8_lru_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_lru_generic(&rte_table_hash_key16_lru_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_lru_generic(&rte_table_hash_key32_lru_ops);
+ &rte_table_hash_key8_lru_ops,
+ 8);
if (status < 0)
return status;
@@ -924,20 +890,7 @@ test_table_hash_ext(void)
{
int status;
- status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_ext_generic(
- &rte_table_hash_key8_ext_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops);
- if (status < 0)
- return status;
-
- status = test_table_hash_ext_generic(&rte_table_hash_key32_ext_ops);
+ status = test_table_hash_ext_generic(&rte_table_hash_key8_ext_ops, 8);
if (status < 0)
return status;
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 11/18] librte_table: rework 16-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (28 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 10/18] librte_table: rework 8-byte key hash tables Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 12/18] librte_table: rework 32-byte " Cristian Dumitrescu
` (6 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the 16-byte key hash tables (both the extendible
bucket and LRU)to use the mask-based hash function and the
unified parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
.../pipeline/pipeline_flow_classification_be.c | 14 +-
lib/librte_table/rte_table_hash.h | 54 ---
lib/librte_table/rte_table_hash_key16.c | 366 +++++++++++----------
test/test-pipeline/pipeline_hash.c | 25 +-
test/test/test_table_combined.c | 40 +--
test/test/test_table_tables.c | 8 +
6 files changed, 233 insertions(+), 274 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index 47e6d35..e4a18e4 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -492,18 +492,6 @@ static void *pipeline_fc_init(struct pipeline_params *params,
/* Tables */
p->n_tables = 1;
{
- struct rte_table_hash_key16_ext_params
- table_hash_key16_params = {
- .n_entries = p_fc->n_flows,
- .n_entries_ext = p_fc->n_flows,
- .signature_offset = p_fc->hash_offset,
- .key_offset = p_fc->key_offset,
- .f_hash = hash_func[(p_fc->key_size / 8) - 1],
- .key_mask = (p_fc->key_mask_present) ?
- p_fc->key_mask : NULL,
- .seed = 0,
- };
-
struct rte_table_hash_params table_hash_params = {
.name = p->name,
.key_size = p_fc->key_size,
@@ -536,7 +524,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
case 16:
table_params.ops = &rte_table_hash_key16_ext_ops;
- table_params.arg_create = &table_hash_key16_params;
+ table_params.arg_create = &table_hash_params;
break;
default:
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 6e25054..079dc93 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -143,62 +143,8 @@ extern struct rte_table_ops rte_table_hash_key8_lru_ops;
extern struct rte_table_ops rte_table_hash_key8_ext_ops;
-/**
- * 16-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key16_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
extern struct rte_table_ops rte_table_hash_key16_lru_ops;
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key16_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Bit-mask to be AND-ed to the key on lookup */
- uint8_t *key_mask;
-};
-
extern struct rte_table_ops rte_table_hash_key16_ext_ops;
/**
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
index 7f6651c..84b59cc 100644
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 16
+#define KEY_SIZE 16
+
+#define KEYS_PER_BUCKET 4
#define RTE_BUCKET_ENTRY_VALID 0x1LLU
@@ -79,14 +81,12 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
uint64_t key_mask[2];
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
/* Extendible buckets */
@@ -99,17 +99,55 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key16_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return (a64[0] != (b64[0] & b_mask64[0])) ||
+ (a64[1] != (b64[1] & b_mask64[1]));
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+ dst64[1] = src64[1] & src_mask64[1];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_buckets */
+ if ((params->n_buckets == 0) ||
+ (!rte_is_power_of_2(params->n_buckets))) {
+ RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
/* f_hash */
if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: f_hash function pointer is NULL\n", __func__);
+ RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
+ __func__);
return -EINVAL;
}
@@ -121,46 +159,67 @@ rte_table_hash_create_key16_lru(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key16_lru_params *p =
- (struct rte_table_hash_key16_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket,
- key_size, bucket_size_cl, total_size, i;
+ uint64_t bucket_size, total_size;
+ uint32_t n_buckets, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_16) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 16;
+
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + n_buckets *
- bucket_size_cl * RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +
+ KEYS_PER_BUCKET * entry_size);
+ total_size = sizeof(struct rte_table_hash) + n_buckets * bucket_size;
+
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -212,19 +271,19 @@ rte_table_hash_entry_add_key16_lru(
uint64_t signature, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -238,13 +297,13 @@ rte_table_hash_entry_add_key16_lru(
/* Key is not present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -257,7 +316,7 @@ rte_table_hash_entry_add_key16_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
bucket->signature[pos] = signature;
- memcpy(bucket->key[pos], key, f->key_size);
+ keycpy(&bucket->key[pos], key, f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
*key_found = 0;
@@ -278,19 +337,19 @@ rte_table_hash_entry_delete_key16_lru(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = 0;
@@ -306,81 +365,71 @@ rte_table_hash_entry_delete_key16_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key16_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: f_hash function pointer is NULL\n", __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key16_ext(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key16_ext_params *p =
- (struct rte_table_hash_key16_ext_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket, key_size,
- bucket_size_cl, stack_size_cl, total_size, i;
+ uint64_t bucket_size, stack_size, total_size;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_16) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 16;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_16) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_16) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
total_size = sizeof(struct rte_table_hash) +
- ((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *
- RTE_CACHE_LINE_SIZE;
+ (p->n_buckets + n_buckets_ext) * bucket_size + stack_size;
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
- RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ RTE_LOG(INFO, TABLE, "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
- f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->n_buckets = p->n_buckets;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -388,10 +437,7 @@ rte_table_hash_create_key16_ext(void *params,
f->n_buckets_ext = n_buckets_ext;
f->stack_pos = n_buckets_ext;
f->stack = (uint32_t *)
- &f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
-
- for (i = 0; i < n_buckets_ext; i++)
- f->stack[i] = i;
+ &f->memory[(p->n_buckets + n_buckets_ext) * f->bucket_size];
if (p->key_mask != NULL) {
f->key_mask[0] = (((uint64_t *)p->key_mask)[0]);
@@ -401,6 +447,9 @@ rte_table_hash_create_key16_ext(void *params,
f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
}
+ for (i = 0; i < n_buckets_ext; i++)
+ f->stack[i] = i;
+
return f;
}
@@ -432,20 +481,20 @@ rte_table_hash_entry_add_key16_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_16 *)
- &f->memory[bucket_index * f->bucket_size];
+ &f->memory[bucket_index * f->bucket_size];
signature |= RTE_BUCKET_ENTRY_VALID;
/* Key is present in the bucket */
for (bucket = bucket0; bucket != NULL; bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -458,17 +507,17 @@ rte_table_hash_entry_add_key16_ext(
/* Key is not present in the bucket */
for (bucket_prev = NULL, bucket = bucket0; bucket != NULL;
- bucket_prev = bucket, bucket = bucket->next)
+ bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -487,7 +536,7 @@ rte_table_hash_entry_add_key16_ext(
bucket_prev->next_valid = 1;
bucket->signature[0] = signature;
- memcpy(bucket->key[0], key, f->key_size);
+ keycpy(&bucket->key[0], key, f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -509,7 +558,7 @@ rte_table_hash_entry_delete_key16_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_16 *)
&f->memory[bucket_index * f->bucket_size];
@@ -520,18 +569,17 @@ rte_table_hash_entry_delete_key16_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = 0;
*key_found = 1;
if (entry)
- memcpy(entry, bucket_data,
- f->entry_size);
+ memcpy(entry, bucket_data, f->entry_size);
if ((bucket->signature[0] == 0) &&
(bucket->signature[1] == 0) &&
@@ -558,26 +606,28 @@ rte_table_hash_entry_delete_key16_ext(
return 0;
}
-#define lookup_key16_cmp(key_in, bucket, pos) \
+#define lookup_key16_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4][2], or[4], signature[4]; \
+ uint64_t xor[4][2], or[4], signature[4], k[2]; \
\
+ k[0] = key_in[0] & f->key_mask[0]; \
+ k[1] = key_in[1] & f->key_mask[1]; \
signature[0] = (~bucket->signature[0]) & 1; \
signature[1] = (~bucket->signature[1]) & 1; \
signature[2] = (~bucket->signature[2]) & 1; \
signature[3] = (~bucket->signature[3]) & 1; \
\
- xor[0][0] = key_in[0] ^ bucket->key[0][0]; \
- xor[0][1] = key_in[1] ^ bucket->key[0][1]; \
+ xor[0][0] = k[0] ^ bucket->key[0][0]; \
+ xor[0][1] = k[1] ^ bucket->key[0][1]; \
\
- xor[1][0] = key_in[0] ^ bucket->key[1][0]; \
- xor[1][1] = key_in[1] ^ bucket->key[1][1]; \
+ xor[1][0] = k[0] ^ bucket->key[1][0]; \
+ xor[1][1] = k[1] ^ bucket->key[1][1]; \
\
- xor[2][0] = key_in[0] ^ bucket->key[2][0]; \
- xor[2][1] = key_in[1] ^ bucket->key[2][1]; \
+ xor[2][0] = k[0] ^ bucket->key[2][0]; \
+ xor[2][1] = k[1] ^ bucket->key[2][1]; \
\
- xor[3][0] = key_in[0] ^ bucket->key[3][0]; \
- xor[3][1] = key_in[1] ^ bucket->key[3][1]; \
+ xor[3][0] = k[0] ^ bucket->key[3][0]; \
+ xor[3][1] = k[1] ^ bucket->key[3][1]; \
\
or[0] = xor[0][0] | xor[0][1] | signature[0]; \
or[1] = xor[1][0] | xor[1][1] | signature[1]; \
@@ -613,14 +663,9 @@ rte_table_hash_entry_delete_key16_ext(
uint64_t *key; \
uint64_t signature = 0; \
uint32_t bucket_index; \
- uint64_t hash_key_buffer[2]; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset);\
- \
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- signature = f->f_hash(hash_key_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed); \
+ signature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed); \
\
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_16 *) \
@@ -635,14 +680,10 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket2, pos); \
+ lookup_key16_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -660,14 +701,10 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket2, pos); \
+ lookup_key16_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -690,15 +727,11 @@ rte_table_hash_entry_delete_key16_ext(
void *a; \
uint64_t pkt_mask, bucket_mask; \
uint64_t *key; \
- uint64_t hash_key_buffer[2]; \
uint32_t pos; \
\
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
- hash_key_buffer[0] = key[0] & f->key_mask[0]; \
- hash_key_buffer[1] = key[1] & f->key_mask[1]; \
- \
- lookup_key16_cmp(hash_key_buffer, bucket, pos); \
+ lookup_key16_cmp(key, bucket, pos, f); \
\
pkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -763,15 +796,11 @@ rte_table_hash_entry_delete_key16_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
uint64_t *key10, *key11; \
- uint64_t hash_offset_buffer[2]; \
uint64_t signature10, signature11; \
uint32_t bucket10_index, bucket11_index; \
\
key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset);\
- hash_offset_buffer[0] = key10[0] & f->key_mask[0]; \
- hash_offset_buffer[1] = key10[1] & f->key_mask[1]; \
- signature10 = f->f_hash(hash_offset_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed);\
+ signature10 = f->f_hash(key10, f->key_mask, KEY_SIZE, f->seed);\
bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_16 *) \
&f->memory[bucket10_index * f->bucket_size]; \
@@ -779,10 +808,7 @@ rte_table_hash_entry_delete_key16_ext(
rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
\
key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset);\
- hash_offset_buffer[0] = key11[0] & f->key_mask[0]; \
- hash_offset_buffer[1] = key11[1] & f->key_mask[1]; \
- signature11 = f->f_hash(hash_offset_buffer, \
- RTE_TABLE_HASH_KEY_SIZE, f->seed);\
+ signature11 = f->f_hash(key11, f->key_mask, KEY_SIZE, f->seed);\
bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_16 *) \
&f->memory[bucket11_index * f->bucket_size]; \
@@ -796,19 +822,13 @@ rte_table_hash_entry_delete_key16_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask; \
uint64_t *key20, *key21; \
- uint64_t hash_key_buffer20[2]; \
- uint64_t hash_key_buffer21[2]; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_key_buffer20[0] = key20[0] & f->key_mask[0]; \
- hash_key_buffer20[1] = key20[1] & f->key_mask[1]; \
- hash_key_buffer21[0] = key21[0] & f->key_mask[0]; \
- hash_key_buffer21[1] = key21[1] & f->key_mask[1]; \
\
- lookup_key16_cmp(hash_key_buffer20, bucket20, pos20); \
- lookup_key16_cmp(hash_key_buffer21, bucket21, pos21); \
+ lookup_key16_cmp(key20, bucket20, pos20, f); \
+ lookup_key16_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -831,19 +851,13 @@ rte_table_hash_entry_delete_key16_ext(
void *a20, *a21; \
uint64_t pkt20_mask, pkt21_mask, bucket20_mask, bucket21_mask;\
uint64_t *key20, *key21; \
- uint64_t hash_key_buffer20[2]; \
- uint64_t hash_key_buffer21[2]; \
uint32_t pos20, pos21; \
\
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
- hash_key_buffer20[0] = key20[0] & f->key_mask[0]; \
- hash_key_buffer20[1] = key20[1] & f->key_mask[1]; \
- hash_key_buffer21[0] = key21[0] & f->key_mask[0]; \
- hash_key_buffer21[1] = key21[1] & f->key_mask[1]; \
\
- lookup_key16_cmp(hash_key_buffer20, bucket20, pos20); \
- lookup_key16_cmp(hash_key_buffer21, bucket21, pos21); \
+ lookup_key16_cmp(key20, bucket20, pos20, f); \
+ lookup_key16_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -994,7 +1008,7 @@ rte_table_hash_lookup_key16_lru(
RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
__builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key16_lru() */
+} /* lookup LRU */
static int
rte_table_hash_lookup_key16_ext(
@@ -1146,7 +1160,7 @@ rte_table_hash_lookup_key16_ext(
RTE_TABLE_HASH_KEY16_STATS_PKTS_LOOKUP_MISS(f, n_pkts_in -
__builtin_popcountll(pkts_mask_out));
return 0;
-} /* rte_table_hash_lookup_key16_ext() */
+} /* lookup EXT */
static int
rte_table_hash_key16_stats_read(void *table, struct rte_table_stats *stats, int clear)
@@ -1167,6 +1181,8 @@ struct rte_table_ops rte_table_hash_key16_lru_ops = {
.f_free = rte_table_hash_free_key16_lru,
.f_add = rte_table_hash_entry_add_key16_lru,
.f_delete = rte_table_hash_entry_delete_key16_lru,
+ .f_add_bulk = NULL,
+ .f_delete_bulk = NULL,
.f_lookup = rte_table_hash_lookup_key16_lru,
.f_stats = rte_table_hash_key16_stats_read,
};
@@ -1176,6 +1192,8 @@ struct rte_table_ops rte_table_hash_key16_ext_ops = {
.f_free = rte_table_hash_free_key16_ext,
.f_add = rte_table_hash_entry_add_key16_ext,
.f_delete = rte_table_hash_entry_delete_key16_ext,
+ .f_add_bulk = NULL,
+ .f_delete_bulk = NULL,
.f_lookup = rte_table_hash_lookup_key16_ext,
.f_stats = rte_table_hash_key16_stats_read,
};
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index f542506..222c48c 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -285,14 +285,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_EXT:
{
- struct rte_table_hash_key16_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
.key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
+ .seed = 0,
};
struct rte_pipeline_table_params table_params = {
@@ -311,13 +312,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_LRU:
{
- struct rte_table_hash_key16_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
- .seed = 0,
.key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
+ .seed = 0,
};
struct rte_pipeline_table_params table_params = {
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index ddcd14b..4cfd955 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -502,13 +502,15 @@ test_table_hash16lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key16_lru_params key16lru_params = {
- .n_entries = 1<<16,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key16lru_params = {
+ .name = "TABLE",
+ .key_size = 16,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key16lru[16];
@@ -537,14 +539,14 @@ test_table_hash16lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key16lru_params.n_entries = 0;
+ key16lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key16_lru_ops,
(void *)&key16lru_params, (void *)key16lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16lru_params.n_entries = 1<<16;
+ key16lru_params.n_keys = 1<<16;
key16lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key16_lru_ops,
@@ -680,14 +682,15 @@ test_table_hash16ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key16_ext_params key16ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key16ext_params = {
+ .name = "TABLE",
+ .key_size = 16,
.key_offset = APP_METADATA_OFFSET(32),
.key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key16ext[16];
@@ -716,14 +719,14 @@ test_table_hash16ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key16ext_params.n_entries = 0;
+ key16ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key16_ext_ops,
(void *)&key16ext_params, (void *)key16ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16ext_params.n_entries = 1<<16;
+ key16ext_params.n_keys = 1<<16;
key16ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key16_ext_ops,
@@ -731,13 +734,6 @@ test_table_hash16ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key16ext_params.f_hash = pipeline_test_hash;
- key16ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key16_ext_ops,
- (void *)&key16ext_params, (void *)key16ext, &table_packets, NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 7f85587..2f916b7 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -878,6 +878,11 @@ test_table_hash_lru(void)
if (status < 0)
return status;
+ status = test_table_hash_lru_generic(
+ &rte_table_hash_key16_lru_ops,
+ 16);
+ if (status < 0)
+ return status;
status = test_lru_update();
if (status < 0)
return status;
@@ -894,6 +899,9 @@ test_table_hash_ext(void)
if (status < 0)
return status;
+ status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops, 16);
+ if (status < 0)
+ return status;
return 0;
}
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 12/18] librte_table: rework 32-byte key hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (29 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 11/18] librte_table: rework 16-byte " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 13/18] librte_table: rework cuckoo hash table Cristian Dumitrescu
` (5 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the 32-byte key hash tables (both the extendible
bucket and LRU)to use the mask-based hash function and the
unified parameter structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 50 -----
lib/librte_table/rte_table_hash_key32.c | 386 ++++++++++++++++++++------------
test/test-pipeline/pipeline_hash.c | 23 +-
test/test/test_table_combined.c | 43 ++--
test/test/test_table_tables.c | 12 +
5 files changed, 283 insertions(+), 231 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 079dc93..16e1dfa 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -147,58 +147,8 @@ extern struct rte_table_ops rte_table_hash_key16_lru_ops;
extern struct rte_table_ops rte_table_hash_key16_ext_ops;
-/**
- * 32-byte key hash tables
- *
- */
-/** LRU hash table parameters */
-struct rte_table_hash_key32_lru_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
-/** LRU hash table operations for pre-computed key signature */
extern struct rte_table_ops rte_table_hash_key32_lru_ops;
-/** Extendible bucket hash table parameters */
-struct rte_table_hash_key32_ext_params {
- /** Maximum number of entries (and keys) in the table */
- uint32_t n_entries;
-
- /** Number of entries (and keys) for hash table bucket extensions. Each
- bucket is extended in increments of 4 keys. */
- uint32_t n_entries_ext;
-
- /** Hash function */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed for the hash function */
- uint64_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-};
-
-/** Extendible bucket hash table operations */
extern struct rte_table_ops rte_table_hash_key32_ext_ops;
/** Cuckoo hash table parameters */
diff --git a/lib/librte_table/rte_table_hash_key32.c b/lib/librte_table/rte_table_hash_key32.c
index f8f6662..3eb7e4f 100644
--- a/lib/librte_table/rte_table_hash_key32.c
+++ b/lib/librte_table/rte_table_hash_key32.c
@@ -42,7 +42,9 @@
#include "rte_table_hash.h"
#include "rte_lru.h"
-#define RTE_TABLE_HASH_KEY_SIZE 32
+#define KEY_SIZE 32
+
+#define KEYS_PER_BUCKET 4
#define RTE_BUCKET_ENTRY_VALID 0x1LLU
@@ -79,13 +81,12 @@ struct rte_table_hash {
/* Input parameters */
uint32_t n_buckets;
- uint32_t n_entries_per_bucket;
uint32_t key_size;
uint32_t entry_size;
uint32_t bucket_size;
- uint32_t signature_offset;
uint32_t key_offset;
- rte_table_hash_op_hash_nomask f_hash;
+ uint64_t key_mask[4];
+ rte_table_hash_op_hash f_hash;
uint64_t seed;
/* Extendible buckets */
@@ -98,10 +99,52 @@ struct rte_table_hash {
};
static int
-check_params_create_lru(struct rte_table_hash_key32_lru_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
+keycmp(void *a, void *b, void *b_mask)
+{
+ uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask;
+
+ return (a64[0] != (b64[0] & b_mask64[0])) ||
+ (a64[1] != (b64[1] & b_mask64[1])) ||
+ (a64[2] != (b64[2] & b_mask64[2])) ||
+ (a64[3] != (b64[3] & b_mask64[3]));
+}
+
+static void
+keycpy(void *dst, void *src, void *src_mask)
+{
+ uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask;
+
+ dst64[0] = src64[0] & src_mask64[0];
+ dst64[1] = src64[1] & src_mask64[1];
+ dst64[2] = src64[2] & src_mask64[2];
+ dst64[3] = src64[3] & src_mask64[3];
+}
+
+static int
+check_params_create(struct rte_table_hash_params *params)
+{
+ /* name */
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "%s: name invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* key_size */
+ if (params->key_size != KEY_SIZE) {
+ RTE_LOG(ERR, TABLE, "%s: key_size invalid value\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_keys */
+ if (params->n_keys == 0) {
+ RTE_LOG(ERR, TABLE, "%s: n_keys is zero\n", __func__);
+ return -EINVAL;
+ }
+
+ /* n_buckets */
+ if ((params->n_buckets == 0) ||
+ (!rte_is_power_of_2(params->n_buckets))) {
+ RTE_LOG(ERR, TABLE, "%s: n_buckets invalid value\n", __func__);
return -EINVAL;
}
@@ -120,51 +163,83 @@ rte_table_hash_create_key32_lru(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key32_lru_params *p =
- (struct rte_table_hash_key32_lru_params *) params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_entries_per_bucket, key_size, bucket_size_cl;
- uint32_t total_size, i;
+ uint64_t bucket_size, total_size;
+ uint32_t n_buckets, i;
/* Check input parameters */
- if ((check_params_create_lru(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
- ((sizeof(struct rte_bucket_4_32) % 64) != 0)) {
+ ((sizeof(struct rte_bucket_4_32) % 64) != 0))
return NULL;
- }
- n_entries_per_bucket = 4;
- key_size = 32;
+
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of buckets (n_buckets) so that there a chance
+ * to store n_keys keys in the table.
+ *
+ * Note: Since the buckets do not get extended, it is not possible to
+ * guarantee that n_keys keys can be stored in the table at any time. In the
+ * worst case scenario when all the n_keys fall into the same bucket, only
+ * a maximum of KEYS_PER_BUCKET keys will be stored in the table. This case
+ * defeats the purpose of the hash table. It indicates unsuitable f_hash or
+ * n_keys to n_buckets ratio.
+ *
+ * MIN(n_buckets) = (n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET
+ */
+ n_buckets = rte_align32pow2(
+ (p->n_keys + KEYS_PER_BUCKET - 1) / KEYS_PER_BUCKET);
+ n_buckets = RTE_MAX(n_buckets, p->n_buckets);
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- bucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- total_size = sizeof(struct rte_table_hash) + n_buckets *
- bucket_size_cl * RTE_CACHE_LINE_SIZE;
-
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +
+ KEYS_PER_BUCKET * entry_size);
+ total_size = sizeof(struct rte_table_hash) + n_buckets * bucket_size;
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
+
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n", __func__,
- total_size);
+ "%s: Hash table %s memory footprint "
+ "is %" PRIu64 " bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
+ if (p->key_mask != NULL) {
+ f->key_mask[0] = ((uint64_t *)p->key_mask)[0];
+ f->key_mask[1] = ((uint64_t *)p->key_mask)[1];
+ f->key_mask[2] = ((uint64_t *)p->key_mask)[2];
+ f->key_mask[3] = ((uint64_t *)p->key_mask)[3];
+ } else {
+ f->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;
+ }
+
for (i = 0; i < n_buckets; i++) {
struct rte_bucket_4_32 *bucket;
@@ -204,7 +279,7 @@ rte_table_hash_entry_add_key32_lru(
uint64_t signature, pos;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -213,10 +288,10 @@ rte_table_hash_entry_add_key32_lru(
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
memcpy(bucket_data, entry, f->entry_size);
@@ -230,13 +305,13 @@ rte_table_hash_entry_add_key32_lru(
/* Key is not present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
lru_update(bucket, i);
*key_found = 0;
@@ -249,10 +324,10 @@ rte_table_hash_entry_add_key32_lru(
/* Bucket full: replace LRU entry */
pos = lru_pos(bucket);
bucket->signature[pos] = signature;
- memcpy(bucket->key[pos], key, f->key_size);
+ keycpy(&bucket->key[pos], key, f->key_mask);
memcpy(&bucket->data[pos * f->entry_size], entry, f->entry_size);
lru_update(bucket, pos);
- *key_found = 0;
+ *key_found = 0;
*entry_ptr = (void *) &bucket->data[pos * f->entry_size];
return 0;
@@ -270,7 +345,7 @@ rte_table_hash_entry_delete_key32_lru(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -279,10 +354,10 @@ rte_table_hash_entry_delete_key32_lru(
/* Key is present in the bucket */
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i * f->entry_size];
bucket->signature[i] = 0;
@@ -299,81 +374,72 @@ rte_table_hash_entry_delete_key32_lru(
return 0;
}
-static int
-check_params_create_ext(struct rte_table_hash_key32_ext_params *params) {
- /* n_entries */
- if (params->n_entries == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries is zero\n", __func__);
- return -EINVAL;
- }
-
- /* n_entries_ext */
- if (params->n_entries_ext == 0) {
- RTE_LOG(ERR, TABLE, "%s: n_entries_ext is zero\n", __func__);
- return -EINVAL;
- }
-
- /* f_hash */
- if (params->f_hash == NULL) {
- RTE_LOG(ERR, TABLE, "%s: f_hash function pointer is NULL\n",
- __func__);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void *
rte_table_hash_create_key32_ext(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_table_hash_key32_ext_params *p =
- params;
+ struct rte_table_hash_params *p = params;
struct rte_table_hash *f;
- uint32_t n_buckets, n_buckets_ext, n_entries_per_bucket;
- uint32_t key_size, bucket_size_cl, stack_size_cl, total_size, i;
+ uint64_t bucket_size, stack_size, total_size;
+ uint32_t n_buckets_ext, i;
/* Check input parameters */
- if ((check_params_create_ext(p) != 0) ||
+ if ((check_params_create(p) != 0) ||
((sizeof(struct rte_table_hash) % RTE_CACHE_LINE_SIZE) != 0) ||
((sizeof(struct rte_bucket_4_32) % 64) != 0))
return NULL;
- n_entries_per_bucket = 4;
- key_size = 32;
+ /*
+ * Table dimensioning
+ *
+ * Objective: Pick the number of bucket extensions (n_buckets_ext) so that
+ * it is guaranteed that n_keys keys can be stored in the table at any time.
+ *
+ * The worst case scenario takes place when all the n_keys keys fall into
+ * the same bucket. Actually, due to the KEYS_PER_BUCKET scheme, the worst
+ * case takes place when (n_keys - KEYS_PER_BUCKET + 1) keys fall into the
+ * same bucket, while the remaining (KEYS_PER_BUCKET - 1) keys each fall
+ * into a different bucket. This case defeats the purpose of the hash table.
+ * It indicates unsuitable f_hash or n_keys to n_buckets ratio.
+ *
+ * n_buckets_ext = n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1
+ */
+ n_buckets_ext = p->n_keys / KEYS_PER_BUCKET + KEYS_PER_BUCKET - 1;
/* Memory allocation */
- n_buckets = rte_align32pow2((p->n_entries + n_entries_per_bucket - 1) /
- n_entries_per_bucket);
- n_buckets_ext = (p->n_entries_ext + n_entries_per_bucket - 1) /
- n_entries_per_bucket;
- bucket_size_cl = (sizeof(struct rte_bucket_4_32) + n_entries_per_bucket
- * entry_size + RTE_CACHE_LINE_SIZE - 1) / RTE_CACHE_LINE_SIZE;
- stack_size_cl = (n_buckets_ext * sizeof(uint32_t) + RTE_CACHE_LINE_SIZE - 1)
- / RTE_CACHE_LINE_SIZE;
+ bucket_size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct rte_bucket_4_32) +
+ KEYS_PER_BUCKET * entry_size);
+ stack_size = RTE_CACHE_LINE_ROUNDUP(n_buckets_ext * sizeof(uint32_t));
total_size = sizeof(struct rte_table_hash) +
- ((n_buckets + n_buckets_ext) * bucket_size_cl + stack_size_cl) *
- RTE_CACHE_LINE_SIZE;
+ (p->n_buckets + n_buckets_ext) * bucket_size + stack_size;
+ if (total_size > SIZE_MAX) {
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
+ return NULL;
+ }
- f = rte_zmalloc_socket("TABLE", total_size, RTE_CACHE_LINE_SIZE, socket_id);
+ f = rte_zmalloc_socket(p->name,
+ (size_t)total_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
if (f == NULL) {
- RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for hash table\n",
- __func__, total_size);
+ RTE_LOG(ERR, TABLE, "%s: Cannot allocate %" PRIu64 " bytes "
+ "for hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
RTE_LOG(INFO, TABLE,
- "%s: Hash table memory footprint is %u bytes\n", __func__,
- total_size);
+ "%s: Hash table %s memory footprint "
+ "is %" PRIu64" bytes\n",
+ __func__, p->name, total_size);
/* Memory initialization */
- f->n_buckets = n_buckets;
- f->n_entries_per_bucket = n_entries_per_bucket;
- f->key_size = key_size;
+ f->n_buckets = p->n_buckets;
+ f->key_size = KEY_SIZE;
f->entry_size = entry_size;
- f->bucket_size = bucket_size_cl * RTE_CACHE_LINE_SIZE;
- f->signature_offset = p->signature_offset;
+ f->bucket_size = bucket_size;
f->key_offset = p->key_offset;
f->f_hash = p->f_hash;
f->seed = p->seed;
@@ -381,7 +447,19 @@ rte_table_hash_create_key32_ext(void *params,
f->n_buckets_ext = n_buckets_ext;
f->stack_pos = n_buckets_ext;
f->stack = (uint32_t *)
- &f->memory[(n_buckets + n_buckets_ext) * f->bucket_size];
+ &f->memory[(p->n_buckets + n_buckets_ext) * f->bucket_size];
+
+ if (p->key_mask != NULL) {
+ f->key_mask[0] = (((uint64_t *)p->key_mask)[0]);
+ f->key_mask[1] = (((uint64_t *)p->key_mask)[1]);
+ f->key_mask[2] = (((uint64_t *)p->key_mask)[2]);
+ f->key_mask[3] = (((uint64_t *)p->key_mask)[3]);
+ } else {
+ f->key_mask[0] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[1] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[2] = 0xFFFFFFFFFFFFFFFFLLU;
+ f->key_mask[3] = 0xFFFFFFFFFFFFFFFFLLU;
+ }
for (i = 0; i < n_buckets_ext; i++)
f->stack[i] = i;
@@ -417,7 +495,7 @@ rte_table_hash_entry_add_key32_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -427,10 +505,10 @@ rte_table_hash_entry_add_key32_ext(
for (bucket = bucket0; bucket != NULL; bucket = bucket->next) {
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
@@ -448,14 +526,14 @@ rte_table_hash_entry_add_key32_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if (bucket_signature == 0) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = signature;
- memcpy(bucket_key, key, f->key_size);
+ keycpy(bucket_key, key, f->key_mask);
memcpy(bucket_data, entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) bucket_data;
@@ -475,7 +553,7 @@ rte_table_hash_entry_add_key32_ext(
bucket_prev->next_valid = 1;
bucket->signature[0] = signature;
- memcpy(bucket->key[0], key, f->key_size);
+ keycpy(&bucket->key[0], key, f->key_mask);
memcpy(&bucket->data[0], entry, f->entry_size);
*key_found = 0;
*entry_ptr = (void *) &bucket->data[0];
@@ -497,7 +575,7 @@ rte_table_hash_entry_delete_key32_ext(
uint64_t signature;
uint32_t bucket_index, i;
- signature = f->f_hash(key, f->key_size, f->seed);
+ signature = f->f_hash(key, f->key_mask, f->key_size, f->seed);
bucket_index = signature & (f->n_buckets - 1);
bucket0 = (struct rte_bucket_4_32 *)
&f->memory[bucket_index * f->bucket_size];
@@ -508,24 +586,23 @@ rte_table_hash_entry_delete_key32_ext(
bucket_prev = bucket, bucket = bucket->next)
for (i = 0; i < 4; i++) {
uint64_t bucket_signature = bucket->signature[i];
- uint8_t *bucket_key = (uint8_t *) bucket->key[i];
+ uint8_t *bucket_key = (uint8_t *) &bucket->key[i];
if ((bucket_signature == signature) &&
- (memcmp(key, bucket_key, f->key_size) == 0)) {
+ (keycmp(bucket_key, key, f->key_mask) == 0)) {
uint8_t *bucket_data = &bucket->data[i *
f->entry_size];
bucket->signature[i] = 0;
*key_found = 1;
if (entry)
- memcpy(entry, bucket_data,
- f->entry_size);
+ memcpy(entry, bucket_data, f->entry_size);
if ((bucket->signature[0] == 0) &&
- (bucket->signature[1] == 0) &&
- (bucket->signature[2] == 0) &&
- (bucket->signature[3] == 0) &&
- (bucket_prev != NULL)) {
+ (bucket->signature[1] == 0) &&
+ (bucket->signature[2] == 0) &&
+ (bucket->signature[3] == 0) &&
+ (bucket_prev != NULL)) {
bucket_prev->next = bucket->next;
bucket_prev->next_valid =
bucket->next_valid;
@@ -546,34 +623,39 @@ rte_table_hash_entry_delete_key32_ext(
return 0;
}
-#define lookup_key32_cmp(key_in, bucket, pos) \
+#define lookup_key32_cmp(key_in, bucket, pos, f) \
{ \
- uint64_t xor[4][4], or[4], signature[4]; \
+ uint64_t xor[4][4], or[4], signature[4], k[4]; \
+ \
+ k[0] = key_in[0] & f->key_mask[0]; \
+ k[1] = key_in[1] & f->key_mask[1]; \
+ k[2] = key_in[2] & f->key_mask[2]; \
+ k[3] = key_in[3] & f->key_mask[3]; \
\
signature[0] = ((~bucket->signature[0]) & 1); \
signature[1] = ((~bucket->signature[1]) & 1); \
signature[2] = ((~bucket->signature[2]) & 1); \
signature[3] = ((~bucket->signature[3]) & 1); \
\
- xor[0][0] = key_in[0] ^ bucket->key[0][0]; \
- xor[0][1] = key_in[1] ^ bucket->key[0][1]; \
- xor[0][2] = key_in[2] ^ bucket->key[0][2]; \
- xor[0][3] = key_in[3] ^ bucket->key[0][3]; \
+ xor[0][0] = k[0] ^ bucket->key[0][0]; \
+ xor[0][1] = k[1] ^ bucket->key[0][1]; \
+ xor[0][2] = k[2] ^ bucket->key[0][2]; \
+ xor[0][3] = k[3] ^ bucket->key[0][3]; \
\
- xor[1][0] = key_in[0] ^ bucket->key[1][0]; \
- xor[1][1] = key_in[1] ^ bucket->key[1][1]; \
- xor[1][2] = key_in[2] ^ bucket->key[1][2]; \
- xor[1][3] = key_in[3] ^ bucket->key[1][3]; \
+ xor[1][0] = k[0] ^ bucket->key[1][0]; \
+ xor[1][1] = k[1] ^ bucket->key[1][1]; \
+ xor[1][2] = k[2] ^ bucket->key[1][2]; \
+ xor[1][3] = k[3] ^ bucket->key[1][3]; \
\
- xor[2][0] = key_in[0] ^ bucket->key[2][0]; \
- xor[2][1] = key_in[1] ^ bucket->key[2][1]; \
- xor[2][2] = key_in[2] ^ bucket->key[2][2]; \
- xor[2][3] = key_in[3] ^ bucket->key[2][3]; \
+ xor[2][0] = k[0] ^ bucket->key[2][0]; \
+ xor[2][1] = k[1] ^ bucket->key[2][1]; \
+ xor[2][2] = k[2] ^ bucket->key[2][2]; \
+ xor[2][3] = k[3] ^ bucket->key[2][3]; \
\
- xor[3][0] = key_in[0] ^ bucket->key[3][0]; \
- xor[3][1] = key_in[1] ^ bucket->key[3][1]; \
- xor[3][2] = key_in[2] ^ bucket->key[3][2]; \
- xor[3][3] = key_in[3] ^ bucket->key[3][3]; \
+ xor[3][0] = k[0] ^ bucket->key[3][0]; \
+ xor[3][1] = k[1] ^ bucket->key[3][1]; \
+ xor[3][2] = k[2] ^ bucket->key[3][2]; \
+ xor[3][3] = k[3] ^ bucket->key[3][3]; \
\
or[0] = xor[0][0] | xor[0][1] | xor[0][2] | xor[0][3] | signature[0];\
or[1] = xor[1][0] | xor[1][1] | xor[1][2] | xor[1][3] | signature[1];\
@@ -604,12 +686,15 @@ rte_table_hash_entry_delete_key32_ext(
rte_prefetch0(RTE_MBUF_METADATA_UINT8_PTR(mbuf0, key_offset));\
}
-#define lookup1_stage1(mbuf1, bucket1, f) \
+#define lookup1_stage1(mbuf1, bucket1, f) \
{ \
+ uint64_t *key; \
uint64_t signature; \
uint32_t bucket_index; \
\
- signature = RTE_MBUF_METADATA_UINT32(mbuf1, f->signature_offset);\
+ key = RTE_MBUF_METADATA_UINT64_PTR(mbuf1, f->key_offset); \
+ signature = f->f_hash(key, f->key_mask, KEY_SIZE, f->seed); \
+ \
bucket_index = signature & (f->n_buckets - 1); \
bucket1 = (struct rte_bucket_4_32 *) \
&f->memory[bucket_index * f->bucket_size]; \
@@ -627,8 +712,7 @@ rte_table_hash_entry_delete_key32_ext(
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- \
- lookup_key32_cmp(key, bucket2, pos); \
+ lookup_key32_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -649,8 +733,7 @@ rte_table_hash_entry_delete_key32_ext(
uint32_t pos; \
\
key = RTE_MBUF_METADATA_UINT64_PTR(mbuf2, f->key_offset);\
- \
- lookup_key32_cmp(key, bucket2, pos); \
+ lookup_key32_cmp(key, bucket2, pos, f); \
\
pkt_mask = (bucket2->signature[pos] & 1LLU) << pkt2_index;\
pkts_mask_out |= pkt_mask; \
@@ -678,7 +761,7 @@ rte_table_hash_entry_delete_key32_ext(
bucket = buckets[pkt_index]; \
key = keys[pkt_index]; \
\
- lookup_key32_cmp(key, bucket, pos); \
+ lookup_key32_cmp(key, bucket, pos, f); \
\
pkt_mask = (bucket->signature[pos] & 1LLU) << pkt_index;\
pkts_mask_out |= pkt_mask; \
@@ -745,22 +828,27 @@ rte_table_hash_entry_delete_key32_ext(
#define lookup2_stage1(mbuf10, mbuf11, bucket10, bucket11, f) \
{ \
- uint64_t signature10, signature11; \
- uint32_t bucket10_index, bucket11_index; \
+ uint64_t *key10, *key11; \
+ uint64_t signature10, signature11; \
+ uint32_t bucket10_index, bucket11_index; \
\
- signature10 = RTE_MBUF_METADATA_UINT32(mbuf10, f->signature_offset);\
- bucket10_index = signature10 & (f->n_buckets - 1); \
+ key10 = RTE_MBUF_METADATA_UINT64_PTR(mbuf10, f->key_offset); \
+ signature10 = f->f_hash(key10, f->key_mask, KEY_SIZE, f->seed); \
+ \
+ bucket10_index = signature10 & (f->n_buckets - 1); \
bucket10 = (struct rte_bucket_4_32 *) \
&f->memory[bucket10_index * f->bucket_size]; \
- rte_prefetch0(bucket10); \
+ rte_prefetch0(bucket10); \
rte_prefetch0((void *)(((uintptr_t) bucket10) + RTE_CACHE_LINE_SIZE));\
rte_prefetch0((void *)(((uintptr_t) bucket10) + 2 * RTE_CACHE_LINE_SIZE));\
\
- signature11 = RTE_MBUF_METADATA_UINT32(mbuf11, f->signature_offset);\
- bucket11_index = signature11 & (f->n_buckets - 1); \
+ key11 = RTE_MBUF_METADATA_UINT64_PTR(mbuf11, f->key_offset); \
+ signature11 = f->f_hash(key11, f->key_mask, KEY_SIZE, f->seed);\
+ \
+ bucket11_index = signature11 & (f->n_buckets - 1); \
bucket11 = (struct rte_bucket_4_32 *) \
&f->memory[bucket11_index * f->bucket_size]; \
- rte_prefetch0(bucket11); \
+ rte_prefetch0(bucket11); \
rte_prefetch0((void *)(((uintptr_t) bucket11) + RTE_CACHE_LINE_SIZE));\
rte_prefetch0((void *)(((uintptr_t) bucket11) + 2 * RTE_CACHE_LINE_SIZE));\
}
@@ -776,8 +864,8 @@ rte_table_hash_entry_delete_key32_ext(
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
\
- lookup_key32_cmp(key20, bucket20, pos20); \
- lookup_key32_cmp(key21, bucket21, pos21); \
+ lookup_key32_cmp(key20, bucket20, pos20, f); \
+ lookup_key32_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
@@ -805,8 +893,8 @@ rte_table_hash_entry_delete_key32_ext(
key20 = RTE_MBUF_METADATA_UINT64_PTR(mbuf20, f->key_offset);\
key21 = RTE_MBUF_METADATA_UINT64_PTR(mbuf21, f->key_offset);\
\
- lookup_key32_cmp(key20, bucket20, pos20); \
- lookup_key32_cmp(key21, bucket21, pos21); \
+ lookup_key32_cmp(key20, bucket20, pos20, f); \
+ lookup_key32_cmp(key21, bucket21, pos21, f); \
\
pkt20_mask = (bucket20->signature[pos20] & 1LLU) << pkt20_index;\
pkt21_mask = (bucket21->signature[pos21] & 1LLU) << pkt21_index;\
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 222c48c..eda63e8 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -339,12 +339,14 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_EXT:
{
- struct rte_table_hash_key32_ext_params table_hash_params = {
- .n_entries = 1 << 24,
- .n_entries_ext = 1 << 23,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
};
@@ -365,11 +367,14 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_LRU:
{
- struct rte_table_hash_key32_lru_params table_hash_params = {
- .n_entries = 1 << 24,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
.key_offset = APP_METADATA_OFFSET(32),
- .f_hash = test_hash,
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
.seed = 0,
};
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index 4cfd955..93603dc 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -563,12 +563,15 @@ test_table_hash32lru(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key32_lru_params key32lru_params = {
- .n_entries = 1<<16,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key32lru_params = {
+ .name = "TABLE",
+ .key_size = 32,
.key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key32lru[32];
@@ -597,14 +600,14 @@ test_table_hash32lru(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key32lru_params.n_entries = 0;
+ key32lru_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key32_lru_ops,
(void *)&key32lru_params, (void *)key32lru, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32lru_params.n_entries = 1<<16;
+ key32lru_params.n_keys = 1<<16;
key32lru_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key32_lru_ops,
@@ -743,13 +746,15 @@ test_table_hash32ext(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_key32_ext_params key32ext_params = {
- .n_entries = 1<<16,
- .n_entries_ext = 1<<15,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
+ struct rte_table_hash_params key32ext_params = {
+ .name = "TABLE",
+ .key_size = 32,
.key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key32ext[32];
@@ -778,14 +783,14 @@ test_table_hash32ext(void)
VERIFY(status, CHECK_TABLE_OK);
/* Invalid parameters */
- key32ext_params.n_entries = 0;
+ key32ext_params.n_keys = 0;
status = test_table_type(&rte_table_hash_key32_ext_ops,
(void *)&key32ext_params, (void *)key32ext, &table_packets,
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32ext_params.n_entries = 1<<16;
+ key32ext_params.n_keys = 1<<16;
key32ext_params.f_hash = NULL;
status = test_table_type(&rte_table_hash_key32_ext_ops,
@@ -793,14 +798,6 @@ test_table_hash32ext(void)
NULL, 0);
VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
- key32ext_params.f_hash = pipeline_test_hash;
- key32ext_params.n_entries_ext = 0;
-
- status = test_table_type(&rte_table_hash_key32_ext_ops,
- (void *)&key32ext_params, (void *)key32ext, &table_packets,
- NULL, 0);
- VERIFY(status, CHECK_TABLE_TABLE_CONFIG);
-
return 0;
}
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 2f916b7..03c2723 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -883,6 +883,13 @@ test_table_hash_lru(void)
16);
if (status < 0)
return status;
+
+ status = test_table_hash_lru_generic(
+ &rte_table_hash_key32_lru_ops,
+ 32);
+ if (status < 0)
+ return status;
+
status = test_lru_update();
if (status < 0)
return status;
@@ -902,6 +909,11 @@ test_table_hash_ext(void)
status = test_table_hash_ext_generic(&rte_table_hash_key16_ext_ops, 16);
if (status < 0)
return status;
+
+ status = test_table_hash_ext_generic(&rte_table_hash_key32_ext_ops, 32);
+ if (status < 0)
+ return status;
+
return 0;
}
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 13/18] librte_table: rework cuckoo hash table
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (30 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 12/18] librte_table: rework 32-byte " Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 14/18] test: add mask-based hash functions Cristian Dumitrescu
` (4 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Rework for the cuckoo hash table to use the unified parameter
structure.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 26 ------
lib/librte_table/rte_table_hash_cuckoo.c | 149 +++++++++++++------------------
test/test-pipeline/pipeline_hash.c | 18 ++--
test/test/test_table_combined.c | 13 +--
test/test/test_table_tables.c | 15 ++--
5 files changed, 82 insertions(+), 139 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 16e1dfa..eb18c7a 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -151,32 +151,6 @@ extern struct rte_table_ops rte_table_hash_key32_lru_ops;
extern struct rte_table_ops rte_table_hash_key32_ext_ops;
-/** Cuckoo hash table parameters */
-struct rte_table_hash_cuckoo_params {
- /** Key size (number of bytes */
- uint32_t key_size;
-
- /** Maximum number of hash table entries */
- uint32_t n_keys;
-
- /** Hash function used to calculate hash */
- rte_table_hash_op_hash_nomask f_hash;
-
- /** Seed value or Init value used by f_hash */
- uint32_t seed;
-
- /** Byte offset within packet meta-data where the 4-byte key signature
- is located. Valid for pre-computed key signature tables, ignored for
- do-sig tables. */
- uint32_t signature_offset;
-
- /** Byte offset within packet meta-data where the key is located */
- uint32_t key_offset;
-
- /** Hash table name */
- const char *name;
-};
-
extern struct rte_table_ops rte_table_hash_cuckoo_ops;
#ifdef __cplusplus
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
index 9b42423..8d26597 100644
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -64,27 +64,30 @@ struct rte_table_hash {
uint32_t key_size;
uint32_t entry_size;
uint32_t n_keys;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint32_t seed;
- uint32_t signature_offset;
uint32_t key_offset;
- const char *name;
/* cuckoo hash table object */
struct rte_hash *h_table;
/* Lookup table */
- uint8_t memory[0] __rte_cache_aligned; };
+ uint8_t memory[0] __rte_cache_aligned;
+};
static int
-check_params_create_hash_cuckoo(const struct
-rte_table_hash_cuckoo_params *params) {
- /* Check for valid parameters */
+check_params_create_hash_cuckoo(struct rte_table_hash_params *params)
+{
if (params == NULL) {
RTE_LOG(ERR, TABLE, "NULL Input Parameters.\n");
return -EINVAL;
}
+ if (params->name == NULL) {
+ RTE_LOG(ERR, TABLE, "Table name is NULL.\n");
+ return -EINVAL;
+ }
+
if (params->key_size == 0) {
RTE_LOG(ERR, TABLE, "Invalid key_size.\n");
return -EINVAL;
@@ -100,11 +103,6 @@ rte_table_hash_cuckoo_params *params) {
return -EINVAL;
}
- if (params->name == NULL) {
- RTE_LOG(ERR, TABLE, "Table name is NULL.\n");
- return -EINVAL;
- }
-
return 0;
}
@@ -113,34 +111,24 @@ rte_table_hash_cuckoo_create(void *params,
int socket_id,
uint32_t entry_size)
{
- struct rte_hash *rte_hash_handle;
+ struct rte_table_hash_params *p = params;
+ struct rte_hash *h_table;
struct rte_table_hash *t;
- uint32_t total_size, total_cl_size;
+ uint32_t total_size;
/* Check input parameters */
- struct rte_table_hash_cuckoo_params *p =
- (struct rte_table_hash_cuckoo_params *) params;
-
if (check_params_create_hash_cuckoo(params))
return NULL;
/* Memory allocation */
- total_cl_size =
- (sizeof(struct rte_table_hash) +
- RTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;
- total_cl_size += (p->n_keys * entry_size +
- RTE_CACHE_LINE_SIZE) / RTE_CACHE_LINE_SIZE;
- total_size = total_cl_size * RTE_CACHE_LINE_SIZE;
-
- t = rte_zmalloc_socket("TABLE",
- total_size,
- RTE_CACHE_LINE_SIZE,
- socket_id);
+ total_size = sizeof(struct rte_table_hash) +
+ RTE_CACHE_LINE_ROUNDUP(p->n_keys * entry_size);
+
+ t = rte_zmalloc_socket(p->name, total_size, RTE_CACHE_LINE_SIZE, socket_id);
if (t == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: Cannot allocate %u bytes for Cuckoo hash table\n",
- __func__,
- (uint32_t)sizeof(struct rte_table_hash));
+ "%s: Cannot allocate %u bytes for cuckoo hash table %s\n",
+ __func__, total_size, p->name);
return NULL;
}
@@ -154,13 +142,13 @@ rte_table_hash_cuckoo_create(void *params,
.name = p->name
};
- rte_hash_handle = rte_hash_find_existing(p->name);
- if (rte_hash_handle == NULL) {
- rte_hash_handle = rte_hash_create(&hash_cuckoo_params);
- if (NULL == rte_hash_handle) {
+ h_table = rte_hash_find_existing(p->name);
+ if (h_table == NULL) {
+ h_table = rte_hash_create(&hash_cuckoo_params);
+ if (h_table == NULL) {
RTE_LOG(ERR, TABLE,
- "%s: failed to create cuckoo hash table. keysize: %u",
- __func__, hash_cuckoo_params.key_len);
+ "%s: failed to create cuckoo hash table %s\n",
+ __func__, p->name);
rte_free(t);
return NULL;
}
@@ -172,26 +160,22 @@ rte_table_hash_cuckoo_create(void *params,
t->n_keys = p->n_keys;
t->f_hash = p->f_hash;
t->seed = p->seed;
- t->signature_offset = p->signature_offset;
t->key_offset = p->key_offset;
- t->name = p->name;
- t->h_table = rte_hash_handle;
+ t->h_table = h_table;
RTE_LOG(INFO, TABLE,
- "%s: Cuckoo Hash table memory footprint is %u bytes\n",
- __func__, total_size);
+ "%s: Cuckoo hash table %s memory footprint is %u bytes\n",
+ __func__, p->name, total_size);
return t;
}
static int
rte_table_hash_cuckoo_free(void *table) {
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
struct rte_table_hash *t = table;
+ if (table == NULL)
+ return -EINVAL;
+
rte_hash_free(t->h_table);
rte_free(t);
@@ -200,25 +184,18 @@ rte_table_hash_cuckoo_free(void *table) {
static int
rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
- int *key_found, void **entry_ptr) {
+ int *key_found, void **entry_ptr)
+{
+ struct rte_table_hash *t = table;
int pos = 0;
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (key == NULL) {
- RTE_LOG(ERR, TABLE, "%s: key parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (entry == NULL) {
- RTE_LOG(ERR, TABLE, "%s: entry parameter is NULL\n", __func__);
+ /* Check input parameters */
+ if ((table == NULL) ||
+ (key == NULL) ||
+ (entry == NULL) ||
+ (key_found == NULL) ||
+ (entry_ptr == NULL))
return -EINVAL;
- }
-
- struct rte_table_hash *t = table;
/* Find Existing entries */
pos = rte_hash_lookup(t->h_table, key);
@@ -231,17 +208,15 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
*entry_ptr = existing_entry;
return 0;
-} else if (pos == -ENOENT) {
- /* Entry not found. Adding new entry */
+ }
+
+ if (pos == -ENOENT) {
+ /* Entry not found. Adding new entry */
uint8_t *new_entry;
pos = rte_hash_add_key(t->h_table, key);
- if (pos < 0) {
- RTE_LOG(ERR, TABLE,
- "%s: Entry not added, status : %u\n",
- __func__, pos);
+ if (pos < 0)
return pos;
- }
new_entry = &t->memory[pos * t->entry_size];
memcpy(new_entry, entry, t->entry_size);
@@ -250,25 +225,22 @@ rte_table_hash_cuckoo_entry_add(void *table, void *key, void *entry,
*entry_ptr = new_entry;
return 0;
}
+
return pos;
}
static int
rte_table_hash_cuckoo_entry_delete(void *table, void *key,
- int *key_found, __rte_unused void *entry) {
+ int *key_found, void *entry)
+{
+ struct rte_table_hash *t = table;
int pos = 0;
- if (table == NULL) {
- RTE_LOG(ERR, TABLE, "%s: table parameter is NULL\n", __func__);
- return -EINVAL;
- }
-
- if (key == NULL) {
- RTE_LOG(ERR, TABLE, "%s: key parameter is NULL\n", __func__);
+ /* Check input parameters */
+ if ((table == NULL) ||
+ (key == NULL) ||
+ (key_found == NULL))
return -EINVAL;
- }
-
- struct rte_table_hash *t = table;
pos = rte_hash_del_key(t->h_table, key);
if (pos >= 0) {
@@ -279,12 +251,13 @@ rte_table_hash_cuckoo_entry_delete(void *table, void *key,
memcpy(entry, entry_ptr, t->entry_size);
memset(&t->memory[pos * t->entry_size], 0, t->entry_size);
+ return 0;
}
+ *key_found = 0;
return pos;
}
-
static int
rte_table_hash_cuckoo_lookup(void *table,
struct rte_mbuf **pkts,
@@ -292,7 +265,7 @@ rte_table_hash_cuckoo_lookup(void *table,
uint64_t *lookup_hit_mask,
void **entries)
{
- struct rte_table_hash *t = (struct rte_table_hash *)table;
+ struct rte_table_hash *t = table;
uint64_t pkts_mask_out = 0;
uint32_t i;
@@ -301,20 +274,19 @@ rte_table_hash_cuckoo_lookup(void *table,
RTE_TABLE_HASH_CUCKOO_STATS_PKTS_IN_ADD(t, n_pkts_in);
if ((pkts_mask & (pkts_mask + 1)) == 0) {
- const uint8_t *keys[64];
- int32_t positions[64], status;
+ const uint8_t *keys[RTE_PORT_IN_BURST_SIZE_MAX];
+ int32_t positions[RTE_PORT_IN_BURST_SIZE_MAX], status;
/* Keys for bulk lookup */
for (i = 0; i < n_pkts_in; i++)
keys[i] = RTE_MBUF_METADATA_UINT8_PTR(pkts[i],
- t->key_offset);
+ t->key_offset);
/* Bulk Lookup */
status = rte_hash_lookup_bulk(t->h_table,
(const void **) keys,
n_pkts_in,
positions);
-
if (status == 0) {
for (i = 0; i < n_pkts_in; i++) {
if (likely(positions[i] >= 0)) {
@@ -326,7 +298,7 @@ rte_table_hash_cuckoo_lookup(void *table,
}
}
}
- } else {
+ } else
for (i = 0; i < (uint32_t)(RTE_PORT_IN_BURST_SIZE_MAX
- __builtin_clzll(pkts_mask)); i++) {
uint64_t pkt_mask = 1LLU << i;
@@ -345,7 +317,6 @@ rte_table_hash_cuckoo_lookup(void *table,
}
}
}
- }
*lookup_hit_mask = pkts_mask_out;
RTE_TABLE_HASH_CUCKOO_STATS_PKTS_LOOKUP_MISS(t,
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index eda63e8..24df81c 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -402,19 +402,15 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_CUCKOO_KEY112:
case e_APP_PIPELINE_HASH_CUCKOO_KEY128:
{
- char hash_name[RTE_HASH_NAMESIZE];
-
- snprintf(hash_name, sizeof(hash_name), "RTE_TH_CUCKOO_%d",
- app.pipeline_type);
-
- struct rte_table_hash_cuckoo_params table_hash_params = {
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
.key_size = key_size,
- .n_keys = (1 << 24) + 1,
- .f_hash = test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
.key_offset = APP_METADATA_OFFSET(32),
- .name = hash_name,
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = (rte_table_hash_op_hash)test_hash,
+ .seed = 0,
};
struct rte_pipeline_table_params table_params = {
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index 93603dc..9515dd0 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -807,14 +807,15 @@ test_table_hash_cuckoo_combined(void)
int status, i;
/* Traffic flow */
- struct rte_table_hash_cuckoo_params cuckoo_params = {
+ struct rte_table_hash_params cuckoo_params = {
+ .name = "TABLE",
.key_size = 32,
- .n_keys = 1<<16,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
.key_offset = APP_METADATA_OFFSET(32),
- .name = "CUCKOO_HASH",
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
uint8_t key_cuckoo[32];
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 03c2723..7f04212 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -932,14 +932,15 @@ test_table_hash_cuckoo(void)
uint32_t entry_size = 1;
/* Initialize params and create tables */
- struct rte_table_hash_cuckoo_params cuckoo_params = {
+ struct rte_table_hash_params cuckoo_params = {
+ .name = "TABLE",
.key_size = 32,
- .n_keys = 1 << 24,
- .f_hash = pipeline_test_hash,
- .seed = 0,
- .signature_offset = APP_METADATA_OFFSET(0),
.key_offset = APP_METADATA_OFFSET(32),
- .name = "CUCKOO",
+ .key_mask = NULL,
+ .n_keys = 1 << 16,
+ .n_buckets = 1 << 16,
+ .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .seed = 0,
};
table = rte_table_hash_cuckoo_ops.f_create(NULL, 0, entry_size);
@@ -969,7 +970,7 @@ test_table_hash_cuckoo(void)
if (table != NULL)
return -4;
- cuckoo_params.f_hash = pipeline_test_hash;
+ cuckoo_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
cuckoo_params.name = NULL;
table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 14/18] test: add mask-based hash functions
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (31 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 13/18] librte_table: rework cuckoo hash table Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 15/18] librte_table: cosmetic enhancements in api file Cristian Dumitrescu
` (3 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Introduce mask-based hash functions in hash_func.h.
Propagate their usage in test/test, test/test-pipeline and
examples/ip_pipeline.
Remove the non-mask-based hash function prototype from API (which
was previously used as build workaround).
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
examples/ip_pipeline/pipeline/hash_func.h | 176 ++++++++++++---------
.../pipeline/pipeline_flow_classification.c | 10 +-
.../pipeline/pipeline_flow_classification_be.c | 9 +-
.../ip_pipeline/pipeline/pipeline_passthrough_be.c | 14 +-
.../ip_pipeline/pipeline/pipeline_routing_be.c | 2 +-
lib/librte_table/rte_table_hash.h | 6 -
test/test-pipeline/main.h | 5 +-
test/test-pipeline/pipeline_hash.c | 113 ++-----------
test/test/test_table.c | 1 +
test/test/test_table.h | 3 +-
test/test/test_table_combined.c | 14 +-
test/test/test_table_tables.c | 12 +-
12 files changed, 155 insertions(+), 210 deletions(-)
diff --git a/examples/ip_pipeline/pipeline/hash_func.h b/examples/ip_pipeline/pipeline/hash_func.h
index b112369..ecd4e05
--- a/examples/ip_pipeline/pipeline/hash_func.h
+++ b/examples/ip_pipeline/pipeline/hash_func.h
@@ -34,48 +34,56 @@
#define __INCLUDE_HASH_FUNC_H__
static inline uint64_t
-hash_xor_key8(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key8(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0;
- xor0 = seed ^ k[0];
+ xor0 = seed ^ (k[0] & m[0]);
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key16(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key16(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0;
- xor0 = (k[0] ^ seed) ^ k[1];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key24(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key24(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0;
- xor0 = (k[0] ^ seed) ^ k[1];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
- xor0 ^= k[2];
+ xor0 ^= k[2] & m[2];
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key32(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
xor0 ^= xor1;
@@ -83,30 +91,34 @@ hash_xor_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_xor_key40(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key40(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
xor0 ^= xor1;
- xor0 ^= k[4];
+ xor0 ^= k[4] & m[4];
return (xor0 >> 32) ^ xor0;
}
static inline uint64_t
-hash_xor_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key48(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1, xor2;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
- xor2 = k[4] ^ k[5];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
+ xor2 = (k[4] & m[4]) ^ (k[5] & m[5]);
xor0 ^= xor1;
@@ -116,17 +128,19 @@ hash_xor_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_xor_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key56(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1, xor2;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
- xor2 = k[4] ^ k[5];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
+ xor2 = (k[4] & m[4]) ^ (k[5] & m[5]);
xor0 ^= xor1;
- xor2 ^= k[6];
+ xor2 ^= k[6] & m[6];
xor0 ^= xor2;
@@ -134,15 +148,17 @@ hash_xor_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_xor_key64(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_xor_key64(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t xor0, xor1, xor2, xor3;
- xor0 = (k[0] ^ seed) ^ k[1];
- xor1 = k[2] ^ k[3];
- xor2 = k[4] ^ k[5];
- xor3 = k[6] ^ k[7];
+ xor0 = ((k[0] & m[0]) ^ seed) ^ (k[1] & m[1]);
+ xor1 = (k[2] & m[2]) ^ (k[3] & m[3]);
+ xor2 = (k[4] & m[4]) ^ (k[5] & m[5]);
+ xor3 = (k[6] & m[6]) ^ (k[7] & m[7]);
xor0 ^= xor1;
xor2 ^= xor3;
@@ -157,26 +173,30 @@ hash_xor_key64(void *key, __rte_unused uint32_t key_size, uint64_t seed)
#include <x86intrin.h>
static inline uint64_t
-hash_crc_key8(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key8(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t crc0;
- crc0 = _mm_crc32_u64(seed, k[0]);
+ crc0 = _mm_crc32_u64(seed, k[0] & m[0]);
return crc0;
}
static inline uint64_t
-hash_crc_key16(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key16(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, crc0, crc1;
- k0 = k[0];
+ k0 = k[0] & m[0];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
crc0 ^= crc1;
@@ -184,16 +204,18 @@ hash_crc_key16(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key24(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key24(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, crc0, crc1;
- k0 = k[0];
- k2 = k[2];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
crc0 = _mm_crc32_u64(crc0, k2);
@@ -203,18 +225,20 @@ hash_crc_key24(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key32(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, crc0, crc1, crc2, crc3;
- k0 = k[0];
- k2 = k[2];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
crc3 = k2 >> 32;
crc0 = _mm_crc32_u64(crc0, crc1);
@@ -226,19 +250,21 @@ hash_crc_key32(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key40(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key40(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, crc0, crc1, crc2, crc3;
- k0 = k[0];
- k2 = k[2];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
crc0 = _mm_crc32_u64(crc0, crc1);
crc1 = _mm_crc32_u64(crc2, crc3);
@@ -249,20 +275,22 @@ hash_crc_key40(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key48(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, k5, crc0, crc1, crc2, crc3;
- k0 = k[0];
- k2 = k[2];
- k5 = k[5];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
+ k5 = k[5] & m[5];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
crc0 = _mm_crc32_u64(crc0, (crc1 << 32) ^ crc2);
crc1 = _mm_crc32_u64(crc3, k5);
@@ -273,22 +301,24 @@ hash_crc_key48(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key56(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5;
- k0 = k[0];
- k2 = k[2];
- k5 = k[5];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
+ k5 = k[5] & m[5];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
- crc4 = _mm_crc32_u64(k5, k[6]);
+ crc4 = _mm_crc32_u64(k5, k[6] & m[6]);
crc5 = k5 >> 32;
crc0 = _mm_crc32_u64(crc0, (crc1 << 32) ^ crc2);
@@ -300,23 +330,25 @@ hash_crc_key56(void *key, __rte_unused uint32_t key_size, uint64_t seed)
}
static inline uint64_t
-hash_crc_key64(void *key, __rte_unused uint32_t key_size, uint64_t seed)
+hash_crc_key64(void *key, void *mask, __rte_unused uint32_t key_size,
+ uint64_t seed)
{
uint64_t *k = key;
+ uint64_t *m = mask;
uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5;
- k0 = k[0];
- k2 = k[2];
- k5 = k[5];
+ k0 = k[0] & m[0];
+ k2 = k[2] & m[2];
+ k5 = k[5] & m[5];
crc0 = _mm_crc32_u64(k0, seed);
- crc1 = _mm_crc32_u64(k0 >> 32, k[1]);
+ crc1 = _mm_crc32_u64(k0 >> 32, k[1] & m[1]);
- crc2 = _mm_crc32_u64(k2, k[3]);
- crc3 = _mm_crc32_u64(k2 >> 32, k[4]);
+ crc2 = _mm_crc32_u64(k2, k[3] & m[3]);
+ crc3 = _mm_crc32_u64(k2 >> 32, k[4] & m[4]);
- crc4 = _mm_crc32_u64(k5, k[6]);
- crc5 = _mm_crc32_u64(k5 >> 32, k[7]);
+ crc4 = _mm_crc32_u64(k5, k[6] & m[6]);
+ crc5 = _mm_crc32_u64(k5 >> 32, k[7] & m[7]);
crc0 = _mm_crc32_u64(crc0, (crc1 << 32) ^ crc2);
crc1 = _mm_crc32_u64(crc3, (crc4 << 32) ^ crc5);
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
index 9ef50cc..70b1938
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification.c
@@ -88,8 +88,10 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
uint32_t *signature)
{
uint8_t buffer[PIPELINE_FC_FLOW_KEY_MAX_SIZE];
+ uint8_t m[PIPELINE_FC_FLOW_KEY_MAX_SIZE]; /* key mask */
void *key_buffer = (key_out) ? key_out : buffer;
+ memset(m, 0xFF, sizeof(m));
switch (key_in->type) {
case FLOW_KEY_QINQ:
{
@@ -101,7 +103,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
qinq->cvlan = rte_cpu_to_be_16(key_in->key.qinq.cvlan);
if (signature)
- *signature = (uint32_t) hash_default_key8(qinq, 8, 0);
+ *signature = (uint32_t) hash_default_key8(qinq, m, 8, 0);
return 0;
}
@@ -118,7 +120,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
ipv4->port_dst = rte_cpu_to_be_16(key_in->key.ipv4_5tuple.port_dst);
if (signature)
- *signature = (uint32_t) hash_default_key16(ipv4, 16, 0);
+ *signature = (uint32_t) hash_default_key16(ipv4, m, 16, 0);
return 0;
}
@@ -136,7 +138,7 @@ app_pipeline_fc_key_convert(struct pipeline_fc_key *key_in,
ipv6->port_dst = rte_cpu_to_be_16(key_in->key.ipv6_5tuple.port_dst);
if (signature)
- *signature = (uint32_t) hash_default_key64(ipv6, 64, 0);
+ *signature = (uint32_t) hash_default_key64(ipv6, m, 64, 0);
return 0;
}
@@ -832,12 +834,12 @@ app_pipeline_fc_add_bulk(struct app_params *app,
}
/* Free resources */
- app_msg_free(app, rsp);
for (i = rsp->n_keys; i < n_keys; i++)
if (new_flow[i])
rte_free(flow[i]);
+ app_msg_free(app, rsp);
rte_free(flow_rsp);
rte_free(flow_req);
rte_free(new_flow);
diff --git a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
index e4a18e4..9846777 100644
--- a/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_flow_classification_be.c
@@ -118,7 +118,7 @@ struct flow_table_entry {
uint32_t pad;
};
-rte_table_hash_op_hash_nomask hash_func[] = {
+rte_table_hash_op_hash hash_func[] = {
hash_default_key8,
hash_default_key16,
hash_default_key24,
@@ -500,7 +500,7 @@ static void *pipeline_fc_init(struct pipeline_params *params,
p_fc->key_mask : NULL,
.n_keys = p_fc->n_flows,
.n_buckets = p_fc->n_flows / 4,
- .f_hash = (rte_table_hash_op_hash)hash_func[(p_fc->key_size / 8) - 1],
+ .f_hash = hash_func[(p_fc->key_size / 8) - 1],
.seed = 0,
};
@@ -519,19 +519,18 @@ static void *pipeline_fc_init(struct pipeline_params *params,
switch (p_fc->key_size) {
case 8:
table_params.ops = &rte_table_hash_key8_ext_ops;
- table_params.arg_create = &table_hash_params;
break;
case 16:
table_params.ops = &rte_table_hash_key16_ext_ops;
- table_params.arg_create = &table_hash_params;
break;
default:
table_params.ops = &rte_table_hash_ext_ops;
- table_params.arg_create = &table_hash_params;
}
+ table_params.arg_create = &table_hash_params;
+
status = rte_pipeline_table_create(p->p,
&table_params,
&p->table_id[0]);
diff --git a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
index 6b57f83..93eedbe 100644
--- a/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_passthrough_be.c
@@ -52,7 +52,7 @@
struct pipeline_passthrough {
struct pipeline p;
struct pipeline_passthrough_params params;
- rte_table_hash_op_hash_nomask f_hash;
+ rte_table_hash_op_hash f_hash;
uint32_t swap_field0_offset[SWAP_DIM];
uint32_t swap_field1_offset[SWAP_DIM];
uint64_t swap_field_mask[SWAP_DIM];
@@ -102,7 +102,7 @@ pkt_work_dma(
/* Read (dma_dst), compute (hash), write (hash) */
if (hash_enabled) {
- uint32_t hash = p->f_hash(dma_dst, dma_size, 0);
+ uint32_t hash = p->f_hash(dma_src, dma_mask, dma_size, 0);
*dma_hash = hash;
if (lb_hash) {
@@ -173,10 +173,10 @@ pkt4_work_dma(
/* Read (dma_dst), compute (hash), write (hash) */
if (hash_enabled) {
- uint32_t hash0 = p->f_hash(dma_dst0, dma_size, 0);
- uint32_t hash1 = p->f_hash(dma_dst1, dma_size, 0);
- uint32_t hash2 = p->f_hash(dma_dst2, dma_size, 0);
- uint32_t hash3 = p->f_hash(dma_dst3, dma_size, 0);
+ uint32_t hash0 = p->f_hash(dma_src0, dma_mask, dma_size, 0);
+ uint32_t hash1 = p->f_hash(dma_src1, dma_mask, dma_size, 0);
+ uint32_t hash2 = p->f_hash(dma_src2, dma_mask, dma_size, 0);
+ uint32_t hash3 = p->f_hash(dma_src3, dma_mask, dma_size, 0);
*dma_hash0 = hash0;
*dma_hash1 = hash1;
@@ -677,7 +677,7 @@ pipeline_passthrough_parse_args(struct pipeline_passthrough_params *p,
return 0;
}
-static rte_table_hash_op_hash_nomask
+static rte_table_hash_op_hash
get_hash_function(struct pipeline_passthrough *p)
{
switch (p->params.dma_size) {
diff --git a/examples/ip_pipeline/pipeline/pipeline_routing_be.c b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
index 796cde7..7aaf467 100644
--- a/examples/ip_pipeline/pipeline/pipeline_routing_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_routing_be.c
@@ -1356,7 +1356,7 @@ pipeline_routing_init(struct pipeline_params *params,
.key_mask = NULL,
.n_keys = p_rt->params.n_arp_entries,
.n_buckets = p_rt->params.n_arp_entries / 4,
- .f_hash = (rte_table_hash_op_hash)hash_default_key8,
+ .f_hash = hash_default_key8,
.seed = 0,
};
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index eb18c7a..3c53d6a 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -129,12 +129,6 @@ struct rte_table_hash_params {
uint64_t seed;
};
-/** Hash function */
-typedef uint64_t (*rte_table_hash_op_hash_nomask)(
- void *key,
- uint32_t key_size,
- uint64_t seed);
-
extern struct rte_table_ops rte_table_hash_ext_ops;
extern struct rte_table_ops rte_table_hash_lru_ops;
diff --git a/test/test-pipeline/main.h b/test/test-pipeline/main.h
index 3685849..26395a3
--- a/test/test-pipeline/main.h
+++ b/test/test-pipeline/main.h
@@ -131,7 +131,10 @@ enum {
void app_main_loop_rx(void);
void app_main_loop_rx_metadata(void);
-uint64_t test_hash(void *key, uint32_t key_size, uint64_t seed);
+uint64_t test_hash(void *key,
+ void *key_mask,
+ uint32_t key_size,
+ uint64_t seed);
void app_main_loop_worker(void);
void app_main_loop_worker_pipeline_stub(void);
diff --git a/test/test-pipeline/pipeline_hash.c b/test/test-pipeline/pipeline_hash.c
index 24df81c..edc1663 100644
--- a/test/test-pipeline/pipeline_hash.c
+++ b/test/test-pipeline/pipeline_hash.c
@@ -169,23 +169,23 @@ app_main_loop_worker_pipeline_hash(void) {
"ring %d\n", i);
}
+ struct rte_table_hash_params table_hash_params = {
+ .name = "TABLE",
+ .key_size = key_size,
+ .key_offset = APP_METADATA_OFFSET(32),
+ .key_mask = NULL,
+ .n_keys = 1 << 24,
+ .n_buckets = 1 << 22,
+ .f_hash = test_hash,
+ .seed = 0,
+ };
+
/* Table configuration */
switch (app.pipeline_type) {
case e_APP_PIPELINE_HASH_KEY8_EXT:
case e_APP_PIPELINE_HASH_KEY16_EXT:
case e_APP_PIPELINE_HASH_KEY32_EXT:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_ext_ops,
.arg_create = &table_hash_params,
@@ -204,17 +204,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_KEY16_LRU:
case e_APP_PIPELINE_HASH_KEY32_LRU:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_lru_ops,
.arg_create = &table_hash_params,
@@ -231,17 +220,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_EXT:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key8_ext_ops,
.arg_create = &table_hash_params,
@@ -258,17 +236,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY8_LRU:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key8_lru_ops,
.arg_create = &table_hash_params,
@@ -285,17 +252,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_EXT:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key16_ext_ops,
.arg_create = &table_hash_params,
@@ -312,17 +268,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY16_LRU:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key16_lru_ops,
.arg_create = &table_hash_params,
@@ -339,17 +284,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_EXT:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key32_ext_ops,
.arg_create = &table_hash_params,
@@ -367,17 +301,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_SPEC_KEY32_LRU:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_key32_lru_ops,
.arg_create = &table_hash_params,
@@ -402,17 +325,6 @@ app_main_loop_worker_pipeline_hash(void) {
case e_APP_PIPELINE_HASH_CUCKOO_KEY112:
case e_APP_PIPELINE_HASH_CUCKOO_KEY128:
{
- struct rte_table_hash_params table_hash_params = {
- .name = "TABLE",
- .key_size = key_size,
- .key_offset = APP_METADATA_OFFSET(32),
- .key_mask = NULL,
- .n_keys = 1 << 24,
- .n_buckets = 1 << 22,
- .f_hash = (rte_table_hash_op_hash)test_hash,
- .seed = 0,
- };
-
struct rte_pipeline_table_params table_params = {
.ops = &rte_table_hash_cuckoo_ops,
.arg_create = &table_hash_params,
@@ -485,6 +397,7 @@ app_main_loop_worker_pipeline_hash(void) {
uint64_t test_hash(
void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed)
{
@@ -547,7 +460,7 @@ app_main_loop_rx_metadata(void) {
} else
continue;
- *signature = test_hash(key, 0, 0);
+ *signature = test_hash(key, NULL, 0, 0);
}
do {
diff --git a/test/test/test_table.c b/test/test/test_table.c
index 9e9eed8..db7d4e6
--- a/test/test/test_table.c
+++ b/test/test/test_table.c
@@ -72,6 +72,7 @@ static void app_init_rings(void);
static void app_init_mbuf_pools(void);
uint64_t pipeline_test_hash(void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed)
{
diff --git a/test/test/test_table.h b/test/test/test_table.h
index 84d1845..8c1df33
--- a/test/test/test_table.h
+++ b/test/test/test_table.h
@@ -94,7 +94,7 @@
APP_METADATA_OFFSET(32)); \
k32 = (uint32_t *) key; \
k32[0] = (value); \
- *signature = pipeline_test_hash(key, 0, 0); \
+ *signature = pipeline_test_hash(key, NULL, 0, 0); \
rte_ring_enqueue((ring), m); \
} while (0)
@@ -131,6 +131,7 @@
/* Function definitions */
uint64_t pipeline_test_hash(
void *key,
+ __attribute__((unused)) void *key_mask,
__attribute__((unused)) uint32_t key_size,
__attribute__((unused)) uint64_t seed);
diff --git a/test/test/test_table_combined.c b/test/test/test_table_combined.c
index 9515dd0..e863788 100644
--- a/test/test/test_table_combined.c
+++ b/test/test/test_table_combined.c
@@ -448,7 +448,7 @@ test_table_hash8lru(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -509,7 +509,7 @@ test_table_hash16lru(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -570,7 +570,7 @@ test_table_hash32lru(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -631,7 +631,7 @@ test_table_hash8ext(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -692,7 +692,7 @@ test_table_hash16ext(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -753,7 +753,7 @@ test_table_hash32ext(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -814,7 +814,7 @@ test_table_hash_cuckoo_combined(void)
.key_mask = NULL,
.n_keys = 1 << 16,
.n_buckets = 1 << 16,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
diff --git a/test/test/test_table_tables.c b/test/test/test_table_tables.c
index 7f04212..a1b0c58 100644
--- a/test/test/test_table_tables.c
+++ b/test/test/test_table_tables.c
@@ -60,7 +60,7 @@ table_test table_tests[] = {
memset(key, 0, 32); \
k32 = (uint32_t *) key; \
k32[0] = (value); \
- *signature = pipeline_test_hash(key, 0, 0); \
+ *signature = pipeline_test_hash(key, NULL, 0, 0); \
} while (0)
unsigned n_table_tests = RTE_DIM(table_tests);
@@ -674,7 +674,7 @@ test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size)
.key_mask = NULL,
.n_keys = 1 << 10,
.n_buckets = 1 << 10,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -691,7 +691,7 @@ test_table_hash_lru_generic(struct rte_table_ops *ops, uint32_t key_size)
if (table != NULL)
return -4;
- hash_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
+ hash_params.f_hash = pipeline_test_hash;
table = ops->f_create(&hash_params, 0, 1);
if (table == NULL)
@@ -777,7 +777,7 @@ test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size)
.key_mask = NULL,
.n_keys = 1 << 10,
.n_buckets = 1 << 10,
- .f_hash = (rte_table_hash_op_hash)pipeline_test_hash,
+ .f_hash = pipeline_test_hash,
.seed = 0,
};
@@ -801,7 +801,7 @@ test_table_hash_ext_generic(struct rte_table_ops *ops, uint32_t key_size)
if (table != NULL)
return -4;
- hash_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
+ hash_params.f_hash = pipeline_test_hash;
table = ops->f_create(&hash_params, 0, 1);
if (table == NULL)
@@ -970,7 +970,7 @@ test_table_hash_cuckoo(void)
if (table != NULL)
return -4;
- cuckoo_params.f_hash = (rte_table_hash_op_hash)pipeline_test_hash;
+ cuckoo_params.f_hash = pipeline_test_hash;
cuckoo_params.name = NULL;
table = rte_table_hash_cuckoo_ops.f_create(&cuckoo_params,
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 15/18] librte_table: cosmetic enhancements in api file
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (32 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 14/18] test: add mask-based hash functions Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 16/18] librte_table: copyright cosmetic updates Cristian Dumitrescu
` (2 subsequent siblings)
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Consolidated hash tables into functional groups.
Minor changes in comments.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 34 +++++++++-------------------------
1 file changed, 9 insertions(+), 25 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 3c53d6a..081464c 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -45,8 +45,6 @@ extern "C" {
* These tables use the exact match criterion to uniquely associate data to
* lookup keys.
*
- * Use-cases: Flow classification table, Address Resolution Protocol (ARP) table
- *
* Hash table types:
* 1. Entry add strategy on bucket full:
* a. Least Recently Used (LRU): One of the existing keys in the bucket is
@@ -59,7 +57,7 @@ extern "C" {
* to the bucket, it also becomes the new MRU key. When a key needs to
* be picked and dropped, the most likely candidate for drop, i.e. the
* current LRU key, is always picked. The LRU logic requires maintaining
- * specific data structures per each bucket.
+ * specific data structures per each bucket. Use-cases: flow cache, etc.
* b. Extendible bucket (ext): The bucket is extended with space for 4 more
* keys. This is done by allocating additional memory at table init time,
* which is used to create a pool of free keys (the size of this pool is
@@ -73,20 +71,8 @@ extern "C" {
* first group of 4 keys, the search continues beyond the first group of
* 4 keys, potentially until all keys in this bucket are examined. The
* extendible bucket logic requires maintaining specific data structures
- * per table and per each bucket.
- * 2. Key signature computation:
- * a. Pre-computed key signature: The key lookup operation is split between
- * two CPU cores. The first CPU core (typically the CPU core performing
- * packet RX) extracts the key from the input packet, computes the key
- * signature and saves both the key and the key signature in the packet
- * buffer as packet meta-data. The second CPU core reads both the key and
- * the key signature from the packet meta-data and performs the bucket
- * search step of the key lookup operation.
- * b. Key signature computed on lookup (do-sig): The same CPU core reads
- * the key from the packet meta-data, uses it to compute the key
- * signature and also performs the bucket search step of the key lookup
- * operation.
- * 3. Key size:
+ * per table and per each bucket. Use-cases: flow table, etc.
+ * 2. Key size:
* a. Configurable key size
* b. Single key size (8-byte, 16-byte or 32-byte key size)
*
@@ -129,22 +115,20 @@ struct rte_table_hash_params {
uint64_t seed;
};
+/** Extendible bucket hash table operations */
extern struct rte_table_ops rte_table_hash_ext_ops;
+extern struct rte_table_ops rte_table_hash_key8_ext_ops;
+extern struct rte_table_ops rte_table_hash_key16_ext_ops;
+extern struct rte_table_ops rte_table_hash_key32_ext_ops;
+/** LRU hash table operations */
extern struct rte_table_ops rte_table_hash_lru_ops;
extern struct rte_table_ops rte_table_hash_key8_lru_ops;
-
-extern struct rte_table_ops rte_table_hash_key8_ext_ops;
-
extern struct rte_table_ops rte_table_hash_key16_lru_ops;
-
-extern struct rte_table_ops rte_table_hash_key16_ext_ops;
-
extern struct rte_table_ops rte_table_hash_key32_lru_ops;
-extern struct rte_table_ops rte_table_hash_key32_ext_ops;
-
+/** Cuckoo hash table operations */
extern struct rte_table_ops rte_table_hash_cuckoo_ops;
#ifdef __cplusplus
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 16/18] librte_table: copyright cosmetic updates
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (33 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 15/18] librte_table: cosmetic enhancements in api file Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 17/18] librte_table: map file updates Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table Cristian Dumitrescu
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Removed incorrect white spaces and updated year in copyrigh headers.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_hash.h | 2 +-
lib/librte_table/rte_table_hash_cuckoo.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_ext.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_key16.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_key32.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_key8.c | 52 ++++++++++++++++----------------
lib/librte_table/rte_table_hash_lru.c | 52 ++++++++++++++++----------------
7 files changed, 157 insertions(+), 157 deletions(-)
diff --git a/lib/librte_table/rte_table_hash.h b/lib/librte_table/rte_table_hash.h
index 081464c..15f1902 100644
--- a/lib/librte_table/rte_table_hash.h
+++ b/lib/librte_table/rte_table_hash.h
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
diff --git a/lib/librte_table/rte_table_hash_cuckoo.c b/lib/librte_table/rte_table_hash_cuckoo.c
index 8d26597..f3845c7
--- a/lib/librte_table/rte_table_hash_cuckoo.c
+++ b/lib/librte_table/rte_table_hash_cuckoo.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2016 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
diff --git a/lib/librte_table/rte_table_hash_ext.c b/lib/librte_table/rte_table_hash_ext.c
index 0a743ad..3af1bca 100644
--- a/lib/librte_table/rte_table_hash_ext.c
+++ b/lib/librte_table/rte_table_hash_ext.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
diff --git a/lib/librte_table/rte_table_hash_key16.c b/lib/librte_table/rte_table_hash_key16.c
index 84b59cc..b541735
--- a/lib/librte_table/rte_table_hash_key16.c
+++ b/lib/librte_table/rte_table_hash_key16.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
diff --git a/lib/librte_table/rte_table_hash_key32.c b/lib/librte_table/rte_table_hash_key32.c
index 3eb7e4f..d4364d6
--- a/lib/librte_table/rte_table_hash_key32.c
+++ b/lib/librte_table/rte_table_hash_key32.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
diff --git a/lib/librte_table/rte_table_hash_key8.c b/lib/librte_table/rte_table_hash_key8.c
index a44f75c..9437304
--- a/lib/librte_table/rte_table_hash_key8.c
+++ b/lib/librte_table/rte_table_hash_key8.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
#include <stdio.h>
diff --git a/lib/librte_table/rte_table_hash_lru.c b/lib/librte_table/rte_table_hash_lru.c
index eebc1cd..139b8fb 100644
--- a/lib/librte_table/rte_table_hash_lru.c
+++ b/lib/librte_table/rte_table_hash_lru.c
@@ -1,34 +1,34 @@
/*-
- * BSD LICENSE
+ * BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- * All rights reserved.
+ * Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
*
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
*
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
*
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <string.h>
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 17/18] librte_table: map file updates
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (34 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 16/18] librte_table: copyright cosmetic updates Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table Cristian Dumitrescu
36 siblings, 0 replies; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Updated map file to current API. Several API items were removed,
so not using inheritance.
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
lib/librte_table/rte_table_version.map | 23 ++++++++++++++++++++++-
1 file changed, 22 insertions(+), 1 deletion(-)
diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map
index e1eaa27..f03fc3f 100644
--- a/lib/librte_table/rte_table_version.map
+++ b/lib/librte_table/rte_table_version.map
@@ -28,7 +28,7 @@ DPDK_2.2 {
rte_table_hash_key16_ext_dosig_ops;
rte_table_hash_key16_lru_dosig_ops;
-};
+} DPDK_2.0;
DPDK_16.07 {
global:
@@ -36,3 +36,24 @@ DPDK_16.07 {
rte_table_hash_cuckoo_dosig_ops;
} DPDK_2.0;
+
+DPDK_17.11 {
+ global:
+
+ rte_table_acl_ops;
+ rte_table_array_ops;
+ rte_table_hash_ext_ops;
+ rte_table_hash_key8_ext_ops;
+ rte_table_hash_key16_ext_ops;
+ rte_table_hash_key32_ext_ops;
+ rte_table_hash_lru_ops;
+ rte_table_hash_key8_lru_ops;
+ rte_table_hash_key16_lru_ops;
+ rte_table_hash_key32_lru_ops;
+ rte_table_hash_cuckoo_ops;
+ rte_table_lpm_ipv6_ops;
+ rte_table_lpm_ops;
+ rte_table_stub_ops;
+
+ local: *;
+};
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
` (35 preceding siblings ...)
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 17/18] librte_table: map file updates Cristian Dumitrescu
@ 2017-10-18 15:03 ` Cristian Dumitrescu
2017-10-18 18:30 ` Mcnamara, John
36 siblings, 1 reply; 53+ messages in thread
From: Cristian Dumitrescu @ 2017-10-18 15:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
doc/guides/rel_notes/deprecation.rst | 6 ------
1 file changed, 6 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 52058f5..2ea19e7 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -86,9 +86,3 @@ Deprecation Notices
* librte_meter: The API will change to accommodate configuration profiles.
Most of the API functions will have an additional opaque parameter.
-
-* librte_table: The ``key_mask`` parameter will be added to all the hash tables
- that currently do not have it, as well as to the hash compute function prototype.
- The non-"do-sig" versions of the hash tables will be removed
- (including the ``signature_offset`` parameter)
- and the "do-sig" versions renamed accordingly.
--
2.7.4
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [dpdk-dev] [PATCH v3 00/18] librte_table: add key mask for hash tables
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: add key mask for hash tables Cristian Dumitrescu
@ 2017-10-18 17:10 ` Dumitrescu, Cristian
0 siblings, 0 replies; 53+ messages in thread
From: Dumitrescu, Cristian @ 2017-10-18 17:10 UTC (permalink / raw)
To: dev; +Cc: thomas
> Cristian Dumitrescu (18):
> test: removing calls to deprecated hash tables
> librte_table: remove deprecated 8-byte key hash tables
> librte_table: remove deprecated 16-byte key hash tables
> librte_table: remove deprecated variable size key ext hash tables
> librte_table: remove deprecated variable size key lru hash tables
> librte_table: rename cuckoo hash table ops
> librte_table: add unified params structure and mask-based hash func
> librte_table: rework variable size key ext hash tables
> librte_table: rework variable size key lru hash table
> librte_table: rework 8-byte key hash tables
> librte_table: rework 16-byte key hash tables
> librte_table: rework 32-byte key hash tables
> librte_table: rework cuckoo hash table
> test: add mask-based hash functions
> librte_table: cosmetic enhancements in api file
> librte_table: copyright cosmetic updates
> librte_table: map file updates
> doc: remove deprecation notice for librte_table
>
Re-applied to next-pipeline (after patch set restructure required by Thomas).
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table Cristian Dumitrescu
@ 2017-10-18 18:30 ` Mcnamara, John
0 siblings, 0 replies; 53+ messages in thread
From: Mcnamara, John @ 2017-10-18 18:30 UTC (permalink / raw)
To: Dumitrescu, Cristian, dev; +Cc: thomas
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Cristian Dumitrescu
> Sent: Wednesday, October 18, 2017 4:04 PM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net
> Subject: [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for
> librte_table
>
> Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>
Acked-by: John McNamara <john.mcnamara@intel.com>
^ permalink raw reply [flat|nested] 53+ messages in thread
end of thread, other threads:[~2017-10-18 18:30 UTC | newest]
Thread overview: 53+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-24 13:53 [dpdk-dev] [PATCH 0/5] table: add key mask for hash tables Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 1/5] " Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 0/5] " Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 1/5] " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 01/18] test: removing calls to deprecated " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 02/18] librte_table: remove deprecated 8-byte key " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 03/18] librte_table: remove deprecated 16-byte " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 04/18] librte_table: remove deprecated variable size key ext " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 05/18] librte_table: remove deprecated variable size key lru " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 06/18] librte_table: rename cuckoo hash table ops Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 07/18] librte_table: add unified params structure and mask-based hash func Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 08/18] librte_table: rework variable size key ext hash tables Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 09/18] librte_table: rework variable size key lru hash table Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 10/18] librte_table: rework 8-byte key hash tables Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 11/18] librte_table: rework 16-byte " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 12/18] librte_table: rework 32-byte " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 13/18] librte_table: rework cuckoo hash table Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 14/18] test: add mask-based hash functions Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 15/18] librte_table: cosmetic enhancements in api file Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 16/18] librte_table: copyright cosmetic updates Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 17/18] librte_table: map file updates Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 00/18] librte_table: add key mask for hash tables Cristian Dumitrescu
2017-10-18 17:10 ` Dumitrescu, Cristian
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 01/18] test: removing calls to deprecated " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 02/18] librte_table: remove deprecated 8-byte key " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 03/18] librte_table: remove deprecated 16-byte " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 04/18] librte_table: remove deprecated variable size key ext " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 05/18] librte_table: remove deprecated variable size key lru " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 06/18] librte_table: rename cuckoo hash table ops Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 07/18] librte_table: add unified params structure and mask-based hash func Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 08/18] librte_table: rework variable size key ext hash tables Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 09/18] librte_table: rework variable size key lru hash table Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 10/18] librte_table: rework 8-byte key hash tables Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 11/18] librte_table: rework 16-byte " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 12/18] librte_table: rework 32-byte " Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 13/18] librte_table: rework cuckoo hash table Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 14/18] test: add mask-based hash functions Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 15/18] librte_table: cosmetic enhancements in api file Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 16/18] librte_table: copyright cosmetic updates Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 17/18] librte_table: map file updates Cristian Dumitrescu
2017-10-18 15:03 ` [dpdk-dev] [PATCH v3 18/18] doc: remove deprecation notice for librte_table Cristian Dumitrescu
2017-10-18 18:30 ` Mcnamara, John
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 2/5] test: update due to api changes in librte_table Cristian Dumitrescu
2017-10-10 11:18 ` [dpdk-dev] [PATCH V2 3/5] test-pipeline: " Cristian Dumitrescu
2017-10-10 11:19 ` [dpdk-dev] [PATCH V2 4/5] ip_pipeline: " Cristian Dumitrescu
2017-10-10 11:19 ` [dpdk-dev] [PATCH V2 5/5] deprecation: removed the librte_table notice Cristian Dumitrescu
2017-10-11 14:22 ` [dpdk-dev] [PATCH V2 0/5] table: add key mask for hash tables Dumitrescu, Cristian
2017-08-24 13:53 ` [dpdk-dev] [PATCH 2/5] test: update due to api changes in librte_table Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 3/5] test-pipeline: " Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 4/5] ip_pipeline: " Cristian Dumitrescu
2017-08-24 13:53 ` [dpdk-dev] [PATCH 5/5] deprecation: removed the librte_table notice Cristian Dumitrescu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).